Proposal/Feature request: AddToBlackList

[spikebike]

Annoyingly many peers will claim they are peers for any requested infohash. Some way to black list them would be useful. Two potential strategies:

1. pick a random infohash, statistically the chance of a collision in 2^160 is low,
   put anyone that replies on black list.
2. For an infohash you want for any peer that responds ask that peer for the infohash+1. If they also claim to be a peer for that infohash+1 they are lying.

[nictuku]

Precision / security isn't usually a concern for most applications, so I'm not sure that belongs to the library.

What you suggest could help, but attackers can just tweak their system a bit and then they are back on the game (e.g: they may ignore the second request coming from a particular host).

I'm OK with adding a missing API to the library: "AddToBlackList", which keeps track of IP/port (or just IPs) that seem to be bogus, and refuses to talk to them in next iterations. This could maybe done with a bloomfilter to save memory.

But you'd then implement the rest of the logic of finding a bogus peer on your own application.

Your application would get the results from PeersRequestResults, then post-process them as you suggested. If you find bogus peers that answer to anything, then you call "dht.AddToBlackList" to permanently prevent it from being used again.

[spikebike]

On 11/10/2013 01:19 AM, Yves Junqueira wrote:

Precision / security isn't usually a concern for most applications, so
I'm not sure that belongs to the library.

I'm not looking for security, just wanted to spend minimal time/CPU on
hosts obviously not valid peers.

What you suggest could help, but attackers can just tweak their system a
bit and then they are back on the game (e.g: they may ignore the second
request coming from a particular host).

If I get popular enough for them to target me I suspect I'll be running
my own DHT by then. In any case doing anything possible to ignore
obviously malicious peers seems like a good thing. Maybe something as
simple as once every 10 minutes generate a fake infohash and black list
anyone that replies. Certainly I could do this with a AddToBlackList
feature.

I'm OK with adding a missing API to the library: "AddToBlackList", which
keeps track of IP/port (or just IPs) that seem to be bogus, and refuses
to talk to them in next iterations. This could maybe done with a
bloomfilter to save memory.

Heh, fancy. I'm hoping there isn't that many, but I haven't tried
quantifying that. Even 1 million would only be 4MB-8MB or so. Unless
DHT got upgraded to IPv6 when I wasn't looking.

But you'd then implement the rest of the logic of finding a bogus peer
on your own application.

Sounds fair.

Your application would get the results from PeersRequestResults, then
post-process them as you suggested. If you find bogus peers that answer
to anything, then you call "dht.AddToBlackList" to permanently prevent
it from being used again.

Sounds good.

[spikebike]

On 11/10/2013 01:19 AM, Yves Junqueira wrote:

I'm OK with adding a missing API to the library: "AddToBlackList", which
keeps track of IP/port (or just IPs) that seem to be bogus, and refuses
to talk to them in next iterations. This could maybe done with a
bloomfilter to save memory.

I still think AddToBlackList is worthwhile, but the problem is not as
bad as I thought. I ran my infohash tester on 4 hosts and published the
same 16 infohashes on each. The goal being all 4 hosts fine each other.

As a result I got 150 replies from random IPs, but each one only replied
to 1 of my 16 infohashes. Maybe they are trying to avoid blacklists.

I summed the number of times a reply for each IP:

hosts
claiming
infohash

peer IP

... [145 or so snipped]
1 93.182.13.50
1 94.155.237.168
1 95.49.156.75
1 95.87.252.52
2 14.213.174.135
53 hostb (one of my hosts)
56 hosta (one of my hosts)
65 hostt (one of my hosts)
67 hostk (one of my hosts)

So basically things are working pretty well.

I'm unsure how 150 hosts could hear of my infohashes though.

Assuming 2M hosts, a random dht.PeersRequest request is going to average
talking to 11 hosts? Assuming my 4 peers end up with a different subset
of the DHT peers that's 44. Ah, that ignore malicious hosts talking to
each other.

Still sounds like a reasonable fraction of peers must be malicious.

[nictuku]

@spikebike , are you still interested in this feature? Nobody else seems to have asked for it.

[spikebike]

I am, but don't feel strongly about it. It would be nice to minimize the overhead of running the DHT, keeping evil peers out of the routing table, etc. But I can mitigate the problem outside of nictuku/dht.

[spikebike]

I just noticed Jech's DHT has a reasonable looking implementation.

In particular https://github.com/jech/dht/blob/master/dht.c lines 657 and 687

Looks like Jech thought it was a good idea.

[nictuku]

jech's blacklist has only 10 slots and it's used for blocking hosts that send malformed messages. It doesn't seem like he ever removes nodes from that list. So I'm not sure that would suit your needs.

But we do have a blacklist of sorts, but it tracks spammy clients. http://godoc.org/github.com/nictuku/nettools#ClientThrottle

It should be OK to add a separate list or map of blocked hosts there, and two methods "Block" and "Unblock". Want to send a pull request?