37 Commits

Author SHA1 Message Date
drbob
af229eb9a7 Bugfix to not count QUERYHASH in processed.
Might improve dht responsiveness.



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@6685 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2013-09-04 05:43:12 +00:00
drbob
3544bc30b2 Added Incoming Query History, for future filtering of aggressive peers.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@6681 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2013-09-03 11:09:30 +00:00
drbob
0608f67d71 Expanded DHT history analysis, and ran some more tests on DHT behaviour.
- Tweaked query parameters (quicker discard, longer requery period).

The analysis suggests that in general the DHT is running okay with some minor issues.
 - BitDHT peers respond to ~50% of FIND_NODE queries. This means we drop them, 
   and so  some ONLINE friends, become UNREACHABLE friends.

This appears to be caused by too many FIND_NODE queries reaching BitDHT, and the 
peer starts dropping them. This appears to come from specific peers, which 
are not RS nodes. Proposed solution is to Track and Filter Over-Querying Peers.
(Only one query per peer every 30 seconds).

 - Should be more lenient on the exact query matches.
Allow some dropped messages.

 - Queries appear to run longer than necessary.
A better query completion criteria needs to be established.

 


git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@6614 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2013-08-26 04:29:27 +00:00
csoler
3a72a3032a corrected debug state committed by mistake in the previous merge operation
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@5788 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2012-11-07 20:13:44 +00:00
csoler
6edb9eb4f8 Merged branch that provide group-based file permissions.
Now users can sort peers into groups in the friend list, and attribute flags and parent groups to the
directories in the share manager.

Flags are B-B-N, meaning in order:
- browsable for peers in the parent groups
- browsable for everyone
- network wide for everyone
Backward compatibility makes previously BN flags been interpreted as -BN, meaning browsable/network wide for everyone.

Be careful with this new feature. It has been tested, but it's a bit early to rely on it for highly sensitive data.

The merge also includes a significant improvement of the naming of flags with incompatible types
which should sort out some existing bugs as well, since inconsistencies in flag usage have been
found during the process.



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@5787 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2012-11-07 20:03:16 +00:00
drbob
cc9e933362 Improved the dht msg history storage.
- specify how long we store for.
 - cleanup old msgs.
 - improve printing of history.
 - add timeline storage as well.
 - disabled by default, enable USE_HISTORY in bdnode.c

There appears to be a bug related to copying bdId's around.
Some of the bootstrap ids are malformed, and this crashes rs.



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@5724 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2012-10-25 22:36:22 +00:00
drbob
83795a7c7d Fixed stupid bug. Labels are the wrong way around.
This meant limit is set to HIGH (50msg/sec) when it should be LOW (5msg/sec)



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@5723 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2012-10-25 18:50:47 +00:00
csoler
9e00688c71 removed debug info
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@5563 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2012-09-17 20:46:23 +00:00
thunder2
4e26884646 Removed some std::ostringstream.
To be continued.

git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@5111 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2012-04-19 20:44:32 +00:00
thunder2
0b1126babf Added thread safe function for inet_ntoa in libbitdht.
Used the existing thread safe function for inet_ntoa in libretroshare.

git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@5033 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2012-03-18 01:33:25 +00:00
drbob
9e5e07d3f0 Reversed Changes for Testing Purposes.
* Disabled lots of debugging.
 * Disabled LOCAL test mode in udpstunner, rsinit
 * Enabled BadPeer checking in bitdht.
 * Restored normal timeout periods for connection attempts.
 * Restored normal connection levels for dht status.



git-svn-id: http://svn.code.sf.net/p/retroshare/code/branches/v0.5-dhtmods@4779 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2012-01-11 10:24:37 +00:00
drbob
e432d89310 Improved the proxy/relay selection.
Limited Search Replies on Relay Servers.
NB: Re-check this code!



git-svn-id: http://svn.code.sf.net/p/retroshare/code/branches/v0.5-dhtmods@4766 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2012-01-08 16:14:40 +00:00
drbob
af5f23e0e7 Changes to enable a LAN Relay Test Environment.
NB: Reverse Most of these changes - before adding to Trunk!!!!

 * Added missing BitDht callback functions for test/example code.
 * Switch tests/example to OSX compile.
 * DISABLE_BAD_PEER_FILTER (as it doesn't like local net)
 * Reduced Minimum Peer Counts for DHT to be considered active.
 * Added LOCALNET_TESTING to rsinit.cc 
	- new Commandline Option -R <lport-uport> to restrict UDP ports.
	- Enables LocalNet in UdpStunner.



git-svn-id: http://svn.code.sf.net/p/retroshare/code/branches/v0.5-dhtmods@4756 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2012-01-07 11:28:00 +00:00
drbob
52e9a27f7b Improvements to DHT to support Relays.
- Updated DhtMode Flags.
 - Allowed DhtMode Flags to be set independently.
 - Set RelayMode to ConnectionManager.
 - Added dropRelayServers() / pingRelayServers()
 - Added Support fns for above two functions.



git-svn-id: http://svn.code.sf.net/p/retroshare/code/branches/v0.5-dhtmods@4729 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-12-20 02:03:25 +00:00
drbob
b6dbdf9396 Improvements to BadPeer Filter.
* Enabling Local BadPeer Filter. 
	- This will remove any peer you detect is spoofing yourself or your friends.
	- This list is also shared with you friends. (in Test Mode).
 * added Cleanup of BadPeer Filter.
	- Instead of permanent ban, peers are be banned for 6 hours.
	- bdManager periodically calls this - which prints out ban list too.
 * added #define to disable the Filter - for testing purposes.

NOTES: This Ip Filter should probably be moved from DHT level to UdpLayer level.
	This will enable it to filter STUN / UDP Connection Packets too.




git-svn-id: http://svn.code.sf.net/p/retroshare/code/branches/v0.5-dhtmods@4716 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-12-08 20:00:20 +00:00
drbob
ed18152596 * Reduced DHT default msg rate to 5/sec. (was 50/sec)
- At this level it takes significantly longer for the searches to complete, but
	  there are no DOS warnings on the router. This should hopefully make users happy.
 * Added DhtMode() Interface call to dynamically change this rate.
	- Can be switched between (HIGH: 50/sec, MED: 20/sec, LOW: 5/sec & TRICKLE: 3/sec)
 * Disable additional Queries if PotentialPeer Queue gets too big.




git-svn-id: http://svn.code.sf.net/p/retroshare/code/branches/v0.5-dhtmods@4703 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-12-01 17:19:30 +00:00
drbob
8d4a7ed4f0 Improvements to the Bad Peer tracking in the DHT.
* Added Interfaces for sharing Bad Peers.
 * Added bdPeerQueue class for storing
 * added doInfoCallback() to inform libretroshare



git-svn-id: http://svn.code.sf.net/p/retroshare/code/branches/v0.5-dhtmods@4686 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-11-25 00:48:33 +00:00
drbob
4cd7523947 Incremental Improvements to DHT:
Added Knowledge of Friends, Friends of Friends and Relays.
	- Added bdfriendlist class to store information.
	- New interface function updateKnownPeer().
	- includes IP addresses, if known. (for filtering)

Bad Peer Filtering.
	- Check IDs against known peer IP addresses.
	- Added checks in checkPotentialPeer().
	- Added checks in addPeer().
	- Running in TestMode at the moment (Bad Peers are only flagged).

Other Fixes:
	- Removed some warning about int/uint comparisons.
	- added bdSpace::flagpeer() fn to support above.



git-svn-id: http://svn.code.sf.net/p/retroshare/code/branches/v0.5-dhtmods@4680 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-11-22 13:16:03 +00:00
thunder2
27842b8592 Removed unused parameter warnings in libbitdht.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4557 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-08-12 12:43:24 +00:00
drbob
f6002b3981 Added External Interface to control "Attached" Mode
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4483 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-07-21 11:23:56 +00:00
drbob
e7049cc483 * #defined most of the Debugging info out.
* increased bdconnection timeout.



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4468 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-07-18 23:52:23 +00:00
drbob
551e2594bb Added Bandwidth and Delay parameters to BitDHT Connect Messages.
* Bandwidth are for RELAY connections.
 * Delays are for Exclusive NAT Proxy connections.
 * Lots of changes to achieve this. (Mainly added "param" to msgs).
 * Various Bugfixes too.
 * Updated BITDHT Version to 01 as we've changed Message Format.
 * Increased Connection Timeout to 180 
 * Added extra checks to kill duplicate connections early.



git-svn-id: http://svn.code.sf.net/p/retroshare/code/branches/v0.5-netupgrade@4462 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-07-17 23:29:53 +00:00
drbob
d6938721b0 Reduced timeout for connections to 20 secs. because they should be quick.
Made the query more robust, by only saying a query is SUCCESSFUL if we have recvd a reply from peer.
NB: These changes are yet to be tested!



git-svn-id: http://svn.code.sf.net/p/retroshare/code/branches/v0.5-netupgrade@4439 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-07-13 11:37:47 +00:00
drbob
5e691935fd Merged from /branches/v0.5-peernet/libbitdht:r4357-4397
Bugfixes and a couple of small improvements to the DHT code.
 * fixed up buggy overloading for the udplayer (for testing).
 * added processing multiple (5) remote processes per tick. 
     - (1 wasn't enough in testing, and led to dropped peers) 
 * removed unused variables from query data structure.
 * #defined out debugging in bdSpace, removed old functions.
 * More agressive attempts to find proxies for connections.
 * made "final query attempt" use exact peer address rather than midid.
     - mid id meant that the target wasn't always returned.
 * tweaked bdconnection debugging.
 * increase CONNECTION_MAX_TIMEOUT from 30 => 45. wasn't enough!
 * Limited bdQuery::QueryIdlePeriod to 15min (was unlimited).
 * added bdQuery::PotentialPeer cleanup functions - for more robust reporting.
 * fixed bdQuery debugging printouts.
 * Implemented BITDHT_QFLAGS_UPDATES flag.



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4398 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-07-06 00:14:58 +00:00
drbob
24351cb460 * added #define WIN32 protection for <malloc> in bdbloom
* disabled ATTACHED mode by default.
 * set reasonable default Startup parameters (had test ones before)



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4397 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-07-05 23:53:40 +00:00
drbob
79b48d5a89 Merging from branches/v0.5-peernet/libbitdht (--- Merging r4354 through r4356 into '.')
* adding Bloom filter to speed up finding RS peers.
 * fixed startup bug. (loops into FAILURE)




git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4357 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-06-29 16:06:33 +00:00
drbob
fff40eceac Merging branches/v0.5-peernet/libbitdht (Merging r4237 through r4353 into '.')
There are many significant improvements to the DHT here. 
See commit logs on v0.5-peernet branch for details.

This is not the final merge, but brings over the majority of expected v0.5-peernet/libbitdht changes 




git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4354 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-06-29 10:46:11 +00:00
drbob
5b70ef6076 A bit of debug - cleaning up.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3722 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-10-31 13:58:54 +00:00
drbob
9cf531fc9a Modified bitdht to enable simultaneous peer searches.
* Added maximum msg limits (20, 50, 100 msgs per sec). 50 = default.
 * Priority send out pings, with 10% queries.
 * switched bdquery list in bdnode to use pointers... so peers can be 
   	easily reordered (queue for queries).
 * Add all find peers, once dht initialised.
 * Added reset if FINDSELF fails. (don't know if this is a good idea!)
 * simplified DHT printouts. verbose versions are enabled with debug.



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3686 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-10-19 22:03:38 +00:00
drbob
74961774cc * Added new interface functions for start / stop DHT and to get stats.
* Implemented start/stop interface in udpbitdht and bemanager
 * added Restart of DHT if it fails to get going.
 * added start / stop functionality to bdnode and bdstore
 * added cleanup code to rest of bitdht classes.
 * reworked NetworkSize calc functions.
 * added thread debugging (prints at start / stop / join).
 * TESTS: added utest.h header file for automated unit testing (from libretroshare)
 * TESTS: bdmetric_test started conversion to automated tests
 * TESTS: udpbitdht_nettest. Added dht start / stop and network reset (thread join) functionality.
 * TESTS: fresh bdboot.txt



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3678 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-10-17 20:55:32 +00:00
drbob
cc5fa5e5dc Fixed up BITDHT versioning.
* Advertises client as DBxxxx where xxxx is provided by using app.
  * Added COMPILE switches to enable ANONYMOUS version or remove BD portion.
  * Fixed version code.
  * Added slow increase in retry timeout, to reduce DHT traffic over time.
      now RETRY PERIOD = query age / 2  
  * initialised bdToken bdVersion etc length.
  * new bdboot.txt 



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3615 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-10-05 21:57:22 +00:00
drbob
20a49a59cf Improvements to DHT, to support UDP connections.
* Changed getDhtPeerAddress() / Value() function to use const *bdNodeId.
 * Filled in getDhtPeerAddress()... to support UDP connections.
 * fixed bug in bdnode:: pong msg... wasn't checking if version existed.
 * printed out version string, as part of debug.
 * added IpAddr field to bdmanager peer search data.
 * stored IpAddr from queries... when peer found.



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3604 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-10-04 21:37:07 +00:00
drbob
c9229fb6e8 bitdht improvements
* Added bdHistory, to monitoring all the p2p dht messages (disabled normally)
 * cleaned up all compile warnings on OSX.
 * added Found timestamp to peer info.
 * disable save, until store is close to full.
 * ...?other stuff?



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3559 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-09-29 19:59:31 +00:00
drbob
f876dcb93b General Bugfixes from RS integration.
* heisenbug fixed. msgout_find_node was inside debugging #defs.
 * fixed ReplyFindNode and ReplyQueryHash counters for stats.
 * added Node Space size() functions.
 * added FINDSELF mode. 60 secs to search for own hash.
 * incremental addition of search ids.



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3326 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-08-01 18:19:27 +00:00
drbob
c1b624832f updates to libbitdht to make it work with retroshare.
* decreased nodes per bucket to 10 - to try reduce traffic.
 * changed operator< to non const.
 * fixed up debuging.



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3322 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-07-31 18:01:24 +00:00
drbob
d70337a642 First pass for windows compilation:
* brought over tou_net / tou_error => bdnet for windoze compatibility.
 * updated udplayer to inline with rs changes + win compatibility.
 * removed unix only networking #includes, replaced with "util/bdnet.h"
 * added subdirectory to #includes.
 * made udpbitdht_nettest use a random node id.



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3302 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-07-18 16:28:51 +00:00
drbob
c415bb6158 Addition of libbitdht.
============================================================

This is intended to be a completely independent library from RS, 
(hosted at sf.net/projects/bitdht) hence is being commited at the top level.

As initial further development / testing will be driven by RS integration
it is being added to the RS repository. Equally important is ensuring
that RS can compile without requiring aux libraries.

Once libbitdht is further developed, this section of the repository
is expected to be removed... But that will not be for a while, I expect.

drbob.



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3276 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-07-10 11:48:24 +00:00