Commit Graph

36 Commits

Author SHA1 Message Date
csoler
8f69251334 small improvements to the chunk checking algorithm
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@5047 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2012-03-24 14:45:33 +00:00
csoler
633b66b1dd fixed missign delete in hash computation code
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@5038 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2012-03-19 13:50:33 +00:00
csoler
c9f99c0be9 removed debug info
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@5030 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2012-03-17 14:45:52 +00:00
csoler
456177ef11 fixed bug responsible for sending wrong checksums on windows systems
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@5029 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2012-03-17 14:08:27 +00:00
csoler
889a2b2433 Added deferred check of chunks during file transfer. Chunk sha1 sums are requested to the sources and checked for downloaded data.
Validated chunks are shared to other peers. Force check is now very simple since it just turns all chunks into "needs checking" mode
and sums are asked to sources. Sources maintain a temporary cache of chunks. Since sums are requested sparsely, this should not 
affect the sources in terms of performance. We can still imagine precomputing and saving sha1 of chunks while hashing them.

For backward compatibility reasons, the following has been setup *temporarily* in this version:
- unvalidated chunks are still considered as already obtained, and are shared and saved
- force check has been disabled
- final file check is maintained
- in case of file fail, the old checking mode will be used.

All changes for next version are kept in the define 'USE_NEW_CHUNK_CHECKING_CODE' that will be made the default in a few weeks.
At start, I expect most chunk to stya yellow during download, until most sources are able to provide chunk hashs.




git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@5019 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2012-03-15 19:55:43 +00:00
csoler
d43a131c04 Improved CRC32Map checking in several ways:
- servers now compute the map in a separate thread
- CRC32Maps are kept in cache for 30 mins
- CRC32Maps requests cannot be used to overflood a server anymore since their number is limited.
- Transfer modules now send keep alive packets to maintain tunnels when asking for a CRC32Map



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4661 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-11-03 23:18:00 +00:00
csoler
c8ea41a4db corrected bug that would allow a direct friend to DL a file that is NETWORK_WIDE only, using direct transfer
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4648 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-10-21 22:25:18 +00:00
thunder2
c6beee44ec Removed unused parameter warnings in libretroshare.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4558 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-08-12 13:42:30 +00:00
thunder2
de87a89437 Added a name to the RsMutex class.
You can enable the debugging of the waiting time for a lock of RsMutex in rsthreads.h with
#define RSMUTEX_DEBUG 300
That means all locks waiting longer than 300ms are logged into the stderr.

git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4392 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-07-04 22:59:39 +00:00
csoler
c9911694ff corrected a bug that caused file descriptors to never get closed. Adding some tracing comments.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3958 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-01-08 23:27:33 +00:00
csoler
9ec813b4b8 corrected quite subtle bug in ftdatamultiplex, due to deleting a filecreator that is also a file server.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3879 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-11-29 20:46:21 +00:00
thunder2
ab5b2a3851 Optimized parameter "hash" of method RsFiles::FileDetails from "std::string" to "const std::string&"
Removed compiler warning of FileEntry.
Recompile of the GUI needed.

git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3703 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-10-27 20:01:31 +00:00
csoler
012808ea00 - Implemented CRC32 map traffic through direct downloads
- replaced the findItems() call into a hand-written search (this was causing a rare bug in displaying downloads)


git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3318 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-07-29 21:07:07 +00:00
csoler
81203b3cfd - modified ftFileProvider so as to keep multiple client peers per instance
- updated TransfersDialog to display this info correctly


git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3316 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-07-27 16:05:21 +00:00
csoler
cee8600a93 - implemented bidirectional exchange of chunk maps for direct downloads, to allow showing proper completion of transfers from direct friends.
- moved the direction flag upward in the pipeline (ftDataSend instead of p3turtle)




git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3313 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-07-25 19:04:31 +00:00
csoler
8bfc74485a First implementation of per-chunk CRC32 check. This is triggered
- by the right-click+Force Check on files.
- when a global hash on a downloaded file does not match the announced hash.

When a CRC map check is ordered, the CRC map is requested to one of the sources for the current file download.
When received, all downloaded chunks are checked w.r.t the reference CRC and marked as not done if the CRCs do not match.
The exchange of CRC32 map and requests has been tested, as well as CRC map checking during download (force check).

To be implemented soon:
    - caching of CRC32 maps (although these are fast to compute)
    - CRC32 map packets for normal downloads. For now these work only for turtle tunnels.
    - handling of errors if the CRC never comes. For now, the download will stay stuck in "Checking..." mode.

So, don't play too much with the force check feature for now...





git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3310 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-07-21 23:14:10 +00:00
csoler
531567997d branch commit 3185: removed deadlock situation in data multiplex
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3186 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-06-21 19:10:55 +00:00
csoler
b80aef1228 Suppressed the possibility for browsable only files to be searched by hash from turtle router.
This avoids that an attacker that captures a broadcast hash request can also ask for the file.

This needed to add back explicit sources in SearchDialog and private chat transfer. I dynamically search
for sources in file lists just before calling FileRequest.

When RS links anchorClick() will work properly with Qt, we can remove the search flag "EXTRA" from the
tunnel digging in p3turtle.cc, otherwise, people having this bug can't transfer files in private chat by
copy+paste of a RS link.
    
	Also:
	    - added missing mutexes in search requests into fimonitor.cc
		- suppressed some debug info in connect manager
		- added check for write fails when dumping config files



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3027 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-05-29 13:17:09 +00:00
csoler
48556396ee added a check to avoid (possibly rare) data races
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@2901 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-05-14 11:29:15 +00:00
csoler
08065b4298 ported branch v0.5.0 commits 2623, 2624, 2626, 2633, 2634, 2643, 2644, 2646, 2651 and 2654 to trunk
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@2660 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-03-28 20:46:45 +00:00
csoler
154ef63474 removed potentially harmful code. Warning: needs make clean in libretroshare
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@2335 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-02-15 20:44:37 +00:00
csoler
cfaaec31c7 - Implemented chunk-based file transfer from partial sources. This in particular means:
- exchange of chunk availability maps from different peers
    - correct handling of what is available to which source before asking the data
    - correct display of chunks in the progress bars
    - generalised the use of compressed chunk maps
    - removed the size parameters from the hash search functions
   
- In addition:
    - suppressed a number of per-value transfers of std::string
    - improved the FileTransferInfo Widget, to show some additional info

Still to be done:
    - chunk map exchange for non anonymous traffic (easy)
    - improve accuracy of completion for uploads (for now it's a integer number of chunks)
    - check compilation on windows




git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@1993 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-01-11 16:00:42 +00:00
csoler
b83e894640 - added a generic RsItem to the turtle router and the methods to route it. This makes the code much more elegant.
- suppressed a cross mutex lock bug that happened rarely while digging tunnels
- changed FileDetails in ftServer so that it's now possiblt to search for hashes of files being downloaded
- improved the search code in ftdatamultiplex
- added some comments to the turtle code



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@1964 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-01-02 21:30:19 +00:00
csoler
f4a2eaecce - implemented a visualisation for currently handled chunks, availability maps, and transfer info
- implemented transfer protocol for chunk availability maps between peers (not enabled yet though)
- suppressed rsiface directory from retroshare-gui
- moved notifyqt.{h,cpp} to gui/

next moves: 
	- send availability maps to clients; 
	- connect turtle search to unfinished files;
	- test multisource download with unfinished files.



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@1939 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2009-12-28 21:11:00 +00:00
csoler
c9467a2bfe this is a fix for the file download problem. It does not correct the wrong chunk size, but it checks that the malloc returns something.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@1442 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2009-07-27 20:31:56 +00:00
csoler
1046bdd846 First commit for the turtle download. It works without perturbating RS traffic, but still needs some (internal) smoothing
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@1275 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2009-05-26 21:42:45 +00:00
csoler
98911955d5 Possibly got rid of this core dump in file uploads. It seems it came from unlocked destroying of dataplex->mServers when they came out of time, while the gui still asked for info about them.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@1030 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2009-02-09 20:26:12 +00:00
csoler
a3d3778c36 Added info for upload peers and rate in Transfer tab
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@1005 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2009-02-04 21:35:39 +00:00
drbob
c60cf84e95 * Switched off lots of debugging.
* Tweaks to the transfer rates.


git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@824 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2008-11-18 21:22:58 +00:00
drbob
80f4686100 Bugfixes and extra debugging for file transfer.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@791 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2008-11-04 23:12:53 +00:00
drbob
88ef2e0e34 Major bugfixes and testing changes for ft.
It is now almost ready for integration with rs.

Added ftserver3test.cc which successfully tests:
	* Hashing of a directory.
	* Transfer of Cache Files.
	* Local Searching.
	* Remote Searching.
	* Download from multiple sources.




git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@773 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2008-10-29 20:58:23 +00:00
drbob
f7fca4295b Improvements/bugfixes to File Transfer.
* Lots more debugging messages.
 * Fixed Sleep / sleep issue on windows.
 * added pthread / WSAStartup.
 * added ownId to ftDataMultiplex for loopback file transfer.
 * now start ftDataMultiplex thread.
 * several bugfixes in ftfilecreator



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@710 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2008-08-30 01:07:24 +00:00
drbob
f243e5f6d3 * Added lots of debugging.
* Partial bug fix for allocate algorithms.



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@706 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2008-08-23 21:28:08 +00:00
drbob
189dcbb482 Completed most of ftserver class.
Also changes to make everything fit together.
changes to Interface to support srcId list.



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@675 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2008-08-09 17:03:24 +00:00
drbob
5c6e558942 More Improvements to FileTransfer:
* Added TlvShallowClear() to serialisers
 * Added RsQueueThread for periodic queue processes.
 * Completed ftDataMultiplex which replaces ftServer/ClientModules.
 * Added Server Queue / Thread to ftDataMultiplex.
 * Added ftdataplextest to exercise ftDataMultiplex.
 * Generalised ftFileSearch to handle an array of ftSearch classes.
 * Tweaked rsfiles.h #defines to match new ftFileSearch scheme.
 * Added Generic ftData Interfaces for Testing.
 * added ftDataSend/Recv Interfaces to ftServer + ftDataMultiplex respectively.
 * Completed much of ftServer (External Interface), but not yet done.
 * Extra debugging and small changes to ftExtraList
 * Integrated new ftTransferModule with minor interface changes.



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@660 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2008-08-03 12:45:53 +00:00
drbob
79727897dd Addition of next stage of new FileTransfer Code:
* Completed rough ftExtraList class (with Test Case)
 * Added data flow interface (ftData.h)
 * Added ftDataMultiplex (server + client modules).
 * Finished parts of ftcontroller / ftserver.
 * Minor Tweaks to ftTransferModules interface for compilation.
Related Changes in other parts of the code:
 * Added new Job/Queue Thread Class.
 * Added more user-friendly directory functions.
 * Added FileInfo print operator.




git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@650 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2008-07-23 22:01:59 +00:00