Commit Graph

72 Commits

Author SHA1 Message Date
csoler
028c9f8c07 split lastRecvTimeStamp into two different time stamps: one for last time the file is wrote (or data is received) and one for last activity that is used (and sometimes reset) by ftcontroller when queuing files
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@6921 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2013-12-04 20:26:54 +00:00
csoler
2e5069cc2a removed debug output
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@6898 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2013-11-08 21:24:57 +00:00
csoler
9c4c5da73d Changes according to experiments from jolavilette:
- set desiredRate of peers to 10MB/s instead of 1MB/s
- maximum number of active chunks per peer is now 20 (that was a serious limiting factor)
- ticks that bring no data are not accounted for in the speed estimate anymore. The data is averaged over all ticks since the previous 
 non zero data chunk -> this brings much more stability to sources and removes lots of oscillation.

The maximum theoretical speed per source peer is therefore 13MB/s (in "Average" DL mode), whereas previously it was 1.3MB/s



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@6897 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2013-11-08 21:23:57 +00:00
thunder2
9095e9bd75 Code maintenance for Qt 5:
- Added some more includes needed by MinGW-w64

git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@6860 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2013-10-21 11:00:49 +00:00
csoler
ef2ebcc1d6 removed unused code
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@6405 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2013-06-05 12:55:48 +00:00
csoler
4b9087b60d removed dead-end for peers not answering downloads, that would get file cache transfers to get stuck indefinitly when peers are too much stressed for a short period. The offline state is entirely given by setPeerState()
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@6206 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2013-03-11 20:33:49 +00:00
csoler
b2e97407b5 added dynamic addign of direct sources from file lists. Useful for channels. Is called once every 61 seconds.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@5850 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2012-11-19 22:45:37 +00:00
csoler
1a8c9d9416 patched fttransfermodule to always fulfill the data suze request, even if the file creator returns small (old pending) chunks, or stops at a chunk boundary. This should improve data flow and hep achievign hight speeds on fast internet connexions
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@5505 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2012-09-02 20:41:25 +00:00
csoler
24f8ee6222 Added dynamic choose of sources for chunk crc requests. Fixes the bug that would let
a transfer not finish if the original surce for a crc is not here anymore.

If you have a unfinished transfer do a force-check after restart to get back chunks that 
where still on verificaiton stage.



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@5315 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2012-07-19 20:52:04 +00:00
csoler
36198b7e6a Enabled full chunk checking code:
- unchecked chunks are not made available to swarming sources, not saved as done.
- force check now uses the simple method to put all chunks in checking mode
- force checked files can be cancelled (finally!)
- improved display (use red for active chunks, yellow for checking)
- cache file are not using chunk checking (assume_availability=true)
The code still contains the #ifdef. It should be removed soon if everything works ok.



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@5235 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2012-06-19 21:36:10 +00:00
csoler
889a2b2433 Added deferred check of chunks during file transfer. Chunk sha1 sums are requested to the sources and checked for downloaded data.
Validated chunks are shared to other peers. Force check is now very simple since it just turns all chunks into "needs checking" mode
and sums are asked to sources. Sources maintain a temporary cache of chunks. Since sums are requested sparsely, this should not 
affect the sources in terms of performance. We can still imagine precomputing and saving sha1 of chunks while hashing them.

For backward compatibility reasons, the following has been setup *temporarily* in this version:
- unvalidated chunks are still considered as already obtained, and are shared and saved
- force check has been disabled
- final file check is maintained
- in case of file fail, the old checking mode will be used.

All changes for next version are kept in the define 'USE_NEW_CHUNK_CHECKING_CODE' that will be made the default in a few weeks.
At start, I expect most chunk to stya yellow during download, until most sources are able to provide chunk hashs.




git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@5019 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2012-03-15 19:55:43 +00:00
csoler
2d2b7230d1 added early sending of chunkmap request to newly added sources
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4950 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2012-02-16 22:58:30 +00:00
csoler
2edb61758b Improvements to file transfer:
- corrected 2 potential flaws of swarming causing request of unavailable data (This should normally be taken care of on the server side!)
- improved swarming test code with fault simulation
- only ask CRC32 maps to peers that have a complete file.
 


git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4947 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2012-02-16 21:10:46 +00:00
csoler
4ab319f29e Adjustment of some values for file transfer: some delays can be larger, now that chunks are shared between sources. This makes file transfer more stable:
- FT_TM_DOWNLOAD_TIMEOUT: 10 -> 20 secs (RTT limit of 10 secs is too low -> transfers in long tunnels kept switching between active and inactive)
- INACTIVE_CHUNKS_CHECK_DELAY: 60 -> 240 secs (Chunks are kept longer before beign cancelled, which is now affordable)
- MAX_TIME_INACTIVE_REQUEUED: 60 -> 120 secs.(Transfers a kept longer before being re-prioritized in queue)



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4707 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-12-03 13:39:37 +00:00
csoler
4d9727f47a - fixed download priority handling. Proof-tested on multiple DL from the same peer.
- removed dependency of mRateIncrease into measured rtt -> would cause very slow transfers through long tunnels



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4671 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-11-16 20:30:20 +00:00
csoler
fa690de1d6 small bug fixes about CRC32 verification when the file is actually complete.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4662 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-11-04 21:29:51 +00:00
csoler
d43a131c04 Improved CRC32Map checking in several ways:
- servers now compute the map in a separate thread
- CRC32Maps are kept in cache for 30 mins
- CRC32Maps requests cannot be used to overflood a server anymore since their number is limited.
- Transfer modules now send keep alive packets to maintain tunnels when asking for a CRC32Map



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4661 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-11-03 23:18:00 +00:00
csoler
97ed1d534f - removed calls to rsfiles->get{Download,Partials}Directory() in RsDiscSpace class, since it would trigger a call to ftController
- added a lock into ftTransferModule::recvFileData() (Crash reported by Costa due to storing data in a deleted transfer module)
- changed names of functions in ftTransferModules to locked_* when appropriate (helps debugging)



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4612 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-09-23 21:08:11 +00:00
thunder2
c6beee44ec Removed unused parameter warnings in libretroshare.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4558 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-08-12 13:42:30 +00:00
thunder2
de87a89437 Added a name to the RsMutex class.
You can enable the debugging of the waiting time for a lock of RsMutex in rsthreads.h with
#define RSMUTEX_DEBUG 300
That means all locks waiting longer than 300ms are logged into the stderr.

git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4392 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-07-04 22:59:39 +00:00
drbob
b683e663d6 Merged Changes /branches/v0.5-peernet/libretroshare/:r4237-4358
Major changes are:
 * Improvements to tcponudp library to allow multiple UdpStacks / ports, with alternative recievers.
 * Resurrected the UdpStunner code, and improved it.
 * Added UdpRelay code.
 * Modified startup code and ssludp code to use the new tcponudp and add a stunner.
 * fixed buggy rs_inet_ntoa
 * fixed a bunch of apple gcc warnings. mainly for(;;); => for(;;) ;

These changes shouldn't affect libretroshare stability... those changes will follow!




git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4359 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-06-29 18:02:44 +00:00
csoler
451c75813d - added a missing pthread_join in file checking, that eventually caused a leak of thread resources
- changed the order of check in tick() to allow cancelling a file that is being checked
- removed the hash step from mutex protection



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4113 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-03-30 22:35:09 +00:00
csoler
827dbd8cf3 added a memory management class for small objects.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4046 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-02-20 21:36:21 +00:00
thunder2
db19db7e60 Optimized some small things in TransferDialog and SearchDialog.
Changed the parameter of the constructor of ftFileCreator and ftFileProvider and some methods of ftTransferModule from "std::string" to "const std::string&"

git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4045 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-02-20 00:59:47 +00:00
csoler
9de341953d removed duplicate hashing function. Now the hash buffer is always 512 bytes.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4038 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-02-14 19:59:06 +00:00
csoler
1e28183c2d changed minimum chunk size to 1KB/s and max desired rate to 10MB/s
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3796 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-11-14 21:38:12 +00:00
drbob
7203229126 Cleaned up Transfer debug, now that we've found the windows bug.
* new uniform #define DEBUG_TRANSFERS to switch on packet timings.
 * switched off by default.
 * enabled BinData printing again.



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3785 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-11-13 11:46:17 +00:00
drbob
06273fa775 Hackish changes to debug Windows Transfers...
Committing these so others can run tests.
Most of these changes will be removed once the problem is fixed!
 * Fixed formatting of various serialiser item::print() fns.
 * Disabled printout of TlvBinaryData - data.
 * added AccurateTimeString() function to util/rsprint.cc
 * print serialise / deserialise of FileRequest & FileData + Timestamps.
 * added print + timestamp in ftFileProvider::getFileData()
 * #define TRANSFER_DEBUG to isolate these changes.
 * enabled debugging in fttransfermodule.cc



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3779 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-11-11 20:51:17 +00:00
csoler
af8ad4e862 removed the "emergency shutdown" feature in tickPeerTransfer(), that would stuck some transfers on the lowest rate 128 B/s. On my tests this emergency shutdown is never needed.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3676 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-10-16 21:09:48 +00:00
csoler
47b48769e4 simplified the code in the post download CRC check routine and removed a potential bug.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3574 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-09-30 21:02:37 +00:00
leander-256
4bb4fc11e2 This intimidating patch renames libretroshare/src/rsiface/ to
libretroshare/src/retroshare/

All the relevant headers have been modified to reflect that change.

This allows installation of libretroshare on a system, headers will
be put in $WHEREVER/retroshare/ and we keep the ability to compile
against them, be it on the system or in the SVN tree.



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3342 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-08-06 09:40:23 +00:00
csoler
1069ed0c58 - corrected 1 bug in serialisation of CRC32Maps for peer transfer
- corrected 1 bug (missing hashing thread deletion on failure)


git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3321 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-07-30 21:01:51 +00:00
csoler
012808ea00 - Implemented CRC32 map traffic through direct downloads
- replaced the findItems() call into a hand-written search (this was causing a rare bug in displaying downloads)


git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3318 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-07-29 21:07:07 +00:00
csoler
cee8600a93 - implemented bidirectional exchange of chunk maps for direct downloads, to allow showing proper completion of transfers from direct friends.
- moved the direction flag upward in the pipeline (ftDataSend instead of p3turtle)




git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3313 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-07-25 19:04:31 +00:00
csoler
8bfc74485a First implementation of per-chunk CRC32 check. This is triggered
- by the right-click+Force Check on files.
- when a global hash on a downloaded file does not match the announced hash.

When a CRC map check is ordered, the CRC map is requested to one of the sources for the current file download.
When received, all downloaded chunks are checked w.r.t the reference CRC and marked as not done if the CRCs do not match.
The exchange of CRC32 map and requests has been tested, as well as CRC map checking during download (force check).

To be implemented soon:
    - caching of CRC32 maps (although these are fast to compute)
    - CRC32 map packets for normal downloads. For now these work only for turtle tunnels.
    - handling of errors if the CRC never comes. For now, the download will stay stuck in "Checking..." mode.

So, don't play too much with the force check feature for now...





git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3310 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-07-21 23:14:10 +00:00
drbob
648e15bcf7 Found cause of OSX network resets / lost connections: inet_ntoa is not thread-safe.
* Added thread-safe rs_inet_ntoa function (util/rsnet.cc)
 * Switched all calls to this function (most of the changes)
 * Modified getLocalInterfaces() and getPreferredInterface() to avoid string conversions.
   - NB: Modified windows functions too, but unable to test (hope it compiles!)

Also:
 * Added  EVP_CIPHER_CTX_rand_key() replacement function for old versions of SSL (pre 0.9.8)  (for OSX 10.5)
   - NB: This code should be reworked to remove these calls anyway.
 * Updated tests/pqi to handle above changes.



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3281 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-07-10 20:34:03 +00:00
csoler
fbd8a33db0 added display of checking stage in GUI (at end of transfer). Gave names to the states in ftTransferModule. Added a flag for ForceCheck files (not functional yet)
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3274 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-07-09 21:04:29 +00:00
csoler
458a8faf70 - implemented post-download hash re-check. For now, if the hash does not match, the download is canceled, but in the near future, per-chunk comparison wil occur
.
- corrected a bug that caused file copy error: a closeFile() was missing when the file is complete. Because of delays in fwrite, the file would not be always co
mplete, nor exist at all for small files (e.g. cache files), which in the later case caused the copy error.

Warning: needs a make clean in libretroshare to recompile.



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3261 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-07-06 05:04:11 +00:00
csoler
cc17744ee3 ported branch v0.5.0 commits 2529 and 2544 to trunk
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@2545 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-03-15 12:57:08 +00:00
csoler
851e9069f6 corrected compled completion and file progress issues
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@2356 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-02-17 22:10:12 +00:00
csoler
365ef569dc suppressed some uninitialised memory reads
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@2267 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-02-09 20:31:49 +00:00
csoler
e08d13397a fixed a completion issue in FT
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@2174 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-02-02 22:15:42 +00:00
csoler
cfaaec31c7 - Implemented chunk-based file transfer from partial sources. This in particular means:
- exchange of chunk availability maps from different peers
    - correct handling of what is available to which source before asking the data
    - correct display of chunks in the progress bars
    - generalised the use of compressed chunk maps
    - removed the size parameters from the hash search functions
   
- In addition:
    - suppressed a number of per-value transfers of std::string
    - improved the FileTransferInfo Widget, to show some additional info

Still to be done:
    - chunk map exchange for non anonymous traffic (easy)
    - improve accuracy of completion for uploads (for now it's a integer number of chunks)
    - check compilation on windows




git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@1993 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-01-11 16:00:42 +00:00
csoler
25a09900e9 - Added a ChunkMap class responsible for allocating new chunks to be downloaded, according to
- a given chunk strategy
	- the availablility map of each source 
- Integrated this into ftFileCreator
- added gui menu in file transfer+right click to change the chunk strategy: streaming vs. random

Next step: 
	- loading/saving file downloading state and availability map
	- displaying chunk details in the selected transfer tab (e.g. list of currently worked chunks, and their current downloading completion)



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@1863 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2009-12-08 22:29:52 +00:00
csoler
211acc6adf suppressed file download memory leak
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@1294 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2009-06-11 08:09:08 +00:00
csoler
48218e98cb turtle tunnel management
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@1289 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2009-06-03 18:47:14 +00:00
csoler
1046bdd846 First commit for the turtle download. It works without perturbating RS traffic, but still needs some (internal) smoothing
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@1275 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2009-05-26 21:42:45 +00:00
csoler
78c07ca367 Turned RsInit into a static class, and changed retroshare invite so that it does not contain signatures anymore.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@1022 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2009-02-08 14:30:28 +00:00
drbob
d89eda510f * switched off some debugging.
* fixed tickrate slow down (when no activity)
 * added extra flags to interface.



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@871 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2008-12-07 14:53:57 +00:00
ewensun
00a0fdfbbf add code for cancen file transfer
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@854 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2008-11-29 22:03:36 +00:00