Commit Graph

56 Commits

Author SHA1 Message Date
Gioacchino Mazzurco
329050a9c2
Use safer rstime_t instead of time_t
Avoid problems to serialization on different platforms, without breaking
nested STL containers serialization.

The conversion have been made with sed, and checked with grep, plus
kdiff3 visual ispection, plus rutime tests, so it should be fine.
2018-10-07 13:07:06 +02:00
csoler
6848a586f3
merged upstream/master 2018-07-08 21:17:48 +02:00
Gioacchino Mazzurco
1825b263f0
ChunkMap::reAskPendingChunk fix unused parameter warning 2018-06-09 17:21:59 +02:00
csoler
b3277824eb
license fix in pqi 2018-05-28 22:03:39 +02:00
csoler
d5627d4b22
fixed license in ft, grouter, and gxs 2018-05-22 22:03:11 +02:00
csoler
e1ad21c357
fixed wrong file count in RsCollectionDialog when downloading files 2018-03-15 13:11:19 +01:00
csoler
b3653d1283
enabled aggressive re-request of pending slices at end of transfer, thus fixing the long delay to finish files with mixed fast/slow sources 2018-03-15 11:32:55 +01:00
Phenom
3a6ad822da Fix warning: this statement may fall through
/libretroshare/src/ft/ftchunkmap.cc:190: warning: this statement may
fall through [-Wimplicit-fallthrough=]
    case FileChunksInfo::CHUNK_CHECKING: _file_is_complete = false ;

/libretroshare/src/ft/ftchunkmap.cc:191: here
    case FileChunksInfo::CHUNK_DONE:   _total_downloaded +=
sizeOfChunk(i) ;
2017-07-16 20:27:09 +02:00
csoler
22d4893721 removed a few compilation warning that might cause compilation errors on some systems 2015-09-24 18:03:19 -04:00
csoler
a1b36cd530 added method in ftServer to allow retrieving data from shared/downloaded files (patch from electron, modified)
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@7999 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2015-03-08 13:46:07 +00:00
chrisparker126
0f29d28b1b merge of branch v0.6-idclean 7180
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@7187 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2014-03-17 20:56:06 +00:00
csoler
162ef94b70 extended max chunk TTL to 1 hour. A short TTL is not anymore needed since chuns are shared between sources
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@6935 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2013-12-09 13:35:06 +00:00
csoler
4e4e81fa8e fixed uninitialized memory read in debug statement
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@6203 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2013-03-10 22:29:25 +00:00
csoler
4c3905e257 simplified the chunk drawing code a little bit
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@6177 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2013-03-01 23:41:42 +00:00
csoler
1fb605cfa9 fixed stupid bug causing streaming mode to get stalled
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@6176 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2013-03-01 23:16:09 +00:00
csoler
4f1dda2ffb added new DL mode: Progressive. This is a random mode that ensures no more than 50MB of free space will be written in the file at once. This should effectively remove the lag when initiating a DL on a big file, while keeping enough entropy to ensure a good dissemination of chunks
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@6164 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2013-02-28 20:42:01 +00:00
csoler
b2214fcd46 removed #ifdef USE_NEW_CHUNK_CHECKING_CODE, since it is enabled by default now. Also removed the use of assume_availablility to put chunks or not in checking mode. This was inconsistent. As a side effect, cache files of size > 1MB get chunk-checked as well, which is good
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@6050 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2012-12-27 21:16:33 +00:00
csoler
d3a3efd492 added additional checking of incoming compressed chunk map, in case the data is currupted, which might happens with a corrupted ft_transfer.cfg file.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@5816 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2012-11-13 20:13:09 +00:00
csoler
24f8ee6222 Added dynamic choose of sources for chunk crc requests. Fixes the bug that would let
a transfer not finish if the original surce for a crc is not here anymore.

If you have a unfinished transfer do a force-check after restart to get back chunks that 
where still on verificaiton stage.



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@5315 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2012-07-19 20:52:04 +00:00
csoler
0d3d1ebc18 fixed bug preventing files < 1MB to terminate properly
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@5240 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2012-06-21 19:24:14 +00:00
csoler
36198b7e6a Enabled full chunk checking code:
- unchecked chunks are not made available to swarming sources, not saved as done.
- force check now uses the simple method to put all chunks in checking mode
- force checked files can be cancelled (finally!)
- improved display (use red for active chunks, yellow for checking)
- cache file are not using chunk checking (assume_availability=true)
The code still contains the #ifdef. It should be removed soon if everything works ok.



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@5235 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2012-06-19 21:36:10 +00:00
csoler
f5c345659a Improved multi-source file transfer in two ways:
- tunnels can borrow data chunks from only slower tunnels, to prevent blocking fast tunnels
- tunnels can handle up to 50 data chunks at once instead of 5
Multi-source transfers are generaly faster and more stable.


git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@5156 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2012-05-09 21:31:05 +00:00
csoler
08a52dc369 fixed bug causing some transfers to never finish
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@5063 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2012-03-30 19:33:58 +00:00
csoler
8f69251334 small improvements to the chunk checking algorithm
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@5047 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2012-03-24 14:45:33 +00:00
csoler
889a2b2433 Added deferred check of chunks during file transfer. Chunk sha1 sums are requested to the sources and checked for downloaded data.
Validated chunks are shared to other peers. Force check is now very simple since it just turns all chunks into "needs checking" mode
and sums are asked to sources. Sources maintain a temporary cache of chunks. Since sums are requested sparsely, this should not 
affect the sources in terms of performance. We can still imagine precomputing and saving sha1 of chunks while hashing them.

For backward compatibility reasons, the following has been setup *temporarily* in this version:
- unvalidated chunks are still considered as already obtained, and are shared and saved
- force check has been disabled
- final file check is maintained
- in case of file fail, the old checking mode will be used.

All changes for next version are kept in the define 'USE_NEW_CHUNK_CHECKING_CODE' that will be made the default in a few weeks.
At start, I expect most chunk to stya yellow during download, until most sources are able to provide chunk hashs.




git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@5019 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2012-03-15 19:55:43 +00:00
csoler
2edb61758b Improvements to file transfer:
- corrected 2 potential flaws of swarming causing request of unavailable data (This should normally be taken care of on the server side!)
- improved swarming test code with fault simulation
- only ask CRC32 maps to peers that have a complete file.
 


git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4947 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2012-02-16 21:10:46 +00:00
csoler
f441f4a407 set max chunk inactivity period to 5 mins instead of 1 mins. Shoudl favor slow DLs. This is nw possible since missing bits will be re-asked regularly
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4925 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2012-02-11 15:26:04 +00:00
csoler
fa690de1d6 small bug fixes about CRC32 verification when the file is actually complete.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4662 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-11-04 21:29:51 +00:00
csoler
1fc8c5ef70 implemented slice sharing between peers.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4650 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-10-23 18:43:58 +00:00
csoler
f5bfc8c92a changed a bit the file transfer strategy. To be tested further.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3898 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-12-07 17:09:21 +00:00
csoler
34a3fc4ea8 removed useless include
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3854 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-11-22 21:01:54 +00:00
csoler
b94e4376a4 only assume availability for cache transfers. All other transfers now use ChunkMaps
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3853 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-11-22 20:57:53 +00:00
csoler
c00e11ed2b switched random chunk strategy to true random
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3777 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-11-10 22:05:34 +00:00
csoler
f2406d4e57 improved random chunk strategy, to limit cluster effects on files partially downloaded in streaming
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3764 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-11-08 22:09:43 +00:00
csoler
8369d42600 corrected a bug in chunkmaps that affected file creators being providers at the same time, which could corrupt data. Added some debug info to chunk maps.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3673 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-10-15 22:12:29 +00:00
csoler
d71dd2889d added safety check to ftChunkMap call to p3turtle, mainly for tests to run
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3582 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-10-01 20:07:29 +00:00
leander-256
4bb4fc11e2 This intimidating patch renames libretroshare/src/rsiface/ to
libretroshare/src/retroshare/

All the relevant headers have been modified to reflect that change.

This allows installation of libretroshare on a system, headers will
be put in $WHEREVER/retroshare/ and we keep the ability to compile
against them, be it on the system or in the SVN tree.



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3342 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-08-06 09:40:23 +00:00
csoler
8bfc74485a First implementation of per-chunk CRC32 check. This is triggered
- by the right-click+Force Check on files.
- when a global hash on a downloaded file does not match the announced hash.

When a CRC map check is ordered, the CRC map is requested to one of the sources for the current file download.
When received, all downloaded chunks are checked w.r.t the reference CRC and marked as not done if the CRCs do not match.
The exchange of CRC32 map and requests has been tested, as well as CRC map checking during download (force check).

To be implemented soon:
    - caching of CRC32 maps (although these are fast to compute)
    - CRC32 map packets for normal downloads. For now these work only for turtle tunnels.
    - handling of errors if the CRC never comes. For now, the download will stay stuck in "Checking..." mode.

So, don't play too much with the force check feature for now...





git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3310 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-07-21 23:14:10 +00:00
csoler
c8fc75c75c added some safety checks to setting chunk strategy against inconsistent values.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3236 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-06-29 20:51:16 +00:00
csoler
5f7486f160 added proper dynamic suppression of sources from chunkmap, and updated the gui
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@2498 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-03-07 14:42:13 +00:00
csoler
e3f2f66702 suppressed bug in file transfer while removing inactive chunks.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@2468 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-03-01 20:46:29 +00:00
csoler
851e9069f6 corrected compled completion and file progress issues
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@2356 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-02-17 22:10:12 +00:00
csoler
5926c79822 corrected bug in file completion
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@2301 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-02-13 20:42:49 +00:00
csoler
e08d13397a fixed a completion issue in FT
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@2174 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-02-02 22:15:42 +00:00
csoler
a70b106005 - added NETWORK_WIDE flag to remote dir model download
- added file priority strategy based on which files are requested first. This provides:
	- equal file speed for files with equal (source,priority)
	- effective priority speed for file of same source but different priority
- removed state variable load/save from turtle, as it's not needed anymore (FileRequest re-opens tunnels as needed)
- manage availability per peer instead of per file type: direct peer ids always assume file availability, while turtle tunnels don't

I still need to make the download queue work, and code this gui for it.



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@2133 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-01-26 20:40:21 +00:00
csoler
0e7f9df638 made contextual menus in FT more contextual, added info about transfer type and chunking strategy, load/save of transfer flags
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@2114 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-01-22 23:23:37 +00:00
csoler
e2b807aad7 added regular cleaning of inactive chunks and slices, so that they can be treated by a different peer
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@2102 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-01-21 12:31:00 +00:00
joss17
f481dbef59 add include for latest gcc. Comment usleep fonction in tou_net.h
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@2101 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-01-20 22:02:43 +00:00
csoler
96147a9734 added flags to file request: RS_FILE_HINTS_ASSUME_AVAILABILITY to not wait/ask for chunk maps. Used for cache transfer and also whenever the turtle traffic is not involved, such as chat file exchange, and RS_FILE_HINTS_NETWORK_WIDE to force using turtle routing for traffic. Also moved turtle tunnel start/stop orders from ftServer to ftController
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@1995 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-01-11 22:38:18 +00:00
csoler
f02b009d7b fixed win32 compilation
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@1994 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-01-11 20:10:04 +00:00