Commit graph

71 commits

Author SHA1 Message Date
chrisparker126
0f29d28b1b merge of branch v0.6-idclean 7180
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@7187 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2014-03-17 20:56:06 +00:00
csoler
235280399f - changed Sha1CheckSum from hand-made class to instance of t_GenericIdType<> (should be backward compatible)
- updated ft/, p3msgservice and p3chatservice accordingly
- added a new class for Sha1Sum in t_GeneridIdType<>, and an additional template argument to make ids of identical size 
	incompatible.



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@7082 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2014-02-01 19:10:45 +00:00
thunder2
e269128bdc Fixed compile on Windows.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@6923 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2013-12-05 17:44:46 +00:00
csoler
028c9f8c07 split lastRecvTimeStamp into two different time stamps: one for last time the file is wrote (or data is received) and one for last activity that is used (and sometimes reset) by ftcontroller when queuing files
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@6921 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2013-12-04 20:26:54 +00:00
csoler
9c4c5da73d Changes according to experiments from jolavilette:
- set desiredRate of peers to 10MB/s instead of 1MB/s
- maximum number of active chunks per peer is now 20 (that was a serious limiting factor)
- ticks that bring no data are not accounted for in the speed estimate anymore. The data is averaged over all ticks since the previous 
 non zero data chunk -> this brings much more stability to sources and removes lots of oscillation.

The maximum theoretical speed per source peer is therefore 13MB/s (in "Average" DL mode), whereas previously it was 1.3MB/s



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@6897 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2013-11-08 21:23:57 +00:00
csoler
3f09bdb0b7 removed false warning that come up when openning a non existing file in ftFileCreator.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@6896 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2013-11-07 21:17:18 +00:00
thunder2
9095e9bd75 Code maintenance for Qt 5:
- Added some more includes needed by MinGW-w64

git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@6860 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2013-10-21 11:00:49 +00:00
csoler
4868a90d81 patch from Henry morgan to fix soem typos (e.g misspelled connexion into connection)
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@6460 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2013-06-24 21:23:50 +00:00
csoler
2dcc35d979 further remove of debug info
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@6247 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2013-03-19 20:21:20 +00:00
csoler
87842640e1 removed debug output
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@6246 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2013-03-19 20:16:40 +00:00
csoler
400f9f63d8 added/fixed some debug info
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@6208 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2013-03-11 20:44:23 +00:00
csoler
4f1dda2ffb added new DL mode: Progressive. This is a random mode that ensures no more than 50MB of free space will be written in the file at once. This should effectively remove the lag when initiating a DL on a big file, while keeping enough entropy to ensure a good dissemination of chunks
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@6164 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2013-02-28 20:42:01 +00:00
csoler
24f8ee6222 Added dynamic choose of sources for chunk crc requests. Fixes the bug that would let
a transfer not finish if the original surce for a crc is not here anymore.

If you have a unfinished transfer do a force-check after restart to get back chunks that 
where still on verificaiton stage.



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@5315 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2012-07-19 20:52:04 +00:00
csoler
c0e066598d switched chunk per peer and max chunk age to previous values, now that chunk cannot be slowed down by slower peers
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@5165 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2012-05-13 20:08:59 +00:00
csoler
f5c345659a Improved multi-source file transfer in two ways:
- tunnels can borrow data chunks from only slower tunnels, to prevent blocking fast tunnels
- tunnels can handle up to 50 data chunks at once instead of 5
Multi-source transfers are generaly faster and more stable.


git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@5156 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2012-05-09 21:31:05 +00:00
csoler
889a2b2433 Added deferred check of chunks during file transfer. Chunk sha1 sums are requested to the sources and checked for downloaded data.
Validated chunks are shared to other peers. Force check is now very simple since it just turns all chunks into "needs checking" mode
and sums are asked to sources. Sources maintain a temporary cache of chunks. Since sums are requested sparsely, this should not 
affect the sources in terms of performance. We can still imagine precomputing and saving sha1 of chunks while hashing them.

For backward compatibility reasons, the following has been setup *temporarily* in this version:
- unvalidated chunks are still considered as already obtained, and are shared and saved
- force check has been disabled
- final file check is maintained
- in case of file fail, the old checking mode will be used.

All changes for next version are kept in the define 'USE_NEW_CHUNK_CHECKING_CODE' that will be made the default in a few weeks.
At start, I expect most chunk to stya yellow during download, until most sources are able to provide chunk hashs.




git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@5019 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2012-03-15 19:55:43 +00:00
csoler
2edb61758b Improvements to file transfer:
- corrected 2 potential flaws of swarming causing request of unavailable data (This should normally be taken care of on the server side!)
- improved swarming test code with fault simulation
- only ask CRC32 maps to peers that have a complete file.
 


git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4947 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2012-02-16 21:10:46 +00:00
csoler
fa690de1d6 small bug fixes about CRC32 verification when the file is actually complete.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4662 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-11-04 21:29:51 +00:00
csoler
a9cb864717 corrected bug in CRC32 map cross-checking: only downloaded chunks would be checked.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4660 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-11-03 23:12:54 +00:00
csoler
dd474cfddf added chunk splitting in case part of the file transfer data does not come. Added strategy to ask again for missing data. The combination of these two cannot result into an infinite loop (as before)
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4610 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-09-22 19:58:47 +00:00
csoler
993e0d225c improved strategy of cleaning unfinished cache transfers
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4171 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-04-25 14:20:57 +00:00
thunder2
bc113326e4 Fixed problem with utf characters in the %APPDATA% path on Windows.
Added function for opening files on Windows and Linux - RsDirUtil::rs_fopen.

git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4124 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-04-03 23:11:38 +00:00
thunder2
127eac1118 Fixed utf8 issues when the partial or the download dir is set to a utf8 dir.
Use the native dialog for selecting the shared folder.

git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4083 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-03-06 20:19:16 +00:00
csoler
e39a04ee7f last fix to missing ftello64 calls (hopefully)
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4057 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-02-24 22:44:41 +00:00
thunder2
db19db7e60 Optimized some small things in TransferDialog and SearchDialog.
Changed the parameter of the constructor of ftFileCreator and ftFileProvider and some methods of ftTransferModule from "std::string" to "const std::string&"

git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4045 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-02-20 00:59:47 +00:00
csoler
9de341953d removed duplicate hashing function. Now the hash buffer is always 512 bytes.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4038 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2011-02-14 19:59:06 +00:00
csoler
40efe19efa added chunk request info, and chunk download info. Corrected bug displaying speed for stalled transfers. Warning: needs full recompilation
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3935 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-12-26 10:58:02 +00:00
csoler
f5bfc8c92a changed a bit the file transfer strategy. To be tested further.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3898 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-12-07 17:09:21 +00:00
csoler
b94e4376a4 only assume availability for cache transfers. All other transfers now use ChunkMaps
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3853 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-11-22 20:57:53 +00:00
csoler
b2b67cfecc added comment
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3839 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-11-21 13:35:10 +00:00
csoler
8369d42600 corrected a bug in chunkmaps that affected file creators being providers at the same time, which could corrupt data. Added some debug info to chunk maps.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3673 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-10-15 22:12:29 +00:00
thunder2
ed737e2c9f Fixed umlauts (utf8) in Windows. Changed some parameters to reference pointer. Added virtual folders to the lib.
Recompile needed.

git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3509 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-09-17 18:27:30 +00:00
csoler
1069ed0c58 - corrected 1 bug in serialisation of CRC32Maps for peer transfer
- corrected 1 bug (missing hashing thread deletion on failure)


git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3321 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-07-30 21:01:51 +00:00
csoler
81203b3cfd - modified ftFileProvider so as to keep multiple client peers per instance
- updated TransfersDialog to display this info correctly


git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3316 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-07-27 16:05:21 +00:00
csoler
8bfc74485a First implementation of per-chunk CRC32 check. This is triggered
- by the right-click+Force Check on files.
- when a global hash on a downloaded file does not match the announced hash.

When a CRC map check is ordered, the CRC map is requested to one of the sources for the current file download.
When received, all downloaded chunks are checked w.r.t the reference CRC and marked as not done if the CRCs do not match.
The exchange of CRC32 map and requests has been tested, as well as CRC map checking during download (force check).

To be implemented soon:
    - caching of CRC32 maps (although these are fast to compute)
    - CRC32 map packets for normal downloads. For now these work only for turtle tunnels.
    - handling of errors if the CRC never comes. For now, the download will stay stuck in "Checking..." mode.

So, don't play too much with the force check feature for now...





git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3310 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-07-21 23:14:10 +00:00
csoler
fbd8a33db0 added display of checking stage in GUI (at end of transfer). Gave names to the states in ftTransferModule. Added a flag for ForceCheck files (not functional yet)
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3274 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-07-09 21:04:29 +00:00
csoler
458a8faf70 - implemented post-download hash re-check. For now, if the hash does not match, the download is canceled, but in the near future, per-chunk comparison wil occur
.
- corrected a bug that caused file copy error: a closeFile() was missing when the file is complete. Because of delays in fwrite, the file would not be always co
mplete, nor exist at all for small files (e.g. cache files), which in the later case caused the copy error.

Warning: needs a make clean in libretroshare to recompile.



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3261 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-07-06 05:04:11 +00:00
csoler
c8fc75c75c added some safety checks to setting chunk strategy against inconsistent values.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3236 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-06-29 20:51:16 +00:00
csoler
a3a75b7aea Ported branch commit 3017:
- added failure tests for fwrite. Some where missing.
	- added a test against the IP 1.0.0.0, which pops up on MacOS (when the peer is not connected to the internet?)
	- put some additional debug messages.



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3018 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-05-28 14:00:35 +00:00
csoler
0634e461fb implemented a safety check for low disc space, and safe drop of disk access if too low
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@2968 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-05-21 20:49:48 +00:00
csoler
b000245ef5 merged branch 0.5.0 commits 2576, 2579, 2581-2583 into trunk
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@2592 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-03-21 21:07:12 +00:00
csoler
f8d6590952 ported v0.5.0 commits 2561,2562 and 2574 to trunk
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@2575 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-03-17 15:34:36 +00:00
csoler
5f7486f160 added proper dynamic suppression of sources from chunkmap, and updated the gui
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@2498 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-03-07 14:42:13 +00:00
csoler
9e469d8baf Added proper dl queue behavior
- suppressed old DwlQueue class
	- turned mDownloads into a list of pointers to allow easy cross-info update between queue and std::map of downloads
	- added queue functionality for moving files top/bottom/up/down
	- added the necessary functions in rsFiles

Put back display of exact sizes in FileTransferInfoWidget

Suppressed some warnings.

**Warning**: this commit requires a make clean.



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@2493 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-03-06 23:29:47 +00:00
csoler
3d11b15490 warning suppression, removed unused p3portservice from compilation
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@2408 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-02-23 22:17:06 +00:00
csoler
851e9069f6 corrected compled completion and file progress issues
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@2356 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-02-17 22:10:12 +00:00
joss17
bf67301d71 remove a maximum of debug outputs for the release
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@2232 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-02-07 21:28:40 +00:00
csoler
e08d13397a fixed a completion issue in FT
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@2174 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-02-02 22:15:42 +00:00
csoler
a70b106005 - added NETWORK_WIDE flag to remote dir model download
- added file priority strategy based on which files are requested first. This provides:
	- equal file speed for files with equal (source,priority)
	- effective priority speed for file of same source but different priority
- removed state variable load/save from turtle, as it's not needed anymore (FileRequest re-opens tunnels as needed)
- manage availability per peer instead of per file type: direct peer ids always assume file availability, while turtle tunnels don't

I still need to make the download queue work, and code this gui for it.



git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@2133 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-01-26 20:40:21 +00:00
csoler
e2b807aad7 added regular cleaning of inactive chunks and slices, so that they can be treated by a different peer
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@2102 b45a01b8-16f6-495d-af2f-9b41ad6348cc
2010-01-21 12:31:00 +00:00