Git-Mirrors/RetroShare
1
0
Fork 0
mirror of https://github.com/RetroShare/RetroShare.git synced 2025-05-15 04:22:27 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions

Added deferred check of chunks during file transfer. Chunk sha1 sums are requested to the sources and checked for downloaded data.

Browse source 
Validated chunks are shared to other peers. Force check is now very simple since it just turns all chunks into "needs checking" mode
and sums are asked to sources. Sources maintain a temporary cache of chunks. Since sums are requested sparsely, this should not 
affect the sources in terms of performance. We can still imagine precomputing and saving sha1 of chunks while hashing them.

For backward compatibility reasons, the following has been setup *temporarily* in this version:
- unvalidated chunks are still considered as already obtained, and are shared and saved
- force check has been disabled
- final file check is maintained
- in case of file fail, the old checking mode will be used.

All changes for next version are kept in the define 'USE_NEW_CHUNK_CHECKING_CODE' that will be made the default in a few weeks.
At start, I expect most chunk to stya yellow during download, until most sources are able to provide chunk hashs.




git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@5019 b45a01b8-16f6-495d-af2f-9b41ad6348cc
This commit is contained in:
csoler 2012-03-15 19:55:43 +00:00
parent 7ab5b54266
commit 889a2b2433
31 changed files with 1540 additions and 35 deletions
Download patch file Download diff file Expand all files Collapse all files

24
libretroshare/src/ft/ftdatamultiplex.h
View file

@ -78,8 +78,17 @@ class ftRequest
void *mData;
};
typedef std::map<std::string,time_t> ChunkCheckSumSourceList ;
class Sha1CacheEntry
{
public:
Sha1Map _map ; // Map of available sha1 sums for every chunk.
time_t last_activity ; // This is used for removing unused entries.
std::vector<uint32_t> _received ; // received chunk ids. To bedispatched.
std::map<uint32_t,ChunkCheckSumSourceList> _to_ask ; // Chunks to ask to sources.
};
class ftDataMultiplex: public ftDataRecv, public RsQueueThread
{
@ -98,6 +107,7 @@ class ftDataMultiplex: public ftDataRecv, public RsQueueThread
bool FileDetails(const std::string &hash, uint32_t hintsflag, FileInfo &info);
void deleteUnusedServers() ;
void handlePendingCrcRequests() ;
/*************** SEND INTERFACE (calls ftDataSend) *******************/
@ -117,6 +127,11 @@ class ftDataMultiplex: public ftDataRecv, public RsQueueThread
/* called from a separate thread */
bool computeAndSendCRC32Map(const std::string& peerId, const std::string& hash) ;
/* called from a separate thread */
bool sendSingleChunkCRCRequests(const std::string& hash, const std::vector<std::pair<uint32_t, std::list<std::string > > >& to_ask) ;
bool dispatchReceivedChunkCheckSum() ;
/*************** RECV INTERFACE (provides ftDataRecv) ****************/
/* Client Recv */
@ -132,6 +147,9 @@ class ftDataMultiplex: public ftDataRecv, public RsQueueThread
virtual bool recvCRC32Map(const std::string& peer_id,const std::string& hash,const CRC32Map& crc_map) ;
/// Receive a CRC map request
virtual bool recvCRC32MapRequest(const std::string& peer_id,const std::string& hash) ;
virtual bool recvSingleChunkCrcRequest(const std::string& peer_id,const std::string& hash,uint32_t chunk_id) ;
virtual bool recvSingleChunkCrc(const std::string& peer_id,const std::string& hash,uint32_t chunk_id,const Sha1CheckSum& sum) ;
// Returns the chunk map from the file uploading client. Also initiates a chunk map request if this
// map is too old. This supposes that the caller will ask again in a few seconds.
@ -153,6 +171,7 @@ class ftDataMultiplex: public ftDataRecv, public RsQueueThread
bool handleRecvClientChunkMapRequest(const std::string& peerId, const std::string& hash) ;
bool handleRecvServerChunkMapRequest(const std::string& peerId, const std::string& hash) ;
bool handleRecvCRC32MapRequest(const std::string& peerId, const std::string& hash) ;
bool handleRecvChunkCrcRequest(const std::string& peerId, const std::string& hash,uint32_t chunk_id) ;
/* We end up doing the actual server job here */
bool locked_handleServerRequest(ftFileProvider *provider, std::string peerId, std::string hash, uint64_t size, uint64_t offset, uint32_t chunksize);
@ -168,6 +187,9 @@ class ftDataMultiplex: public ftDataRecv, public RsQueueThread
std::list<CRC32Thread *> _crc32map_threads ;
std::map<std::string,std::pair<time_t,CRC32Map> > _cached_crc32maps ;
std::map<std::string,Sha1CacheEntry> _cached_sha1maps ; // one cache entry per file hash. Handled dynamically.
ftDataSend *mDataSend;
ftSearch *mSearch;
std::string mOwnId;