a transfer not finish if the original surce for a crc is not here anymore.
If you have a unfinished transfer do a force-check after restart to get back chunks that
where still on verificaiton stage.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@5315 b45a01b8-16f6-495d-af2f-9b41ad6348cc
- unchecked chunks are not made available to swarming sources, not saved as done.
- force check now uses the simple method to put all chunks in checking mode
- force checked files can be cancelled (finally!)
- improved display (use red for active chunks, yellow for checking)
- cache file are not using chunk checking (assume_availability=true)
The code still contains the #ifdef. It should be removed soon if everything works ok.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@5235 b45a01b8-16f6-495d-af2f-9b41ad6348cc
- tunnels can borrow data chunks from only slower tunnels, to prevent blocking fast tunnels
- tunnels can handle up to 50 data chunks at once instead of 5
Multi-source transfers are generaly faster and more stable.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@5156 b45a01b8-16f6-495d-af2f-9b41ad6348cc
Validated chunks are shared to other peers. Force check is now very simple since it just turns all chunks into "needs checking" mode
and sums are asked to sources. Sources maintain a temporary cache of chunks. Since sums are requested sparsely, this should not
affect the sources in terms of performance. We can still imagine precomputing and saving sha1 of chunks while hashing them.
For backward compatibility reasons, the following has been setup *temporarily* in this version:
- unvalidated chunks are still considered as already obtained, and are shared and saved
- force check has been disabled
- final file check is maintained
- in case of file fail, the old checking mode will be used.
All changes for next version are kept in the define 'USE_NEW_CHUNK_CHECKING_CODE' that will be made the default in a few weeks.
At start, I expect most chunk to stya yellow during download, until most sources are able to provide chunk hashs.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@5019 b45a01b8-16f6-495d-af2f-9b41ad6348cc
Added -Wextra to the Windows compile.
Added new define in libbitdht for snprintf and fprintf to proper handling of "%ll" under Windows.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4951 b45a01b8-16f6-495d-af2f-9b41ad6348cc
- corrected 2 potential flaws of swarming causing request of unavailable data (This should normally be taken care of on the server side!)
- improved swarming test code with fault simulation
- only ask CRC32 maps to peers that have a complete file.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4947 b45a01b8-16f6-495d-af2f-9b41ad6348cc
- FT_TM_DOWNLOAD_TIMEOUT: 10 -> 20 secs (RTT limit of 10 secs is too low -> transfers in long tunnels kept switching between active and inactive)
- INACTIVE_CHUNKS_CHECK_DELAY: 60 -> 240 secs (Chunks are kept longer before beign cancelled, which is now affordable)
- MAX_TIME_INACTIVE_REQUEUED: 60 -> 120 secs.(Transfers a kept longer before being re-prioritized in queue)
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4707 b45a01b8-16f6-495d-af2f-9b41ad6348cc
- removed dependency of mRateIncrease into measured rtt -> would cause very slow transfers through long tunnels
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4671 b45a01b8-16f6-495d-af2f-9b41ad6348cc
- servers now compute the map in a separate thread
- CRC32Maps are kept in cache for 30 mins
- CRC32Maps requests cannot be used to overflood a server anymore since their number is limited.
- Transfer modules now send keep alive packets to maintain tunnels when asking for a CRC32Map
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4661 b45a01b8-16f6-495d-af2f-9b41ad6348cc
re-organises (e.g. defragments) data such that
- when the DL is finished, the file is in the correct order (no need to re-order it)
- during the DL, only the n*_chunk_size first bytes of the partial file are written, where n is the total number of downloaded chunks.
- the total amount of copy operations does not exceed the total size of the file. In practice, it's much lower.
That suppresses the lag when downloading large files due to writing isolated chunks in the middle of the file.
Next:
- integration into ftFileCreator
- load/save in ft_transfers
- retrocompatibility with existing transfers.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4658 b45a01b8-16f6-495d-af2f-9b41ad6348cc
* Compilation for FreeBSD.
* Notes that MT is not cryptographically secure.
* modified sorting of DHT Window.
Only one change that I didn't commit:
bitdht {
- LIBS += ../../libbitdht/src/lib/libbitdht.a
- PRE_TARGETDEPS *= ../../libbitdht/src/lib/libbitdht.a
+ LIBS += ../../libbitdht/src/libbitdht.a
+ PRE_TARGETDEPS *= ../../libbitdht/src/libbitdht.a
}
As this would have broken compilation on the other platforms.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4619 b45a01b8-16f6-495d-af2f-9b41ad6348cc
- added a lock into ftTransferModule::recvFileData() (Crash reported by Costa due to storing data in a deleted transfer module)
- changed names of functions in ftTransferModules to locked_* when appropriate (helps debugging)
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4612 b45a01b8-16f6-495d-af2f-9b41ad6348cc
- set default of the signature checkbox in ConnectFriendWizard to disabled
- set the initial option for the download directory to network wide only
- set the default of a new shared directory to network wide
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4578 b45a01b8-16f6-495d-af2f-9b41ad6348cc
You can enable the debugging of the waiting time for a lock of RsMutex in rsthreads.h with
#define RSMUTEX_DEBUG 300
That means all locks waiting longer than 300ms are logged into the stderr.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4392 b45a01b8-16f6-495d-af2f-9b41ad6348cc
Major changes are:
* Improvements to tcponudp library to allow multiple UdpStacks / ports, with alternative recievers.
* Resurrected the UdpStunner code, and improved it.
* Added UdpRelay code.
* Modified startup code and ssludp code to use the new tcponudp and add a stunner.
* fixed buggy rs_inet_ntoa
* fixed a bunch of apple gcc warnings. mainly for(;;); => for(;;) ;
These changes shouldn't affect libretroshare stability... those changes will follow!
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4359 b45a01b8-16f6-495d-af2f-9b41ad6348cc
* Added Old Cache Storage... so these can be retrieved.
* Connected up the CancelCacheFile() -> FileCancel.
Other Changes.
* Removed ipaddr = 1 bug (thought someone else had commited this earlier???)
* added #ifdefs to remove debugging output in p3BitDht.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4330 b45a01b8-16f6-495d-af2f-9b41ad6348cc
- ported LinksCloud to a new plugin, and removed it from main sources
Next moves:
- add gui for managing plugins
- handle windows compilation
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4275 b45a01b8-16f6-495d-af2f-9b41ad6348cc
- corrected bug that would wipe out the cache file every other start
- missing root directories are not discarded anymore, but kept empty, unless the user really removes them
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4271 b45a01b8-16f6-495d-af2f-9b41ad6348cc
Removed extra "/" of the filename (e.g. "E://") of the hashed file when sharing a whole drive.
Recompile of the gui needed.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4200 b45a01b8-16f6-495d-af2f-9b41ad6348cc
- changed the order of check in tick() to allow cancelling a file that is being checked
- removed the hash step from mutex protection
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4113 b45a01b8-16f6-495d-af2f-9b41ad6348cc
Combined the two methods for share and unshare of the download dir.
Added share incoming directory to the QuickStartWizard.
Recompile of the GUI needed.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4089 b45a01b8-16f6-495d-af2f-9b41ad6348cc
Changed the parameter of the constructor of ftFileCreator and ftFileProvider and some methods of ftTransferModule from "std::string" to "const std::string&"
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@4045 b45a01b8-16f6-495d-af2f-9b41ad6348cc
Optimized parameter of rsForums from "std::string" to "const std::string&". Recompile of the GUI needed.
Cleared childTS in RsDistribMsg::clear.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3949 b45a01b8-16f6-495d-af2f-9b41ad6348cc
Added new icon in MainWindow and systray icon for existing completed downloads.
You can enable/disable the toaster and the systray icon in NotifyPage.
Recompile of the GUI needed.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3868 b45a01b8-16f6-495d-af2f-9b41ad6348cc
* new uniform #define DEBUG_TRANSFERS to switch on packet timings.
* switched off by default.
* enabled BinData printing again.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3785 b45a01b8-16f6-495d-af2f-9b41ad6348cc
Committing these so others can run tests.
Most of these changes will be removed once the problem is fixed!
* Fixed formatting of various serialiser item::print() fns.
* Disabled printout of TlvBinaryData - data.
* added AccurateTimeString() function to util/rsprint.cc
* print serialise / deserialise of FileRequest & FileData + Timestamps.
* added print + timestamp in ftFileProvider::getFileData()
* #define TRANSFER_DEBUG to isolate these changes.
* enabled debugging in fttransfermodule.cc
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3779 b45a01b8-16f6-495d-af2f-9b41ad6348cc
msgs now save from field in received msgs
also allowed user to delete file added in createChanMsg dialog
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3727 b45a01b8-16f6-495d-af2f-9b41ad6348cc
- Sort the ChanMsgItems of the channel by date
ChanMsgItem:
- Fixed adding all files (SubFileItem) again by pressing the unsubscribe button (only in Friend Storm)
- Enable unsubscribe button only when channel is subscribed (only in Friend Storm)
- Fixed adding files with umlauts (utf8) on Windows
- Enabled Download and Play button in ChanMsgItem, when they are enabled in the SubFileItems
ftServer:
- Added the method alreadyHaveFile for use in the GUI - recompile of GUI needed
SubFileItem:
- Added state strings like LOCAL, REMOTE, ERROR, ... for translating
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3614 b45a01b8-16f6-495d-af2f-9b41ad6348cc
libretroshare/src/retroshare/
All the relevant headers have been modified to reflect that change.
This allows installation of libretroshare on a system, headers will
be put in $WHEREVER/retroshare/ and we keep the ability to compile
against them, be it on the system or in the SVN tree.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3342 b45a01b8-16f6-495d-af2f-9b41ad6348cc
- replaced the findItems() call into a hand-written search (this was causing a rare bug in displaying downloads)
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3318 b45a01b8-16f6-495d-af2f-9b41ad6348cc
- by the right-click+Force Check on files.
- when a global hash on a downloaded file does not match the announced hash.
When a CRC map check is ordered, the CRC map is requested to one of the sources for the current file download.
When received, all downloaded chunks are checked w.r.t the reference CRC and marked as not done if the CRCs do not match.
The exchange of CRC32 map and requests has been tested, as well as CRC map checking during download (force check).
To be implemented soon:
- caching of CRC32 maps (although these are fast to compute)
- CRC32 map packets for normal downloads. For now these work only for turtle tunnels.
- handling of errors if the CRC never comes. For now, the download will stay stuck in "Checking..." mode.
So, don't play too much with the force check feature for now...
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3310 b45a01b8-16f6-495d-af2f-9b41ad6348cc
* Added thread-safe rs_inet_ntoa function (util/rsnet.cc)
* Switched all calls to this function (most of the changes)
* Modified getLocalInterfaces() and getPreferredInterface() to avoid string conversions.
- NB: Modified windows functions too, but unable to test (hope it compiles!)
Also:
* Added EVP_CIPHER_CTX_rand_key() replacement function for old versions of SSL (pre 0.9.8) (for OSX 10.5)
- NB: This code should be reworked to remove these calls anyway.
* Updated tests/pqi to handle above changes.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3281 b45a01b8-16f6-495d-af2f-9b41ad6348cc
.
- corrected a bug that caused file copy error: a closeFile() was missing when the file is complete. Because of delays in fwrite, the file would not be always co
mplete, nor exist at all for small files (e.g. cache files), which in the later case caused the copy error.
Warning: needs a make clean in libretroshare to recompile.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3261 b45a01b8-16f6-495d-af2f-9b41ad6348cc
* Added existing ft tests to tests directory (don't work yet!)
* Added dbase tests to tests directory (don't work yet!)
* moved net_setup test to tests/general directory (don't work yet!)
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3176 b45a01b8-16f6-495d-af2f-9b41ad6348cc
Corrects wrong logic when using Win32 API for handling the filesystem, Win32 functions return 0 for an error while POSIX functions return 0 for a success.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3146 b45a01b8-16f6-495d-af2f-9b41ad6348cc
3099: libretroshare/src/tcponudp/udplayer.cc
libretroshare/src/tcponudp/udplayer.h
libretroshare/src/util/rsnet.cc
libretroshare/src/util/rsnet.h
moved operator<<(ostream&,sockaddr_in) to rsnet.h, as it is used at
different places
3100: libretroshare/src/util/rsdiscspace.cc
suppressed debug mode
3101: libretroshare/src/serialiser/rsdiscitems.cc
put correct (i.e. non conservative) size for RsDiscReply packets
3102: libretroshare/src/services/p3disc.cc
Made a pass on the code in p3disc. Corrected some bugs:
- suppressed an unwanted return in packet treatment
- prevented sending info to a peer about itself
- changed askInfoToAllPeers() such as not to discard info
from peers with NODISC flag (because we especially need
info for these)
- enabled receiving p3disc info even if p3discovery is
disabled. Indeed, disabeling p3disc is a measure of
protection, so it should limit the export of p3disc info,
not the import.
- removed test discarding info about dummy friends, because
it is useless
- don't discard info about peers that have the NODISC flag
(meaning that we especially need info for them)
- added safety check about received GPG keys. Before we
relied on this test being performed by
p3ConnMgr::addFriend()
- added some debug info
3103: libretroshare/src/ft/ftcontroller.cc
removed debug info
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3104 b45a01b8-16f6-495d-af2f-9b41ad6348cc
- memory leaks in pqi/cleanupxpgp.cc , 2 new char[] => 2 delete[]
- reordering some constructors' initializing order
- char* myString = "foo" constructs must be prefixed by "const"
- using size_t for variable types when dealing with STL
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3079 b45a01b8-16f6-495d-af2f-9b41ad6348cc
3068: added verification for file size computation (bug correction)
3069: set heartbeat values to intermediate values
3070: suppressed uninitialised memory read
3071: automatic removal of file lists from deleted peers, at restart (After
double check that this does not alter exchange of file lists in any way)
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3072 b45a01b8-16f6-495d-af2f-9b41ad6348cc
This avoids that an attacker that captures a broadcast hash request can also ask for the file.
This needed to add back explicit sources in SearchDialog and private chat transfer. I dynamically search
for sources in file lists just before calling FileRequest.
When RS links anchorClick() will work properly with Qt, we can remove the search flag "EXTRA" from the
tunnel digging in p3turtle.cc, otherwise, people having this bug can't transfer files in private chat by
copy+paste of a RS link.
Also:
- added missing mutexes in search requests into fimonitor.cc
- suppressed some debug info in connect manager
- added check for write fails when dumping config files
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3027 b45a01b8-16f6-495d-af2f-9b41ad6348cc
- added failure tests for fwrite. Some where missing.
- added a test against the IP 1.0.0.0, which pops up on MacOS (when the peer is not connected to the internet?)
- put some additional debug messages.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@3018 b45a01b8-16f6-495d-af2f-9b41ad6348cc
The problem was that libretroshare handles files in UTF-8 but Windows's ANSI/POSIX C functions automatically assume that the char* parameters are encoded with the system's code page. There is no way to set that code page as UTF-8. So now under Windows the code translates the file name to UTF-16 before feeding it to one of the Unicode functions (they are usually prefixed or suffixed by 'w'). Please note that it is not very efficient.
Furthermore, Windows does not provide a Unicode version of opendir/readdir/closedir, so it was necessary to use FindFirstFileW/FindNextFileW/FindClose which has a different behaviour as well as different structures. The FolderIterator class was created in order to mimic the Unix way of traversing folders contents. Hence the algorithm is unchanged and the systems differences masked.
As it was necessary to use some functions from the Windows API, <windows.h> had to be included in a few files were it didn't appear before, creating macros and #define conflicts. In order to solve them, util/rswin.h must be included first in a file (if necessary). Otherwise the preprocessor will fail on purpose to avoid the code being compiled with different _WIN32_WINNT values.
As another side-effect, rstlvutil.h and rstlvutil.cc have been removed from libretroshare.pro file. They are only used by testing units and include util/utest.h which defines a macro FAILED that already exists in <windows.h>. I don't know if unit tests are still in use and I don't plan on coding often on Windows, so I'll leave that as an exercise (hot potato?) to a motivated fellow Windows programmer.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@2924 b45a01b8-16f6-495d-af2f-9b41ad6348cc
- ftController::copyFile -> buffer and file pointer leaks in case of error
- PeersDialog::insertSendList -> new QTreeWidgetItem for all online peers, this function is not ready yet
- PeersDialog::insertPeers -> gpg_item = new QTreeWidgetItem(0); in case of no more accept connection
- pqissl::reset -> missing SSL_free of ssl_connection
Missing return:
- IntroPage::nextId
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@2891 b45a01b8-16f6-495d-af2f-9b41ad6348cc
- suppressed old DwlQueue class
- turned mDownloads into a list of pointers to allow easy cross-info update between queue and std::map of downloads
- added queue functionality for moving files top/bottom/up/down
- added the necessary functions in rsFiles
Put back display of exact sizes in FileTransferInfoWidget
Suppressed some warnings.
**Warning**: this commit requires a make clean.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@2493 b45a01b8-16f6-495d-af2f-9b41ad6348cc
- added file priority strategy based on which files are requested first. This provides:
- equal file speed for files with equal (source,priority)
- effective priority speed for file of same source but different priority
- removed state variable load/save from turtle, as it's not needed anymore (FileRequest re-opens tunnels as needed)
- manage availability per peer instead of per file type: direct peer ids always assume file availability, while turtle tunnels don't
I still need to make the download queue work, and code this gui for it.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@2133 b45a01b8-16f6-495d-af2f-9b41ad6348cc
- exchange of chunk availability maps from different peers
- correct handling of what is available to which source before asking the data
- correct display of chunks in the progress bars
- generalised the use of compressed chunk maps
- removed the size parameters from the hash search functions
- In addition:
- suppressed a number of per-value transfers of std::string
- improved the FileTransferInfo Widget, to show some additional info
Still to be done:
- chunk map exchange for non anonymous traffic (easy)
- improve accuracy of completion for uploads (for now it's a integer number of chunks)
- check compilation on windows
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@1993 b45a01b8-16f6-495d-af2f-9b41ad6348cc
- suppressed a cross mutex lock bug that happened rarely while digging tunnels
- changed FileDetails in ftServer so that it's now possiblt to search for hashes of files being downloaded
- improved the search code in ftdatamultiplex
- added some comments to the turtle code
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@1964 b45a01b8-16f6-495d-af2f-9b41ad6348cc
- implemented transfer protocol for chunk availability maps between peers (not enabled yet though)
- suppressed rsiface directory from retroshare-gui
- moved notifyqt.{h,cpp} to gui/
next moves:
- send availability maps to clients;
- connect turtle search to unfinished files;
- test multisource download with unfinished files.
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@1939 b45a01b8-16f6-495d-af2f-9b41ad6348cc
The side effect is that even when stopped, downloads show the correct downloaded size (up to the size of one chunk).
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@1865 b45a01b8-16f6-495d-af2f-9b41ad6348cc
- a given chunk strategy
- the availablility map of each source
- Integrated this into ftFileCreator
- added gui menu in file transfer+right click to change the chunk strategy: streaming vs. random
Next step:
- loading/saving file downloading state and availability map
- displaying chunk details in the selected transfer tab (e.g. list of currently worked chunks, and their current downloading completion)
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@1863 b45a01b8-16f6-495d-af2f-9b41ad6348cc
en visible, and each time it gets showed. I already made some RS pages derive from this page.
- Modified and renamed the taskGraphPainterWidget to display info about currently selected file transfers. For know, it only shows file chunks info, but we should use it to display additional info about the transfer. Warning: in its
current state, only fake chunk info is displayed.
- Suppressed p3files.h, because it is not used.
Remains to do (in order):
- Implement a chunk map class in ft/ to properly manage which chunks should be downloaded
- Display correct chunk info by requesting the transfer's chunk map
- Add check-sum verification of chunks
- allow RS to import a partially downloaded file by force re-checking chunks
- Add chunk map exchange from file source peer id to file destination peer id.
- Manage chunks from different peers.
- Allow search (turtle search) for partially downloaded files
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@1832 b45a01b8-16f6-495d-af2f-9b41ad6348cc
-downloading directories from search dialog not implemented yet
-needs more code clean and beautify the directory tree
git-svn-id: http://svn.code.sf.net/p/retroshare/code/trunk@1566 b45a01b8-16f6-495d-af2f-9b41ad6348cc
