add two RSA based ciphers for Windows/depends compatibility
also enforce server cipher ordering
also set ECDH to auto because vtnerd says it is good :)
When built with the depends system, openssl does not include any
cipher on the current whitelist, so add this one, which fixes the
problem, and does seem sensible.
It can allocate a lot when getting a lot of connections
(in particular, the stress test on windows apparently pushes
that memory to actual use, rather than just allocated)
When closing connections due to exiting, the IO service is
already gone, so the data exchange needed for a gracious SSL
shutdown cannot happen. We just close the socket in that case.
An override for the wallet to daemon connection is provided, but not for
other SSL contexts. The intent is to prevent users from supplying a
system CA as the "user" whitelisted certificate, which is less secure
since the key is controlled by a third party.
This allows "chain" certificates to be used with the fingerprint
whitelist option. A user can get a system-ca signature as backup while
clients explicitly whitelist the server certificate. The user specified
CA can also be combined with fingerprint whitelisting.
The former has the same behavior with single self signed certificates
while allowing the server to have separate short-term authentication
keys with long-term authorization keys.
If the verification mode is `system_ca`, clients will now do hostname
verification. Thus, only certificates from expected hostnames are
allowed when SSL is enabled. This can be overridden by forcible setting
the SSL mode to autodetect.
Clients will also send the hostname even when `system_ca` is not being
performed. This leaks possible metadata, but allows servers providing
multiple hostnames to respond with the correct certificate. One example
is cloudflare, which getmonero.org is currently using.
If SSL is "enabled" via command line without specifying a fingerprint or
certificate, the system CA list is checked for server verification and
_now_ fails the handshake if that check fails. This change was made to
remain consistent with standard SSL/TLS client behavior. This can still
be overridden by using the allow any certificate flag.
If the SSL behavior is autodetect, the system CA list is still checked
but a warning is logged if this fails. The stream is not rejected
because a re-connect will be attempted - its better to have an
unverified encrypted stream than an unverified + unencrypted stream.
Using `verify_peer` on server side requests a certificate from the
client. If no certificate is provided, the server silently accepts the
connection and rejects if the client sends an unexpected certificate.
Adding `verify_fail_if_no_cert` has no affect on client and for server
requires that the peer sends a certificate or fails the handshake. This
is the desired behavior when the user specifies a fingerprint or CA file.
Currently a client must provide a certificate, even if the server is
configured to allow all certificates. This drops that requirement from
the client - unless the server is configured to use a CA file or
fingerprint(s) for verification - which is the standard behavior for SSL
servers.
The "system-wide" CA is not being used as a "fallback" to verify clients
before or after this patch.
Specifying SSL certificates for peer verification does an exact match,
making it a not-so-obvious alias for the fingerprints option. This
changes the checks to OpenSSL which loads concatenated certificate(s)
from a single file and does a certificate-authority (chain of trust)
check instead. There is no drop in security - a compromised exact match
fingerprint has the same worse case failure. There is increased security
in allowing separate long-term CA key and short-term SSL server keys.
This also removes loading of the system-default CA files if a custom
CA file or certificate fingerprint is specified.
Manually initialize the array_entry_t iterator to ensure it points
to the correct m_array, thereby preventing a potential use-after-free
situation.
Signed-off-by: Guido Vranken <guidovranken@gmail.com>
This avoids the annoying case where the shell prints its prompt
after the last line from Monero output, causing line editing to
sometimes go wonky, for lack of a better term
RPC connections now have optional tranparent SSL.
An optional private key and certificate file can be passed,
using the --{rpc,daemon}-ssl-private-key and
--{rpc,daemon}-ssl-certificate options. Those have as
argument a path to a PEM format private private key and
certificate, respectively.
If not given, a temporary self signed certificate will be used.
SSL can be enabled or disabled using --{rpc}-ssl, which
accepts autodetect (default), disabled or enabled.
Access can be restricted to particular certificates using the
--rpc-ssl-allowed-certificates, which takes a list of
paths to PEM encoded certificates. This can allow a wallet to
connect to only the daemon they think they're connected to,
by forcing SSL and listing the paths to the known good
certificates.
To generate long term certificates:
openssl genrsa -out /tmp/KEY 4096
openssl req -new -key /tmp/KEY -out /tmp/REQ
openssl x509 -req -days 999999 -sha256 -in /tmp/REQ -signkey /tmp/KEY -out /tmp/CERT
/tmp/KEY is the private key, and /tmp/CERT is the certificate,
both in PEM format. /tmp/REQ can be removed. Adjust the last
command to set expiration date, etc, as needed. It doesn't
make a whole lot of sense for monero anyway, since most servers
will run with one time temporary self signed certificates anyway.
SSL support is transparent, so all communication is done on the
existing ports, with SSL autodetection. This means you can start
using an SSL daemon now, but you should not enforce SSL yet or
nothing will talk to you.
Further speedups to icu compilation, it is faster to run the
pre-generated configure scripts.
Ensure that the native protobuf installation only generates the required
libraries and binaries.
Disable qt compilation when running travis on windows. Qt is used for
lrelease, the travis recipe instead usese the a local installation of
lrelease.
Remove various packages and options from the travis recipe.
Update Readline to version 8.0. The previously used url 404'd sometimes,
use the official gnu ftp server instead.
Remove unused cmake config.
Building with docker is arguably easier and more familiar to most people
than either kvm, or lxc.
This commit also relaxes the back compat requirement a bit. 32 bit linux
now uses glibc version 2.0. Also, the docker shell could not handle gcc arguments
containing spaces, so the explicit '-DFELT_TYPE' declaration was dropped.
Lastly, this removes some packages from the osx descriptor.
RPC connections now have optional tranparent SSL.
An optional private key and certificate file can be passed,
using the --{rpc,daemon}-ssl-private-key and
--{rpc,daemon}-ssl-certificate options. Those have as
argument a path to a PEM format private private key and
certificate, respectively.
If not given, a temporary self signed certificate will be used.
SSL can be enabled or disabled using --{rpc}-ssl, which
accepts autodetect (default), disabled or enabled.
Access can be restricted to particular certificates using the
--rpc-ssl-allowed-certificates, which takes a list of
paths to PEM encoded certificates. This can allow a wallet to
connect to only the daemon they think they're connected to,
by forcing SSL and listing the paths to the known good
certificates.
To generate long term certificates:
openssl genrsa -out /tmp/KEY 4096
openssl req -new -key /tmp/KEY -out /tmp/REQ
openssl x509 -req -days 999999 -sha256 -in /tmp/REQ -signkey /tmp/KEY -out /tmp/CERT
/tmp/KEY is the private key, and /tmp/CERT is the certificate,
both in PEM format. /tmp/REQ can be removed. Adjust the last
command to set expiration date, etc, as needed. It doesn't
make a whole lot of sense for monero anyway, since most servers
will run with one time temporary self signed certificates anyway.
SSL support is transparent, so all communication is done on the
existing ports, with SSL autodetection. This means you can start
using an SSL daemon now, but you should not enforce SSL yet or
nothing will talk to you.
- Support for ".onion" in --add-exclusive-node and --add-peer
- Add --anonymizing-proxy for outbound Tor connections
- Add --anonymous-inbounds for inbound Tor connections
- Support for sharing ".onion" addresses over Tor connections
- Support for broadcasting transactions received over RPC exclusively
over Tor (else broadcast over public IP when Tor not enabled).
The blockchain prunes seven eighths of prunable tx data.
This saves about two thirds of the blockchain size, while
keeping the node useful as a sync source for an eighth
of the blockchain.
No other data is currently pruned.
There are three ways to prune a blockchain:
- run monerod with --prune-blockchain
- run "prune_blockchain" in the monerod console
- run the monero-blockchain-prune utility
The first two will prune in place. Due to how LMDB works, this
will not reduce the blockchain size on disk. Instead, it will
mark parts of the file as free, so that future data will use
that free space, causing the file to not grow until free space
grows scarce.
The third way will create a second database, a pruned copy of
the original one. Since this is a new file, this one will be
smaller than the original one.
Once the database is pruned, it will stay pruned as it syncs.
That is, there is no need to use --prune-blockchain again, etc.
This includes more fine grained configure options and skipping the
openssl and zlib dependencies when compiling qt. The zlib and libevent
packages are removed.
Number matching semantics are slightly changed: since this is used
as a filter to check whether a number is signed and/or floating
point, we can speed this up further. strto* functions are called
afterwards and will error out where necessary. We now also accept
numbers like .4 which were not accepted before.
The strto* calls on a boost::string_ref will not access unallocated
memory since the parsers always stop at the first bad character,
and the original string is zero terminated.
in arbitrary time measurement units for some arbitrary test case:
match_number2: 235 -> 70
match_word2: 330 -> 108
a13eb0a1 epee: speed up string matching a bit (moneromooo-monero)
3a3858dc epee: avoid string allocation when parsing a pod from string (moneromooo-monero)
- docker protobuf dependencies, cross-compilation
- device/trezor protobuf build fixes, try_compile
- libusb built under all platforms, used by trezor for direct connect
Depends still contained some leftovers, like the `wallet` target that
included bdb from bitcoin. This commit removes these unneeded targets,
the miniupnpc package and the berkeley db package. Reflect the changes
in the README as well.
ab783b17 easylogging++: ensure logger is initialized before main (moneromooo-monero)
9b69a0ae daemon: print monero version at startup when calling a detached daemon (moneromooo-monero)
4d71d463 mlocker: remove early page size log (moneromooo-monero)
b36353e2 unit_tests: add some hex parsing test for non hex input (xiphon)
6671110c unit_tests: add a test for parse_hexstr_to_binbuff (moneromooo-monero)
f6187cd8 epee: speed up parse_hexstr_to_binbuff a little (Howard Chu)
It comes before the logger is initialized, so gets displayed
even though it should not be by default, and apparenly comes
too early for (some versions of) Android, where it crashes.
Turns out getting the global shared_ptr hits the profile,
and passing it around still keeps it at close to ~1% CPU,
which is too much for mostly silent logging.
Leak the object instead, which is even safer for late logging.
- webusb transport based on libusb added. Provides direct access to Trezor via USB, no need for Trezor bridge.
- trezor protocol message handler improved, no recursion used. Ready for upcoming integration tests.
- libusb (for docker) bumped from v1.0.9 to v1.0.22, newer version required for webusb transport, for device enumeration.
- cmake improvements and fixes. Cmake Trezor checks are moved to a dedicated CheckTrezor.cmake file. In case of a problem Trezor is excluded from build.
- ifdefs made consistent to Ledger.
- UDP Transport enumeration disabled by default in release mode
This prevents exceptions from showing up in various awkward
places such as dtors, since the only exception that can be
thrown is a lock failure, and nothing handles a lock failure
anyway.
The version prefix 'v' should just be set constantly.
Reflect this change in the README as well.
This should allow building commits as well, if a commit
is passed in instead of a tag.
00901e9c epee: initialize a few data members where it seems to be appropriate (moneromooo-monero)
144a6c32 abstract_tcp_server2: move m_period to subclass (moneromooo-monero)
758d7684 connection_basic: remove unused floating time start time (moneromooo-monero)
e5108a29 Catch more exceptions in dtors (moneromooo-monero)
Windows is built with a seperate descriptor to handle additional changes
that need to be done to the end binary. Consolidate the gitian-build
script for this change.
The signature prepare tool and the gitian-builder git repo should be
checked for their content. For this purpose, checkout the gitian-builder
repo at a specific commit and take the sha256sum of the osslsigncode
tool.
This adds a build script to run gitian builds for linux.
The build script was copied from bitcoin and then adapted for monero.
Build step documentation is outlined in the README in the contrib/gitian
directory.
2fbf38ee Fix 32bit depends builds (TheCharlatan)
17142ec9 malloc scratchpad for all supported android archs (m2049r)
6a781408 Make depends use self built clang for darwin (TheCharlatan)
69da14e1 fixes make debug compilation on OSX (Dusan Klinec)
fe125647 Fixup RENAME_DB() macro (Howard Chu)
b2972927 osx compilation fix: missing boost libs added (Dusan Klinec)
174f31bf simplewallet: don't complain about payment id on pool mined blocks (moneromooo-monero)
89288863 README: mention ASAN usage alongside valgrind (moneromooo-monero)
83debef9 wallet_rpc_server: remove verbose field in incoming_transfers query (moneromooo-monero)
a69271fa Fixed a typo (Piotr Kąkol)
92d1da28 unit_tests: fix build with GCC 5.4.0 on ubuntu (moneromooo-monero)
a21da905 Wallet: use unique_ptr for WalletImpl members (oneiric)
7a056f44 WalletAPI: multisigSignData bug fixed (naughtyfox)
43a06350 ringdb: use cursors to be a bit faster (moneromooo-monero)
7964d4f8 wallet2: handle corner case in picking fake outputs (moneromooo-monero)
6f5360b3 bump version to 0.13.0.1 (Riccardo Spagni)
cf470bf3 switch from master to rc (Riccardo Spagni)
bcf3f6af fuzz_tests: catch unhandled exceptions (moneromooo-monero)
3ebd05d4 miner: restore stream flags after changing them (moneromooo-monero)
a093092e levin_protocol_handler_async: do not propagate exception through dtor (moneromooo-monero)
1eebb82b net_helper: do not propagate exceptions through dtor (moneromooo-monero)
fb6a3630 miner: do not propagate exceptions through dtor (moneromooo-monero)
2e2139ff epee: do not propagate exception through dtor (moneromooo-monero)
0749a8bd db_lmdb: do not propagate exceptions in dtor (moneromooo-monero)
1b0afeeb wallet_rpc_server: exit cleanly on unhandled exceptions (moneromooo-monero)
418a9936 unit_tests: catch unhandled exceptions (moneromooo-monero)
ea7f9543 threadpool: do not propagate exceptions through the dtor (moneromooo-monero)
6e855422 gen_multisig: nice exit on unhandled exception (moneromooo-monero)
53df2deb db_lmdb: catch error in mdb_stat calls during migration (moneromooo-monero)
e67016dd blockchain_blackball: catch failure to commit db transaction (moneromooo-monero)
661439f4 mlog: don't remove old logs if we failed to rename the current file (moneromooo-monero)
5fdcda50 easylogging++: test for NULL before dereference (moneromooo-monero)
7ece1550 performance_test: fix bad last argument calling add_arg (moneromooo-monero)
a085da32 unit_tests: add check for page size > 0 before dividing (moneromooo-monero)
d8b1ec8b unit_tests: use std::shared_ptr to shut coverity up about leaks (moneromooo-monero)
02563bf4 simplewallet: top level exception catcher to print nicer messages (moneromooo-monero)
c57a65b2 blockchain_blackball: fix shift range for 32 bit archs (moneromooo-monero)
The configure script in hidapi and libsodium tried to find clang in /usr/bin,
even though the correct prefix was passed in. This sets the correct CC flag.
This was previously undetected, because clang and the sdk where
installed in the global environment.
This also fixes a subsequent error, where IOKIT and CoreFoundation are
not found, again for the reason stated above.
Clang needs to get its cctools path passed directly for the hid build to
succeed.
Make gperf a permanent external dependency.
Remove pcsc from depends.
26a42fe5 Added features to epee::span<T> : - Support for classes - Added `remove_prefix` function - Added `to_mut_span` and `as_mut_byte_span` (Lee Clagett)
56b50faa wallet: use wipeable_string in more places where a secret is used (moneromooo-monero)
07ec748c wipeable_string: add hex_to_pod function (moneromooo-monero)
42397359 Fixup 32bit arm build (TheCharlatan)
a06d2581 Fix Windows build (TheCharlatan)
ecaf5b3f Add libsodium to the packages, the arm build was complaining about it. (TheCharlatan)
cbbf4d24 Adapt translations to upstream changes (TheCharlatan)
db571546 Updated pcsc url (TheCharlatan)
f0ba19fd Add lrelease to the depends (TheCharlatan)
cfb30462 Add Miniupnp submodule (TheCharlatan)
5f7da005 Unbound is now a submodule. Adapt depends for this. (TheCharlatan)
d6b9bdd3 Update readmes to reflect the usage of depends (TheCharlatan)
56b6e41e Add support for apple and arm building (TheCharlatan)
29311fd1 Disable stack unwinding for mingw32 depends build. (TheCharlatan)
8db3d573 Modify depends for monero's dependencies (TheCharlatan)
0806a23a Initial depends addition (TheCharlatan)
4469b0c4 abstract_tcp_server2: fix binding to the wrong IP (moneromooo-monero)
8eab6147 epee: use the socket::bind variant which does not throw (moneromooo-monero)
Drop miniupnp and unbound depends builds. Make sure that build variables are propageted properly to unbound and miniupnp.
Rebase to after the v0.12 release
Add pcsc-lite to linux builds
Fixup windows icu4c linking with depends, the static libraries have an 's' appended to them
Compiling depends arm-linux-gnueabihf will allow you to compile armv6zk monero binaries
Add readline, ldns, graphviz, unbound to depends packages
Add a cmake toolchain file to depends that is uniquely created for every build and placed in triple/share/toolchain.cmake
This file is then passed to cmake with -DCMAKE_TOOLCHAIN_FILE=/path/to/triple/share/toolchain.cmake
Add the boost locale package to depends
In the depends cmake toolchain file, a DEPENDS flag is added
to exclude, or change cmake checks done that are required for depends
Link miniupnpc and unwind from depends and not external
Add libiconv and icu4c to depends, required for mingw32 builds.
Headers (winsock) need to be lower case in order to compile on unix systems.
This should not affect building on windows.
Depends cross compiles project dependencies for linux, mac and windows and multiple architectures.
Depends is original work by Cory Fields and used in bitcoin and a wide range of bitcoin related projects.
29dea03 epee: resize vectors where possible in serialization (moneromooo-monero)
76affd9 epee: some speedup in parsing (moneromooo-monero)
dc6c069 db_lmdb: speedup the get_output_distribution common case (moneromooo-monero)
76ac5a8 wallet2: ask for a binary output distribution, for speed (moneromooo-monero)
This class will allow mlocking small objects, of which there
may be several per page. It adds refcounting so pages are only
munlocked when the last object on that page munlocks.
The secret spend key is kept encrypted in memory, and
decrypted on the fly when needed.
Both spend and view secret keys are kept encrypted in a JSON
field in the keys file. This avoids leaving the keys in
memory due to being manipulated by the JSON I/O API.
f8dd433 epee: fix detection of 172.16.0.0/172.31.255.255 local IP range (moneromooo-monero)
5db9e3c unit_tests: add tests for local IP range detection (moneromooo-monero)
a connection's timeout is halved for every extra connection
from the same host.
Also keep track of when we don't need to use a connection
anymore, so we can close it and free the resource for another
connection.
Also use the longer timeout for non routable local addresses.
In file included from src/cryptonote_basic/hardfork.cpp:33:
In file included from src/blockchain_db/blockchain_db.h:42:
In file included from src/cryptonote_basic/hardfork.h:31:
contrib/epee/include/syncobj.h:37:10: fatal error: 'boost/thread/v2/thread.hpp' file not found
#include <boost/thread/v2/thread.hpp>
^~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from src/rpc/daemon_handler.cpp:29:
In file included from src/rpc/daemon_handler.h:36:
In file included from src/p2p/net_node.h:41:
In file included from contrib/epee/include/net/levin_server_cp2.h:32:
In file included from contrib/epee/include/net/abstract_tcp_server2.h:324:
contrib/epee/include/net/abstract_tcp_server2.inl:44:10: fatal error: 'boost/thread/v2/thread.hpp' file not found
#include <boost/thread/v2/thread.hpp> // TODO
^~~~~~~~~~~~~~~~~~~~~~~~~~~~
contrib/epee/include/math_helper.h: In member function 'bool epee::math_helper::average<val, default_base>::set_base()':
contrib/epee/include/syncobj.h:227:56: error: 'sleep_for' is not a member of 'boost::this_thread'
#define CRITICAL_REGION_LOCAL(x) {boost::this_thread::sleep_for(boost::chrono::milliseconds(epee::debug::g_test_dbg_lock_sleep()));} epee::critical_region_t<decltype(x)> critical_region_var(x)
^
contrib/epee/include/syncobj.h:227:56: note: in definition of macro 'CRITICAL_REGION_LOCAL'
#define CRITICAL_REGION_LOCAL(x) {boost::this_thread::sleep_for(boost::chrono::milliseconds(epee::debug::g_test_dbg_lock_sleep()));} epee::critical_region_t<decltype(x)> critical_region_var(x)
^~~~~~~~~
contrib/epee/include/syncobj.h:227:56: note: suggested alternative: 'sleep'
#define CRITICAL_REGION_LOCAL(x) {boost::this_thread::sleep_for(boost::chrono::milliseconds(epee::debug::g_test_dbg_lock_sleep()));} epee::critical_region_t<decltype(x)> critical_region_var(x)
^
contrib/epee/include/syncobj.h:227:56: note: in definition of macro 'CRITICAL_REGION_LOCAL'
#define CRITICAL_REGION_LOCAL(x) {boost::this_thread::sleep_for(boost::chrono::milliseconds(epee::debug::g_test_dbg_lock_sleep()));} epee::critical_region_t<decltype(x)> critical_region_var(x)
^~~~~~~~~
e4646379 keccak: fix mdlen bounds sanity checking (moneromooo-monero)
2e3e90ac pass large parameters by const ref, not value (moneromooo-monero)
61defd89 blockchain: sanity check number of precomputed hash of hash blocks (moneromooo-monero)
9af6b2d1 ringct: fix infinite loop in unused h2b function (moneromooo-monero)
8cea8d0c simplewallet: double check a new multisig wallet is multisig (moneromooo-monero)
9b98a6ac threadpool: catch exceptions in dtor, to avoid terminate (moneromooo-monero)
24803ed9 blockchain_export: fix buffer overflow in exporter (moneromooo-monero)
f3f7da62 perf_timer: rewrite to make it clear there is no division by zero (moneromooo-monero)
c6ea3df0 performance_tests: remove add_arg call stray extra param (moneromooo-monero)
fa6b4566 fuzz_tests: fix an uninitialized var in setup (moneromooo-monero)
03887f11 keccak: fix sanity check bounds test (moneromooo-monero)
ad11db91 blockchain_db: initialize m_open in base class ctor (moneromooo-monero)
bece67f9 miner: restore std::cout precision after modification (moneromooo-monero)
1aabd14c db_lmdb: check hard fork info drop succeeded (moneromooo-monero)
cb9aa23c levin_protocol_handler_async: another attempt at fixing at exception (moneromooo-monero)
64d23ce3 Revert "epee: keep a ref to a connection we're deleting" (moneromooo-monero)
bd5cce07 network_throttle: fix ineffective locking (moneromooo-monero)
e0a61299 network_throttle: remove unused xxx static member (moneromooo-monero)
24f584d9 cryptonote_core: remove unused functions with off by one bugs (moneromooo-monero)
b1634aa3 blockchain: don't leave dangling pointers in this (moneromooo-monero)
8e60b81c cryptonote_core: fix db leak on error (moneromooo-monero)
213e326c abstract_tcp_server2: log init_server errors as fatal (moneromooo-monero)
b51dc566 use const refs in for loops for non tiny types (moneromooo-monero)
f0568ca6 net_parse_helpers: fix regex error checking (moneromooo-monero)
b49ddc76 check accessing an element past the end of a container (moneromooo-monero)
2305bf26 check return value for generate_key_derivation and derive_public_key (moneromooo-monero)
a4240d9f catch const exceptions (moneromooo-monero)
45a1c4c0 add empty container sanity checks when using front() and back() (moneromooo-monero)
56fa6ce1 tests: fix a buffer overread in a unit test (moneromooo-monero)
b4524892 rpc: guard against json parsing a non object (moneromooo-monero)
c2ed8618 easylogging++: avoid buffer underflow (moneromooo-monero)
187a6ab2 epee: trap failure to parse URI from request (moneromooo-monero)
061789b5 checkpoints: trap failure to load JSON checkpoints (moneromooo-monero)
ba2fefb9 checkpoints: pass std::string by const ref, not const value (moneromooo-monero)
38c8f4e0 mlog: terminate a string at last char, just in case (moneromooo-monero)
d753d716 fix a few leaks by throwing objects, not newed pointers to objects (moneromooo-monero)
fe568db8 p2p: use size_t for arbitrary counters instead of uint8_t (moneromooo-monero)
46d6fa35 cryptonote_protocol: sanity check chain hashes from peer (moneromooo-monero)
25584f86 cryptonote_protocol: print peer versions when unexpected (moneromooo-monero)
490a5d41 rpc: do not try to use an invalid txid in relay_tx (moneromooo-monero)
43f5269f Wallets now do not depend on the daemon rpc lib (moneromooo-monero)
bb89ae8b move connection_basic and network_throttle from src/p2p to epee (moneromooo-monero)
4abf25f3 cryptonote_core does not depend on p2p anymore (moneromooo-monero)
Partially implements #74.
Securely erases keys from memory after they are no longer needed. Might have a
performance impact, which I haven't measured (perf measurements aren't
generally reliable on laptops).
Thanks to @stoffu for the suggestion to specialize the pod_to_hex/hex_to_pod
functions. Using overloads + SFINAE instead generalizes it so other types can
be marked as scrubbed without adding more boilerplate.
3dffe71b new wipeable_string class to replace std::string passphrases (moneromooo-monero)
7a2a5741 utils: initialize easylogging++ in on_startup (moneromooo-monero)
54950829 use memwipe in a few relevant places (moneromooo-monero)
000666ff add a memwipe function (moneromooo-monero)
27aa8ce9 net_utils_base: fix peer list parsing (moneromooo-monero)
fe5ab2c4 epee: fix kv_unserialize return value when a field is not found (moneromooo-monero)
Deleted 3 out of 4 calls to method connection_basic::sleep_before_packet
that were erroneous / superfluous, which enabled the elimination of a
"fudge" factor of 2.1 in connection_basic::set_rate_up_limit;
also ended the multiplying of limit values and numbers of bytes
transferred by 1024 before handing them over to the global throttle
objects
0d9c0db9 Do not build against epee_readline if it was not built (Howard Chu)
178014c9 split off readline code into epee_readline (moneromooo-monero)
a9e14a19 link against readline only for monerod and wallet-wallet-{rpc,cli} (moneromooo-monero)
437421ce wallet: move some scoped_message_writer calls from the libs (moneromooo-monero)
e89994e9 wallet: rejig to avoid prompting in wallet2 (moneromooo-monero)
ec5135e5 move input_line from command_line to simplewallet (moneromooo-monero)
082db75f move cryptonote command line options to cryptonote_core (moneromooo-monero)
This fixes linking when path to openssl
is defined manually:
cmake -DOPENSSL_ROOT_DIR='/usr/include/openssl-1.0;/usr/lib/openssl-1.0' ...
This is useful for building with OpenSSL v1.0
when default system installation is v1.1.
The linking error is undefined SSL_load_error_strings symbol.
This is due to -L /usr/lib/openssl-1.0 not making it onto
the linkline (so -lssl pulls in the default system openssl).
Fixes compile error when building with OpenSSL v1.1:
contrib/epee/include/net/net_helper.h: In member function ‘void epee::net_utils::blocked_mode_client::shutdown_ssl()’:
contrib/epee/include/net/net_helper.h:579:106: error: ‘SSL_R_SHORT_READ’ was not declared in this scope
if (ec.category() == boost::asio::error::get_ssl_category() && ec.value() != ERR_PACK(ERR_LIB_SSL, 0, SSL_R_SHORT_READ))
^
contrib/epee/include/net/net_helper.h:579:106: note: suggested alternative: ‘SSL_F_SSL_READ’
See boost/asio/ssl/error.hpp.
Boost handles differences between OpenSSL versions.
cmake: fail if Boost is too old for OpenSSL v1.1
The commands handler must not be destroyed before the config
object, or we'll be accessing freed memory.
An earlier attempt at using boost::shared_ptr to control object
lifetime turned out to be very invasive, though would be a
better solution in theory.
15e62581 connection_context: initialize m_callback_request_count to 0 (moneromooo-monero)
5fa199c6 copyable_atomic: add a ctor with initial value (moneromooo-monero)
- internal nullptr checks
- prevent modifications to network_address (shallow copy issues)
- automagically works with any type containing interface functions
- removed fnv1a hashing
- ipv4_network_address now flattened with no base class
792ba4f0 Log categories can now be added to and removed from (moneromooo-monero)
48f92eb6 easylogging++: add categories getter (moneromooo-monero)
f35afe62 epee: factor log level/categories setting (moneromooo-monero)
ef005f5e p2p: add a couple early outs when the stop signal is received (moneromooo-monero)
80d361c7 abstract_tcp_server2: improve tracking/cancelling of early connections (moneromooo-monero)
close might end up dropping a ref, ending up removing the
connection from m_connects, as the lock is recursive. This'd
cause an out of bounds exception and kill the idle connection
maker thread
df0cffed cryptonote_protocol: warn if we see a higher top version we expect (moneromooo-monero)
317ab21a cryptonote_protocol: less strict check on top version on connect (moneromooo-monero)
cc81a371 cryptonote_protocol: update target height when syncing too (moneromooo-monero)
e2ad372b cryptonote_protocol: simplify and remove unnecessary casts (moneromooo-monero)
727e67ca cryptonote_protocol: print peer top height along with its version (moneromooo-monero)
b5345ef4 crypto: use malloc instead of alloca (moneromooo-monero)
80794b31 thread_group: set thread size to THREAD_STACK_SIZE (moneromooo-monero)
5524bc31 print peer id in 0 padded hex for consistency (moneromooo-monero)
8f8cc09b contrib: add sync_info to rlwrap command set (moneromooo-monero)
70b8c6d7 cryptonote_protocol: misc fixes to the new sync algorithm (moneromooo-monero)
We don't actually need to keep them past the call to start, as this
adds them to the config object list, and so they'll then be cancelled
already when the stop signal arrives. This allows removing the periodic
call to cleanup connections.
b7d6ec83 simplewallet: add (out of sync) or (no daemon) markers in the prompt (moneromooo-monero)
fa23a500 wallet2: add a is_synced function (moneromooo-monero)
f1307bbd node_rpc_proxy: add a proxy for target height (moneromooo-monero)
5d4ef719 core: speed up output index unique set calculation (moneromooo-monero)
19d7f568 perf_timer: allow profiling more granular than millisecond (moneromooo-monero)
bda8c598 epee: add nanosecond timer and pause/restart profiling macros (moneromooo-monero)
dc0b312f wallet_rpc_server: optionally return tx blobs on transfer calls (moneromooo-monero)
5eb79983 epee: add a KV_SERIALIZE variant for optional parameters (moneromooo-monero)
A block queue is now placed between block download and
block processing. Blocks are now requested only from one
peer (unless starved).
Includes a new sync_info coommand.
Nanosecond timer precision won't work on Windows, but we don't
care since I'm using that just for profiling incremental code
paths, but a Windows coder is welcome to add it if there's a way.
This was the case for monero-wallet-cli already, but not for
monerod, which was making it pretty spammy as it was duplicating
intended output. Since my original intent was to ensure logs
included command output for debugging, this achieves both.
76043b17 monero-wallet-cli: hang on exit in readline code (#2117) (moneromooo-monero)
a73a42a6 monero-wallet-cli: hang on exit in readline code (#2117) (moneromooo-monero)
be9d4f04 Fix multiline wallet cli output with readline (Jethro Grassie)
readline_buffer: fix start/stop threads being starved by process
process could run for quite some time re-acquiring the process
lock, leaving start/stop starving. Yielding after unlock in
process is much better but doesn't seem to be enough to reliably
yield, so we sleep for a millisecond, which should be transparent
for user input anyway.
Signed-off-by: Jethro Grassie <jtg@xtrabass.com>
monero-wallet-cli commands which have multine output sometimes causes
issues with the readline support. This patch fixes show_transfers,
payments and incoming_transfers.
Existing tests: block, transaction, signature, cold outputs,
cold transaction.
Data for these is in tests/data/fuzz.
A convenience shell script is in contrib/fuzz_testing/fuzz.sh, eg:
contrib/fuzz_testing/fuzz.sh signature
The fuzzer will run indefinitely, ^C to stop.
Fuzzing is currently supported for GCC only. I can't get CLANG
to build Monero here as it dies on some system headers, so if
someone wants to make it work on both, that'd be great.
In particular, the __AFL_LOOP construct should be made to work
so that a given run can fuzz multiple inputs, as the C++ load
time is substantial.
07c4276c Don't issue a new timedsync while one is already in progress (Howard Chu)
cf3a376c Don't timeout a slow operation that's making progress (Howard Chu)
340830de Fix PR#2039 (Howard Chu)
It'd eat up a core constantly, due to spending its time jumping
back and forth between userland and kernel. We now wait for up
to a millisecond in kernel, which will be transparent to the user
and drop to idle most of the time.
This PR adds readline support to the daemon and monero-wallet-cli. Only
GNU readline is supported (e.g. not libedit) and there are cmake checks
to ensure this.
There is a cmake variable, Readline_ROOT_DIR that can specify a
directory to find readline, otherwise some default paths are searched.
There is also a cmake option, USE_READLINE, that defaults to ON. If set
to ON, if readline is not found, the build continues but without
readline support.
One negative side effect of using readline is that the color prompt in
the wallet-cli now has no color and just uses terminal default. I know
how to fix this but it's quite a big change so will tackle another time.
A timedsync is issued every minute on a connection, but the input
tineout is 2 minutes. This means a new sync request could be issued
while a slow sync request was already in progress. The additional
request will further clog the network on a slow connection, and
cause a premature timeout.
All code which was using ip and port now uses a new IPv4 object,
subclass of a new network_address class. This will allow easy
addition of I2P addresses later (and also IPv6, etc).
Both old style and new style peer lists are now sent in the P2P
protocol, which is inefficient but allows peers using both
codebases to talk to each other. This will be removed in the
future. No other subclasses than IPv4 exist yet.
- Performance improvements
- Added `span` for zero-copy pointer+length arguments
- Added `std::ostream` overload for direct writing to output buffers
- Removal of unused `string_tools::buff_to_hex`
Because some people just won't even try to read what is written
and freak out because the word FATAL is in here, despite the
context making it clear it's not an error.
8bdc86be protocol: speed up sync by minimizing duplicate work (moneromooo-monero)
61dfa310 epee: fix some log macros not printing context nicely (moneromooo-monero)
c02e1cb9 Updates to epee HTTP client code - http_simple_client now uses std::chrono for timeouts - http_simple_client accepts timeouts per connect / invoke call - shortened names of epee http invoke functions - invoke command functions only take relative path, connection is not automatically performed (Lee Clagett)
- http_simple_client now uses std::chrono for timeouts
- http_simple_client accepts timeouts per connect / invoke call
- shortened names of epee http invoke functions
- invoke command functions only take relative path, connection
is not automatically performed
using the MONERO_LOG_FORMAT environment variable.
Default is:
%datetime{%Y-%M-%d %H:%m:%s.%g}\t%thread\t%level\t%logger\t%loc\t%msg
Field list in easylogging++ documentation.
Don't forget to escape as needed.
This replaces the epee and data_loggers logging systems with
a single one, and also adds filename:line and explicit severity
levels. Categories may be defined, and logging severity set
by category (or set of categories). epee style 0-4 log level
maps to a sensible severity configuration. Log files now also
rotate when reaching 100 MB.
To select which logs to output, use the MONERO_LOGS environment
variable, with a comma separated list of categories (globs are
supported), with their requested severity level after a colon.
If a log matches more than one such setting, the last one in
the configuration string applies. A few examples:
This one is (mostly) silent, only outputting fatal errors:
MONERO_LOGS=*:FATAL
This one is very verbose:
MONERO_LOGS=*:TRACE
This one is totally silent (logwise):
MONERO_LOGS=""
This one outputs all errors and warnings, except for the
"verify" category, which prints just fatal errors (the verify
category is used for logs about incoming transactions and
blocks, and it is expected that some/many will fail to verify,
hence we don't want the spam):
MONERO_LOGS=*:WARNING,verify:FATAL
Log levels are, in decreasing order of priority:
FATAL, ERROR, WARNING, INFO, DEBUG, TRACE
Subcategories may be added using prefixes and globs. This
example will output net.p2p logs at the TRACE level, but all
other net* logs only at INFO:
MONERO_LOGS=*:ERROR,net*:INFO,net.p2p:TRACE
Logs which are intended for the user (which Monero was using
a lot through epee, but really isn't a nice way to go things)
should use the "global" category. There are a few helper macros
for using this category, eg: MGINFO("this shows up by default")
or MGINFO_RED("this is red"), to try to keep a similar look
and feel for now.
Existing epee log macros still exist, and map to the new log
levels, but since they're used as a "user facing" UI element
as much as a logging system, they often don't map well to log
severities (ie, a log level 0 log may be an error, or may be
something we want the user to see, such as an important info).
In those cases, I tried to use the new macros. In other cases,
I left the existing macros in. When modifying logs, it is
probably best to switch to the new macros with explicit levels.
The --log-level options and set_log commands now also accept
category settings, in addition to the epee style log levels.
5783dd8c tests: add unit tests for uri parsing (moneromooo-monero)
82ba2108 wallet: add API and RPC to create/parse monero: URIs (moneromooo-monero)
d9001b43 epee: add functions to convert from URL format (ie, %XX values) (moneromooo-monero)
When receiving an answer packet, the command code was passed
to the callback instead of the error code. This was hiding
the "command not found" failure from the peer, and in turn
causing the code to attempt to deserialize a non existent
reply string.
48b57d8 monero.supp: valgrind suppressions file (moneromooo-monero)
ffd8c41 ringct: check the size of amount_keys is the same as destinations (moneromooo-monero)
836669d ringct: always shutdown the boost io service (moneromooo-monero)
This is intended to catch traffic coming from a web browser,
so we avoid issues with a web page sending a transfer RPC to
the wallet. Requiring a particular user agent can act as a
simple password scheme, while we wait for 0MQ and proper
authentication to be merged.
Support building internal libraries as shared. This reduces
development time by eliminating the need to re-link all
binaries every time non-interface code in the library changes.
Instead, can hack on libxyz, then `make libxyz`, and re-run
monerod.
By default BUILD_SHARED_LIBS is OFF in release build type,
and ON in debug build type, but can be overriden with -D.
This adds [snap](https://snapcraft.io) packaging to the project. See the
link for more information on snaps. Snap packages install on all Linux
distributions. On Ubuntu, snap confinement with apparmor and seccomp
provide an additional layer of security.
This snap sets up monerod as a systemd service, which should start
immediately on install. To access the wallet CLI, simply run `monero`
(/snap/bin/monero). I think it's a really quick & easy way to get
started with monero.
I've made some opinionated decisions in the packaging just to kick this
off, but I'm happy to iterate on this stuff.
The noexcept specs were added to make GCC 6.1.1 happy (#846), but this
one was missing (because GCC did not complain about it on Linux, but
does complain on OSX).
5dc09f2 wallet_rpc_server: fix some string values being returned between <> (moneromooo-monero)
f8213c0 Require 64/16 characters for payment ids (moneromooo-monero)
The default behavior for hex string parsing would allow the
last digit to be made from a single hexadecimal character,
which is correct, but we typically do not want that as it
gets confusing and easy to not spot wrong input size.
The destructors get a noexcept(true) spec by default, but these
destructors in fact throw exceptions. An alternative fix might be to not
throw (most if not all of these throws are non-essential
error-reporting/logging).
1c0bffb Restrict also 'get_connections' and 'getbans' APIs. (osensei)
9f8bc49 Don't allow 'flush_txpool' and 'setbans' JSON_RPC methods when running in restricted mode. (osensei)
When the send queue limit is reached, it is likely to not drain
any time soon. If we call close on the connection, it will stay
alive, waiting for the queue to drain before actually closing,
and will hit that check again and again. Since the queue size
limit is the reason we're closing in the first place, we call
shutdown directly.
If we reach the send queue size limit, we need to release the lock,
or we will deadlock and it will never drain.
If we reach that limit, it's likely there's another problem in the
first place though, so it will probably not drain in practice either,
unless some kind of transient network timeout.
Since connections from the ::connect method are now kept in
a deque to be able to cancel them on exit, this leaks both
memory and a file descriptor. Here, we clean those up after
30 seconds, to avoid this. 30 seconds is higher then the
5 second timeout used in the async code, so this should be
safe. However, this is an assumption which would break if
that async code was to start relying on longer timeouts.
When the boost ioservice is stopped, pending work notifications
will not happen. This includes deadline timers, which would
otherwise time out the now cancelled I/O operations. When this
happens just after starting a new connect operation, this can
leave that operations in a state where it won't receive either
the completion notification nor a timeout, causing a hang.
This is fixed by keeping a list of connections corresponding
to the connect operations, and cancelling them before stopping
the boost ioservice.
Note that the list of these connections can grow unbounded, as
they're never cleaned up. Cleaning them up would involve
working out which connections do not have any pending work,
and it's not quite clear yet how to go about this.
This prevents an exception from existing the loop without
calling the exit handler, if one is defined.
The daemon defines one, which stops the p2p layer, and will
only exit once the p2p layer is shut down. This would cause
a hang upon an exception, as the input thread would have
exited and the daemon would wait forever with no console
user input.
The daemon will be polled every 90 seconds for new blocks.
It is enabled by default, and can be turned on/off with
set auto-refresh 1 and set auto-refresh 0 in the wallet.
The daemon registers a custom exit command, which cause the
loop to stop. Catch this case before printing "Failed to read line"
as this is an expected case.
If process path name isn't found, then leave log file name blank.
This also applies if a process name is found, but it's blank after
removing a trailing dot extension.
It uses the async console handler differently than simplewallet,
and wasn't running the same exit code, causing it to never actually
exit after breaking out of the console entry loop.
Add fix for compile error with multiple uses of peerid_type (uint64_t)
variable in lambda expression.
- known GCC issue: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65843
epee: replace return value of nullptr for expected boolean with false.
Fixes#231.
Daemon interactive mode is now working again.
RPC mapped calls in daemon and wallet have both had connection_context
removed as an argument as that argument was not being used anywhere.
+toc -doc -drmonero
Fixed the windows path, and improved logging and data
(for graph) logging, fixed some locks and added more checks.
Still there is a locking error,
not added by my patches, but present in master version
(locking of map/list of peers).
many RPC functions added by the daemonize changes
(and related changes on the upstream dev branch that were not merged)
were commented out (apart from return). Other than that, this *should*
work...at any rate, it builds, and that's something.
new update of the pr with network limits
more debug options:
discarding downloaded blocks all or after given height.
trying to trigger the locking errors.
debug levels polished/tuned to sane values.
debug/logging improved.
warning: this pr should be correct code, but it could make
an existing (in master version) locking error appear more often.
it's a race on the list (map) of peers, e.g. between closing/deleting
them versus working on them in net-limit sleep in sending chunk.
the bug is not in this code/this pr, but in the master version.
the locking problem of master will be fixed in other pr.
problem is ub, and in practice is seems to usually cause program abort
(tested on debian stable with updated gcc). see --help for option
to add sleep to trigger the error faster.
Update of the PR with network limits
works very well for all speeds
(but remember that low download speed can stop upload
because we then slow down downloading of blockchain
requests too)
more debug options
fixed pedantic warnings in our code
should work again on Mac OS X and FreeBSD
fixed warning about size_t
tested on Debian, Ubuntu, Windows(testing now)
TCP options and ToS (QoS) flag
FIXED peer number limit
FIXED some spikes in ingress/download
FIXED problems when other up and down limit
commands and options for network limiting
works very well e.g. for 50 KiB/sec up and down
ToS (QoS) flag
peer number limit
TODO some spikes in ingress/download
TODO problems when other up and down limit
added "otshell utils" - simple logging (with colors, text files channels)
