If a queued job uses a waiter, then we want to run that waiter's
jobs in the current thread if all threads are busy, even if the
queue is empty, since there is no guarantee that any thread will
free up to take care of that new job, since all the threads might
be running a job which spawns such a recursive job and will block
till that recursive job is done, which it will never be since it
relies on the queue being polled by one of those blocked threads.
Partially implements #74.
Securely erases keys from memory after they are no longer needed. Might have a
performance impact, which I haven't measured (perf measurements aren't
generally reliable on laptops).
Thanks to @stoffu for the suggestion to specialize the pod_to_hex/hex_to_pod
functions. Using overloads + SFINAE instead generalizes it so other types can
be marked as scrubbed without adding more boilerplate.
3dffe71b new wipeable_string class to replace std::string passphrases (moneromooo-monero)
7a2a5741 utils: initialize easylogging++ in on_startup (moneromooo-monero)
54950829 use memwipe in a few relevant places (moneromooo-monero)
000666ff add a memwipe function (moneromooo-monero)
If a translation file exists in a "translations" directory located in
the same directory as the binary, it is used in priority (this can be
useful when working on translations as you don't have to recompile the
whole program all the time), and if no such file is found the embedded
translation file is used (if it exists).
0d9c0db9 Do not build against epee_readline if it was not built (Howard Chu)
178014c9 split off readline code into epee_readline (moneromooo-monero)
a9e14a19 link against readline only for monerod and wallet-wallet-{rpc,cli} (moneromooo-monero)
437421ce wallet: move some scoped_message_writer calls from the libs (moneromooo-monero)
e89994e9 wallet: rejig to avoid prompting in wallet2 (moneromooo-monero)
ec5135e5 move input_line from command_line to simplewallet (moneromooo-monero)
082db75f move cryptonote command line options to cryptonote_core (moneromooo-monero)
wallet2 is a library, and should not prompt for stdin. Instead,
pass a function so simplewallet can prompt on stdin, and a GUI
might display a window, etc.
In practice, this seems to cause monero-wallet-rpc to exit
when ^C quits whatever its output is piped into (such as tee),
but it saves, while it did not before.
6137a0b9 blockchain: reject unsorted ins and outs from v7 (moneromooo-monero)
16afab90 core: sort ins and outs key key image and public key, respectively (moneromooo-monero)
0c36b9f9 common: add apply_permutation file and function (moneromooo-monero)
0299cb77 Fix various oversights/bugs in ZMQ RPC server code (Thomas Winget)
77986023 json serialization for rpc-relevant monero types (Thomas Winget)
5c1e08fe Refactor some things into more composable (smaller) functions (Thomas Winget)
9ac2ad07 DRY refactoring (Thomas Winget)
Structured {de-,}serialization methods for (many new) types
which are used for requests or responses in the RPC.
New types include RPC requests and responses, and structs which compose
types within those.
# Conflicts:
# src/cryptonote_core/blockchain.cpp
3dd34a49 Cleanup test impact of moving blockchain_db_types() (Howard Chu)
80344740 More DB support cleanup (Howard Chu)
4c7f8ac0 DB cleanup (Howard Chu)
df0cffed cryptonote_protocol: warn if we see a higher top version we expect (moneromooo-monero)
317ab21a cryptonote_protocol: less strict check on top version on connect (moneromooo-monero)
cc81a371 cryptonote_protocol: update target height when syncing too (moneromooo-monero)
e2ad372b cryptonote_protocol: simplify and remove unnecessary casts (moneromooo-monero)
727e67ca cryptonote_protocol: print peer top height along with its version (moneromooo-monero)
b5345ef4 crypto: use malloc instead of alloca (moneromooo-monero)
80794b31 thread_group: set thread size to THREAD_STACK_SIZE (moneromooo-monero)
5524bc31 print peer id in 0 padded hex for consistency (moneromooo-monero)
8f8cc09b contrib: add sync_info to rlwrap command set (moneromooo-monero)
70b8c6d7 cryptonote_protocol: misc fixes to the new sync algorithm (moneromooo-monero)
Hide DB types from db_types.h - no reason to recompile dependencies
when DB types change.
Also remove lingering in-memory DB references, they've been
obsolete since 9e82b694da
e499ff33 simplewallet: factor out message_writer (moneromooo-monero)
7ed5ab47 scoped_message_writer: pause readline to match simplewallet (moneromooo-monero)
5d4ef719 core: speed up output index unique set calculation (moneromooo-monero)
19d7f568 perf_timer: allow profiling more granular than millisecond (moneromooo-monero)
bda8c598 epee: add nanosecond timer and pause/restart profiling macros (moneromooo-monero)
This avoids having to include p2p_protocol_defs.h in util.h,
as util.h is used a lot, and p2p_protocol_defs.h includes
a lot of other things that most users don't need.
Library code should definitely not ask for console input unless
it's clearly an input function. Delegating the user interaction
part to the caller means it can now be used by a GUI, or have a
decision algorithm better adapted to a particular caller.
This was the case for monero-wallet-cli already, but not for
monerod, which was making it pretty spammy as it was duplicating
intended output. Since my original intent was to ensure logs
included command output for debugging, this achieves both.
This PR adds readline support to the daemon and monero-wallet-cli. Only
GNU readline is supported (e.g. not libedit) and there are cmake checks
to ensure this.
There is a cmake variable, Readline_ROOT_DIR that can specify a
directory to find readline, otherwise some default paths are searched.
There is also a cmake option, USE_READLINE, that defaults to ON. If set
to ON, if readline is not found, the build continues but without
readline support.
One negative side effect of using readline is that the color prompt in
the wallet-cli now has no color and just uses terminal default. I know
how to fix this but it's quite a big change so will tackle another time.
load_txt_records_from_dns attempts to distribute `a = 0, b = -1` where
(b = dns_urls.size() - 1) and IntType is signed integer. This results in
an infinite recursion which leads to SIGSEGV.
c02e1cb9 Updates to epee HTTP client code - http_simple_client now uses std::chrono for timeouts - http_simple_client accepts timeouts per connect / invoke call - shortened names of epee http invoke functions - invoke command functions only take relative path, connection is not automatically performed (Lee Clagett)
- http_simple_client now uses std::chrono for timeouts
- http_simple_client accepts timeouts per connect / invoke call
- shortened names of epee http invoke functions
- invoke command functions only take relative path, connection
is not automatically performed
This replaces the epee and data_loggers logging systems with
a single one, and also adds filename:line and explicit severity
levels. Categories may be defined, and logging severity set
by category (or set of categories). epee style 0-4 log level
maps to a sensible severity configuration. Log files now also
rotate when reaching 100 MB.
To select which logs to output, use the MONERO_LOGS environment
variable, with a comma separated list of categories (globs are
supported), with their requested severity level after a colon.
If a log matches more than one such setting, the last one in
the configuration string applies. A few examples:
This one is (mostly) silent, only outputting fatal errors:
MONERO_LOGS=*:FATAL
This one is very verbose:
MONERO_LOGS=*:TRACE
This one is totally silent (logwise):
MONERO_LOGS=""
This one outputs all errors and warnings, except for the
"verify" category, which prints just fatal errors (the verify
category is used for logs about incoming transactions and
blocks, and it is expected that some/many will fail to verify,
hence we don't want the spam):
MONERO_LOGS=*:WARNING,verify:FATAL
Log levels are, in decreasing order of priority:
FATAL, ERROR, WARNING, INFO, DEBUG, TRACE
Subcategories may be added using prefixes and globs. This
example will output net.p2p logs at the TRACE level, but all
other net* logs only at INFO:
MONERO_LOGS=*:ERROR,net*:INFO,net.p2p:TRACE
Logs which are intended for the user (which Monero was using
a lot through epee, but really isn't a nice way to go things)
should use the "global" category. There are a few helper macros
for using this category, eg: MGINFO("this shows up by default")
or MGINFO_RED("this is red"), to try to keep a similar look
and feel for now.
Existing epee log macros still exist, and map to the new log
levels, but since they're used as a "user facing" UI element
as much as a logging system, they often don't map well to log
severities (ie, a log level 0 log may be an error, or may be
something we want the user to see, such as an important info).
In those cases, I tried to use the new macros. In other cases,
I left the existing macros in. When modifying logs, it is
probably best to switch to the new macros with explicit levels.
The --log-level options and set_log commands now also accept
category settings, in addition to the epee style log levels.
07b9138c support importing unportable outputs (kenshi84)
2ac80075 also use portable serializer for boost_serialization_helper.h and net_node.inl, completely adandon boost/archive/binary_oarchive.hpp (kenshi84)
d1d6e27a moved boost cpp into hpp since they're supposed to be header only (kenshi84)
66e6af89 added experimental boost::archive::portable_binary_{i|o}archive (kenshi84)
This is intended to catch traffic coming from a web browser,
so we avoid issues with a web page sending a transfer RPC to
the wallet. Requiring a particular user agent can act as a
simple password scheme, while we wait for 0MQ and proper
authentication to be merged.
Keep the immediate direct deps at the library that depends on them,
declare deps as PUBLIC so that targets that link against that library
get the library's deps as transitive deps.
Break dep cycle between blockchain_db <-> crytonote_core.
No code refactoring, just hide cycle from cmake so that
it doesn't complain (cycles are allowed only between
static libs, not shared libs).
This is in preparation for supproting BUILD_SHARED_LIBS cmake
built-in option for building internal libs as shared.
By default the flag is enabled whenever libunwind is found on the
system, with the exception of static build on OSX (for which we can't
install the throw hook #932 due to lack of support for --wrap in OSX
ld64 linker).
Tested that it builds with:
gcc 6.1.1, STATIC=OFF,i686
gcc 6.1.1, STATIC=OFF,armv7h
clang 3.8, STATIC=OFF,i686
clang 3.8, STATIC=OFF,armv7h
gcc 6.1.1, STATIC=ON,i686
clang 3.8, STATIC=ON,i686
Also tested that stack trace is generated fine on exception on:
i686, gcc 6.1.1, STATIC=OFF
(didn't bother testing all the other platforms/configs)
This should fix the build problem on OSX (#871, #901), but
I don't have OSX, so I could only test Clang on Linux.
The former was a faulty "fix" for gmtime_r not existing on Windows. The latter is needed only for dynamic builds, and is not included with msys2, which ends up fine because Windows is only built static at this time.
It sets the max number of threads to use for a parallel job.
This is different that the number of total threads, since monero
binaries typically start a lot of them.
We also replace the --fakechain option with an optional structure
containing details about configuration for the core/blockchain,
for test purposes. This seems more future friendly.
Sample use:
DNS_PUBLIC=tcp torsocks bin/bitmonerod --p2p-bind-ip 127.0.0.1
Test:
Run above with --log-level 4 with and without DNS_PUBLIC environment
variable set.
DNS debugging info should show successful DNS lookups only when
DNS_PUBLIC is set to "tcp":
DNS lookup for seeds.moneroseeds.se: 17 results
DNS lookup for seeds.moneroseeds.ae.org: 17 results
DNS lookup for seeds.moneroseeds.ch: 12 results
DNS lookup for seeds.moneroseeds.li: 12 results
The core tests use the blockchain, and reset it to be able
to add test data to it. This does not play nice with the
databases, since those will save that data without an explicit
save call.
We add a fakechain flag that the tests will set, which tells
the core and blockchain code to use a separate database, as
well as skip a few things like checkpoints and fixup, which
only make sense for real data.
This fixes coretests, which does not register daemon specific arguments,
but uses core, which uses those arguments. Also gets rid of an unwanted
dependency on daemon code from core.
^C while in manual refresh will cancel the refresh, since that's
often an annoying thing to have to wait for. Also, a manual refresh
command will interrupt any running background refresh and take
over, rather than wait for the background refresh to be done, and
look to be hanging.
There are various locale related bugs in various versions of boost,
where exceptions are thrown in boost::filesystem APIs when the
current locale is not to boost's liking. It's not clear what "not
to boost's liking" means in detail, though "en" and "en_US.UTF-8"
are not to its liking.
Fix it by running a test function that's known to throw in such
a case, and resetting LANG and LC_ALL to C if an exception is
thrown. In simplewallet, the locale is queried before that so the
correct translations will still be used.
This ensures one can't instanciate a DNSResolver object by
mistake, but uses the singleton. A separate create static
function is added for cases where a new object is explicitely
needed.
Based on tewinget's update.
Make OpenAlias address format independent of existing DNS functions.
Add tests.
Test:
make debug-test
cd build/debug/tests/unit_tests
# test that regular DNS functions work, including IPv4 lookups.
# also test function that converts OpenAlias address format
make && ./unit_tests --gtest_filter=DNSResolver*
# test that OpenAlias addresses like donate@getmonero.org work from
# wallet tools
make && ./unit_tests --gtest_filter=AddressFromURL.Success
Due to a bug in unbound, we were passing a string containing a null
character to ub_ctx_resolvconf and ub_ctx_hosts rather than a NULL
pointer. On *nix this wasn't causing headache, but on Windows this was
causing unbound to not correctly load DNS settings from the OS.
Note on the bug: in a Windows-specific code branch in the function
ub_ctx_hosts(), if the hosts file specified was a NULL pointer, a call
to getenv() was stored in a local char* and later freed. This is
incorrect, as we do not own that data, and caused the program to crash.
DNSSEC is now implemented with the hardcoded key from unbound.
This will need to be not hardcoded in the future, but is okay for now.
Unit tests updated for DNSSEC (as well as for the fact that, contrary to
previous assumption, example.com does not have a static IP address).
many RPC functions added by the daemonize changes
(and related changes on the upstream dev branch that were not merged)
were commented out (apart from return). Other than that, this *should*
work...at any rate, it builds, and that's something.
Note: DNSResolver does not yet *use* DNSSEC, but rather this commit is
preparation for including DNSSEC validation. The function in
src/wallet/wallet2.cpp that uses DNSResolver still needs its parameters
updated accordingly.
ldns dependency was only still around for constants defined in ldns/rr.h,
but those constants are RFC specified DNS constants, and to reduce deps
have been replicated in dns_utils.h instead of including ldns/rr.h.
The previous implementation was almost certainly a typo.
full_block_size is the maximum index in the encoded_block_sizes array,
and size is used as an index in this array. So now 1 <= size <=
full_block_size == 8 instead of 1 <= size <= sizeof(full_block_size) ==
size_of(size_t) == ? (maybe 4 on 32-bit systems!)
