Commit Graph

11765 Commits

Author SHA1 Message Date
j-berman
513dae88dc small touchups 2024-09-17 11:23:28 -07:00
j-berman
21664f99b2 fcmp++: use fe * instead of vector<fe> + clean up 2024-09-17 11:23:28 -07:00
j-berman
330b82ff9a Fix build errs + warnings 2024-09-17 11:23:28 -07:00
j-berman
b2ea86235d fcmp++: use batch inversion when converting outputs to leaf tupels 2024-09-17 11:23:28 -07:00
j-berman
358a66f6b1 crypto: fe_batch_invert using Montgomery's trick
Speeds up inverting many elems at once 95%+
2024-09-17 11:23:28 -07:00
j-berman
5f4b20b8b6 fcmp++: fix build errs for ki_context_t hash specialization
- plus slightly cleaner hash
2024-09-12 18:35:09 -07:00
j-berman
7fa01d20c0 fcmp++: key image migration
- Removing the sign bit from key images enables an optimization for
fcmp's.
- If an fcmp includes a key image with sign bit cleared,while the
same key image with sign bit set exists in the chain already via a
ring signature, then the fcmp would be a double spend attempt and
the daemon must be able to detect and reject it.
- In order for the daemon to detect such double spends, upon
booting the daemon, we clear the sign bit from all key images
already in the db. We also make sure that all key images held in
memory by the pool have sign bit cleared as well.
- Key images with sign bit cleared are a new type:
`crypto::key_image_y`. The sign bit can be cleared via
`crypto::key_image_to_y`.
- The `_y` denotes that the encoded point is now the point's y
coordinate.
- In order to maintain backwards compatibility with current RPC
consumers, the daemon keeps track of which key images have sign
bit cleared and not, so that upon serving
`spent_key_image_info::id_hash`, the daemon can re-construct the
original key image and serve it to clients.
2024-09-12 18:35:02 -07:00
j-berman
043597426b fcmp++ lmdb: migration touchups
- If locked output migration step completes, then program exits
while migration step to grow the tree is in progress, make sure
the migration picks back up where it left off growing the tree.
- Make sure db cursor gets set in all cases when renaming
block infn table.
2024-09-12 18:34:50 -07:00
j-berman
b055eb3f61 fcmp++: output may not be present in locked outputs table on remove
- If the output is invalid/unspendable, upon unlock it will be
deleted from the locked outputs table and then won't be used to
grow the tree. Upon reorg/pop blocks, the invalid output won't be
re-added to the locked outputs table upon trimming the tree. Thus,
it's possible for an invalid/unspendable output to not be present
in the locked outputs table upon remove.
2024-09-11 10:13:48 -07:00
j-berman
8fc87d72a2 fcmp++: multithreaded convert valid outputs into leaf tuples 2024-09-11 10:13:48 -07:00
j-berman
90164e3d6b fcmp++: multithreaded hashing children chunks into tree 2024-09-11 10:13:48 -07:00
j-berman
e636c384a9 fix ubuntu 20 compile 2024-09-11 10:13:48 -07:00
j-berman
f47c60ec63 fcmp++ tests: init tree in db once, then copy db for tests
- speeds up trim_tree test by 60%+
2024-09-11 10:13:48 -07:00
j-berman
9784ced3db fcmp++ tests: init tree once in memory and grow copy 2024-09-11 10:13:48 -07:00
j-berman
0ff1eaf948 Use macro to de-dupe curve trees test logic 2024-09-11 10:13:48 -07:00
j-berman
b71f2440bc fcmp++: better test names 2024-09-11 10:13:48 -07:00
j-berman
0688538110 fcmp++: add test to trim the tree then grow again after trimming 2024-09-11 10:13:48 -07:00
j-berman
072a82dd3f fcmp++: tests now test trimming to empty tree 2024-09-11 10:13:48 -07:00
j-berman
2890e8cf38 fcmp++: LMDB touchups
- fixes usage of MDB_NEXT and MDB_NEXT_DUP, allowing the db call
to set key and value
2024-09-11 10:13:48 -07:00
j-berman
acc7d05885 fcmp++ LMDB: use dummy key optimization on leaves table
- Save 8 bytes per leaf by using DUPFIXED table and dummy
"zerokval" key and attaching leaf_idx as prefix to data to serve
as DUPSORT key
2024-09-11 10:13:48 -07:00
j-berman
0a604a9786 fcmp++: Enable trimming to empty tree 2024-09-11 10:13:48 -07:00
j-berman
16ff6a9e68 fcmp++: trim tree when removing a block
- trim_tree now re-adds trimmed outputs back to the locked outputs
table. remove_output then deletes from the locked output table.
- Since outputs added to the tree in a specific block may have
originated from distinct younger blocks (thanks to distinct unlock
times), we need to store the 8 byte output_id in the leaves table
as well, so that in the event of a reorg, upon removing outputs
from the tree we can add them back to the locked outputs table
in the correct order.
2024-09-11 10:13:48 -07:00
j-berman
41b1985f63 Fix compile on arm-linux-androideabi (32-bit) using a newer NDK
- @tobtoht reported err with size_t -> uint64_t
- Also address some PR comments (@vtnerd namespace comment +
@boog900 freeing unallocated rust type)
- Some cleaning
2024-09-03 14:35:26 -07:00
j-berman
47d47bdd20 fcmp++: proof len from inputs *AND merkle tree depth 2024-08-14 11:42:30 -07:00
j-berman
ee19361ea0 Remove extra gcc install fixes windows build 2024-08-14 08:58:57 -07:00
j-berman
918befb0f5 new_leaf_tuples -> new_outputs 2024-08-13 09:25:43 -07:00
j-berman
67f5546d10 lmdb touchup && OutputsByUnlockBlock map -> unordered_map 2024-08-13 09:24:43 -07:00
j-berman
83d56597e2 Clean lmbd impl
- Reverted back to storing output_id in locked_outputs table; it's
required to make sure outputs enter the tree in chain order I see
no other simple way.
- Removed unnecessary comments and db flags (MDB_APPENDDUP already
makes sure key/value doesn't already exist, and when inserting,
every global output id should be unique, so should never get that
error)
2024-08-10 02:20:55 -07:00
j-berman
6525df113c Don't store output_id in locked_outpust table, table stays ordered 2024-08-09 18:31:18 -07:00
j-berman
7389cb6bee add missing files 2024-08-09 16:35:25 -07:00
j-berman
b6bcca9899 Remove ringct dep in fcmp_pp, impl in fcmp_pp_crypto 2024-08-09 16:29:31 -07:00
j-berman
9ad49189bb link correct cncrypto and ringct_basic libs 2024-08-09 16:24:21 -07:00
j-berman
f17db01250 fcmp++: store {output pubkey, commitment} in db, pre-torsion clear
- We must use the output pubkey to calculate key image generator I
- Since torsion cleared outputs can be spent via ring sig today,
if we torsion clear outputs **before** calculating I, then the key
image of torsioned outputs will be different when constructing
fcmp's, effectively enabling a double spend of torsioned outputs
via ring sig before fcmp's and again via fcmp.
- Storing {output pubkey, commitment} instead of {O.x,I.x,C.x} to
save 32 bytes per output.
2024-08-09 15:43:18 -07:00
j-berman
8b12a335c6 fcmp++: implement iterative audit_tree function
- Recursion goes too deep
2024-08-08 19:26:08 -07:00
j-berman
9f0dd859e6 fix clang compile errors 2024-08-08 14:55:49 -07:00
j-berman
d4847f649e Rename everything from fcmp* to fcmp_pp 2024-08-08 14:10:34 -07:00
j-berman
d72f405cee Revert DELETE_DB macro to original spot 2024-08-08 13:43:54 -07:00
j-berman
3a5cf70066 32-bit platform compatibility in Rust FFI (untested) 2024-08-08 13:38:45 -07:00
j-berman
10c6c12b18 fcmp++: compilation fixes + misc. cleanup
- Removed call to hash_init_point in constructor
- Replaced global static CURVE_TREES_V1 with a smart pointer
- Don't need to link Rust static lib when including curve_trees.h
- leaves table doesn't need dupsort flags, all leaves should be
unique by key
- rename fcmp -> fcmp_pp
- return when 0 leaves passed into trim_tree
2024-08-08 13:31:21 -07:00
j-berman
edded7e6e3 fcmp++: Restart migration from where it leaves off 2024-08-08 13:01:04 -07:00
j-berman
5e76191afe cleaner crypto for converting output to leaf tuple 2024-08-02 23:22:39 -07:00
j-berman
30fc80b33e Don't copy when flattening leaves 2024-08-02 22:34:06 -07:00
j-berman
cbf6a5d618 Optimize conversion from output to leaf tuple 2024-08-02 22:32:03 -07:00
j-berman
b90cee8bab Store {O,C} for each leaf tuple instead of {O.x,I.x,C.x}
- Can derive {O.x,I.x,C.x} from {O,C}
- Note: this slows down tests since they do the derivation both
on insertion into the tree, and when auditing the tree
- At the hard fork, we don't need to store {O,C} in the
output_amounts table anymore since that table will no longer be
useful
2024-08-02 10:28:13 -07:00
j-berman
34eafa85f3 Store points in the tree in compressed encoding (32 bytes) 2024-08-01 10:04:51 -07:00
j-berman
95114f9253 Use explicit code instead of macro in custom fcmp_pp serialization
It's slightly different than other usages of the macro, so figured
it makes sense to just write out the code to do what I expect it
to do
2024-07-31 23:15:08 -07:00
j-berman
e40c5bb0fc fix json tagging in fcmp_pp serialization 2024-07-31 22:50:17 -07:00
j-berman
54d5d0d5c7 fcmp++: add support for new fcmp types in cryptonote::transaction
- Replace CLSAGs with a single fcmp_pp
- fcmp_pp is an opaque vector of bytes. The length of the vector
is calculated from the number of inputs on serialization (i.e. the
length is not serialized, only the raw bytes are serialized)
- Includes tests for binary serialization happy path and errors
2024-07-31 18:13:23 -07:00
Luke Parker
16a8ce3a45 Add * point from bytes 2024-07-29 03:40:44 -07:00
j-berman
729e31df74 include fcmp/curve_trees.h in db_lmdb.h 2024-07-29 03:40:44 -07:00