Commit Graph

11727 Commits

Author SHA1 Message Date
j-berman
36f1e1965f Fix grow_tree, restructure it, and clean the approach
The initial impl didn't capture the following edge case:

  - Tree has 3 (or more) layers + 1 leaf layeri
  - Leaf layer last chunk IS full
  - Layer 0 last chunk is NOT full
  - Layer 1 last chunk is NOT full
  - Layer 2 last chunk IS NOT full

In this case, when updating layer 1, we need to use layer 0's old
last hash to update layer 1's old last hash. Same for Layer 2.

The solution is to use logic that checks the *prev* layer when
updating a layer to determine if the old last hash from the prev
layer is needed.

This commit restructures the grow_tree impl to account for this
and simplifies the approach as follows:

1. Read the tree to get num leaf tuples + last hashes in each layer

2. Get the tree extension using the above values + new leaf tuples
2a. Prior to updating the leaf layer, call the function
get_update_leaf_layer_metadata. This function uses existing totals
in the leaf layer, the new total of leaf tuples,  and tree params
to calculate how the layer after the leaf layer should be updated.
2b. For each subsequent layer, call the function
get_update_layer_metadata. This function uses the existing totals
in the *prev* layer, the new total of children in the *prev* layer,
and tree params to calculate how the layer should be updated.

3. Grow the tree using the tree extension.

This approach isolates update logic and actual hashing into neat
structured functions, rather than mix the two. This makes the code
easier to follow without needing to keep so much in your head at
one time.
2024-07-29 03:40:44 -07:00
j-berman
8287ba6f78 faster trim_tree tests 2024-07-29 03:40:44 -07:00
j-berman
ed040cacc1 implement trim_tree_in_memory 2024-07-29 03:40:44 -07:00
j-berman
e8af7090b0 expose and test hash_trim from rust lib 2024-07-29 03:40:44 -07:00
j-berman
ae89fddc00 Set up trim_tree_in_memory test 2024-07-29 03:40:44 -07:00
j-berman
c7c6c6afff CurveTreesUnitTest -> CurveTreesGlobalTree class 2024-07-29 03:40:44 -07:00
j-berman
17b1f421c0 cleaner lmdb test structure for curve trees 2024-07-29 03:40:44 -07:00
j-berman
6045357f6a implement db->audit_tree, and flesh out db test to init/extend tree 2024-07-29 03:40:44 -07:00
j-berman
1ffde095c8 Implement get_tree_last_chunks in db, can now extend tree in db 2024-07-29 03:40:44 -07:00
j-berman
ab7c74136b Simplify edge case handling in hash_layer
- When retrieving last chunks, set next_start_child_chunk_index
so can know the correct start index without needing to modify
the offset
- Other smaller cleanup
2024-07-29 03:40:44 -07:00
Luke Parker
988c4eae40 Remove reference from m_hash_init_point
Identified by kayabaNerve, patch suggested by j-berman.
2024-07-29 03:40:44 -07:00
j-berman
a1ee603132 explicit type response to hash_grow 's 2024-07-29 03:40:44 -07:00
j-berman
517f5a301e don't want to expose generator lengths in ffi 2024-07-29 03:40:44 -07:00
Luke Parker
effa9eea0e Only pass a single prior child 2024-07-29 03:40:44 -07:00
Luke Parker
c792b21535 Use statics on the Rust side for generators 2024-07-29 03:40:44 -07:00
Luke Parker
42f6ef273d fmt, clippy 2024-07-29 03:40:44 -07:00
Luke Parker
af47a135eb Remove cxx and expose scalars/points directly 2024-07-29 03:40:44 -07:00
j-berman
af9b74fcc7 start LMDB grow_tree 2024-07-29 03:40:44 -07:00
j-berman
e68ea2e054 small cleanup 2024-07-29 03:40:44 -07:00
j-berman
5ad026975a Cleaner template usage, moved static functions out of CurveTrees class 2024-07-29 03:40:44 -07:00
j-berman
9ba00be519 Move curve_trees.h implementations into curve_trees.cpp file 2024-07-29 03:40:44 -07:00
j-berman
4ade675939 Consolidate hash_leaf_layer into hash_layer 2024-07-29 03:40:44 -07:00
j-berman
26009ba5f1 slight simplification to CurveTrees::hash_layer 2024-07-29 03:40:44 -07:00
j-berman
9e68475ebd Use widths from fcmp++ repo test & align tests with width 2024-07-29 03:40:44 -07:00
j-berman
29e0fe759e Add Curve class, and Helios & Selene classes that derive from Curve 2024-07-29 03:40:44 -07:00
j-berman
d9390c7b08 Implement CurveTrees & CurveTreesUnitTest classes to simplify callers 2024-07-29 03:40:44 -07:00
j-berman
5103a94ee9 template all curve_trees types & funcs, rename tower_cycle_types to tower_cycle 2024-07-29 03:40:44 -07:00
j-berman
ad8872a76b Cleaner file organization 2024-07-29 03:40:44 -07:00
j-berman
c05bd80ee5 actual indexing fix, tests now passing 2024-07-29 03:40:44 -07:00
j-berman
33ad50b176 fix c1 c2 layer indexing issue in test helper get_last_chunk 2024-07-29 03:40:44 -07:00
j-berman
9e6b93b94d test validates lowest layer in tree 2024-07-29 03:40:44 -07:00
j-berman
1ba876bcc2 remove whitespaces 2024-07-29 03:40:44 -07:00
j-berman
e1c03f4d5a rough fcmp++ tree impl (lots of work remaining to clean it up and fix) 2024-07-29 03:40:44 -07:00
luigi1111
caa62bc9ea
Merge pull request #9392
b894042 epee: fix mlog filename compare bug. (0xFFFC0000)
2024-07-16 19:02:24 -04:00
luigi1111
3b53db47cf
Merge pull request #9386
1d024f0 chore: fix some comments (haouvw)
2024-07-16 19:01:09 -04:00
luigi1111
330212888a
Merge pull request #9383
9f83e74 Daemon RPC: /getblocks.bin return chain's top_block_hash in resp (j-berman)
2024-07-16 19:00:39 -04:00
luigi1111
4ed5bc5436
Merge pull request #9382
356829a Daemon RPC: high_height_ok req boolean field /getblocks.bin (j-berman)
2024-07-16 19:00:06 -04:00
luigi1111
8dbbc47804
Merge pull request #9379
6c28bdb wallet2: move get_rpc_status func to RPC server definition file (j-berman)
2024-07-16 18:58:08 -04:00
luigi1111
cfd7f5222d
Merge pull request #9370
9fc48f4 Fix ZMQ Tx Pruning (Lee Clagett)
2024-07-16 18:56:44 -04:00
luigi1111
66c5917876
Merge pull request #9353
32f3245 cryptonote_protocol: prevent duplicate txs in fluff queue (0xFFFC0000)
2024-07-16 18:51:30 -04:00
luigi1111
c830fc05d8
Merge pull request #9349
9817293 workflows: don't hardcode repo name (selsta)
2024-07-16 18:50:40 -04:00
luigi1111
e3cfaed278
Merge pull request #9346
0fad1a8 src: update checkpoints to match v0.18.3.4 (selsta)
2024-07-16 18:49:44 -04:00
luigi1111
c2fceb2939
Merge pull request #9344
a026d5a wallet2: validate fetched block height and parent hash (Oscar Mira)
c634c26 wallet2: use start_height consistently as const in process_parsed_blocks (Oscar Mira)
2024-07-16 18:48:43 -04:00
luigi1111
baf6df06c2
Merge pull request #9343
302fa9b wallet: fetch pool txs in pruned form (jeffro256)
2024-07-16 18:46:04 -04:00
luigi1111
b02bf31b83
Merge pull request #9342
b07a97c epee: partially revert c56ee140 to fix linking errors (jeffro256)
2024-07-16 18:44:04 -04:00
luigi1111
040bccc22f
Merge pull request #9338
4a376d6 copyright: fix vector.h typo (hinto.janaiyo)
d61e8e9 workflows: delete copyright.yml (plowsof)
341771a copyright: bump to 2024 (copyCat)
2024-07-16 18:42:49 -04:00
luigi1111
2a6b6ea0f2
Merge pull request #9326
004ead1 gpg_keys: renew jeffro256 (jeffro256)
2024-07-16 18:41:57 -04:00
luigi1111
978842c284
Merge pull request #9324
2e061be README.md: sponsors;remove forked networking/globee (plowsof)
2024-07-16 18:41:20 -04:00
luigi1111
85cf0b01fa
Merge pull request #9289
025021d BlockchainDB: shorter fixup() (jeffro256)
2024-07-16 18:40:46 -04:00
luigi1111
99e33996f4
Merge pull request #9288
262ef47 BlockchainDB: remove lock/unlock methods (jeffro256)
2024-07-16 18:32:42 -04:00