Howard Chu
b2f1c58805
Use cursors in some remove functions
...
Helps when they're called repeatedly in one txn
2016-04-05 20:57:31 +01:00
Howard Chu
118dd69dd5
Use DUPFIXED for block_info and output_txs
...
Saves another ~150MB or so on the full blockchain
2016-04-05 20:55:16 +01:00
Howard Chu
6225716f3c
More outputs consolidation
...
Also bumped DB VERSION to 1
Another significant speedup and space savings:
Get rid of global_output_indices, remove indirection from output to keys
This is the change warptangent described on irc but never got to finish.
2016-04-05 20:55:12 +01:00
Howard Chu
7c5abdc3a3
Use DUPFIXED for output_keys
...
Saves another 90MB on 200000 block import.
Had to bring back compare_uint64 for this, but it's safe since
this table is always 64-bit aligned.
2016-04-05 20:54:42 +01:00
Howard Chu
8e9d8e3364
Use DUPFIXED for tx_indices
...
Small space savings, no measurable speedup
2016-04-05 20:54:39 +01:00
Howard Chu
a12f9365f8
Use DUPFIXED for block_heights
...
Only a small savings...
2016-04-05 20:54:07 +01:00
Howard Chu
38c2277d6f
Use DUPFIXED for spent_keys
2016-04-05 20:54:07 +01:00
warptangent
bf769c32ba
Add back changes from revert.
...
m_tx_outputs doesn't need to be changed, as it's no longer dup list.
2016-04-05 20:54:07 +01:00
warptangent
46b991b362
Use MDB_APPEND mode with two tx subdbs
...
This is possible on those using a tx index as a key.
2016-04-05 20:54:07 +01:00
warptangent
9aadedb1d0
Schema update: tx_indices - consolidate the tx subdbs from 5 to 3
2016-04-05 20:54:06 +01:00
warptangent
a2f518aa01
Schema update: tx_indices - yet less indirection
2016-04-05 20:54:06 +01:00
warptangent
8d12a8df2c
Schema update: tx_indices - improve further with less indirection
2016-04-05 20:54:06 +01:00
warptangent
7c013f66e9
Add batch warning for further review
2016-04-05 20:54:06 +01:00
warptangent
ae0854a431
Schema update: tx_indices
2016-04-05 20:54:06 +01:00
Howard Chu
8d252a4214
Consolidated block info
2016-04-05 20:53:59 +01:00
warptangent
132c666f67
Update schema for "tx_outputs" to use array containing amount output indices
...
This speeds up wallet refresh by directly retrieving a tx's amount output indices.
It removes the indirection and walking the amount output duplicate list
for every amount in each requested tx.
"tx_outputs" is used by:
Amount output indices are needed for wallet refresh.
Global output indices are needed for removing a tx.
Both amount output indices and global output indices are now stored in
an array of 64-bit unsigned ints:
tx_outputs[<tx_hash>] -> [ <a1_oi, a1_gi, a2_oi, a2_gi, ...> ]
Previously it was:
tx_outputs[<tx_hash>] -> duplicate list of <a1_gi, a2_gi, a3_gi, ...>
The amount output list had to be walked for every amount in order to
find each amount's output index, by comparing the amount's global output
index with each one in the duplicate list until a match was found.
See also d045dfa7ce
2016-04-05 20:30:50 +01:00
Riccardo Spagni
32dc08d19a
Merge pull request #700
...
bdec7cbc5932eba49c355
2016-03-05 23:10:38 +02:00
Riccardo Spagni
c3af15702f
Merge pull request #698
...
ee7a8b8
2016-03-05 23:06:26 +02:00
warptangent
bdec7cbfb3
BlockchainLMDB: Use DB error helper consistently
2016-03-03 21:27:13 -08:00
warptangent
c5932eb5ea
BlockchainLMDB: Add DB error to exception
2016-03-03 20:15:37 -08:00
Howard Chu
ee7a8b87f9
Get rid of lmdb_cur
...
We don't need it now with per-txn cursors.
2016-03-03 19:06:27 +00:00
Howard Chu
f5affbef26
More for df239428c0
...
Let ARMv7 work again
2016-03-03 00:13:13 +00:00
warptangent
2c823c1589
Make partial revert consistent
...
If user-defined comparator is used, subdb shouldn't be opened with
MDB_INTEGERKEY.
TODO: Again, this will be added back with future schema updates.
2016-03-01 02:47:16 -08:00
warptangent
df239428c0
Revert part of 7db89ed2ee
...
For now, so existing databases work.
TODO: add these back with future schema updates.
2016-02-29 18:03:41 -08:00
warptangent
c9c4060dc6
BlockchainLMDB: Add height and db error to exception
2016-02-29 17:52:54 -08:00
Riccardo Spagni
d60bf4ee36
Merge pull request #686
...
7db89ed5a07cef8cc7a3686a7f2bea5fa5e
2016-02-29 17:29:26 -05:00
Howard Chu
7db89ed2ee
ARMv7: fix unaligned accesses
...
And cleanup some key comparators
2016-02-25 13:57:00 +00:00
Howard Chu
8cc7a36f0b
read txn/cursor stuff
...
Could wrap more later.
2016-02-23 20:47:15 +00:00
moneromooo-monero
0eb4d054f6
db_bdb: add missing include for std::condition_variable
2016-02-20 15:38:43 +00:00
warptangent
57e75fa33f
BlockchainBDB: Check if hard fork subdbs need reset
...
See f7e337e625
2016-02-18 06:38:56 -08:00
warptangent
47f6cf8bea
BlockchainBDB: Support blockchain_import --drop-hard-fork command
...
See c657e772c4
2016-02-18 06:37:45 -08:00
Howard Chu
02abe3590d
Use MDB_PREV_MULTIPLE
...
in get_global_output_indices
2016-02-17 16:14:50 +00:00
Howard Chu
7a4755d3a4
Fixup after lmdb master resync
2016-02-17 16:14:38 +00:00
Howard Chu
3b13a7473e
Shutup about VERSION 0
2016-02-17 05:06:19 +00:00
Howard Chu
1537477c9f
Use cursor in get_output_key
2016-02-17 04:05:29 +00:00
Howard Chu
f2faf8cdd9
Use MDB_APPEND mode where possible
...
When keys are contiguous and monotonically increasing, this gets
denser page utilization (doesn't leave padding in page splits).
Can't be used for keys that are inserted in random order (e.g. hashes)
In total this only saves around 1.5% of space compared to original
DB code. The previous patch accounted for 0.8% savings on its own;
the blocks tables just aren't that big.
2016-02-17 04:05:29 +00:00
Howard Chu
090b548c3b
Use cursors in write txns
...
Saves a bit of seek overhead. LMDB frees them automatically
in txn_(commit|abort) so they need no cleanup.
2016-02-17 04:05:29 +00:00
Howard Chu
ed08d2152e
Keep a running blocksize count
...
Used in batch size estimation, avoids rereading already processed
blocks during import
2016-02-17 04:05:28 +00:00
Howard Chu
0fc93345ea
Win32 import batchsize tweaks
...
Reduce frequency of resizes: bump minimum increase from 128MB to 512MB
Use a bigger safety margin at small batch sizes
2016-02-17 04:05:28 +00:00
Riccardo Spagni
63e2d4195b
Merge pull request #657
...
1995923c16cc209118d0af5581c3
2016-02-13 19:54:47 +02:00
warptangent
1995923559
BlockchainLMDB: Deal with DB exceptions at block level with particularity
...
Add another DB error exception type to distinguish failed txn setup from
general use of txn.
This keeps the error handling flow the same as before the block-level
txn setup changes that moved control up a layer to BlockchainDB.
2016-02-13 05:12:39 -08:00
warptangent
c16cc204d3
BlockchainLMDB: Add sanity check for inconsistent state
...
This hasn't been known to occur in block-level txn abort, but throw
exception if it does.
2016-02-13 04:17:37 -08:00
warptangent
9118d0a44c
BlockchainLMDB: Call destructor on allocated txn if setup fails
2016-02-13 04:17:37 -08:00
warptangent
f5581c3536
BlockchainLMDB: Replace remaining txn pointer NULLs with nullptr
...
For consistency.
2016-02-13 04:17:36 -08:00
moneromooo-monero
8649b9f1ef
blockchain_db: pass hard fork object as a simple pointer
2016-02-08 20:56:31 +00:00
warptangent
e02577f594
Move HardFork DB update to BlockchainDB::add_block()
...
Ensures the database is consistent.
Also simplifes blockchain_import in that verify mode off has less to
work around.
2016-02-08 09:28:19 -08:00
warptangent
3800875406
Make HardFork object available to BlockchainDB and derived DB implementations
...
This will later allow the HardFork object's DB update functions to be
called when the DB transaction that persists across block add/remove is
open.
2016-02-08 09:28:17 -08:00
warptangent
fd46c96dce
BlockchainDB/LMDB: Refactor block-scope DB txn handling for add block
...
Move block-scope txn start and stop from BlockchainLMDB to BlockchainDB.
2016-02-08 09:28:15 -08:00
warptangent
f3a6000094
BlockchainDB/LMDB/BDB: Extract DB txn functions for block add/remove
2016-02-08 09:28:14 -08:00
warptangent
f47d5b0fe3
BlockchainLMDB: Allow two HardFork functions to update DB during block add
...
Note that this doesn't yet cause them to be called during block add.
2016-02-08 09:28:12 -08:00
