* use 0-based positions in methods pmmr_leaf_to_insertion_index and bintree_postorder_height; add round_up_to_leaf_pos method
* use 0-based positions in method insertion_to_pmmr_index
* use 0-based positions in method is_leaf
* use 0-based positions in method family()
* use 0-based positions in method is_left_sibling
* use 0-based positions in method family_branch
* use 0-based positions in methods bintree_{left,right}most
* use 0-based positions in method bintree_pos_iter
* use 0-based positions in method bintree_range
* use 0-based positions in method bintree_leaf_pos_iter
* rename last_pos in MMR related structs to size
* use 0-based positions in method prune
* use 0-based positions in method push and apply_output return value
* use 0-based position argument of method merkle_proof
* use 0-based outputs in method pmmr::peaks
* fix peaks() code comments
* refix peaks() code comments
* use 0-based positions in method get_peak_from_file
* use 0-based positions in methods get_data_from_file
* use 0-based positions in methods get_from_file
* use 0-based positions in methods get_data
* use 0-based positions in methods get_hash
* use 0-based positions in method peak_path
* use 0-based positions in method bag_the_rhs
* use 0-based positions in method Backend::remove
* use 0-based positions in method leaf_pos_iter
* use 0-based positions in method self.LeafSet::includes
* use 0-based positions in methods self.LeafSet::{add,remove}
* use 0-based positions in methods is_pruned,is_pruned_root,is_compacted
* use 0-based positions in methods PruneList::append
* use 0-based positions in methods append_pruned_subtree
* use 0-based positions in method calculate_next_leaf_shift
* use 0-based positions in method append_single
* use 0-based positions in method calculate_next_shift
* use 0-based positions in method segment_pos_range
* use 0-based positions in method reconstruct_root
* use 0-based positions in method validate_with
* use 0-based positions in method validate
* rename size (formerly last_pos) to mmr_size
* use 0-based positions in Segment's hash_pos and leaf_pos
* minimize use of saturating_sub(1) and rename some pos/idx to size
* use 0-based positions in methods get_output_pos
* use 0-based positions in method get_unspent_output_at
* use 0-based positions in method get_header_hash
* use 0-based positions in methods MerkleProof::verify{,_consume}
* use 0-based positions in method cleanup_subtree
* don't allow 0 in prunelist bitmap
* use 0-based positions in methods get_{,leaf_}shift
* rename some 1-based pos to pos1; identify TODO
* Address yeastplume's PR review comments
* refactor prune_list with aim of allowing pruned subtree appending
* add test coverage around pmmr::is_leaf() and pmmr::bintree_leaf_pos_iter()
* comments
* cleanup
* implement append pruned subtree for prune_list
* commit
* we can now append to prune_list
* fix our prune_list corruption...
* rework how we rewrite the prune list during compaction
* test coverage for improved prune list api
* continuing to merge
* finish merge, tests passing again
* add function pmmr_leaf_to_insertion_index, and modify bintree_lef_pos_iter to use it. Note there's still an unwrap that needs to be dealt with sanely
* change pmmr_leaf_to_insertion_index to simpler version + handle conversion between 1 and 0 based in bintree_leaf_pos_iter
Co-authored-by: antiochp <30642645+antiochp@users.noreply.github.com>
We have a pattern in the code - return Vec, turn it into an iterator, filter and collect to another Vec. The idea was to return iterator where possible to avoid allocating intermediate vecs.
* add bitmap accumulator
refactor vec backend so we can use it outside of tests
introduce a "hash only" vec backend for the accumulator
* get core tests passing
* initial test coverage for bitmap_accumulator
* better test coverage for bitmap accumulator and cleanup code
* refactor txhashset roots, call validate() on roots during block validation
* fix store tests
* log the "merged" root when validating roots
* cleanup, revise based on feedback
* cleanup
* rework it to pass explicit size into bitmap accumulator when applying
* add function to retrieve a set of pmmr indices between a given block height range
* typo
* change APU to just return required indices
* change pmmr index retrieval, change new function to only return pmmr indices between blocks
* header MMR in use within txhashset itself
works with fast sync
not yet in place for initial header sync
* add the (currently unused) sync_head mmr
* use sync MMR during fast sync
rebuild header MMR after we validate full txhashset after download
* support missing header MMR (rebuild as necessary) for legacy nodes
* rustfmt
* comments/docs
* rustfmt
* cleanup DBBackend
* cleanup DBBackend
* cleanup
* rename to HashOnly
* rustfmt
* cleanup backend.append()
* simply pmmr append api
no need to pass position when appending
* cleanup
* simplify vec_backend to match simpler append api
* rustfmt
* docs/comments
* rustfmt
* cleanup
* Include commitments non-duplicate checks in aggregate
* Remove said check from the pool
* Block building now uses tx aggregation to reduce duplication
* add new core mmr routine upon which others should be built more efficiently
* tons of pmmr optimizations and simplifications
* fix typo and extend docs
* change initial spaces to tab to fix indentation
* implement prune_list as a bitmap
and simplify
* cleanup prune_list, use maximum()
* handle migration of prune_list to new bitmap prun file
* legacy filename consts
* cleanup and docs
* only store leaves in rm_log
* cleanup
* commit
* add failing test to cover case where we compact
an already compacted data file
* fix the logic for pruning the data file
* rm_log only contains leaves
prunelist only contains non-leaf subtree roots
* cleanup
* commit
* bitmap impl running in parallel
* rustfmt
* this is better - rewind unremoves outpu pos spent by rewound inputs
* commit
* commit
* pass bitmap around when rewinding
* store and retrieve input bitmap per block from the db
* Replace the RemoveLog with the UTXO set.
* utxo set starting to pass tests
* stuff works...
* split store types out
added some tests to compare the performance of the rm_log and the proposed utxo_set
* pull prune_list out into standalone file
* cleanup, get rid of unused height param
* cleanup and add doc comments
* add support for migrating rm_log to utxo_set
* take snapshot of utxo file during fast sync
implement migration of rm_log -> utxo_set
* rename rewound_copy to snapshot
* fixup pmmr tests to reflect cutoff_pos
* cleanup unused import
* check_compact needs to rewind the utxo_set as appropriate
* fix pool tests
* fixup core tests
* cache block_input_bitmaps via LruCache in store
* cache block header on initial write to db
* rename utxo_set -> leaf_set
and remove references to "spent" in grin_store
* better document the rewind behavior
