* replace bitvec with more efficient bitpack algorithm
* optimise proof_unpack_len
* move proof pack length calculation
* small refactor
* first pass attempt at not deserializing proof nonces in difficulty iter
* another 10 seconds gained by not deserialising the proof from the difficulty iterator
* add new deser parameters to tests where needed
* add skip_proof variants to store
* remove hash from difficulty iterator struct, rename HeaderInfo to HeaderDifficultyInfo
* replace bitvec with more efficient bitpack algorithm
* optimise proof_unpack_len
* move proof pack length calculation
* small refactor
* first pass attempt at not deserializing proof nonces in difficulty iter
* another 10 seconds gained by not deserialising the proof from the difficulty iterator
* add new deser parameters to tests where needed
* add skip_proof variants to store
* remove hash from difficulty iterator struct, rename HeaderInfo to HeaderDifficultyInfo
* initial commit of WIP pibd explorations
* correct calling for obtaining and validating first segment
* update test to properly iterate through each segment of the test pmmrs, validating each segment as it goes
* updated test to fully segment and validate PMMRs from compacted and uncompacted sample data. Also contains method of running test againt live chain data
* remove logger change
* change test file name
* change test file name
* change directory reference in test for CI
* add initial (experimental) structure for PIBD desegmenting
* move bitmap desegmentation logic into desegmenter
* added txhashset methods to apply pibd segments (note this only works for fully unpruned trees atm)
* change last_pos to mmr_size
* fix to pmmr::peaks call
* don't verify POW when copying headers
* prepare for commit of work thus far'
* update test paths
* few updates based on early review
* use 0-based positions in methods pmmr_leaf_to_insertion_index and bintree_postorder_height; add round_up_to_leaf_pos method
* use 0-based positions in method insertion_to_pmmr_index
* use 0-based positions in method is_leaf
* use 0-based positions in method family()
* use 0-based positions in method is_left_sibling
* use 0-based positions in method family_branch
* use 0-based positions in methods bintree_{left,right}most
* use 0-based positions in method bintree_pos_iter
* use 0-based positions in method bintree_range
* use 0-based positions in method bintree_leaf_pos_iter
* rename last_pos in MMR related structs to size
* use 0-based positions in method prune
* use 0-based positions in method push and apply_output return value
* use 0-based position argument of method merkle_proof
* use 0-based outputs in method pmmr::peaks
* fix peaks() code comments
* refix peaks() code comments
* use 0-based positions in method get_peak_from_file
* use 0-based positions in methods get_data_from_file
* use 0-based positions in methods get_from_file
* use 0-based positions in methods get_data
* use 0-based positions in methods get_hash
* use 0-based positions in method peak_path
* use 0-based positions in method bag_the_rhs
* use 0-based positions in method Backend::remove
* use 0-based positions in method leaf_pos_iter
* use 0-based positions in method self.LeafSet::includes
* use 0-based positions in methods self.LeafSet::{add,remove}
* use 0-based positions in methods is_pruned,is_pruned_root,is_compacted
* use 0-based positions in methods PruneList::append
* use 0-based positions in methods append_pruned_subtree
* use 0-based positions in method calculate_next_leaf_shift
* use 0-based positions in method append_single
* use 0-based positions in method calculate_next_shift
* use 0-based positions in method segment_pos_range
* use 0-based positions in method reconstruct_root
* use 0-based positions in method validate_with
* use 0-based positions in method validate
* rename size (formerly last_pos) to mmr_size
* use 0-based positions in Segment's hash_pos and leaf_pos
* minimize use of saturating_sub(1) and rename some pos/idx to size
* use 0-based positions in methods get_output_pos
* use 0-based positions in method get_unspent_output_at
* use 0-based positions in method get_header_hash
* use 0-based positions in methods MerkleProof::verify{,_consume}
* use 0-based positions in method cleanup_subtree
* don't allow 0 in prunelist bitmap
* use 0-based positions in methods get_{,leaf_}shift
* rename some 1-based pos to pos1; identify TODO
* Address yeastplume's PR review comments
* initial commit of WIP pibd explorations
* correct calling for obtaining and validating first segment
* update test to properly iterate through each segment of the test pmmrs, validating each segment as it goes
* updated test to fully segment and validate PMMRs from compacted and uncompacted sample data. Also contains method of running test againt live chain data
* remove logger change
* change test file name
* change test file name
* change directory reference in test for CI
* remove +1 from segment count
* prediction comment change
* add ignore of full-chain test
* refactor prune_list with aim of allowing pruned subtree appending
* add test coverage around pmmr::is_leaf() and pmmr::bintree_leaf_pos_iter()
* comments
* cleanup
* implement append pruned subtree for prune_list
* commit
* we can now append to prune_list
* fix our prune_list corruption...
* rework how we rewrite the prune list during compaction
* test coverage for improved prune list api
* continuing to merge
* finish merge, tests passing again
* add function pmmr_leaf_to_insertion_index, and modify bintree_lef_pos_iter to use it. Note there's still an unwrap that needs to be dealt with sanely
* change pmmr_leaf_to_insertion_index to simpler version + handle conversion between 1 and 0 based in bintree_leaf_pos_iter
Co-authored-by: antiochp <30642645+antiochp@users.noreply.github.com>
* fix for comparing scaled vs unscaled difficulty
fix mix use of scaled and unscaled, add additional comments to clarify which values are which
improved consistency - scaled and unscaled difficulty
* combine from_proof_unscaled() and to_unscaled_difficulty{}
* code review - rust syntax
* add FeeFields type
* use FeeFields with ::zero and try_into().unwrap()
* fixed tests
* avoid 0 accept_base_fee
* add aggregate_fee_fields method for transaction
* implement std::fmt::Display trait for FeeFields
* make base_fee argument non-optional in libtx::mod::tx_fee
* add global and thread local accept_fee_base; use to simplify tests
* set unusually high fee base for a change
* revert to optional base fee argument; default coming from either grin-{server,wallet}.toml
* remove optional base fee argument; can be set with global::set_local_accept_fee_base instead
* define constant global::DEFAULT_ACCEPT_FEE_BASE and set global value
* add Transaction::accept_fee() method and use
* Manual (de)ser impl on FeeFields
* fix comment bug
* Serialize FeeFields as int in tx
* allow feefields: u32:into() for tests
* try adding height args everywhere
* make FeeFields shift/fee methods height dependent
* prior to hf4 feefield testing
* rename selected fee_fields back to fee for serialization compatibility
* fix test_fee_fields test, merge conflict, and doctest use of obsolete fee_fields
* make accept_fee height dependent
* Accept any u64 in FeeFields deser
Co-authored-by: Jasper van der Maarel <j@sper.dev>
* make min_wtema difficulty chain type dependent
* fix bad graph weights for automated and user testing
* only mainnet can get wtema difficulty unstuck at 59s blocks
* HardFork4 initial commit
* HF4 obsoletes secondary_scale, which becomes an nonce extension 8->12 bytes
* fix TESTNET_FOURTH_HARD_FORK height
* improve predicted height for testnet fork on dec 8
* move local variable as suggested by antiochp
* More efficient serialization for bitmap segments
* Rename a const
* Correctly count number of chunks in a segment
* Enum for BitmapBlock (de)ser mode
* Add more segments in test
* Fix duplicate function
* add segmenter for generating segments from txhashset with consistent rewind
* rework segmenter to take a txhashset wrapped in rwlock
rework our rewindable pmmr so we can convert to readonly easily
* placeholder code for rewinding readonly txhashset extension to build a rangeproof segment
* segment creation for outputs/rangeproofs/kernels/bitmaps
* placeholder segment impl
* commit
* rework segmenter to use a cached bitmap (rewind is expensive)
* cache segmenter instance based on current archive header
* integrate the real segment and segment identifier with our segmenter
* exercise the segmenter code on chain init
* wrap accumulator in an arc, no need to clone each time
* Chunk generation and validation
* Rename chunk -> segment
* Missed a few
* Generate and validate merkle proof
* Fix bugs in generation and validation
* Add test for unprunable MMR of various sizes
* Add missing docs
* Remove unused functions
* Remove segment error variant on chain error type
* Simplify calculation by using a Vec instead of HashMap
* Use vectors in segment definition
* Compare subtree root during tests
* Add test of segments for a prunable mmr
* Remove assertion
* Only send intermediary hashes for prunable MMRs
* Get hash from file directly
* Require both leaves if one of them is not pruned
* More pruning tests
* Add segment (de)serialization
* Require sorted vectors in segment deser
* Store pos and data separately in segment
* Rename log_size -> height
* Fix bitmap index in root calculation
* Add validation function for output (bitmap) MMRs
* Remove left over debug statements
* Fix test
* Edge case: final segment with uneven number of leaves
* Use last_pos instead of segment_last_pos
* Simplify pruning in test
* Add leaf and hash iterators
* Support fully pruned segments
* Drop backend before deleting dir in pruned_segment test
* Simplify output of first_unpruned_parent
* Introduce CommitOnly variant of Inputs.
Introduce CommitWrapper so we can sort commit only inputs correctly.
* rememebr to resort if converting
* write inputs based on variant and protocol version
* read and write protocol version specific inputs
* store full blocks in local db in v3
convert to v2 when relaying to v2 peers
* add debug version_str for inputs
* no assumptions about spent index sort order
* add additional version debug logs
* fix ser/deser tests for proto v3
* cleanup coinbase maturity
* rework pool to better handle v2 conversion robustly
* cleanup txpool add_to_pool
* fix nrd kernel test
* move init conversion earlier
* cleanup
* cleanup based on PR feedback
