* Update versioning on master to 5.4.0-alpha.0
* updates for 1.80 and other accumulated warnings
* further warning cleanups
* move dead code tag to function defn rather than module
* replace bitvec with more efficient bitpack algorithm
* optimise proof_unpack_len
* move proof pack length calculation
* small refactor
* first pass attempt at not deserializing proof nonces in difficulty iter
* another 10 seconds gained by not deserialising the proof from the difficulty iterator
* add new deser parameters to tests where needed
* add skip_proof variants to store
* remove hash from difficulty iterator struct, rename HeaderInfo to HeaderDifficultyInfo
* replace bitvec with more efficient bitpack algorithm
* optimise proof_unpack_len
* move proof pack length calculation
* small refactor
* first pass attempt at not deserializing proof nonces in difficulty iter
* another 10 seconds gained by not deserialising the proof from the difficulty iterator
* add new deser parameters to tests where needed
* add skip_proof variants to store
* remove hash from difficulty iterator struct, rename HeaderInfo to HeaderDifficultyInfo
* use 0-based positions in methods pmmr_leaf_to_insertion_index and bintree_postorder_height; add round_up_to_leaf_pos method
* use 0-based positions in method insertion_to_pmmr_index
* use 0-based positions in method is_leaf
* use 0-based positions in method family()
* use 0-based positions in method is_left_sibling
* use 0-based positions in method family_branch
* use 0-based positions in methods bintree_{left,right}most
* use 0-based positions in method bintree_pos_iter
* use 0-based positions in method bintree_range
* use 0-based positions in method bintree_leaf_pos_iter
* rename last_pos in MMR related structs to size
* use 0-based positions in method prune
* use 0-based positions in method push and apply_output return value
* use 0-based position argument of method merkle_proof
* use 0-based outputs in method pmmr::peaks
* fix peaks() code comments
* refix peaks() code comments
* use 0-based positions in method get_peak_from_file
* use 0-based positions in methods get_data_from_file
* use 0-based positions in methods get_from_file
* use 0-based positions in methods get_data
* use 0-based positions in methods get_hash
* use 0-based positions in method peak_path
* use 0-based positions in method bag_the_rhs
* use 0-based positions in method Backend::remove
* use 0-based positions in method leaf_pos_iter
* use 0-based positions in method self.LeafSet::includes
* use 0-based positions in methods self.LeafSet::{add,remove}
* use 0-based positions in methods is_pruned,is_pruned_root,is_compacted
* use 0-based positions in methods PruneList::append
* use 0-based positions in methods append_pruned_subtree
* use 0-based positions in method calculate_next_leaf_shift
* use 0-based positions in method append_single
* use 0-based positions in method calculate_next_shift
* use 0-based positions in method segment_pos_range
* use 0-based positions in method reconstruct_root
* use 0-based positions in method validate_with
* use 0-based positions in method validate
* rename size (formerly last_pos) to mmr_size
* use 0-based positions in Segment's hash_pos and leaf_pos
* minimize use of saturating_sub(1) and rename some pos/idx to size
* use 0-based positions in methods get_output_pos
* use 0-based positions in method get_unspent_output_at
* use 0-based positions in method get_header_hash
* use 0-based positions in methods MerkleProof::verify{,_consume}
* use 0-based positions in method cleanup_subtree
* don't allow 0 in prunelist bitmap
* use 0-based positions in methods get_{,leaf_}shift
* rename some 1-based pos to pos1; identify TODO
* Address yeastplume's PR review comments
* refactor prune_list with aim of allowing pruned subtree appending
* add test coverage around pmmr::is_leaf() and pmmr::bintree_leaf_pos_iter()
* comments
* cleanup
* implement append pruned subtree for prune_list
* commit
* we can now append to prune_list
* fix our prune_list corruption...
* rework how we rewrite the prune list during compaction
* test coverage for improved prune list api
* continuing to merge
* finish merge, tests passing again
* add function pmmr_leaf_to_insertion_index, and modify bintree_lef_pos_iter to use it. Note there's still an unwrap that needs to be dealt with sanely
* change pmmr_leaf_to_insertion_index to simpler version + handle conversion between 1 and 0 based in bintree_leaf_pos_iter
Co-authored-by: antiochp <30642645+antiochp@users.noreply.github.com>
* add FeeFields type
* use FeeFields with ::zero and try_into().unwrap()
* fixed tests
* avoid 0 accept_base_fee
* add aggregate_fee_fields method for transaction
* implement std::fmt::Display trait for FeeFields
* make base_fee argument non-optional in libtx::mod::tx_fee
* add global and thread local accept_fee_base; use to simplify tests
* set unusually high fee base for a change
* revert to optional base fee argument; default coming from either grin-{server,wallet}.toml
* remove optional base fee argument; can be set with global::set_local_accept_fee_base instead
* define constant global::DEFAULT_ACCEPT_FEE_BASE and set global value
* add Transaction::accept_fee() method and use
* Manual (de)ser impl on FeeFields
* fix comment bug
* Serialize FeeFields as int in tx
* allow feefields: u32:into() for tests
* try adding height args everywhere
* make FeeFields shift/fee methods height dependent
* prior to hf4 feefield testing
* rename selected fee_fields back to fee for serialization compatibility
* fix test_fee_fields test, merge conflict, and doctest use of obsolete fee_fields
* make accept_fee height dependent
* Accept any u64 in FeeFields deser
Co-authored-by: Jasper van der Maarel <j@sper.dev>
* make min_wtema difficulty chain type dependent
* fix bad graph weights for automated and user testing
* only mainnet can get wtema difficulty unstuck at 59s blocks
* HardFork4 initial commit
* HF4 obsoletes secondary_scale, which becomes an nonce extension 8->12 bytes
* fix TESTNET_FOURTH_HARD_FORK height
* improve predicted height for testnet fork on dec 8
* move local variable as suggested by antiochp
* Chunk generation and validation
* Rename chunk -> segment
* Missed a few
* Generate and validate merkle proof
* Fix bugs in generation and validation
* Add test for unprunable MMR of various sizes
* Add missing docs
* Remove unused functions
* Remove segment error variant on chain error type
* Simplify calculation by using a Vec instead of HashMap
* Use vectors in segment definition
* Compare subtree root during tests
* Add test of segments for a prunable mmr
* Remove assertion
* Only send intermediary hashes for prunable MMRs
* Get hash from file directly
* Require both leaves if one of them is not pruned
* More pruning tests
* Add segment (de)serialization
* Require sorted vectors in segment deser
* Store pos and data separately in segment
* Rename log_size -> height
* Fix bitmap index in root calculation
* Add validation function for output (bitmap) MMRs
* Remove left over debug statements
* Fix test
* Edge case: final segment with uneven number of leaves
* Use last_pos instead of segment_last_pos
* Simplify pruning in test
* Add leaf and hash iterators
* Support fully pruned segments
* Drop backend before deleting dir in pruned_segment test
* Simplify output of first_unpruned_parent
* Introduce CommitOnly variant of Inputs.
Introduce CommitWrapper so we can sort commit only inputs correctly.
* rememebr to resort if converting
* write inputs based on variant and protocol version
* read and write protocol version specific inputs
* store full blocks in local db in v3
convert to v2 when relaying to v2 peers
* add debug version_str for inputs
* no assumptions about spent index sort order
* add additional version debug logs
* fix ser/deser tests for proto v3
* cleanup coinbase maturity
* rework pool to better handle v2 conversion robustly
* cleanup txpool add_to_pool
* fix nrd kernel test
* move init conversion earlier
* cleanup
* cleanup based on PR feedback
* refactor output to have internal output identifier
refactor to use AsRef for output identifier
make the output MMR explicit in terms of output identifiers
* put the serde deser back for rangeproof
* add json test for transactions
* use mmr sizes in header to validate block weight given header only
* add global weight validaton to headers when reading them untrusted off the network
* fixup tests, one pending test
* add test for deserializing untrusted block header exceeeding weight limits
* validate header mmr sizes, must grow for each block
* remove redundant height check
* fix tests after rebase
block specific TooHeavy error
* wip
* commit
* add block level validation rule around NRD kernels and HF3 block height
* wip
* test coverage for kernel ser/deser for NRD kernels
* add some type safety around ser/deser of nrd relative height
* cleanup
* cleanup tx sig handling and add tests around NRD kernel sig
* test coverage for chain apply block containing NRD kernel
* verify kernel variants when adding tx to pool
NRD kernels will not be accepted until HF3
* add txpool test for NRD kernel handling
* fix merge
* fix api for NRD kernels
* commit
* wip - feature flag for NRD kernel support
* wip
* global config and thread local feature flag for NRD kernel support
* add feature flag support for NRD kernels
default to false when starting up node
allow it to be set to true in context of individual tests
* feature flag
* explicit testing of NRD kernel via feature flag
* add chain_type and feature flag logging on startup
* PR feedback and cleanup
* PR feedback
* cleanup how we handle key splitting for transaction offset
add test to demonstrate a pair of transaction halves sharing same kernel excess
* cleanup
* cleanup
We have a pattern in the code - return Vec, turn it into an iterator, filter and collect to another Vec. The idea was to return iterator where possible to avoid allocating intermediate vecs.
* Introduce GLOBAL_CHAIN_TYPE and make CHAIN_TYPE thread_local.
This makes testing more explicit and significantly more robust.
* set_local_chain_type() in tests
* cleanup - weird
* get pool tests working with explicit local chain_type config
* core tests working with explicit local chain_type
* p2p tests working with explicit local chain_type
* store tests working
* cleanup, feedback
Currently Writable accepts trait Write as a type parameter but Readable
takes Read as a trait object, which is not symmetrical and also less performant. This PR changes Readable trait and all places where it's used
Currently we pass a Vec. This requires an extra allocation and copy of all elements if a caller doesn't have a Vec already, which is at least 95% of cases.
Another, a smaller issue, we have a function util::to_hex and some structs implement to_hex() on top of it, so we have a mix of it in the code. This PR introduces a trait and a blanket impl for AsRef<[u8]> which brings a uniform API (obj.to_hex()). One unfortunate case is arrays of size bigger than 32 - Rust doesn't implement AsRef for them so it requires an ugly hack (&array[..]).to_hex().
* no need for AsFixedBytes we can just use AsRef<[u8]>
* cleanup FixedLength trait
* revert this change for now
* fix store tests
* cleanup and fix tests after rebase
* fix tests
* update based on PR review
less hard-coded values now
* cleanup
* Implement validation and generation of block header by proof and pre pow
* Optimize
* handle missed ser result errors and test invalid pre_pow
* switch to specific error mapping
* clean import
* header version now explicit, no "default" version
"empty" block can safely have verion 0 as it needs to be set later
* default block header uses version 1
* HeaderVersion(1)
* add 2nd HF and cuckaroom
* add cuckaroom and hardfork tests
* remove all traces of later phaseouts
* refactor header_version from valid_header_version
* add bitmap accumulator
refactor vec backend so we can use it outside of tests
introduce a "hash only" vec backend for the accumulator
* get core tests passing
* initial test coverage for bitmap_accumulator
* better test coverage for bitmap accumulator and cleanup code
* refactor txhashset roots, call validate() on roots during block validation
* fix store tests
* log the "merged" root when validating roots
* cleanup, revise based on feedback
* cleanup
* rework it to pass explicit size into bitmap accumulator when applying
* fix: split state validation status into kernel and rproof updates. And fix sync status for these two states
* fix: show correct number of leaves for pruned MMR as well as unpruned
* docs: better docs for kernel/range proof validation
* fix: ordering of kernel and rproofs validation in TUI
* fix: typo in rangeproofs api and comments
We use crate renaming to remove grin prefix, as result imports for util and keychain crates start with crate::
which looks like a part of the current crate. This PR fixes it.
Also some small improvements were made.
If this approach works I'll replicate it to other crates.
* add function to retrieve a set of pmmr indices between a given block height range
* typo
* change APU to just return required indices
* change pmmr index retrieval, change new function to only return pmmr indices between blocks
* tx combinators now take operate on Result to allow for more robust errors handling
replace with_fee() and with_lock_height() with a more flexible with_features()
* pass kernel features in as arg to build::transaction()
* fix chain tests
* fix pool tests
* do not pass kernel around in the combinators
just set it once on the tx when building
* build::partial_transaction now takes a existing tx to build on
