* document what we do during pipe::apply_block()
* rustfmt
* wip
* rustfmt
* wip
* additional check_known_store and check_known_mmr checks in process_block
* rustfmt
* cleanup coinbase maturity check in process_block
* consolidate the "check in store" logic
add TODOs around the 50 block OldBlock logic
* rustfmt
* cleanup
* improve: HeaderSync optimization (#1372)
* remove get_locator() optimization, which should be an independent pr for security review
* refactoring: move 'headers_streaming_body()' from Message to Protocol
* move 2 headers utils functions out of Protocol, and remove 'pub'
* support reading variable size of BlockHeader, from Cuckoo30 to Cuckoo36
* fix: use global::min_sizeshift() instead of hardcoded 30, because Cuckoo10 will be used for AutomatedTesting chain
* fix: should use global::proofsize() instead of hardcoded 42 when calculate serialized_size_of_header
* replace another 42 with global::proofsize()
* simplify tx validation and aggregation
we *only* need to account for reward when building a block from txs
* rustfmt
* cleanup and tests passing
* rustfmt
* better comments in with_reward()
* fix wallet tests
* Sort pool tx by fee over weight for mining. Fixes#1105
* Bucketing dependent transactions before weighing. Minor tx weight fixes.
* Limit length of tx chain, cleanup and test fixes
* Cleanup all mining references to a hardcoded tx count
* Small test improvement, cleanup
* Include commitments non-duplicate checks in aggregate
* Remove said check from the pool
* Block building now uses tx aggregation to reduce duplication
* Transaction max weight expressed in terms of block weight
* Cleaned up max block sizes, so everything is in therms of weight
* Cleanup block max constants
* Rename verify_size -> verify_weight
* add new core mmr routine upon which others should be built more efficiently
* tons of pmmr optimizations and simplifications
* fix typo and extend docs
* change initial spaces to tab to fix indentation
Fuzz test found the folowing issues with reading block header:
* Unbounded cuckou_sizeshift field in Proof
* Different timestamp range after migration to chrono crate
* Check txhashset content before zip/unzip
* Add header in txhashset verification
* Add copy function and test
* Add file util
* Now check and remove unexpected files instead of just crashing
* improve: time crate flagged as deprecated, switch to actively maintained chrono crate
* improve: complete the switching from deprecated time crate to chrono crate, for all the tests part
* improve: complete switching to chrono crate, for the left tests in 'chain' and 'wallet'
* Make grin.toml config optional. Mirror exisiting config parameters in grin.toml to source code, so binary can run without a config file. Add test for it.
* fixup! Make grin.toml config optional
* fix misbehaving simulnet fastsync test
cleanup redundant cutoff vs bitmap params is rewind and check_compact
* make sure we do not verify full kernel history on a writeable txhashset extension
rework simulnet simulate_fast_sync test to be more robust
* fixup store tests
* sleep for a bit longer to give nodes time to update
their sync_state correctly
* tweak timing of simulate_block_propagation
* Handle SIGINT and SIGTERM (#1180)
* Handle SIGINT and SIGTERM
* pin croaring to version 0.3.2 until we get the clang conflict sorted out (#1183)
* document proposals for - (#1190)
* conditional timelocked outputs, and
* conditional relative timelocked outputs
* problem with validation on rewind
* update difficulty scenarios to display much more data
* correct average sum output
* update pre-genesis block padding to just use previous block's data
* Added sizeshift to proof of work and encoding as VLQ
* Fix some loose ends to get things to compile
* Rename sizehift to min_sizeshift
* Pipeline checks for PoW Cuckoo Cycle size shift and adjustment
factor before difficulty comparison.
* Working delta+VLQ encoding of u64 PoW, unfortunately still a
little larger than u32, at least for Cuckoo30.
* Changed binary encoding of proof of work to a sequence of exact
compact bit representation of each nonce. Somewhat simpler,
shorter and matches the data to be hashed.
* Few fixes based on @tromp feedback:
* Max nonce is 2^(N-1)
* Need a separate constant for reference sizeshift
* Cuckoo implementation now conserves sizeshift. Test providing a
block at higher sizeshift.
* Last small overflow protection
* we do not need Merkle proofs to spend coinbase outputs
we only need the output_mmr_size from the block header
* tests working with no Merkle proofs in inputs
* implement prune_list as a bitmap
and simplify
* cleanup prune_list, use maximum()
* handle migration of prune_list to new bitmap prun file
* legacy filename consts
* cleanup and docs
We used to hash the hash of the header, needlessly. This also
makes a few function signatures more explicit.
Note that this is consensus breaking and should be accompanied
with a similar change on the cuckoo-pow repository:
6fcecf516aFixes#1066
* Migrate main node store to LMDB
In preparation to using LMDB as a wallet database, migrate the
node db. There's no point in having 2 key-value stores.
In addition LMDB provides a few advantages as a node db, namely a
much faster build (compared to RocksDb), lesser dependencies and
transactions.
* Migrated p2p store to lmdb, stuff compiles
* More fixes, chain tests starting to pass
* Fixed txhashset rollback messing with block save and general batch delimitation. Chain tests passing.
* rustfmt
* LMDB max map size of 10MB isn't really workable. Half TB seems reasonable.
* Fix wallet tests
* Rather crucial commit was missing
* rustfmt
* Fixing new merged tests following lmdb changes
* rustfmt
* * Make txhashset validation read-only on fast sync to avoid having
a really long open transaction.
* Fix deadlock in new block processing, batch should always be
created within a txhashset lock (when they interact).
* Comment about batch and txhashset interlacing
* Fix store tests to use batch
* Externalize wallet config and seed
* Converted direct read access to file outputs map to an iterator
* Cleaned up and simplified wallet Backend trait:
* No more direct mutable access to internal structures (HashMap)
* Batch interface for all writes
* Remove unneeded read wrapper (read_wallet)
* rustfmt
* First (incomplete) pass at wallet LMDB backend
* Progressing on lmdb backent iml
* Added batch impl for LMDB wallet backend. Pretty much done with it, but not sure how to deal with commit (owned).
* rustfmt
* Wrapping LMDB batch around a refcell to work around borrow rules
* Compilation up to grin chain
* Add kernel commitments sum and kernel and output MMR sizes to block header
* Sum a block without including previous sums, cleanup. Blocks are now summed and validated based on their own totals and not the totals since genesis. This allows to get rid of BlockSum and simplified the setting of a new block's roots, kernel sum and MMR sizes. Fixes#116
* Additional kernel MMR validation to check all prior header roots successively
* Wallet tests fix
* only store leaves in rm_log
* cleanup
* commit
* add failing test to cover case where we compact
an already compacted data file
* fix the logic for pruning the data file
* rm_log only contains leaves
prunelist only contains non-leaf subtree roots
* cleanup
* commit
* bitmap impl running in parallel
* rustfmt
* this is better - rewind unremoves outpu pos spent by rewound inputs
* commit
* commit
* pass bitmap around when rewinding
* store and retrieve input bitmap per block from the db
* Replace the RemoveLog with the UTXO set.
* utxo set starting to pass tests
* stuff works...
* split store types out
added some tests to compare the performance of the rm_log and the proposed utxo_set
* pull prune_list out into standalone file
* cleanup, get rid of unused height param
* cleanup and add doc comments
* add support for migrating rm_log to utxo_set
* take snapshot of utxo file during fast sync
implement migration of rm_log -> utxo_set
* rename rewound_copy to snapshot
* fixup pmmr tests to reflect cutoff_pos
* cleanup unused import
* check_compact needs to rewind the utxo_set as appropriate
* fix pool tests
* fixup core tests
* cache block_input_bitmaps via LruCache in store
* cache block header on initial write to db
* rename utxo_set -> leaf_set
and remove references to "spent" in grin_store
* better document the rewind behavior
* First pass at restructuring the keychain crate and introducing a Keychain trait
* Parameterized everything that had to. Stuff compiles.
* More stuff compiles, fix most tests
* Big merge, pushing down opening the keychain forced adding factory methods on trait
* Test fixes for pool and servers crate
Small refactoring of one folder, if it makes sense I could extend the scope.
* Remove some cloning (real and just verbosity in the code)
* Naming conventions like to/into*
* Some Clippy's suggestions
I found that we don't use field init shorthand syntax, so I didn't touch this part, was it discussed before?
* move checker and rename to updater
* rustfmt
* complete checker/updater move
* rustfmt
* move libwallet error into separate file
* rustfmt
* starting to sort our error types
* updating errors in libtx and libwallet
* rustfmt
* factor out error type
* rustfmt
* compiling, errors split into libwallet and wallet errors
* rustfmt
* changing user error reporting to new format
* rustfmt
* clean up error types
* clean up error types
* move restore into libwallet
* rustfmt
* verify a tx like we verify a block (experimental)
* first minimal_pool test up and running but not testing what we need to
* rework tx_pool validation to use txhashset extension
* minimal tx pool wired up but rough
* works locally (rough statew though)
delete "legacy" pool and graph code
* rework the new pool into TransactionPool and Pool impls
* rework pool to store pool entries
with associated timer and source etc.
* all_transactions
* extra_txs so we can validate stempool against existing txpool
* rework reconcile_block
* txhashset apply_raw_tx can now rewind to a checkpoint (prev raw tx)
* wip - txhashset tx tests
* more flexible rewind on MMRs
* add tests to cover apply_raw_txs on txhashset extension
* add_to_stempool and add_to_txpool
* deaggregate multi kernel tx when adding to txpoool
* handle freshness in stempool
handle propagation of stempool txs via dandelion monitor
* patience timer and fluff if we cannot propagate
to next relay
* aggregate and fluff stempool is we have no relay
* refactor coinbase maturity
* rewrote basic tx pool tests to use a real txhashset via chain adapter
* rework dandelion monitor to reflect recent discussion
works locally but needs a cleanup
* refactor dandelion_monitor - split out phases
* more pool test coverage
* remove old test code from pool (still wip)
* block_building and block_reconciliation tests
* tracked down chain test failure...
* fix test_coinbase_maturity
* dandelion_monitor now runs...
* refactor dandelion config, shared across p2p and pool components
* fix pool tests with new config
* fix p2p tests
* rework tx pool to deal with duplicate commitments (testnet2 limitation)
* cleanup and address some PR feedback
* add big comment about pre_tx...
* Beginning to rework aggsig library workflow
* more refactoring of transaction api
* whoever does round 1 first creates offset
* slate finalisation now context-free, so anyone can do it
* remove concept of transaction phase
* remove slate phase enum
* update actual send/receive code with new transaction lib workflow
* refactoring transaction building code
* serialise return transaction
* move shared functions into transactions, ensure wallet manipulation is only done outside of aggsig transaction lib
* remove unneeded wallet config from fn
* adding test functions to facilitate libwallet transaction testing
* rustfmt
* refactoring checker somewhat, adding ability to create and transactions against local copy of chain for simpler testing
* finish transaction testing functionality which verifies transactions work properly
* Remove wallet output manipulation from transaction building lib
* ensure sender expects full transaction back on last phase
* ensure sender expects full transaction back on last phase
* beginning to refactor keychain into wallet lib
* rustfmt
* more refactor of aggsig lib, simplify aggsig context manager, hold instance statically for now
* clean some warnings
* clean some warnings
* fix wallet send test a bit
* fix core tests, move wallet dependent tests into integration tests
* repair chain tests
* refactor/fix pool tests
* fix wallet tests, moved from keychain
* add wallet tests
* block sums and reworked block validation
read and write block_sums
refactor validate on both block and txhashset
write block_sum on fast sync
we store the kernel_sum (need to account for the offset)
* block_sums
* rustfmt
* cleanup
* wip BlockMarker struct, get rid of PMMRMetadata
* use rewind to init the txhashet correctly on startup, we do not need to track pos via metadata (we have block markers), we do not need to open the txhashset with specific pos (we have rewind)
* better logging on init
* keep rewinding and validating on init, to find a good block
* use validate_roots on chain init
* Test multi kernel deaggregation
* Add aggregate without cut_through and deaggregate function
* Add deaggregate function in pool and test
* Rustfmt
* Add deaggregate_and_add_to_memory_pool
* Deaggregate regular multi kernel transaction by default
* Rustfmt
* Add error type faileddeaggregation
* Add find candidates function
* Rustfmt
* Use intersection of sets instead of for comparisons
* Rustfmt
* Removed unnecessary if
* Stricter verification with is_subset
* Rustfmt
* Beginning to remove in-process miner
* rustfmt
* rustfmt
* rustfmt
* rustfmt
* remove pow crate and put remnants into core
* rustfmt
* fix test compilation in core and chain
* rustfmt
* Updating server tests to use test miner
* rustfmt
* rustfmt
* remove pow from test matrix
* adding basic stratum stats to TUI
* run stratum server at all times, and halt messages while syncing
* fix core tests
* add ability to run internal test miner for cuckoo 16 testing
* modify build instructions
