* include kernel features in msg being signed
hash the msg before signing it (for consistent 32 bytes)
* rustfmt
* fix various tests
* no HF for this (mainnet only)
* Chain init now handles genesis body properly, related unit test creating the genesis with reward
* Avoid making block body public by adding a with_reward method
* apply_block in all genesis cases works
* start wallet command refactoring
* another re-structuring attempt
* rustfmt
* begin splitting up wallet commands
* rustfmt
* clean up wallet arg checking
* rustfmt
* macro for arg parsing
* rustfmt
* factor out init commands
* rustfmt
* move recover to new format
* rustfmt
* add listen command to new format
* rustfmt
* Finish moving commands to new format
* rustfmt
* rustfmt
* propogate errors more cleanly
* rustfmt
* error handling cleanup
* introduce DataFile<T> to hide the LEN handling
use slices rather than vec<u8> where possible
* cleanup
* use DataFile<Hash> and not HashFile
* fix core tests
* Cuckatoo size shift upgrade schedule
* Move the schedule into graph_weight instead of messing with min edge bits
* Cleanup and fixes now that we have an agreed upon schedule
* refactor header MMR to use regular backend
this lets us store header hashes in the mmr data file
the hashes in the hash file are "hash with index"
* rustfmt
* get_header_hash(pos) on header extension
* rustfmt
* cleanup
* cleanup
* comments
* cleanup
* fixup testelem so tests work
* PoW context is now properly picked depending on the chain type,
edge bits and block height. Height const for T4 hard fork leaving
a couple weeks to have miners in place. Removed now unused Cuckoo context.
* Simplified block siphash
* Fix servers crate compilation
* Tiny bit cleaner block siphash. Maybe.
* Cuckatoo min edge bits update for T4 and mainnet
* Fix header size tests, Cuckatoo31 default means one more bit per edge
* Remove redundant param from verify_size
* First pass at iterative siphash
* Generalizing our siphash24 implementation slightly to make it friendlier to repeated hashing
* Block siphash algorithm, Cuckaroo placeholder
* Cuckaroo validator, still needs to be tested with vectors from the @tromp implementation
* Working cuckaroo validation with test vectors for cuckaroo19, will add cuckaroo29 vectors when a lean or mean implementation can find some solutions
* headers msg is now "streamed" off the tcp stream
* rustfmt
* cleanup
* move StreamingReader into ser.rs
extract read_exact out into util crate
* rustfmt
* use FixedLength to define serialized size in bytes of various structs
replace usages of mem::size_of() with ::LEN so we correctly calculate serialized sizes in bytes
* rustfmt
* split horizon into two explicit values for cut through and txhashset request
* let node which has 2-7 days of history be able to handle forks larger than 2 days
* add test simulate_long_fork
* add pause/resume feature on p2p for tests
* refactor the state_sync
* ignore the test case simulate_long_fork for normal Travis-CI
* refactor function check_txhashset_needed to be shared with body_sync
* fix: state TxHashsetDone should allow header sync
Fuzz test found that we don't read IPV6 addr (as part of p2p message)
properly. The code is supposed to read 8 dwords, but [0..8] is not a
slice of 8 ints, but a slice of one Range, so we always read just one
dword
* cleanup pmmrable and len()
introduce FixedLength trait with a const LEN
make Hash impl FixedLength for consistency
* rustfmt
* store tests cleanup
* rustfmt
* whats going on with those comments and rustfmt?
* Update secp dependency to use crates.io version
* Add more details to various Cargo.toml
* Remove use of env variables that aren't reliably provided by cargo
* Initial expired peers removal
* Stop expired peers
* Simplify peer removal and remove only Defunct peers
* Make seed to check for expired peers every hour
* Get rid of unused vector of peers to remove
* Make peer deletion predicate closure immutable
* Replace logging backend to flexi-logger and add log rotation
* Changed flexi_logger to log4rs
* Disable logging level filtering in Root logger
* Support different logging levels for file and stdout
* Don't log messages from modules other than Grin-related
* Fix formatting
* Place backed up compressed log copies into log file directory
* Increase default log file size to 16 MiB
* Add comment to config file on log_max_size option
* header MMR in use within txhashset itself
works with fast sync
not yet in place for initial header sync
* add the (currently unused) sync_head mmr
* use sync MMR during fast sync
rebuild header MMR after we validate full txhashset after download
* support missing header MMR (rebuild as necessary) for legacy nodes
* rename to HashOnly
* cleanup backend.append()
* simplify vec_backend to match simpler append api
* First pass at secondary proof of work difficulty adjustments
* Core and chain test fixes
* Next difficulty calc now needs a height. Scaling calculation fixes. Setting scaling on mined block.
* Change factor to u32 instead of u64.
* Cleanup structs used by next_difficulty
* Fix header size calc with u32 scaling
* change keychain to use bip32 paths
* convert keychain to use bip32
* change identifier to be serialisation of 4-level bip32 path
* wallet changes compiling, pass parent key into all wallet functions
* rustfmt
* fix tests in chain
* rustfmt
* core tests passing
* rustfmt
* pool tests
* rustfmt
* fixing wallet tests
* rustfmt
* remove file wallet
* wallet tests compiling
* rustfmt
* remove db_migrate
* successful tx exchange test using BIP32 paths
* rustfmt
* fix wallet derivation paths to m/0/0/0
* wallet test fixed again, working with default path
* rustfmt
* fix server tests
* rustfmt
* make parent_id a trait on walletbackend
* rustfmt
* add ability for wallet to switch between multiple named accounts, and tests (not complete)
* rustfmt
* account switching tests in place and passing
* rustfmt
* compile and test with latest libsecp changes
* added public key sum to calculated e for aggsig
* rustfmt
* Update secp to 26
* bulletproof bip32 path integration
* rustfmt
* wallet restore updated with bip32 paths, also restores accounts
* rustfmt
* rustfmt
* remove old extkey
* remove old extkey
* rustfmt
* add wallet account commands
* rustfmt
* update wallet documentation
* rustfmt
* merge from master
* update libsecp tag
* merge from upstream and fix server test
* rustfmt
* rustfmt
* merge from master
* update latest libsecp merge
* fix commitment to zero value generation
* cuck placeholder
* rustfmt
* cuckatoo, early days
* rustfmt
* data structures are in place, siphash key creation is consistent with @tromp
* solver in place, (not yet working)
* cuckatoo test solver working with test nonce
* rustfmt
* update solver to remove adjacency list removals
* verifier functioning
* rustfmt
* Proper error handing in Cuckatoo module, couple of tests
* modify cuckoo/cuckatoo solvers and verifiers to function identically, in advance of trait refactoring
* rustfmt
* refactor PoW context into trait, default to using cuckoo context
* rustfmt
* create macros for integer casting/unwraps
* don't instantiate structs when just verifying, add test validation vector for cuckatoo 29
* rustfmt
* don't init cuckoo structs if just validating
* test fix
* ensure BH hashing for POW is only done within miner/validators
* introduce rewindable_kernel_view
cleanup header in extension
* cleanup and docs/comments
* txhashset does not need to be mutable here
* pull validate_kernel_history out into fn
* block_sums and full kernel sum verification
* rustfmt
* add docs/comments
* docs
* rustfmt
* comment on fact total_kernel_sum is redundant now
* make sure we setup block_sums correctly on a fork
* rustfmt
* replace those asserts with errors
* rustfmt
* Move Proof and Difficulty types to pow module. Difficulty scaling calculation.
* Diffculty scaling and size shift for 2nd PoW
* Backport e7eb26ee
* Test compilation fixes
* Scaling only need to go one way, as @tromp pointed out
* document what we do during pipe::apply_block()
* rustfmt
* wip
* rustfmt
* wip
* additional check_known_store and check_known_mmr checks in process_block
* rustfmt
* cleanup coinbase maturity check in process_block
* consolidate the "check in store" logic
add TODOs around the 50 block OldBlock logic
* rustfmt
* cleanup
* improve: HeaderSync optimization (#1372)
* remove get_locator() optimization, which should be an independent pr for security review
* refactoring: move 'headers_streaming_body()' from Message to Protocol
* move 2 headers utils functions out of Protocol, and remove 'pub'
* support reading variable size of BlockHeader, from Cuckoo30 to Cuckoo36
* fix: use global::min_sizeshift() instead of hardcoded 30, because Cuckoo10 will be used for AutomatedTesting chain
* fix: should use global::proofsize() instead of hardcoded 42 when calculate serialized_size_of_header
* replace another 42 with global::proofsize()
* simplify tx validation and aggregation
we *only* need to account for reward when building a block from txs
* rustfmt
* cleanup and tests passing
* rustfmt
* better comments in with_reward()
* fix wallet tests
* Sort pool tx by fee over weight for mining. Fixes#1105
* Bucketing dependent transactions before weighing. Minor tx weight fixes.
* Limit length of tx chain, cleanup and test fixes
* Cleanup all mining references to a hardcoded tx count
* Small test improvement, cleanup
* Include commitments non-duplicate checks in aggregate
* Remove said check from the pool
* Block building now uses tx aggregation to reduce duplication
* Transaction max weight expressed in terms of block weight
* Cleaned up max block sizes, so everything is in therms of weight
* Cleanup block max constants
* Rename verify_size -> verify_weight
* add new core mmr routine upon which others should be built more efficiently
* tons of pmmr optimizations and simplifications
* fix typo and extend docs
* change initial spaces to tab to fix indentation
Fuzz test found the folowing issues with reading block header:
* Unbounded cuckou_sizeshift field in Proof
* Different timestamp range after migration to chrono crate
* Check txhashset content before zip/unzip
* Add header in txhashset verification
* Add copy function and test
* Add file util
* Now check and remove unexpected files instead of just crashing
* improve: time crate flagged as deprecated, switch to actively maintained chrono crate
* improve: complete the switching from deprecated time crate to chrono crate, for all the tests part
* improve: complete switching to chrono crate, for the left tests in 'chain' and 'wallet'
* Make grin.toml config optional. Mirror exisiting config parameters in grin.toml to source code, so binary can run without a config file. Add test for it.
* fixup! Make grin.toml config optional
* fix misbehaving simulnet fastsync test
cleanup redundant cutoff vs bitmap params is rewind and check_compact
* make sure we do not verify full kernel history on a writeable txhashset extension
rework simulnet simulate_fast_sync test to be more robust
* fixup store tests
* sleep for a bit longer to give nodes time to update
their sync_state correctly
* tweak timing of simulate_block_propagation
* Handle SIGINT and SIGTERM (#1180)
* Handle SIGINT and SIGTERM
* pin croaring to version 0.3.2 until we get the clang conflict sorted out (#1183)
* document proposals for - (#1190)
* conditional timelocked outputs, and
* conditional relative timelocked outputs
* problem with validation on rewind
* update difficulty scenarios to display much more data
* correct average sum output
* update pre-genesis block padding to just use previous block's data
* Added sizeshift to proof of work and encoding as VLQ
* Fix some loose ends to get things to compile
* Rename sizehift to min_sizeshift
* Pipeline checks for PoW Cuckoo Cycle size shift and adjustment
factor before difficulty comparison.
* Working delta+VLQ encoding of u64 PoW, unfortunately still a
little larger than u32, at least for Cuckoo30.
* Changed binary encoding of proof of work to a sequence of exact
compact bit representation of each nonce. Somewhat simpler,
shorter and matches the data to be hashed.
* Few fixes based on @tromp feedback:
* Max nonce is 2^(N-1)
* Need a separate constant for reference sizeshift
* Cuckoo implementation now conserves sizeshift. Test providing a
block at higher sizeshift.
* Last small overflow protection
* we do not need Merkle proofs to spend coinbase outputs
we only need the output_mmr_size from the block header
* tests working with no Merkle proofs in inputs
* implement prune_list as a bitmap
and simplify
* cleanup prune_list, use maximum()
* handle migration of prune_list to new bitmap prun file
* legacy filename consts
* cleanup and docs
We used to hash the hash of the header, needlessly. This also
makes a few function signatures more explicit.
Note that this is consensus breaking and should be accompanied
with a similar change on the cuckoo-pow repository:
6fcecf516aFixes#1066
* Migrate main node store to LMDB
In preparation to using LMDB as a wallet database, migrate the
node db. There's no point in having 2 key-value stores.
In addition LMDB provides a few advantages as a node db, namely a
much faster build (compared to RocksDb), lesser dependencies and
transactions.
* Migrated p2p store to lmdb, stuff compiles
* More fixes, chain tests starting to pass
* Fixed txhashset rollback messing with block save and general batch delimitation. Chain tests passing.
* rustfmt
* LMDB max map size of 10MB isn't really workable. Half TB seems reasonable.
* Fix wallet tests
* Rather crucial commit was missing
* rustfmt
* Fixing new merged tests following lmdb changes
* rustfmt
* * Make txhashset validation read-only on fast sync to avoid having
a really long open transaction.
* Fix deadlock in new block processing, batch should always be
created within a txhashset lock (when they interact).
* Comment about batch and txhashset interlacing
* Fix store tests to use batch
* Externalize wallet config and seed
* Converted direct read access to file outputs map to an iterator
* Cleaned up and simplified wallet Backend trait:
* No more direct mutable access to internal structures (HashMap)
* Batch interface for all writes
* Remove unneeded read wrapper (read_wallet)
* rustfmt
* First (incomplete) pass at wallet LMDB backend
* Progressing on lmdb backent iml
* Added batch impl for LMDB wallet backend. Pretty much done with it, but not sure how to deal with commit (owned).
* rustfmt
* Wrapping LMDB batch around a refcell to work around borrow rules
* Compilation up to grin chain
* Add kernel commitments sum and kernel and output MMR sizes to block header
* Sum a block without including previous sums, cleanup. Blocks are now summed and validated based on their own totals and not the totals since genesis. This allows to get rid of BlockSum and simplified the setting of a new block's roots, kernel sum and MMR sizes. Fixes#116
* Additional kernel MMR validation to check all prior header roots successively
* Wallet tests fix
* only store leaves in rm_log
* cleanup
* commit
* add failing test to cover case where we compact
an already compacted data file
* fix the logic for pruning the data file
* rm_log only contains leaves
prunelist only contains non-leaf subtree roots
* cleanup
* commit
* bitmap impl running in parallel
* rustfmt
* this is better - rewind unremoves outpu pos spent by rewound inputs
* commit
* commit
* pass bitmap around when rewinding
* store and retrieve input bitmap per block from the db
* Replace the RemoveLog with the UTXO set.
* utxo set starting to pass tests
* stuff works...
* split store types out
added some tests to compare the performance of the rm_log and the proposed utxo_set
* pull prune_list out into standalone file
* cleanup, get rid of unused height param
* cleanup and add doc comments
* add support for migrating rm_log to utxo_set
* take snapshot of utxo file during fast sync
implement migration of rm_log -> utxo_set
* rename rewound_copy to snapshot
* fixup pmmr tests to reflect cutoff_pos
* cleanup unused import
* check_compact needs to rewind the utxo_set as appropriate
* fix pool tests
* fixup core tests
* cache block_input_bitmaps via LruCache in store
* cache block header on initial write to db
* rename utxo_set -> leaf_set
and remove references to "spent" in grin_store
* better document the rewind behavior
* First pass at restructuring the keychain crate and introducing a Keychain trait
* Parameterized everything that had to. Stuff compiles.
* More stuff compiles, fix most tests
* Big merge, pushing down opening the keychain forced adding factory methods on trait
* Test fixes for pool and servers crate
Small refactoring of one folder, if it makes sense I could extend the scope.
* Remove some cloning (real and just verbosity in the code)
* Naming conventions like to/into*
* Some Clippy's suggestions
I found that we don't use field init shorthand syntax, so I didn't touch this part, was it discussed before?
* move checker and rename to updater
* rustfmt
* complete checker/updater move
* rustfmt
* move libwallet error into separate file
* rustfmt
* starting to sort our error types
* updating errors in libtx and libwallet
* rustfmt
* factor out error type
* rustfmt
* compiling, errors split into libwallet and wallet errors
* rustfmt
* changing user error reporting to new format
* rustfmt
* clean up error types
* clean up error types
* move restore into libwallet
* rustfmt
* verify a tx like we verify a block (experimental)
* first minimal_pool test up and running but not testing what we need to
* rework tx_pool validation to use txhashset extension
* minimal tx pool wired up but rough
* works locally (rough statew though)
delete "legacy" pool and graph code
* rework the new pool into TransactionPool and Pool impls
* rework pool to store pool entries
with associated timer and source etc.
* all_transactions
* extra_txs so we can validate stempool against existing txpool
* rework reconcile_block
* txhashset apply_raw_tx can now rewind to a checkpoint (prev raw tx)
* wip - txhashset tx tests
* more flexible rewind on MMRs
* add tests to cover apply_raw_txs on txhashset extension
* add_to_stempool and add_to_txpool
* deaggregate multi kernel tx when adding to txpoool
* handle freshness in stempool
handle propagation of stempool txs via dandelion monitor
* patience timer and fluff if we cannot propagate
to next relay
* aggregate and fluff stempool is we have no relay
* refactor coinbase maturity
* rewrote basic tx pool tests to use a real txhashset via chain adapter
* rework dandelion monitor to reflect recent discussion
works locally but needs a cleanup
* refactor dandelion_monitor - split out phases
* more pool test coverage
* remove old test code from pool (still wip)
* block_building and block_reconciliation tests
* tracked down chain test failure...
* fix test_coinbase_maturity
* dandelion_monitor now runs...
* refactor dandelion config, shared across p2p and pool components
* fix pool tests with new config
* fix p2p tests
* rework tx pool to deal with duplicate commitments (testnet2 limitation)
* cleanup and address some PR feedback
* add big comment about pre_tx...
* Beginning to rework aggsig library workflow
* more refactoring of transaction api
* whoever does round 1 first creates offset
* slate finalisation now context-free, so anyone can do it
* remove concept of transaction phase
* remove slate phase enum
* update actual send/receive code with new transaction lib workflow
