* Fix secondary scaling bugs; rename is_testnet -> is_floonet (#2215)
* add global::is_mainnet()
* use it to change pre-genesis pow type
* rename is_testnet -> is_floonet
* Support multiple chain configurations (#2217)
* Support multiple chain configurations
Supports generating the proper configuration for each chain type
(mainnet, floonet, usernet). Will run them by default under
their respective root directory (~/.grin/main, ~/.grin/floo, etc).
Assigned default ports for mainnet, overriding them to keep Floonet
ports unchanged.
For now, starting on mainnet will abort.
* Fixed usernet command line help message. Fixes#2217
* Differing magic numbers for each chain type (#2208)
* stick to e=H(R|P|m) when use schnorr signature (#2200)
* stick to e=H(R|P|m) when use schnorr signature
* (1)add verify_slate_messages for wallet receive (2)log the message content
* remove debug log on verify_slate_messages
* verify the sender's message signature when receive_tx in wallet listen
* Revert "remove debug log on verify_slate_messages"
This reverts commit 65ea32a407.
* Revert "rustfmt"
This reverts commit c380ab9185.
* Revert "(1)add verify_slate_messages for wallet receive (2)log the message content"
This reverts commit 9584ca7a89.
* [re-floonet] Keychain Floonet BIP32 version/network option (#2235)
* add 'is_floonet' property to keychain
* fix hex encoding and tests
* Fix couple floonet loose ends (#2230)
* Fix couple floonet loose ends. Fixes#2216
* Doc fix for sig message
* Refuse unkown kernel features (#2244)
* Minor: magic number change for re-floonet
* Set pre genesis is_secondary to true (#2247)
* Minor: tx validation error display underlying
* New floonet genesis
* genesis rustfmt
* Use chain-specific config for wallet toml gen
* Fix default wallet_listener_url
* New more reasonable genesis block, bumped version
* genesis rustfmt
* Couple minor fixes to genesis generation script
* Issue #1348 - cleanup pmmr_leaf snapshot files (from fast-sync)
* create a new function in store::pmmr to delete files
* call the new function during compaction
* create test coverage for the new function
* formatting
* Fix rustdoc issue
I had to add a text annotation to the preformatted block to prevent cargo from creating a doctest.
* don't delete files that were recently accessed
* parameterize the age in the cleanup function
* add a new constant defining how long the rewind files should be saved for
* enhance the unit test to include files that were too new to be deleted and files that were old enough to be deleted
* formatting
* formatting
* remove errant println statement
* functional style change
* introduce DataFile<T> to hide the LEN handling
use slices rather than vec<u8> where possible
* cleanup
* use DataFile<Hash> and not HashFile
* fix core tests
* refactor header MMR to use regular backend
this lets us store header hashes in the mmr data file
the hashes in the hash file are "hash with index"
* rustfmt
* get_header_hash(pos) on header extension
* rustfmt
* cleanup
* cleanup
* comments
* cleanup
* fixup testelem so tests work
* cleanup pmmrable and len()
introduce FixedLength trait with a const LEN
make Hash impl FixedLength for consistency
* rustfmt
* store tests cleanup
* rustfmt
* whats going on with those comments and rustfmt?
* Update secp dependency to use crates.io version
* Add more details to various Cargo.toml
* Remove use of env variables that aren't reliably provided by cargo
* Replace logging backend to flexi-logger and add log rotation
* Changed flexi_logger to log4rs
* Disable logging level filtering in Root logger
* Support different logging levels for file and stdout
* Don't log messages from modules other than Grin-related
* Fix formatting
* Place backed up compressed log copies into log file directory
* Increase default log file size to 16 MiB
* Add comment to config file on log_max_size option
* header MMR in use within txhashset itself
works with fast sync
not yet in place for initial header sync
* add the (currently unused) sync_head mmr
* use sync MMR during fast sync
rebuild header MMR after we validate full txhashset after download
* support missing header MMR (rebuild as necessary) for legacy nodes
* rename to HashOnly
* cleanup backend.append()
* simplify vec_backend to match simpler append api
* change keychain to use bip32 paths
* convert keychain to use bip32
* change identifier to be serialisation of 4-level bip32 path
* wallet changes compiling, pass parent key into all wallet functions
* rustfmt
* fix tests in chain
* rustfmt
* core tests passing
* rustfmt
* pool tests
* rustfmt
* fixing wallet tests
* rustfmt
* remove file wallet
* wallet tests compiling
* rustfmt
* remove db_migrate
* successful tx exchange test using BIP32 paths
* rustfmt
* fix wallet derivation paths to m/0/0/0
* wallet test fixed again, working with default path
* rustfmt
* fix server tests
* rustfmt
* make parent_id a trait on walletbackend
* rustfmt
* add ability for wallet to switch between multiple named accounts, and tests (not complete)
* rustfmt
* account switching tests in place and passing
* rustfmt
* compile and test with latest libsecp changes
* added public key sum to calculated e for aggsig
* rustfmt
* Update secp to 26
* bulletproof bip32 path integration
* rustfmt
* wallet restore updated with bip32 paths, also restores accounts
* rustfmt
* rustfmt
* remove old extkey
* remove old extkey
* rustfmt
* add wallet account commands
* rustfmt
* update wallet documentation
* rustfmt
* merge from master
* update libsecp tag
* merge from upstream and fix server test
* rustfmt
* rustfmt
* merge from master
* update latest libsecp merge
* fix commitment to zero value generation
* Check txhashset content before zip/unzip
* Add header in txhashset verification
* Add copy function and test
* Add file util
* Now check and remove unexpected files instead of just crashing
* improve: time crate flagged as deprecated, switch to actively maintained chrono crate
* improve: complete the switching from deprecated time crate to chrono crate, for all the tests part
* improve: complete switching to chrono crate, for the left tests in 'chain' and 'wallet'
* beginnings of transaction log for db
* add migrate utility for file-wallet to db wallet
* rustfmt + missing file
* update transaction log entry status on confirmed txs, add basic tests for transaction log
* fix misbehaving simulnet fastsync test
cleanup redundant cutoff vs bitmap params is rewind and check_compact
* make sure we do not verify full kernel history on a writeable txhashset extension
rework simulnet simulate_fast_sync test to be more robust
* fixup store tests
* sleep for a bit longer to give nodes time to update
their sync_state correctly
* tweak timing of simulate_block_propagation
* Handle SIGINT and SIGTERM (#1180)
* Handle SIGINT and SIGTERM
* pin croaring to version 0.3.2 until we get the clang conflict sorted out (#1183)
* document proposals for - (#1190)
* conditional timelocked outputs, and
* conditional relative timelocked outputs
* problem with validation on rewind
* update difficulty scenarios to display much more data
* correct average sum output
* update pre-genesis block padding to just use previous block's data
Only rewind kernels to avoid requiring proper bitmap extraction. Also avoids maintaining bitmap data for kernels by introducing a "prunable" flag on PMMR backend.
* implement prune_list as a bitmap
and simplify
* cleanup prune_list, use maximum()
* handle migration of prune_list to new bitmap prun file
* legacy filename consts
* cleanup and docs
* Migrate main node store to LMDB
In preparation to using LMDB as a wallet database, migrate the
node db. There's no point in having 2 key-value stores.
In addition LMDB provides a few advantages as a node db, namely a
much faster build (compared to RocksDb), lesser dependencies and
transactions.
* Migrated p2p store to lmdb, stuff compiles
* More fixes, chain tests starting to pass
* Fixed txhashset rollback messing with block save and general batch delimitation. Chain tests passing.
* rustfmt
* LMDB max map size of 10MB isn't really workable. Half TB seems reasonable.
* Fix wallet tests
* Rather crucial commit was missing
* rustfmt
* Fixing new merged tests following lmdb changes
* rustfmt
* * Make txhashset validation read-only on fast sync to avoid having
a really long open transaction.
* Fix deadlock in new block processing, batch should always be
created within a txhashset lock (when they interact).
* Comment about batch and txhashset interlacing
* Fix store tests to use batch
* Externalize wallet config and seed
* Converted direct read access to file outputs map to an iterator
* Cleaned up and simplified wallet Backend trait:
* No more direct mutable access to internal structures (HashMap)
* Batch interface for all writes
* Remove unneeded read wrapper (read_wallet)
* rustfmt
* First (incomplete) pass at wallet LMDB backend
* Progressing on lmdb backent iml
* Added batch impl for LMDB wallet backend. Pretty much done with it, but not sure how to deal with commit (owned).
* rustfmt
* Wrapping LMDB batch around a refcell to work around borrow rules
* Compilation up to grin chain
* Add kernel commitments sum and kernel and output MMR sizes to block header
* Sum a block without including previous sums, cleanup. Blocks are now summed and validated based on their own totals and not the totals since genesis. This allows to get rid of BlockSum and simplified the setting of a new block's roots, kernel sum and MMR sizes. Fixes#116
* Additional kernel MMR validation to check all prior header roots successively
* Wallet tests fix
* only store leaves in rm_log
* cleanup
* commit
* add failing test to cover case where we compact
an already compacted data file
* fix the logic for pruning the data file
* rm_log only contains leaves
prunelist only contains non-leaf subtree roots
* cleanup
* commit
* bitmap impl running in parallel
* rustfmt
* this is better - rewind unremoves outpu pos spent by rewound inputs
* commit
* commit
* pass bitmap around when rewinding
* store and retrieve input bitmap per block from the db
* Replace the RemoveLog with the UTXO set.
* utxo set starting to pass tests
* stuff works...
* split store types out
added some tests to compare the performance of the rm_log and the proposed utxo_set
* pull prune_list out into standalone file
* cleanup, get rid of unused height param
* cleanup and add doc comments
* add support for migrating rm_log to utxo_set
* take snapshot of utxo file during fast sync
implement migration of rm_log -> utxo_set
* rename rewound_copy to snapshot
* fixup pmmr tests to reflect cutoff_pos
* cleanup unused import
* check_compact needs to rewind the utxo_set as appropriate
* fix pool tests
* fixup core tests
* cache block_input_bitmaps via LruCache in store
* cache block header on initial write to db
* rename utxo_set -> leaf_set
and remove references to "spent" in grin_store
* better document the rewind behavior
* verify a tx like we verify a block (experimental)
* first minimal_pool test up and running but not testing what we need to
* rework tx_pool validation to use txhashset extension
* minimal tx pool wired up but rough
* works locally (rough statew though)
delete "legacy" pool and graph code
* rework the new pool into TransactionPool and Pool impls
* rework pool to store pool entries
with associated timer and source etc.
* all_transactions
* extra_txs so we can validate stempool against existing txpool
* rework reconcile_block
* txhashset apply_raw_tx can now rewind to a checkpoint (prev raw tx)
* wip - txhashset tx tests
* more flexible rewind on MMRs
* add tests to cover apply_raw_txs on txhashset extension
* add_to_stempool and add_to_txpool
* deaggregate multi kernel tx when adding to txpoool
* handle freshness in stempool
handle propagation of stempool txs via dandelion monitor
* patience timer and fluff if we cannot propagate
to next relay
* aggregate and fluff stempool is we have no relay
* refactor coinbase maturity
* rewrote basic tx pool tests to use a real txhashset via chain adapter
* rework dandelion monitor to reflect recent discussion
works locally but needs a cleanup
* refactor dandelion_monitor - split out phases
* more pool test coverage
* remove old test code from pool (still wip)
* block_building and block_reconciliation tests
* tracked down chain test failure...
* fix test_coinbase_maturity
* dandelion_monitor now runs...
* refactor dandelion config, shared across p2p and pool components
* fix pool tests with new config
* fix p2p tests
* rework tx pool to deal with duplicate commitments (testnet2 limitation)
* cleanup and address some PR feedback
* add big comment about pre_tx...
* beginning to refactor keychain into wallet lib
* rustfmt
* more refactor of aggsig lib, simplify aggsig context manager, hold instance statically for now
* clean some warnings
* clean some warnings
* fix wallet send test a bit
* fix core tests, move wallet dependent tests into integration tests
* repair chain tests
* refactor/fix pool tests
* fix wallet tests, moved from keychain
* add wallet tests
Adds a `BufWrite` around each file that gets incrementally written
to in the MMR store main types. Should speed up a few operations
quite a bit, but particularly flushing the remove log. Saving
txhashsets on new blocks is now 10x to 20x faster, making full
block syncs a lot faster.
* wip BlockMarker struct, get rid of PMMRMetadata
* use rewind to init the txhashet correctly on startup, we do not need to track pos via metadata (we have block markers), we do not need to open the txhashset with specific pos (we have rewind)
* better logging on init
* keep rewinding and validating on init, to find a good block
* use validate_roots on chain init
* Bump up crates versions
* Finally add a Cargo.lock to avoid dependency breakages
* Build doc update for testnet2
* Fix test framework not really using its mining config
* Testnet2 genesis, best so far at 128 difficulty (a nice number)
* Minor build doc update
* speed up Travis startup
- keep all test tempdata in target/tmp
- purge before saving target/* to S3 cache
* allow 4 minutes for cache up/download from Travis' S3 cache stores
* rewind to header as part of txhashset validation
otherwise we risk including a new block and the roots do not match
* fix bug in rm_log rewind (wants to be inclusive of provided index)
* put block marker in the index so we can rewind correctly
during validation of the new txhashset
* rustfmt
* hash_with_index on non-leaf nodes
rework the Merkle proof to include the pos of each sibling in the path
* rustfmt
* cleanup
* cleanup
* use get_from_file in validate (children may have been "removed")
* rustfmt
* fixup store tests
* wip
* failing test for being too eager when pruning a sibling
* commit
* rustfmt
* [WIP] modified get_shift and get_leaf_shift to account for leaving "pruned but not compacted" leaves in place
Note: this currently breaks check_compact as nothing else is aware of the modified behavior
* rustfmt
* commit
* rustfmt
* basic prune/compact/shift working
* rustfmt
* commit
* rustfmt
* next_pruned_idx working (I think)
* commit
* horizon test uncovered some subtle issues - wip
* rustfmt
* cleanup
* rustfmt
* commit
* cleanup
* cleanup
* commit
* rustfmt
* contains -> binary_search
* rustfmt
* no need for height==0 special case
* wip - works for single compact, 2nd one breaks the mmr hashes
* commit
* rustfmt
* fixed it (needs a lot of cleanup)
we were not traversing all the way up to the peak if we pruned an entire tree
so rm_log and prune list were inconsistent
* multiple compact steps are working
data file not being copmacted currently (still to investigate)
* cleanup store tests
* cleanup
* cleanup up debug
* rustfmt
* take kernel offsets into account when summing kernels and outputs for full txhashset validation
validate chain state pre and post compaction
* rustfmt
* fix wallet refresh (we need block height to be refreshed on non-coinbase outputs)
otherwise we cannot spend them...
* rustfmt
* Implementation of compaction for the chain. Single entry point on the chain triggers compaction of all MMRs as well as the cleanup of the positional index and full blocks.
* API endpoint, additional tests and more fixes for compaction
* Also prune PMMR metadata, minor bug fix
* PMMR store tests fix
