* shutdown the tcpstream when peer connect or accept fail
* remove the unnecessary 3 times retry on peer connecting
* connect/accept are actually handshakings instead of tcpstream connect/accept
* split horizon into two explicit values for cut through and txhashset request
* let node which has 2-7 days of history be able to handle forks larger than 2 days
* add test simulate_long_fork
* add pause/resume feature on p2p for tests
* refactor the state_sync
* ignore the test case simulate_long_fork for normal Travis-CI
* refactor function check_txhashset_needed to be shared with body_sync
* fix: state TxHashsetDone should allow header sync
* Replace logging backend to flexi-logger and add log rotation
* Changed flexi_logger to log4rs
* Disable logging level filtering in Root logger
* Support different logging levels for file and stdout
* Don't log messages from modules other than Grin-related
* Fix formatting
* Place backed up compressed log copies into log file directory
* Increase default log file size to 16 MiB
* Add comment to config file on log_max_size option
* get rid of RwLock around peer instances
* rustfmt
* rename stuff to live_info
* rustfmt
* serialize into PeerInfoDisplay
limit live_info write lock to peer_info.update()
rename to PeerLiveInfo
* simplify broadcast logic a bit more
connected peers are connected by definition
* return true/false on broadcast so we can send to more peers
* rustfmt
* fix p2p and server tests for PeerInfoDisplay
* commit
* Move Proof and Difficulty types to pow module. Difficulty scaling calculation.
* Diffculty scaling and size shift for 2nd PoW
* Backport e7eb26ee
* Test compilation fixes
* Scaling only need to go one way, as @tromp pointed out
* fix for unstable travis-ci test on servers module
* skip the PeerWithSelf connection request on sending side
* logs and comments change; and remove an unnecessary unwrap() in test
* Use internal hashmap for count and contains methods instead of relying on
connected_peers method which is expensive (creates vector from hashmap)
* Reduce number of `clone()`
* Refactor broadcast_xxx
* Remove Iron dependecy and update hyper to version 0.12 #876
* REMOVE ME
* Revert "REMOVE ME"
This reverts commit e9a976eee98a2d5a4dfae5d9e1e4f5ed640c05d3.
* Rebase and start updating libwallet
Libwallet doesn't compile yet.
* Wallet compiles
* Grin compiles
* No compilation errors in tests
* All tests pass
* Reeturn future from handler
* Refactoring
* Fix lifetime issue one more time
I have to force push to rollback all the work done in last 2 days
* Fix wallet send issue
* Clean up
* Migrate main node store to LMDB
In preparation to using LMDB as a wallet database, migrate the
node db. There's no point in having 2 key-value stores.
In addition LMDB provides a few advantages as a node db, namely a
much faster build (compared to RocksDb), lesser dependencies and
transactions.
* Migrated p2p store to lmdb, stuff compiles
* More fixes, chain tests starting to pass
* Fixed txhashset rollback messing with block save and general batch delimitation. Chain tests passing.
* rustfmt
* LMDB max map size of 10MB isn't really workable. Half TB seems reasonable.
* Fix wallet tests
* Rather crucial commit was missing
* rustfmt
* Fixing new merged tests following lmdb changes
* rustfmt
* * Make txhashset validation read-only on fast sync to avoid having
a really long open transaction.
* Fix deadlock in new block processing, batch should always be
created within a txhashset lock (when they interact).
* Comment about batch and txhashset interlacing
* Fix store tests to use batch
* Externalize wallet config and seed
* Converted direct read access to file outputs map to an iterator
* Cleaned up and simplified wallet Backend trait:
* No more direct mutable access to internal structures (HashMap)
* Batch interface for all writes
* Remove unneeded read wrapper (read_wallet)
* rustfmt
* First (incomplete) pass at wallet LMDB backend
* Progressing on lmdb backent iml
* Added batch impl for LMDB wallet backend. Pretty much done with it, but not sure how to deal with commit (owned).
* rustfmt
* Wrapping LMDB batch around a refcell to work around borrow rules
* Compilation up to grin chain
* verify a tx like we verify a block (experimental)
* first minimal_pool test up and running but not testing what we need to
* rework tx_pool validation to use txhashset extension
* minimal tx pool wired up but rough
* works locally (rough statew though)
delete "legacy" pool and graph code
* rework the new pool into TransactionPool and Pool impls
* rework pool to store pool entries
with associated timer and source etc.
* all_transactions
* extra_txs so we can validate stempool against existing txpool
* rework reconcile_block
* txhashset apply_raw_tx can now rewind to a checkpoint (prev raw tx)
* wip - txhashset tx tests
* more flexible rewind on MMRs
* add tests to cover apply_raw_txs on txhashset extension
* add_to_stempool and add_to_txpool
* deaggregate multi kernel tx when adding to txpoool
* handle freshness in stempool
handle propagation of stempool txs via dandelion monitor
* patience timer and fluff if we cannot propagate
to next relay
* aggregate and fluff stempool is we have no relay
* refactor coinbase maturity
* rewrote basic tx pool tests to use a real txhashset via chain adapter
* rework dandelion monitor to reflect recent discussion
works locally but needs a cleanup
* refactor dandelion_monitor - split out phases
* more pool test coverage
* remove old test code from pool (still wip)
* block_building and block_reconciliation tests
* tracked down chain test failure...
* fix test_coinbase_maturity
* dandelion_monitor now runs...
* refactor dandelion config, shared across p2p and pool components
* fix pool tests with new config
* fix p2p tests
* rework tx pool to deal with duplicate commitments (testnet2 limitation)
* cleanup and address some PR feedback
* add big comment about pre_tx...
* documentation and spelling
* unused values in tests; just .unwrap()
* unneeded macro_use (router)
* unneeded macro_use and use-imports
* unused value -> .unwrap
* unused variables
* clarify that " is required (see 107ea3c4dd)
* first steps to dig into #933 test_p2p issues
* less simulnet build warnings
* move some debug! to trace!
* more informative debugs
* standardising on always showing chain tips as "cumulative difficulty @ height [hash]"
* make 2 debug outputs into a single
* "no peers" as warning (not info) to let it stand out more clearly
* move fn param (used only in this one debug line)
* clarify difficulty "units"
* adding appropriate message to bulletproofs to allow for restore
* rustfmt
* should work, now test
* rustfmt
* fix to wallet restore, works now
* fix pool tests
* fix pool tests
* rustfmt
* Initial Dandelion Commit
* Changed stem_tx_pool to tx_stempool
* Introduction of stem memory pool and stem pool config
* Pool push now send to stem memory pool
* Add stem transaction functions
* Add stem transaction pool
* Drastically simplified code structure
* Add monitor transactions
* Add Dandelion monitor and remove transactions from stempool
* Add peer relay monitor
* Reconcile block with stempool
* Fix total size bug
* Add fluff option for pool push
* Added details on dandelion monitor
* Fix issue with missing parent
* Child transaction with stempool parent are now forced stem
* Update Dandelion Relay from outgoing peers
* Fix missing pool reconciliation
* Add the ability to fluff a transaction directly
* Fix tests for Dandelion
* Missing send_stem_transaction method...
* Add fluff handler for wallet
* Add logger when successfully updated Dandelion relay
* Launch transaction monitor last
* Fix dandelion relay misplaced
* Add logging and updating for stempool
* Additionnal check for stem transaction
* Fix 2 Locks in a row
* wip - basic tx pool lookup for kernel short ids (compact block hydration)
* use the nonce in the compact_block to correctly generate short_ids for lookup
* query the tx pool based on kernel short_ids
* tests passing
* cleanup some logging
* cleanup logging
* Clean server shutdown, generalizes usage of `Weak`. Introduces 2 main changes:
* A shared `AtomicBool` that signals server shutdown. All server
threads regularly check it to break out of their main loops when
it changes to `true`.
* Breaking of circular `Arc` references, which can never be
destroyed, by downgrading to `Weak` instead. Only the main server
keeps the `Arc` while all other components get the `Weak` variant.
Both of these are required for all long-living structs to be
cleanly destroyed. Note that in Rust this is fairly important as
most resource-freeing logic is associated with `drop`,
which is only called when said struct is free of scope or `Arc`
references.
Should address most of #536 (only need the stop hook to call
`Server` shutdown).
* Test for fast sync, followed by restart, followed by re-fast-sync
* P2P test fix
* Double sync taking too long for Travis, commenting out for now
* Util to zip and unzip directories
* First pass at sumtree request/response. Add message types, implement the exchange in the protocol, zip up the sumtree directory and stream the file over, with necessary adapter hooks.
* Implement the sumtree archive receive logicGets the sumtree archive data stream from the network and write it to a file. Unzip the file, place it at the right spot and reconstruct the sumtree data structure, rewinding where to the right spot.
* Sumtree hash structure validation
* Simplify sumtree backend buffering logic. The backend for a sumtree has to implement some in-memory buffering logic to provide a commit/rollback interface. The backend itself is an aggregate of 3 underlying storages (an append only file, a remove log and a skip list). The buffering was previously implemented both by the backend and some of the underlying storages. Now pushing back all buffering logic to the storages to keep the backend simpler.
* Add kernel append only store file to sumtrees. The chain sumtrees structure now also saves all kernels to a dedicated file. As that storage is implemented by the append only file wrapper, it's also rewind-aware.
* Full state validation. Checks that:
- MMRs are sane (hash and sum each node)
- Tree roots match the corresponding header
- Kernel signatures are valid
- Sum of all kernel excesses equals the sum of UTXO commitments
minus the supply
* Fast sync handoff to body sync. Once the fast-sync state is fully setup, get bacj in body sync
mode to get the full bodies of the last blocks we're missing.
* First fully working fast sync
* Facility in p2p conn to deal with attachments (raw binary after message).
* Re-introduced sumtree send and receive message handling using the above.
* Fixed test and finished updating all required db state after sumtree validation.
* Massaged a little bit the pipeline orphan check to still work after the new sumtrees have been setup.
* Various cleanup. Consolidated fast sync and full sync into a single function as they're very similar. Proper conditions to trigger a sumtree request and some checks on receiving it.
* Re-introduce peer regular ping/pong
* Fix partial header reads. Turns out that on async tcp streams, even a small chunk like a
header can be read partially. So header read needs to rely on our
fixed-up `read_exact`, with an additional boolean to allow yield
when no bytes were read.
