* PoW context is now properly picked depending on the chain type,
edge bits and block height. Height const for T4 hard fork leaving
a couple weeks to have miners in place. Removed now unused Cuckoo context.
* Simplified block siphash
* Fix servers crate compilation
* Tiny bit cleaner block siphash. Maybe.
* Cuckatoo min edge bits update for T4 and mainnet
* Fix header size tests, Cuckatoo31 default means one more bit per edge
* Remove redundant param from verify_size
* block_accepted via adapter is now reorg aware
we skip reconciling the txpool is block is not most work
we skip reconciling the reorg_cache is not a reorg
* rustfmt
* logging tweaks
* rework block_accepted interface
* rustfmt
* rework reorg determination
* introduce BlockStatus to represent next vs reorg vs fork
* rustfmt
* cleanup logging
* log from adapter, even during sync
* default to minimum_confirmation = 10
pass in minimum_confirmations to grin wallet info
* rustfmt
* rebase against master
* fixup wallet tests
* fixup server tests
* remove receive_coinbase fn from wallet to wallet client
* start moving clients into separate mod
* rustfmt
* move wallet client + start listener into wallet 'clients'
* rustfmt
* refactor API to make it more modular and completely decouple sending
* rustfmt
* further decouple API from sending methods
* rustfmt
* remove wallet to wallet client trait
* rustfmt
* rename / refactor client + adapters
* rustfmt
* add adapter concept
* add midding node_clients dir
* add file and null adapters
* rustfmt
* remove receive from owner api
* factor out receiving slates into trait
* rustfmt
* adding listen trait
* rustfmt
* change listener to use trait
* rustfmt
* add test for file-based exchange
* replace http api send command
* rustfmt
* move controller out of libwallet and into top-level wallet dir
* rustfmt
* add moved controller
* split horizon into two explicit values for cut through and txhashset request
* let node which has 2-7 days of history be able to handle forks larger than 2 days
* add test simulate_long_fork
* add pause/resume feature on p2p for tests
* refactor the state_sync
* ignore the test case simulate_long_fork for normal Travis-CI
* refactor function check_txhashset_needed to be shared with body_sync
* fix: state TxHashsetDone should allow header sync
* Cleanup syncer and sync header (locator)
* Simplify body sync
* Remove duplicate head in locator, add greater case in close_enough
* Various sync small fixes and tuning after testing
* More close_enough tests and related minor fixes
* Update secp dependency to use crates.io version
* Add more details to various Cargo.toml
* Remove use of env variables that aren't reliably provided by cargo
* Add a struct to encapsulate common references and avoid passing
them around on every function.
* Consolidate `skip_sync_wait` and `awaiting_peers` in an
additional sync status.
* New awaiting peer status is initial too
* Initial expired peers removal
* Stop expired peers
* Simplify peer removal and remove only Defunct peers
* Make seed to check for expired peers every hour
* Get rid of unused vector of peers to remove
* Make peer deletion predicate closure immutable
* Replace logging backend to flexi-logger and add log rotation
* Changed flexi_logger to log4rs
* Disable logging level filtering in Root logger
* Support different logging levels for file and stdout
* Don't log messages from modules other than Grin-related
* Fix formatting
* Place backed up compressed log copies into log file directory
* Increase default log file size to 16 MiB
* Add comment to config file on log_max_size option
* Add peers used bandwidth calculation and display in TUI
* Fix formatting
* Change Mutex to RwLock from peer's used bandwidth statistics in Tracker
* Make used bandwidth column in TUI peers list sort by sum of bytes
We select a peer to ask a block randomly. Peer's send channel has
capacity 10. If we need too many blocks we limit number of blocks to asks as a number of peers
* 10, which means that there is some probability (pretty high) that we
will overflow send buffer capacity.
This fix freezes a peer list (which gives also some performance boost)
and create a cycle iteraror to equally distribute requests among the
peers.
There is a risk that a peer may be disconnected while we are sending a
request to the chanel, but stricltly speaking it was possible in the old
code too, perhaps with a lower probability.
Fixes#1748
* header MMR in use within txhashset itself
works with fast sync
not yet in place for initial header sync
* add the (currently unused) sync_head mmr
* use sync MMR during fast sync
rebuild header MMR after we validate full txhashset after download
* support missing header MMR (rebuild as necessary) for legacy nodes
* rename to HashOnly
* cleanup backend.append()
* simplify vec_backend to match simpler append api
* header MMR in use within txhashset itself
works with fast sync
not yet in place for initial header sync
* add the (currently unused) sync_head mmr
* use sync MMR during fast sync
rebuild header MMR after we validate full txhashset after download
* support missing header MMR (rebuild as necessary) for legacy nodes
* rustfmt
* comments/docs
* rustfmt
* cleanup DBBackend
* cleanup DBBackend
* cleanup
* rename to HashOnly
* rustfmt
* cleanup backend.append()
* simply pmmr append api
no need to pass position when appending
* cleanup
* simplify vec_backend to match simpler append api
* rustfmt
* docs/comments
* rustfmt
* cleanup
* First pass at secondary proof of work difficulty adjustments
* Core and chain test fixes
* Next difficulty calc now needs a height. Scaling calculation fixes. Setting scaling on mined block.
* Change factor to u32 instead of u64.
* Cleanup structs used by next_difficulty
* Fix header size calc with u32 scaling
* fix: avoid a confusing log when fastsync start (#1720)
(cherry picked from commit 4f462cdfdc)
* small speed optimization for header sync (#1719)
(cherry picked from commit d3589d1bf5)
* change keychain to use bip32 paths
* convert keychain to use bip32
* change identifier to be serialisation of 4-level bip32 path
* wallet changes compiling, pass parent key into all wallet functions
* rustfmt
* fix tests in chain
* rustfmt
* core tests passing
* rustfmt
* pool tests
* rustfmt
* fixing wallet tests
* rustfmt
* remove file wallet
* wallet tests compiling
* rustfmt
* remove db_migrate
* successful tx exchange test using BIP32 paths
* rustfmt
* fix wallet derivation paths to m/0/0/0
* wallet test fixed again, working with default path
* rustfmt
* fix server tests
* rustfmt
* make parent_id a trait on walletbackend
* rustfmt
* add ability for wallet to switch between multiple named accounts, and tests (not complete)
* rustfmt
* account switching tests in place and passing
* rustfmt
* compile and test with latest libsecp changes
* added public key sum to calculated e for aggsig
* rustfmt
* Update secp to 26
* bulletproof bip32 path integration
* rustfmt
* wallet restore updated with bip32 paths, also restores accounts
* rustfmt
* rustfmt
* remove old extkey
* remove old extkey
* rustfmt
* add wallet account commands
* rustfmt
* update wallet documentation
* rustfmt
* merge from master
* update libsecp tag
* merge from upstream and fix server test
* rustfmt
* rustfmt
* merge from master
* update latest libsecp merge
* fix commitment to zero value generation
* get rid of RwLock around peer instances
* rustfmt
* rename stuff to live_info
* rustfmt
* serialize into PeerInfoDisplay
limit live_info write lock to peer_info.update()
rename to PeerLiveInfo
* simplify broadcast logic a bit more
connected peers are connected by definition
* return true/false on broadcast so we can send to more peers
* rustfmt
* fix p2p and server tests for PeerInfoDisplay
* commit
* Go back to db for head and header_head
* Use batch consistently for db access
* Pass active txhashset and batch in the ctx
* Only update head in db if total work increases
* Only updated header_head if total work (on header chain) increases
Picks a random defunct peer and marks it healthy again anytime
we're querying peers to find more. Over enough time, any peer will
see another as defunct, leading to peer list atrophy. This
allows us random retries.
In addition, we need to cleanup peers after a while, we quickly
accumulate hundreds of dead peers over time. This should be
tracked by a different issue however.
Related: #1632
* Allow TLS for Wallet APIs
This PR adds an optional support of TLS for wallet APIs. Only PKCS12 format is supported, will address .pem support in next PR and provide some documentation.
Address #1425
* cuck placeholder
* rustfmt
* cuckatoo, early days
* rustfmt
* data structures are in place, siphash key creation is consistent with @tromp
* solver in place, (not yet working)
* cuckatoo test solver working with test nonce
* rustfmt
* update solver to remove adjacency list removals
* verifier functioning
* rustfmt
* Proper error handing in Cuckatoo module, couple of tests
* modify cuckoo/cuckatoo solvers and verifiers to function identically, in advance of trait refactoring
* rustfmt
* refactor PoW context into trait, default to using cuckoo context
* rustfmt
* create macros for integer casting/unwraps
* don't instantiate structs when just verifying, add test validation vector for cuckatoo 29
* rustfmt
* don't init cuckoo structs if just validating
* test fix
* ensure BH hashing for POW is only done within miner/validators
* Add api_secret
* Add to base64 method
* Add basic auth in API
* Add Basic Auth to owner API
* Add flag to enable disable basic auth
* Add .api_secret file
* Move Proof and Difficulty types to pow module. Difficulty scaling calculation.
* Diffculty scaling and size shift for 2nd PoW
* Backport e7eb26ee
* Test compilation fixes
* Scaling only need to go one way, as @tromp pointed out
Currently on startup we wait for 30 secs if we don't have at least 4 peers with more work. If a node was quickly restarted it is already fully synced so thereare no peers with more work at all. This pr assumes that if we've done some work already and we have enough peers but still 0 with more work we are fully synced and good to go.
* fix for unstable travis-ci test on servers module
* skip the PeerWithSelf connection request on sending side
* logs and comments change; and remove an unnecessary unwrap() in test
* improve: HeaderSync optimization (#1372)
* remove get_locator() optimization, which should be an independent pr for security review
* refactoring: move 'headers_streaming_body()' from Message to Protocol
* move 2 headers utils functions out of Protocol, and remove 'pub'
* support reading variable size of BlockHeader, from Cuckoo30 to Cuckoo36
* fix: use global::min_sizeshift() instead of hardcoded 30, because Cuckoo10 will be used for AutomatedTesting chain
* fix: should use global::proofsize() instead of hardcoded 42 when calculate serialized_size_of_header
* replace another 42 with global::proofsize()
* config file can now be generated by executable
* rustfmt
* remove now-unnecessary config defaults test
* set up paths and config file creation in user's home directory
* rustfmt
* remove default grin.toml
* add grin configuration command to spit out config file
* Split configuration into wallet and server
* rustfmt
* Restore logging to wallet configurations
* rustfmt
* simplify tx validation and aggregation
we *only* need to account for reward when building a block from txs
* rustfmt
* cleanup and tests passing
* rustfmt
* better comments in with_reward()
* fix wallet tests
It produces 1000+ otputs on a fast machine (?) and generates invalid URL as result (too many paramaters). We don't care about sending all outputs in this test, we just need something.
* Local test container used different folder than regular server, so test could not find txhashset files
* Error stacktrace was printed in cases when error was expected, which created some noise and confusion
* refactoring get_locator(), to ensure the gap between neighbours >= 2^n
* security enhancement for get_locator
* in case of sync head and the header head are different, clear history locators
Header sync used to run when complete (right count of blocks
received) or every 10 sec regardless. Sometimes 10 secs is not
enough, especially when a big batch of blocks bodies is arriving
at the same time.
This tweaks the time-based heuristic just a bit to only ask for
headers when stalled, meaning that the header head has not
*progressed* for 10 seconds.
* Fix bad header sync timeout logic
* Improve stability of fast sync test
Adds regular pings so that the 2 test servers know each others'
heights (necessary to find most work peer). Stop mining on the
first server once the 2nd kicks in. Avoids chasing a fast moving
target on Cuckoo10.
* Sort pool tx by fee over weight for mining. Fixes#1105
* Bucketing dependent transactions before weighing. Minor tx weight fixes.
* Limit length of tx chain, cleanup and test fixes
* Cleanup all mining references to a hardcoded tx count
* Small test improvement, cleanup
* first addition of static wallet file server
* rustfmt
* add custom build functions to download and untar particular release of the web-wallet
* rustfmt
* remove comments
* Include commitments non-duplicate checks in aggregate
* Remove said check from the pool
* Block building now uses tx aggregation to reduce duplication
* fix: in case txhashset validation fail, 'TxHashsetDownload' should retry (#1293)
And add 2 more fixes:
1. handle sending request failure of 'send_txhashset_request';
2. use a 10-mins timeout to handle noresponse failure, in case a peer ignore our txhashset request, or in case a peer fail to send a file for some reason.
