* First tests for API handlers
* Test chain UTXO handler
* Add test sumtrees handlers
* Fix typo
* Removed unused logger
* Update bitflags to ^1.0
* Missing test for bitflags update
* Moved test to grin directory
* Remove tests dependencies
* Add API P2P tests
* Fix hex string commitment
* Fix conflicting port
* Fix directory conflict and server port
* Wait for at least one block is mined on Travis
* large updates for mining, fix async mode, will list changes in PR
* reset config and build defaults
* change to difficulty calculations
* tweaking mining params and tests
* tweaking to tests
* including pre-genesis data for difficulty adjustments, adding adjustment scenario tests
* further clarifying next_difficulty function
* moving tests out of consensus.rs
* pow test fix
* changing pre-genesis generation
* [wip] short_id implementation (first attempt)
todo - make this more reusable (a trait?) so we can use it for inputs/outputs/kernels easily
* factor short_id support out into ShortIdentifiable trait
* block can now be converted to compact_block
rename existing block.compact() -> block.cut_through()
* expose compact block representation via block api endpoint
optional with ?compact query param
* experiment with lock_heights on outputs
* playing around with lock_height as part of the switch commitment hash
* cleanup
* include features in the switch commit hash key
* commit
* rebase off master
* commit
* cleanup
* missing docs
* rework coinbase maturity test to build valid tx
* pool and chain tests passing (inputs have switch commitments)
* commit
* cleanup
* check inputs spending coinbase outputs have valid lock_heights
* wip - got it building (tests still failing)
* use zero key for non coinbase switch commit hash
* fees and height wrong order...
* send output lock_height over to wallet via api
* no more header by height index
workaround this for wallet refresh and wallet restore
* refresh heights for unspent wallet outputs where missing
* TODO - might be slow?
* simplify - do not pass around lock_height for non coinbase outputs
* commit
* fix tests after merge
* build input vs coinbase_input
switch commit hash key encodes lock_height
cleanup output by commit index (currently broken...)
* is_unspent and get_unspent cleanup - we have no outputs, only switch_commit_hashes
* separate concept of utxo vs output in the api
utxos come from the sumtrees (and only the sumtrees, limited info)
outputs come from blocks (and we need to look them up via block height)
* cleanup
* better api support for block outputs with range proofs
* basic wallet operations appear to work
restore is not working fully
refresh refreshes heights correctly (at least appears to)
* wallet refresh and wallet restore appear to be working now
* fix core tests
* fix some mine_simple_chain tests
* fixup chain tests
* rework so pool tests pass
* wallet restore now safely habndles duplicate commitments (reused wallet keys)
for coinbase outputs where lock_height is _very_ important
* wip
* validate_coinbase_maturity
got things building
tests are failing
* lite vs full versions of is_unspent
* builds and working locally
zero-conf - what to do here?
* handle zero-conf edge case (use latest block)
* introduce OutputIdentifier, avoid leaking SumCommit everywhere
* fix the bad merge
* pool verifies coinbase maturity via is_matured
this uses sumtree in a consistent way
* cleanup
* add docs, cleanup build warnings
* fix core tests
* fix chain tests
* fix pool tests
* cleanup debug logging that we no longer need
* make out_block optional on an input (only care about it for spending coinbase outputs)
* cleanup
* bump the build
* Make total_difficulty the sum of network difficulty, not whatever the miner happened to mine.
- Only for Testnet2 / Mainnet (hardforks Testnet1)
- update chain::pipe validate_header to validate according to Testnet2 rules for cumulative difference
Fixes#280
* tests that should ignore (network) difficulty
* fn new_block is explained as "utility to create a block without worrying about the key or previous header" so it gets network difficulty := 0 too
* update tx pool tests (going with "minimum" for network difficulty for now)
* add ERR outputs about bannable offences
(#406 should know about these)
* whitespace fix
* mine_simple_chain: Probably DON'T overwrite difficulty (?)
* core/mod tests "reward_empty_block" and "reward_with_tx_block" tests set to use lowest network difficulty possible
* First steps converting transaction workflow to be aggsig-enable
* integrating updated version of aggsig, which gives greater control over the contents of e
* added wallet transaction test to testing framework to enable testing the whole thing, completed interaction as far as inital response from recipient
* more aggsig work, final signature is produced now
* Construction of aggsig transaction now working to the point of the signature being built
* aggsig transactions working end-to-end in the nominal case
* refactor aggsig verify from commit and fix some tests
* more cleanup and test fixing
* cleaning up automated tests
* test+formatting fix
* added debug logging to see what we're doing in save_pos_index
* add counts to logging
* make sure inputs/outputs in tx are sorted
tweak debug logging
* DRY up sorting of inputs/outputs/kernels via a simple sort() in ser/deser
* add explicit validation fo sort order during tx and block validation
* sort order consensus rule for inputs/outputs/kernels in blocks
* we want a max of 10*num_peers blocks during sync...
not min
* fix test imports
* Very quick peer banning endpoint, helps with #406
* Ping heights (#407)
* add height to ping/ping
* reformat output
* fix p2p test
* Fix orphan handling, not related to current head. Fixes#412
* Check before borrow, fixes#267
* Not finding an output commit in pos is an AlreadySpent
* Fix race condition, sending before conn is ready
* Explicit error for unknown pos of a forked block
* Remove config outdated tests. Fix#333
* Check ref and try before borrow, fix#400
* We do not want to sync with old peers anyway
* Hide cargo compiler warning for unused NoopAdapter and unused test code. Add TODOs
* Limit number of inputs in a block, fix#261
* MAX_BLOCK_INPUTS = 300_000 // IO-time bound DoS protection. 5MB/s disk random reads gives ~150k UTXO lookups/s, so set limit at 300k UTXO inputs
* beginning to add wallet restore... api endpoints and basic restore
* basic restore working, still missing features
* rustfmt
* large speed up to output search, should be more or less working
* properly mark coinbase status
Renamed mining parameter mode to chain type, with existing types
of CI testing, user testing, testnet1 or mainnet. The public
chains (testnet1 and mainnet) come with their fully pre-defined
genesis block.
Still need to set the nonce and cycle for testnet1 genesis.
* allow selecting a commit while providing a key index
* misnamed variable
* added static reference to libsecp that can be called throughout
* don't serialise rangeproof to json if it's not desired
* forgotten new file
* amounts input and displayed in wallet are now in full grins, with optional decimal place
* rustfmt
* merge branch
* better acknowledgement of transaction being sent
* allow selecting a commit while providing a key index
* added static reference to libsecp that can be called throughout
* don't serialise rangeproof to json if it's not desired
Moved the HTTP APIs away from the REST endpoint abstraction and
to simpler Hyper handlers. Re-established all routes as v1.
Changed wallet receiver port to 13415 to avoid a gap in port
numbers.
Finally, rustfmt seems to have ignored specific files arguments,
running on everything.
* adding more useful handlers
* added method to return last n leaf nodes inserted into the sum tree
* endpoints in place for getting last n sumtree nodes
* Fix for the chain pipeline partly relying on an outdated head,
leading to not properly recognizing a fork and inconsistent sum
tree state.
* Do not drop block requests during sync that don't get satisfied,
retry enough time to get them and avoid stall.
* Always validate header, even in sync where we may have validated
it already. We don't want a block coming from a peer that could
squeeze through with an invalid header.
* When syncing, do not mark blocks that were errored by the chain
as received (typical case: orphan). Keep retrying.
* Improved chain state dump for debugging.
* Do not add to orphans blocks too far in the future.
* Better error reporting on db errors.
* Related sync test fixes.
TODO figure out why syncing peers timeout so often, very useful
to test but not that great for a fast sync experience.
* Adding switch commit to grin outputs
* logging output fix
* adding switch commitment hash to sum tree node
* added hash_with to Hashed trait, to allow for hashing to include another writeable element
* adding hash_with as method in hashed trait
* fail deserialization if input or outputs are not sorted for txs, fail deserialization if inputs or outputs or kernels are not sorted for blocks, make sure we sort them before serialization
* extract consensus rule into verify_sort_order on VerifySortOrder trait
* rework the sort and write into WriteableSorted trait
* fix wallet sending bug (fees related) so we can test
* refactored read_and_verify_ordered out into ser, used in transaction and block
* rename pubkey -> key_id, these are key identifiers, not public keys
* reduce identifier seize to 10 bytes, get rid of fingerprints (identifiers are now small enough to use as-is)
* IDENTIFIER_SIZE const
* add FeeDispute error for when sender and recipient disagre on fee calculation (should never happen)
* does not need to be mut
* cleaned up some warnings
* add lock_height to Transaction and TxKernel, sign msg including both fee and lock_height in kernel
* make the order of the fields in tx and kernel more consistent
* rename to kernel_sig_msg
* add test to cover adding timelocked tx to pool, where tx is invalid based on current height of the blockchain
* add tests for adding timelocked txs to blocks (valid and otherwise)
* store wallet output data in hashmap
* cleanup up commented out code
* pass pubkey/identifier and not derivation around to miner etc.
* fix failing tests
Update coinbase building and block summation to account for half of
the fees going to the coinbase. Forcing fees to be even as a
consequence. Now that we can't build the coinbase independently
from the block (because fees), had to update the miner to keep the
key derivation so a new derivation isn't made any time a new block
gets worked on.
Minor doc and warning cleanups.
* introduce grin_keychain, encapsulate derivation of secret_keys
* core compiles against keychain, tests don't run yet
* core tests are now passing against keychain
* wip - getting wallet working with keychain
* add util and keychain to travis test matrix
* basic test around key derivation
* Integrate PMMR and its persistent backend with the Chain
* Chain can set tree roots; PMMR backend discard
* Check spent and prune for each input in new block
* Handling of forks by rewinding the state
* More PMMR tests and fixes, mostly around rewind
* Rewrite get_unspent to use the sumtrees, fix remaining compilation issues
* workaround the "commitment is not a public key" problem by using to_two_keys() from rust-secp256k1-zkp
* remove secp256k1zkp, update the various Cargo.toml to use rust-secp256k1-zkp from external git repo, update code to not rely on to_pubkey (and use verify_from_commit)
* secp256k1zkp from git
* cleanup secp256k1 dependencies
* gitignore node1 and node2 dirs
* use head_header in add_to_memory_pool
* add COINBASE_MATURITY const to consensus
* add coinbase maturity (wip) validaton rule to validate_block
* add coinbase maturity check to validate_block
* map errors in adapters - specific errors still wip
* reworked so adapter translates chain errors to pool errors (core errors not required)
* add test for spending immature coinbase in memory pool
* wip - add test to cover spending coinbase output in chain.process_block
* added test coverage around process_block - we have a problem with coinbase output commitments
* add a comment on the failing test
* process_block will now fail validation if we attempt to spend coinbase that has not yet matured
(remember to use a new reward_key for every block). test coverage in place to verify this
* Base MMR storage structures
Implementations of the MMR append-only file structure and its
remove log. The append-only file is backed by a mmap for read
access. The remove log is stored in memory for quick checking
and backed by a simple file to persist it.
* Add PMMR backend buffer, make PMMR Backend mutable
* The Backend trait now has &mut self methods, and an &mut
reference in PMMR. This simplifies the implementation of all
backends by not forcing them to be interior mutable. Slight
drawback is that a backend can't be used directly as long as it's
used by a PMMR instance.
* Introduced a buffer in the PMMR persistent backend to allow
reads before the underlying files are fully flushed. Implemented
with a temporary VecBackend.
* Implement a prune list to use with dense backends
The PruneList is useful when implementing compact backends for a PMMR (for
example a single large byte array or a file). As nodes get pruned and
removed from the backend to free space, the backend will get more compact
but positions of a node within the PMMR will not match positions in the
backend storage anymore. The PruneList accounts for that mismatch and does
the position translation.
* PMMR store compaction
Implement actual pruning of the underlying PMMR storage by
flushing the remove log. This triggers a rewrite of the PMMR nodes
data (hashes and sums), removing pruned nodes. The information of
what has been removed is kept in a prune list and the remove log
is truncated.
* PMMR store pruning tests and fixes
* initial pass at indexing block header hashes by commitments
* check block height matches currently indexed block height
when getting block header for an output commit
* add some test coverage
* document get_block_header_by_output_commit
* Adding output stats from plugin, when available
* adding grin config
* moving pow crate from core/pow to it's own crate
* moving POW/mining stuff into pow module, and removing mining dependency from chain
* refactored most of mining into pow module...miner.rs still in server
* update grin tests
* updating genesis block mining to use configured miner, if available
* chain tests back into chain, done with pow refactor
* use tag of cuckoo-miner for pr
* playing around with changing cuckoo sizes on the fly
* modifying tests to use global cuckoo parameters, and checking results
* check for pow size
* Changing global function names, and removing length from POW serialization
* changes to use blake2 for cuckoo seed generation
* updating tests to blake 2
* minor cleanup - unused imports
* cleanup build warnings - unused vars
* make structs pub to get rid of the private_in_public lint warning
* missing docs on RangeProof
* add missing docs to store delete function
* cleaned up deprecation warning -
tokio_core -> tokio_io
complete() -> send()
* playing around with changing cuckoo sizes on the fly
* modifying tests to use global cuckoo parameters, and checking results
* check for pow size
* Changing global function names, and removing length from POW serialization
* Adding ability to serialise parts of the header, pre-nonce and post-nonce
* Some test integration of queueing functions in cuckoo-miner
* more cuckoo-miner async mode integration, now more or less working
* integrating async miner workflow
* rocksdb update
* u64 internal difficulty representation, and integration of latest Cuckoo-miner API
* change to cuckoo-miner notify function
* Issue in testing, and if use_async value is None in grin.toml
* making async mode explicit in tests - 2
* fiddle with port numbers for CI
* update tag to ensure cuckoo-miner build doesn't fail on windows
* change the order in which tests are run
Couple improvements to the sum tree implementation. First change
is to not store the data but only its hashes, assuming a higher
level layer will take care of the data (KV store).
Second minor improvement is renaming Node into NodeData and vice
versa.
* Adding cuckoo-miner into grin as a proper dependency now
* Defaulting to using cuckoo-miner
* Updates to tests to use cuckoo_miner by default, (using cuckoo16)
* core: partially implement sumtree; appending and pruning work
* core: add simple merkle-root-sum-calculating function.
* core: implement replacement in sumtree
* core: implement de/serialization for sumtree
* core: make sumtree index use hashes rather than data
* core: prevent double-adds to sumtrees
* core: replace sumtree's separate sums with a Summable trait
* core: make SumTree::contains() return the index if an element exists
* Refactoring to allow for different miner implementations. Added conditional support for compiling and loading the cuckoo-miner plugin project.
* Small changes to experimentally integrate with cuckoo-miner and compatibility with latest version of cuckoo-miner.
* Turning off inclusion of cuckoo_miner by default
* Disabling simulate_parallel_mining test for now
* Beginning to add a POW description, and some minor changes to mining testing, addition of further debug information to mining output.
* Many additions to create first draft of POW documentation
* Fixes to difficult adjustments by adding a MINIMUM_DIFFICULTY consensus value. Otherwise never adjusted above 1 due to flooring.
Integrate the new difficulty calculation into the block chain
validation, the miner and tests. As the difficulty calculation
doesn't use varying Cuckoo sizes anymore and we narrowed down
reasonable final Cuckoo Cycle parameters, removed all Cuckoo
Cycle sizes from block headers.
Formalized easier Cuckoo Cycle sizes for testing (and possibly
testnet) by introducing a test mode in configuration. Updated
all tests.
* Transactions coming from the network are now pushed to the pool
through the net adapter.
* New blocks accepted by the chain are sent to the pool for
eviction.
* The miner requests transactions from the pool to build its
blocks.
* The push API adds to the pool, removing the mock.
* Implementation of the adapter to the chain required by the pool
to get consistent UTXOs. Grossly unoptimized until we have the UTXO
MMR ready.
