* experiment with lock_heights on outputs
* playing around with lock_height as part of the switch commitment hash
* cleanup
* include features in the switch commit hash key
* commit
* rebase off master
* commit
* cleanup
* missing docs
* rework coinbase maturity test to build valid tx
* pool and chain tests passing (inputs have switch commitments)
* commit
* cleanup
* check inputs spending coinbase outputs have valid lock_heights
* wip - got it building (tests still failing)
* use zero key for non coinbase switch commit hash
* fees and height wrong order...
* send output lock_height over to wallet via api
* no more header by height index
workaround this for wallet refresh and wallet restore
* refresh heights for unspent wallet outputs where missing
* TODO - might be slow?
* simplify - do not pass around lock_height for non coinbase outputs
* commit
* fix tests after merge
* build input vs coinbase_input
switch commit hash key encodes lock_height
cleanup output by commit index (currently broken...)
* is_unspent and get_unspent cleanup - we have no outputs, only switch_commit_hashes
* separate concept of utxo vs output in the api
utxos come from the sumtrees (and only the sumtrees, limited info)
outputs come from blocks (and we need to look them up via block height)
* cleanup
* better api support for block outputs with range proofs
* basic wallet operations appear to work
restore is not working fully
refresh refreshes heights correctly (at least appears to)
* wallet refresh and wallet restore appear to be working now
* fix core tests
* fix some mine_simple_chain tests
* fixup chain tests
* rework so pool tests pass
* wallet restore now safely habndles duplicate commitments (reused wallet keys)
for coinbase outputs where lock_height is _very_ important
* wip
* validate_coinbase_maturity
got things building
tests are failing
* lite vs full versions of is_unspent
* builds and working locally
zero-conf - what to do here?
* handle zero-conf edge case (use latest block)
* introduce OutputIdentifier, avoid leaking SumCommit everywhere
* fix the bad merge
* pool verifies coinbase maturity via is_matured
this uses sumtree in a consistent way
* cleanup
* add docs, cleanup build warnings
* fix core tests
* fix chain tests
* fix pool tests
* cleanup debug logging that we no longer need
* make out_block optional on an input (only care about it for spending coinbase outputs)
* cleanup
* bump the build
* Make total_difficulty the sum of network difficulty, not whatever the miner happened to mine.
- Only for Testnet2 / Mainnet (hardforks Testnet1)
- update chain::pipe validate_header to validate according to Testnet2 rules for cumulative difference
Fixes#280
* tests that should ignore (network) difficulty
* fn new_block is explained as "utility to create a block without worrying about the key or previous header" so it gets network difficulty := 0 too
* update tx pool tests (going with "minimum" for network difficulty for now)
* add ERR outputs about bannable offences
(#406 should know about these)
* whitespace fix
* mine_simple_chain: Probably DON'T overwrite difficulty (?)
* core/mod tests "reward_empty_block" and "reward_with_tx_block" tests set to use lowest network difficulty possible
* First steps converting transaction workflow to be aggsig-enable
* integrating updated version of aggsig, which gives greater control over the contents of e
* added wallet transaction test to testing framework to enable testing the whole thing, completed interaction as far as inital response from recipient
* more aggsig work, final signature is produced now
* Construction of aggsig transaction now working to the point of the signature being built
* aggsig transactions working end-to-end in the nominal case
* refactor aggsig verify from commit and fix some tests
* more cleanup and test fixing
* cleaning up automated tests
* test+formatting fix
Due to the construction of PMMRs the last element, when its a leaf,
can never be pruned as it has no parent yet and it will be needed
to calculate that hash. To work around this, we now insert coinbase
outputs first to add at least one output of padding.
Also changed the `set_sumtree_root` function on chain a bit to allow
setting the roots on a fork. Mostly useful for tests.
Added new test case to handle both the issue above and spending
transactions within a fork.
When applying a fork, everything is done in memory to be able to
rewind if it's either invalid or doesn't have more work. But
checking for the UTXO index was done only against the store. Now
checking the memory as well to find UTXOs that haven't been
stored yet.
The method `is_unspent` hadn't been fixed with the other sumtree
functions to check the MMR before deciding whether something is
really unspent. This is now fixed and also checks the output
hash is the one we expect.
* added debug logging to see what we're doing in save_pos_index
* add counts to logging
* make sure inputs/outputs in tx are sorted
tweak debug logging
* DRY up sorting of inputs/outputs/kernels via a simple sort() in ser/deser
* add explicit validation fo sort order during tx and block validation
* sort order consensus rule for inputs/outputs/kernels in blocks
* we want a max of 10*num_peers blocks during sync...
not min
* fix test imports
* wip
* rework check_orphans to be smart about _which_ orphan(s) to check
* cleanup
* limit max of 100 blocks at a time, and corresponding 100 max orphan blocks
* distinguish select (among futures) from select coins. Regex search in project for select\b shows we hardly use select, but maybe could use it to add timeouts more cleanly. ("Want to add a timeout to any future? Just do a select of that future and a timeout future!" from https://aturon.github.io/blog/2016/08/11/futures/)
* remove a trailing space
* FAQ.md - fix typo
* wallet: display problematic tx
* update FAQ build troubleshooting to cover #443
* stdout_log_level = Info
file_log_level = Debug
* sync: show total diff @ height when syncronization is completed
* better wallet send dest format error
* move INFO "Client conn ... lost" and "Connected to peer" down to Debug
* move some level=Info to Debug, add 1000-block outputs
* timeout connection already tracks "expected" responses
use this to deduplicate requests and do not ask a peer for the same thing again
(until either success or timeout)
* do not ask for orphan blocks repeatedly
allow more than preferred number of peers (clean if we exceed max number)
Without a backup, the in-memory data structure stays truncated
even if the rewind is abandoned. Also add some logging on our
most problematic block in case it still is problematic.
* Very quick peer banning endpoint, helps with #406
* Ping heights (#407)
* add height to ping/ping
* reformat output
* fix p2p test
* Fix orphan handling, not related to current head. Fixes#412
* Check before borrow, fixes#267
* Not finding an output commit in pos is an AlreadySpent
* Fix race condition, sending before conn is ready
* Explicit error for unknown pos of a forked block
* Remove config outdated tests. Fix#333
* Check ref and try before borrow, fix#400
* We do not want to sync with old peers anyway
* Hide cargo compiler warning for unused NoopAdapter and unused test code. Add TODOs
When rewinding, it's possible to have a commitment (or kernel)
identical to the one we're adding both in the index and in the
data file (because it's not truncated by rewind, by design). To
detect this, we just need to make sure we don't look for anything
past the size of the rewound MMR.
Added support for peer banning on the p2p server. The peer status
is changed and the peer is disconnected. A banned peer won't be
able to reconnect as well.
Tracking of chain errors due to a block that's intrinsically bad
and banning of the peer that sent it. If we're syncing, resetting
the header chain to the same as the main chain to force
backtracking.
* port simple_sync across from testnet1, sync head & improved fork handling
* introduce new sync_head for tracking header chain during sync
* add debug logging in body_sync for state of various heads
* update header_head during sync if header is now the one with most work, pass in sync and head contexts to pipe
* port across sync_head changes from master, put the 1s sleep back in...
* let sync run even if the full 512 headers are all known, give sync a chance to sync against a fork that forked a long time ago
* handle height jumping during a reorg
block validaton should not check for height+1 based on head
no assumptions should be made about height indices
* quick(er) check for previous block in process_block (we cannot check height against ctx head)
* make body_sync a _lot_ faster by finding the forked block more efficiently...
* fix monitoring peers log msg
* fix chain tests
* fix grin tests - we were using the wrong genesis hash (wrong chain type)
* apparently needs setting in both places...
* body -> header -> sync (#414)
* port over body -> header -> sync changes from testnet1
* port across "simple_sync" changes from testnet1 branch
Cleanup direct refs to peer map or peer store
P2P server acts as a facade, handling the list of connected peers
and the storage of their information. Everything else goes through
the p2p server instead of having a peer map reference or going
straight to the store.
Fix p2p tests
* fix "monitoring peers" log msg (use connected_peers)
Duplicate commitment and ducplicate kernel errors, as well as
`is_unspent` did not check the total PMMR size, leading to
wrong detection of duplicates or unspent.
* beginning to add wallet restore... api endpoints and basic restore
* basic restore working, still missing features
* rustfmt
* large speed up to output search, should be more or less working
* properly mark coinbase status
* ensure only unspent utxos are returned from sumtree
Fixes#281 by doing:
- "Starting validation pipeline for " -> "Processing "
- stdout logging default = Warning
- ERR -> INFO for "Transaction rejected: Already in pool" -- only for netadapter transaction_received, which (hopefully!) only gets tx from peers, and lots of those will be duplicates, that's good and not an error.
* Downgrade ERR even down to DEBUG.
Also let Transaction rejected show tx hash,
so users can check if such a rejection is for the
payment they expected, or just any random peer
sending in some old and already known tx.
Renamed mining parameter mode to chain type, with existing types
of CI testing, user testing, testnet1 or mainnet. The public
chains (testnet1 and mainnet) come with their fully pre-defined
genesis block.
Still need to set the nonce and cycle for testnet1 genesis.
* bad case of trying to rewind to block header height 0
* rewind_to_genesis appears to work
* do not assume genesis block at height 0 is empty, pass full block in to rewind, check for last output and kernel, use index 0 if block is empty
* allow selecting a commit while providing a key index
* added static reference to libsecp that can be called throughout
* don't serialise rangeproof to json if it's not desired
Moved the HTTP APIs away from the REST endpoint abstraction and
to simpler Hyper handlers. Re-established all routes as v1.
Changed wallet receiver port to 13415 to avoid a gap in port
numbers.
Finally, rustfmt seems to have ignored specific files arguments,
running on everything.
* adding more useful handlers
* added method to return last n leaf nodes inserted into the sum tree
* endpoints in place for getting last n sumtree nodes
* Fix for the chain pipeline partly relying on an outdated head,
leading to not properly recognizing a fork and inconsistent sum
tree state.
* Do not drop block requests during sync that don't get satisfied,
retry enough time to get them and avoid stall.
* Always validate header, even in sync where we may have validated
it already. We don't want a block coming from a peer that could
squeeze through with an invalid header.
* When syncing, do not mark blocks that were errored by the chain
as received (typical case: orphan). Keep retrying.
* Improved chain state dump for debugging.
* Do not add to orphans blocks too far in the future.
* Better error reporting on db errors.
* Related sync test fixes.
TODO figure out why syncing peers timeout so often, very useful
to test but not that great for a fast sync experience.
Addresses a corner case of sync. If we're still in sync mode but
just caught up to the head, a block could be added through normal
gossip. So we can't short-circuit some of the header handling
even though during sync the header should have already been
validated and saved, because we can still get a block from gossip.
