Commit graph

70 commits

Author SHA1 Message Date
Yeastplume
05d1c6c817
Take the 'Sum' out of 'Sumtree' (#702)
* beginning to remove sum

* continuing to remove sumtree sums

* finished removing sums from pmmr core

* renamed sumtree files, and completed changes+test updates in core and store

* updating grin/chain to include removelogs

* integration of flatfile structure, changes to chain/sumtree to start using them

* tests on chain, core and store passing

* cleaning up api and tests

* formatting

* flatfiles stored as part of PMMR backend instead

* all compiling and tests running

* documentation

* added remove + pruning to flatfiles

* remove unneeded enum

* adding sumtree root struct
2018-02-22 13:45:13 +00:00
Antioch Peverell
4022b82817
cleanup various build warnings (#714)
* cleanup various build warnings

* use serde_json in tests only
2018-02-17 12:56:22 -05:00
Ignotus Peverell
76796738c1
Clean server shutdown, generalizes usage of Weak (#700)
* Clean server shutdown, generalizes usage of `Weak`. Introduces 2 main changes:

  * A shared `AtomicBool` that signals server shutdown. All server
threads regularly check it to break out of their main loops when
it changes to `true`.
  * Breaking of circular `Arc` references, which can never be
destroyed, by downgrading to `Weak` instead. Only the main server
keeps the `Arc` while all other components get the `Weak` variant.

Both of these are required for all long-living structs to be
cleanly destroyed. Note that in Rust this is fairly important as
most resource-freeing logic is associated with `drop`,
which is only called when said struct is free of scope or `Arc`
references.

Should address most of #536 (only need the stop hook to call
`Server` shutdown).

* Test for fast sync, followed by restart, followed by re-fast-sync

* P2P test fix

* Double sync taking too long for Travis, commenting out for now
2018-02-13 00:38:52 +00:00
Quentin Le Sceller
fb46fad0ac Remove unused import and uniformize crates (#686)
* Removed unused crates
* Add listconnectedpeers in grin client
* Removed unused import and uniformize crates
2018-02-05 19:46:56 +00:00
Quentin Le Sceller
8a7eb94759 Update bitflags to ^0.1 (#682)
* Removed unused crates
* Add listconnectedpeers in grin client
* Update bitflags to ^0.1 globally
2018-02-05 19:43:54 +00:00
Quentin Le Sceller
33cb0902bd Fix 672 (#673) 2018-01-31 13:29:45 -05:00
Antioch Peverell
c75026153c
only include kern_ids in compact block (we can safely omit input and outputs ids) (#670) 2018-01-31 10:23:42 -05:00
Matthew Slipper
d754b8d1e4 Fix #566: serialize commit to hex (#574)
* Fix #566: Use serdes to serialize commit to hex

* Add printable UTXO
2018-01-29 09:36:09 -05:00
AntiochP
ba4c450d2f
coinbase output(s) and kernel(s) always full in compact block (#641)
all other tx inputs/outputs/kernels can be reduced to their short_ids
2018-01-19 22:29:48 -05:00
AntiochP
9085e548f7
ShortId implementation (and CompactBlock) (#637)
* [wip] short_id implementation (first attempt)
todo - make this more reusable (a trait?) so we can use it for inputs/outputs/kernels easily

* factor short_id support out into ShortIdentifiable trait

* block can now be converted to compact_block
rename existing block.compact() -> block.cut_through()

* expose compact block representation via block api endpoint
optional with ?compact query param
2018-01-19 17:43:02 -05:00
AntiochP
cbd3b2ff87
hash (features|commitment) in output mmr (#615)
* experiment with lock_heights on outputs

* playing around with lock_height as part of the switch commitment hash

* cleanup

* include features in the switch commit hash key

* commit

* rebase off master

* commit

* cleanup

* missing docs

* rework coinbase maturity test to build valid tx

* pool and chain tests passing (inputs have switch commitments)

* commit

* cleanup

* check inputs spending coinbase outputs have valid lock_heights

* wip - got it building (tests still failing)

* use zero key for non coinbase switch commit hash

* fees and height wrong order...

* send output lock_height over to wallet via api

* no more header by height index
workaround this for wallet refresh and wallet restore

* refresh heights for unspent wallet outputs where missing

* TODO - might be slow?

* simplify - do not pass around lock_height for non coinbase outputs

* commit

* fix tests after merge

* build input vs coinbase_input
switch commit hash key encodes lock_height
cleanup output by commit index (currently broken...)

* is_unspent and get_unspent cleanup - we have no outputs, only switch_commit_hashes

* separate concept of utxo vs output in the api
utxos come from the sumtrees (and only the sumtrees, limited info)
outputs come from blocks (and we need to look them up via block height)

* cleanup

* better api support for block outputs with range proofs

* basic wallet operations appear to work
restore is not working fully
refresh refreshes heights correctly (at least appears to)

* wallet refresh and wallet restore appear to be working now

* fix core tests

* fix some mine_simple_chain tests

* fixup chain tests

* rework so pool tests pass

* wallet restore now safely habndles duplicate commitments (reused wallet keys)
for coinbase outputs where lock_height is _very_ important

* wip

* validate_coinbase_maturity
got things building
tests are failing

* lite vs full versions of is_unspent

* builds and working locally
zero-conf - what to do here?

* handle zero-conf edge case (use latest block)

* introduce OutputIdentifier, avoid leaking SumCommit everywhere

* fix the bad merge

* pool verifies coinbase maturity via is_matured
this uses sumtree in a consistent way

* cleanup

* add docs, cleanup build warnings

* fix core tests

* fix chain tests

* fix pool tests

* cleanup debug logging that we no longer need

* make out_block optional on an input (only care about it for spending coinbase outputs)

* cleanup

* bump the build
2018-01-16 22:03:40 -05:00
Yeastplume
9704cc35bd Aggsig Serialization (#618)
* changing ser of aggsig signature
* serialise Signature as raw 64 bytes
* remove compact sig
* remove to_compact
* remove unused import
2018-01-15 20:45:26 +00:00
Quentin Le Sceller
cbc40c9824 Update ``grin client status`` (#599)
* Add status type
* Add /v1/status handler
* Updated client for API::Status type
* Remove unused dependency
* Moved user agent up
2018-01-11 05:25:48 +00:00
Ignotus Peverell
3515bf748c
Improved receive error handling
Better errors and messages returned by the push tx API. And better
message in the command line receive. Fixes #585
2018-01-06 23:27:21 +00:00
AntiochP
26c2669fe8
pmmr should not depend on switch_commit_hash to identify utxo (#576)
* pmmr does not depend on switch_commit_hash
for a given chain a commitment shold be sufficient to identify a utxo

* fix the tests
2018-01-04 13:38:46 -05:00
Quentin Le Sceller
1c3034b17f Fix typo ''atheight'' instead of ''byheight'' in utxo handler (#577)
* Fix typo atheight instead of byheight in utxo handler

* Change atheight to byheight
2018-01-04 10:39:30 -05:00
Quentin Le Sceller
c8383dac93 Add POST peers/a.b.c.d/unban in REST API (#571)
Add unban peer in rest API and clean TODO
2018-01-04 03:25:14 +00:00
Quentin Le Sceller
7b13b25782 Add GET peers/a.b.c.d handler in REST API (#564)
* Change name of get_peer to get_connected_peer and add a get_peer method with store
* Change to get_connected_peer
* Added handler for GET peers/a.b.c.d
2018-01-03 01:03:44 +00:00
yeastplume
f08d386790 semversion regex dependency, as rocksdb build was failing as a result 2017-12-31 10:35:23 +00:00
Ignotus Peverell
707a2073a5
Fix is_unspent to consider MMR and not only index
The method `is_unspent` hadn't been fixed with the other sumtree
functions to check the MMR before deciding whether something is
really unspent. This is now fixed and also checks the output
hash is the one we expect.
2017-12-25 00:31:51 +00:00
Simon B
1159834321 update api index (GET /v1) to also list peer banning (#486) 2017-12-14 12:44:06 -05:00
Ignotus Peverell
e50703d79e Externalized all peers selection and handling (#468)
Moved handling to the peer map out of the p2p server and into
its own struct. Allowed factoring code from the net adapter and
simplification of some interactions. Also removes the need for
the adapter to reference the p2p server or peers.

Fixes #430, #453 and #456
2017-12-14 01:13:05 +01:00
Ignotus Peverell
4a03b90190 Name all threads 2017-12-14 01:13:04 +01:00
Simon B
17d5898677 Forgotten testnet1 cherries (#475)
* Very quick peer banning endpoint, helps with #406
* Ping heights (#407)
* add height to ping/ping
* reformat output
* fix p2p test
* Fix orphan handling, not related to current head. Fixes #412
* Check before borrow, fixes #267
* Not finding an output commit in pos is an AlreadySpent
* Fix race condition, sending before conn is ready
* Explicit error for unknown pos of a forked block
* Remove config outdated tests. Fix #333
* Check ref and try before borrow, fix #400
* We do not want to sync with old peers anyway
* Hide cargo compiler warning for unused NoopAdapter and unused test code. Add TODOs
2017-12-13 21:52:21 +00:00
AntiochP
8e6f54779e
not sure how this ever worked... (#465) 2017-12-11 12:54:26 -05:00
AntiochP
001fd3789c
specify serde_json as "=1.0.7" to pin the version (#451) 2017-12-09 11:59:54 -05:00
Johnny Gannon
1ac8aa9978 Revert BlockHeaderInfo struct to contain only the fields needed for wallet reconstruction. Create a new struct for use in the /blocks api. (#442) 2017-12-07 09:27:55 +00:00
Yeastplume
f7866d9f32
fix to header info for wallet restore (#435) 2017-12-06 12:54:33 +00:00
Johnny Gannon
72fdceb0d6 API endpoints to browse blocks (#416)
* Implement /block api endpoint displaying basic information
* Add block inputs and kernels to the api output.
* Add fields to BlockHeaderInfo and TxKernelPrintable
* Add features debug string to TxKernelPrintable.
* Return 400 and 404 statuses from the blocks api endpoint.
* For the blocks api, return a 404 if a block is not found at the requested height
* Add back hash to BlockHeader api output.
2017-12-05 18:55:32 +00:00
AntiochP
442ef3b255
port across "simple_sync" changes from testnet1 branch (#409)
* port across "simple_sync" changes from testnet1 branch
Cleanup direct refs to peer map or peer store
P2P server acts as a facade, handling the list of connected peers
and the storage of their information. Everything else goes through
the p2p server instead of having a peer map reference or going
straight to the store.
Fix p2p tests

* fix "monitoring peers" log msg (use connected_peers)
2017-11-30 10:27:50 -05:00
AntiochP
6352dfbac9
add total_diff to ping/pong msgs (#350)
* add total_diff to ping/pong msgs
debug log for total_diff on each ping/pong

* expose peer addr to the handle_payload fn
so we know where it came from

* fix p2p tests for ping

* default to 0 if we cannot read total_difficulty

* updating a connected peer in place

* actually update peer info diff

* fixup p2p tests
2017-11-21 09:24:29 -05:00
Yeastplume
6ad403fbf8
Wallet recovery return unspent only (#340)
* beginning to add wallet restore... api endpoints and basic restore

* basic restore working, still missing features

* rustfmt

* large speed up to output search, should be more or less working

* properly mark coinbase status

* ensure only unspent utxos are returned from sumtree
2017-11-20 10:38:49 +00:00
Yeastplume
4b3a374d98
Wallet Restore feature (#338)
* beginning to add wallet restore... api endpoints and basic restore

* basic restore working, still missing features

* rustfmt

* large speed up to output search, should be more or less working

* properly mark coinbase status
2017-11-20 00:50:09 +00:00
Simon B
7986829d58 Quick placeholder api route index (#286) (#324)
* Aiming for #286 - quick placeholder to have at least something to help your memory

* corrections:pool/push get->post, add sumtrees/(roots|lastutxos|lastrangeproofs|lastkernels)
2017-11-19 15:24:14 -05:00
Yeastplume
48a60858ba Wallet output selection performance (#238)
* allow selecting a commit while providing a key index
* added static reference to libsecp that can be called throughout
* don't serialise rangeproof to json if it's not desired
2017-11-07 11:48:37 -05:00
AntiochP
8a42a692ce add new /v1/peers endpoints to the api (#225)
* add new /v1/peers endpoint to the api
* /peers/connected and /peers/all endpoints
2017-11-02 12:49:33 -04:00
AntiochP
9e36b820f6
fix the wallet receiver api (#222) 2017-11-01 14:32:34 -04:00
Ignotus Peverell
e4ebb7c7cb
Cleanup HTTP APIs, update ports to avoid gap, rustfmt
Moved the HTTP APIs away from the REST endpoint abstraction and
to simpler Hyper handlers. Re-established all routes as v1.
Changed wallet receiver port to 13415 to avoid a gap in port
numbers.

Finally, rustfmt seems to have ignored specific files arguments,
running on everything.
2017-10-31 19:42:56 -04:00
Yeastplume
05d22cb632
changing secp import to tag, and consolidating Cargo.toml import (#220) 2017-10-31 23:20:55 +00:00
Yeastplume
7f8d307cc8 Sum tree and improved chain API Endpoints (#214)
* adding more useful handlers
* added method to return last n leaf nodes inserted into the sum tree
* endpoints in place for getting last n sumtree nodes
2017-10-27 21:57:04 +00:00
AntiochP
8b324f7429 add retry logic to miner when hitting wallet coinbase API (#213)
* mount v2 router for flexibility, wallet checker now refreshes multiple outputs via single api call
* fix the api router
* wallet api handlers, miner uses wallet_client
* retry logic via tokio_retry, miner creates new coinbase output via wallet API (retries several times)
* move wallet client into wallet crateand rework the lock acquisition logic to use tokio_retry
2017-10-27 17:36:03 +00:00
AntiochP
7178b400b8 refresh multiple wallet outputs in single api call (#205)
* mount v2 router for flexibility - wallet checker now refreshes multiple outputs via single api call
* fix the api router
2017-10-25 17:57:48 +00:00
Yeastplume
49797853d9 tweaks to logging configuration (#176)
* tweaks to logging configuration
* doc port update
2017-10-13 16:42:04 +00:00
Yeastplume
8e382a7593 slog-rs logging (#171)
* added global slog instance, changed all logging macro formats to include logger instance
* adding configuration to logging, allowing for multiple log outputs
* updates to test, changes to build docs
* rustfmt
* moving logging functions into util crate
2017-10-12 16:56:44 +00:00
Ignotus Peverell
b85006ebe5 Wallet operation to burn some coins (#172)
* Burn transaction for testing
* Burn bug fixes, embed burn key in keychain
* Better error logging in API, wallet fee calc fix
2017-10-12 03:35:40 +00:00
AntiochP
bf7c1fb44f add lock_height to Transaction and TxKernel (#167)
* add lock_height to Transaction and TxKernel, sign msg including both fee and lock_height in kernel
* make the order of the fields in tx and kernel more consistent
* rename to kernel_sig_msg
* add test to cover adding timelocked tx to pool, where tx is invalid based on current height of the blockchain
* add tests for adding timelocked txs to blocks (valid and otherwise)
2017-10-11 18:12:01 +00:00
Yeastplume
da21388131 wee warning cleanup + build doc updates (#156)
* warning cleanup
* test compilation
* clarification on architecture in build doc, path export statement fix
2017-10-06 16:46:18 +00:00
Yeastplume
e68a6a69bb Cuckoo miner fixes and configurable coinbase maturity (#154)
* cuckoo miner update+coinbase maturity
* set mining parameter mode on immature coinbase test
2017-10-04 17:44:22 +00:00
Ignotus Peverell
8504efb796
Cargo fmt all the things 2017-09-29 18:44:25 +00:00
Yeastplume
53d9ca630c Cuckoo-miner update - Multiple Plugin mining (#139)
* Adding ability to load multiple mining plugins in parallel via cuckoo miner
* updating with newest cuckoo-miner changes
* revert default config value
* update pow document with link to podcast
2017-09-26 17:58:56 +00:00