* introduce protocol version to deserialize and read
* thread protocol version through our reader
* cleanup
* cleanup
* streaming_reader cleanup
* Pass protocol version into BinWriter to allow for version specific serialization rules.
* rustfmt
* read and write now protocol version specific
* always stem local txs if configured that way (unless explicitly fluff from wallet)
this overrides current epoch behavior for txs coming in via "push-api"
rename "local" to "our" txs
* TxSource is now an enum for type safety.
* introduce protocol version to deserialize and read
* thread protocol version through our reader
* example protocol version access in kernel read
* fix our StreamingReader impl (WouldBlock woes)
* debug log progress of txhashset download
* introduce HeaderVersion (was u16) for type safety
cleanup pow ser/deser (version unused)
* fixup tests for HeaderVersion
* validate header version during header deserialization
* cleanup legacy "3 dot" check (#2625)
* Allow to peers behind NAT to get up to preferred_max connections (#2543)
Allow to peers behind NAT to get up to preffered_max connections
If peer has only outbound connections it's mot likely behind NAT and we should not stop it from getting more outbound connections
* Reduce usage of unwrap in p2p crate (#2627)
Also change store crate a bit
* Simplify (and fix) output_pos cleanup during chain compaction (#2609)
* expose leaf pos iterator
use it for various things in txhashset when iterating over outputs
* fix
* cleanup
* rebuild output_pos index (and clear it out first) when compacting the chain
* fixup tests
* refactor to match on (output, proof) tuple
* add comments to compact() to explain what is going on.
* get rid of some boxing around the leaf_set iterator
* cleanup
* [docs] Add switch commitment documentation (#2526)
* remove references to no-longer existing switch commitment hash
(as switch commitments were removed in ca8447f3bd
and moved into the blinding factor of the Pedersen Commitment)
* some rewording (points vs curves) and fix of small formatting issues
* Add switch commitment documentation
* [docs] Documents in grin repo had translated in Korean. (#2604)
* Start to M/W intro translate in Korean
* translate in Korean
* add korean translation on intro
* table_of_content.md translate in Korean.
* table_of_content_KR.md finish translate in Korean, start to translate State_KR.md
* add state_KR.md & commit some translation in State_KR.md
* WIP stat_KR.md translation
* add build_KR.md && stratum_KR.md
* finish translate stratum_KR.md & table_of_content_KR.md
* rename intro.KR.md to intro_KR.md
* add intro_KR.md file path each language's intro.md
* add Korean translation file path to stratum.md & table_of_contents.md
* fix difference with grin/master
* Fix TxHashSet file filter for Windows. (#2641)
* Fix TxHashSet file filter for Windows.
* rustfmt
* Updating regexp
* Adding in test case
* Display the current download rate rather than the average when syncing the chain (#2633)
* When syncing the chain, calculate the displayed download speed using the current rate from the most recent iteration, rather than the average download speed from the entire syncing process.
* Replace the explicitly ignored variables in the pattern with an implicit ignore
* remove root = true from editorconfig (#2655)
* Add Medium post to intro (#2654)
Spoke to @yeastplume who agreed it makes sense to add the "Grin Transactions Explained, Step-by-Step" Medium post to intro.md
Open for suggestions on a better location.
* add a new configure item for log_max_files (#2601)
* add a new configure item for log_max_files
* rustfmt
* use a constant instead of multiple 32
* rustfmt
* Fix the build warning of deprecated trim_right_matches (#2662)
* [DOC] state.md, build.md and chain directory documents translate in Korean. (#2649)
* add md files for translation.
* start to translation fast-sync, code_structure. add file build_KR.md, states_KR.md
* add dandelion_KR.md && simulation_KR.md for Korean translation.
* add md files for translation.
* start to translation fast-sync, code_structure. add file build_KR.md, states_KR.md
* add dandelion_KR.md && simulation_KR.md for Korean translation.
* remove some useless md files for translation. this is rearrange set up translation order.
* add dot end of sentence & translate build.md in korean
* remove fast-sync_KR.md
* finish build_KR.md translation
* finish build_KR.md translation
* finish translation state_KR.md & add phrase in state.md to move other language md file
* translate blocks_and_headers.md && chain_sync.md in Korean
* add . in chain_sync.md , translation finished in doc/chain dir.
* fix some miss typos
* Api documentation fixes (#2646)
* Fix the API documentation for Chain Validate (v1/chain/validate). It was documented as a POST, but it is actually a GET request, which can be seen in its handler ChainValidationHandler
* Update the API V1 route list response to include the headers and merkleproof routes. Also clarify that for the chain/outputs route you must specify either byids or byheight to select outputs.
* refactor(ci): reorganize CI related code (#2658)
Break-down the CI related code into smaller more maintainable pieces.
* Specify grin or nanogrins in API docs where applicable (#2642)
* Set Content-Type in API client (#2680)
* Reduce number of unwraps in chain crate (#2679)
* fix: the restart of state sync doesn't work sometimes (#2687)
* let check_txhashset_needed return true on abnormal case (#2684)
* Reduce number of unwwaps in api crate (#2681)
* Reduce number of unwwaps in api crate
* Format use section
* Small QoL improvements for wallet developers (#2651)
* Small changes for wallet devs
* Move create_nonce into Keychain trait
* Replace match by map_err
* Add flag to Slate to skip fee check
* Fix secp dependency
* Remove check_fee flag in Slate
* Add Japanese edition of build.md (#2697)
* catch the panic to avoid peer thread quit early (#2686)
* catch the panic to avoid peer thread quit before taking the chance to ban
* move catch wrapper logic down into the util crate
* log the panic info
* keep txhashset.rs untouched
* remove a warning
* [DOC] dandelion.md, simulation.md ,fast-sync.md and pruning.md documents translate in Korean. (#2678)
* Show response code in API client error message (#2683)
It's hard to investigate what happens when an API client error is
printed out
* Add some better logging for get_outputs_by_id failure states (#2705)
* Switch commitment doc fixes (#2645)
Fix some typos and remove the use of parentheses in a
couple of places to make the reading flow a bit better.
* docs: update/add new README.md badges (#2708)
Replace existing badges with SVG counterparts and add a bunch of new ones.
* Update intro.md (#2702)
Add mention of censoring attack prevented by range proofs
* use sandbox folder for txhashset validation on state sync (#2685)
* use sandbox folder for txhashset validation on state sync
* rustfmt
* use temp directory as the sandbox instead actual db_root txhashset dir
* rustfmt
* move txhashset overwrite to the end of full validation
* fix travis-ci test
* rustfmt
* fix: hashset have 2 folders including txhashset and header
* rustfmt
*
(1)switch to rebuild_header_mmr instead of copy the sandbox header mmr
(2)lock txhashset when overwriting and opening and rebuild
* minor improve on sandbox_dir
* add Japanese edition of state.md (#2703)
* Attempt to fix broken TUI locale (#2713)
Can confirm that on the same machine 1.0.2 TUI looks great and is broken on
the current master. Bump of `cursive` version fixed it for me.
Fixes#2676
* clean the header folder in sandbox (#2716)
* forgot to clean the header folder in sandbox in #2685
* Reduce number of unwraps in servers crate (#2707)
It doesn't include stratum server which is sufficiently changed in 1.1
branch and adapters, which is big enough for a separate PR.
* rustfmt
* change version to beta
* wip
* big refactor, regretting doing this now
* PeerAddr everywhere
* cleanup
* fixup server tests
* peers api working for GET, POST is still WIP
* we can now ban/unban peers by ip only (port optional)
* chore: replace trim_right with $ sed -i'' 's/trim_right/trim_end/' **/*.rs
* docs: individually document macros to avoid warning, add TODO to make to_edge hygenic
* docs: document impl_array_newtype macros, refactor: move all impl_array_newtype macro traits into impl_array_netype_index
* adding optional mmr index to key ids
* rustfmt
* update index
* add mmr index to output display
* change restore to match on commit instead of ID, add extensive restore/check tests for multiple wallets using same seed
* rustfmt
* ensure check restores unknown accounts as well
* rustfmt
* remove storage of commit from wallet
* Fix secondary scaling bugs; rename is_testnet -> is_floonet (#2215)
* add global::is_mainnet()
* use it to change pre-genesis pow type
* rename is_testnet -> is_floonet
* Support multiple chain configurations (#2217)
* Support multiple chain configurations
Supports generating the proper configuration for each chain type
(mainnet, floonet, usernet). Will run them by default under
their respective root directory (~/.grin/main, ~/.grin/floo, etc).
Assigned default ports for mainnet, overriding them to keep Floonet
ports unchanged.
For now, starting on mainnet will abort.
* Fixed usernet command line help message. Fixes#2217
* Differing magic numbers for each chain type (#2208)
* stick to e=H(R|P|m) when use schnorr signature (#2200)
* stick to e=H(R|P|m) when use schnorr signature
* (1)add verify_slate_messages for wallet receive (2)log the message content
* remove debug log on verify_slate_messages
* verify the sender's message signature when receive_tx in wallet listen
* Revert "remove debug log on verify_slate_messages"
This reverts commit 65ea32a407.
* Revert "rustfmt"
This reverts commit c380ab9185.
* Revert "(1)add verify_slate_messages for wallet receive (2)log the message content"
This reverts commit 9584ca7a89.
* [re-floonet] Keychain Floonet BIP32 version/network option (#2235)
* add 'is_floonet' property to keychain
* fix hex encoding and tests
* Fix couple floonet loose ends (#2230)
* Fix couple floonet loose ends. Fixes#2216
* Doc fix for sig message
* Refuse unkown kernel features (#2244)
* Minor: magic number change for re-floonet
* Set pre genesis is_secondary to true (#2247)
* Minor: tx validation error display underlying
* New floonet genesis
* genesis rustfmt
* Use chain-specific config for wallet toml gen
* Fix default wallet_listener_url
* New more reasonable genesis block, bumped version
* genesis rustfmt
* Couple minor fixes to genesis generation script
* add feature for height locked kernels
* add function to compute kernel features appropriate for lock height, and use it
* only sign kernel-features relevant fields; refactor Features
* simplify invalid kernel logic
* remove unused height arg to reward::output and run some rustfmt
* replace nested if/else by match
* start wallet command refactoring
* another re-structuring attempt
* rustfmt
* begin splitting up wallet commands
* rustfmt
* clean up wallet arg checking
* rustfmt
* macro for arg parsing
* rustfmt
* factor out init commands
* rustfmt
* move recover to new format
* rustfmt
* add listen command to new format
* rustfmt
* Finish moving commands to new format
* rustfmt
* rustfmt
* propogate errors more cleanly
* rustfmt
* error handling cleanup
* remove receive_coinbase fn from wallet to wallet client
* start moving clients into separate mod
* rustfmt
* move wallet client + start listener into wallet 'clients'
* rustfmt
* refactor API to make it more modular and completely decouple sending
* rustfmt
* further decouple API from sending methods
* rustfmt
* remove wallet to wallet client trait
* rustfmt
* rename / refactor client + adapters
* rustfmt
* add adapter concept
* add midding node_clients dir
* add file and null adapters
* rustfmt
* remove receive from owner api
* factor out receiving slates into trait
* rustfmt
* adding listen trait
* rustfmt
* change listener to use trait
* rustfmt
* add test for file-based exchange
* replace http api send command
* rustfmt
* move controller out of libwallet and into top-level wallet dir
* rustfmt
* add moved controller
* Update secp dependency to use crates.io version
* Add more details to various Cargo.toml
* Remove use of env variables that aren't reliably provided by cargo
* Replace logging backend to flexi-logger and add log rotation
* Changed flexi_logger to log4rs
* Disable logging level filtering in Root logger
* Support different logging levels for file and stdout
* Don't log messages from modules other than Grin-related
* Fix formatting
* Place backed up compressed log copies into log file directory
* Increase default log file size to 16 MiB
* Add comment to config file on log_max_size option
* header MMR in use within txhashset itself
works with fast sync
not yet in place for initial header sync
* add the (currently unused) sync_head mmr
* use sync MMR during fast sync
rebuild header MMR after we validate full txhashset after download
* support missing header MMR (rebuild as necessary) for legacy nodes
* rename to HashOnly
* cleanup backend.append()
* simplify vec_backend to match simpler append api
* Use secp crate directly without extra use statement (#1738)
(cherry picked from commit 80d28f94ea)
* suppress the test error of test_start_api. Note: this is not a fix.
(cherry picked from commit 6f29685daf)
* get rid of RwLock around peer instances
* rustfmt
* rename stuff to live_info
* rustfmt
* serialize into PeerInfoDisplay
limit live_info write lock to peer_info.update()
rename to PeerLiveInfo
* simplify broadcast logic a bit more
connected peers are connected by definition
* return true/false on broadcast so we can send to more peers
* rustfmt
* fix p2p and server tests for PeerInfoDisplay
* commit
Mostly to support let's encrypt. It requires to switch from native-tls and friends to rustls and friends, which perhap is a good thing per se, rustls looks more modern and for sure more Rusty.
Alternative would be manually convert pkcs12 certificates to pem, which requires openssl tools to be installed and make transparent integration whith let's encrypt much harder (this is out of the scope for now, perhaps in near future)
* Allow TLS for Wallet APIs
This PR adds an optional support of TLS for wallet APIs. Only PKCS12 format is supported, will address .pem support in next PR and provide some documentation.
Address #1425
According to rfc2616[1], the response from a server to a request with
bad credentials should be a 401 instead of a 403. Grin does not have
the concept of identities so it does not actually recognize a user
request with bad credentials.
[1] https://tools.ietf.org/html/rfc2616#section-10.4.2
* Add api_secret
* Add to base64 method
* Add basic auth in API
* Add Basic Auth to owner API
* Add flag to enable disable basic auth
* Add .api_secret file
We used to launch a thread for API server inside the wallet crate, now we do it inside api crate, so the cmd tool launches API and exit. This fix makes sure that command will wait for API thread.
* Middleware and handler (mw implements the same trait) could be attached to multiple nodes inside the router
* Middleware could be attached to the router (syntactic sugar, it is attached to the root node) as well as to any node
* Handler's call method receives an iterator of handlers and responsible for calling the next handler if needed
* Support TLS in ApiServer
This is ground work to support TLS in Grin APIs (like wallet ot node). Particular API implemention needs to decide if TLS is used or not and pass certificate data etc.
* P12 format support
* New method to start TLS server
* Transparent TLS support in API client (depends on URL scheme http/https)
* Refactoring
* Initial support for graceful shutdown (commentred out int this PR, unstable for now)
* API server tests (TLS server test is disabled by default, hyper client rejects self-signed certificates, so extra step is needed to install local CA (I used mkcert)
* Add a cert file to make test complile
* Make it simpler to implement middleware
* Switch from the current thread runtime to the default one. It enables us to inject TLS support later one and potentially more scalable, unfortunately it involves some additonal cloning of the router, because we can't rely on thread local vars anymore
* Introduce `call` entrypoint for Handler, so it's possible to handle any HTTP method in one place, handy for middleware
* Implement example of middleware
Address ##1525 in particular and improve error messages in general.
Instead of `Request Error: Error { inner:` a client would get:
`Generic error: Invalid request body: missing field `method` at line 1 column 162`
* Fix get outputs call in wallet. It generates an invalid url if there are 1000+ outputs.
* Also switched to async http client for performance reasons
* Couple unrelated cleanups
* fixup! Fix get outputs call in wallet
Fixes [#1308]. The main change is to switch from Core to Runtime inside the client.
I also used this as an opportunity to provide async methods for get and post, so we can
use it in places where futures are acceptable, which is not the case for the wallet.
* Remove Iron dependecy and update hyper to version 0.12 #876
* REMOVE ME
* Revert "REMOVE ME"
This reverts commit e9a976eee98a2d5a4dfae5d9e1e4f5ed640c05d3.
* Rebase and start updating libwallet
Libwallet doesn't compile yet.
* Wallet compiles
* Grin compiles
* No compilation errors in tests
* All tests pass
* Reeturn future from handler
* Refactoring
* Fix lifetime issue one more time
I have to force push to rollback all the work done in last 2 days
* Fix wallet send issue
* Clean up
* improve: time crate flagged as deprecated, switch to actively maintained chrono crate
* improve: complete the switching from deprecated time crate to chrono crate, for all the tests part
* improve: complete switching to chrono crate, for the left tests in 'chain' and 'wallet'
* Migrate main node store to LMDB
In preparation to using LMDB as a wallet database, migrate the
node db. There's no point in having 2 key-value stores.
In addition LMDB provides a few advantages as a node db, namely a
much faster build (compared to RocksDb), lesser dependencies and
transactions.
* Migrated p2p store to lmdb, stuff compiles
* More fixes, chain tests starting to pass
* Fixed txhashset rollback messing with block save and general batch delimitation. Chain tests passing.
* rustfmt
* LMDB max map size of 10MB isn't really workable. Half TB seems reasonable.
* Fix wallet tests
* Rather crucial commit was missing
* rustfmt
* Fixing new merged tests following lmdb changes
* rustfmt
* * Make txhashset validation read-only on fast sync to avoid having
a really long open transaction.
* Fix deadlock in new block processing, batch should always be
created within a txhashset lock (when they interact).
* Comment about batch and txhashset interlacing
* Fix store tests to use batch
* Externalize wallet config and seed
* Converted direct read access to file outputs map to an iterator
* Cleaned up and simplified wallet Backend trait:
* No more direct mutable access to internal structures (HashMap)
* Batch interface for all writes
* Remove unneeded read wrapper (read_wallet)
* rustfmt
* First (incomplete) pass at wallet LMDB backend
* Progressing on lmdb backent iml
* Added batch impl for LMDB wallet backend. Pretty much done with it, but not sure how to deal with commit (owned).
* rustfmt
* Wrapping LMDB batch around a refcell to work around borrow rules
* Compilation up to grin chain
* Add kernel commitments sum and kernel and output MMR sizes to block header
* Sum a block without including previous sums, cleanup. Blocks are now summed and validated based on their own totals and not the totals since genesis. This allows to get rid of BlockSum and simplified the setting of a new block's roots, kernel sum and MMR sizes. Fixes#116
* Additional kernel MMR validation to check all prior header roots successively
* Wallet tests fix
Small refactoring of one folder, if it makes sense I could extend the scope.
* Remove some cloning (real and just verbosity in the code)
* Naming conventions like to/into*
* Some Clippy's suggestions
I found that we don't use field init shorthand syntax, so I didn't touch this part, was it discussed before?
* verify a tx like we verify a block (experimental)
* first minimal_pool test up and running but not testing what we need to
* rework tx_pool validation to use txhashset extension
* minimal tx pool wired up but rough
* works locally (rough statew though)
delete "legacy" pool and graph code
* rework the new pool into TransactionPool and Pool impls
* rework pool to store pool entries
with associated timer and source etc.
* all_transactions
* extra_txs so we can validate stempool against existing txpool
* rework reconcile_block
* txhashset apply_raw_tx can now rewind to a checkpoint (prev raw tx)
* wip - txhashset tx tests
* more flexible rewind on MMRs
* add tests to cover apply_raw_txs on txhashset extension
* add_to_stempool and add_to_txpool
* deaggregate multi kernel tx when adding to txpoool
* handle freshness in stempool
handle propagation of stempool txs via dandelion monitor
* patience timer and fluff if we cannot propagate
to next relay
* aggregate and fluff stempool is we have no relay
* refactor coinbase maturity
* rewrote basic tx pool tests to use a real txhashset via chain adapter
* rework dandelion monitor to reflect recent discussion
works locally but needs a cleanup
* refactor dandelion_monitor - split out phases
* more pool test coverage
* remove old test code from pool (still wip)
* block_building and block_reconciliation tests
* tracked down chain test failure...
* fix test_coinbase_maturity
* dandelion_monitor now runs...
* refactor dandelion config, shared across p2p and pool components
* fix pool tests with new config
* fix p2p tests
* rework tx pool to deal with duplicate commitments (testnet2 limitation)
* cleanup and address some PR feedback
* add big comment about pre_tx...
This PR adresses #166
Error handling in wallet was ported to failure in https://github.com/mimblewimble/grin/pull/713
Using the same error model makes wallet code simpler and may simplify migration to Hyper.
* update pmmr to get batch of elements by insertion position
* update pmmr to get batch of elements by insertion position
* add api + chain calls to get traversed outputs back out
* add api + chain calls to get traversed outputs back out
* first pass getting wallet restore to work again with updated utxo-walking api
* Bump up crates versions
* Finally add a Cargo.lock to avoid dependency breakages
* Build doc update for testnet2
* Fix test framework not really using its mining config
* Testnet2 genesis, best so far at 128 difficulty (a nice number)
* Minor build doc update
* adding appropriate message to bulletproofs to allow for restore
* rustfmt
* should work, now test
* rustfmt
* fix to wallet restore, works now
* fix pool tests
* fix pool tests
* rustfmt
* Initial Dandelion Commit
* Changed stem_tx_pool to tx_stempool
* Introduction of stem memory pool and stem pool config
* Pool push now send to stem memory pool
* Add stem transaction functions
* Add stem transaction pool
* Drastically simplified code structure
* Add monitor transactions
* Add Dandelion monitor and remove transactions from stempool
* Add peer relay monitor
* Reconcile block with stempool
* Fix total size bug
* Add fluff option for pool push
* Added details on dandelion monitor
* Fix issue with missing parent
* Child transaction with stempool parent are now forced stem
* Update Dandelion Relay from outgoing peers
* Fix missing pool reconciliation
* Add the ability to fluff a transaction directly
* Fix tests for Dandelion
* Missing send_stem_transaction method...
* Add fluff handler for wallet
* Add logger when successfully updated Dandelion relay
* Launch transaction monitor last
* Fix dandelion relay misplaced
* Add logging and updating for stempool
* Additionnal check for stem transaction
* Fix 2 Locks in a row
* remove difficulty field from block_header (derive from total_difficulty)
* address feedback on PR and general cleanup
* rustfmt
* rework pow/difficulty validation in pipe::validate_header
now that we only have total_difficulty available
* cleanup various todos
* rustfmt
* rework DifficultyIterator to track header and prev_header state
* rustfmt caught some garbage syntax
* cleanup
* wip
* failing test for being too eager when pruning a sibling
* commit
* rustfmt
* [WIP] modified get_shift and get_leaf_shift to account for leaving "pruned but not compacted" leaves in place
Note: this currently breaks check_compact as nothing else is aware of the modified behavior
* rustfmt
* commit
* rustfmt
* basic prune/compact/shift working
* rustfmt
* commit
* rustfmt
* next_pruned_idx working (I think)
* commit
* horizon test uncovered some subtle issues - wip
* rustfmt
* cleanup
* rustfmt
* commit
* cleanup
* cleanup
* commit
* rustfmt
* contains -> binary_search
* rustfmt
* no need for height==0 special case
* wip - works for single compact, 2nd one breaks the mmr hashes
* commit
* rustfmt
* fixed it (needs a lot of cleanup)
we were not traversing all the way up to the peak if we pruned an entire tree
so rm_log and prune list were inconsistent
* multiple compact steps are working
data file not being copmacted currently (still to investigate)
* cleanup store tests
* cleanup
* cleanup up debug
* rustfmt
* take kernel offsets into account when summing kernels and outputs for full txhashset validation
validate chain state pre and post compaction
* rustfmt
* fix wallet refresh (we need block height to be refreshed on non-coinbase outputs)
otherwise we cannot spend them...
* rustfmt
* Cleanup MMRs positional indexes. Get rid of the kernel position index.
Introduced a new block marker index that keeps, for each block, the
respective positions in the output and kernel MMRs. This is now sufficient
for rewind.
* Block marker compaction
* Implementation of compaction for the chain. Single entry point on the chain triggers compaction of all MMRs as well as the cleanup of the positional index and full blocks.
* API endpoint, additional tests and more fixes for compaction
* Also prune PMMR metadata, minor bug fix
* PMMR store tests fix
* family_branch() to recursively call family() up the branch
todo
- we hit a peak, then we need to get to the root somehow
- actually get the hashes to build the proof
* wip
* some additional testing around merkle tree branches
* track left/right branch for each sibling as we build the merkle path up
* MerkleProof and basic (incomplete) verify fn
* I think a MerkleProof verifies correctly now
need to test on test case with multiple peaks
* basic pmmr merkle proof working
* MerkleProof now serializable/deserializable
* coinbase maturity via merkle proof basically working
* ser/deser merkle proof into hex in api and wallet.dat
* cleanup
* wip - temporarily saving merkle proofs to the commit index
* assert merkle proof in store matches the rewound version
there are cases where it does not...
* commit
* commit
* can successfully rewind the output PMMR and generate a Merkle proof
need to fix the tests up now
and cleanup the code
and add docs for functions etc.
* core tests passing
* fixup chain tests using merkle proofs
* pool tests working with merkle proofs
* api tests working with merkle proof
* fix the broken comapct block hashing behavior
made nonce for short_ids explicit to help with this
* cleanup and comment as necessary
* cleanup variety of TODOs
* beginning to remove sum
* continuing to remove sumtree sums
* finished removing sums from pmmr core
* renamed sumtree files, and completed changes+test updates in core and store
* updating grin/chain to include removelogs
* integration of flatfile structure, changes to chain/sumtree to start using them
* tests on chain, core and store passing
* cleaning up api and tests
* formatting
* flatfiles stored as part of PMMR backend instead
* all compiling and tests running
* documentation
* added remove + pruning to flatfiles
* remove unneeded enum
* adding sumtree root struct
* Clean server shutdown, generalizes usage of `Weak`. Introduces 2 main changes:
* A shared `AtomicBool` that signals server shutdown. All server
threads regularly check it to break out of their main loops when
it changes to `true`.
* Breaking of circular `Arc` references, which can never be
destroyed, by downgrading to `Weak` instead. Only the main server
keeps the `Arc` while all other components get the `Weak` variant.
Both of these are required for all long-living structs to be
cleanly destroyed. Note that in Rust this is fairly important as
most resource-freeing logic is associated with `drop`,
which is only called when said struct is free of scope or `Arc`
references.
Should address most of #536 (only need the stop hook to call
`Server` shutdown).
* Test for fast sync, followed by restart, followed by re-fast-sync
* P2P test fix
* Double sync taking too long for Travis, commenting out for now
