Commit graph

220 commits

Author SHA1 Message Date
Quentin Le Sceller
1255ea2224
Upgrade and uniformize dependencies (#1138)
* Remove unused DandelionConfig

* Cargo upgrade and cleanup
2018-06-05 13:26:32 -04:00
Quentin Le Sceller
663733e72b
Replace try! for ? (#1134)
* Replace try for ?
2018-06-04 12:05:49 -04:00
Antioch Peverell
0ecadd3486
move verify_kernel_sums into committed trait (#1131) 2018-06-02 19:00:44 +01:00
Antioch Peverell
0a80023527
pass in next height when applying raw txs (#1129)
rewind correctly to the previous height as necessary
2018-06-02 12:29:18 +01:00
hashmap
2fa32d15ce Rustify core/src/core (#1122)
Small refactoring of one folder, if it makes sense I could extend the scope.
* Remove some cloning (real and just verbosity in the code)
* Naming conventions like to/into*
* Some Clippy's suggestions

I found that we don't use field init shorthand syntax, so I didn't touch this part, was it discussed before?
2018-06-01 20:41:26 +01:00
Antioch Peverell
4fda7a6899
Minimal Transaction Pool (#1067)
* verify a tx like we verify a block (experimental)

* first minimal_pool test up and running but not testing what we need to

* rework tx_pool validation to use txhashset extension

* minimal tx pool wired up but rough

* works locally (rough statew though)
delete "legacy" pool and graph code

* rework the new pool into TransactionPool and Pool impls

* rework pool to store pool entries
with associated timer and source etc.

* all_transactions

* extra_txs so we can validate stempool against existing txpool

* rework reconcile_block

* txhashset apply_raw_tx can now rewind to a checkpoint (prev raw tx)

* wip - txhashset tx tests

* more flexible rewind on MMRs

* add tests to cover apply_raw_txs on txhashset extension

* add_to_stempool and add_to_txpool

* deaggregate multi kernel tx when adding to txpoool

* handle freshness in stempool
handle propagation of stempool txs via dandelion monitor

* patience timer and fluff if we cannot propagate
to next relay

* aggregate and fluff stempool is we have no relay

* refactor coinbase maturity

* rewrote basic tx pool tests to use a real txhashset via chain adapter

* rework dandelion monitor to reflect recent discussion
works locally but needs a cleanup

* refactor dandelion_monitor - split out phases

* more pool test coverage

* remove old test code from pool (still wip)

* block_building and block_reconciliation tests

* tracked down chain test failure...

* fix test_coinbase_maturity

* dandelion_monitor now runs...

* refactor dandelion config, shared across p2p and pool components

* fix pool tests with new config

* fix p2p tests

* rework tx pool to deal with duplicate commitments (testnet2 limitation)

* cleanup and address some PR feedback

* add big comment about pre_tx...
2018-05-30 16:57:13 -04:00
Quentin Le Sceller
b6c31ebc78 Remove unused error type switch commitment (#1104)
* Remove unused switch commitment error

* Rustfmt

* Updated doc
2018-05-30 14:45:27 -04:00
Yeastplume
82ed280625
[WIP] Factoring out wallet traits and continued wallet library work (#1096)
* rename wallet libs

* rename transaction.rs to slate.rs

* rename transaction.rs to slate.rs

* move some types into libwallet/types

* rustfmt

* rename libtransaction libtx

* rename libtransaction libtx

* change types.rs to file_wallet

* rustfmt

* rename WalletData to FileWallet

* refactoring WalletBackend types out

* rustfmt

* All compiling, at least

* rustfmt

* fix FileWallet paths to persist

* rustfmt

* re-ignore wallet integration test
2018-05-30 17:48:32 +01:00
Antioch Peverell
601c9e769e
all duplicates (incorrectly) to maintain compatibility with testnet2 (#1103) 2018-05-30 10:15:37 -04:00
Antioch Peverell
37e2dc86f6
fix the hash comparison (#1091)
for identifying duplicate commitments in apply_output...
2018-05-28 14:22:22 -04:00
Antioch Peverell
0d9242e5dc
refactor on master, pass in offset as blinding_factor (#1064) 2018-05-13 13:59:27 -04:00
Ignotus Peverell
d3a33c790e
Automatically trigger compaction every 2000 blocks (#1054)
Does so in a separate thread as it can take some time. Also
remove validation pre-compaction as it's hopefully unecessary
now.
2018-05-11 17:58:52 +01:00
Yeastplume
4121ea1240
Wallet+Keychain refactoring (#1035)
* beginning to refactor keychain into wallet lib

* rustfmt

* more refactor of aggsig lib, simplify aggsig context manager, hold instance statically for now

* clean some warnings

* clean some warnings

* fix wallet send test a bit

* fix core tests, move wallet dependent tests into integration tests

* repair chain tests

* refactor/fix pool tests

* fix wallet tests, moved from keychain

* add wallet tests
2018-05-09 10:15:58 +01:00
Antioch Peverell
f90506d869
txhashset extension now implements Committed (#1049)
* txhashset now implements committed for consistency

* rustfmt and cleanup
2018-05-08 10:23:33 -04:00
Antioch Peverell
4dd94ff39e
[testnet2] Store output sum (#1043)
* block sums and reworked block validation
read and write block_sums
refactor validate on both block and txhashset
write block_sum on fast sync
we store the kernel_sum (need to account for the offset)

* block_sums

* rustfmt

* cleanup
2018-05-07 09:21:41 -04:00
Yeastplume
59664181e4
update wallet restore, generate new merkle proof for coinbase outputs (#1008) 2018-04-26 14:01:01 +01:00
Antioch Peverell
e22d025dc8 Simplify block marker metadata (#985)
* wip BlockMarker struct, get rid of PMMRMetadata
* use rewind to init the txhashet correctly on startup, we do not need to track pos via metadata (we have block markers), we do not need to open the txhashset with specific pos (we have rewind)
* better logging on init
* keep rewinding and validating on init, to find a good block
* use validate_roots on chain init
2018-04-24 19:53:01 +00:00
Yeastplume
4c6a193e01
[WIP] Miner removal (#979)
* Beginning to remove in-process miner

* rustfmt

* rustfmt

* rustfmt

* rustfmt

* remove pow crate and put remnants into core

* rustfmt

* fix test compilation in core and chain

* rustfmt

* Updating server tests to use test miner

* rustfmt

* rustfmt

* remove pow from test matrix

* adding basic stratum stats to TUI

* run stratum server at all times, and halt messages while syncing

* fix core tests

* add ability to run internal test miner for cuckoo 16 testing

* modify build instructions
2018-04-24 09:18:24 +01:00
Antioch Peverell
46a7245ec1 Cleanup unused fn on rm_log (#999) 2018-04-24 00:46:06 +00:00
Antioch Peverell
ef304ee9c1
add block header cache to store (#996)
* WIP - add blocks header cache to store
The DifficultyIter is expensive to use when validating block headers.

* lru_cache

* rustfmt

* cleanup
2018-04-23 14:55:25 -04:00
Antioch Peverell
77172057ff
header sync does not touch txhashset so no need to take a write lock on it (#990) 2018-04-21 15:15:01 -04:00
Antioch Peverell
242b67182c
use less cloning when validating a txhashset (#989) 2018-04-21 14:14:43 -04:00
Antioch Peverell
c36e72d8c9
fix off by one in setup_height (#988) 2018-04-20 21:11:41 -04:00
Antioch Peverell
b7e29fee55
introduce some minimal sync state (#978)
* commit

* rustfmt

* only fast sync once
then treat as full sync

* commit

* add some debug logging so we can track progress when verifying rangeproofs

* rustfmt
2018-04-18 20:12:39 +01:00
Antioch Peverell
2503811e89
init_head and not reset_head on startup (#971)
so we do not reset the sync_head if we restart during a sync
2018-04-18 01:41:51 +01:00
hashmap
b28de95da4 Error handling using failure in API (#949)
This PR adresses #166
Error handling in wallet was ported to failure in https://github.com/mimblewimble/grin/pull/713
Using the same error model makes wallet code simpler and may simplify migration to Hyper.
2018-04-16 10:00:32 +01:00
hashmap
8e26d4dbac Fix Bus Error (core dumped) when validating fast sync txhashset (#956)
This PR fixes #953 by introducing a lock for txhashet_write. It's not enough
to synchronize access to in memory data, files also needs to be protected, so
a general txhashset lock was introduced.
2018-04-12 10:10:11 +01:00
Yeastplume
dcdf654bc9
[WIP] Core PMMR and API updates to support wallet restore (#950)
* update pmmr to get batch of elements by insertion position

* update pmmr to get batch of elements by insertion position

* add api + chain calls to get traversed outputs back out

* add api + chain calls to get traversed outputs back out

* first pass getting wallet restore to work again with updated utxo-walking api
2018-04-11 10:02:07 +01:00
Antioch Peverell
4d6e0378fe Introduce extending_readonly to simplify a forcing and cancelling rollbacks (#945)
readonly views of the txhashset
2018-04-09 16:37:46 +00:00
Ignotus Peverell
65d5bd40e3
Fix 33c5a983 2018-04-06 19:03:57 +01:00
Ignotus Peverell
33c5a9835a
Avoid double-locking on add eviction. Fixes #936 2018-04-06 18:46:07 +01:00
Antioch Peverell
bc6342ea65 Fix issue where we have no metadata for a block (#938)
when restarting node before initial sync completed
2018-04-06 17:14:50 +00:00
Simon B
98efaf88df mixed small stuff (#934)
* documentation and spelling
* unused values in tests; just .unwrap()
* unneeded macro_use (router)
* unneeded macro_use and use-imports
* unused value -> .unwrap
* unused variables
* clarify that " is required (see 107ea3c4dd)
* first steps to dig into #933 test_p2p issues
* less simulnet build warnings
2018-04-05 23:31:34 +00:00
Ignotus Peverell
c460f9876a
Improvements to orphan handling to help sync (#931)
Our chain orphans data structure now does:

* Eviction based on a max total number of orphaned blocks
* Evicts further away first, then too old
* Avoids all cloning (blocks can be big)

This allows sync to be a little more stupid, only reducing the
number of blocks it requests when orphans get nearly full.

Fixes #897
2018-04-05 03:24:43 +00:00
hashmap
ac979d021d Build index header by height after fast sync (#925)
Make sure that all headers are indexed
2018-04-04 04:45:45 +00:00
hashmap
923adf0fe5 Fix header_by_height (#922)
Currently for fast sync index header_by_height is created only for headers above the horizon
2018-04-02 23:10:30 +00:00
Ignotus Peverell
5ba0dbf38d
Fix and cleanup of fast sync triggering logic (#916)
* Fix and cleanup of fast sync triggering logic
* New txhashset on fast sync has to be applied, not rolled back
* Do not block if peer send buffer is full, fixes #912
2018-03-30 06:02:40 +00:00
Simon B
3e3fe6cae6 Testnet2 small fixes (#899)
* add hint for gcc-5 for building CUDA plugin
* add N_BLOCks comment for >=6GB GPU cards
* minor logging cleanup
* mark to be kept + reformat as "@ height [hash]"
* show CuckooNN + better wording
2018-03-29 15:56:46 +00:00
Antioch Peverell
696097370e
do not save header during header first sync (#891)
just use the header as a hint to go request the block
2018-03-27 16:48:09 -04:00
Yeastplume
6a6e600e48
tests for difficulty adjustment scenarios (#887)
* tests for difficulty adjustment scenarios

* ignore test
2018-03-27 19:15:42 +01:00
Antioch Peverell
5bd0c251dc
log some more fork related info when rewinding and applying a fork (#884) 2018-03-27 11:11:21 -04:00
Ignotus Peverell
16beeca8c9
Correct chain initialization at genesis 2018-03-26 17:38:27 +01:00
Yeastplume
6f264b2f7e
Testnet2 test fixes (#870)
* use dev genesis for automated tests

* never test block time on automated tests

* automated genesis block creation
2018-03-26 12:07:04 +01:00
Ignotus Peverell
327293d28e Genesis, version changes and doc for testnet2 (#865)
* Bump up crates versions

* Finally add a Cargo.lock to avoid dependency breakages

* Build doc update for testnet2

* Fix test framework not really using its mining config

* Testnet2 genesis, best so far at 128 difficulty (a nice number)

* Minor build doc update
2018-03-26 10:48:46 +01:00
John Tromp
a327427178 0-based positions, bag peaks right-to-left with mmr size as index (#863)
* 0-based positions, bag peaks right-to-left with mmr size as index
* fix remaining instances of hash_with_index
* fix pmmr test
2018-03-26 02:26:11 +00:00
Simon B
9285de2c24 Here is a case where ~ is needed, to allow cargo update to only touch the patch level version (third number). Anything else than ~ means that both first numbers can be raised, so "2.1" updaates to "2.2" (#866) 2018-03-25 20:08:38 +00:00
Ignotus Peverell
2bf656646c
Force slog version to 2.1 (2.2 broke some traits) 2018-03-25 19:54:41 +01:00
Simon B
f9b87164e7 Idiomatic cargo toml (#862)
* Specify versions SemVer-style
 - slog and serde
* Comments for not update-able pre-1.0 crates:
 - urlencoded at "0.5" since 0.6+ lacks trait `plugin::Plugin<iron::Request<'_, '_>>`
* alpha-sort deps
* Specify versions SemVer-style
 - backtrace, bitflags, blake2-rfc, bodyparser, built, byteorder
 - chrono, clap, demonize, enum_primitive, env_logger
 - failure, failure_derive, futures
 - hyper, iron, itertools, lazy_static, libc
 - memmap, mount, net2, num_bigint, prettytable-rs
 - rand, regex, rocksdb, router, siphasher
 - term, time, tokio-core, tokio-retry, urlencoded, uuid, walkdir, zip
2018-03-25 17:44:27 +00:00
Simon B
a8fc82d8c1 Console debug fixes (#861)
* move some debug! to trace!

* more informative debugs

* standardising on always showing chain tips as "cumulative difficulty @ height [hash]"

* make 2 debug outputs into a single

* "no peers" as warning (not info) to let it stand out more clearly

* move fn param (used only in this one debug line)

* clarify difficulty "units"
2018-03-25 12:41:12 -04:00
Antioch Peverell
7a8d6147f4 Split pmmr.get() into get_hash() and get_data() (#855) 2018-03-23 23:33:59 +00:00