Commit graph

210 commits

Author SHA1 Message Date
Antioch Peverell
0d9242e5dc
refactor on master, pass in offset as blinding_factor (#1064) 2018-05-13 13:59:27 -04:00
Ignotus Peverell
d3a33c790e
Automatically trigger compaction every 2000 blocks (#1054)
Does so in a separate thread as it can take some time. Also
remove validation pre-compaction as it's hopefully unecessary
now.
2018-05-11 17:58:52 +01:00
Yeastplume
4121ea1240
Wallet+Keychain refactoring (#1035)
* beginning to refactor keychain into wallet lib

* rustfmt

* more refactor of aggsig lib, simplify aggsig context manager, hold instance statically for now

* clean some warnings

* clean some warnings

* fix wallet send test a bit

* fix core tests, move wallet dependent tests into integration tests

* repair chain tests

* refactor/fix pool tests

* fix wallet tests, moved from keychain

* add wallet tests
2018-05-09 10:15:58 +01:00
Antioch Peverell
f90506d869
txhashset extension now implements Committed (#1049)
* txhashset now implements committed for consistency

* rustfmt and cleanup
2018-05-08 10:23:33 -04:00
Antioch Peverell
4dd94ff39e
[testnet2] Store output sum (#1043)
* block sums and reworked block validation
read and write block_sums
refactor validate on both block and txhashset
write block_sum on fast sync
we store the kernel_sum (need to account for the offset)

* block_sums

* rustfmt

* cleanup
2018-05-07 09:21:41 -04:00
Yeastplume
59664181e4
update wallet restore, generate new merkle proof for coinbase outputs (#1008) 2018-04-26 14:01:01 +01:00
Antioch Peverell
e22d025dc8 Simplify block marker metadata (#985)
* wip BlockMarker struct, get rid of PMMRMetadata
* use rewind to init the txhashet correctly on startup, we do not need to track pos via metadata (we have block markers), we do not need to open the txhashset with specific pos (we have rewind)
* better logging on init
* keep rewinding and validating on init, to find a good block
* use validate_roots on chain init
2018-04-24 19:53:01 +00:00
Yeastplume
4c6a193e01
[WIP] Miner removal (#979)
* Beginning to remove in-process miner

* rustfmt

* rustfmt

* rustfmt

* rustfmt

* remove pow crate and put remnants into core

* rustfmt

* fix test compilation in core and chain

* rustfmt

* Updating server tests to use test miner

* rustfmt

* rustfmt

* remove pow from test matrix

* adding basic stratum stats to TUI

* run stratum server at all times, and halt messages while syncing

* fix core tests

* add ability to run internal test miner for cuckoo 16 testing

* modify build instructions
2018-04-24 09:18:24 +01:00
Antioch Peverell
46a7245ec1 Cleanup unused fn on rm_log (#999) 2018-04-24 00:46:06 +00:00
Antioch Peverell
ef304ee9c1
add block header cache to store (#996)
* WIP - add blocks header cache to store
The DifficultyIter is expensive to use when validating block headers.

* lru_cache

* rustfmt

* cleanup
2018-04-23 14:55:25 -04:00
Antioch Peverell
77172057ff
header sync does not touch txhashset so no need to take a write lock on it (#990) 2018-04-21 15:15:01 -04:00
Antioch Peverell
242b67182c
use less cloning when validating a txhashset (#989) 2018-04-21 14:14:43 -04:00
Antioch Peverell
c36e72d8c9
fix off by one in setup_height (#988) 2018-04-20 21:11:41 -04:00
Antioch Peverell
b7e29fee55
introduce some minimal sync state (#978)
* commit

* rustfmt

* only fast sync once
then treat as full sync

* commit

* add some debug logging so we can track progress when verifying rangeproofs

* rustfmt
2018-04-18 20:12:39 +01:00
Antioch Peverell
2503811e89
init_head and not reset_head on startup (#971)
so we do not reset the sync_head if we restart during a sync
2018-04-18 01:41:51 +01:00
hashmap
b28de95da4 Error handling using failure in API (#949)
This PR adresses #166
Error handling in wallet was ported to failure in https://github.com/mimblewimble/grin/pull/713
Using the same error model makes wallet code simpler and may simplify migration to Hyper.
2018-04-16 10:00:32 +01:00
hashmap
8e26d4dbac Fix Bus Error (core dumped) when validating fast sync txhashset (#956)
This PR fixes #953 by introducing a lock for txhashet_write. It's not enough
to synchronize access to in memory data, files also needs to be protected, so
a general txhashset lock was introduced.
2018-04-12 10:10:11 +01:00
Yeastplume
dcdf654bc9
[WIP] Core PMMR and API updates to support wallet restore (#950)
* update pmmr to get batch of elements by insertion position

* update pmmr to get batch of elements by insertion position

* add api + chain calls to get traversed outputs back out

* add api + chain calls to get traversed outputs back out

* first pass getting wallet restore to work again with updated utxo-walking api
2018-04-11 10:02:07 +01:00
Antioch Peverell
4d6e0378fe Introduce extending_readonly to simplify a forcing and cancelling rollbacks (#945)
readonly views of the txhashset
2018-04-09 16:37:46 +00:00
Ignotus Peverell
65d5bd40e3
Fix 33c5a983 2018-04-06 19:03:57 +01:00
Ignotus Peverell
33c5a9835a
Avoid double-locking on add eviction. Fixes #936 2018-04-06 18:46:07 +01:00
Antioch Peverell
bc6342ea65 Fix issue where we have no metadata for a block (#938)
when restarting node before initial sync completed
2018-04-06 17:14:50 +00:00
Simon B
98efaf88df mixed small stuff (#934)
* documentation and spelling
* unused values in tests; just .unwrap()
* unneeded macro_use (router)
* unneeded macro_use and use-imports
* unused value -> .unwrap
* unused variables
* clarify that " is required (see 107ea3c4dd)
* first steps to dig into #933 test_p2p issues
* less simulnet build warnings
2018-04-05 23:31:34 +00:00
Ignotus Peverell
c460f9876a
Improvements to orphan handling to help sync (#931)
Our chain orphans data structure now does:

* Eviction based on a max total number of orphaned blocks
* Evicts further away first, then too old
* Avoids all cloning (blocks can be big)

This allows sync to be a little more stupid, only reducing the
number of blocks it requests when orphans get nearly full.

Fixes #897
2018-04-05 03:24:43 +00:00
hashmap
ac979d021d Build index header by height after fast sync (#925)
Make sure that all headers are indexed
2018-04-04 04:45:45 +00:00
hashmap
923adf0fe5 Fix header_by_height (#922)
Currently for fast sync index header_by_height is created only for headers above the horizon
2018-04-02 23:10:30 +00:00
Ignotus Peverell
5ba0dbf38d
Fix and cleanup of fast sync triggering logic (#916)
* Fix and cleanup of fast sync triggering logic
* New txhashset on fast sync has to be applied, not rolled back
* Do not block if peer send buffer is full, fixes #912
2018-03-30 06:02:40 +00:00
Simon B
3e3fe6cae6 Testnet2 small fixes (#899)
* add hint for gcc-5 for building CUDA plugin
* add N_BLOCks comment for >=6GB GPU cards
* minor logging cleanup
* mark to be kept + reformat as "@ height [hash]"
* show CuckooNN + better wording
2018-03-29 15:56:46 +00:00
Antioch Peverell
696097370e
do not save header during header first sync (#891)
just use the header as a hint to go request the block
2018-03-27 16:48:09 -04:00
Yeastplume
6a6e600e48
tests for difficulty adjustment scenarios (#887)
* tests for difficulty adjustment scenarios

* ignore test
2018-03-27 19:15:42 +01:00
Antioch Peverell
5bd0c251dc
log some more fork related info when rewinding and applying a fork (#884) 2018-03-27 11:11:21 -04:00
Ignotus Peverell
16beeca8c9
Correct chain initialization at genesis 2018-03-26 17:38:27 +01:00
Yeastplume
6f264b2f7e
Testnet2 test fixes (#870)
* use dev genesis for automated tests

* never test block time on automated tests

* automated genesis block creation
2018-03-26 12:07:04 +01:00
Ignotus Peverell
327293d28e Genesis, version changes and doc for testnet2 (#865)
* Bump up crates versions

* Finally add a Cargo.lock to avoid dependency breakages

* Build doc update for testnet2

* Fix test framework not really using its mining config

* Testnet2 genesis, best so far at 128 difficulty (a nice number)

* Minor build doc update
2018-03-26 10:48:46 +01:00
John Tromp
a327427178 0-based positions, bag peaks right-to-left with mmr size as index (#863)
* 0-based positions, bag peaks right-to-left with mmr size as index
* fix remaining instances of hash_with_index
* fix pmmr test
2018-03-26 02:26:11 +00:00
Simon B
9285de2c24 Here is a case where ~ is needed, to allow cargo update to only touch the patch level version (third number). Anything else than ~ means that both first numbers can be raised, so "2.1" updaates to "2.2" (#866) 2018-03-25 20:08:38 +00:00
Ignotus Peverell
2bf656646c
Force slog version to 2.1 (2.2 broke some traits) 2018-03-25 19:54:41 +01:00
Simon B
f9b87164e7 Idiomatic cargo toml (#862)
* Specify versions SemVer-style
 - slog and serde
* Comments for not update-able pre-1.0 crates:
 - urlencoded at "0.5" since 0.6+ lacks trait `plugin::Plugin<iron::Request<'_, '_>>`
* alpha-sort deps
* Specify versions SemVer-style
 - backtrace, bitflags, blake2-rfc, bodyparser, built, byteorder
 - chrono, clap, demonize, enum_primitive, env_logger
 - failure, failure_derive, futures
 - hyper, iron, itertools, lazy_static, libc
 - memmap, mount, net2, num_bigint, prettytable-rs
 - rand, regex, rocksdb, router, siphasher
 - term, time, tokio-core, tokio-retry, urlencoded, uuid, walkdir, zip
2018-03-25 17:44:27 +00:00
Simon B
a8fc82d8c1 Console debug fixes (#861)
* move some debug! to trace!

* more informative debugs

* standardising on always showing chain tips as "cumulative difficulty @ height [hash]"

* make 2 debug outputs into a single

* "no peers" as warning (not info) to let it stand out more clearly

* move fn param (used only in this one debug line)

* clarify difficulty "units"
2018-03-25 12:41:12 -04:00
Antioch Peverell
7a8d6147f4 Split pmmr.get() into get_hash() and get_data() (#855) 2018-03-23 23:33:59 +00:00
Antioch Peverell
90a7f3d0f6 Validating an empty chain no longer panics (#850) 2018-03-23 01:48:05 +00:00
Antioch Peverell
ce2fb6ecb7 Rewind force rollback (#847)
* rewind() vs rewind_readonly()
* cleanup comments
2018-03-22 16:57:37 +00:00
Ignotus Peverell
ca8447f3bd
Removed all switch commitment usages, including restore (#841)
* Removed all switch commitment usages, including restore
* Fixed pool tests
* Fix keychain tests
* Get rid of the switch key in keychain
2018-03-22 00:10:11 +00:00
Antioch Peverell
c595a4b35c
GET /v1/chain/validate (#832)
chain.validate() now takes skip_rproof=true|false
2018-03-21 08:28:05 -04:00
Antioch Peverell
b56f21a9ac
split out verify_rangeproofs from sum_outputs (#823)
* split out verify_rangeproofs from sum_outputs
added some timing to the debug msgs
force_rollback at start of chain.validate()

* cleanup
2018-03-20 20:34:19 -04:00
Antioch Peverell
1d7a7457fc
force rollback on txhashset extension when validating (#824) 2018-03-20 17:46:29 -04:00
Simon B
4fa9ccc4f7 Cargo.toml (#818)
* Make sure no grin crates get published by mistake: https://doc.rust-lang.org/cargo/reference/manifest.html#the-publish--field-optional
* add missing workspace = '..', [workspace docs for interested readers](https://doc.rust-lang.org/cargo/reference/manifest.html#the-workspace-section)
* dev- or dev_ works equally well, but dev- is what's typically used
* authors = Grin Developers <maillist>
* [dependencies] keep grin-* at the end + readability
2018-03-20 18:21:29 +00:00
Antioch Peverell
7816f35238
rewind to header as part of txhashset validation (#808)
* rewind to header as part of txhashset validation
otherwise we risk including a new block and the roots do not match

* fix bug in rm_log rewind (wants to be inclusive of provided index)

* put block marker in the index so we can rewind correctly
during validation of the new txhashset

* rustfmt
2018-03-19 21:31:57 -04:00
Ignotus Peverell
154d52e548
Unecessary block cloning in chain 2018-03-19 21:02:02 +00:00
Yeastplume
b53b84b0c6 Move nonce/pow to end of block header, change post-pow hash (#794)
* move nonce/pow to end of block header, update call to cuckoo-miner
* change header hash to be pow only
* fix genesis block miner and tests
2018-03-16 19:04:31 +00:00