Commit graph

130 commits

Author SHA1 Message Date
Quentin Le Sceller
24ed4b787a
Add a maximum size to stempool (#1895)
* Add a maximum size to stempool

* Simplify logic
2018-11-05 13:51:52 +01:00
Antioch Peverell
12be191ecd
validate root of header MMR when processing headers (during sync and full blocks) (#1836)
validate root of header MMR when processing headers (during sync and full blocks) (#1836)
2018-11-01 09:51:32 +00:00
Ignotus Peverell
46051ee174
improve minimum difficulty handling (#1791)
* rename MIN_DIFFICULTY to UNIT_DIFFICULTY; define Difficulty::unit, fix INITIAL_DIFFICULTY
* improve minimum difficulty handling
* replace all Difficulty::one by ::min
* revert secondary scaling default in HeaderInfo; rename scaling_difficulty; refactor difficulty_data_to_vector
2018-10-27 10:37:44 -07:00
Ignotus Peverell
c083312ad4
Improve crate descriptors to be more cargo-friendly (#1844)
* Update secp dependency to use crates.io version
* Add more details to various Cargo.toml
* Remove use of env variables that aren't reliably provided by cargo
2018-10-25 17:44:50 -07:00
Ignotus Peverell
a42d66efff
Merge branch 'master' into unitdiff 2018-10-25 14:20:41 -07:00
hashmap
ab7a59b1c2
Pool crate cleanup (#1835)
* Refactor retrieve_transaction
* Cleanup transaction_pool
2018-10-25 14:21:36 +02:00
Antioch Peverell
4050f7fccb
handle re-orgs in transaction_pool (#1829)
* use reorg_cache in transaction_pool to safely handle txs during a re-org

* rustfmt

* comments
2018-10-24 17:57:31 +01:00
Antioch Peverell
3d59cce7c5
rework pool tx validation logic a bit (#1826) 2018-10-24 10:55:42 +01:00
eupn
1195071f5b Replace logging backend to log4rs and add log rotation (#1789)
* Replace logging backend to flexi-logger and add log rotation
* Changed flexi_logger to log4rs
* Disable logging level filtering in Root logger
* Support different logging levels for file and stdout
* Don't log messages from modules other than Grin-related
* Fix formatting
* Place backed up compressed log copies into log file directory
* Increase default log file size to 16 MiB
* Add comment to config file on log_max_size option
2018-10-21 13:30:56 -07:00
Gary Yu
0d06561a91
replace stdlib RwLock and Mutex with parking_lot (#1793)
* replace all stdlib RwLock with parking_lot RwLock

* replace stdlib Mutex with parking_lot Mutex

* rustfmt
2018-10-20 08:13:07 +08:00
John Tromp
846b38308c replace all Difficulty::one by ::min 2018-10-18 21:18:16 +02:00
yeastplume
545d9d6116 t4 merge - rustfmt on entire tree 2018-10-18 11:40:58 +01:00
yeastplume
9423865f92 update cargo versioning 2018-10-15 10:34:33 +00:00
Yeastplume
d756da062a [T4] warning cleanup (#1713)
* clean up warnings
* rustfmt
2018-10-10 08:56:15 -07:00
Yeastplume
6c8c483172
[T4 ONLY] BIP32 Wallet Compliance - Aggsig Updates - Bulletproof Updates (#1501)
* change keychain to use bip32 paths

* convert keychain to use bip32

* change identifier to be serialisation of 4-level bip32 path

* wallet changes compiling, pass parent key into all wallet functions

* rustfmt

* fix tests in chain

* rustfmt

* core tests passing

* rustfmt

* pool tests

* rustfmt

* fixing wallet tests

* rustfmt

* remove file wallet

* wallet tests compiling

* rustfmt

* remove db_migrate

* successful tx exchange test using BIP32 paths

* rustfmt

* fix wallet derivation paths to m/0/0/0

* wallet test fixed again, working with default path

* rustfmt

* fix server tests

* rustfmt

* make parent_id a trait on walletbackend

* rustfmt

* add ability for wallet to switch between multiple named accounts, and tests (not complete)

* rustfmt

* account switching tests in place and passing

* rustfmt

* compile and test with latest libsecp changes

* added public key sum to calculated e for aggsig

* rustfmt

* Update secp to 26

* bulletproof bip32 path integration

* rustfmt

* wallet restore updated with bip32 paths, also restores accounts

* rustfmt

* rustfmt

* remove old extkey

* remove old extkey

* rustfmt

* add wallet account commands

* rustfmt

* update wallet documentation

* rustfmt

* merge from master

* update libsecp tag

* merge from upstream and fix server test

* rustfmt

* rustfmt

* merge from master

* update latest libsecp merge

* fix commitment to zero value generation
2018-10-10 10:11:01 +01:00
Antioch Peverell
a676eb1b39
Replace RefCell with RwLock in OneTime (and cleanup Weak usage) (#1694)
* rustfmt

* use RwLock in OneTime (and not RefCell)

* rustfmt

* put weak refs back and cleanup

* rustfmt

* revert weak in adapters
2018-10-09 16:53:57 +01:00
Antioch Peverell
11f2d7b6d4
UTXOView (readonly, minimal, output only txhashset extension) (#1584)
* make the utxo view explicit

* introduce utxo_view (readonly, minimal txhashset extension)

* rustfmt

* cleanup

* cleanup

* rustfmt

* cleanup build warnings, comments etc.

* rustfmt

* utxo_view even more readonly now

* cleanup

* refactor pmmr, split out readonly_pmmr and backend

* rustfmt
2018-09-25 11:01:19 +01:00
Antioch Peverell
e72d8b58e4
[WIP] txpool (tx validation) using block sums for full validation (#1567)
tx validation (txpool) rework/simplify
2018-09-24 09:24:10 +01:00
Ignotus Peverell
9cc7aed713
Dual proof of work, first step (#1534)
* Move Proof and Difficulty types to pow module. Difficulty scaling calculation.
* Diffculty scaling and size shift for 2nd PoW
* Backport e7eb26ee
* Test compilation fixes
* Scaling only need to go one way, as @tromp pointed out
2018-09-18 15:12:57 -07:00
Antioch Peverell
a4476443bb
Compact transactions (initial prep work) (#1548)
* introduce CompactTransaction (unused currently)

* rustfmt

* fix comments
2018-09-18 15:25:26 +01:00
Quentin Le Sceller
345a6cb53e Update rand crate to 0.5 (#1542)
* Update rand crate to 0.5

* Rustfmt

* Misplaced import
2018-09-18 09:39:45 +01:00
Gary Yu
a0deb18e77
Update version of some crates (#1536)
Update version of some crates
2018-09-17 10:47:48 +08:00
Antioch Peverell
0cbfeaab94
Faster tx known check (#1468)
* fast check for known tx when adding to pool

* rustfmt
2018-09-03 12:35:37 +01:00
Antioch Peverell
939d391e0e
cache rangeproof and kernel signature verification results (#1419)
* wip - cache rangeproof verification

* rustfmt

* rustfmt

* rename to ok_verifier

* rustfmt

* wire in caching verifier in more places
skip rangeproof and kernel signature verification during deserialization

* rustfmt

* actually cache verification results via ok verifier

* rustfmt

* cleanup

* pass a cache around, not a verifier

* rustfmt

* cleanup trait shenanigans

* rustfmt

* core tests passing

* rustfmt

* rustfmt

* tests passing

* rustfmt

* verifier cache now takes vecs of data in and out

* rustfmt

* logging + rustfmt

* add docs and comments
2018-08-30 15:44:34 +01:00
Antioch Peverell
c334c557aa
Simplify tx.validate() and transaction::aggregate() (#1436)
* simplify tx validation and aggregation
we *only* need to account for reward when building a block from txs

* rustfmt

* cleanup and tests passing

* rustfmt

* better comments in with_reward()

* fix wallet tests
2018-08-28 20:00:25 +01:00
Antioch Peverell
a6bd81fdb3 Cleanup generics (where not needed) (#1429)
BlockChain does not need to be generic
2018-08-27 17:22:48 -04:00
Antioch Peverell
7dfca6077c
Refactor compact block body (#1394)
* introduce CompactBlockBody

* rustfmt

* implement From<Block> for CompactBlock

* rustfmt

* remove debug logging

* wip

* rustfmt
2018-08-22 20:19:37 +01:00
Antioch Peverell
8c820a8929 Fast check for duplicate tx in the pool (#1392) 2018-08-20 18:02:28 -04:00
Antioch Peverell
bb7bc2c4f2
Refactor rewind single header (#1390)
* refactor txhashset to only take a single header
no need to pass in current head_header each time

* rustfmt

* fix bug introduced in rewind with LMDB

* rustfmt

* add some comments
2018-08-20 20:34:12 +01:00
Antioch Peverell
82a467ac3c
Fix txpool race condition (#1385)
* fix txpool race condition

* rustfmt

* fixup pool tests

* rustfmt

* rebase against master, pool tests passing
2018-08-20 14:48:05 +01:00
Ignotus Peverell
ef4f426474
Sort pool tx by fee over weight for mining (#1346)
* Sort pool tx by fee over weight for mining. Fixes #1105
* Bucketing dependent transactions before weighing. Minor tx weight fixes.
* Limit length of tx chain, cleanup and test fixes
* Cleanup all mining references to a hardcoded tx count
* Small test improvement, cleanup
2018-08-19 18:50:43 -04:00
Antioch Peverell
5abefbff33
improve logging when receiving blocks and txs (msg_len) (#1383)
* better logging for msg_len and # kernels

* rustfmt
2018-08-19 18:15:42 +01:00
Ignotus Peverell
6775112f61
Introduce cut_through method to avoid code duplication (#1355)
* Introduce cut_through method to avoid code duplication
* Improve error handling, fix sorting.
2018-08-15 16:20:33 -07:00
hashmap
99a66c1960
Consolidate validation in Block and Transaction (#1354)
* Consolidate validation in Block and Transaction

Introduce TransactionBody which is included into block and tx.
Fixes #1333
2018-08-15 23:14:48 +02:00
Ignotus Peverell
e9c987c075
Consolidate and cleanup tx aggregation (#1332)
* Include commitments non-duplicate checks in aggregate
* Remove said check from the pool
* Block building now uses tx aggregation to reduce duplication
2018-08-12 19:08:08 -07:00
Antioch Peverell
ed88ad8bbc
cannot use unstable rustfmt features outside nightly toolchain (#1338)
* cannot use unstable rustfmt features outside nightly toolchain

* cleanup redundant rustfmt.toml files
2018-08-10 14:54:09 +01:00
Gary Yu
76f4915bf9 improve: time crate flagged as deprecated, switch to actively maintained chrono crate (#1297)
* improve: time crate flagged as deprecated, switch to actively maintained chrono crate

* improve: complete the switching from deprecated time crate to chrono crate, for all the tests part

* improve: complete switching to chrono crate, for the left tests in 'chain' and 'wallet'
2018-07-30 09:33:28 +01:00
Ivan Sorokin
5c029e3f87 Make grin.toml config optional (#1278)
* Make grin.toml config optional. Mirror exisiting config parameters in grin.toml to source code, so binary can run without a config file. Add test for it.
* fixup! Make grin.toml config optional
2018-07-29 17:48:24 -07:00
Quentin Le Sceller
b5fe22dbd5 Bump to 0.3 (#1225) 2018-07-06 18:29:39 +01:00
Antioch Peverell
1df409fa69
verify coinbase maturity via output_mmr_size (#1203)
* we do not need Merkle proofs to spend coinbase outputs
we only need the output_mmr_size from the block header

* tests working with no Merkle proofs in inputs
2018-06-28 21:56:07 -04:00
Ignotus Peverell
d189770080 Wallet LMDB backend (#1151)
* Migrate main node store to LMDB

In preparation to using LMDB as a wallet database, migrate the
node db. There's no point in having 2 key-value stores.
In addition LMDB provides a few advantages as a node db, namely a
much faster build (compared to RocksDb), lesser dependencies and
transactions.

* Migrated p2p store to lmdb, stuff compiles

* More fixes, chain tests starting to pass

* Fixed txhashset rollback messing with block save and general batch delimitation. Chain tests passing.

* rustfmt

* LMDB max map size of 10MB isn't really workable. Half TB seems reasonable.

* Fix wallet tests

* Rather crucial commit was missing

* rustfmt

* Fixing new merged tests following lmdb changes

* rustfmt

* * Make txhashset validation read-only on fast sync to avoid having
a really long open transaction.
* Fix deadlock in new block processing, batch should always be
created within a txhashset lock (when they interact).

* Comment about batch and txhashset interlacing

* Fix store tests to use batch

* Externalize wallet config and seed

* Converted direct read access to file outputs map to an iterator

* Cleaned up and simplified wallet Backend trait:

* No more direct mutable access to internal structures (HashMap)
* Batch interface for all writes
* Remove unneeded read wrapper (read_wallet)

* rustfmt

* First (incomplete) pass at wallet LMDB backend

* Progressing on lmdb backent iml

* Added batch impl for LMDB wallet backend. Pretty much done with it, but not sure how to deal with commit (owned).

* rustfmt

* Wrapping LMDB batch around a refcell to work around borrow rules

* Compilation up to grin chain
2018-06-22 09:08:06 +01:00
Ignotus Peverell
0967a5302b
Kernel sum and MMR sizes in block header (#1163)
* Add kernel commitments sum and kernel and output MMR sizes to block header
* Sum a block without including previous sums, cleanup. Blocks are now summed and validated based on their own totals and not the totals since genesis. This allows to get rid of BlockSum and simplified the setting of a new block's roots, kernel sum and MMR sizes. Fixes #116
* Additional kernel MMR validation to check all prior header roots successively
* Wallet tests fix
2018-06-21 02:30:22 +01:00
Antioch Peverell
8fee3b6922
[consensus breaking] use hash_with_index(pos - 1) when checking the hash in the MMR (#1114)
* use hash_with_index(pos - 1) when checking the hash in the MMR

* fix tests, get rid of todo related to duplicate commitments
2018-06-20 16:11:46 -04:00
Antioch Peverell
0ff6763ee6
[consensus breaking] New Improved Merkle Proofs (#1119)
* new improved Merkle proofs

* fix pool and chain tests

* fixup core tests after the merge
2018-06-20 15:18:52 -04:00
Antioch Peverell
028b14d9d9
Replace rm_log with leaf_set (compact bitmap) (#1147)
* only store leaves in rm_log

* cleanup

* commit

* add failing test to cover case where we compact
an already compacted data file

* fix the logic for pruning the data file

* rm_log only contains leaves
prunelist only contains non-leaf subtree roots

* cleanup

* commit

* bitmap impl running in parallel

* rustfmt

* this is better - rewind unremoves outpu pos spent by rewound inputs

* commit

* commit

* pass bitmap around when rewinding

* store and retrieve input bitmap per block from the db

* Replace the RemoveLog with the UTXO set.

* utxo set starting to pass tests

* stuff works...

* split store types out
added some tests to compare the performance of the rm_log and the proposed utxo_set

* pull prune_list out into standalone file

* cleanup, get rid of unused height param

* cleanup and add doc comments

* add support for migrating rm_log to utxo_set

* take snapshot of utxo file during fast sync
implement migration of rm_log -> utxo_set

* rename rewound_copy to snapshot

* fixup pmmr tests to reflect cutoff_pos

* cleanup unused import

* check_compact needs to rewind the utxo_set as appropriate

* fix pool tests

* fixup core tests

* cache block_input_bitmaps via LruCache in store

* cache block header on initial write to db

* rename utxo_set -> leaf_set
and remove references to "spent" in grin_store

* better document the rewind behavior
2018-06-18 11:18:38 -04:00
Quentin Le Sceller
c412bc6c60 Fix empty candidate txs (#1168) 2018-06-14 18:37:21 +01:00
Quentin Le Sceller
a30ee88236
Cleanup imports (#1161)
Reorganize imports
2018-06-14 08:16:14 -04:00
Yeastplume
88616fd341
[WIP] Updates to support web-wallet (#1160)
* updates to support web wallet workflow

* rustfmt

* functions to support wallet, error handling

* rustfmt

* rebase rustfmt

* test fix
2018-06-13 21:58:45 +01:00
Quentin Le Sceller
8f4dbfa540 Many typo fixes (#1158)
* Typo fix
* Parameterise
2018-06-13 17:03:34 +01:00
Ignotus Peverell
af178f82f8
Refactor the Keychain to be based on a trait (#1146)
* First pass at restructuring the keychain crate and introducing a Keychain trait
* Parameterized everything that had to. Stuff compiles.
* More stuff compiles, fix most tests
* Big merge, pushing down opening the keychain forced adding factory methods on trait
* Test fixes for pool and servers crate
2018-06-08 06:21:54 +01:00