Commit graph

137 commits

Author SHA1 Message Date
John Tromp
48efb693e2
Fixfees (#3481)
* add FeeFields type

* use FeeFields with ::zero and try_into().unwrap()

* fixed tests

* avoid 0 accept_base_fee

* add aggregate_fee_fields method for transaction

* implement std::fmt::Display trait for FeeFields

* make base_fee argument non-optional in libtx::mod::tx_fee

* add global and thread local accept_fee_base; use to simplify tests

* set unusually high fee base for a change

* revert to optional base fee argument; default coming from either grin-{server,wallet}.toml

* remove optional base fee argument; can be set with global::set_local_accept_fee_base instead

* define constant global::DEFAULT_ACCEPT_FEE_BASE and set global value

* add Transaction::accept_fee() method and use

* Manual (de)ser impl on FeeFields

* fix comment bug

* Serialize FeeFields as int in tx

* allow feefields: u32:into() for tests

* try adding height args everywhere

* make FeeFields shift/fee methods height dependent

* prior to hf4 feefield testing

* rename selected fee_fields back to fee for serialization compatibility

* fix test_fee_fields test, merge conflict, and doctest use of obsolete fee_fields

* make accept_fee height dependent

* Accept any u64 in FeeFields deser

Co-authored-by: Jasper van der Maarel <j@sper.dev>
2020-11-26 18:03:06 +01:00
deevope
fd5dfaaec4
reorg cache fix (#3495)
* reorg cache period configurable

* fix comment

* u32 type

Co-authored-by: deevope <you@example.com>
2020-11-25 17:59:37 +00:00
Antioch Peverell
7dc94576bd
Introduce CommitOnly variant of Inputs (#3419)
* Introduce CommitOnly variant of Inputs.
Introduce CommitWrapper so we can sort commit only inputs correctly.

* rememebr to resort if converting

* write inputs based on variant and protocol version

* read and write protocol version specific inputs

* store full blocks in local db in v3
convert to v2 when relaying to v2 peers

* add debug version_str for inputs

* no assumptions about spent index sort order

* add additional version debug logs

* fix ser/deser tests for proto v3

* cleanup coinbase maturity

* rework pool to better handle v2 conversion robustly

* cleanup txpool add_to_pool

* fix nrd kernel test

* move init conversion earlier

* cleanup

* cleanup based on PR feedback
2020-09-07 16:58:41 +01:00
Antioch Peverell
133089e985
Refactor Output Identifiers (#3371)
* refactor output to have internal output identifier

refactor to use AsRef for output identifier

make the output MMR explicit in terms of output identifiers

* put the serde deser back for rangeproof

* add json test for transactions
2020-09-03 10:59:54 +01:00
Antioch Peverell
1cff387f61
Use mmr sizes in header to validate header weight (#3395)
* use mmr sizes in header to validate block weight given header only

* add global weight validaton to headers when reading them untrusted off the network

* fixup tests, one pending test

* add test for deserializing untrusted block header exceeeding weight limits

* validate header mmr sizes, must grow for each block

* remove redundant height check

* fix tests after rebase
block specific TooHeavy error
2020-08-19 09:40:56 +01:00
Antioch Peverell
4732a0b62b
convert tx for v2 compatibility on way into stempool/txpool (#3412)
cleanup passing extra_tx around

use output identifiers when converting tx to v2
2020-08-02 13:26:28 +01:00
Antioch Peverell
83b269961a
introduce Inputs enum variants for future commit only support (#3406) 2020-07-28 21:21:57 +01:00
Antioch Peverell
80841f16d2
pass slices around and not refs to vecs (#3404)
* pass slices around and not refs to vecs

* use slice.swap()

* use inputs() not body.inputs
2020-07-27 11:07:18 +01:00
Antioch Peverell
20e5c1910b
NRD rules and "recent" kernel pos index (#3302)
* variants for output_pos linked list entries (head/tail/middle/unique)
next and prev and Vec<u8> lmdb keys

* get_pos on enum

* break list and list entries out into separate enums

* track output features in the new output_pos index, so we can determine coinbase maturity

* push entry impl for none and unique

* some test coverage for output_pos_list

* commit

* wip - FooListEntry

* use instance of the index

* linked list of output_pos and commit_pos both now supported

* linked_list

* cleanup and rename

* rename

* peek_pos

* push some, peek some, pop some

* cleanup

* commit pos
cleanup

* split list and entry out into separate db prefixes

* cleanup and add placeholder for pop_back

* pop_pos_back (for popping off the back of the linked list)
test coverage for pop_pos_back

* wip

* placeholder for prune via a trait
pos must always increase in the index

* rewind kernel_pos_idx when calling rewind_single_block

* RewindableListIndex with rewind support.

* test coverage for rewindable list index

* test coverage for rewind back to 0

* rewind past end of list

* add tests for kernel_pos_idx with multiple commits

* commit

* cleanup

* hook NRD relative lock height validation into block processing and tx validation

* cleanup

* set local chain type for kernel_idx tests

* add test coverage for NRD rules in block processing

* NRD test coverage and cleanup

* NRD relative height 1 test

* test coverage for NRD kernels in block processing

* cleanup

* start of test coverage for txpool NRD kernel rules

* wip

* rework pool tests to use real chain (was mock chain) to better reflect reality (tx/block validation rules etc.)

* cleanup

* cleanup pruneable trait for kernel pos index

* add clear() to kernel_pos idx and test coverage

* hook kernel_pos rebuild into node startup, compaction and fast sync

* verify full NRD history on fast sync

* return early if nrd disabled

* fix header sync issue
2020-06-10 16:38:29 +01:00
Antioch Peverell
c54568e69f
Rework pool tests to use real chain (was mock chain) (#3342)
* rework pool tests to use real chain (was mock chain) to better reflect reality (tx/block validation rules etc.)

* cleanup
2020-06-07 09:26:08 +01:00
Antioch Peverell
988a05f023
Enable NRD kernel support (noop) feature flagged (#3303)
* wip

* commit

* add block level validation rule around NRD kernels and HF3 block height

* wip

* test coverage for kernel ser/deser for NRD kernels

* add some type safety around ser/deser of nrd relative height

* cleanup

* cleanup tx sig handling and add tests around NRD kernel sig

* test coverage for chain apply block containing NRD kernel

* verify kernel variants when adding tx to pool
NRD kernels will not be accepted until HF3

* add txpool test for NRD kernel handling

* fix merge

* fix api for NRD kernels

* commit

* wip - feature flag for NRD kernel support

* wip

* global config and thread local feature flag for NRD kernel support

* add feature flag support for NRD kernels
default to false when starting up node
allow it to be set to true in context of individual tests

* feature flag

* explicit testing of NRD kernel via feature flag

* add chain_type and feature flag logging on startup

* PR feedback and cleanup

* PR feedback
2020-05-29 09:56:24 +01:00
Antioch Peverell
a8b8dc3a7f
add test to demonstrate pair of "half" kernels sharing same public excess (#3314)
* cleanup how we handle key splitting for transaction offset
add test to demonstrate a pair of transaction halves sharing same kernel excess

* cleanup

* cleanup
2020-05-28 15:26:18 +01:00
Antioch Peverell
6faa0e8d75
thread local chain type vs global chain type (#3327)
* Introduce GLOBAL_CHAIN_TYPE and make CHAIN_TYPE thread_local.
This makes testing more explicit and significantly more robust.

* set_local_chain_type() in tests

* cleanup - weird

* get pool tests working with explicit local chain_type config

* core tests working with explicit local chain_type

* p2p tests working with explicit local chain_type

* store tests working

* cleanup, feedback
2020-05-22 12:51:58 +01:00
Antioch Peverell
133a8da53e
Refactor bucket_transactions() and evict_transaction() (#3319) 2020-05-14 14:21:41 -04:00
hashmap
9e51e86538
Use generic types instead of trait objects in tx pool (#3308)
Tx pool takes some parameters as trait objects. It's not an idiomatic Rust code, in this particular case we should use generic types. Trait object makes sense when we accept in runtime different concrete types which implement the trait as a value of the same field. It's not the case here. Trait objects come with a price - instead of method dispatch in compile time we have to accept runtime dispatch. My guess we did it to not clutter the code with type parameters, which is understandable but still suboptimal.
2020-04-30 17:41:49 +02:00
Quentin Le Sceller
04a0123752
Less cloning and additional pattern simplifications (#3223)
* API Cleanup

* Chain Cleanup

* Core Cleanup

* Keychain Cleanup

* P2P Cleanup

* Pool Cleanup

* Store Cleanup

* Util Cleanup

* Cleanup clone_from_slice

* Address jasper comments
2020-02-12 19:35:33 +01:00
François Garillot
dcdbdd4bcc
Some simple Option / Result / iterator pattern simplifications (#3205) 2020-01-29 14:20:57 +00:00
Quentin Le Sceller
6e5afe496b Update License to 2020 (#3196) 2020-01-20 11:40:58 +00:00
Quentin Le Sceller
cdb2d6c72c Node API v2 (#3094)
* Node API v2

* Update gitignore

* Add get_pmmr_indices method

* Add Foreign and Owner API each one with specific secret

* Fix failing tests

* Revert to 'Option<u64>'
2019-12-06 10:57:53 +00:00
Antioch Peverell
8d2c43d7e8
add tx pool kernel counts to tui status page (#3111)
* add tx pool kernel counts to tui status page

* fix formatting on tui
2019-11-14 16:21:27 +00:00
Yeastplume
4e37c1c9e7
Update Serde annotations in pool crate (#3081)
* update serde annotations in pool crate

* rustfmt
2019-10-04 11:44:46 +01:00
Quentin Le Sceller
6be6391225 Change 2018 to 2019 in copyright files (#3072) 2019-10-02 09:40:20 +01:00
Antioch Peverell
f4eb3e3d4b
Always stem local txs if configured that way (unless explicitly fluffed) (#2876)
* always stem local txs if configured that way (unless explicitly fluff from wallet)
this overrides current epoch behavior for txs coming in via "push-api"
rename "local" to "our" txs

* TxSource is now an enum for type safety.
2019-07-04 11:56:42 +01:00
Antioch Peverell
82775164e8
take age of txs into consideraton when bucketing and sorting txs from the pool (#2878) 2019-06-27 17:50:10 +01:00
eupn
56b62a319b Fill BlindingFactor with zeros on Drop (#2847)
* Implement simple zeroing of BlindingFactor in Drop

* rustfmt

* Make Debug implementation for BlindingFactor empty

* Implement BlindingFactor zeroing unit test

* mnemonic.rs: fix deprecated warning in test_bip39_random test

* Use zeroize crate to clear BlindingFactor

* Fix comment and implement dummy Debug trait for BlindingFactor

* Fix formatter
2019-05-30 23:16:53 +02:00
eupn
7e57c10ec7 Remove TODO (#2854) 2019-05-27 13:15:47 -04:00
Quentin Le Sceller
59db5e3bd1 Evict transaction from transaction pool (#2797)
* Evict transaction from transaction pool

* Remove Result from evict function
2019-05-13 15:03:47 -07:00
Antioch Peverell
b2b96f3f81
bucket_transactions is now fee_to_weight aware (#2758)
* bucket_transactions is now fee_to_weight aware
bucket_transactions now returns the underlying txs

* cleanup

* simplify pool bucket sorting, no need for depth sort key
2019-04-30 12:14:02 +01:00
Antioch Peverell
a2adf2dfe8
Dandelion++ Rewrite (#2628)
* reworked the dandelion rewrite (dandelion++)

* fallback to fluff/broadcast if we cannot stem the tx for any reason

* rework stem vs fluff logic during accepting tx

* cleanup docs

* add is_stem to logging

* cleanup

* rustfmt

* cleanup monitor and logging

* rework dandelion monitor to use simple cutoff for aggregation

* transition to next epoch *after* processing tx
so we fluff final outstanding txs

* fluff all txs in stempool if any are older than 30s
aggressively aggregate when we can

* fix rebase onto 1.1.0

* default config comments for Dandelion

* fix code to reflect our tests - fallback to txpool on stempool error

* log fluff and expire errors in dandelion monitor

* cleanup

* fix off by one

* cleanup

* cleanup

* various fixes

* one less clone

* cleanup
2019-03-20 13:08:56 +00:00
hashmap
5d904250d5 Use failure for Pool and Committed errors (#2570)
It doesn't play nice with failure-based error management, we have to
throw away an exisiting error and create a new one (failure-based)
2019-02-13 09:33:25 -08:00
Antioch Peverell
a82e2a0126
Rework "bucket transactions" logic (buckets are now weight limited) (#2487)
* rework bucket txs

* introduce max_block_weight to global (based on chain_type)
not yet in use anywhere...

* now using global::max_block_weight() everywhere

* testing max_block_weight needs to be big enough to cover existing chain tests...

* add some test coverage for prepare mineable transactions at the block weight limit
introduce AsLimitedTx{max_weight} so we can build artifically small blocks (per mining config)

* cleanup

* default_mineable_max_weight is just the default max_block_weight
we do not need to account for coinbase reward here (tx vs block)

* 75 change outputs in a test is not valid now that we have a low block weight limit...
2019-02-01 10:44:04 +00:00
Antioch Peverell
c8fd0575ed Pool tx weight verification (#2466) 2019-01-25 12:48:15 -08:00
Antioch Peverell
a02c44080a Dandelion cycle fix (#2185)
* modify stempool behavior - fluff if tx seen before (any kernels)

* rustfmt
2019-01-07 18:09:04 -05:00
Antioch Peverell
16641fe149
Cleanup compact block logging (#2126)
* cleanup logging, error logging around compact blocks

* rustfmt
2018-12-11 14:11:58 +00:00
hashmap
aedac483f5
Convert to Rust 2018 edition (#2084)
* Convert to Rust 2018 edition

* Update gitignore
2018-12-08 00:59:40 +01:00
Antioch Peverell
66f2545186
Block accepted reorg aware (#2003)
* block_accepted via adapter is now reorg aware
we skip reconciling the txpool is block is not most work
we skip reconciling the reorg_cache is not a reorg

* rustfmt

* logging tweaks

* rework block_accepted interface

* rustfmt

* rework reorg determination

* introduce BlockStatus to represent next vs reorg vs fork

* rustfmt

* cleanup logging

* log from adapter, even during sync
2018-11-21 14:35:38 +00:00
Antioch Peverell
61f1f6103b
Reorg cache time based (cache 30mins of txs) (#2002)
* cache 30 mins of txs for reorg handling
hard limit set the same as the txpool size limit

* rustfmt
2018-11-20 21:47:59 +00:00
Ignotus Peverell
e4be820671
Configurable max block weight for mining (aka soft limit) (#1976) 2018-11-14 10:22:08 -08:00
Antioch Peverell
d97a6c2189
[DNM] broadcast tx kernel hash (if supported by peer) (#1929)
broadcast tx kernel hash (if supported by peer)
2018-11-07 09:28:17 +00:00
Quentin Le Sceller
24ed4b787a
Add a maximum size to stempool (#1895)
* Add a maximum size to stempool

* Simplify logic
2018-11-05 13:51:52 +01:00
hashmap
ab7a59b1c2
Pool crate cleanup (#1835)
* Refactor retrieve_transaction
* Cleanup transaction_pool
2018-10-25 14:21:36 +02:00
Antioch Peverell
4050f7fccb
handle re-orgs in transaction_pool (#1829)
* use reorg_cache in transaction_pool to safely handle txs during a re-org

* rustfmt

* comments
2018-10-24 17:57:31 +01:00
Antioch Peverell
3d59cce7c5
rework pool tx validation logic a bit (#1826) 2018-10-24 10:55:42 +01:00
eupn
1195071f5b Replace logging backend to log4rs and add log rotation (#1789)
* Replace logging backend to flexi-logger and add log rotation
* Changed flexi_logger to log4rs
* Disable logging level filtering in Root logger
* Support different logging levels for file and stdout
* Don't log messages from modules other than Grin-related
* Fix formatting
* Place backed up compressed log copies into log file directory
* Increase default log file size to 16 MiB
* Add comment to config file on log_max_size option
2018-10-21 13:30:56 -07:00
Gary Yu
0d06561a91
replace stdlib RwLock and Mutex with parking_lot (#1793)
* replace all stdlib RwLock with parking_lot RwLock

* replace stdlib Mutex with parking_lot Mutex

* rustfmt
2018-10-20 08:13:07 +08:00
yeastplume
545d9d6116 t4 merge - rustfmt on entire tree 2018-10-18 11:40:58 +01:00
Yeastplume
d756da062a [T4] warning cleanup (#1713)
* clean up warnings
* rustfmt
2018-10-10 08:56:15 -07:00
Yeastplume
6c8c483172
[T4 ONLY] BIP32 Wallet Compliance - Aggsig Updates - Bulletproof Updates (#1501)
* change keychain to use bip32 paths

* convert keychain to use bip32

* change identifier to be serialisation of 4-level bip32 path

* wallet changes compiling, pass parent key into all wallet functions

* rustfmt

* fix tests in chain

* rustfmt

* core tests passing

* rustfmt

* pool tests

* rustfmt

* fixing wallet tests

* rustfmt

* remove file wallet

* wallet tests compiling

* rustfmt

* remove db_migrate

* successful tx exchange test using BIP32 paths

* rustfmt

* fix wallet derivation paths to m/0/0/0

* wallet test fixed again, working with default path

* rustfmt

* fix server tests

* rustfmt

* make parent_id a trait on walletbackend

* rustfmt

* add ability for wallet to switch between multiple named accounts, and tests (not complete)

* rustfmt

* account switching tests in place and passing

* rustfmt

* compile and test with latest libsecp changes

* added public key sum to calculated e for aggsig

* rustfmt

* Update secp to 26

* bulletproof bip32 path integration

* rustfmt

* wallet restore updated with bip32 paths, also restores accounts

* rustfmt

* rustfmt

* remove old extkey

* remove old extkey

* rustfmt

* add wallet account commands

* rustfmt

* update wallet documentation

* rustfmt

* merge from master

* update libsecp tag

* merge from upstream and fix server test

* rustfmt

* rustfmt

* merge from master

* update latest libsecp merge

* fix commitment to zero value generation
2018-10-10 10:11:01 +01:00
Antioch Peverell
a676eb1b39
Replace RefCell with RwLock in OneTime (and cleanup Weak usage) (#1694)
* rustfmt

* use RwLock in OneTime (and not RefCell)

* rustfmt

* put weak refs back and cleanup

* rustfmt

* revert weak in adapters
2018-10-09 16:53:57 +01:00
Antioch Peverell
11f2d7b6d4
UTXOView (readonly, minimal, output only txhashset extension) (#1584)
* make the utxo view explicit

* introduce utxo_view (readonly, minimal txhashset extension)

* rustfmt

* cleanup

* cleanup

* rustfmt

* cleanup build warnings, comments etc.

* rustfmt

* utxo_view even more readonly now

* cleanup

* refactor pmmr, split out readonly_pmmr and backend

* rustfmt
2018-09-25 11:01:19 +01:00