Commit graph

593 commits

Author SHA1 Message Date
Quentin Le Sceller
64b2fddbf1
[5.0.x] Fix PIBD segments (#3532) (#3583)
Co-authored-by: jaspervdm <j@sper.dev>
2021-03-03 14:36:47 -05:00
Antioch Peverell
03b7518884
cleanup redundant sync_head and associated MMR (#3556) 2021-02-24 19:11:08 +00:00
Antioch Peverell
7649d361e4
Block sync hash traversal perf (#3558)
* sync traversal performance improvements

* rework how body_sync determines block hashes to request
2021-02-15 13:47:59 +00:00
Antioch Peverell
a3aed4aae5
fix for missing block under certain startup conditions (#3555) 2021-02-09 09:13:39 +00:00
jaspervdm
35ebdea967
Bump master version to 5.1.0-alpha.1 (#3510) 2021-01-04 15:29:50 +00:00
Antioch Peverell
eaf9bcf2bf
Block migration db flag (only migrate once) (#3512)
* rework migrate_db_v2_v3

* db flag to track block migration
2020-12-08 09:37:12 +00:00
Antioch Peverell
f48a23655d
bump 5.0.0-beta.2 after fixing windows release script (#3508) 2020-11-26 21:35:10 +00:00
jaspervdm
64c8e0cf0a
Bump version to 5.0.0-beta.1 (#3505) 2020-11-26 18:48:16 +01:00
John Tromp
48efb693e2
Fixfees (#3481)
* add FeeFields type

* use FeeFields with ::zero and try_into().unwrap()

* fixed tests

* avoid 0 accept_base_fee

* add aggregate_fee_fields method for transaction

* implement std::fmt::Display trait for FeeFields

* make base_fee argument non-optional in libtx::mod::tx_fee

* add global and thread local accept_fee_base; use to simplify tests

* set unusually high fee base for a change

* revert to optional base fee argument; default coming from either grin-{server,wallet}.toml

* remove optional base fee argument; can be set with global::set_local_accept_fee_base instead

* define constant global::DEFAULT_ACCEPT_FEE_BASE and set global value

* add Transaction::accept_fee() method and use

* Manual (de)ser impl on FeeFields

* fix comment bug

* Serialize FeeFields as int in tx

* allow feefields: u32:into() for tests

* try adding height args everywhere

* make FeeFields shift/fee methods height dependent

* prior to hf4 feefield testing

* rename selected fee_fields back to fee for serialization compatibility

* fix test_fee_fields test, merge conflict, and doctest use of obsolete fee_fields

* make accept_fee height dependent

* Accept any u64 in FeeFields deser

Co-authored-by: Jasper van der Maarel <j@sper.dev>
2020-11-26 18:03:06 +01:00
jaspervdm
96afc766a1
PIBD segment p2p messages (#3496)
* Define PIBD segment p2p messages

* Respond to segment requests

* Use specialized (de)ser for output bitmap segments

* Allowed segment height ranges in const
2020-11-25 21:52:09 +01:00
Antioch Peverell
6e73e00d6b
fix calls to next_diff() in tests to use next height correctly (#3501) 2020-11-25 12:47:30 +00:00
John Tromp
f86102b2a6
fixDAA; implement wtema (#3477)
* implement wtema

* reduce FTL from 12 to 5 mins; fix inaccuracies in cuckoo doc

* rename difficulty window for DampedMovingAverage DAA

* fix FTL error msg

* secondary_scale becomes extended nonce past HF4

* secondary_scale change actually belongs in hardfork4 PR

* make future time limit configurable

* define constant global::DEFAULT_FUTURE_TIME_LIMIT and refactor a bit

* fix typos

* as_ref() feels safer than as_mut() :-)

* remove obsolete mut

* make future_time_limit non-mandatory in grin-server.toml
2020-11-25 10:58:39 +00:00
John Tromp
a5b8968826
HardFork4 (#3478)
* HardFork4 initial commit

* HF4 obsoletes secondary_scale, which becomes an nonce extension 8->12 bytes

* fix TESTNET_FOURTH_HARD_FORK height

* improve predicted height for testnet fork on dec 8

* move local variable as suggested by antiochp
2020-11-24 16:13:28 +00:00
jaspervdm
055b684416
More efficient serialization for bitmap segments (#3492)
* More efficient serialization for bitmap segments

* Rename a const

* Correctly count number of chunks in a segment

* Enum for BitmapBlock (de)ser mode

* Add more segments in test

* Fix duplicate function
2020-11-24 15:19:07 +01:00
Antioch Peverell
cba3137338
add segmenter for generating segments from txhashset with consistent rewind (#3482)
* add segmenter for generating segments from txhashset with consistent rewind

* rework segmenter to take a txhashset wrapped in rwlock
rework our rewindable pmmr so we can convert to readonly easily

* placeholder code for rewinding readonly txhashset extension to build a rangeproof segment

* segment creation for outputs/rangeproofs/kernels/bitmaps

* placeholder segment impl

* commit

* rework segmenter to use a cached bitmap (rewind is expensive)

* cache segmenter instance based on current archive header

* integrate the real segment and segment identifier with our segmenter

* exercise the segmenter code on chain init

* wrap accumulator in an arc, no need to clone each time
2020-11-23 19:07:07 +00:00
Quentin Le Sceller
cf2a65242d
Rename Floonet to Testnet (#3431)
* Rename Floonet to Testnet

* Fix test

* Rename test and docker
2020-10-07 09:36:02 -04:00
Antioch Peverell
e7bbda81a0
migrate blocks in batches (rework db iterator impl) (#3450)
* rework db iterator
more robust block migration based on low level iterator

* cleanup

* cleanup

* fixup
2020-10-07 11:47:25 +01:00
Antioch Peverell
c3b1ccff3c
Cleanup usage of legacy bitmaps. (#3455)
The legacy bitmaps are no longer required and
have not been in use for a while.
2020-10-02 09:40:04 -04:00
jaspervdm
0aec8b533b
Refactor PMMR read methods into trait (#3454) 2020-09-29 15:57:33 +02:00
Antioch Peverell
62783997df
refactor get_ser to use get_with internally (#3451)
less code duplication and a cleaner code path
batch.exists() now aware of current write tx
2020-09-25 12:47:04 +01:00
Antioch Peverell
a22d98e9d5
bump working version on master to 4.2.0-alpha.1 (#3443)
we now have a current/4.1.x branch for 4.1.0 release
2020-09-15 17:26:44 +01:00
Antioch Peverell
7dc94576bd
Introduce CommitOnly variant of Inputs (#3419)
* Introduce CommitOnly variant of Inputs.
Introduce CommitWrapper so we can sort commit only inputs correctly.

* rememebr to resort if converting

* write inputs based on variant and protocol version

* read and write protocol version specific inputs

* store full blocks in local db in v3
convert to v2 when relaying to v2 peers

* add debug version_str for inputs

* no assumptions about spent index sort order

* add additional version debug logs

* fix ser/deser tests for proto v3

* cleanup coinbase maturity

* rework pool to better handle v2 conversion robustly

* cleanup txpool add_to_pool

* fix nrd kernel test

* move init conversion earlier

* cleanup

* cleanup based on PR feedback
2020-09-07 16:58:41 +01:00
Antioch Peverell
133089e985
Refactor Output Identifiers (#3371)
* refactor output to have internal output identifier

refactor to use AsRef for output identifier

make the output MMR explicit in terms of output identifiers

* put the serde deser back for rangeproof

* add json test for transactions
2020-09-03 10:59:54 +01:00
Antioch Peverell
caa6b8c747
further tweaks to block_accepted logs for clarity (#3379)
* further tweaks to block_accepted logs for clarity

* fix tests

* depth based off prev_head
2020-08-20 17:28:35 +01:00
Antioch Peverell
1cff387f61
Use mmr sizes in header to validate header weight (#3395)
* use mmr sizes in header to validate block weight given header only

* add global weight validaton to headers when reading them untrusted off the network

* fixup tests, one pending test

* add test for deserializing untrusted block header exceeeding weight limits

* validate header mmr sizes, must grow for each block

* remove redundant height check

* fix tests after rebase
block specific TooHeavy error
2020-08-19 09:40:56 +01:00
Antioch Peverell
29cffe9b3c
verify_cut_through and test coverage (#3424)
* fix inconsistent verify_cut_through() logic

* add test coverage for chain::process_block() and cut_through logic

* fix comment
2020-08-18 20:09:54 +01:00
Antioch Peverell
efece9e0c7
use sync pmmr for building locator (#3397) 2020-08-17 13:21:54 -04:00
Antioch Peverell
599bf22cfc
Simplify api commits (#3423)
* simplify api with unspent by commitment

* fix chain tests
2020-08-17 13:19:29 -04:00
Antioch Peverell
34adaf797e
orphan check needs to handle fast sync "edge case" (#3418) 2020-08-08 11:21:38 +01:00
Antioch Peverell
4732a0b62b
convert tx for v2 compatibility on way into stempool/txpool (#3412)
cleanup passing extra_tx around

use output identifiers when converting tx to v2
2020-08-02 13:26:28 +01:00
Antioch Peverell
04c8713d83
refactor apply_inputs and support converting block for v2 compatibility (#3409)
* wip

* convert incoming block to v2, rework orphan and duplicate check earlier than conversion

* cleanup process_block_single

* cleanup block conversion

* cleanup

* leverage utxo_view when applying block inputs
2020-07-31 14:36:20 +01:00
Antioch Peverell
83b269961a
introduce Inputs enum variants for future commit only support (#3406) 2020-07-28 21:21:57 +01:00
Antioch Peverell
80841f16d2
pass slices around and not refs to vecs (#3404)
* pass slices around and not refs to vecs

* use slice.swap()

* use inputs() not body.inputs
2020-07-27 11:07:18 +01:00
Antioch Peverell
3c06672e9b
include height even for spent outputs in get_outputs api (#3400) 2020-07-22 08:32:09 -04:00
Antioch Peverell
824c51a346
bump version to 4.1.0 on master (4.0.1 on 4.0.x branch) (#3390) 2020-07-13 12:16:43 +01:00
Antioch Peverell
84ad59c5be
more robust handling of min_height and max_height in get_kernel_height() (#3387)
* more robust handling of min_height and max_height in get_kernel_height()

* cleanup
2020-07-13 11:31:52 +01:00
Antioch Peverell
32253194b5
log reorg (and fork) depth correctly (#3376)
* log reorg (and fork) depth correctly
depth should be based on "fork point" not prev head

* correct depth for fork/reorg (0 for alternate head etc.)
2020-07-08 09:02:28 +01:00
Antioch Peverell
30db9c410e
bump master to 4.0.1-alpha.1 to differentiate from 4.0.0 official tagged build (#3374) 2020-07-02 15:51:11 +01:00
Yeastplume
7abe9fcb39
Versioning for 4.0.0 release (#3373) 2020-07-02 10:29:10 +01:00
Yeastplume
098d25e569
Version bump for 4.0.0-rc.1 (#3366) 2020-06-26 13:10:42 +01:00
Antioch Peverell
20b4500625
make sure header PMMR is init correctly based on header_head from db (#3362)
* make sure header PMMR is init correctly based on header_head from db

* fix to ensure header PMMR is consistent with header_head in db on chain init

* change order of operations - validate header earlier
and avoid applying header to header PMMR before validation
as this potentially leaves the PMMR in a tricky state to rewind from
2020-06-22 10:04:09 -04:00
Antioch Peverell
55a929e214
cleanup old unused code (#3355) 2020-06-15 15:05:47 +01:00
Antioch Peverell
dd88d28a7a
bump version to 4.0.0-beta.2 (#3351) 2020-06-12 14:08:12 +01:00
Antioch Peverell
20e5c1910b
NRD rules and "recent" kernel pos index (#3302)
* variants for output_pos linked list entries (head/tail/middle/unique)
next and prev and Vec<u8> lmdb keys

* get_pos on enum

* break list and list entries out into separate enums

* track output features in the new output_pos index, so we can determine coinbase maturity

* push entry impl for none and unique

* some test coverage for output_pos_list

* commit

* wip - FooListEntry

* use instance of the index

* linked list of output_pos and commit_pos both now supported

* linked_list

* cleanup and rename

* rename

* peek_pos

* push some, peek some, pop some

* cleanup

* commit pos
cleanup

* split list and entry out into separate db prefixes

* cleanup and add placeholder for pop_back

* pop_pos_back (for popping off the back of the linked list)
test coverage for pop_pos_back

* wip

* placeholder for prune via a trait
pos must always increase in the index

* rewind kernel_pos_idx when calling rewind_single_block

* RewindableListIndex with rewind support.

* test coverage for rewindable list index

* test coverage for rewind back to 0

* rewind past end of list

* add tests for kernel_pos_idx with multiple commits

* commit

* cleanup

* hook NRD relative lock height validation into block processing and tx validation

* cleanup

* set local chain type for kernel_idx tests

* add test coverage for NRD rules in block processing

* NRD test coverage and cleanup

* NRD relative height 1 test

* test coverage for NRD kernels in block processing

* cleanup

* start of test coverage for txpool NRD kernel rules

* wip

* rework pool tests to use real chain (was mock chain) to better reflect reality (tx/block validation rules etc.)

* cleanup

* cleanup pruneable trait for kernel pos index

* add clear() to kernel_pos idx and test coverage

* hook kernel_pos rebuild into node startup, compaction and fast sync

* verify full NRD history on fast sync

* return early if nrd disabled

* fix header sync issue
2020-06-10 16:38:29 +01:00
Antioch Peverell
b98e5e06a6
fix race condition (#3344) 2020-06-10 13:56:33 +01:00
Quentin Le Sceller
093c500fed
Cargo Update (#3338) 2020-06-08 17:15:32 -04:00
Yeastplume
c7c9a32b9b
4.0.0-beta.1 Version Bump (#3339) 2020-06-04 14:52:26 +01:00
Antioch Peverell
5b825fbf0d
check_known now takes total_difficulty into consideration (#3298)
* check_known now takes total_difficulty into consideration

* update based on feedback
2020-05-29 10:21:12 +01:00
Antioch Peverell
988a05f023
Enable NRD kernel support (noop) feature flagged (#3303)
* wip

* commit

* add block level validation rule around NRD kernels and HF3 block height

* wip

* test coverage for kernel ser/deser for NRD kernels

* add some type safety around ser/deser of nrd relative height

* cleanup

* cleanup tx sig handling and add tests around NRD kernel sig

* test coverage for chain apply block containing NRD kernel

* verify kernel variants when adding tx to pool
NRD kernels will not be accepted until HF3

* add txpool test for NRD kernel handling

* fix merge

* fix api for NRD kernels

* commit

* wip - feature flag for NRD kernel support

* wip

* global config and thread local feature flag for NRD kernel support

* add feature flag support for NRD kernels
default to false when starting up node
allow it to be set to true in context of individual tests

* feature flag

* explicit testing of NRD kernel via feature flag

* add chain_type and feature flag logging on startup

* PR feedback and cleanup

* PR feedback
2020-05-29 09:56:24 +01:00
Antioch Peverell
6faa0e8d75
thread local chain type vs global chain type (#3327)
* Introduce GLOBAL_CHAIN_TYPE and make CHAIN_TYPE thread_local.
This makes testing more explicit and significantly more robust.

* set_local_chain_type() in tests

* cleanup - weird

* get pool tests working with explicit local chain_type config

* core tests working with explicit local chain_type

* p2p tests working with explicit local chain_type

* store tests working

* cleanup, feedback
2020-05-22 12:51:58 +01:00