Commit graph

517 commits

Author SHA1 Message Date
Yeastplume
41f3aafb9a
Merge DNSSeed scope changes into pibd impl branch (#3708)
* update Cargo.lock for next release

* visibility scope tweaks to aid seed test utilities (#3707)
2022-05-16 10:11:31 +01:00
Yeastplume
5efd70a17b
[PIBD_IMPL] BitmapAccumulator Serialization Fix (#3705)
* fix for writing / calculating incorrect length for negative indices

* update capabilities with new version of PIBD hist

* remove incorrect comment

* fix capabilities flag, trace output

* test fix
2022-04-20 13:44:53 +01:00
Yeastplume
aa2a2a98df
add pibd abort timeout case (#3704) 2022-04-05 12:46:05 +01:00
Yeastplume
eda31ab9e5
documentation updates + todo fixes (#3703) 2022-04-01 13:42:12 +01:00
Yeastplume
6a7b66b329
[PIBD_IMPL] Catch-Up functionality + Fixes based on testing (#3702)
* ensure desegmenter attempts to apply correct block after a resumte

* ensure txhashset's committed implementation takes into account output bitmap for summing purposes

* remove check to de-apply outputs during segment application

* return removal of spent outputs during pibd

* remove unneeded status

* remove uneeded change to rewind function
2022-03-30 13:47:42 +01:00
Yeastplume
21b1ac50d7
[PIBD_IMPL] Thread simplification + More TUI Updates + Stop State Propagation (#3698)
* change pibd stat display to show progress as a percentage of downloaded leaves

* attempt some inline rp validation

* propagate shutdown state through kernel validation

* change validation loop timing

* simplify validator threading

* add more detailed tracking of kernel history validation to tui, allow stop state during

* adding more stop state + tui progress indication

* remove progressive validate

* test fix
2022-03-01 13:52:16 +00:00
Yeastplume
bf48e52374
[PIBD_IMPL] Update number of simultaneous peer requests for segments (#3696)
* cleanup of segment request list

* allow for more simultaneous requests during state sync

* up number of simultaneous peer requests for segments
2022-02-24 15:20:35 +00:00
Yeastplume
5630cf2e10
[PIBD_IMPL] PIBD Stats + Retry on validation errors (#3694)
* start to add stats and reset chain state after errors detected

* add functions to reset prune list when resetting chain pibd state

* debug statement

* remove test function
2022-02-17 14:10:56 +00:00
Yeastplume
3ea233d5eb
[PIBD_IMPL] Finalize PIBD download and move state to chain validation (#3692)
* investigations as to why a slight rewind is needed on startup during PIBD

* move validation code into desegmenter validation thread (for now)

* ensure genesis entries in pmmrs are removed if they're removed in the first segment

* validation all working except for verifying kernel sums

* remove unneeded pmmr rollbacks on resume now root cause was found

* updates to remove unpruned leaves from leaf set when rebuilding pmmr

* remove + 1 to segment traversal iter length
2022-02-15 13:04:14 +00:00
Yeastplume
169e106e70
[PIBD_IMPL] PIBD tree sync via network and kill/resume functionality (#3691)
* add functions to determing latest verifiable block height for the given pibd state

* attempting to allow for pibd to resume after killing process

* fix to ensure prune list is properly flushed during pibd sync

* removal of unneeded code

* ignore test for now (fix before full merge)
2022-02-08 12:24:07 +00:00
Yeastplume
24202f0442
[PIBD_IMPL] PMMR Reassembly from Segments (#3690)
* update pibd copy test to use new desgmenter structure

* begin reconstruction of output pmmr

* clean up hash/leaf insertion logic

* push pruned subtree appears to be working, now also calculates left hand hashes correctly

* factor out ordering of segment/hash order array

* refactor for pmmr application code

* test of chain copy appears to be working

* add rangeproof functions to desegmenter

* add kernel functions, attempt refactor

* small test cleanup, reconstruction of live chain working in manual copy test
2022-01-28 11:32:54 +00:00
Yeastplume
436bacf17e
[PIBD_IMPL] Bitmap accumulator reconstruction + TxHashset set reconstruction (#3689)
* application of received bitmap segments to local accumulator

* add all required elements to send/receive output segment requests and responses

* testing of output sync

* add special cases to pmmr segment request
2022-01-25 10:38:13 +00:00
Yeastplume
41a86b4cd9
[PIBD_IMPL] PIBD Desegmenter State (#3688)
* add functions to desegmenter to report next desired segments, begin to add state to determine which segments have been requested

* add segmentidentifier type to id requested segments uniquely

* make a call on where to keep track of which PIBD segments have been requested

* move segmenttype definition, add functions to manipulate peer segment list

* remove desegmenter state enum

* change chain desegmenter function to provide rwlock

* trace, warning cleanup

* udpate to test compliation
2022-01-20 09:57:21 +00:00
Yeastplume
009a02eec1
add pibd receive messages to network, and basic calls to desegmenter from each (#3686) 2022-01-17 10:08:58 +00:00
Yeastplume
89730b7d6d
[PIBD_IMPL] Introduce PIBD state into sync workflow (#3685)
* experimental addition of pibd download state for testnet only

* fixes to bitmap number of segments calculation + conversion of bitmap accumulator to bitmap

* attempt to call a test message

* add p2p methods for receiving bitmap segment and applying to desegmenter associated with chain

* fixes to state sync
2022-01-12 13:02:59 +00:00
Yeastplume
78c9794d30
TUI Freeze Fix - Add manually maintained hash to difficulty iterator (#3684)
* add manually managed hash to difficulty iterator

* text fix

* fix for hash-of-hash, review feedback
2022-01-07 14:23:58 +00:00
deevope
a3eebbc0ab
verify-chain node client arg (#3678)
* add parameter fast_validation to the api

* arg `verify-chain` on node client

* arg verify chain + read_timeout parameter

* change timeout to 2hours

* update with tromp review

* 2nd comment from tromp
2022-01-07 13:45:48 +00:00
deevope
c92d2e9fba
graceful api shutdown (#3677) 2022-01-07 13:24:54 +00:00
Yeastplume
2237f42144
Handle attempts to get block headers at invalid heights (#3683)
* handle attempts to get block headers at invalid heights

* compare requested height against header pmmr size

* gte
2021-12-31 09:47:04 +00:00
Yeastplume
63c65605bb
[SYNC PERFORMANCE] Adjust DifficultyIterator to no longer deserialize PoW proof nonces (#3671)
* replace bitvec with more efficient bitpack algorithm

* optimise proof_unpack_len

* move proof pack length calculation

* small refactor

* first pass attempt at not deserializing proof nonces in difficulty iter

* another 10 seconds gained by not deserialising the proof from the difficulty iterator

* add new deser parameters to tests where needed

* add skip_proof variants to store

* remove hash from difficulty iterator struct, rename HeaderInfo to HeaderDifficultyInfo

* replace bitvec with more efficient bitpack algorithm

* optimise proof_unpack_len

* move proof pack length calculation

* small refactor

* first pass attempt at not deserializing proof nonces in difficulty iter

* another 10 seconds gained by not deserialising the proof from the difficulty iterator

* add new deser parameters to tests where needed

* add skip_proof variants to store

* remove hash from difficulty iterator struct, rename HeaderInfo to HeaderDifficultyInfo
2021-12-10 11:39:39 +00:00
Quentin Le Sceller
2e2149859f
bump to 5.2.0-alpha.1 on master (#3652) 2021-08-19 09:06:52 +02:00
Quentin Le Sceller
11e5efc089
Tagging v5.1.1 release (#3651) 2021-08-17 20:57:13 +02:00
John Tromp
f51b6e1376
implement fix past fees RFC with fee shift since genesis (#3629) 2021-05-11 16:39:09 +01:00
Antioch Peverell
a9f1dd7bcd
bump to 5.2.0-alpha.1 on master (#3640) 2021-05-06 13:46:03 +01:00
Antioch Peverell
72a6eb81e8
bump to 5.1.0 to prep for release (#3639) 2021-05-06 13:02:01 +01:00
Antioch Peverell
87729a2227
bump crate version to 5.1.0-beta.2 to prep for release (#3636) 2021-04-29 13:05:28 +01:00
Antioch Peverell
89c06ddab7
client invalidateheader and resetchainhead (#3618)
* wip - "reset_head" via owner api functionality

* jsonrpc pass hash in as a string

* sort of works

* not a reorg if we simply accept several blocks at once

* remember to reset header MMR separately
as it is readonly when interacting with txhashset extension

* basic client integration
needs error handling etc.

* reset sync status when reset chain head

* track "denylist" (todo) and validate headers against this via the ctx

* track denylist (header hashes) in chain itself

* header denylist in play

* expose invalidateheader as client cmd

* rework reset_chain_head - rewind txhashset then header MMR
2021-04-29 11:05:05 +01:00
Blade Doyle
9ed0cd65ef
fix share difficulty validation - compare stratum server minimum_share_difficulty as unscaled difficulty (#3624)
* fix for comparing scaled vs unscaled difficulty

fix mix use of scaled and unscaled, add additional comments to clarify which values are which

improved consistency - scaled and unscaled difficulty

* combine from_proof_unscaled() and  to_unscaled_difficulty{}

* code review - rust syntax
2021-04-23 09:28:29 -04:00
Antioch Peverell
34413c1cab
track sync_head on header_sync sync status (#3626)
* track sync_head on header_sync sync status
follow header fork as appropriate

* track highest_diff alongside highest_height on HeaderSync status

* cleanup

* not not
2021-04-06 11:16:20 +01:00
Antioch Peverell
f6ec77a592
never speak of the verifier cache again (#3628) 2021-04-01 15:04:53 +01:00
Antioch Peverell
6690b25f05
Enable block archival sync (#3579)
* wip - body sync for full archive

* allow chain compaction during sync

* placeholder for logic to ensure archive nodes sync from archive nodes

* body sync from archival peers

* allow chain compaction during sync

* placeholder for logic to ensure archive nodes sync from archive nodes
2021-03-16 12:04:09 +00:00
Blade Doyle
846b8f82e8
fix for missing increment of stratum stats blocks_found (#3593) 2021-03-11 10:13:15 -05:00
Quentin Le Sceller
0259ed23ea
Update copyright year to 2021 (#3592)
* Update copyright year to 2021
2021-03-10 10:14:48 -05:00
Quentin Le Sceller
725622da7a
Remove imports and cleanups (#3590)
* Remove unused import and if else chain
2021-03-08 11:09:41 -05:00
Antioch Peverell
35740204a1
no more conversion to support v2 blocks (#3587)
get_block simply returns None if v2 support required
2021-03-06 21:40:20 +00:00
Blade Doyle
059e6ad545
Mining tui updates (#3584)
* add column ordering, set mining workers table to sort the connected workers to the top by default

* add column ordering, set mining workers table to sort the connected workers to the top by default

* Mining TUI fixes and updates
2021-03-04 09:39:14 -05:00
Quentin Le Sceller
64b2fddbf1
[5.0.x] Fix PIBD segments (#3532) (#3583)
Co-authored-by: jaspervdm <j@sper.dev>
2021-03-03 14:36:47 -05:00
Antioch Peverell
4284458e17
more consistent handling of peers_preferred during startup (#3578) 2021-03-02 20:34:55 +00:00
Antioch Peverell
03b7518884
cleanup redundant sync_head and associated MMR (#3556) 2021-02-24 19:11:08 +00:00
Antioch Peverell
08523b2b39
add BLOCK_HIST capabilities on startup if config archive_mode=true (#3563) 2021-02-23 11:40:48 +00:00
Antioch Peverell
4ab72902e0
add rate limiting to outbound p2p msg sending (#3560)
treat peers as abusive based on incoming msgs, not outgoing msg rates
2021-02-15 13:49:31 +00:00
Antioch Peverell
7649d361e4
Block sync hash traversal perf (#3558)
* sync traversal performance improvements

* rework how body_sync determines block hashes to request
2021-02-15 13:47:59 +00:00
jaspervdm
35ebdea967
Bump master version to 5.1.0-alpha.1 (#3510) 2021-01-04 15:29:50 +00:00
Antioch Peverell
1baa59c44d
prefer outbound peers when syncing (consistently) (#3521)
but use inbound peer for header and body sync if necessary
sync state from inbound peer if no outbound peers to sync from
2020-12-15 19:11:51 +00:00
Antioch Peverell
f48a23655d
bump 5.0.0-beta.2 after fixing windows release script (#3508) 2020-11-26 21:35:10 +00:00
jaspervdm
64c8e0cf0a
Bump version to 5.0.0-beta.1 (#3505) 2020-11-26 18:48:16 +01:00
John Tromp
48efb693e2
Fixfees (#3481)
* add FeeFields type

* use FeeFields with ::zero and try_into().unwrap()

* fixed tests

* avoid 0 accept_base_fee

* add aggregate_fee_fields method for transaction

* implement std::fmt::Display trait for FeeFields

* make base_fee argument non-optional in libtx::mod::tx_fee

* add global and thread local accept_fee_base; use to simplify tests

* set unusually high fee base for a change

* revert to optional base fee argument; default coming from either grin-{server,wallet}.toml

* remove optional base fee argument; can be set with global::set_local_accept_fee_base instead

* define constant global::DEFAULT_ACCEPT_FEE_BASE and set global value

* add Transaction::accept_fee() method and use

* Manual (de)ser impl on FeeFields

* fix comment bug

* Serialize FeeFields as int in tx

* allow feefields: u32:into() for tests

* try adding height args everywhere

* make FeeFields shift/fee methods height dependent

* prior to hf4 feefield testing

* rename selected fee_fields back to fee for serialization compatibility

* fix test_fee_fields test, merge conflict, and doctest use of obsolete fee_fields

* make accept_fee height dependent

* Accept any u64 in FeeFields deser

Co-authored-by: Jasper van der Maarel <j@sper.dev>
2020-11-26 18:03:06 +01:00
jaspervdm
96afc766a1
PIBD segment p2p messages (#3496)
* Define PIBD segment p2p messages

* Respond to segment requests

* Use specialized (de)ser for output bitmap segments

* Allowed segment height ranges in const
2020-11-25 21:52:09 +01:00
deevope
fd5dfaaec4
reorg cache fix (#3495)
* reorg cache period configurable

* fix comment

* u32 type

Co-authored-by: deevope <you@example.com>
2020-11-25 17:59:37 +00:00
Antioch Peverell
3efe382e9c
add peer capabilities to tui peers screen (#3490) 2020-11-25 16:29:33 +00:00