Commit graph

2433 commits

Author SHA1 Message Date
Antioch Peverell
a3c9b478e2
[5.0.x] inefficient locking on recv of peers lists can result in failure to get peers lock (#3566) (#3570)
* fix for: inefficient locking of peers lists can result in failure to get peers lock

* dont hold the peers Vec lock while writing to the peers lmdb

Co-authored-by: Blade Doyle <bladedoyle@gmail.com>
2021-02-22 15:18:09 +00:00
Antioch Peverell
4de2d92433
Revert "rework prune rewrite with iterators (#3568)" (#3569)
This reverts commit 5092652b0c.
2021-02-22 14:18:23 +00:00
Antioch Peverell
5092652b0c
rework prune rewrite with iterators (#3568)
* rework prune rewrite with iterators

* fix
2021-02-19 11:59:40 +00:00
Antioch Peverell
5b9664bccf
inc quietly on small batches of headers (#3564) 2021-02-16 14:36:28 +00:00
Antioch Peverell
524dbd0170
pibd capabilities test was only relevant temporarily (#3565)
while we did not enable it by default
2021-02-16 11:28:01 +00:00
Antioch Peverell
4ab72902e0
add rate limiting to outbound p2p msg sending (#3560)
treat peers as abusive based on incoming msgs, not outgoing msg rates
2021-02-15 13:49:31 +00:00
Antioch Peverell
7649d361e4
Block sync hash traversal perf (#3558)
* sync traversal performance improvements

* rework how body_sync determines block hashes to request
2021-02-15 13:47:59 +00:00
Antioch Peverell
a3aed4aae5
fix for missing block under certain startup conditions (#3555) 2021-02-09 09:13:39 +00:00
jaspervdm
61982efdd9
Safety: check length during deserialization (#3551) 2021-02-02 09:56:00 -05:00
John Tromp
4877f30e2c
override old accept_fee_base setting of 1000000 to new default 500000 (#3539)
* override old accept_fee_base setting of 1000000 to new default 500000

* document exceptional accept_fee_base behaviour in grin-server.toml

* document more accurately
2021-01-06 15:24:11 +00:00
David Burkett
6f142b4c75
Downgrade CI VM to Ubuntu 18.04 (#3535) 2021-01-05 08:44:41 +00:00
jaspervdm
35ebdea967
Bump master version to 5.1.0-alpha.1 (#3510) 2021-01-04 15:29:50 +00:00
Antioch Peverell
1baa59c44d
prefer outbound peers when syncing (consistently) (#3521)
but use inbound peer for header and body sync if necessary
sync state from inbound peer if no outbound peers to sync from
2020-12-15 19:11:51 +00:00
John Tromp
acba73bf40
reomve unused function (#3509) 2020-12-11 09:26:58 -05:00
Antioch Peverell
eaf9bcf2bf
Block migration db flag (only migrate once) (#3512)
* rework migrate_db_v2_v3

* db flag to track block migration
2020-12-08 09:37:12 +00:00
Antioch Peverell
f48a23655d
bump 5.0.0-beta.2 after fixing windows release script (#3508) 2020-11-26 21:35:10 +00:00
Antioch Peverell
623e7d2e59
refreshenv as part of windows release script (#3507) 2020-11-26 21:27:37 +00:00
jaspervdm
64c8e0cf0a
Bump version to 5.0.0-beta.1 (#3505) 2020-11-26 18:48:16 +01:00
John Tromp
48efb693e2
Fixfees (#3481)
* add FeeFields type

* use FeeFields with ::zero and try_into().unwrap()

* fixed tests

* avoid 0 accept_base_fee

* add aggregate_fee_fields method for transaction

* implement std::fmt::Display trait for FeeFields

* make base_fee argument non-optional in libtx::mod::tx_fee

* add global and thread local accept_fee_base; use to simplify tests

* set unusually high fee base for a change

* revert to optional base fee argument; default coming from either grin-{server,wallet}.toml

* remove optional base fee argument; can be set with global::set_local_accept_fee_base instead

* define constant global::DEFAULT_ACCEPT_FEE_BASE and set global value

* add Transaction::accept_fee() method and use

* Manual (de)ser impl on FeeFields

* fix comment bug

* Serialize FeeFields as int in tx

* allow feefields: u32:into() for tests

* try adding height args everywhere

* make FeeFields shift/fee methods height dependent

* prior to hf4 feefield testing

* rename selected fee_fields back to fee for serialization compatibility

* fix test_fee_fields test, merge conflict, and doctest use of obsolete fee_fields

* make accept_fee height dependent

* Accept any u64 in FeeFields deser

Co-authored-by: Jasper van der Maarel <j@sper.dev>
2020-11-26 18:03:06 +01:00
Antioch Peverell
14f4683ca1
enable PIBD_HIST capability by default (#3500) 2020-11-26 09:53:00 +00:00
jaspervdm
96afc766a1
PIBD segment p2p messages (#3496)
* Define PIBD segment p2p messages

* Respond to segment requests

* Use specialized (de)ser for output bitmap segments

* Allowed segment height ranges in const
2020-11-25 21:52:09 +01:00
Quentin Le Sceller
9abb6e3e01
Disable API v1 (#3503)
* Disable API v1

* Fix missing path in foreign_api_secret
2020-11-25 19:36:29 +00:00
Antioch Peverell
cbd6c3350d
fix, no more FULL_NODE (#3504) 2020-11-25 18:31:38 +00:00
deevope
fd5dfaaec4
reorg cache fix (#3495)
* reorg cache period configurable

* fix comment

* u32 type

Co-authored-by: deevope <you@example.com>
2020-11-25 17:59:37 +00:00
Antioch Peverell
3efe382e9c
add peer capabilities to tui peers screen (#3490) 2020-11-25 16:29:33 +00:00
John Tromp
97425af637
make min_wtema difficulty chain type dependent (#3502)
* make min_wtema difficulty chain type dependent

* fix bad graph weights for automated and user testing

* only mainnet can get wtema difficulty unstuck at 59s blocks
2020-11-25 15:13:43 +00:00
Antioch Peverell
6e73e00d6b
fix calls to next_diff() in tests to use next height correctly (#3501) 2020-11-25 12:47:30 +00:00
John Tromp
f86102b2a6
fixDAA; implement wtema (#3477)
* implement wtema

* reduce FTL from 12 to 5 mins; fix inaccuracies in cuckoo doc

* rename difficulty window for DampedMovingAverage DAA

* fix FTL error msg

* secondary_scale becomes extended nonce past HF4

* secondary_scale change actually belongs in hardfork4 PR

* make future time limit configurable

* define constant global::DEFAULT_FUTURE_TIME_LIMIT and refactor a bit

* fix typos

* as_ref() feels safer than as_mut() :-)

* remove obsolete mut

* make future_time_limit non-mandatory in grin-server.toml
2020-11-25 10:58:39 +00:00
John Tromp
a5b8968826
HardFork4 (#3478)
* HardFork4 initial commit

* HF4 obsoletes secondary_scale, which becomes an nonce extension 8->12 bytes

* fix TESTNET_FOURTH_HARD_FORK height

* improve predicted height for testnet fork on dec 8

* move local variable as suggested by antiochp
2020-11-24 16:13:28 +00:00
Antioch Peverell
2125c05020
use extend_from_slice when appending multiple hashes to hash file (#3497)
* use extend_from_slice when appending multiple hashes to hash file

* bump
2020-11-24 14:24:19 +00:00
jaspervdm
055b684416
More efficient serialization for bitmap segments (#3492)
* More efficient serialization for bitmap segments

* Rename a const

* Correctly count number of chunks in a segment

* Enum for BitmapBlock (de)ser mode

* Add more segments in test

* Fix duplicate function
2020-11-24 15:19:07 +01:00
Antioch Peverell
b3938de8b3
we have no patch file and nothing to patch (#3499) 2020-11-23 15:02:02 -05:00
Antioch Peverell
cba3137338
add segmenter for generating segments from txhashset with consistent rewind (#3482)
* add segmenter for generating segments from txhashset with consistent rewind

* rework segmenter to take a txhashset wrapped in rwlock
rework our rewindable pmmr so we can convert to readonly easily

* placeholder code for rewinding readonly txhashset extension to build a rangeproof segment

* segment creation for outputs/rangeproofs/kernels/bitmaps

* placeholder segment impl

* commit

* rework segmenter to use a cached bitmap (rewind is expensive)

* cache segmenter instance based on current archive header

* integrate the real segment and segment identifier with our segmenter

* exercise the segmenter code on chain init

* wrap accumulator in an arc, no need to clone each time
2020-11-23 19:07:07 +00:00
Antioch Peverell
5282ecb12f
try and call refreshenv before running tests on windows (#3498)
* try and call refreshenv before running tests on windows

* try again
2020-11-23 16:58:59 +00:00
Antioch Peverell
b5c6d24b40
default capabilities cleanup (#3484)
* default capabilities cleanup

* fix tests for unused pibd capabilities
2020-11-21 21:49:19 +00:00
jaspervdm
8faba4ef83
PMMR segment creation and validation (#3453)
* Chunk generation and validation

* Rename chunk -> segment

* Missed a few

* Generate and validate merkle proof

* Fix bugs in generation and validation

* Add test for unprunable MMR of various sizes

* Add missing docs

* Remove unused functions

* Remove segment error variant on chain error type

* Simplify calculation by using a Vec instead of HashMap

* Use vectors in segment definition

* Compare subtree root during tests

* Add test of segments for a prunable mmr

* Remove assertion

* Only send intermediary hashes for prunable MMRs

* Get hash from file directly

* Require both leaves if one of them is not pruned

* More pruning tests

* Add segment (de)serialization

* Require sorted vectors in segment deser

* Store pos and data separately in segment

* Rename log_size -> height

* Fix bitmap index in root calculation

* Add validation function for output (bitmap) MMRs

* Remove left over debug statements

* Fix test

* Edge case: final segment with uneven number of leaves

* Use last_pos instead of segment_last_pos

* Simplify pruning in test

* Add leaf and hash iterators

* Support fully pruned segments

* Drop backend before deleting dir in pruned_segment test

* Simplify output of first_unpruned_parent
2020-11-17 19:38:44 +01:00
Antioch Peverell
e6145db743
Disable merkle proofs for v2 get_block (#3487)
* api v2 get_blocks should not include Merkle proofs

* default to not including merkle proofs in v2 get_blocks api
2020-11-09 14:56:01 -05:00
Antioch Peverell
49cccea035
on_block_accepted logging and webhooks all the time (#3486) 2020-11-09 12:11:27 -05:00
Antioch Peverell
25fcefada2
Flexible peer filtering (#3458)
* first pass at peers iter cleanup

* more flexible peer with diff lookups

* PeersIter with impl Iterator

* sync against outbound peers
reorder peers filters so expensive filters come later

* filter peers by capabilities during sync

* prefer outbound peers with high total difficulty

* with_difficulty now takes a fn to allow more flexible comparisons based on difficulty

* rename peers_iter() -> iter()
2020-10-27 12:36:00 +00:00
Daniel Lehnberg
cea546ceb8
fix: correct typo in SECURITY.md (#3462) 2020-10-08 09:46:48 -04:00
Antioch Peverell
145c2874f5
fix v2 conversion to ensure we provide blocks in correct format (#3465) 2020-10-08 09:27:00 +01:00
Quentin Le Sceller
cf2a65242d
Rename Floonet to Testnet (#3431)
* Rename Floonet to Testnet

* Fix test

* Rename test and docker
2020-10-07 09:36:02 -04:00
jaspervdm
4c6d1dd4bd
Define get_data_from_file function on ReadablePMMR trait (#3464) 2020-10-07 14:09:27 +02:00
Antioch Peverell
e7bbda81a0
migrate blocks in batches (rework db iterator impl) (#3450)
* rework db iterator
more robust block migration based on low level iterator

* cleanup

* cleanup

* fixup
2020-10-07 11:47:25 +01:00
Quentin Le Sceller
eab26b3ae4
CI: no directories in checksum file (#3460) 2020-10-02 13:14:02 -04:00
Antioch Peverell
c3b1ccff3c
Cleanup usage of legacy bitmaps. (#3455)
The legacy bitmaps are no longer required and
have not been in use for a while.
2020-10-02 09:40:04 -04:00
Antioch Peverell
f25b7ae032
use choose_multiple to clean get_peers() up (#3457) 2020-10-01 14:42:57 -04:00
Antioch Peverell
cc4e608d1b
cleanup validate() no need to return kernel sums (#3456) 2020-10-01 14:56:37 +01:00
jaspervdm
0aec8b533b
Refactor PMMR read methods into trait (#3454) 2020-09-29 15:57:33 +02:00
jaspervdm
defc714f6e
Refactor p2p reader (#3433)
* Refactor p2p reader

* Rename Output enum

* Consume enum takes owned values instead of references

* Deserialization in codec, remove Consume enum

* Calculate block header size

* Read headers in batches

* Remove headers type from deserializer
2020-09-28 14:43:37 +01:00