Commit graph

476 commits

Author SHA1 Message Date
Antioch Peverell
7dc94576bd
Introduce CommitOnly variant of Inputs (#3419)
* Introduce CommitOnly variant of Inputs.
Introduce CommitWrapper so we can sort commit only inputs correctly.

* rememebr to resort if converting

* write inputs based on variant and protocol version

* read and write protocol version specific inputs

* store full blocks in local db in v3
convert to v2 when relaying to v2 peers

* add debug version_str for inputs

* no assumptions about spent index sort order

* add additional version debug logs

* fix ser/deser tests for proto v3

* cleanup coinbase maturity

* rework pool to better handle v2 conversion robustly

* cleanup txpool add_to_pool

* fix nrd kernel test

* move init conversion earlier

* cleanup

* cleanup based on PR feedback
2020-09-07 16:58:41 +01:00
Antioch Peverell
133089e985
Refactor Output Identifiers (#3371)
* refactor output to have internal output identifier

refactor to use AsRef for output identifier

make the output MMR explicit in terms of output identifiers

* put the serde deser back for rangeproof

* add json test for transactions
2020-09-03 10:59:54 +01:00
Antioch Peverell
caa6b8c747
further tweaks to block_accepted logs for clarity (#3379)
* further tweaks to block_accepted logs for clarity

* fix tests

* depth based off prev_head
2020-08-20 17:28:35 +01:00
Antioch Peverell
1cff387f61
Use mmr sizes in header to validate header weight (#3395)
* use mmr sizes in header to validate block weight given header only

* add global weight validaton to headers when reading them untrusted off the network

* fixup tests, one pending test

* add test for deserializing untrusted block header exceeeding weight limits

* validate header mmr sizes, must grow for each block

* remove redundant height check

* fix tests after rebase
block specific TooHeavy error
2020-08-19 09:40:56 +01:00
Antioch Peverell
29cffe9b3c
verify_cut_through and test coverage (#3424)
* fix inconsistent verify_cut_through() logic

* add test coverage for chain::process_block() and cut_through logic

* fix comment
2020-08-18 20:09:54 +01:00
Antioch Peverell
efece9e0c7
use sync pmmr for building locator (#3397) 2020-08-17 13:21:54 -04:00
Antioch Peverell
599bf22cfc
Simplify api commits (#3423)
* simplify api with unspent by commitment

* fix chain tests
2020-08-17 13:19:29 -04:00
Antioch Peverell
34adaf797e
orphan check needs to handle fast sync "edge case" (#3418) 2020-08-08 11:21:38 +01:00
Antioch Peverell
4732a0b62b
convert tx for v2 compatibility on way into stempool/txpool (#3412)
cleanup passing extra_tx around

use output identifiers when converting tx to v2
2020-08-02 13:26:28 +01:00
Antioch Peverell
04c8713d83
refactor apply_inputs and support converting block for v2 compatibility (#3409)
* wip

* convert incoming block to v2, rework orphan and duplicate check earlier than conversion

* cleanup process_block_single

* cleanup block conversion

* cleanup

* leverage utxo_view when applying block inputs
2020-07-31 14:36:20 +01:00
Antioch Peverell
83b269961a
introduce Inputs enum variants for future commit only support (#3406) 2020-07-28 21:21:57 +01:00
Antioch Peverell
80841f16d2
pass slices around and not refs to vecs (#3404)
* pass slices around and not refs to vecs

* use slice.swap()

* use inputs() not body.inputs
2020-07-27 11:07:18 +01:00
Antioch Peverell
3c06672e9b
include height even for spent outputs in get_outputs api (#3400) 2020-07-22 08:32:09 -04:00
Antioch Peverell
84ad59c5be
more robust handling of min_height and max_height in get_kernel_height() (#3387)
* more robust handling of min_height and max_height in get_kernel_height()

* cleanup
2020-07-13 11:31:52 +01:00
Antioch Peverell
32253194b5
log reorg (and fork) depth correctly (#3376)
* log reorg (and fork) depth correctly
depth should be based on "fork point" not prev head

* correct depth for fork/reorg (0 for alternate head etc.)
2020-07-08 09:02:28 +01:00
Antioch Peverell
20b4500625
make sure header PMMR is init correctly based on header_head from db (#3362)
* make sure header PMMR is init correctly based on header_head from db

* fix to ensure header PMMR is consistent with header_head in db on chain init

* change order of operations - validate header earlier
and avoid applying header to header PMMR before validation
as this potentially leaves the PMMR in a tricky state to rewind from
2020-06-22 10:04:09 -04:00
Antioch Peverell
55a929e214
cleanup old unused code (#3355) 2020-06-15 15:05:47 +01:00
Antioch Peverell
20e5c1910b
NRD rules and "recent" kernel pos index (#3302)
* variants for output_pos linked list entries (head/tail/middle/unique)
next and prev and Vec<u8> lmdb keys

* get_pos on enum

* break list and list entries out into separate enums

* track output features in the new output_pos index, so we can determine coinbase maturity

* push entry impl for none and unique

* some test coverage for output_pos_list

* commit

* wip - FooListEntry

* use instance of the index

* linked list of output_pos and commit_pos both now supported

* linked_list

* cleanup and rename

* rename

* peek_pos

* push some, peek some, pop some

* cleanup

* commit pos
cleanup

* split list and entry out into separate db prefixes

* cleanup and add placeholder for pop_back

* pop_pos_back (for popping off the back of the linked list)
test coverage for pop_pos_back

* wip

* placeholder for prune via a trait
pos must always increase in the index

* rewind kernel_pos_idx when calling rewind_single_block

* RewindableListIndex with rewind support.

* test coverage for rewindable list index

* test coverage for rewind back to 0

* rewind past end of list

* add tests for kernel_pos_idx with multiple commits

* commit

* cleanup

* hook NRD relative lock height validation into block processing and tx validation

* cleanup

* set local chain type for kernel_idx tests

* add test coverage for NRD rules in block processing

* NRD test coverage and cleanup

* NRD relative height 1 test

* test coverage for NRD kernels in block processing

* cleanup

* start of test coverage for txpool NRD kernel rules

* wip

* rework pool tests to use real chain (was mock chain) to better reflect reality (tx/block validation rules etc.)

* cleanup

* cleanup pruneable trait for kernel pos index

* add clear() to kernel_pos idx and test coverage

* hook kernel_pos rebuild into node startup, compaction and fast sync

* verify full NRD history on fast sync

* return early if nrd disabled

* fix header sync issue
2020-06-10 16:38:29 +01:00
Antioch Peverell
b98e5e06a6
fix race condition (#3344) 2020-06-10 13:56:33 +01:00
Antioch Peverell
5b825fbf0d
check_known now takes total_difficulty into consideration (#3298)
* check_known now takes total_difficulty into consideration

* update based on feedback
2020-05-29 10:21:12 +01:00
hashmap
2397407dc4
Always return a typed structure from lmdb store (#3312)
We have a method get which returns a Vec, it's used in cases when we can't implement FromLmdbBytes or Readable on a desired type, eg when both traits and type are external for the current crate. This approach requires an extra allocation of a Vec, this PR introduces a method which accepts desirialzer as closure, which allows to desrialize in place without an intermidiate heap allocation.

It's still possible to get a Vec if needed, just pass a Vec constructor to get_with.
2020-05-05 11:00:23 +02:00
hashmap
8a22fb516a
Reduce number of allocations in to_key calls (#3311)
We have to make an extra allocation per db get request because key generation function to_key takes Vec. Taking byte slice (AsRef<[u8]> to be precise) also simplifes the code a bit.
2020-04-30 17:47:44 +02:00
hashmap
a82041d0ed
Refactor Readable trait (#3309)
Currently Writable accepts trait Write as a type parameter but Readable
takes Read as a trait object, which is not symmetrical and also less performant. This PR changes Readable trait and all places where it's used
2020-04-30 17:42:19 +02:00
Quentin Le Sceller
be4779c923
Less cloning and pattern simplifications (#3305)
* Cleanup 
* Pattern simplification
2020-04-24 11:19:34 +02:00
hashmap
d8c6eef485
Refactor SyncState (#3297)
* Refactor SyncState

Method sync_error() retrun type was simplified.
update_txhashset_download() was  made type safe, which eliminates a runtime enum variant's  check, added an atomic status update
2020-04-20 12:30:04 +02:00
Antioch Peverell
bab9f41343
go back to storing header_head in the db (#3268)
* go back to storing header_head in the db

* simplify and cleanup
2020-04-10 16:12:53 +01:00
Antioch Peverell
e49eecae5c
Cleanup path handling with AsRef<Path> (#3284)
* AsRef<Path>

* bump
2020-03-30 11:35:21 +01:00
Antioch Peverell
3bbf60ee92
simplify when block_sums and spent_index are added to the db (#3253)
* simplify when block_sums and spent_index are added to the db

* fix pool tests

* cleanup
2020-03-10 14:58:32 +00:00
jaspervdm
d5b523248b
API: don't error on missing output (#3256)
* Node API: don't error on missing output

* Propagate errors from get_output*

* Rename is_unspent and small refactor

* Forgot to rename function in tests

* Change Batch get_output_pos_height type signature
2020-03-04 23:42:10 +01:00
Antioch Peverell
5f5b1d2f13
no need to rehash with index to compare output with input spending it (#3260)
* no need to rehash with index to compare output with input spending it

* compare output identifier when checking is_unspent()

* output identifier from cleanup
2020-03-04 08:36:33 +00:00
Antioch Peverell
2527006e8d
keep output and rangeproof mmrs together (#3254) 2020-03-03 15:32:01 +00:00
Antioch Peverell
bcf0e0b4eb
we need to be lenient when deleting entries from the output_pos index (may be missing) (#3251) 2020-02-28 12:41:40 +00:00
Antioch Peverell
5cb0aac55e
only apply full rewind to bitmap, not per block (#3246) 2020-02-26 19:15:13 +00:00
Antioch Peverell
3790b2cba5
cleanup unused import (#3238) 2020-02-24 16:08:16 +01:00
Antioch Peverell
cb2b909090
Block input bitmap rework (#3236)
* first pass at rewind_single_block
and reworking rewind to simply iterate over blocks, rewinding each incrementally

* commit

* commit

* cleanup

* add test coverage for output_pos index transactional semantics during rewind

* commit

* do not store commitments in spent_index
just use the order of the inputs in the block

* compare key with commitment when cleaning output_pos index

* remove unused OutputPos struct
2020-02-24 09:48:51 +00:00
Antioch Peverell
2d4a2c30ce
Cleanup output pos idx (#3227)
* cleanup legacy output pos index
cleanup unused migration code
consolidate use of output_pos index

* refactor to cleanup duplicate code
2020-02-14 20:42:07 +00:00
Antioch Peverell
3c040b96e1
account for the utxo rewind when attempting to fix a bad chain init (#3229) 2020-02-14 15:03:45 +00:00
Antioch Peverell
4c081b8f73
use backend file when compacting the output pos index (#3226) 2020-02-13 10:26:56 +00:00
Quentin Le Sceller
04a0123752
Less cloning and additional pattern simplifications (#3223)
* API Cleanup

* Chain Cleanup

* Core Cleanup

* Keychain Cleanup

* P2P Cleanup

* Pool Cleanup

* Store Cleanup

* Util Cleanup

* Cleanup clone_from_slice

* Address jasper comments
2020-02-12 19:35:33 +01:00
Quentin Le Sceller
c4e69717ab
Less cloning and pattern simplifications (#3216)
* Less cloning and pattern simplifications

* Revert inclusive range and remove unecessary Error:From
2020-02-05 11:02:07 -05:00
Antioch Peverell
616dad43fd
Cleanup redundant AsFixedBytes and FixedLength traits (#3131)
* no need for AsFixedBytes we can just use AsRef<[u8]>

* cleanup FixedLength trait

* revert this change for now

* fix store tests

* cleanup and fix tests after rebase

* fix tests

* update based on PR review
less hard-coded values now

* cleanup
2020-01-29 13:41:50 +00:00
Antioch Peverell
9ec9d04457 Refactor to pass batch around explicitly (#3176)
* Refactor to pass batch around explicitly rather than keeping it in the extension.
We would like to be able to pass an extension around and potentially clone it.

* cleanup, pass header around where it makes sense
2020-01-28 12:23:11 -05:00
cgilliard
c15aedf190 Bug fix for highest_index in output API. (#3200)
https://github.com/mimblewimble/grin/issues/3199
2020-01-21 11:05:36 +01:00
Quentin Le Sceller
6e5afe496b Update License to 2020 (#3196) 2020-01-20 11:40:58 +00:00
Quentin Le Sceller
d31427f694 Fix compiler warning 'unnecessary parentheses around type' (#3175) 2020-01-06 09:53:49 +00:00
Antioch Peverell
ea2092ae21
rework orphan block validation (pow_only) (#3172) 2019-12-17 15:17:45 +00:00
Quentin Le Sceller
cdb2d6c72c Node API v2 (#3094)
* Node API v2

* Update gitignore

* Add get_pmmr_indices method

* Add Foreign and Owner API each one with specific secret

* Fix failing tests

* Revert to 'Option<u64>'
2019-12-06 10:57:53 +00:00
Antioch Peverell
0b21ee607a
height (header version) specific output PMMR root rules (#3147)
* height (header version) specific output PMMR root rules

* cleanup

* cleanup based on PR feedback

* address feedback
2019-12-05 11:55:10 +00:00
Antioch Peverell
3a333ae00f
deadlock risk in check_orphans called from txhashset_write (#3156) 2019-12-05 10:32:01 +00:00
Antioch Peverell
e2795b1593
call zip_read fro within the txhashet lock for a consistent view on the files (#3142) 2019-11-26 21:17:28 +00:00