Commit graph

55 commits

Author SHA1 Message Date
John Tromp
53414ae105
Fixmmr part2 (#3666)
* use 0-based positions in  methods pmmr_leaf_to_insertion_index and bintree_postorder_height; add round_up_to_leaf_pos method

* use 0-based positions in method insertion_to_pmmr_index

* use 0-based positions in method is_leaf

* use 0-based positions in method family()

* use 0-based positions in method is_left_sibling

* use 0-based positions in method family_branch

* use 0-based positions in methods bintree_{left,right}most

* use 0-based positions in method bintree_pos_iter

* use 0-based positions in method bintree_range

* use 0-based positions in method bintree_leaf_pos_iter

* rename last_pos in MMR related structs to size

* use 0-based positions in method prune

* use 0-based positions in method push and apply_output return value

* use 0-based position argument of method merkle_proof

* use 0-based outputs in method pmmr::peaks

* fix peaks() code comments

* refix peaks() code comments

* use 0-based positions in method get_peak_from_file

* use 0-based positions in methods get_data_from_file

* use 0-based positions in methods get_from_file

* use 0-based positions in methods get_data

* use 0-based positions in methods get_hash

* use 0-based positions in method peak_path

* use 0-based positions in method bag_the_rhs

* use 0-based positions in method Backend::remove

* use 0-based positions in method leaf_pos_iter

* use 0-based positions in method self.LeafSet::includes

* use 0-based positions in methods self.LeafSet::{add,remove}

* use 0-based positions in methods is_pruned,is_pruned_root,is_compacted

* use 0-based positions in methods PruneList::append

* use 0-based positions in methods append_pruned_subtree

* use 0-based positions in method calculate_next_leaf_shift

* use 0-based positions in method append_single

* use 0-based positions in method calculate_next_shift

* use 0-based positions in method segment_pos_range

* use 0-based positions in method reconstruct_root

* use 0-based positions in method validate_with

* use 0-based positions in method validate

* rename size (formerly last_pos) to mmr_size

* use 0-based positions in Segment's hash_pos and leaf_pos

* minimize use of saturating_sub(1) and rename some pos/idx to size

* use 0-based positions in methods get_output_pos

* use 0-based positions in method get_unspent_output_at

* use 0-based positions in method get_header_hash

* use 0-based positions in methods MerkleProof::verify{,_consume}

* use 0-based positions in method cleanup_subtree

* don't allow 0 in prunelist bitmap

* use 0-based positions in methods get_{,leaf_}shift

* rename some 1-based pos to pos1; identify TODO

* Address yeastplume's PR review comments
2021-11-26 11:25:10 +00:00
Antioch Peverell
89c06ddab7
client invalidateheader and resetchainhead (#3618)
* wip - "reset_head" via owner api functionality

* jsonrpc pass hash in as a string

* sort of works

* not a reorg if we simply accept several blocks at once

* remember to reset header MMR separately
as it is readonly when interacting with txhashset extension

* basic client integration
needs error handling etc.

* reset sync status when reset chain head

* track "denylist" (todo) and validate headers against this via the ctx

* track denylist (header hashes) in chain itself

* header denylist in play

* expose invalidateheader as client cmd

* rework reset_chain_head - rewind txhashset then header MMR
2021-04-29 11:05:05 +01:00
Antioch Peverell
34413c1cab
track sync_head on header_sync sync status (#3626)
* track sync_head on header_sync sync status
follow header fork as appropriate

* track highest_diff alongside highest_height on HeaderSync status

* cleanup

* not not
2021-04-06 11:16:20 +01:00
Antioch Peverell
f6ec77a592
never speak of the verifier cache again (#3628) 2021-04-01 15:04:53 +01:00
Quentin Le Sceller
0259ed23ea
Update copyright year to 2021 (#3592)
* Update copyright year to 2021
2021-03-10 10:14:48 -05:00
Antioch Peverell
25fcefada2
Flexible peer filtering (#3458)
* first pass at peers iter cleanup

* more flexible peer with diff lookups

* PeersIter with impl Iterator

* sync against outbound peers
reorder peers filters so expensive filters come later

* filter peers by capabilities during sync

* prefer outbound peers with high total difficulty

* with_difficulty now takes a fn to allow more flexible comparisons based on difficulty

* rename peers_iter() -> iter()
2020-10-27 12:36:00 +00:00
Antioch Peverell
133089e985
Refactor Output Identifiers (#3371)
* refactor output to have internal output identifier

refactor to use AsRef for output identifier

make the output MMR explicit in terms of output identifiers

* put the serde deser back for rangeproof

* add json test for transactions
2020-09-03 10:59:54 +01:00
Antioch Peverell
599bf22cfc
Simplify api commits (#3423)
* simplify api with unspent by commitment

* fix chain tests
2020-08-17 13:19:29 -04:00
Antioch Peverell
ec3ea9c3ff
pass ref to chain around (#3401) 2020-07-27 11:05:12 +01:00
Antioch Peverell
20e5c1910b
NRD rules and "recent" kernel pos index (#3302)
* variants for output_pos linked list entries (head/tail/middle/unique)
next and prev and Vec<u8> lmdb keys

* get_pos on enum

* break list and list entries out into separate enums

* track output features in the new output_pos index, so we can determine coinbase maturity

* push entry impl for none and unique

* some test coverage for output_pos_list

* commit

* wip - FooListEntry

* use instance of the index

* linked list of output_pos and commit_pos both now supported

* linked_list

* cleanup and rename

* rename

* peek_pos

* push some, peek some, pop some

* cleanup

* commit pos
cleanup

* split list and entry out into separate db prefixes

* cleanup and add placeholder for pop_back

* pop_pos_back (for popping off the back of the linked list)
test coverage for pop_pos_back

* wip

* placeholder for prune via a trait
pos must always increase in the index

* rewind kernel_pos_idx when calling rewind_single_block

* RewindableListIndex with rewind support.

* test coverage for rewindable list index

* test coverage for rewind back to 0

* rewind past end of list

* add tests for kernel_pos_idx with multiple commits

* commit

* cleanup

* hook NRD relative lock height validation into block processing and tx validation

* cleanup

* set local chain type for kernel_idx tests

* add test coverage for NRD rules in block processing

* NRD test coverage and cleanup

* NRD relative height 1 test

* test coverage for NRD kernels in block processing

* cleanup

* start of test coverage for txpool NRD kernel rules

* wip

* rework pool tests to use real chain (was mock chain) to better reflect reality (tx/block validation rules etc.)

* cleanup

* cleanup pruneable trait for kernel pos index

* add clear() to kernel_pos idx and test coverage

* hook kernel_pos rebuild into node startup, compaction and fast sync

* verify full NRD history on fast sync

* return early if nrd disabled

* fix header sync issue
2020-06-10 16:38:29 +01:00
hashmap
9e51e86538
Use generic types instead of trait objects in tx pool (#3308)
Tx pool takes some parameters as trait objects. It's not an idiomatic Rust code, in this particular case we should use generic types. Trait object makes sense when we accept in runtime different concrete types which implement the trait as a value of the same field. It's not the case here. Trait objects come with a price - instead of method dispatch in compile time we have to accept runtime dispatch. My guess we did it to not clutter the code with type parameters, which is understandable but still suboptimal.
2020-04-30 17:41:49 +02:00
Quentin Le Sceller
be4779c923
Less cloning and pattern simplifications (#3305)
* Cleanup 
* Pattern simplification
2020-04-24 11:19:34 +02:00
hashmap
d8c6eef485
Refactor SyncState (#3297)
* Refactor SyncState

Method sync_error() retrun type was simplified.
update_txhashset_download() was  made type safe, which eliminates a runtime enum variant's  check, added an atomic status update
2020-04-20 12:30:04 +02:00
jaspervdm
6bdf31f23d
Remove unused code in message handling (#3269) 2020-03-13 12:50:35 +01:00
Antioch Peverell
8ca381a9c2
cleanup util::from_hex() (#3265)
* cleanup our from_hex()

* fix keychain tests

* add test coverage for empty hex string
2020-03-10 10:36:18 +00:00
jaspervdm
d5b523248b
API: don't error on missing output (#3256)
* Node API: don't error on missing output

* Propagate errors from get_output*

* Rename is_unspent and small refactor

* Forgot to rename function in tests

* Change Batch get_output_pos_height type signature
2020-03-04 23:42:10 +01:00
Antioch Peverell
cb2b909090
Block input bitmap rework (#3236)
* first pass at rewind_single_block
and reworking rewind to simply iterate over blocks, rewinding each incrementally

* commit

* commit

* cleanup

* add test coverage for output_pos index transactional semantics during rewind

* commit

* do not store commitments in spent_index
just use the order of the inputs in the block

* compare key with commitment when cleaning output_pos index

* remove unused OutputPos struct
2020-02-24 09:48:51 +00:00
jaspervdm
6bca34c6a8
Update hyper/tokio/futures dependencies (#3214)
* Update hyper, tokio, futures versions

* Update stratum server

* Update API

* Update webhooks
2020-02-18 23:45:27 +01:00
Quentin Le Sceller
04a0123752
Less cloning and additional pattern simplifications (#3223)
* API Cleanup

* Chain Cleanup

* Core Cleanup

* Keychain Cleanup

* P2P Cleanup

* Pool Cleanup

* Store Cleanup

* Util Cleanup

* Cleanup clone_from_slice

* Address jasper comments
2020-02-12 19:35:33 +01:00
François Garillot
dcdbdd4bcc
Some simple Option / Result / iterator pattern simplifications (#3205) 2020-01-29 14:20:57 +00:00
hashmap
4449c91c04 Fix return code in peers API (#3186)
Rust compiler found this issue, we were preparing a response and the dropping it and returning a default one
2020-01-21 11:30:29 -05:00
Quentin Le Sceller
6e5afe496b Update License to 2020 (#3196) 2020-01-20 11:40:58 +00:00
Quentin Le Sceller
cdb2d6c72c Node API v2 (#3094)
* Node API v2

* Update gitignore

* Add get_pmmr_indices method

* Add Foreign and Owner API each one with specific secret

* Fix failing tests

* Revert to 'Option<u64>'
2019-12-06 10:57:53 +00:00
Antioch Peverell
0b21ee607a
height (header version) specific output PMMR root rules (#3147)
* height (header version) specific output PMMR root rules

* cleanup

* cleanup based on PR feedback

* address feedback
2019-12-05 11:55:10 +00:00
Joseph Goulden
6d864a813c fix: split state validation status into kernel and rproof updates. (#3096)
* fix: split state validation status into kernel and rproof updates. And fix sync status for these two states

* fix: show correct number of leaves for pruned MMR as well as unpruned

* docs: better docs for kernel/range proof validation

* fix: ordering of kernel and rproofs validation in TUI

* fix: typo in rangeproofs api and comments
2019-11-17 22:12:10 +01:00
Yeastplume
38e6497919
Retrieve outputs within a block height range (#3103)
* add function to retrieve a set of pmmr indices between a given block height range

* typo

* change APU to just return required indices

* change pmmr index retrieval, change new function to only return pmmr indices between blocks
2019-11-04 15:04:21 +00:00
Quentin Le Sceller
eefd87aa2e
More robust peer banning (#3086)
More robust peer banning
2019-10-04 18:00:49 -04:00
Quentin Le Sceller
6be6391225 Change 2018 to 2019 in copyright files (#3072) 2019-10-02 09:40:20 +01:00
Antioch Peverell
bc6108cf12
Kernels v2 (variable size) (#3034)
* wip

* exhaustive match

* write with fixed v1 strategy when writing for hashing

* local protocol version is 2

* cleanup "size" tests that exercise v1 vs v2 vs default protocol versions

* add proto version to Connected! log msg

* cleanup docs

* negotiate protocol version min(local, peer) when doing hand/shake
2019-09-19 14:31:46 +01:00
Quentin Le Sceller
b209244d17
Add sync_status in status Node API (#2966)
* Add sync_status in status Node API
2019-09-17 11:12:42 -04:00
jaspervdm
0e62b7e43e Optionally add rangeproofs to block api (#2999) 2019-09-03 15:52:33 +01:00
jaspervdm
30156cdcf9 Kernel lookup api method (#3000)
* Kernel lookup api

* Min and max height parameters, scan backwards

* Return null instead of 404 if kernel not found

* Return TxKernel instead of TxKernelPrintable

* Update description of KernelHandler
2019-08-30 10:03:12 +01:00
Gary Yu
d36a0b29ef store both mmr index and block height into database for output (#2903)
* store both mmr index and block height into database for output

* rustfmt

* fix: mmr position is 1-based instead of 0-based

* (Hash, u64, u64) deserves a type
2019-08-29 17:14:39 +01:00
Quentin Le Sceller
121e6c8c46
[DOC] Translate documents & fix some typo (#2952)
*  add files for translation

*  WIP translation

* finish to translate coinbase_maturity.md && fix some typo to understand easily

*  add doc link in coinbase_maturity.md

*  remove wrong file

* Fix typo and unecessary word
2019-07-18 11:42:34 +02:00
Antioch Peverell
d284d8f6de
[2.x.x] Writeable protocol version aware (#2856)
* introduce protocol version to deserialize and read

* thread protocol version through our reader

* cleanup

* cleanup

* streaming_reader cleanup

* Pass protocol version into BinWriter to allow for version specific serialization rules.

* rustfmt

* read and write now protocol version specific
2019-07-06 15:51:03 +01:00
Antioch Peverell
f4eb3e3d4b
Always stem local txs if configured that way (unless explicitly fluffed) (#2876)
* always stem local txs if configured that way (unless explicitly fluff from wallet)
this overrides current epoch behavior for txs coming in via "push-api"
rename "local" to "our" txs

* TxSource is now an enum for type safety.
2019-07-04 11:56:42 +01:00
Antioch Peverell
5aaf2d058d
[2.x.x] deserialization now protocol version aware (#2824)
* introduce protocol version to deserialize and read

* thread protocol version through our reader

* example protocol version access in kernel read

* fix our StreamingReader impl (WouldBlock woes)

* debug log progress of txhashset download
2019-06-27 17:19:41 +01:00
jaspervdm
5d6defb2eb Upate version api call (#2899) 2019-06-15 19:24:12 +01:00
Yeastplume
c2153fada2
Add version endpoint to node API, rename pool/push (#2897)
* add node version API, tweak pool/push parameter

* rustfmt
2019-06-15 09:54:45 +01:00
Aleksandrov Vladimir
2cb37913ba add flag 'no_merkle_proof' to /v1/blocks api to get blocks without checking merkle proof (#2843) 2019-05-31 08:49:40 +02:00
Antioch Peverell
e56cd55980
initial support for kernel data download (#2765)
* initial support for kernel data download

* fix vec backend for tests

* cleanup after rebase
2019-05-14 17:17:38 +01:00
Mark Renten
c3cd98cae7 Add some better logging for get_outputs_by_id failure states (#2705) 2019-03-25 15:17:47 +01:00
hashmap
7fad5b040f
Reduce number of unwwaps in api crate (#2681)
* Reduce number of unwwaps in api crate

* Format use section
2019-03-18 19:34:35 +01:00
Gary Yu
608973c498 Fix the build warning of deprecated trim_right_matches (#2662) 2019-03-10 20:13:42 -07:00
Antioch Peverell
23cb9e2514
Map peers by ip only (ignoring port unless on loopback ip) (#2540)
* wip

* big refactor, regretting doing this now

* PeerAddr everywhere

* cleanup

* fixup server tests

* peers api working for GET, POST is still WIP

* we can now ban/unban peers by ip only (port optional)
2019-02-18 12:15:32 +00:00
hashmap
ac6ed71abd
Refactor API handlers (#2572)
Also add some API tests
2019-02-15 22:17:00 +01:00
Mark Renten
3b74a9ba3f Add content disposition for OK responses (#2545) 2019-02-11 10:54:21 -08:00
Simon B
dd1a24dcbc Compiler warnings 20190123 (#2462)
* cleanup compiler warnings: unused imports

* rustfmt

* more compiler warnings
2019-01-25 10:09:32 +00:00
Simon B
87a4e1e2d7 clippy says: unneeded return https://rust-lang.github.io/rust-clippy/master/index.html#needless_return 2019-01-24 01:56:50 +01:00
Simon B
8935f3c7b4 clippy says: use char (',') instead of string (",")
https://rust-lang.github.io/rust-clippy/master/index.html#single_char_pattern
2019-01-24 01:53:35 +01:00