* change pibd stat display to show progress as a percentage of downloaded leaves
* attempt some inline rp validation
* propagate shutdown state through kernel validation
* change validation loop timing
* simplify validator threading
* add more detailed tracking of kernel history validation to tui, allow stop state during
* adding more stop state + tui progress indication
* remove progressive validate
* test fix
* start to add stats and reset chain state after errors detected
* add functions to reset prune list when resetting chain pibd state
* debug statement
* remove test function
* investigations as to why a slight rewind is needed on startup during PIBD
* move validation code into desegmenter validation thread (for now)
* ensure genesis entries in pmmrs are removed if they're removed in the first segment
* validation all working except for verifying kernel sums
* remove unneeded pmmr rollbacks on resume now root cause was found
* updates to remove unpruned leaves from leaf set when rebuilding pmmr
* remove + 1 to segment traversal iter length
* add functions to determing latest verifiable block height for the given pibd state
* attempting to allow for pibd to resume after killing process
* fix to ensure prune list is properly flushed during pibd sync
* removal of unneeded code
* ignore test for now (fix before full merge)
* update pibd copy test to use new desgmenter structure
* begin reconstruction of output pmmr
* clean up hash/leaf insertion logic
* push pruned subtree appears to be working, now also calculates left hand hashes correctly
* factor out ordering of segment/hash order array
* refactor for pmmr application code
* test of chain copy appears to be working
* add rangeproof functions to desegmenter
* add kernel functions, attempt refactor
* small test cleanup, reconstruction of live chain working in manual copy test
* application of received bitmap segments to local accumulator
* add all required elements to send/receive output segment requests and responses
* testing of output sync
* add special cases to pmmr segment request
* add functions to desegmenter to report next desired segments, begin to add state to determine which segments have been requested
* add segmentidentifier type to id requested segments uniquely
* make a call on where to keep track of which PIBD segments have been requested
* move segmenttype definition, add functions to manipulate peer segment list
* remove desegmenter state enum
* change chain desegmenter function to provide rwlock
* trace, warning cleanup
* udpate to test compliation
* experimental addition of pibd download state for testnet only
* fixes to bitmap number of segments calculation + conversion of bitmap accumulator to bitmap
* attempt to call a test message
* add p2p methods for receiving bitmap segment and applying to desegmenter associated with chain
* fixes to state sync
* wip - "reset_head" via owner api functionality
* jsonrpc pass hash in as a string
* sort of works
* not a reorg if we simply accept several blocks at once
* remember to reset header MMR separately
as it is readonly when interacting with txhashset extension
* basic client integration
needs error handling etc.
* reset sync status when reset chain head
* track "denylist" (todo) and validate headers against this via the ctx
* track denylist (header hashes) in chain itself
* header denylist in play
* expose invalidateheader as client cmd
* rework reset_chain_head - rewind txhashset then header MMR
* track sync_head on header_sync sync status
follow header fork as appropriate
* track highest_diff alongside highest_height on HeaderSync status
* cleanup
* not not
* wip - body sync for full archive
* allow chain compaction during sync
* placeholder for logic to ensure archive nodes sync from archive nodes
* body sync from archival peers
* allow chain compaction during sync
* placeholder for logic to ensure archive nodes sync from archive nodes
* Introduce CommitOnly variant of Inputs.
Introduce CommitWrapper so we can sort commit only inputs correctly.
* rememebr to resort if converting
* write inputs based on variant and protocol version
* read and write protocol version specific inputs
* store full blocks in local db in v3
convert to v2 when relaying to v2 peers
* add debug version_str for inputs
* no assumptions about spent index sort order
* add additional version debug logs
* fix ser/deser tests for proto v3
* cleanup coinbase maturity
* rework pool to better handle v2 conversion robustly
* cleanup txpool add_to_pool
* fix nrd kernel test
* move init conversion earlier
* cleanup
* cleanup based on PR feedback
Tx pool takes some parameters as trait objects. It's not an idiomatic Rust code, in this particular case we should use generic types. Trait object makes sense when we accept in runtime different concrete types which implement the trait as a value of the same field. It's not the case here. Trait objects come with a price - instead of method dispatch in compile time we have to accept runtime dispatch. My guess we did it to not clutter the code with type parameters, which is understandable but still suboptimal.
* Refactor SyncState
Method sync_error() retrun type was simplified.
update_txhashset_download() was made type safe, which eliminates a runtime enum variant's check, added an atomic status update
* wip
* sync sort of works now
* get rid of the deadlock during compaction
there is *always* a deadlock in there when we make changes like this...
* cleanup how we rebuild the sync MMR on init
* cleanup rewind logic
* roll the "fix invalid root" changes into this PR
* move rebuild_height_pos_index into txhashset
and pass in header_pmmr for header lookups
* cleanup and remember to setup sync head on init
* cleanup unnecessary ref muts
* rebuild_height_pos_index when writing txhashset
* Ban on cannot get block header in txhashset_write
* Rusfmt
* Fix typo
* Missing error handling
* Rustfmt
* Only accept txhashset from corresponding peer
* Switch to AtomicBool instead of RwLock<bool>
* Rustfmt
* always stem local txs if configured that way (unless explicitly fluff from wallet)
this overrides current epoch behavior for txs coming in via "push-api"
rename "local" to "our" txs
* TxSource is now an enum for type safety.
* Calculate reorg depth in BlockStatus::Reorg enum member
* rustfmt
* Fix reorg height calculation and implement reorg test
* rustfmt
* Report reorg depth in webhook payload
* Add optional depth field to the block webhook JSON reply
* use a grin specific tmp dir instead of a system tmp dir
* rustfmt
* using create_new instead of create
* switch the txhashset sandbox from system tmp dir to grin specific tmp dir
* rustfmt
* adapter get_tmp_dir delegate to chain implementation
* simply the logic of protolc for get_tmpfile_pathname
* rustfmt
* fix the merge mistake
Most of the methods return nothing or bool which is used to decide if a
sender of a message should be banned or not. However underlying chain
implementation may fail so we need a way to reflect this fact in API.
Also it allows to reduce number of unwraps and makes the code more robust.
* cleanup legacy "3 dot" check (#2625)
* Allow to peers behind NAT to get up to preferred_max connections (#2543)
Allow to peers behind NAT to get up to preffered_max connections
If peer has only outbound connections it's mot likely behind NAT and we should not stop it from getting more outbound connections
* Reduce usage of unwrap in p2p crate (#2627)
Also change store crate a bit
* Simplify (and fix) output_pos cleanup during chain compaction (#2609)
* expose leaf pos iterator
use it for various things in txhashset when iterating over outputs
* fix
* cleanup
* rebuild output_pos index (and clear it out first) when compacting the chain
* fixup tests
* refactor to match on (output, proof) tuple
* add comments to compact() to explain what is going on.
* get rid of some boxing around the leaf_set iterator
* cleanup
* [docs] Add switch commitment documentation (#2526)
* remove references to no-longer existing switch commitment hash
(as switch commitments were removed in ca8447f3bd
and moved into the blinding factor of the Pedersen Commitment)
* some rewording (points vs curves) and fix of small formatting issues
* Add switch commitment documentation
* [docs] Documents in grin repo had translated in Korean. (#2604)
* Start to M/W intro translate in Korean
* translate in Korean
* add korean translation on intro
* table_of_content.md translate in Korean.
* table_of_content_KR.md finish translate in Korean, start to translate State_KR.md
* add state_KR.md & commit some translation in State_KR.md
* WIP stat_KR.md translation
* add build_KR.md && stratum_KR.md
* finish translate stratum_KR.md & table_of_content_KR.md
* rename intro.KR.md to intro_KR.md
* add intro_KR.md file path each language's intro.md
* add Korean translation file path to stratum.md & table_of_contents.md
* fix difference with grin/master
* Fix TxHashSet file filter for Windows. (#2641)
* Fix TxHashSet file filter for Windows.
* rustfmt
* Updating regexp
* Adding in test case
* Display the current download rate rather than the average when syncing the chain (#2633)
* When syncing the chain, calculate the displayed download speed using the current rate from the most recent iteration, rather than the average download speed from the entire syncing process.
* Replace the explicitly ignored variables in the pattern with an implicit ignore
* remove root = true from editorconfig (#2655)
* Add Medium post to intro (#2654)
Spoke to @yeastplume who agreed it makes sense to add the "Grin Transactions Explained, Step-by-Step" Medium post to intro.md
Open for suggestions on a better location.
* add a new configure item for log_max_files (#2601)
* add a new configure item for log_max_files
* rustfmt
* use a constant instead of multiple 32
* rustfmt
* Fix the build warning of deprecated trim_right_matches (#2662)
* [DOC] state.md, build.md and chain directory documents translate in Korean. (#2649)
* add md files for translation.
* start to translation fast-sync, code_structure. add file build_KR.md, states_KR.md
* add dandelion_KR.md && simulation_KR.md for Korean translation.
* add md files for translation.
* start to translation fast-sync, code_structure. add file build_KR.md, states_KR.md
* add dandelion_KR.md && simulation_KR.md for Korean translation.
* remove some useless md files for translation. this is rearrange set up translation order.
* add dot end of sentence & translate build.md in korean
* remove fast-sync_KR.md
* finish build_KR.md translation
* finish build_KR.md translation
* finish translation state_KR.md & add phrase in state.md to move other language md file
* translate blocks_and_headers.md && chain_sync.md in Korean
* add . in chain_sync.md , translation finished in doc/chain dir.
* fix some miss typos
* Api documentation fixes (#2646)
* Fix the API documentation for Chain Validate (v1/chain/validate). It was documented as a POST, but it is actually a GET request, which can be seen in its handler ChainValidationHandler
* Update the API V1 route list response to include the headers and merkleproof routes. Also clarify that for the chain/outputs route you must specify either byids or byheight to select outputs.
* refactor(ci): reorganize CI related code (#2658)
Break-down the CI related code into smaller more maintainable pieces.
* Specify grin or nanogrins in API docs where applicable (#2642)
* Set Content-Type in API client (#2680)
* Reduce number of unwraps in chain crate (#2679)
* fix: the restart of state sync doesn't work sometimes (#2687)
* let check_txhashset_needed return true on abnormal case (#2684)
* Reduce number of unwwaps in api crate (#2681)
* Reduce number of unwwaps in api crate
* Format use section
* Small QoL improvements for wallet developers (#2651)
* Small changes for wallet devs
* Move create_nonce into Keychain trait
* Replace match by map_err
* Add flag to Slate to skip fee check
* Fix secp dependency
* Remove check_fee flag in Slate
* Add Japanese edition of build.md (#2697)
* catch the panic to avoid peer thread quit early (#2686)
* catch the panic to avoid peer thread quit before taking the chance to ban
* move catch wrapper logic down into the util crate
* log the panic info
* keep txhashset.rs untouched
* remove a warning
* [DOC] dandelion.md, simulation.md ,fast-sync.md and pruning.md documents translate in Korean. (#2678)
* Show response code in API client error message (#2683)
It's hard to investigate what happens when an API client error is
printed out
* Add some better logging for get_outputs_by_id failure states (#2705)
* Switch commitment doc fixes (#2645)
Fix some typos and remove the use of parentheses in a
couple of places to make the reading flow a bit better.
* docs: update/add new README.md badges (#2708)
Replace existing badges with SVG counterparts and add a bunch of new ones.
* Update intro.md (#2702)
Add mention of censoring attack prevented by range proofs
* use sandbox folder for txhashset validation on state sync (#2685)
* use sandbox folder for txhashset validation on state sync
* rustfmt
* use temp directory as the sandbox instead actual db_root txhashset dir
* rustfmt
* move txhashset overwrite to the end of full validation
* fix travis-ci test
* rustfmt
* fix: hashset have 2 folders including txhashset and header
* rustfmt
*
(1)switch to rebuild_header_mmr instead of copy the sandbox header mmr
(2)lock txhashset when overwriting and opening and rebuild
* minor improve on sandbox_dir
* add Japanese edition of state.md (#2703)
* Attempt to fix broken TUI locale (#2713)
Can confirm that on the same machine 1.0.2 TUI looks great and is broken on
the current master. Bump of `cursive` version fixed it for me.
Fixes#2676
* clean the header folder in sandbox (#2716)
* forgot to clean the header folder in sandbox in #2685
* Reduce number of unwraps in servers crate (#2707)
It doesn't include stratum server which is sufficiently changed in 1.1
branch and adapters, which is big enough for a separate PR.
* rustfmt
* change version to beta
* use sandbox folder for txhashset validation on state sync
* rustfmt
* use temp directory as the sandbox instead actual db_root txhashset dir
* rustfmt
* move txhashset overwrite to the end of full validation
* fix travis-ci test
* rustfmt
* fix: hashset have 2 folders including txhashset and header
* rustfmt
*
(1)switch to rebuild_header_mmr instead of copy the sandbox header mmr
(2)lock txhashset when overwriting and opening and rebuild
* minor improve on sandbox_dir
