Commit graph

114 commits

Author SHA1 Message Date
Quentin Le Sceller
6be6391225 Change 2018 to 2019 in copyright files () 2019-10-02 09:40:20 +01:00
Antioch Peverell
973a03c826
support for v2 txhashset (but only using v1 in mainnent and floonet) () 2019-09-24 09:57:34 +01:00
Gary Yu
7a07541c26
Clean the unused build_index () 2019-09-10 07:55:24 +08:00
Antioch Peverell
383985292c Split header MMR (and sync MMR) out from txhashset ()
* wip

* sync sort of works now

* get rid of the deadlock during compaction
there is *always* a deadlock in there when we make changes like this...

* cleanup how we rebuild the sync MMR on init

* cleanup rewind logic

* roll the "fix invalid root" changes into this PR

* move rebuild_height_pos_index into txhashset
and pass in header_pmmr for header lookups

* cleanup and remember to setup sync head on init

* cleanup unnecessary ref muts

* rebuild_height_pos_index when writing txhashset
2019-09-07 07:28:26 +08:00
Gary Yu
d55ebe8a2b fix: the race condition in compact () 2019-09-04 19:51:28 +01:00
Gary Yu
928279a676 schnorr signature batch verification ()
* schnorr signature batch verification

* Split the fee and lock_height verification out from the batch signature verification

* Remove the new added fee_height_verify to comply with 

* fix: the last n could not be leaf?
2019-09-03 15:44:10 +01:00
jaspervdm
30156cdcf9 Kernel lookup api method ()
* Kernel lookup api

* Min and max height parameters, scan backwards

* Return null instead of 404 if kernel not found

* Return TxKernel instead of TxKernelPrintable

* Update description of KernelHandler
2019-08-30 10:03:12 +01:00
Gary Yu
d36a0b29ef store both mmr index and block height into database for output ()
* store both mmr index and block height into database for output

* rustfmt

* fix: mmr position is 1-based instead of 0-based

* (Hash, u64, u64) deserves a type
2019-08-29 17:14:39 +01:00
Antioch Peverell
357bc11746
Fix invalid root ()
* use the correct (header) head for the header extension

* add test coverage for header first fork scenario
2019-08-29 10:15:41 +01:00
j01tz
d220410571 Improve error handling when computing PMMR roots ()
* Improve error handling for invalid PMMR roots

* Update tests that rely on pmmr root

* Fix pmmr store tests
2019-08-09 15:10:54 +01:00
Antioch Peverell
515fa54614
Rewind head and header_head consistently. ()
* maintain header_head as distinctly separate from head

* cleanup corrupted storage log msg

* simplify process_header and check_header_known

* remember to commit the batch when successfully processing a header...

* rework sync_block_headers for consistency with process_block_header

* cleanup unrelated code

* fix pool tests

* cleanup chain tests

* cleanup chain tests (reuse helpers more)

* cleanup - head not header on an extension
shortcircuit "rewind and apply fork" for headers if next header
2019-07-26 08:36:24 +01:00
Quentin Le Sceller
4966dc04f7
[2.x.x] Generate txhashset archives on 720 block intervals. ()
Generate txhashset archives on 720 block intervals.
2019-07-23 10:46:29 +02:00
Antioch Peverell
1395074a25
[2.x.x] simplify txhashset zip creation and extraction ()
* simplify how we extract files from the txhashset zip archive

* bump version of zip-rs
cleanup create_zip (was compress)
use explicit list of files when creating zip archive

* rework zip tests for our simplified impl

* periodic debug during txhashset download

* rework list_files to return something usable
2019-07-12 10:50:27 +01:00
eupn
249fe03d5c Remove needless lifetimes ()
* Remove lifetimes that could be elided

* rustfmt
2019-05-25 17:13:36 +01:00
eupn
a2a350dac7 txhashset: remove unnecessary mutability from self references in getters ()
* txhashset.rs: make self references immutable in getters

* rustfmt
2019-05-25 17:12:32 +01:00
Antioch Peverell
e56cd55980
initial support for kernel data download ()
* initial support for kernel data download

* fix vec backend for tests

* cleanup after rebase
2019-05-14 17:17:38 +01:00
Gary Yu
b61b773cee Get back the lost code of () 2019-04-22 16:53:24 -07:00
Antioch Peverell
3d817f6cd0 [1.1.0] Support variable size PMMR (optional size_file) ()
* Introduce optional size_file.
Support fixed size data file via an optional elmt_size.
Support variable size data file via optional size_file.

* remember to release the size_file

* fix scoping for windows support
2019-04-12 15:41:21 -07:00
Yeastplume
5cb8025ddd
[1.1.0] Merge master into 1.1.0 ()
* cleanup legacy "3 dot" check ()

* Allow to peers behind NAT to get up to preferred_max connections ()

Allow to peers behind NAT to get up to preffered_max connections

If peer has only outbound connections it's mot likely behind NAT and we should not stop it from getting more outbound connections

* Reduce usage of unwrap in p2p crate ()

Also change store crate a bit

* Simplify (and fix) output_pos cleanup during chain compaction ()

* expose leaf pos iterator
use it for various things in txhashset when iterating over outputs

* fix

* cleanup

* rebuild output_pos index (and clear it out first) when compacting the chain

* fixup tests

* refactor to match on (output, proof) tuple

* add comments to compact() to explain what is going on.

* get rid of some boxing around the leaf_set iterator

* cleanup

* [docs] Add switch commitment documentation ()

* remove references to no-longer existing switch commitment hash

(as switch commitments were removed in ca8447f3bd
and moved into the blinding factor of the Pedersen Commitment)

* some rewording (points vs curves) and fix of small formatting issues

* Add switch commitment documentation

* [docs] Documents in grin repo had translated in Korean.  ()

*  Start to M/W intro translate in Korean
*  translate in Korean
*  add korean translation  on intro
* table_of_content.md translate in Korean.
*  table_of_content_KR.md finish translate in Korean, start to translate State_KR.md
*  add state_KR.md & commit some translation in State_KR.md
*  WIP stat_KR.md translation
*  add build_KR.md && stratum_KR.md
*  finish translate stratum_KR.md & table_of_content_KR.md
*  rename intro.KR.md to intro_KR.md
*  add intro_KR.md file path each language's  intro.md
*  add Korean translation file path to stratum.md & table_of_contents.md
*  fix difference with grin/master

* Fix TxHashSet file filter for Windows. ()

* Fix TxHashSet file filter for Windows.

* rustfmt

* Updating regexp

* Adding in test case

* Display the current download rate rather than the average when syncing the chain ()

* When syncing the chain, calculate the displayed download speed using the current rate from the most recent iteration, rather than the average download speed from the entire syncing process.

* Replace the explicitly ignored variables in the pattern with an implicit ignore

* remove root = true from editorconfig ()

* Add Medium post to intro ()

Spoke to @yeastplume who agreed it makes sense to add the "Grin Transactions Explained, Step-by-Step" Medium post to intro.md

Open for suggestions on a better location.

* add a new configure item for log_max_files ()

* add a new configure item for log_max_files

* rustfmt

* use a constant instead of multiple 32

* rustfmt

* Fix the build warning of deprecated trim_right_matches ()

* [DOC] state.md, build.md and chain directory documents translate in Korean.  ()

*  add md files for translation.

*  start to translation fast-sync, code_structure. add file build_KR.md, states_KR.md

* add dandelion_KR.md && simulation_KR.md for Korean translation.

*  add md files for translation.

*  start to translation fast-sync, code_structure. add file build_KR.md, states_KR.md

* add dandelion_KR.md && simulation_KR.md for Korean translation.

* remove some useless md files for translation. this is rearrange set up translation order.

*  add dot end of sentence & translate build.md in korean

*  remove fast-sync_KR.md

*  finish build_KR.md translation

*  finish build_KR.md translation

*  finish translation state_KR.md & add phrase in state.md to move other language md file

* translate blocks_and_headers.md && chain_sync.md in Korean

*  add . in chain_sync.md , translation finished in doc/chain dir.

* fix some miss typos

* Api documentation fixes ()

* Fix the API documentation for Chain Validate (v1/chain/validate).  It was documented as a POST, but it is actually a GET request, which can be seen in its handler ChainValidationHandler
* Update the API V1 route list response to include the headers and merkleproof routes.  Also clarify that for the chain/outputs route you must specify either byids or byheight to select outputs.

* refactor(ci): reorganize CI related code ()

Break-down the CI related code into smaller more maintainable pieces.

* Specify grin or nanogrins in API docs where applicable ()

* Set Content-Type in API client ()

* Reduce number of unwraps in chain crate ()

* fix: the restart of state sync doesn't work sometimes ()

* let check_txhashset_needed return true on abnormal case ()

*  Reduce number of unwwaps in api crate  ()

* Reduce number of unwwaps in api crate

* Format use section

* Small QoL improvements for wallet developers ()

* Small changes for wallet devs

* Move create_nonce into Keychain trait

* Replace match by map_err

* Add flag to Slate to skip fee check

* Fix secp dependency

* Remove check_fee flag in Slate

* Add Japanese edition of build.md ()

* catch the panic to avoid peer thread quit early ()

* catch the panic to avoid peer thread quit before taking the chance to ban
* move catch wrapper logic down into the util crate
* log the panic info
* keep txhashset.rs untouched
* remove a warning

* [DOC] dandelion.md, simulation.md ,fast-sync.md and pruning.md documents translate in Korean. ()

* Show response code in API client error message ()

It's hard to investigate what happens when an API client error is
printed out

* Add some better logging for get_outputs_by_id failure states ()

* Switch commitment doc fixes ()

Fix some typos and remove the use of parentheses in a
couple of places to make the reading flow a bit better.

* docs: update/add new README.md badges ()

Replace existing badges with SVG counterparts and add a bunch of new ones.

* Update intro.md ()

Add mention of censoring attack prevented by range proofs

* use sandbox folder for txhashset validation on state sync ()

* use sandbox folder for txhashset validation on state sync

* rustfmt

* use temp directory as the sandbox instead actual db_root txhashset dir

* rustfmt

* move txhashset overwrite to the end of full validation

* fix travis-ci test

* rustfmt

* fix: hashset have 2 folders including txhashset and header

* rustfmt

* 
(1)switch to rebuild_header_mmr instead of copy the sandbox header mmr 
(2)lock txhashset when overwriting and opening and rebuild

* minor improve on sandbox_dir

* add Japanese edition of state.md ()

* Attempt to fix broken TUI locale ()

Can confirm that on the same machine 1.0.2 TUI looks great and is broken on
the current master. Bump of `cursive` version fixed it for me.
Fixes 

* clean the header folder in sandbox ()

* forgot to clean the header folder in sandbox in 

* Reduce number of unwraps in servers crate ()

It doesn't include stratum server which is sufficiently changed in 1.1
branch and adapters, which is big enough for a separate PR.

* rustfmt

* change version to beta
2019-04-01 11:47:48 +01:00
Yeastplume
6fa137a4ff
Fix TxHashSet file filter for Windows. () ()
* Fix TxHashSet file filter for Windows.

* rustfmt

* Updating regexp

* Adding in test case
2019-02-28 21:39:38 +00:00
yeastplume
91c91469a3
merge from master 2019-02-25 12:15:12 +00:00
hashmap
391e311f4c Don't extract unexpected files from txhashset archive ()
We verify and remove such files later one, it's safer to ignore them
during unpacking
2019-02-24 22:57:56 -08:00
hashmap
e71eca1977 Remove some unwrap/expect in chain crate ()
* Return Result instead of calling expect in root(). It would kill peer's thread. Perhaps we should ban this peer as malicious.
* Remove some unwraps
2019-02-24 21:57:21 -08:00
Antioch Peverell
2df633b622
log timing for rebuild_index () 2019-02-20 08:16:13 +00:00
Antioch Peverell
5dc01b3de6
debug how long it takes to verify the kernel sums for fast sync () 2019-02-19 13:22:26 +00:00
Gary Yu
2f5fbb3ce8 clean-up the txhashset zip file and temp folder ()
* clean-up the txhashset zip file and temp folder
* adapt test_txhashset with the new zip_read function
* fix: the wrong folder when cleaning up old zips
2019-02-15 10:36:26 -08:00
Gary Yu
271042c3b0 suppress the debug log for unexpected file report ()
* supress the debug log for unexpected file report in check_and_remove_files

* revise according to review comments

* rustfmt
2019-02-15 14:33:10 +00:00
Jeremy Rubin
99494c6fa6 get rid of default implementation of Hashed trait for writable things () 2019-02-15 13:41:19 +00:00
Yoni Svechinsky
ee4eed71ea Feature/slate serialization ()
* - Add backwards compatability
- Add hex serialization

* rustfmt

* rustfmt

* Windows Compatibility Fixes  ()

* initial changes for windows build and unit/integration tests

* rustfmt

* wallet+store tests

* rustfmt

* fix linux daemonize

* better encapsulate file rename

* rustfmt

* remove daemonize commands

* rustfmt

* remove server start/stop commands

* add ability to drop pmmr backend files explicitly for txhashset unzip

* rustfmt

* fix pmmr tests

* rustfmt

* Windows TUI Fix ()

* switch pancurses backend to win32

* revert changes to restore test

* compatibility fix + debug messages

* rustfmt

* Add content disposition for OK responses  ()

* Testing http send and fixing accordingly

* add repost method into wallet owner api ()

* add repost method into wallet owner api

* rustfmt

* Add ability to compare selection strategies ()

Before tx creation user can estimate fee and locked amount
with different selection strategies by providing `-e` flag for
`wallet send` command.
2019-02-13 13:29:44 +00:00
Yeastplume
15c7a270eb
Windows Compatibility Fixes ()
* initial changes for windows build and unit/integration tests

* rustfmt

* wallet+store tests

* rustfmt

* fix linux daemonize

* better encapsulate file rename

* rustfmt

* remove daemonize commands

* rustfmt

* remove server start/stop commands

* add ability to drop pmmr backend files explicitly for txhashset unzip

* rustfmt

* fix pmmr tests

* rustfmt
2019-02-09 21:14:27 +00:00
Ignotus Peverell
e3a1c2c6b1
Fix/serializing hash ()
* Remvoe 3 dots from hash formatting
* Move 3 dots to the TUI
* Add backwards check
* Add triple dot to check_and_remove_files + typo
2019-01-12 13:07:03 -08:00
Yoni
51ad711063 Add triple dot to check_and_remove_files + typo 2019-01-12 21:11:03 +02:00
Yeastplume
e93b380a06
Recover outputs from multiple wallets using same seed ()
* adding optional mmr index to key ids

* rustfmt

* update index

* add mmr index to output display

* change restore to match on commit instead of ID, add extensive restore/check tests for multiple wallets using same seed

* rustfmt

* ensure check restores unknown accounts as well

* rustfmt

* remove storage of commit from wallet
2019-01-12 18:39:29 +00:00
John Tromp
f43df5f601 [Floonet] add feature for height locked kernels ()
* add feature for height locked kernels

* add function to compute kernel features appropriate for lock height, and use it

* only sign kernel-features relevant fields; refactor Features

* simplify invalid kernel logic

* remove unused height arg to reward::output and run some rustfmt

* replace nested if/else by match
2018-12-18 18:26:34 +00:00
Antioch Peverell
c188b60a38
readonly verify_coinbase_maturity ()
* move verify_coinbase_maturity into utxo_view
no longer need a write lock on txhashset

* rustfmt
2018-12-16 09:26:39 +00:00
Antioch Peverell
793e3843f0
HeaderEntry for storing subset of header info in the header MMR ()
* HeaderEntry for storing subset of header info in the header MMR

* cleanup

* cleanup
2018-12-13 09:57:24 +00:00
hashmap
aedac483f5
Convert to Rust 2018 edition ()
* Convert to Rust 2018 edition

* Update gitignore
2018-12-08 00:59:40 +01:00
Ignotus Peverell
ef55b35416
Support for genesis block that includes a coinbase ()
* Chain init now handles genesis body properly, related unit test creating the genesis with reward
* Avoid making block body public by adding a with_reward method
* apply_block in all genesis cases works
2018-12-06 15:38:15 -08:00
Antioch Peverell
02b3bf4b59 utxo_view hash compare ()
* utxo_view validate_input now checks the data in the output MMR
* cleanup apply_input and validate_input
2018-12-05 09:58:40 -08:00
Antioch Peverell
bf815aa5cd
Readonly pmmr cleanup ()
* avoid locking txhashset by using a readonly PMMR

* rustfmt
2018-12-05 14:52:53 +00:00
Antioch Peverell
a24d613b05
no need for write lock on txhashset for chain.is_unspent() ()
* no need for write lock on txhashset

* commit
2018-12-03 09:35:40 +00:00
Antioch Peverell
8bad93c8c9
Reduce amount of clone() going on when applying data to MMRs ()
* less cloning when pushing to PMMR

* pass a ref to data when applying to the MMR and appending to the backend

* cleanup

* rustfmt

* fixup tests
2018-12-02 19:59:24 +00:00
Antioch Peverell
f05d3dd920
decouple outputs from output MMR () 2018-11-29 10:17:44 +00:00
Antioch Peverell
8e62130a7a
decouple kernels from the kernel MMR () 2018-11-29 10:01:59 +00:00
Antioch Peverell
8e3a3e1a40
Replace "header by height" index with reads into the header MMR ()
* replace header_by_height index with reads into the header MMR

* rustfmt

* cleanup

* cleanup chain tests

* fix locate_headers to stop on our max header

* fix the deadlock in comact_blocks_db...

* cleanup and docs/comments
2018-11-29 09:26:24 +00:00
Antioch Peverell
79d540cf86
Header MMR Refactor ()
* refactor header MMR to use regular backend
this lets us store header hashes in the mmr data file
the hashes in the hash file are "hash with index"

* rustfmt

* get_header_hash(pos) on header extension

* rustfmt

* cleanup

* cleanup

* comments

* cleanup

* fixup testelem so tests work
2018-11-27 12:32:39 +00:00
Antioch Peverell
207621b545
Tx kernel entry ()
* Wrap tx kernels in tx kernel entries when storing them in the MMR.
Decouples the storage impl of kernels.

* rustfmt
2018-11-11 16:03:03 +01:00
Gary Yu
1b264d3554
protection of parallel requests of TxHashSetRequest ()
* protection of parallel requests of TxHashSetRequest

* fix: adapt the test of test_unexpected_zip
2018-11-05 22:56:45 +08:00
Antioch Peverell
d3a8613e43
use hash_file abstraction for pmmr backend ()
* use HashFile (wraps an AppendOnlyFile) in the PMMR backend

* rustfmt

* cleanup

* rustfmt
2018-11-05 12:01:24 +00:00
Antioch Peverell
d23dec73d0
PMMRable cleanup ()
* cleanup pmmrable and len()
introduce FixedLength trait with a const LEN
make Hash impl FixedLength for consistency

* rustfmt

* store tests cleanup

* rustfmt

* whats going on with those comments and rustfmt?
2018-11-01 20:14:46 +00:00
hashmap
368e1a461e
Tiny store crate cleanup () 2018-11-01 12:34:47 +01:00
Antioch Peverell
12be191ecd
validate root of header MMR when processing headers (during sync and full blocks) ()
validate root of header MMR when processing headers (during sync and full blocks) ()
2018-11-01 09:51:32 +00:00
Antioch Peverell
d51522a9cd Genesis is height 0 (not height 1) () 2018-10-31 13:24:21 -07:00
hashmap
3efa7bdac9
Small style changes in core crate () 2018-10-23 21:09:16 +02:00
eupn
1195071f5b Replace logging backend to log4rs and add log rotation ()
* Replace logging backend to flexi-logger and add log rotation
* Changed flexi_logger to log4rs
* Disable logging level filtering in Root logger
* Support different logging levels for file and stdout
* Don't log messages from modules other than Grin-related
* Fix formatting
* Place backed up compressed log copies into log file directory
* Increase default log file size to 16 MiB
* Add comment to config file on log_max_size option
2018-10-21 13:30:56 -07:00
yeastplume
6980278b95 merge T4 into master - rustfmt 2018-10-18 11:23:22 +01:00
yeastplume
f94ede9af3 merge T4 into master 2018-10-18 11:23:04 +01:00
Antioch Peverell
fbf955dd11
Commit to prev_root in block headers ()
* commit to prev_root in block headers

* prev_root ready to go, mergeable onto testnet4
2018-10-17 10:06:38 +01:00
John Tromp
8540e4f723 [T4] tweaks and fixes ()
* refactor consensus.rs, tweaking some values
* move scale() there
* fix set_header_nonce bug
* remove maturity soft-fork code
* increase diff target precision
* fix weight comments and try resolve PR conflict
2018-10-16 16:14:22 -07:00
Antioch Peverell
86c1d7683b
The Header MMR (One MMR To Rule Them All) () ()
* header MMR in use within txhashset itself
works with fast sync
not yet in place for initial header sync

* add the (currently unused) sync_head mmr

* use sync MMR during fast sync
rebuild header MMR after we validate full txhashset after download

* support missing header MMR (rebuild as necessary) for legacy nodes

* rename to HashOnly

* cleanup backend.append()

* simplify vec_backend to match simpler append api
2018-10-15 19:24:01 +01:00
Antioch Peverell
8b7a20f8b0
The Header MMR (One MMR To Rule Them All) ()
* header MMR in use within txhashset itself
works with fast sync
not yet in place for initial header sync

* add the (currently unused) sync_head mmr

* use sync MMR during fast sync
rebuild header MMR after we validate full txhashset after download

* support missing header MMR (rebuild as necessary) for legacy nodes

* rustfmt

* comments/docs

* rustfmt

* cleanup DBBackend

* cleanup DBBackend

* cleanup

* rename to HashOnly

* rustfmt

* cleanup backend.append()

* simply pmmr append api
no need to pass position when appending

* cleanup

* simplify vec_backend to match simpler append api

* rustfmt

* docs/comments

* rustfmt

* cleanup
2018-10-15 17:16:34 +01:00
Antioch Peverell
acf61db463
skip rangeproof and kernel signature verification unless we are doing a "full" chain validation () 2018-10-07 13:39:40 +01:00
Antioch Peverell
985bced99d
remove duplicate dir creation code () 2018-09-29 09:50:32 +01:00
Antioch Peverell
acec59e249
introduce rewindable_kernel_view (readonly but rewindable kernel MMR view) ()
* introduce rewindable_kernel_view
cleanup header in extension

* cleanup and docs/comments

* txhashset does not need to be mutable here

* pull validate_kernel_history out into fn
2018-09-26 09:59:00 +01:00