Commit graph

87 commits

Author SHA1 Message Date
Antioch Peverell
2d4538c428
Clean shutdown via stop_state (#2117)
* clean shutdown wip

* rustfmt

* introduce StopState that we can lock on

* rustfmt

* take lock on stop_state during critical processing (process_block_single etc.)

* rustfmt

* take lock on stop_state during chain::init()

* cleanup

* cleanup

* rustfmt

* docs/comments

* fixup servers tests

* cleanup p2p tests
2018-12-11 11:07:41 +00:00
David Burkett
0ce8f3e1cd Changing compression method to 'stored' instead of bzip2. (#2111) 2018-12-10 10:20:21 -08:00
hashmap
aedac483f5
Convert to Rust 2018 edition (#2084)
* Convert to Rust 2018 edition

* Update gitignore
2018-12-08 00:59:40 +01:00
Ignotus Peverell
87bfd9aaa4
Mark all crates as publishable 2018-12-07 00:08:15 +00:00
Antioch Peverell
7b07fe99a7
bump 0.4.1 -> 0.4.2 so we can watch the HF (#2057) 2018-11-30 11:20:12 +00:00
Antioch Peverell
f0fa410273
Streaming headers (#1989)
* headers msg is now "streamed" off the tcp stream

* rustfmt

* cleanup

* move StreamingReader into ser.rs
extract read_exact out into util crate

* rustfmt
2018-11-16 11:00:39 +00:00
Antioch Peverell
cb652e7ef1
do not treat txhashset.zip download as abusive behavior (#1973)
* do not treat txhashset.zip download as abusive behavior

* count times, not bytes so we exclude quiet increments

* use inc_quiet when tracking sent bytes via an attachment

* add comment

* fixup "quiet" counter entries

* rustfmt
2018-11-13 21:34:45 +00:00
Antioch Peverell
183d9c9f3e
Refactor kernel sig msg creation (#1954)
* refactor kernel msg creation

* rustfmt

* Error not secp::Error

* fix tests for refactored errors
2018-11-10 16:24:11 +01:00
Antioch Peverell
39ebb33ba1
bump 0.4.0 -> 0.4.1 (tx kernel broadcast support) (#1950) 2018-11-09 14:57:25 +00:00
Ignotus Peverell
e9c50ccb56
Time comparisons are difficult. Or not. 2018-11-08 00:40:28 +00:00
Ignotus Peverell
f7d0fe1840
Bug fix, tuning of peer message rate counting 2018-11-07 22:15:12 +00:00
Ignotus Peverell
8b546632fe
Peer rate limiting (#1933)
* Rate counter in peer conn to monitor traffic 
* Ban peers that trigger is_abusive
2018-11-06 17:51:22 -08:00
Gary Yu
d56ef364c4
fix: init_test_logger update for log4rs (#1921) 2018-11-05 11:57:59 +08:00
eupn
40d727a01c fix: more readable timestamp format for logging (#1912)
* Use more readable timestamp format for logging

* Add logging level highlighting

* Change date and time format to YYYYMMDD from MM-DD
2018-11-02 15:13:12 +00:00
eupn
0959bf376c Avoid stdout and panic logging when TUI is running (#1837) 2018-10-27 10:37:03 -07:00
Ignotus Peverell
c083312ad4
Improve crate descriptors to be more cargo-friendly (#1844)
* Update secp dependency to use crates.io version
* Add more details to various Cargo.toml
* Remove use of env variables that aren't reliably provided by cargo
2018-10-25 17:44:50 -07:00
eupn
3d74d7fe85 Fix path to log file in panic message (#1815) 2018-10-24 09:13:57 -07:00
eupn
1195071f5b Replace logging backend to log4rs and add log rotation (#1789)
* Replace logging backend to flexi-logger and add log rotation
* Changed flexi_logger to log4rs
* Disable logging level filtering in Root logger
* Support different logging levels for file and stdout
* Don't log messages from modules other than Grin-related
* Fix formatting
* Place backed up compressed log copies into log file directory
* Increase default log file size to 16 MiB
* Add comment to config file on log_max_size option
2018-10-21 13:30:56 -07:00
Gary Yu
0d06561a91
replace stdlib RwLock and Mutex with parking_lot (#1793)
* replace all stdlib RwLock with parking_lot RwLock

* replace stdlib Mutex with parking_lot Mutex

* rustfmt
2018-10-20 08:13:07 +08:00
yeastplume
9423865f92 update cargo versioning 2018-10-15 10:34:33 +00:00
Gary Yu
3b0006934e
suppress the test error of test_start_api (#1740)
* Use secp crate directly without extra use statement (#1738)

(cherry picked from commit 80d28f94ea)

* suppress the test error of test_start_api. Note: this is not a fix.

(cherry picked from commit 6f29685daf)
2018-10-14 21:15:38 +08:00
Yeastplume
6c8c483172
[T4 ONLY] BIP32 Wallet Compliance - Aggsig Updates - Bulletproof Updates (#1501)
* change keychain to use bip32 paths

* convert keychain to use bip32

* change identifier to be serialisation of 4-level bip32 path

* wallet changes compiling, pass parent key into all wallet functions

* rustfmt

* fix tests in chain

* rustfmt

* core tests passing

* rustfmt

* pool tests

* rustfmt

* fixing wallet tests

* rustfmt

* remove file wallet

* wallet tests compiling

* rustfmt

* remove db_migrate

* successful tx exchange test using BIP32 paths

* rustfmt

* fix wallet derivation paths to m/0/0/0

* wallet test fixed again, working with default path

* rustfmt

* fix server tests

* rustfmt

* make parent_id a trait on walletbackend

* rustfmt

* add ability for wallet to switch between multiple named accounts, and tests (not complete)

* rustfmt

* account switching tests in place and passing

* rustfmt

* compile and test with latest libsecp changes

* added public key sum to calculated e for aggsig

* rustfmt

* Update secp to 26

* bulletproof bip32 path integration

* rustfmt

* wallet restore updated with bip32 paths, also restores accounts

* rustfmt

* rustfmt

* remove old extkey

* remove old extkey

* rustfmt

* add wallet account commands

* rustfmt

* update wallet documentation

* rustfmt

* merge from master

* update libsecp tag

* merge from upstream and fix server test

* rustfmt

* rustfmt

* merge from master

* update latest libsecp merge

* fix commitment to zero value generation
2018-10-10 10:11:01 +01:00
Antioch Peverell
a676eb1b39
Replace RefCell with RwLock in OneTime (and cleanup Weak usage) (#1694)
* rustfmt

* use RwLock in OneTime (and not RefCell)

* rustfmt

* put weak refs back and cleanup

* rustfmt

* revert weak in adapters
2018-10-09 16:53:57 +01:00
Quentin Le Sceller
62fd8f2124 Implement Basic Auth for API and Owner API (#1566)
* Add api_secret

* Add to base64 method

* Add basic auth in API

* Add Basic Auth to owner API

* Add flag to enable disable basic auth

* Add .api_secret file
2018-09-26 22:38:44 +02:00
Gary Yu
8d62247c99 Small cleanup of rand usage (#1576)
Clean 3 build warning of deprecated item 'rand::Rand::rand'
2018-09-24 15:15:55 -07:00
Quentin Le Sceller
345a6cb53e Update rand crate to 0.5 (#1542)
* Update rand crate to 0.5

* Rustfmt

* Misplaced import
2018-09-18 09:39:45 +01:00
Gary Yu
a0deb18e77
Update version of some crates (#1536)
Update version of some crates
2018-09-17 10:47:48 +08:00
Yeastplume
63880f71c7
BIP32 Lib Integration (#1454)
* Test integrating BIP-32 implementation (not complete)

* Test integrating BIP-32 implementation (not complete)

* factor out bip32 crypto functions into trait

* rustfmt

* compliation

* rustfmt

* fixes for test vectors.. all work now with hashes specified in BIP32

* rustfmt

* move reference hasher out of test
2018-09-04 10:58:26 +01:00
Gary Yu
60d62bfb97 add a test case for transaction deserialize (#1356) (#1381)
* add a test case for transaction deserialize (#1356)
* update rust-secp256k1-zkp to grin_integration_23
* add transaction explicit validation
2018-09-01 15:14:10 -04:00
Yeastplume
8440aad7ea
Retrieve header by output commit (#1388)
* Add get_header_for_output function to chain

* add api call to retrieve associated header for a given output

* rustfmt
2018-08-20 19:02:44 +01:00
Gary Yu
54d3fc0af2 improve: use batch verification for txhashset rangeproof validation (#1321) (#1363)
* improve: use bullet rangeproof batch verification for txhashset validation (#1321)

* update rust-secp256k1-zkp to tag 'grin_integration_22'
2018-08-17 16:18:48 +01:00
Ignotus Peverell
dcfbda9c89
Fix test logger, debug level, no file 2018-08-16 19:32:03 +01:00
hashmap
779921f71a Fix compiler warnings (#1361)
* env::home_dir is deprecated, compiler suggests to use crate `dirs`
* add superficial comments to file util module
2018-08-15 17:11:05 -07:00
Yeastplume
e70ff53c3c
Use tagged version of rust-libsecp256k1 (#1359) 2018-08-15 17:02:06 +01:00
Quentin Le Sceller
3df050cc93 Check content before zip/unzip the txhashset (#1174)
* Check txhashset content before zip/unzip
* Add header in txhashset verification
* Add copy function and test
* Add file util
* Now check and remove unexpected files instead of just crashing
2018-08-02 19:16:16 -07:00
Ivan Sorokin
5c029e3f87 Make grin.toml config optional (#1278)
* Make grin.toml config optional. Mirror exisiting config parameters in grin.toml to source code, so binary can run without a config file. Add test for it.
* fixup! Make grin.toml config optional
2018-07-29 17:48:24 -07:00
Quentin Le Sceller
b5fe22dbd5 Bump to 0.3 (#1225) 2018-07-06 18:29:39 +01:00
Yeastplume
38a7936521
Use updated bulletproof API (#1194) 2018-06-25 12:28:56 +01:00
yeastplume
e7c380836b rustfmt 2018-06-22 09:44:38 +01:00
Ignotus Peverell
d189770080 Wallet LMDB backend (#1151)
* Migrate main node store to LMDB

In preparation to using LMDB as a wallet database, migrate the
node db. There's no point in having 2 key-value stores.
In addition LMDB provides a few advantages as a node db, namely a
much faster build (compared to RocksDb), lesser dependencies and
transactions.

* Migrated p2p store to lmdb, stuff compiles

* More fixes, chain tests starting to pass

* Fixed txhashset rollback messing with block save and general batch delimitation. Chain tests passing.

* rustfmt

* LMDB max map size of 10MB isn't really workable. Half TB seems reasonable.

* Fix wallet tests

* Rather crucial commit was missing

* rustfmt

* Fixing new merged tests following lmdb changes

* rustfmt

* * Make txhashset validation read-only on fast sync to avoid having
a really long open transaction.
* Fix deadlock in new block processing, batch should always be
created within a txhashset lock (when they interact).

* Comment about batch and txhashset interlacing

* Fix store tests to use batch

* Externalize wallet config and seed

* Converted direct read access to file outputs map to an iterator

* Cleaned up and simplified wallet Backend trait:

* No more direct mutable access to internal structures (HashMap)
* Batch interface for all writes
* Remove unneeded read wrapper (read_wallet)

* rustfmt

* First (incomplete) pass at wallet LMDB backend

* Progressing on lmdb backent iml

* Added batch impl for LMDB wallet backend. Pretty much done with it, but not sure how to deal with commit (owned).

* rustfmt

* Wrapping LMDB batch around a refcell to work around borrow rules

* Compilation up to grin chain
2018-06-22 09:08:06 +01:00
Antioch Peverell
028b14d9d9
Replace rm_log with leaf_set (compact bitmap) (#1147)
* only store leaves in rm_log

* cleanup

* commit

* add failing test to cover case where we compact
an already compacted data file

* fix the logic for pruning the data file

* rm_log only contains leaves
prunelist only contains non-leaf subtree roots

* cleanup

* commit

* bitmap impl running in parallel

* rustfmt

* this is better - rewind unremoves outpu pos spent by rewound inputs

* commit

* commit

* pass bitmap around when rewinding

* store and retrieve input bitmap per block from the db

* Replace the RemoveLog with the UTXO set.

* utxo set starting to pass tests

* stuff works...

* split store types out
added some tests to compare the performance of the rm_log and the proposed utxo_set

* pull prune_list out into standalone file

* cleanup, get rid of unused height param

* cleanup and add doc comments

* add support for migrating rm_log to utxo_set

* take snapshot of utxo file during fast sync
implement migration of rm_log -> utxo_set

* rename rewound_copy to snapshot

* fixup pmmr tests to reflect cutoff_pos

* cleanup unused import

* check_compact needs to rewind the utxo_set as appropriate

* fix pool tests

* fixup core tests

* cache block_input_bitmaps via LruCache in store

* cache block header on initial write to db

* rename utxo_set -> leaf_set
and remove references to "spent" in grin_store

* better document the rewind behavior
2018-06-18 11:18:38 -04:00
Quentin Le Sceller
8f4dbfa540 Many typo fixes (#1158)
* Typo fix
* Parameterise
2018-06-13 17:03:34 +01:00
Quentin Le Sceller
1255ea2224
Upgrade and uniformize dependencies (#1138)
* Remove unused DandelionConfig

* Cargo upgrade and cleanup
2018-06-05 13:26:32 -04:00
Yeastplume
85285473bd
[WIP] Wallet refactor - part 3 (#1072)
* Beginning to rework aggsig library workflow

* more refactoring of transaction api

* whoever does round 1 first creates offset

* slate finalisation now context-free, so anyone can do it

* remove concept of transaction phase

* remove slate phase enum

* update actual send/receive code with new transaction lib workflow
2018-05-21 16:28:11 +01:00
Antioch Peverell
4dd94ff39e
[testnet2] Store output sum (#1043)
* block sums and reworked block validation
read and write block_sums
refactor validate on both block and txhashset
write block_sum on fast sync
we store the kernel_sum (need to account for the offset)

* block_sums

* rustfmt

* cleanup
2018-05-07 09:21:41 -04:00
Dawid Ciężarkiewicz
2b2e13be63 Optimize logging (#992)
`Async<Discard>` means "drain that will send each logging
record to an async thread, that will discard it". This means
logging record is sent to another thread for no reason.
2018-04-24 11:02:49 +01:00
Ignotus Peverell
327293d28e Genesis, version changes and doc for testnet2 (#865)
* Bump up crates versions

* Finally add a Cargo.lock to avoid dependency breakages

* Build doc update for testnet2

* Fix test framework not really using its mining config

* Testnet2 genesis, best so far at 128 difficulty (a nice number)

* Minor build doc update
2018-03-26 10:48:46 +01:00
Simon B
ccf0c11da6 Fix 2 compiler warnings. More \n around panic outputs (#867)
* avoid compiler warnings on 3 tests
* more newlines around logging a panic
2018-03-26 01:35:28 +00:00
Simon B
9285de2c24 Here is a case where ~ is needed, to allow cargo update to only touch the patch level version (third number). Anything else than ~ means that both first numbers can be raised, so "2.1" updaates to "2.2" (#866) 2018-03-25 20:08:38 +00:00
Ignotus Peverell
2bf656646c
Force slog version to 2.1 (2.2 broke some traits) 2018-03-25 19:54:41 +01:00