Commit graph

35 commits

Author SHA1 Message Date
Ignotus Peverell
d189770080 Wallet LMDB backend (#1151)
* Migrate main node store to LMDB

In preparation to using LMDB as a wallet database, migrate the
node db. There's no point in having 2 key-value stores.
In addition LMDB provides a few advantages as a node db, namely a
much faster build (compared to RocksDb), lesser dependencies and
transactions.

* Migrated p2p store to lmdb, stuff compiles

* More fixes, chain tests starting to pass

* Fixed txhashset rollback messing with block save and general batch delimitation. Chain tests passing.

* rustfmt

* LMDB max map size of 10MB isn't really workable. Half TB seems reasonable.

* Fix wallet tests

* Rather crucial commit was missing

* rustfmt

* Fixing new merged tests following lmdb changes

* rustfmt

* * Make txhashset validation read-only on fast sync to avoid having
a really long open transaction.
* Fix deadlock in new block processing, batch should always be
created within a txhashset lock (when they interact).

* Comment about batch and txhashset interlacing

* Fix store tests to use batch

* Externalize wallet config and seed

* Converted direct read access to file outputs map to an iterator

* Cleaned up and simplified wallet Backend trait:

* No more direct mutable access to internal structures (HashMap)
* Batch interface for all writes
* Remove unneeded read wrapper (read_wallet)

* rustfmt

* First (incomplete) pass at wallet LMDB backend

* Progressing on lmdb backent iml

* Added batch impl for LMDB wallet backend. Pretty much done with it, but not sure how to deal with commit (owned).

* rustfmt

* Wrapping LMDB batch around a refcell to work around borrow rules

* Compilation up to grin chain
2018-06-22 09:08:06 +01:00
Quentin Le Sceller
1255ea2224
Upgrade and uniformize dependencies (#1138)
* Remove unused DandelionConfig

* Cargo upgrade and cleanup
2018-06-05 13:26:32 -04:00
Quentin Le Sceller
d6b689bada
Simplify DandelionConfig (#1115)
* Simplify DandelionConfig and make fields optional
2018-06-01 18:41:39 -04:00
Ignotus Peverell
327293d28e Genesis, version changes and doc for testnet2 (#865)
* Bump up crates versions

* Finally add a Cargo.lock to avoid dependency breakages

* Build doc update for testnet2

* Fix test framework not really using its mining config

* Testnet2 genesis, best so far at 128 difficulty (a nice number)

* Minor build doc update
2018-03-26 10:48:46 +01:00
Simon B
9285de2c24 Here is a case where ~ is needed, to allow cargo update to only touch the patch level version (third number). Anything else than ~ means that both first numbers can be raised, so "2.1" updaates to "2.2" (#866) 2018-03-25 20:08:38 +00:00
Ignotus Peverell
2bf656646c
Force slog version to 2.1 (2.2 broke some traits) 2018-03-25 19:54:41 +01:00
Simon B
f9b87164e7 Idiomatic cargo toml (#862)
* Specify versions SemVer-style
 - slog and serde
* Comments for not update-able pre-1.0 crates:
 - urlencoded at "0.5" since 0.6+ lacks trait `plugin::Plugin<iron::Request<'_, '_>>`
* alpha-sort deps
* Specify versions SemVer-style
 - backtrace, bitflags, blake2-rfc, bodyparser, built, byteorder
 - chrono, clap, demonize, enum_primitive, env_logger
 - failure, failure_derive, futures
 - hyper, iron, itertools, lazy_static, libc
 - memmap, mount, net2, num_bigint, prettytable-rs
 - rand, regex, rocksdb, router, siphasher
 - term, time, tokio-core, tokio-retry, urlencoded, uuid, walkdir, zip
2018-03-25 17:44:27 +00:00
Simon B
4fa9ccc4f7 Cargo.toml (#818)
* Make sure no grin crates get published by mistake: https://doc.rust-lang.org/cargo/reference/manifest.html#the-publish--field-optional
* add missing workspace = '..', [workspace docs for interested readers](https://doc.rust-lang.org/cargo/reference/manifest.html#the-workspace-section)
* dev- or dev_ works equally well, but dev- is what's typically used
* authors = Grin Developers <maillist>
* [dependencies] keep grin-* at the end + readability
2018-03-20 18:21:29 +00:00
Quentin Le Sceller
fb46fad0ac Remove unused import and uniformize crates (#686)
* Removed unused crates
* Add listconnectedpeers in grin client
* Removed unused import and uniformize crates
2018-02-05 19:46:56 +00:00
Ignotus Peverell
c7418cfe04
[WIP] Rewrite peer-to-peer logic to replace tokio with simple threading (#664) 2018-02-02 02:03:12 +00:00
Quentin Le Sceller
1e50d56c7e Add unit tests for API Endpoint (#653)
* First tests for API handlers

* Test chain UTXO handler

* Add test sumtrees handlers

* Fix typo

* Removed unused logger

* Update bitflags to ^1.0

* Missing test for bitflags update

* Moved test to grin directory

* Remove tests dependencies

* Add API P2P tests

* Fix hex string commitment

* Fix conflicting port

* Fix directory conflict and server port

* Wait for at least one block is mined on Travis
2018-02-01 13:14:32 -05:00
Simon B
38741c91e2 cherry-pick "[testnet1] cpu_pool and buffered fold in read_msg (#479)" (#482) 2017-12-14 07:19:43 -05:00
AntiochP
bffd955c26
Revert "introduce cpu_pool to read_msg (#477)" (#478)
This reverts commit 45b9962109.
2017-12-13 16:30:59 -05:00
AntiochP
45b9962109
introduce cpu_pool to read_msg (#477)
* introduce cpu_pool to read_msg

* cleanup and less unwrap() in handler

* expose cpu_pool as part of net_adapter (reuse across all peer connections and handlers)

* fix DummyAdaptor for test usage

* add new() to DummyAdapter for convenience
2017-12-13 15:05:25 -05:00
Yeastplume
49797853d9 tweaks to logging configuration (#176)
* tweaks to logging configuration
* doc port update
2017-10-13 16:42:04 +00:00
Yeastplume
8e382a7593 slog-rs logging (#171)
* added global slog instance, changed all logging macro formats to include logger instance
* adding configuration to logging, allowing for multiple log outputs
* updates to test, changes to build docs
* rustfmt
* moving logging functions into util crate
2017-10-12 16:56:44 +00:00
AntiochP
677d0a3a95 keychain crate (no more secretkeys in core) (#146)
* introduce grin_keychain, encapsulate derivation of secret_keys
* core compiles against keychain, tests don't run yet
* core tests are now passing against keychain
* wip - getting wallet working with keychain
* add util and keychain to travis test matrix
* basic test around key derivation
2017-10-03 00:02:31 +00:00
Seamus Finnigan
2e819c72aa Fix deprecated futures items (#148)
* Update futures deprecated send item
* Update futures deprecated iter item
* Fix cargo req for futures
2017-10-01 21:34:14 +00:00
AntiochP
7c5b7d23e7 Replace secp256k1 directory with repository dependency (#126)
* workaround the "commitment is not a public key" problem by using to_two_keys() from rust-secp256k1-zkp
* remove secp256k1zkp, update the various Cargo.toml to use rust-secp256k1-zkp from external git repo, update code to not rely on to_pubkey (and use verify_from_commit)
* secp256k1zkp from git
* cleanup secp256k1 dependencies
* gitignore node1 and node2 dirs
2017-09-21 00:07:40 +00:00
Jacob Payne
6a15100c96 [WIP] Rate Limiting (#57)
* Setup tests for rate limiting
* Add send and receive rate configs to P2P Config
* Setup ThrottledConnection and ThrottledWrite
* Add Tokio io
* Fix ThrottledReader + Add Bytes
* Attach Rate Limiting to Connection
2017-06-18 01:15:46 +02:00
Ignotus Peverell
40d23db6c1
Revert "[WIP] Partial Transition from Writeable/Readable to Codecs (#51)"
This reverts commit de4ebdde71.
2017-05-31 15:47:52 -07:00
Ignotus Peverell
f79fb8ef95
Sending end of the wallet
Most of the logic to build a transaction that sends coin to
another party. Still requires more debugging and clean up.
Main changes and additions are:

* Update to serde 1.0
* API endpoint to retrieve an Output
* Output is now Serialize and Deserialize
* Wallet configuration
* Command line for the send operation
* Wallet data checker to update created outputs into confirmed
* Wallet-specific configuration
2017-05-28 20:21:29 -07:00
Jacob Payne
de4ebdde71 [WIP] Partial Transition from Writeable/Readable to Codecs (#51)
* Sample Signatures for put_enc and get_dec
* Implement put_enc and get_dec
* Implement ChainCodec in grin_chain
* Truncate src only on complete Blocks
* Truncate src only on complete Tip + Check Len
* Move BlockHeader Encoding to BlockHeaderCodec
* Define put_enc for store::Batch
* Replace BlockCodec and BlockHeaderCodec with generic BlockCodec<T>
* Implement Default for BlockCodec Manually
* Replace get_ser/put_ser with get_enc/get_dec for chain::ChainKVStore
* Remove Writeable/Readable for chain::Tip
* Add Tokio-io and Bytes to grin_p2p
* Additional Setup for Message enum + Msg{Encode,Decode} traits
* base msg ping pong encoding and test
* fill out msg-codec tests
* Implement Hand Encoding/Decoding
* msg-encode shake
* msg-encode getpeeraddr
* codec peer-addrs message, SockAddr struct wierdness
* header message codec
* msg encoding finished prelim
* Implement PeerCodec Encoding/Decoding
* Set PeerStore to use PeerCodec for Encoding/Decoding
* Add a DecIterator
* Prune PeerStore
* Replace Decoding and Encoding in handle_payload
* Prune Writeable/Readable methods in store::Store
* Remove Incomplete Frame Testing ( Not Nessesary right now )
* separate block and tx codec tests
* Refactor {Tx,Block}Codec Tests
2017-05-22 19:16:13 +02:00
Ignotus Peverell
3fe28b0f03
Server types cleanup and serde for configuration types
Moved error and configuration types from server.rs to types.rs
where they should have been from the start. Configuration types
for the server and p2p modules now derive serde Serialize and
Deserialize so we can have a simple JSON configuration file.
2017-04-24 18:49:57 -07:00
Ignotus Peverell
382754e54c
Add workspace support for all crates 2017-04-05 23:41:49 -07:00
Ignotus Peverell
786da24653
A db store for peer data. 2017-02-09 20:16:34 -08:00
Ignotus Peverell
7f029cb4c0
Added a Connection wrapper to handle timeouts when we want information from a peer in a request/response style. 2017-02-01 19:05:17 -08:00
Ignotus Peverell
572c1951e1
Introduced a higher-level connection abstraction, allowing the protocol to stay relatively simple. Deals with the lower level details of sending, receiving data, timeouts and dealing with futures. 2017-01-29 22:43:20 -08:00
Ignotus Peverell
19a700b81e
Minor fixes, testing server peer count. 2016-12-13 16:05:24 -08:00
Ignotus Peverell
4b5c010b05
Rewrote most of p2p code to use futures-rs instead of mioco. Need some cleanup and support for a few more message types. 2016-12-10 19:11:49 -08:00
Ignotus Peverell
ea425dc614
Protocol cleanup. Tests cleanup. Additional test for transaction broadcast. 2016-11-05 16:31:45 -07:00
Garrick Ollivander
d7dcd6d3ae
nicer processing of message types 2016-10-31 13:08:48 +01:00
Ignotus Peverell
fdaf2ba6af
First working test with related fixes demonstrating a full peer handshake. 2016-10-27 14:28:02 -07:00
Ignotus Peverell
b1762cb5f4
p2p module getting close to first integration test allowing 2 peers to connect. Fleshed out handshake and protocol use. Fixed most errors, just need to make the borrow checker happy. 2016-10-25 22:06:13 -07:00
Ignotus Peverell
c8aa8d7c18
Getting started on the p2p networking code. 2016-10-23 17:02:02 -04:00