Commit graph

88 commits

Author SHA1 Message Date
Antioch Peverell
d284d8f6de
[2.x.x] Writeable protocol version aware (#2856)
* introduce protocol version to deserialize and read

* thread protocol version through our reader

* cleanup

* cleanup

* streaming_reader cleanup

* Pass protocol version into BinWriter to allow for version specific serialization rules.

* rustfmt

* read and write now protocol version specific
2019-07-06 15:51:03 +01:00
Antioch Peverell
5aaf2d058d
[2.x.x] deserialization now protocol version aware (#2824)
* introduce protocol version to deserialize and read

* thread protocol version through our reader

* example protocol version access in kernel read

* fix our StreamingReader impl (WouldBlock woes)

* debug log progress of txhashset download
2019-06-27 17:19:41 +01:00
hashmap
9ab23f6eef Implement graceful shutdown (#2812)
* Remove stop status mutex
* remove some deadlocks
* Rewrite stop channel handling
* fix deadlock in peers object
* add missing test fixes
2019-05-15 08:51:35 -07:00
Antioch Peverell
e56cd55980
initial support for kernel data download (#2765)
* initial support for kernel data download

* fix vec backend for tests

* cleanup after rebase
2019-05-14 17:17:38 +01:00
Antioch Peverell
ff1c55193f Wrap MsgHeader in MsgHeaderWrapper for Known/Unknown msg type support (#2791)
* wrap MsgHeader in MsgHeaderWrapper for Known/Unknown msg type support.
* cleanup based on feedback
2019-05-14 09:07:51 -07:00
Gary Yu
97e96e4fcf use a grin specific tmp dir instead of a system tmp dir (#2753) (#2772)
* use a grin specific tmp dir instead of a system tmp dir

* rustfmt

* using create_new instead of create

* switch the txhashset sandbox from system tmp dir to grin specific tmp dir

* rustfmt

* adapter get_tmp_dir delegate to chain implementation

* simply the logic of protolc for get_tmpfile_pathname

* rustfmt

* fix the merge mistake
2019-04-22 16:54:36 -07:00
Antioch Peverell
13c6160e15
pass peer_info around rather than peer_addr (includes protocol version) (#2761) 2019-04-18 14:11:06 +01:00
hashmap
94732b0d58
Return Result from methods of ChainAdapter (#2722)
Most of the methods return nothing or bool which is used to decide if a
sender of a message should be banned or not. However underlying chain
implementation may fail so we need a way to reflect this fact in API.

Also it allows to reduce number of unwraps and makes the code more robust.
2019-04-08 22:13:28 +02:00
hashmap
fe9fa51f32
Reduce usage of unwrap in p2p crate (#2627)
Also change store crate a bit
2019-02-25 19:48:54 +01:00
Antioch Peverell
23cb9e2514
Map peers by ip only (ignoring port unless on loopback ip) (#2540)
* wip

* big refactor, regretting doing this now

* PeerAddr everywhere

* cleanup

* fixup server tests

* peers api working for GET, POST is still WIP

* we can now ban/unban peers by ip only (port optional)
2019-02-18 12:15:32 +00:00
Ignotus Peverell
f9a20aef0d
Track blocks we requested, always broadcast otherwise (#2349) 2019-01-12 09:28:03 -08:00
hashmap
aedac483f5
Convert to Rust 2018 edition (#2084)
* Convert to Rust 2018 edition

* Update gitignore
2018-12-08 00:59:40 +01:00
Antioch Peverell
a3523028dc
Sync headers slice (#2085)
* pass slices of headers around, not vec of headers

* rustfmt
2018-12-05 16:50:32 +00:00
antiochp
f7608f5b9c
rustfmt 2018-11-16 14:34:05 +00:00
antiochp
4b48eebdfe
refactor some p2p to use Read/Write over TcpStream 2018-11-16 14:33:35 +00:00
Antioch Peverell
f0fa410273
Streaming headers (#1989)
* headers msg is now "streamed" off the tcp stream

* rustfmt

* cleanup

* move StreamingReader into ser.rs
extract read_exact out into util crate

* rustfmt
2018-11-16 11:00:39 +00:00
Antioch Peverell
cb652e7ef1
do not treat txhashset.zip download as abusive behavior (#1973)
* do not treat txhashset.zip download as abusive behavior

* count times, not bytes so we exclude quiet increments

* use inc_quiet when tracking sent bytes via an attachment

* add comment

* fixup "quiet" counter entries

* rustfmt
2018-11-13 21:34:45 +00:00
Gary Yu
7ff1ee5fde
Peer send receive bytes statistics missed response messages and attachment (#1953)
* fix: peer send receive bytes statistics missed response message and attachment

* rustfmt
2018-11-11 07:30:57 +08:00
Antioch Peverell
d97a6c2189
[DNM] broadcast tx kernel hash (if supported by peer) (#1929)
broadcast tx kernel hash (if supported by peer)
2018-11-07 09:28:17 +00:00
eupn
1195071f5b Replace logging backend to log4rs and add log rotation (#1789)
* Replace logging backend to flexi-logger and add log rotation
* Changed flexi_logger to log4rs
* Disable logging level filtering in Root logger
* Support different logging levels for file and stdout
* Don't log messages from modules other than Grin-related
* Fix formatting
* Place backed up compressed log copies into log file directory
* Increase default log file size to 16 MiB
* Add comment to config file on log_max_size option
2018-10-21 13:30:56 -07:00
John Tromp
34646ddf51 [T4] Rename all shiftsize / cuckoo_size to edge_bits and change value for T4 (#1752)
* replace all size_shift / cuckoo_size by edge_bits and change some constants for T4
* replace remaining occurrences of sizeshift
2018-10-15 16:14:23 -07:00
Gary Yu
2e6a242827
small improvement on the servers test (#1737)
* cherry-pick from master for #1736
2018-10-14 20:13:49 +08:00
Ignotus Peverell
43f4f92730
[T4] Secondary proof of work difficulty adjustments (#1709)
* First pass at secondary proof of work difficulty adjustments
* Core and chain test fixes
* Next difficulty calc now needs a height. Scaling calculation fixes. Setting scaling on mined block.
* Change factor to u32 instead of u64.
* Cleanup structs used by next_difficulty
* Fix header size calc with u32 scaling
2018-10-13 13:57:01 -07:00
Gary Yu
9a716aea72
feature: txhashset downloading progress display on tui (#1729) (#1730)
(cherry picked from commit 5c0eb11a7d)
2018-10-13 10:12:13 +08:00
Gary Yu
d3589d1bf5
small speed optimization for header sync (#1719) 2018-10-11 16:47:27 +08:00
Antioch Peverell
c9c829514b
Trace get block (#1628)
* trace on GetBlock msg

* rustfmt
2018-10-01 22:12:39 +01:00
Antioch Peverell
c291c48436 Simplify block broadcast (#1549)
* removed random "block vs compact_block"
2018-09-18 10:51:37 -07:00
Ignotus Peverell
ecf20602d5
Hard fork tryout: better PoW encapsulation in block header (#1478)
* Improve encapsulation with ProofOfWork struct
* Add dual pow scaling factor, fix test
* Fix pre_pow serialization, chain tests
* Adjust header serialized size calc
* Hard fork handling, version-based serialization
2018-09-10 15:36:57 -07:00
Quentin Le Sceller
544a296f77
Fix typos and unused imports (#1466)
* Fix typos and unused imports
2018-09-03 13:09:28 +02:00
Gary Yu
8a39f1fc75 Use 10us sleep instead of 1ms sleep (#1453) 2018-08-31 13:20:59 -04:00
Gary Yu
d719493483 HeaderSync optimization (#1372) (#1400)
* improve: HeaderSync optimization (#1372)
* remove get_locator() optimization, which should be an independent pr for security review
* refactoring: move 'headers_streaming_body()' from Message to Protocol
* move 2 headers utils functions out of Protocol, and remove 'pub'
* support reading variable size of BlockHeader, from Cuckoo30 to Cuckoo36
* fix: use global::min_sizeshift() instead of hardcoded 30, because Cuckoo10 will be used for AutomatedTesting chain
* fix: should use global::proofsize() instead of hardcoded 42 when calculate serialized_size_of_header
* replace another 42 with global::proofsize()
2018-08-30 19:50:55 -04:00
Quentin Le Sceller
8cd1b23f7a Add a peers_preferred option in Grin (#1401)
* Preferred peers option
* Move P2P configuration into proper config part
* Fix tests
2018-08-23 15:16:04 -04:00
Antioch Peverell
7dfca6077c
Refactor compact block body (#1394)
* introduce CompactBlockBody

* rustfmt

* implement From<Block> for CompactBlock

* rustfmt

* remove debug logging

* wip

* rustfmt
2018-08-22 20:19:37 +01:00
Antioch Peverell
7ca4d45bf5
Cleanup build warnings (#1391)
* cleanup some unused vars etc.

* rustfmt
2018-08-20 21:56:32 +01:00
Antioch Peverell
5abefbff33
improve logging when receiving blocks and txs (msg_len) (#1383)
* better logging for msg_len and # kernels

* rustfmt
2018-08-19 18:15:42 +01:00
Ignotus Peverell
630d0a0438
Buffered txhashset temp file write 2018-08-18 02:52:44 +01:00
e-max
25e3d9e7d3 Remove Iron dependency and update hyper to version 0.12 (#1241)
* Remove Iron dependecy and update hyper to version 0.12 #876

* REMOVE ME

* Revert "REMOVE ME"

This reverts commit e9a976eee98a2d5a4dfae5d9e1e4f5ed640c05d3.

* Rebase and start updating libwallet

Libwallet doesn't compile yet.

* Wallet compiles

* Grin compiles

* No compilation errors in tests

* All tests pass

* Reeturn future from handler

* Refactoring

* Fix lifetime issue one more time

I have to force push to rollback all the work done in last 2 days

* Fix wallet send issue

* Clean up
2018-08-01 10:44:07 +01:00
Gary Yu
3b6b85ec57 Fix fast sync mode failure when 'sync_state' was switched to 'HeaderSync' (#1286) 2018-07-23 10:37:35 -07:00
Ignotus Peverell
bdfd5405c0
Leverage sync state to accept txhashset (#1251)
Fixes #1246
2018-07-12 17:06:52 +01:00
Ignotus Peverell
1a410f115c
Cleanup a bunch of build warnings 2018-07-02 00:25:20 +01:00
Quentin Le Sceller
a30ee88236
Cleanup imports (#1161)
Reorganize imports
2018-06-14 08:16:14 -04:00
Quentin Le Sceller
ec3c2e89a4 Store and communicate ban reason (#1085)
* Add read_i32 and write_i32
* Store and send reason for ban
* Client corruption ban reason
* Forgotten max message size
2018-05-29 03:45:31 +01:00
Antioch Peverell
bc6342ea65 Fix issue where we have no metadata for a block (#938)
when restarting node before initial sync completed
2018-04-06 17:14:50 +00:00
Antioch Peverell
4fcf5fdbca
drop msgs received from a banned peer (#886) 2018-03-27 12:09:41 -04:00
Simon B
a8fc82d8c1 Console debug fixes (#861)
* move some debug! to trace!

* more informative debugs

* standardising on always showing chain tips as "cumulative difficulty @ height [hash]"

* make 2 debug outputs into a single

* "no peers" as warning (not info) to let it stand out more clearly

* move fn param (used only in this one debug line)

* clarify difficulty "units"
2018-03-25 12:41:12 -04:00
Quentin Le Sceller
fcfe7bc6a4 Basic Dandelion transaction routing (#719)
* Initial Dandelion Commit
* Changed stem_tx_pool to tx_stempool
* Introduction of stem memory pool and stem pool config
* Pool push now send to stem memory pool
* Add stem transaction functions
* Add stem transaction pool
* Drastically simplified code structure
* Add monitor transactions
* Add Dandelion monitor and remove transactions from stempool
* Add peer relay monitor
* Reconcile block with stempool
* Fix total size bug
* Add fluff option for pool push
* Added details on dandelion monitor
* Fix issue with missing parent
* Child transaction with stempool parent are now forced stem
* Update Dandelion Relay from outgoing peers
* Fix missing pool reconciliation
* Add the ability to fluff a transaction directly
* Fix tests for Dandelion
* Missing send_stem_transaction method...
* Add fluff handler for wallet
* Add logger when successfully updated Dandelion relay
* Launch transaction monitor last
* Fix dandelion relay misplaced
* Add logging and updating for stempool
* Additionnal check for stem transaction
* Fix 2 Locks in a row
2018-03-20 03:18:54 +00:00
Yeastplume
1143d84238
Remove Sumtree References and disambiguate some naming (#747)
* start of renamathon

* api renaming

* Rename UTXO-Output to lessen ambiguity

* compile warning

* compile error

* readme fix

* remove file commit in error
2018-03-05 19:33:44 +00:00
Ignotus Peverell
be8d9633e4
rustfmt all the things 2018-03-04 00:19:54 +00:00
Ignotus Peverell
22c521eec8
[WIP] Abridged sync (#440)
* Util to zip and unzip directories
* First pass at sumtree request/response. Add message types, implement the exchange in the protocol, zip up the sumtree directory and stream the file over, with necessary adapter hooks.
* Implement the sumtree archive receive logicGets the sumtree archive data stream from the network and write it to a file. Unzip the file, place it at the right spot and reconstruct the sumtree data structure, rewinding where to the right spot.
* Sumtree hash structure validation
* Simplify sumtree backend buffering logic. The backend for a sumtree has to implement some in-memory buffering logic to provide a commit/rollback interface. The backend itself is an aggregate of 3 underlying storages (an append only file, a remove log and a skip list). The buffering was previously implemented both by the backend and some of the underlying storages. Now pushing back all buffering logic to the storages to keep the backend simpler.
* Add kernel append only store file to sumtrees. The chain sumtrees structure now also saves all kernels to a dedicated file. As that storage is implemented by the append only file wrapper, it's also rewind-aware.
* Full state validation. Checks that:
    - MMRs are sane (hash and sum each node)
    - Tree roots match the corresponding header
    - Kernel signatures are valid
    - Sum of all kernel excesses equals the sum of UTXO commitments
minus the supply
* Fast sync handoff to body sync. Once the fast-sync state is fully setup, get bacj in body sync
mode to get the full bodies of the last blocks we're missing.
* First fully working fast sync
* Facility in p2p conn to deal with attachments (raw binary after message).
* Re-introduced sumtree send and receive message handling using the above.
* Fixed test and finished updating all required db state after sumtree validation.
* Massaged a little bit the pipeline orphan check to still work after the new sumtrees have been setup.
* Various cleanup. Consolidated fast sync and full sync into a single function as they're very similar. Proper conditions to trigger a sumtree request and some checks on receiving it.
2018-02-09 22:32:16 +00:00
Antioch Peverell
78a6447a09
fix bad merge - missing support for header msg type (#690) 2018-02-07 11:26:52 -05:00