Commit graph

68 commits

Author SHA1 Message Date
Antioch Peverell
dcd405e263
cleanup deprecation warnings about missing dyn with trait objects (#2997) 2019-08-26 21:17:47 +01:00
Antioch Peverell
5bf813ee93
improve checking for p2p connection limits (revised) (#2993)
* Add check for p2p connection limits

* Simplify undesirable connection shutdown

* Make inbound and outbound connections more explicit

* Cleanup inbound and outbound connections

* Cleanup an outbound peers check

* Rename healthy_peers_mix to enough_outbound_peers

* be a lot less restrictive when picking some candidate peers to connect to
keep the peer address queue drained but actually attempt a healthy number of connections
as most of these attempts are going to fail due to majority of nodes not being publicly accessible
2019-08-21 19:58:43 +01:00
antiochp
17dddeeb0d
Revert "Improve checking for p2p connection limits (#2985)"
This reverts commit 24f0a52437.
2019-08-21 15:02:10 +01:00
j01tz
24f0a52437 Improve checking for p2p connection limits (#2985)
* Add check for p2p connection limits

* Simplify undesirable connection shutdown

* Make inbound and outbound connections more explicit

* Cleanup inbound and outbound connections

* Cleanup an outbound peers check

* Rename healthy_peers_mix to enough_outbound_peers
2019-08-21 12:43:09 +01:00
Quentin Le Sceller
4966dc04f7
[2.x.x] Generate txhashset archives on 720 block intervals. (#2951)
Generate txhashset archives on 720 block intervals.
2019-07-23 10:46:29 +02:00
hashmap
9ab23f6eef Implement graceful shutdown (#2812)
* Remove stop status mutex
* remove some deadlocks
* Rewrite stop channel handling
* fix deadlock in peers object
* add missing test fixes
2019-05-15 08:51:35 -07:00
Antioch Peverell
e56cd55980
initial support for kernel data download (#2765)
* initial support for kernel data download

* fix vec backend for tests

* cleanup after rebase
2019-05-14 17:17:38 +01:00
Antioch Peverell
4ef4212f1f
Simplify peer connection handling (#2801)
* connection no longer wrapped in an Option in peer

* introduce peer.send()

* remove some Arc indirection

* self.send() cleanup

* extract Peer:new() from connect and accept

* fixup

* cleanup
2019-05-03 15:56:25 +01:00
David Burkett
0e6a249126 Increasing thread sleeps for socket listener thread and per-connection threads. (#2773) 2019-04-23 08:29:24 +02:00
Gary Yu
97e96e4fcf use a grin specific tmp dir instead of a system tmp dir (#2753) (#2772)
* use a grin specific tmp dir instead of a system tmp dir

* rustfmt

* using create_new instead of create

* switch the txhashset sandbox from system tmp dir to grin specific tmp dir

* rustfmt

* adapter get_tmp_dir delegate to chain implementation

* simply the logic of protolc for get_tmpfile_pathname

* rustfmt

* fix the merge mistake
2019-04-22 16:54:36 -07:00
Antioch Peverell
13c6160e15
pass peer_info around rather than peer_addr (includes protocol version) (#2761) 2019-04-18 14:11:06 +01:00
Yeastplume
12fe928112
merge master into 1.1.0 2019-04-16 11:24:43 +01:00
hashmap
94732b0d58
Return Result from methods of ChainAdapter (#2722)
Most of the methods return nothing or bool which is used to decide if a
sender of a message should be banned or not. However underlying chain
implementation may fail so we need a way to reflect this fact in API.

Also it allows to reduce number of unwraps and makes the code more robust.
2019-04-08 22:13:28 +02:00
Yeastplume
beaae28d70
Dynamic LMDB mapsize allocation [1.1.0] (#2605)
* dynamically resize lmdb

* rustfmt

* explicitly close db before resizing

* rustfmt

* test fix

* rustfmt

* pool tests

* chain fix

* merge

* move RwLock into Store, ensure resize gives a min threshold

* rustfmt

* move locks based on testing

* rustfmt
2019-02-27 09:47:46 +00:00
Antioch Peverell
23cb9e2514
Map peers by ip only (ignoring port unless on loopback ip) (#2540)
* wip

* big refactor, regretting doing this now

* PeerAddr everywhere

* cleanup

* fixup server tests

* peers api working for GET, POST is still WIP

* we can now ban/unban peers by ip only (port optional)
2019-02-18 12:15:32 +00:00
Ignotus Peverell
f9a20aef0d
Track blocks we requested, always broadcast otherwise (#2349) 2019-01-12 09:28:03 -08:00
Ignotus Peverell
5efcdbc077
Refuse duplicate peer IPs (#2347)
* Refuse duplicate peer IPs
* Explicitly mentioning NAT in code comment
* Well of course that would mess with tests. Only trigger using IP test when on the wild internet (~DNS seeding).
2019-01-11 15:51:38 -08:00
Ignotus Peverell
f86c90ade5
rustfmt 2019-01-10 01:23:02 +00:00
Ignotus Peverell
6a244e904a
Fix add_peer for ban, remove useless disconnect 2019-01-10 01:22:48 +00:00
Ignotus Peverell
cf088f6f51
Ban peers that fail handshake 2019-01-09 20:31:09 +00:00
Ignotus Peverell
26d250bbea
Fix borrow error 2019-01-09 20:24:57 +00:00
Ignotus Peverell
7a44ee9aa0
Force client conn shutdown on error 2019-01-09 19:24:59 +00:00
Ignotus Peverell
684e8164f5
Break out of main peer loop on error 2019-01-09 19:14:50 +00:00
Gary Yu
e79123fd5c
clean the unused p2p sockets (i.e. not in peers list) (#2298)
* clean the unused p2p sockets (i.e. not in peers list)

* add function comments
2019-01-07 14:41:41 +08:00
Gary Yu
738d49d560
fix: one peer occupy multiple tcp connections (#2262)
* shutdown the tcpstream when peer connect or accept fail

* remove the unnecessary 3 times retry on peer connecting

* connect/accept are actually handshakings instead of tcpstream connect/accept
2018-12-31 19:24:30 +08:00
Gary Yu
ea7eea3f84
Skip connecting PeerWithSelf by checking ip address (#2253)
* skip the connecting to self
2018-12-31 07:15:58 +08:00
Antioch Peverell
2d4538c428
Clean shutdown via stop_state (#2117)
* clean shutdown wip

* rustfmt

* introduce StopState that we can lock on

* rustfmt

* take lock on stop_state during critical processing (process_block_single etc.)

* rustfmt

* take lock on stop_state during chain::init()

* cleanup

* cleanup

* rustfmt

* docs/comments

* fixup servers tests

* cleanup p2p tests
2018-12-11 11:07:41 +00:00
hashmap
aedac483f5
Convert to Rust 2018 edition (#2084)
* Convert to Rust 2018 edition

* Update gitignore
2018-12-08 00:59:40 +01:00
Antioch Peverell
a3523028dc
Sync headers slice (#2085)
* pass slices of headers around, not vec of headers

* rustfmt
2018-12-05 16:50:32 +00:00
Gary Yu
2352275dff
Suppress some peers logs by trace level instead of debug (#1952)
* suppress some peers boring logs by trace instead of debug

* rustfmt
2018-11-11 07:30:14 +08:00
Gary Yu
9af9ca9518
refactor the state_sync to handle the long fork (#1902)
* split horizon into two explicit values for cut through and txhashset request

* let node which has 2-7 days of history be able to handle forks larger than 2 days

* add test simulate_long_fork

* add pause/resume feature on p2p for tests

* refactor the state_sync

* ignore the test case simulate_long_fork for normal Travis-CI

* refactor function check_txhashset_needed to be shared with body_sync

* fix: state TxHashsetDone should allow header sync
2018-11-10 11:27:52 +08:00
Antioch Peverell
d97a6c2189
[DNM] broadcast tx kernel hash (if supported by peer) (#1929)
broadcast tx kernel hash (if supported by peer)
2018-11-07 09:28:17 +00:00
hashmap
b01fcd2f56 Remove unused unsafe declarations (#1901) 2018-10-31 13:19:28 -07:00
Ignotus Peverell
a42d66efff
Merge branch 'master' into unitdiff 2018-10-25 14:20:41 -07:00
Antioch Peverell
38cbd6eafb
full node == fast sync (no full archival guarantees) (#1809)
* wip - fast sync only

* wip

* cleanup

* cleanup

* cleanup comments in default config file around capabilities

* fixup p2p tests
2018-10-23 13:01:19 +01:00
eupn
1195071f5b Replace logging backend to log4rs and add log rotation (#1789)
* Replace logging backend to flexi-logger and add log rotation
* Changed flexi_logger to log4rs
* Disable logging level filtering in Root logger
* Support different logging levels for file and stdout
* Don't log messages from modules other than Grin-related
* Fix formatting
* Place backed up compressed log copies into log file directory
* Increase default log file size to 16 MiB
* Add comment to config file on log_max_size option
2018-10-21 13:30:56 -07:00
John Tromp
846b38308c replace all Difficulty::one by ::min 2018-10-18 21:18:16 +02:00
Gary Yu
5c0eb11a7d feature: txhashset downloading progress display on tui (#1729) 2018-10-12 15:53:50 -07:00
Antioch Peverell
8cfe9e64ac
Remove RwLock around individual peer instances (#1673)
* get rid of RwLock around peer instances

* rustfmt

* rename stuff to live_info

* rustfmt

* serialize into PeerInfoDisplay
limit live_info write lock to peer_info.update()
rename to PeerLiveInfo

* simplify broadcast logic a bit more
connected peers are connected by definition

* return true/false on broadcast so we can send to more peers

* rustfmt

* fix p2p and server tests for PeerInfoDisplay

* commit
2018-10-09 08:27:34 +01:00
Gary Yu
1a6101f870 refactor: remove thread p2p-moniter, move the Ping to the connect_and_monitor (#1670) 2018-10-07 10:24:06 -07:00
Ignotus Peverell
85d5feafa3 Fix concurrency issue around peer add and start. Fixes #1585 (#1633)
* Fix concurrency issue around peer add and start. Fixes #1585
2018-10-02 10:17:29 -04:00
Gary Yu
56ffbee7e9
fix: all fast sync nodes report to peers as full nodes (#1547)
* fix: all fast sync nodes report to peers as full nodes
2018-09-20 07:09:39 +08:00
Ignotus Peverell
9cc7aed713
Dual proof of work, first step (#1534)
* Move Proof and Difficulty types to pow module. Difficulty scaling calculation.
* Diffculty scaling and size shift for 2nd PoW
* Backport e7eb26ee
* Test compilation fixes
* Scaling only need to go one way, as @tromp pointed out
2018-09-18 15:12:57 -07:00
Gary Yu
ed73db671f fix for unstable travis-ci test on servers module (#1434)
* fix for unstable travis-ci test on servers module

* skip the PeerWithSelf connection request on sending side

* logs and comments change; and remove an unnecessary unwrap() in test
2018-09-04 10:52:11 +02:00
hashmap
631201358f Refactor and optimise peers.rs (#1389)
* Use internal hashmap for count and contains methods instead of relying on
connected_peers method which is expensive (creates vector from hashmap)
* Reduce number of `clone()`
* Refactor broadcast_xxx
2018-08-20 18:32:13 -04:00
Ignotus Peverell
d47a3bc225
Simplify and improve headers_received (#1362) 2018-08-16 22:30:05 -04:00
e-max
25e3d9e7d3 Remove Iron dependency and update hyper to version 0.12 (#1241)
* Remove Iron dependecy and update hyper to version 0.12 #876

* REMOVE ME

* Revert "REMOVE ME"

This reverts commit e9a976eee98a2d5a4dfae5d9e1e4f5ed640c05d3.

* Rebase and start updating libwallet

Libwallet doesn't compile yet.

* Wallet compiles

* Grin compiles

* No compilation errors in tests

* All tests pass

* Reeturn future from handler

* Refactoring

* Fix lifetime issue one more time

I have to force push to rollback all the work done in last 2 days

* Fix wallet send issue

* Clean up
2018-08-01 10:44:07 +01:00
Ignotus Peverell
bdfd5405c0
Leverage sync state to accept txhashset (#1251)
Fixes #1246
2018-07-12 17:06:52 +01:00
Ignotus Peverell
1a410f115c
Cleanup a bunch of build warnings 2018-07-02 00:25:20 +01:00
Ignotus Peverell
d189770080 Wallet LMDB backend (#1151)
* Migrate main node store to LMDB

In preparation to using LMDB as a wallet database, migrate the
node db. There's no point in having 2 key-value stores.
In addition LMDB provides a few advantages as a node db, namely a
much faster build (compared to RocksDb), lesser dependencies and
transactions.

* Migrated p2p store to lmdb, stuff compiles

* More fixes, chain tests starting to pass

* Fixed txhashset rollback messing with block save and general batch delimitation. Chain tests passing.

* rustfmt

* LMDB max map size of 10MB isn't really workable. Half TB seems reasonable.

* Fix wallet tests

* Rather crucial commit was missing

* rustfmt

* Fixing new merged tests following lmdb changes

* rustfmt

* * Make txhashset validation read-only on fast sync to avoid having
a really long open transaction.
* Fix deadlock in new block processing, batch should always be
created within a txhashset lock (when they interact).

* Comment about batch and txhashset interlacing

* Fix store tests to use batch

* Externalize wallet config and seed

* Converted direct read access to file outputs map to an iterator

* Cleaned up and simplified wallet Backend trait:

* No more direct mutable access to internal structures (HashMap)
* Batch interface for all writes
* Remove unneeded read wrapper (read_wallet)

* rustfmt

* First (incomplete) pass at wallet LMDB backend

* Progressing on lmdb backent iml

* Added batch impl for LMDB wallet backend. Pretty much done with it, but not sure how to deal with commit (owned).

* rustfmt

* Wrapping LMDB batch around a refcell to work around borrow rules

* Compilation up to grin chain
2018-06-22 09:08:06 +01:00