Commit graph

350 commits

Author SHA1 Message Date
hashmap
a63cfa7138 Allow to have more than min_peers/2 outbound peers (#2537)
* Allow to have more than min_peers/2 outbound peers

Default value is 8, so we allow only 4 outbound peers, unfortunately if a node is behind NAT it can't get inbound peers and it stucks with 4 peers as maximum.
This PR allows a number of outbound peers to be between min_peers and max_peers/2. Node behind NAT will eventually have just min_peers number of peers.

* Extract into healthy_peers_mix function
2019-02-07 12:26:23 -08:00
Antioch Peverell
19ce9cc28e
bump for v1.0.1 (#2511) 2019-02-01 14:43:18 +00:00
Antioch Peverell
a82e2a0126
Rework "bucket transactions" logic (buckets are now weight limited) (#2487)
* rework bucket txs

* introduce max_block_weight to global (based on chain_type)
not yet in use anywhere...

* now using global::max_block_weight() everywhere

* testing max_block_weight needs to be big enough to cover existing chain tests...

* add some test coverage for prepare mineable transactions at the block weight limit
introduce AsLimitedTx{max_weight} so we can build artifically small blocks (per mining config)

* cleanup

* default_mineable_max_weight is just the default max_block_weight
we do not need to account for coinbase reward here (tx vs block)

* 75 change outputs in a test is not valid now that we have a low block weight limit...
2019-02-01 10:44:04 +00:00
Simon B
dd1a24dcbc Compiler warnings 20190123 (#2462)
* cleanup compiler warnings: unused imports

* rustfmt

* more compiler warnings
2019-01-25 10:09:32 +00:00
Antioch Peverell
ef5d83817e
Maintain min outbound peers (min_preferred_peers / 2) (#2417)
* outbound connection count

* rustfmt

* display outbound peer count

* rustfmt

* wip - allow connections to exceed max when trying to create new outcound connections
any in excess of total max will be cleaned up subsequently

* fix

* we care about connected outbound peer count when deciding to connect to new peers
2019-01-19 16:13:09 +00:00
Ignotus Peverell
e7485ab5f1
Version bump to 1.0.0 2019-01-15 16:24:29 +00:00
Ignotus Peverell
f9a20aef0d
Track blocks we requested, always broadcast otherwise (#2349) 2019-01-12 09:28:03 -08:00
Ignotus Peverell
5efcdbc077
Refuse duplicate peer IPs (#2347)
* Refuse duplicate peer IPs
* Explicitly mentioning NAT in code comment
* Well of course that would mess with tests. Only trigger using IP test when on the wild internet (~DNS seeding).
2019-01-11 15:51:38 -08:00
Gary Yu
7698b28e16
fix: wait_for_min_peers shouldn't wait forever when all peers are in same work (#2340) 2019-01-12 07:38:27 +08:00
Ignotus Peverell
1a6b46b09d
Ban peers that can't complete a handshake (#2320)
* Break out of main peer loop on error
* Force client conn shutdown on error
* Fix borrow error
* Ban peers that fail handshake
* Fix add_peer for ban, remove useless disconnect
2019-01-11 10:00:32 -08:00
Ignotus Peverell
60d3ee3c44
Bump version to 0.5.2 for release 2019-01-11 01:26:30 +00:00
Ignotus Peverell
f86c90ade5
rustfmt 2019-01-10 01:23:02 +00:00
Ignotus Peverell
6a244e904a
Fix add_peer for ban, remove useless disconnect 2019-01-10 01:22:48 +00:00
Ignotus Peverell
cf088f6f51
Ban peers that fail handshake 2019-01-09 20:31:09 +00:00
Ignotus Peverell
26d250bbea
Fix borrow error 2019-01-09 20:24:57 +00:00
Ignotus Peverell
7a44ee9aa0
Force client conn shutdown on error 2019-01-09 19:24:59 +00:00
Ignotus Peverell
684e8164f5
Break out of main peer loop on error 2019-01-09 19:14:50 +00:00
Ignotus Peverell
2c5ba853f7
Version bump to 0.5.1 2019-01-07 19:41:07 +00:00
Gary Yu
0b73e87f8c
fix: connection close should stay outside the poll loop (#2308) 2019-01-07 14:44:07 +08:00
Gary Yu
e79123fd5c
clean the unused p2p sockets (i.e. not in peers list) (#2298)
* clean the unused p2p sockets (i.e. not in peers list)

* add function comments
2019-01-07 14:41:41 +08:00
Gary Yu
6165c0873c fix: shutdown the tcpstream when we initiate to close a peer connection (#2288) 2019-01-04 09:08:44 -08:00
Gary Yu
7e8296dce0
fix: peer connect panic on log (#2272) 2019-01-02 09:54:17 +08:00
Gary Yu
23df6fa976
fix: check_banned must care about ip address only, and ignore the peer port info (#2267) 2019-01-01 11:43:05 +08:00
Yoni
972cea5fe9 more idiomatic way of picking the peer 2018-12-31 22:48:24 +02:00
Yoni
a5e5ae83b6 typo + styling 2018-12-31 20:00:33 +02:00
Yoni
b2cc5ba7e1 Enable setting a fixed dandelion peer 2018-12-31 18:49:05 +02:00
Gary Yu
738d49d560
fix: one peer occupy multiple tcp connections (#2262)
* shutdown the tcpstream when peer connect or accept fail

* remove the unnecessary 3 times retry on peer connecting

* connect/accept are actually handshakings instead of tcpstream connect/accept
2018-12-31 19:24:30 +08:00
Gary Yu
ea7eea3f84
Skip connecting PeerWithSelf by checking ip address (#2253)
* skip the connecting to self
2018-12-31 07:15:58 +08:00
Ignotus Peverell
9234e3c251
Re-launch floonet (#2249)
*  Fix secondary scaling bugs;  rename is_testnet -> is_floonet (#2215)
* add global::is_mainnet()
* use it to change pre-genesis pow type
* rename is_testnet -> is_floonet
* Support multiple chain configurations (#2217)
* Support multiple chain configurations
Supports generating the proper configuration for each chain type
(mainnet, floonet, usernet). Will run them by default under
their respective root directory (~/.grin/main, ~/.grin/floo, etc).
Assigned default ports for mainnet, overriding them to keep Floonet
ports unchanged.
For now, starting on mainnet will abort.
* Fixed usernet command line help message. Fixes #2217
* Differing magic numbers for each chain type (#2208)
* stick to e=H(R|P|m) when use schnorr signature (#2200)
* stick to e=H(R|P|m) when use schnorr signature
* (1)add verify_slate_messages for wallet receive (2)log the message content
* remove debug log on verify_slate_messages
* verify the sender's message signature when receive_tx in wallet listen
* Revert "remove debug log on verify_slate_messages"

This reverts commit 65ea32a407.

* Revert "rustfmt"

This reverts commit c380ab9185.

* Revert "(1)add verify_slate_messages for wallet receive (2)log the message content"

This reverts commit 9584ca7a89.

* [re-floonet] Keychain Floonet BIP32 version/network option (#2235)

* add 'is_floonet' property to keychain
* fix hex encoding and tests

* Fix couple floonet loose ends (#2230)

* Fix couple floonet loose ends. Fixes #2216
* Doc fix for sig message

* Refuse unkown kernel features (#2244)

* Minor: magic number change for re-floonet

* Set pre genesis is_secondary to true (#2247)

* Minor: tx validation error display underlying

* New floonet genesis

* genesis rustfmt

* Use chain-specific config for wallet toml gen

* Fix default wallet_listener_url

* New more reasonable genesis block, bumped version

* genesis rustfmt

* Couple minor fixes to genesis generation script
2018-12-28 14:46:21 -08:00
Gary Yu
c0507b2ab1 Ban a fraud peer which claims a fake height and total difficulty (#2237)
* ban a peer with highest height but can't send us the proper headers
* reset self.stalling_ts if syncing goes well
* fix: switch to peers.ban_peer() instead of peer.set_banned()
2018-12-28 14:09:09 -08:00
Gary Yu
55334caec8 fix: peers::is_banned need check peers data from database (#2243) 2018-12-28 13:54:37 -08:00
Ignotus Peverell
7284cfec72
Changed magic number and seeds for Floonet (#2188)
* Changed magic number and seeds for Floonet.
* Genesis generator now loads a local wallet seed to build coinbase.
2018-12-20 11:08:01 -08:00
Antioch Peverell
2d4538c428
Clean shutdown via stop_state (#2117)
* clean shutdown wip

* rustfmt

* introduce StopState that we can lock on

* rustfmt

* take lock on stop_state during critical processing (process_block_single etc.)

* rustfmt

* take lock on stop_state during chain::init()

* cleanup

* cleanup

* rustfmt

* docs/comments

* fixup servers tests

* cleanup p2p tests
2018-12-11 11:07:41 +00:00
hashmap
aedac483f5
Convert to Rust 2018 edition (#2084)
* Convert to Rust 2018 edition

* Update gitignore
2018-12-08 00:59:40 +01:00
hashmap
a6bb6344ac
Fix seed thread crash (#2096)
We get overflow in some tests, peraps in real life too
Fixes #2095
2018-12-07 17:09:26 +01:00
Ignotus Peverell
87bfd9aaa4
Mark all crates as publishable 2018-12-07 00:08:15 +00:00
Antioch Peverell
a3523028dc
Sync headers slice (#2085)
* pass slices of headers around, not vec of headers

* rustfmt
2018-12-05 16:50:32 +00:00
Antioch Peverell
7b07fe99a7
bump 0.4.1 -> 0.4.2 so we can watch the HF (#2057) 2018-11-30 11:20:12 +00:00
antiochp
f7608f5b9c
rustfmt 2018-11-16 14:34:05 +00:00
antiochp
4b48eebdfe
refactor some p2p to use Read/Write over TcpStream 2018-11-16 14:33:35 +00:00
Antioch Peverell
f0fa410273
Streaming headers (#1989)
* headers msg is now "streamed" off the tcp stream

* rustfmt

* cleanup

* move StreamingReader into ser.rs
extract read_exact out into util crate

* rustfmt
2018-11-16 11:00:39 +00:00
Antioch Peverell
cb652e7ef1
do not treat txhashset.zip download as abusive behavior (#1973)
* do not treat txhashset.zip download as abusive behavior

* count times, not bytes so we exclude quiet increments

* use inc_quiet when tracking sent bytes via an attachment

* add comment

* fixup "quiet" counter entries

* rustfmt
2018-11-13 21:34:45 +00:00
Antioch Peverell
01900f2e4d Enable TX_KERNEL_HASH capability by default (#1971) 2018-11-13 09:28:36 -08:00
Antioch Peverell
236fe56f60
ask for peers with PEER_LIST capability when asking for more peers (#1969)
do not simply look for peers with the same capabilities we have
2018-11-13 10:59:33 +00:00
Antioch Peverell
c631b45ab6
cleanup how we deal with flexible rproof size in ser/deser (#1965) 2018-11-13 09:30:40 +00:00
Antioch Peverell
a3c1c6d603
Do not use mem::size_of() when calculating serialized size of data (#1967)
* use FixedLength to define serialized size in bytes of various structs
replace usages of mem::size_of() with ::LEN so we correctly calculate serialized sizes in bytes

* rustfmt
2018-11-13 09:30:02 +00:00
Gary Yu
7ff1ee5fde
Peer send receive bytes statistics missed response messages and attachment (#1953)
* fix: peer send receive bytes statistics missed response message and attachment

* rustfmt
2018-11-11 07:30:57 +08:00
Gary Yu
2352275dff
Suppress some peers logs by trace level instead of debug (#1952)
* suppress some peers boring logs by trace instead of debug

* rustfmt
2018-11-11 07:30:14 +08:00
Gary Yu
9af9ca9518
refactor the state_sync to handle the long fork (#1902)
* split horizon into two explicit values for cut through and txhashset request

* let node which has 2-7 days of history be able to handle forks larger than 2 days

* add test simulate_long_fork

* add pause/resume feature on p2p for tests

* refactor the state_sync

* ignore the test case simulate_long_fork for normal Travis-CI

* refactor function check_txhashset_needed to be shared with body_sync

* fix: state TxHashsetDone should allow header sync
2018-11-10 11:27:52 +08:00
Antioch Peverell
39ebb33ba1
bump 0.4.0 -> 0.4.1 (tx kernel broadcast support) (#1950) 2018-11-09 14:57:25 +00:00
Ignotus Peverell
f7d0fe1840
Bug fix, tuning of peer message rate counting 2018-11-07 22:15:12 +00:00
Antioch Peverell
ab30f714fc
Capabilities now deserialize safely (#1946)
* deserialize capabilities dropping unknown flag bits

* add tests for capabilities deserialization

* rustfmt

* FULL_NODE needs to be handled safely for legacy nodes
2018-11-07 11:15:37 +00:00
Antioch Peverell
e25a3b9d72
Fix capabilities (#1944)
* we cannot safely update nodes to use capabilities bits = 15 ...

* rustfmt
2018-11-07 10:00:42 +00:00
Antioch Peverell
949f75c77c
fix the lock on the new mutex (#1942) 2018-11-07 09:40:39 +00:00
Antioch Peverell
d97a6c2189
[DNM] broadcast tx kernel hash (if supported by peer) (#1929)
broadcast tx kernel hash (if supported by peer)
2018-11-07 09:28:17 +00:00
Ignotus Peverell
8b546632fe
Peer rate limiting (#1933)
* Rate counter in peer conn to monitor traffic 
* Ban peers that trigger is_abusive
2018-11-06 17:51:22 -08:00
hashmap
d2088ff48c Fix recently introduced peer deadlock (#1941) 2018-11-06 16:42:36 -08:00
hashmap
8742c334dd Remove unsafe sync/send impl for Peers (#1938)
This is really unsafe. Fixes #1937

* Switch to better mutex
2018-11-06 08:59:46 -08:00
hashmap
0af1f13bf9 Fix IPV6 address deserialization (#1932)
Fuzz test found that we don't read IPV6 addr (as part of p2p message)
properly. The code is supposed to read 8 dwords, but [0..8] is not a
slice of 8 ints, but a slice of one Range, so we always read just one
dword
2018-11-05 20:38:41 -08:00
hashmap
109a426990
Add fuzz tests for p2p crate (#1931)
Add fuzz tests for p2p crate
2018-11-05 21:50:16 +01:00
hashmap
b01fcd2f56 Remove unused unsafe declarations (#1901) 2018-10-31 13:19:28 -07:00
Ignotus Peverell
46051ee174
improve minimum difficulty handling (#1791)
* rename MIN_DIFFICULTY to UNIT_DIFFICULTY; define Difficulty::unit, fix INITIAL_DIFFICULTY
* improve minimum difficulty handling
* replace all Difficulty::one by ::min
* revert secondary scaling default in HeaderInfo; rename scaling_difficulty; refactor difficulty_data_to_vector
2018-10-27 10:37:44 -07:00
Ignotus Peverell
c083312ad4
Improve crate descriptors to be more cargo-friendly (#1844)
* Update secp dependency to use crates.io version
* Add more details to various Cargo.toml
* Remove use of env variables that aren't reliably provided by cargo
2018-10-25 17:44:50 -07:00
Ignotus Peverell
a42d66efff
Merge branch 'master' into unitdiff 2018-10-25 14:20:41 -07:00
eupn
711fad6c24 Remove TODO for counting of bytes that was already done (#1838) 2018-10-25 15:57:53 +02:00
Antioch Peverell
38cbd6eafb
full node == fast sync (no full archival guarantees) (#1809)
* wip - fast sync only

* wip

* cleanup

* cleanup

* cleanup comments in default config file around capabilities

* fixup p2p tests
2018-10-23 13:01:19 +01:00
Ignotus Peverell
7f60e2076d
Default last connected to migrate older db (#1812)
Fixes backward compatibility issue with #1794
2018-10-22 18:15:10 -07:00
eupn
8f42f7306c feat: remove expired peers from the storage (#1794)
* Initial expired peers removal
* Stop expired peers
* Simplify peer removal and remove only Defunct peers
* Make seed to check for expired peers every hour
* Get rid of unused vector of peers to remove
* Make peer deletion predicate closure immutable
2018-10-22 13:59:40 -07:00
eupn
1195071f5b Replace logging backend to log4rs and add log rotation (#1789)
* Replace logging backend to flexi-logger and add log rotation
* Changed flexi_logger to log4rs
* Disable logging level filtering in Root logger
* Support different logging levels for file and stdout
* Don't log messages from modules other than Grin-related
* Fix formatting
* Place backed up compressed log copies into log file directory
* Increase default log file size to 16 MiB
* Add comment to config file on log_max_size option
2018-10-21 13:30:56 -07:00
Gary Yu
0d06561a91
replace stdlib RwLock and Mutex with parking_lot (#1793)
* replace all stdlib RwLock with parking_lot RwLock

* replace stdlib Mutex with parking_lot Mutex

* rustfmt
2018-10-20 08:13:07 +08:00
John Tromp
846b38308c replace all Difficulty::one by ::min 2018-10-18 21:18:16 +02:00
yeastplume
f94ede9af3 merge T4 into master 2018-10-18 11:23:04 +01:00
Gary Yu
53b10a083c
kick stuck peer out of connected peers. (#1782)
* cherry-picking commit 7754adb8 from master for #1746
2018-10-18 10:04:05 +08:00
Ignotus Peverell
b5cb227322
Last genesis for T4, unless I messed up something 2018-10-17 19:36:12 +00:00
eupn
b22fb55245 feat: add peers used bandwidth calculation and display in TUI (#1770)
* Add peers used bandwidth calculation and display in TUI
* Fix formatting
* Change Mutex to RwLock from peer's used bandwidth statistics in Tracker
* Make used bandwidth column in TUI peers list sort by sum of bytes
2018-10-17 10:01:42 -07:00
Yeastplume
b43d6e4326
[T4] Set genesis block data + initial secondary scaling correctly (#1776)
* ensure genesis block+pre genesis is populated correctly with secondary scaling

* rustfmt
2018-10-17 16:53:31 +01:00
Yeastplume
5cec885ef5
[T4] diff change (#1769)
* pre-t4 again

* rustfmt
2018-10-17 13:48:18 +01:00
Yeastplume
404165a8fd
[T4] Add sec pow info to TUI, change magic number, genesis diff to 1 (temporarily) (#1768)
* add sec scaling stats to tui

* rustfmt
2018-10-17 10:37:28 +01:00
hashmap
85433c659d Introduce a constant for peer send channel capacity (#1761)
We implicitly use it also in body_sync, so it's hard to keep it in sync.
2018-10-16 16:31:00 -07:00
Gary Yu
7754adb834 kick stuck peer out of connected peers (#1746)
* kick stuck peer out of connected peers
* adjust kick time to 2 hours
* stuck detection take into account own difficulty compare
2018-10-16 09:14:16 -07:00
yeastplume
4bb31dbdb4 update new genesis block, change p2p msg magic number 2018-10-16 10:16:54 +01:00
John Tromp
34646ddf51 [T4] Rename all shiftsize / cuckoo_size to edge_bits and change value for T4 (#1752)
* replace all size_shift / cuckoo_size by edge_bits and change some constants for T4
* replace remaining occurrences of sizeshift
2018-10-15 16:14:23 -07:00
yeastplume
9423865f92 update cargo versioning 2018-10-15 10:34:33 +00:00
Gary Yu
2e6a242827
small improvement on the servers test (#1737)
* cherry-pick from master for #1736
2018-10-14 20:13:49 +08:00
Gary Yu
6f4eb8dd2e
small improvement on the servers test (#1736)
* small improve on the servers test, to avoid Travis-CI failure on "job exceeded the maximum log length"

* rustfmt
2018-10-14 17:01:34 +08:00
Ignotus Peverell
00be540e0c Fix small txhashset download timeouts (#1735) 2018-10-14 15:25:17 +08:00
Ignotus Peverell
43f4f92730
[T4] Secondary proof of work difficulty adjustments (#1709)
* First pass at secondary proof of work difficulty adjustments
* Core and chain test fixes
* Next difficulty calc now needs a height. Scaling calculation fixes. Setting scaling on mined block.
* Change factor to u32 instead of u64.
* Cleanup structs used by next_difficulty
* Fix header size calc with u32 scaling
2018-10-13 13:57:01 -07:00
Gary Yu
9a716aea72
feature: txhashset downloading progress display on tui (#1729) (#1730)
(cherry picked from commit 5c0eb11a7d)
2018-10-13 10:12:13 +08:00
Gary Yu
5c0eb11a7d feature: txhashset downloading progress display on tui (#1729) 2018-10-12 15:53:50 -07:00
Gary Yu
3fb4669d0a
small speed optimization for header sync (#1728)
* fix: avoid a confusing log when fastsync start (#1720)

(cherry picked from commit 4f462cdfdc)

* small speed optimization for header sync (#1719)

(cherry picked from commit d3589d1bf5)
2018-10-12 08:44:35 +08:00
Gary Yu
d3589d1bf5
small speed optimization for header sync (#1719) 2018-10-11 16:47:27 +08:00
Antioch Peverell
8cfe9e64ac
Remove RwLock around individual peer instances (#1673)
* get rid of RwLock around peer instances

* rustfmt

* rename stuff to live_info

* rustfmt

* serialize into PeerInfoDisplay
limit live_info write lock to peer_info.update()
rename to PeerLiveInfo

* simplify broadcast logic a bit more
connected peers are connected by definition

* return true/false on broadcast so we can send to more peers

* rustfmt

* fix p2p and server tests for PeerInfoDisplay

* commit
2018-10-09 08:27:34 +01:00
Antioch Peverell
de5b6868fb
Add last_seen to peer info (#1688)
* add last_seen to peer and update on ping/pong
expose last_seen in tui

* rustfmt

* chrono serde features
2018-10-08 13:39:52 +01:00
Gary Yu
1a6101f870 refactor: remove thread p2p-moniter, move the Ping to the connect_and_monitor (#1670) 2018-10-07 10:24:06 -07:00
Gary Yu
463567b19e
optimization of peer state write lock (#1669)
* optimization of peer state write lock
2018-10-07 13:01:32 +08:00
Gary Yu
a37f7bbcab
improvement on peer lock for ping/pong (#1668)
* peer lock optimization for ping/pong
* is_banned peer lock use read lock instead of write
2018-10-07 08:00:53 +08:00
Antioch Peverell
c86578c2a9 Peers broadcast fixes (#1667)
* peers broadcast fixes
* stop at num_peers...
2018-10-05 16:53:55 -07:00
Gary Yu
697805752c fix: seed thread panicked on a try_read unwrap 2018-10-05 21:16:44 +08:00
Ignotus Peverell
85d5feafa3 Fix concurrency issue around peer add and start. Fixes #1585 (#1633)
* Fix concurrency issue around peer add and start. Fixes #1585
2018-10-02 10:17:29 -04:00
Antioch Peverell
c9c829514b
Trace get block (#1628)
* trace on GetBlock msg

* rustfmt
2018-10-01 22:12:39 +01:00