grin/chain/tests/data_file_integrity.rs

179 lines
4.7 KiB
Rust
Raw Normal View History

// Copyright 2018 The Grin Developers
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
use self::chain::types::NoopAdapter;
use self::chain::Chain;
use self::core::core::verifier_cache::LruVerifierCache;
use self::core::core::{Block, BlockHeader, Transaction};
use self::core::global::{self, ChainTypes};
use self::core::libtx;
use self::core::pow::{self, Difficulty};
use self::core::{consensus, genesis};
use self::keychain::{ExtKeychain, ExtKeychainPath, Keychain};
use self::util::RwLock;
use chrono::Duration;
use grin_chain as chain;
use grin_core as core;
use grin_keychain as keychain;
use grin_util as util;
use std::fs;
use std::sync::Arc;
fn clean_output_dir(dir_name: &str) {
let _ = fs::remove_dir_all(dir_name);
}
fn setup(dir_name: &str) -> Chain {
util::init_test_logger();
clean_output_dir(dir_name);
global::set_mining_mode(ChainTypes::AutomatedTesting);
let genesis_block = pow::mine_genesis_block().unwrap();
let verifier_cache = Arc::new(RwLock::new(LruVerifierCache::new()));
chain::Chain::init(
dir_name.to_string(),
Arc::new(NoopAdapter {}),
genesis_block,
pow::verify_size,
verifier_cache,
false,
)
.unwrap()
}
fn reload_chain(dir_name: &str) -> Chain {
let verifier_cache = Arc::new(RwLock::new(LruVerifierCache::new()));
chain::Chain::init(
dir_name.to_string(),
Arc::new(NoopAdapter {}),
genesis::genesis_dev(),
pow::verify_size,
verifier_cache,
false,
)
.unwrap()
}
#[test]
fn data_files() {
let chain_dir = ".grin_df";
//new block so chain references should be freed
{
let chain = setup(chain_dir);
Re-launch floonet (#2249) * Fix secondary scaling bugs; rename is_testnet -> is_floonet (#2215) * add global::is_mainnet() * use it to change pre-genesis pow type * rename is_testnet -> is_floonet * Support multiple chain configurations (#2217) * Support multiple chain configurations Supports generating the proper configuration for each chain type (mainnet, floonet, usernet). Will run them by default under their respective root directory (~/.grin/main, ~/.grin/floo, etc). Assigned default ports for mainnet, overriding them to keep Floonet ports unchanged. For now, starting on mainnet will abort. * Fixed usernet command line help message. Fixes #2217 * Differing magic numbers for each chain type (#2208) * stick to e=H(R|P|m) when use schnorr signature (#2200) * stick to e=H(R|P|m) when use schnorr signature * (1)add verify_slate_messages for wallet receive (2)log the message content * remove debug log on verify_slate_messages * verify the sender's message signature when receive_tx in wallet listen * Revert "remove debug log on verify_slate_messages" This reverts commit 65ea32a407bfd57d02bf169803f1483ba611962e. * Revert "rustfmt" This reverts commit c380ab91856344b73595bb04eef1fc087dedd84d. * Revert "(1)add verify_slate_messages for wallet receive (2)log the message content" This reverts commit 9584ca7a893b22a768dea061039140033c07e8eb. * [re-floonet] Keychain Floonet BIP32 version/network option (#2235) * add 'is_floonet' property to keychain * fix hex encoding and tests * Fix couple floonet loose ends (#2230) * Fix couple floonet loose ends. Fixes #2216 * Doc fix for sig message * Refuse unkown kernel features (#2244) * Minor: magic number change for re-floonet * Set pre genesis is_secondary to true (#2247) * Minor: tx validation error display underlying * New floonet genesis * genesis rustfmt * Use chain-specific config for wallet toml gen * Fix default wallet_listener_url * New more reasonable genesis block, bumped version * genesis rustfmt * Couple minor fixes to genesis generation script
2018-12-29 01:46:21 +03:00
let keychain = ExtKeychain::from_random_seed(false).unwrap();
for n in 1..4 {
let prev = chain.head_header().unwrap();
[1.1.0] Merge master into 1.1.0 (#2720) * cleanup legacy "3 dot" check (#2625) * Allow to peers behind NAT to get up to preferred_max connections (#2543) Allow to peers behind NAT to get up to preffered_max connections If peer has only outbound connections it's mot likely behind NAT and we should not stop it from getting more outbound connections * Reduce usage of unwrap in p2p crate (#2627) Also change store crate a bit * Simplify (and fix) output_pos cleanup during chain compaction (#2609) * expose leaf pos iterator use it for various things in txhashset when iterating over outputs * fix * cleanup * rebuild output_pos index (and clear it out first) when compacting the chain * fixup tests * refactor to match on (output, proof) tuple * add comments to compact() to explain what is going on. * get rid of some boxing around the leaf_set iterator * cleanup * [docs] Add switch commitment documentation (#2526) * remove references to no-longer existing switch commitment hash (as switch commitments were removed in ca8447f3bd49e80578770da841e5fbbac2c23cde and moved into the blinding factor of the Pedersen Commitment) * some rewording (points vs curves) and fix of small formatting issues * Add switch commitment documentation * [docs] Documents in grin repo had translated in Korean. (#2604) * Start to M/W intro translate in Korean * translate in Korean * add korean translation on intro * table_of_content.md translate in Korean. * table_of_content_KR.md finish translate in Korean, start to translate State_KR.md * add state_KR.md & commit some translation in State_KR.md * WIP stat_KR.md translation * add build_KR.md && stratum_KR.md * finish translate stratum_KR.md & table_of_content_KR.md * rename intro.KR.md to intro_KR.md * add intro_KR.md file path each language's intro.md * add Korean translation file path to stratum.md & table_of_contents.md * fix difference with grin/master * Fix TxHashSet file filter for Windows. (#2641) * Fix TxHashSet file filter for Windows. * rustfmt * Updating regexp * Adding in test case * Display the current download rate rather than the average when syncing the chain (#2633) * When syncing the chain, calculate the displayed download speed using the current rate from the most recent iteration, rather than the average download speed from the entire syncing process. * Replace the explicitly ignored variables in the pattern with an implicit ignore * remove root = true from editorconfig (#2655) * Add Medium post to intro (#2654) Spoke to @yeastplume who agreed it makes sense to add the "Grin Transactions Explained, Step-by-Step" Medium post to intro.md Open for suggestions on a better location. * add a new configure item for log_max_files (#2601) * add a new configure item for log_max_files * rustfmt * use a constant instead of multiple 32 * rustfmt * Fix the build warning of deprecated trim_right_matches (#2662) * [DOC] state.md, build.md and chain directory documents translate in Korean. (#2649) * add md files for translation. * start to translation fast-sync, code_structure. add file build_KR.md, states_KR.md * add dandelion_KR.md && simulation_KR.md for Korean translation. * add md files for translation. * start to translation fast-sync, code_structure. add file build_KR.md, states_KR.md * add dandelion_KR.md && simulation_KR.md for Korean translation. * remove some useless md files for translation. this is rearrange set up translation order. * add dot end of sentence & translate build.md in korean * remove fast-sync_KR.md * finish build_KR.md translation * finish build_KR.md translation * finish translation state_KR.md & add phrase in state.md to move other language md file * translate blocks_and_headers.md && chain_sync.md in Korean * add . in chain_sync.md , translation finished in doc/chain dir. * fix some miss typos * Api documentation fixes (#2646) * Fix the API documentation for Chain Validate (v1/chain/validate). It was documented as a POST, but it is actually a GET request, which can be seen in its handler ChainValidationHandler * Update the API V1 route list response to include the headers and merkleproof routes. Also clarify that for the chain/outputs route you must specify either byids or byheight to select outputs. * refactor(ci): reorganize CI related code (#2658) Break-down the CI related code into smaller more maintainable pieces. * Specify grin or nanogrins in API docs where applicable (#2642) * Set Content-Type in API client (#2680) * Reduce number of unwraps in chain crate (#2679) * fix: the restart of state sync doesn't work sometimes (#2687) * let check_txhashset_needed return true on abnormal case (#2684) * Reduce number of unwwaps in api crate (#2681) * Reduce number of unwwaps in api crate * Format use section * Small QoL improvements for wallet developers (#2651) * Small changes for wallet devs * Move create_nonce into Keychain trait * Replace match by map_err * Add flag to Slate to skip fee check * Fix secp dependency * Remove check_fee flag in Slate * Add Japanese edition of build.md (#2697) * catch the panic to avoid peer thread quit early (#2686) * catch the panic to avoid peer thread quit before taking the chance to ban * move catch wrapper logic down into the util crate * log the panic info * keep txhashset.rs untouched * remove a warning * [DOC] dandelion.md, simulation.md ,fast-sync.md and pruning.md documents translate in Korean. (#2678) * Show response code in API client error message (#2683) It's hard to investigate what happens when an API client error is printed out * Add some better logging for get_outputs_by_id failure states (#2705) * Switch commitment doc fixes (#2645) Fix some typos and remove the use of parentheses in a couple of places to make the reading flow a bit better. * docs: update/add new README.md badges (#2708) Replace existing badges with SVG counterparts and add a bunch of new ones. * Update intro.md (#2702) Add mention of censoring attack prevented by range proofs * use sandbox folder for txhashset validation on state sync (#2685) * use sandbox folder for txhashset validation on state sync * rustfmt * use temp directory as the sandbox instead actual db_root txhashset dir * rustfmt * move txhashset overwrite to the end of full validation * fix travis-ci test * rustfmt * fix: hashset have 2 folders including txhashset and header * rustfmt * (1)switch to rebuild_header_mmr instead of copy the sandbox header mmr (2)lock txhashset when overwriting and opening and rebuild * minor improve on sandbox_dir * add Japanese edition of state.md (#2703) * Attempt to fix broken TUI locale (#2713) Can confirm that on the same machine 1.0.2 TUI looks great and is broken on the current master. Bump of `cursive` version fixed it for me. Fixes #2676 * clean the header folder in sandbox (#2716) * forgot to clean the header folder in sandbox in #2685 * Reduce number of unwraps in servers crate (#2707) It doesn't include stratum server which is sufficiently changed in 1.1 branch and adapters, which is big enough for a separate PR. * rustfmt * change version to beta
2019-04-01 13:47:48 +03:00
let next_header_info = consensus::next_difficulty(1, chain.difficulty_iter().unwrap());
2018-10-10 12:11:01 +03:00
let pk = ExtKeychainPath::new(1, n as u32, 0, 0, 0).to_identifier();
let reward = libtx::reward::output(
&keychain,
&libtx::ProofBuilder::new(&keychain),
&pk,
0,
false,
)
.unwrap();
let mut b =
core::core::Block::new(&prev, vec![], next_header_info.clone().difficulty, reward)
.unwrap();
b.header.timestamp = prev.timestamp + Duration::seconds(60);
b.header.pow.secondary_scaling = next_header_info.secondary_scaling;
chain.set_txhashset_roots(&mut b).unwrap();
pow::pow_size(
&mut b.header,
next_header_info.difficulty,
global::proofsize(),
global::min_edge_bits(),
)
.unwrap();
2018-03-04 03:19:54 +03:00
chain
.process_block(b.clone(), chain::Options::MINE)
.unwrap();
chain.validate(false).unwrap();
}
}
// Now reload the chain, should have valid indices
{
let chain = reload_chain(chain_dir);
chain.validate(false).unwrap();
}
// Cleanup chain directory
clean_output_dir(chain_dir);
}
fn _prepare_block(kc: &ExtKeychain, prev: &BlockHeader, chain: &Chain, diff: u64) -> Block {
let mut b = _prepare_block_nosum(kc, prev, diff, vec![]);
chain.set_txhashset_roots(&mut b).unwrap();
b
}
fn _prepare_block_tx(
kc: &ExtKeychain,
2018-03-04 03:19:54 +03:00
prev: &BlockHeader,
chain: &Chain,
diff: u64,
txs: Vec<&Transaction>,
) -> Block {
let mut b = _prepare_block_nosum(kc, prev, diff, txs);
chain.set_txhashset_roots(&mut b).unwrap();
b
}
fn _prepare_fork_block(kc: &ExtKeychain, prev: &BlockHeader, chain: &Chain, diff: u64) -> Block {
let mut b = _prepare_block_nosum(kc, prev, diff, vec![]);
chain.set_txhashset_roots_forked(&mut b, prev).unwrap();
b
}
fn _prepare_fork_block_tx(
kc: &ExtKeychain,
2018-03-04 03:19:54 +03:00
prev: &BlockHeader,
chain: &Chain,
diff: u64,
txs: Vec<&Transaction>,
) -> Block {
let mut b = _prepare_block_nosum(kc, prev, diff, txs);
chain.set_txhashset_roots_forked(&mut b, prev).unwrap();
b
}
fn _prepare_block_nosum(
kc: &ExtKeychain,
2018-03-04 03:19:54 +03:00
prev: &BlockHeader,
diff: u64,
txs: Vec<&Transaction>,
) -> Block {
2018-10-10 12:11:01 +03:00
let key_id = ExtKeychainPath::new(1, diff as u32, 0, 0, 0).to_identifier();
let fees = txs.iter().map(|tx| tx.fee()).sum();
let reward =
libtx::reward::output(kc, &libtx::ProofBuilder::new(kc), &key_id, fees, false).unwrap();
Minimal Transaction Pool (#1067) * verify a tx like we verify a block (experimental) * first minimal_pool test up and running but not testing what we need to * rework tx_pool validation to use txhashset extension * minimal tx pool wired up but rough * works locally (rough statew though) delete "legacy" pool and graph code * rework the new pool into TransactionPool and Pool impls * rework pool to store pool entries with associated timer and source etc. * all_transactions * extra_txs so we can validate stempool against existing txpool * rework reconcile_block * txhashset apply_raw_tx can now rewind to a checkpoint (prev raw tx) * wip - txhashset tx tests * more flexible rewind on MMRs * add tests to cover apply_raw_txs on txhashset extension * add_to_stempool and add_to_txpool * deaggregate multi kernel tx when adding to txpoool * handle freshness in stempool handle propagation of stempool txs via dandelion monitor * patience timer and fluff if we cannot propagate to next relay * aggregate and fluff stempool is we have no relay * refactor coinbase maturity * rewrote basic tx pool tests to use a real txhashset via chain adapter * rework dandelion monitor to reflect recent discussion works locally but needs a cleanup * refactor dandelion_monitor - split out phases * more pool test coverage * remove old test code from pool (still wip) * block_building and block_reconciliation tests * tracked down chain test failure... * fix test_coinbase_maturity * dandelion_monitor now runs... * refactor dandelion config, shared across p2p and pool components * fix pool tests with new config * fix p2p tests * rework tx pool to deal with duplicate commitments (testnet2 limitation) * cleanup and address some PR feedback * add big comment about pre_tx...
2018-05-30 23:57:13 +03:00
let mut b = match core::core::Block::new(
prev,
txs.into_iter().cloned().collect(),
Difficulty::from_num(diff),
reward,
) {
2018-03-04 03:19:54 +03:00
Err(e) => panic!("{:?}", e),
Ok(b) => b,
};
b.header.timestamp = prev.timestamp + Duration::seconds(60);
b.header.pow.total_difficulty = Difficulty::from_num(diff);
b
}