Priority: TODO:

Research testeth on threads boost test does not suuport multithreading

enhancment disable random test at macOS9 build try fix it. no way to reproduce or log the error
bug add -a option to RPC tests in test repo. https://github.com/ethereum/cpp-ethereum/issues/4582 it does not help
maybe make expect sections for Byzantine in genStateTests
maybe add docs on Blockchain test fillers
maybe review revert tests. classify scenarious
maybe more comments and descriptions of functions in the codebase CPP Progress
maybe optimize changeLog copy operations in stateTests for --statediff (dont copy the vector in functions while return if --statediff not set) https://github.com/ethereum/cpp-ethereum/issues/4050
maybe add parity to fuzztesting
someday (Felix) add transaction RLP to state tests
someday Add RPC test asking for non-existing transaction receipt. RPC tests research. why RPC tests failing on Azure? https://github.com/ethereum/tests/issues/377
someday fix gasUsed https://github.com/ethereum/tests/blob/develop/BlockchainTests/bcValidBlockTest.json#L711 : a gasUsed: "0x64" in genesis . Is that the intent of the test?
someday make testeth to correct wrong boost cases names (print suggestions). enhancment
someday Limit memory usage on tests to 1 GB, move other tests to --memory flag ask if anyone still have issues with that
someday migrate from json_spirit to faster json engine. research if it's faster first

beforeConstantinople zero Transaction + Create Transaction + Create2 tests

beforeMetropolis check callcode oog on initial fee

testcase more blockgas limit tests around minimum gaslimit https://github.com/ethereum/cpp-ethereum/issues/4515#issuecomment-330246206
Research libFuzz path coverage with fuzzed tests
Yooichi enhance tests for zeroKnowledge. Add odd input like for modexp
Yooichi makeAllFieldsHex might be not needed if we create filled file in a new mjson object instead replacing in the original mjson.
Yooichi testeth check that all fields are hex. the format. (hashes and addreses no 0x prefix) (min value 0x00) (addresses as keys no 0x prefix) (no int fields) no comment fields. all values are string
enhancment make genesis config to load accounts code from external file

testcase all possible situations with precompiled (see in crossTests) make yellow paper
guido integrade libfuzer corpus files into random tests generator
testcase check that returndatabuffer is empty when create fails with RETURN, STOP, REVERT, SELFDESTRURCT
soon add transaction test with chainid > 110 depends on make general transaction tests

Yoichi not setting returndata buffer after `REVERT`

Jared add unit test with all valid opcodes as LLLC being checked that are compiled correctly to Travis https://github.com/ethereum/cpp-ethereum/issues/4519
bug JUMPDEST in LLL might be broken. need to rewrite those tests. connsider using Julya + yaml sources
testcase `codesize` works when deploying, but `extcodesize` does not. check that it is true Bell: -9000r
enhancment genesis block larger thatn 0 in blockchain tests Pksa: 670 STRJ
slow updates YP review and better documentation for evm https://github.com/winsvega/yellowpaper/blob/master/evm.rst Ann: 14 + 30 eth | 20 etc
research lllc alternatives ? test sources in yml files and julya syntax testeth should support lllc / wasm / julya compilation
afterMetropolis copy VMTests into GeneralStateTests or convert VMTests into hive tests (better) EVM Manual
afterMetropolis review the blockchain tests that dropp invalid transactions before including it into block https://workflowy.com/s/Pnm.ca8tPgXP3c#/61425dc12501
testcase https://github.com/ethereum/tests/issues/380 add new testcase group https://github.com/winsvega/yellowpaper/blob/master/evm.rst
research experiment with swarm + mist + smart contract + ens

0 0

done add more tests where function arguments are 0 but not const. (chfast request) https://github.com/ethereum/tests/pull/389 https://github.com/ethereum/cpp-ethereum/pull/4744
January 2018
done support wasm in testeth https://github.com/ethereum/cpp-ethereum/pull/4754 January 2018
>> add wasm test suite to testeth https://github.com/ethereum/cpp-ethereum/pull/4764 January 2018
done better testeth error when json is malformed https://github.com/ethereum/cpp-ethereum/pull/4794 January 2018
done testeth support PR review 4754, 4783, 4779 January 2018
enhancment reduce the amount of tests which are not useful January 2018
>> move makeBlockchainTests out of state tests into separate function https://github.com/ethereum/cpp-ethereum/pull/4777 January 2018
>> replace networks vector with set in testeth https://github.com/ethereum/cpp-ethereum/pull/4801 January 2018
>> expand expect section network field https://github.com/ethereum/cpp-ethereum/pull/4804 January 2018
>> testeth should fill tests only on networks that set in expect section https://github.com/ethereum/cpp-ethereum/pull/4809 January 2018
>> optimize and update the tests https://github.com/ethereum/tests/pull/413 January 2018
now debug macos random state test issue https://github.com/ethereum/cpp-ethereum/pull/4787 January 2018
>> store randomcode seed on macos https://github.com/ethereum/cpp-ethereum/pull/4789 January 2018

now review cpp-ethereum testeth issues

>> testeth should error when data or code not prefixed by 0x (branch testethdata0x) https://github.com/ethereum/cpp-ethereum/issues/4287 wait untill test optimized January 2018
>> testeth should error when test filename is too long https://github.com/ethereum/cpp-ethereum/issues/4090 wait untill test optimized January 2018

optimized bc Tests stCallCodes

stCallCreateCallCodeTest

issue https://github.com/ethereum/cpp-ethereum/pull/4683 debug December 2017

done fix remaining issues, finish and merge pull requests December 2017
done read .yaml test sources December 2017
done fix travis '.' output when using testeth debug options https://github.com/ethereum/cpp-ethereum/pull/4696 December 2017
done ethvm/ethvm --network Constantinople --code 67FFFFFFFFFFFFFFFF600160006000FB https://github.com/ethereum/tests/pull/386 December 2017
done @winsvega I've found a bug in EVMJIT -- incorrectly putting CREATE returned address in RETURNBUFFER. https://github.com/ethereum/tests/pull/375/files#diff-851d485e8819932e5b34a7a273b632c0R35 December 2017
Can you check there is a opposite case somewhere in tests: when CREATE fails and returns error message and this message should be placed in RETURNBUFFER.
done testeth should trow an error if code field in compiled test is empty while not being empty in source file December 2017
done convert testeth sources with clang format cancelled by Pawel December 2017
done create random test refactoring https://github.com/ethereum/cpp-ethereum/pull/4724 December 2017
done replace lll code in transaction data https://github.com/ethereum/cpp-ethereum/pull/4735 December 2017

done create random test from seed option November 2017

done fix test name check when using --singletest November 2017
done https://github.com/ethereum/tests/blob/develop/src/GeneralStateTestsFiller/stStaticCall/static_CALL_ZeroVCallSuicideFiller.json
check the return value of staticcall https://github.com/ethereum/tests/blob/develop/src/GeneralStateTestsFiller/stStaticCall/static_CALL_OneVCallSuicideFiller.json
November 2017
done make createRandomTest options loadable from config file basicOptions + refactoring + more options November 2017
done run --singletest <pathToFile> a test withoug filler refactoring November 2017
done make general transaction tests rebase November 2017
done testeth should throw a warning id -d -g -v not found a transaction in compiled test merge November 2017
done add options to createRandomTest November 2017

done if there is a test but there is no filler for that test. testeth should throw a warning. check filler hash in the final test October 2017
done Fix test fillers October 2017
done check that test .json is update with Filler.json compare hash of the filler October 2017
done "_info" : { "comment" : "Simple test", from source test. should go to the filled test October 2017
done add TMPDIR=/dev/shm to runalltests script October 2017
done split large test suitest into smaller ones for better in threads execution October 2017
done add modexp with fuzzed input discovered by guido fuzztool October 2017
done add a test with bytecode 6001600160000360003e00 October 2017
done check that transaction arrays are not empty in testeth October 2017
done refactor fuzzTesting code in cpp-ethereum October 2017
done make determenate randomCode generator (from seed) October 2017
done create with huge value in staticcall https://github.com/ethereum/tests/pull/365/files#diff-577295cbb9de725d230ee76a0e099ae3R67 October 2017
done move zeroSigTransaction tests into a pending PR in tests repo October 2017

done split Transaction and VMtests into separate files September 2017
done rebuild testeth docker September 2017
done tests support: PR review, discussions and calls. testeth refactoring, keep hive tests updated September 2017
done add --version option for testeth September 2017
done enhance tests for zeroKnowledge we could easily add gas below 100000 and slightly above 100000 and above 180000 to existing general state tests. September 2017
done update precompiled gas prices, regenerate hive tests (2d) September 2017
done add Transaction test for V value range 1f....0xffff. add crashing transaction from issue https://github.com/ethereum/tests/issues/143 September 2017
done separate bench tests under --all flag September 2017
done fix total difficulty leafs test. check that cpp throws when address collision September 2017
done fix ctest cmake to parse all suites with any depth September 2017
done more extcodecopy tests September 2017
done review Issues in tests repo. resolve whats possible September 2017
done make difficulty tests for Byzantium as .json files September 2017
done optimize testeth + "/" + filepath construction to boost crossplatform "/" September 2017
done fix createRandomTest no output issue, make random code options constant September 2017
done refactor executeTests function in testeth September 2017

Jared add tests for all opcodes in init code that are valid September 2017
Yooichi the test case would contain two pairs (g1a, g2a), (g1b, g2b) September 2017
Yoichi Throw an exception if collision happens createJS_ExampleContract September 2017
Jared add test where SSTORE in init code September 2017
Yooichi make getTestPath to return boost::path September 2017

done --jsontrace testeth option August 2017

done disable Frontier and Homestead tests run (move it under --perfomance --all flag) August 2017
done update hive tests for Byzantium and Constantinople August 2017
done fix tests for Metropolis -> Byzantium and Constantinople split August 2017
done remove checkstate option (enable it by default) August 2017
done update testeth documentation to cpp-ethereum/docs August 2017
done clean up // comment fields from the tests on filling stage August 2017
done setup a test server for random and swarm tests August 2017
done make a random test generation option for the testeth August 2017
done logs field in general state tests with a hash + remove old fields from state test fillers (random tests: logs, out) August 2017
done setup fuzzserver and configure geth and testeth to run random tests August 2017
done add randomTest642 a call to 007 precompile issue fails on geth August 2017
done fuzz testing using testeth. compare results with geth August 2017
done tests support: PR review, discussions and calls. testeth refactoring August 2017
Youichi fix hive tests August 2017

done geneal blockchain tests July 2017

difficulty does not affect the blockchain tests. reduced mining threads to 1.
done make faster blockchain mining generation still lots of exec time spent on starting and stopping the mining thread July 2017
done testeth stop execution if unknown network is specified for --singlenet July 2017
blockchain test network now defined in "network" section in .json files
done add implicit field specifiyng the network for blockhain tests TransitionNetTest not affected. July 2017
done remove old state test code July 2017
done check that sum of gas (preState) == sum of gas (postState) {State Tests} July 2017
done upgrade TransitionNetTest to Frontier->Homestead, Homestead->EIP150, EIP150->EIP158, EIP158->Metropolis or maybe general transition test? July 2017
done fix test fillers with transaction creation by old rules on Metropolis July 2017
done fetching up tests with latest develop (regenerate Hive tests) July 2017
done add a test run of a blockchain tests generated from state tests (hive tests to develop) (!will double the test time) July 2017

done (Yoichi) testeth throw if gen state test has multiple json object in one file. Jun 2017
done limit gasLimit to reasonable value. Vitalik suggested 250M gas cap for the tests (stBounds) Jun 2017
done (Yoichi) check odd length of hex fields when filling the tests. fix pairing test missed 0 Jun 2017
done move Blockhash tests from stateGeneral to Blockchain tests Jun 2017
done do not touch other test files when using --singletest with state tests Jun 2017
done (Yoichi) add vitalik fillers for modexp https://github.com/ethereum/pyethereum/tree/state_revamp/precompile_tests_temp Jun 2017
done (Yoichi) reveiw RevertDepthCreateAddressCollisionFiller https://github.com/ethereum/cpp-ethereum/issues/4130 Jun 2017
done add information field to the final test file (filled with client: cpp-ethereum. commit version) Jun 2017
done (Yann) fix eip86Create PR failing on frontier tests Jun 2017
done merge EIP86 PR to develop Jun 2017
done revive random code option in testeth Jun 2017
done more staticcall tests Jun 2017
done check all blockchain tests on Metropolis Jun 2017
Tests: difficulty tests
https://github.com/ethereum/tests/blob/develop/BlockchainTests/TestNetwork/bcMetropolis.json https://github.com/ethereum/tests/tree/develop/BasicTests
more tests once Blockchain tests will be checked against Metropolis rules

Difficulty calculation tests

Parent block has uncles
Parent block has no uncles
Difficulty of transition to Metropolis block
Large timestamp difference with parent block - check that difficulty change is limited
Parent block has two uncles
ucle difficulty. uncle from before the transition point
special difficulty tests in blockchain form https://github.com/ethereum/tests/issues/318

add block gasLimit tests around minimum block gaslimit (needed to test KEVM, which does not specify the min gas limit)
Assignment Cases
Yoichi StaticCall -> CREATE2
StaticCall -> returnDataCopy
...

add suicide in init code to a contract. a) transaction sending founds to "" suicide occures to account x. check that no account been created and that x has balance that transaction was sending to "" (create)
b) transaction not sending funds, suicide occure in init code. no contract being created becase of the suicide. account x unchanged.
c) account x is empty account in pre state. transaction not sending funds to "". in init code suicide occures to account x. no contract being created because of the suicide. account x is cleaned because it was touched by suicide from init code

staticcall -> A | call -> A | delegatecall -> A | callcaode -> A | means successfullty finish the call and return to the callee to continue the execution

for each precompile

transaction call to | a call to | a transaction correct data that runs oog | a call that then leads to oog | a Create to
1) non exist in state precompile
2) empty in state precompile
3) precompile that has only nonce = 1
4) precompile that has only balance = 1
5) precompile that has working code in state

div(timestamp, something-zero) but 'something-zero' is MSIZE or CALLDATALOAD

New! STATICCALL->CREATE(with big value) https://github.com/ethereum/tests/pull/365/files#diff-577295cbb9de725d230ee76a0e099ae3R67

STATICCALL->CALL(with only two stack elements) https://github.com/ethereum/tests/pull/365/files#diff-577295cbb9de725d230ee76a0e099ae3R67
 Zero Knowledge Precompiled tests

Source https://github.com/ethereum/tests/tree/develop/src/GeneralStateTestsFiller/stZeroKnowledge
Tests
as Blockchain https://github.com/ethereum/tests/tree/snark_test_filled/BlockchainTests/GeneralStateTests/stZeroKnowledge
as GeneralState https://github.com/ethereum/tests/tree/snark_test_filled/GeneralStateTests/stZeroKnowledge

Cases:
Yoichi uncovered scenario https://github.com/ethereum/tests/issues/314
enhance tests for zeroKnowledge. Add odd input like for modexp "odd input"? what is meant here?
Truncated input (ECADD, first point) that results in a valid curve
https://github.com/ethereum/EIPs/pull/213#discussion_r151803836
point.
Truncated input (ECADD, second point) that results in a valid curve
https://github.com/ethereum/EIPs/pull/213#discussion_r151803836
point.
Truncated input (ECMUL) that results in a valid curve point. https://github.com/ethereum/EIPs/pull/213#discussion_r151803836

ZEC tests:
Paring tests (based on zec transaction) and chriseth tests https://github.com/ethereum/cpp-ethereum/blob/58607943f19bccae2251a1bd971f66fb59fcc0cf/test/libdevcrypto/LibSnark.cpp
[0] proof.A + vk.A + negateG1(proof.Ap) + P2,
[1] vk.B + proof.B + negateG1(proof.Bp) + P2,
[2] proof.C + vk.C + negateG1(proof.Cp) + P2,
[3] proof.K + vk.gamma + negateG1(addG1(vkx, addG1(proof.A, proof.C))) + vk.gammaBeta2 + negateG1(vk.gammaBeta1) + proof.B,
[4] addG1(vkx, proof.A) + proof.B + negateG1(proof.H) + vk.Z +negateG1(proof.C) + P2,
[5] proof.A + vk.A + proof.Ap + P2
lots of test cases from Vitalik python client

with different values of x and -1

(1,2) + (1,2), [0] (x,y) + (x,y) == (x,y) * 2, (x,y) + (x,y) == (x,y) * 2,
(0,0) + (0,0), [1] (x,y) + (x,y) + (x,y) == (x,y) * 3, (x,y) + (x,y) + (x,y) == (x,y) * 3,
(x,x) + (invalid), [2] (-x, -y) + (x,y) == (x,y) * 0, (-x, -y) + (x,y) == (x,y) * 0,
(x,x) + (0,0), [3] (-x, -y) + (0, 0) == (x, y) * -1, (-x, -y) + (-x, -y) == (-x,-y) * 2,
(0,0) + (x,x), [4] (-x, -y) + (-x, -y) == (x, y) * -2, (-x, -y) + (0, 0) == (x, y) * -1,
(invalid) + (x,x), [5] (x, y) + (0, 0) == (x, y) * 1, (-x, -y) + (-x, -y) == (x, y) * -2,
(invalid) + (invalid), [6] (-x, -y) + (0, 0) == (x, y) * -1, (x, y) + (0, 0) == (x, y) * 1,
(x,x) + (x,x), [7] (-x, -y) + (-x, -y) == (x, y) * -2, (1, 2) + (1, 2) == (1, 2) * -1,
(x,x) + (y,y), [8] (-x, -y) + (0, 0) == (x, y) * -1, (1, 2) + (-1, -2) == (1, 2) * 0,
(x,x) + (1,2) [9] (-x, -y) + (-x, -y) == (x, y) * -2,
[10] (x,y) + (x,y) == (x,y) * 2,
(invalid) is point not from the curve [11] (x,y) + (x,y) + (x,y) == (x,y) * 3,
(x,x) is point X from the curve (x1, x2) actually [12] (-x, -y) + (x,y) == (x,y) * 0,
[13] (-x, -y) + (-x, -y) == (-x,-y) * 2,
[14] (-x, -y) + (0, 0) == (x, y) * -1,
[15] (-x, -y) + (-x, -y) == (x, y) * -2,
[16] (x, y) + (0, 0) == (x, y) * 1,
[17] (-x, -y) + (x,y) == (x,y) * 0,
[18] (-x, -y) + (-x, -y) == (-x,-y) * 2,
[19] (-x, -y) + (0, 0) == (x, y) * -1,
[20] (-x, -y) + (-x, -y) == (x, y) * -2,
[21] (-x, -y) + (x,y) == (x,y) * 0,
[22] (-x, -y) + (-x, -y) == (-x,-y) * 2,
[23] (-x, -y) + (0, 0) == (x, y) * -1,
[24] (-x, -y) + (-x, -y) == (x, y) * -2,
[25] (x,y) + (x,y) + (x,y) == (x,y) * 3,
[26] (-x, -y) + (x,y) == (x,y) * 0,
[27] (-x, -y) + (0, 0) == (x, y) * -1,
[28] (-x, -y) + (-x, -y) == (x, y) * -2,
[29] (x, y) + (0, 0) == (x, y) * 1,
[30] (-x, -y) + (x,y) == (x,y) * 0,
[31] (-x, -y) + (-x, -y) == (-x,-y) * 2,
 [32] (-x, -y) + (0, 0) == (x, y) * -1,
[33] (-x, -y) + (-x, -y) == (x, y) * -2,
[34] (-x, -y) + (x,y) == (x,y) * 0,
[35] (-x, -y) + (-x, -y) == (-x,-y) * 2,
[36] (-x, -y) + (0, 0) == (x, y) * -1,
[37] (-x, -y) + (-x, -y) == (x, y) * -2

when the input has zero length, it is interpreted as (0,0) + (0,0)

when the input has zero length, it is interpreted as 0 \times (0, 0)
assigned
Yoichi RETURNDATACOPY at the beginning of a program should throw exception if offset is not zero https://github.com/ethereum/tests/pull/181
Yoichi RETURNDATASIZE at the beginning of a program should return 0 https://github.com/ethereum/tests/pull/174
Yoichi RETURNDATACOPYing returndata[0-96) should copy the returndata from a previous successful CALL https://github.com/ethereum/tests/pull/174
Yoichi RETURNDATACOPYing with start = 2^{256} - 4 and size = 100 should throw https://github.com/ethereum/tests/pull/181
Yoichi RETURNDATASIZE after a successful CREATE should return 0 https://github.com/ethereum/tests/pull/174
Yoichi RETURNDATACOPY at the beginning of a program should throw exception of size is not zero https://github.com/ethereum/tests/pull/181
Yoichi RETURNDATACOPY at the beginning of a program should not throw if size=offset= zero https://github.com/ethereum/tests/pull/194
Yoichi RETURNDATACOPY after a successful CREATE should throw exception if size is not zero returndatacopy_following_create
Yoichi RETURNDATACOPY after a successful CREATE should not throw if size = offset = zero https://github.com/ethereum/tests/pull/196
Yoichi RETURNDATACOPY after a successful CREATE should throw exception if offset is not zero https://github.com/ethereum/tests/pull/196
Yoichi RETURNDATACOPY after a failing CALL should throw exception if offset is not zero https://github.com/ethereum/tests/pull/200
Yoichi RETURNDATACOPY at the beginning of a program should throw an exception if offset + size = 2^64 https://github.com/ethereum/tests/pull/202
Yoichi RETURNDATACOPY at the beginning of a program should throw an exception if offset + size = 2^256 https://github.com/ethereum/tests/pull/208
Jared RETURNDATASIZE after successful CALLCODE https://github.com/ethereum/tests/pull/290/files
Jared RETURNDATASIZE after successful STATICCALL https://github.com/ethereum/tests/pull/283
Jared RETURNDATASIZE after successful DELEGATECALL https://github.com/ethereum/tests/pull/283
Jared RETURNDATACOPY after successful CALLCODE https://github.com/ethereum/tests/pull/283
Jared RETURNDATACOPY after successful STATICCALL https://github.com/ethereum/tests/pull/290/files
Jared RETURNDATACOPY after successful DELEGATECALL https://github.com/ethereum/tests/pull/283
Jared RETURNDATASIZE after failing CALLCODE https://github.com/ethereum/tests/pull/290/files
Jared RETURNDATASIZE after failing STATICCALL https://github.com/ethereum/tests/pull/290/files
Jared RETURNDATASIZE after failing DELEGATECALL https://github.com/ethereum/tests/pull/290/files
Jared RETURNDATACOPY after failing CALLCODE https://github.com/ethereum/tests/pull/290/files
Jared RETURNDATACOPY after failing STATICCALL https://github.com/ethereum/tests/pull/290/files
Jared RETURNDATACOPY after failing DELEGATECALL https://github.com/ethereum/tests/pull/290/files
Yoichi RETURNDATACOPY after a failing CALL (due to insufficient balance) should throw exception if offset is not zero https://github.com/ethereum/tests/pull/343#pullrequestreview-65816906
Yoichi RETURNDATACOPY after a failing CALL (because of REVERT) should throw exception if offset is not zero <- the expectation is wrong, and the scenario is already tested
Yoichi transaction calls A (CALL B(CALL C(RETURN) OOG) 'check buffers') https://github.com/ethereum/tests/pull/356
https://github.com/ethereum/tests/pull/343#discussion_r141633982

Dimitry RETURNDATASIZE after a failing CALL (due to insufficient balance) should return 0 https://github.com/ethereum/tests/pull/360

have a call to e.g. identity built-in that specifies output size less than input size. Check that all of the output data is still available with RETURNDATA,
Yoichi a successful call to modexp with modsize = 0 followed by returndatasize https://github.com/ethereum/tests/pull/363
Yoichi a successful call to modexp with modsize = 1 followed by returndatasize https://github.com/ethereum/tests/pull/366
a successful call to modexp with modsize = 100 followed by returndatasize already in
a successful call to modexp with modsize = 256 followed by returndatasize already in
a successful call to modexp with modsize = 999188 followed by returndatasize https://github.com/ethereum/tests/pull/405

already in Consensus bug in Parity: RETURNDATASIZE from a precompile returns non-zero size even if nothing is returned/written. https://gitter.im/ethereum/fuzztest?at=59d61b3abac826f0543a2f54
src/GeneralStateTestsFiller/stReturnDataTest/call_ecrec_success_empty_then_returndatasizeFiller.json
Dimitry October 11: Consensus bug in Python, RETURNDATACOPY erroneously expands memory beyond the size argument. Fix: ethereum/pyethereum#810.
https://github.com/ethereum/tests/pull/365
Dimitry October 12: Consensus bug in Parity: how RETURNDATACOPY is handled. Basically, returndatacopy(0,-1) sets one byte in memory, even if returndata buffer is empty. Overflow during addition of offset + length. Fix: paritytech/parity#6718.
https://github.com/ethereum/tests/pull/355/files#diff-3c07d7cde60de1e1669c28fdc42c9b0dR4
Dimitry RETURNDATACOPY after failing create (due to 0xfd code) https://github.com/ethereum/tests/pull/361
Dimitry RETURNDATASIZE after failing create (due to 0xfd code) https://github.com/ethereum/tests/pull/361
Dimitry RETURNDATACOPY/RETURNDATASIZE after failing CALL/DELEGATECALL/CREATE/... that has used REVERT with non-empty data.
split below
The REVERT data should be in the RETURNDATA buffer after the call.
Yoichi RETURNDATASIZE/RETURNDATACOPY after CALL that fails for a REVERT with nonempty data RETURNDATASIZE:
RETURNDATACOPY:
https://github.com/ethereum/tests/blob/develop/src/GeneralStateTestsFiller/stRevertTest/RevertOpcodeInCallsOnNonEmptyReturnDataFiller.json
https://github.com/ethereum/tests/blob/develop/src/GeneralStateTestsFiller/stReturnDataTest/returndatacopy_following_revertFiller.json
Yoichi RETURNDATASIZE/RETURNDATACOPY after CREATE that fails for a REVERT with nonempty data RETURNDATASIZE
https://github.com/ethereum/tests/pull/375
is done in RevertOpcodeInCreateReturns
Yoichi RETURNDATASIZE/RETURNDATACOPY after STATICCALL that fails for a REVERT with nonempty data https://github.com/ethereum/tests/pull/379
https://github.com/ethereum/tests/pull/378
Yoichi RETURNDATASIZE/RETURNDATACOPY after DELEGATECALL that fails for a REVERT with nonempty data https://github.com/ethereum/tests/pull/384
Yoichi RETURNDATASIZE/RETURNDATACOPY after CALLCODE that fails for a REVERT with nonempty data https://github.com/ethereum/tests/pull/387
Yoichi RETURNDATASIZE/RETURNDATACOPY (in init code) after CREATE (from init code) that fails for a REVERT with nonempty data https://github.com/ethereum/tests/pull/405
 Revert Tests

Source https://github.com/ethereum/tests/tree/develop/src/GeneralStateTestsFiller/stRevertTest/RevertOpcode*
Tests https://github.com/ethereum/tests/commit/05bffb4d7d6c2a5d6ee763693cf1c6a8d684b5ae

Cases:
check that state changes are reverted
*state reverted only inside call. for instance if revert happen in subcall only subcall changes would be reverted
call -> callA-> revert | callB -> revert | some code * and OOG scenarious
check revert in contract creation code
check that sender nonce is incremented even after revert
check sender nonce after multiple reverts + and some OOG scenario
check that sender nonce is incremented even after revert inside CREATE/INIT call
check that no instructions after REVERT are executed
revert + callcode + call + delegatecall scenarious * different depth levels
check that output data is really returned
check that output data is accessible with RETURNDATACOPY https://github.com/ethereum/tests/issues/188
check that the remaining gas is not consumed after revert
revert (bignumber, 0) (1, 0) (0, 0) (0, bignumber) (0, 32)
revert in contract creation mode should not deploy code RevertOpcodeInInitFiller.json
Yoichi revert in contract creation mode, should leave data visible by RETURNDATACOPY, RETURNDATASIZE https://github.com/ethereum/tests/pull/192
Yoichi revert in CALLCODE https://github.com/ethereum/tests/pull/311
Yoichi revert in DELEGATECALL https://github.com/ethereum/tests/pull/326
Yoichi revert in STATICCALL https://github.com/ethereum/tests/pull/333
Yoichi status code after a call transaction REVERTs https://github.com/ethereum/tests/pull/302
Yoichi status code after contract creation REVERTS https://github.com/ethereum/tests/pull/303
status code after transaction -- CALL --> internal call -- REVERT <-- STOP already done
Yoichi non-empty returndata buffer --CALL --> REVERT <-- RETURNDATASIZE, to detect "not setting returndata buffer after REVERT"
https://github.com/ethereum/tests/pull/338
Call CREATE. Immediately in the new context, do a REVERT with returndata >24576 (larger than max code size). This should be executed with sufficient gas to not OOG here. Then finished. Expe
Yoichi https://github.com/ethereum/tests/pull/331
collision on an accout with no code but nonzero nonce https://github.com/ethereum/tests/pull/327#pullrequestreview-63394766

dfg
gfd
dfg
sdg
sdfg
 Test
https://github.com/ethereum/tests/blob/develop/GeneralStateTests/stPreCompiledContracts/modexp.json

Modexp precompiled tests

Zero base
Zero exp
Zero mod
Zero base & exp & mod
Zero-length base - interpreted as zero
Zero-length exp - interpreted as zero
Zero-length mod - interpreted as zero
Non-zero length mod, but passed calldata has not enough total length (check implicit zero-padding)
Non-zero length mod, but passed data has no mod (interpreted as zero mod)
Non-zero length exp & mod, but passed data has only base and part of exp
Non-zero length exp & mod, but passed data has only base
Non-zero length base & exp & mod and no values in input data - interpreted as all zeroes

Price calculation when baseLength > modLength

Price calculation when baseLength < modLength
Price calculation when exp length = 0
bigint_expmod precompiled contract returns 1 for 0 ^ 0 mod 100
bigint_expmod precompiled contract should return zero when modulo is zero
Yoichi Price calculation when exp length > 256, and the first word of exponent is zero https://github.com/ethereum/tests/pull/210
Yoichi test baseLength = 2^64 (trying to confuze implementations using size_t) https://github.com/ethereum/tests/pull/215
Yoichi test expLength = 2^64 (trying to confuze implementations using size_t) https://github.com/ethereum/tests/pull/215
Yoichi baseLength = modLength = 0, but expLength = 2^255, should return the empty byte sequence https://github.com/ethereum/tests/pull/220
Yoichi The case found on 10.10.2017 https://github.com/ethereum/tests/pull/349
https://github.com/ethereum/tests/pull/349/files
Yoichi When the modexp input is 0x00000000000000000000000000000000000000000000000000000000000000ff2a1e5300000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
https://github.com/ethereum/tests/pull/349
Dimitry modexp input 00000000000000000000000000000000000000000000000000000000000000E300000000000000000000000000000000000000000000000000
https://github.com/ethereum/tests/pull/354
Dimitry modexp input 00000000008000000000000000000000000000000000000000000000000000000000000400000000000000000000000A https://github.com/ethereum/tests/pull/354
Dimitry October 13: Consensus big in Parity: gas calculation of modexp is subject to overflow for certain inputs. Found via manual review.https://github.com/ethereum/tests/pull/354/files
A case was found where consensus issue could be achieved in ~500K gas.
 Tests:
https://github.com/ethereum/tests/tree/staticcall/GeneralStateTests/stStaticCall
Source: https://github.com/ethereum/tests/tree/staticcall/src/GeneralStateTestsFiller/stStaticCall

Cases
Make a copy of CALL tests replacing CALL with STATICCALL (correct oog tests properly!!! OOG using state change)
DELEGATECALL during a static call should be allowed
STATICCALL during a static call should be allowed
check that staticall reurns 1 when nothing is changed in the state (double the copied tests.
add mstore instructions instead of sstore for OOG. check gas. OOG using MSTORE.
add MSTORE before and after static calls but not leading to OOG for gas usage checks)

static call -> static call success | OOG / write operation check staticcall subcall gas calculation
<delegate|call|callcode> -> static call | static call OOG | static call -> delegatecall kind of scenarious
static call operating with memory read/write operations should work as it does not touch storage
static call changing storage to the same value value->value | 0->0
staticcall -> staticcall -> call, change value, OOG | some memory calcs internal static call goes OOG (statechange) but this does not make upper staticcall changing the state. check that memory calcs after subcall staticcall consume gas and execution continues.
LOG0 in a staticcall should cause an exception
staticcall->call->LOG0 should cause an exception in the deepest call
staticcall->callcode with positive value should not succeed
staticcall-> subcall with value transfer
call -> contract and change it then staticcall to that contract and dont change it then call to contract and change it + case with OOG Revert at the end
create contract during static call different depth level
if possible. static call in transaction init code exec static call to itself. transaction with value transfer
create contract. static call to that contract. | staticcall with change | staticcall (gas) + normal call + oog (calculate gas usage)
(static call to newly created contract with and without changes)

while in staticcall on different depth put (ADDRESS ORIGIN CALLER CALLVALUE) and such codes into memory and compare the values so then go OOG if it does not match the expected values
LLL condition (if (= 1 2) (MSTORE 1 1) (SSTORE 1 2) )
if staticcall exec sstore it goes oog. if mstore then it just spend some gas. it is possible to check exact values of (ADDRESS ORIGIN CALLER CALLVALUE)
all cases with: use --vmtrace to see the exact value. test cases to check: (-> means going into a subcall) ( | means going up from subcall)
v: zero, non zero staticcall -> (check)
gas: enough,
not enough (oog at the middle of the execution) staticcall -> staticcall -> (check)
staticcall -> (call|callcode|delegatecall) -> check
staticcall -> (call|callcode|delegatecall) -> staticcall -> check
staticcall -> (check) | staticcall -> (check)
staticcall -> staticcall -> staticcall (loop). check oog scenario when OOG hapens because of the base staticcall price substracting from available gas
(call|callcode|delegatecall) -> staticcall -> check
Moved to Constantinople
Transaction tests
Transaction fields (data, gaslimit, vrs, sender revocery)
Source https://github.com/ethereum/tests/tree/zeroTransaction/src/TransactionTestsFiller/Metropolis
State Transition tests
Source https://github.com/ethereum/tests/tree/zeroTransaction/src/GeneralStateTestsFiller/stTransactionTest/zeroSigTransaction*
Tests
https://github.com/ethereum/tests/tree/develop/TransactionTests/Metropolis
https://github.com/ethereum/tests/tree/develop/GeneralStateTests/stTransactionTest

Cases
zeroSig transaction that creates a new contract ( zero / non zero gasPrice)
zeroSig transaction with zero gasPrice (value = 1 | 0)
zeroSig transaction with invChain ID
zeroSig transaction with correct nonce, but != 0 (= sender nonce)
zeroSig transaction with incorrect nonce (!= sender nonce != 0)
zeroSig transaction when sender zero account does not exist / exist and has funds
all of the above with oog scenarious
zeroSig transaction to zeroAddress
stTransactionTest/TransactionToAddressh160minusOne.json
[19:38:00] Vitalik Buterin: in this test, the to address of the tx contains a 0x prefix
[19:38:36] Vitalik Buterin: in all others it doesn't
add zeroSig transaction to zeroAddress call tests
zeroSig transaction calls a contract that looks at ORIGIN and CALLER
zeroSig transaction creates a contract with initialization code looking at ORIGIN and CALLER
When an external account (not zeroSig, creates a contract) creates a contract, the address is computed with the new formula
CREATE_P2SH increments the nonce of the creating contract
make a copy of all CREATE tests replacing create with new CREATESH
make custom CREATEP2SH tests scenarious (including OOG / Revert / diffrenet depth / double creatin of the same contract and revert)
check create collision issues
Vitalik Buterin: Here's a test we should have:
[2:21:10] Vitalik Buterin: (i) set storage key 1 of contract C to 3. (ii) self-destruct contract C. (iii) in the next transaction create contract C, and read its storage key
[2:21:19] Vitalik Buterin: the read should give 0, not 3
[2:21:29] Vitalik Buterin: it has to be a block test, because it covers cross-transaction activity
[2:22:42] Vitalik Buterin: another one is: (i) self-destruct contract C, (ii) in the next transaction, send 1 wei to C, (iii) verify that C's balance is 1 wei
[2:23:01] Vitalik Buterin: the reason this is important now is that with EIP98, clients will be switching to not "committing" their caches to the trie at the end of every tx
[2:23:18] Vitalik Buterin: and this means that selfdestruct has to be handled more carefully than before
[2:23:49] Vitalik Buterin: pyethereum screwed it up and it had to get fixed, though pyethereum's bug had to do with pre-spurious dragon "existence" so it may not be relevant now

Generic CREATE collisions, create a contract to the addres that:

- has nonzero balance
- has nonzero nonce
- has not empty storage
- has code
- is precompiled contract
- is 0 address
- has selfdestructed before in the same transactoin
- has selfdestructed before, in one of the previous transactions

A contract which performs SUICIDE, and is then attempted to be recreated (different code, same init-code) during the same transaction. This ought to fail, since the code is not cleaned out until after the transaction is ended.
A transaction with v = r = z = 0 is not valid (unless the chain_id is zero). An early version of the EIP specified this format, but was deprecated.
CREATE2 with the empty init code should fail if the destination acount has non-zero nonce

Create a block filled with several zero-sig transactions which have the same hash (are identical)
Create two subsequent blocks with the same zero-sig transaction in both blocks.

Create a contract at an account with zero-nonce, empty-code, but with some storage.
A case when zero address is a contract and while being called by another contract is creating a new contract
 Moved to Constantinople
assigned
check the nonce of the SYSTEM_ACCOUNT that performs blockhash contract maintenance, after the first block in metropolis
check the nonce of the SYSTEM_ACCOUNT that performs blockhash contract maintenance, after the second block in metropolis
BLOCKHASH contract: get the code and check if it matches the expected one.

test the gas consumption at the 256-th block in Metropolis

test the gas consumption at the 257-th block in Metropolis
test the gas consumption at the first block in Metropolis

256 blocks into Metropolis, check the return value of BLOCKHASH

New test case: EXTCODECOPY from the contract that the new BLOCKHASH uses