Add Ethereum state test runner for Giga executor validation

[pdrobnjak]

Summary

• Add comprehensive test runner comparing Giga (evmone) vs Geth execution using Ethereum's official GeneralStateTests
• Fix evmc.Error type conversion bug in host_context.go that caused panics on nested call errors
• Include 9MB compressed test fixtures that auto-extract on first run

Changes

New Test Infrastructure (giga/tests/state_test.go)

• TestGigaVsGeth_StateTests: Runs Ethereum state tests through both executors and compares results
• TestGigaVsGeth_StateTest_Simple: Self-contained sanity test
• Auto-extracts test fixtures from embedded .tgz archive

Bug Fix (giga/executor/internal/host_context.go)

• Added toEvmcError() to convert geth errors to evmc.Error type
• Prevents panic when nested CALL/CREATE operations return errors

Test Fixtures

• Embedded fixtures_general_state_tests.tgz (9MB, 277MB uncompressed)
• Full Ethereum GeneralStateTests coverage
• .gitignore for extracted directory

Test Results

• 127 passed, 3 failed (97.6% pass rate)
• 3 failures are cosmetic error code differences (both executors correctly reject invalid jumps)

Usage

# Tests work out of the box - no setup needed
go test -v ./giga/tests/... -run TestGigaVsGeth_StateTests

# Optional: Override with custom test path
ETHEREUM_TESTS_PATH=/path/to/tests go test -v ./giga/tests/...

Test plan

• Run TestGigaVsGeth_StateTest_Simple - sanity check passes
• Run TestGigaVsGeth_StateTests - 97.6% pass rate
• Verify auto-extraction from archive works
• Verify env var override still works

🤖 Generated with Claude Code

[pdrobnjak]

Code Review: PR #2707 - Giga Executor State Tests and Bug Fix

Summary

This PR introduces a comprehensive test framework for comparing the Giga (evmone-based) EVM executor against the standard Geth interpreter. The key components are:

1. State Test Runner (giga/tests/state_test.go) - Loads and executes Ethereum's official GeneralStateTests to validate Giga vs Geth equivalence
2. Bug Fix (giga/executor/internal/host_context.go) - Added toEvmcError() function to properly convert Go errors to EVMC error types
3. Test Fixtures - 9MB embedded .tgz archive containing Ethereum's official state tests
4. Supporting Infrastructure - New executor framework, config, and comprehensive comparison tests

Overall, this is well-structured code that establishes important test infrastructure. However, there are several issues that should be addressed.

---

Critical Issues

1. [CRITICAL] Incomplete Error Mapping in toEvmcError()

File: giga/executor/internal/host_context.go (lines 293-306)

func toEvmcError(err error) error {
    if err == nil {
        return nil
    }
    if _, ok := err.(evmc.Error); ok {
        return err
    }
    if errors.Is(err, vm.ErrExecutionReverted) {
        return evmc.Revert
    }
    return evmc.Failure
}

Problem: The function only handles vm.ErrExecutionReverted and maps all other errors to evmc.Failure. This loses important error context and may cause incorrect behavior for different error types like:

• vm.ErrOutOfGas - Should map to evmc.OutOfGas
• vm.ErrInsufficientBalance - Has specific semantics
• vm.ErrDepth - Stack depth exceeded
• vm.ErrCodeStoreOutOfGas - Contract creation out of gas

Recommendation: Extend the error mapping to handle more geth VM errors explicitly.

---

Important Findings

2. [IMPORTANT] Interpreter Does Not Update Gas After Execution

File: giga/executor/internal/interpreter.go (lines 62-69)

Problem: The returned GasLeft from Execute() is being discarded. The contract's gas should be updated after execution to reflect actual gas consumed. This could lead to incorrect gas accounting in nested calls.

3. [IMPORTANT] Missing DELEGATECALL Context Handling (Multiple TODOs)

Files:

• giga/executor/internal/host_context.go (line 193)
• giga/executor/internal/interpreter.go (line 58)

Multiple TODO comments indicate that DELEGATECALL sender/recipient propagation may not be correct. DELEGATECALL has specific semantics where msg.sender and msg.value should be preserved from the calling context, not updated. Incorrect handling could lead to security issues.

4. [IMPORTANT] Create Address Not Populated

File: giga/executor/internal/host_context.go (lines 161-167)

Contract creation returns an empty address, which breaks contracts that deploy other contracts and need to know their addresses.

---

Suggestions

5. [SUGGESTION] Test Subtest Naming Bug

File: giga/tests/state_test.go (lines 443-446)

subtestName := testName
if len(cancunPosts) > 1 {
    subtestName = testName + "/" + string(rune('0'+i))
}

Problem: For indices >= 10, string(rune('0'+i)) produces non-numeric characters.

Recommendation: Use fmt.Sprintf("%s/%d", testName, i) instead.

6. [SUGGESTION] Silent Parse Failures in parseHexBig()

Parse failures silently return zero, which could mask test data issues. Consider logging a warning or returning an error.

7. [SUGGESTION] Expand State Test Coverage

Currently only tests stArgsZeroOneBalance and stExample directories. Consider adding CI jobs to run more comprehensive state test categories.

---

Positive Observations

• Well-structured test framework with clear separation of concerns
• Good error recovery patterns with panic handling
• Correct OCC abort handling
• Clean platform-specific build tags for library loading
• Good use of transient store workaround for OCC mode

---

🤖 Generated with Claude Code

[pdrobnjak]

[CRITICAL] Incomplete Error Mapping

The function only handles vm.ErrExecutionReverted and maps all other errors to evmc.Failure. This loses important error context.

Missing mappings:

• vm.ErrOutOfGas → evmc.OutOfGas
• vm.ErrInsufficientBalance → specific handling
• vm.ErrDepth → stack depth exceeded

Suggestion:

switch {
case errors.Is(err, vm.ErrExecutionReverted):
    return evmc.Revert
case errors.Is(err, vm.ErrOutOfGas):
    return evmc.OutOfGas
default:
    return evmc.Failure
}

[pdrobnjak]

[SUGGESTION] Subtest Naming Bug

For indices >= 10, string(rune('0'+i)) produces non-numeric characters (e.g., ':' for i=10, ';' for i=11).

Suggestion: Use fmt.Sprintf for subtest naming:

subtestName = fmt.Sprintf("%s/%d", testName, i)

[pdrobnjak]

[SUGGESTION] Silent Parse Failures

Parse failures silently return zero, which could mask test data issues.

result, ok := new(big.Int).SetString(s, 16)
if !ok {
    return new(big.Int)  // Silently returns 0 on parse failure
}

Recommendation: Either log a warning or return an error when parsing fails to avoid masking issues in test fixtures.

[github-actions]

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

Build	Format	Lint	Breaking	Updated (UTC)
✅ passed	✅ passed	✅ passed	✅ passed	Jan 15, 2026, 4:14 PM