Adversarial Tests

The adversarial test suite validates TeaLeaf’s error handling and robustness using crafted malformed inputs, binary corruption, compression edge cases, and large-corpus stress tests. All tests are isolated in the adversarial-tests/ directory to avoid touching core project files.

Current count: 58 tests across 9 categories.

Running Tests

# Run all adversarial tests
cd adversarial-tests/core-harness
cargo test --test adversarial

# With output
cargo test --test adversarial -- --nocapture

# Run via script (PowerShell)
./adversarial-tests/scripts/run_core_harness.ps1

# CLI adversarial tests
./adversarial-tests/scripts/run_cli_adversarial.ps1

# .NET adversarial harness
./adversarial-tests/scripts/run_dotnet_harness.ps1

Test Input Files

TeaLeaf Format (.tl) — 13 files

Crafted .tl files testing parser error paths:

Note: bad_number_overflow.tl does not cause a parse error. Numbers exceeding i64/u64 range are stored as Value::JsonNumber (exact decimal string), not rejected.

Edge cases that should succeed:

JSON Format (.json) — 6 files

Files testing from_json error and edge-case paths:

Binary Format (.tlbx) — 4 files (unused)

These fixture files exist but are not referenced by any test. All binary adversarial tests generate malformed data inline using tempfile::tempdir(). These files are only used by the CLI adversarial scripts in results/cli/.

Test Functions

Parse Error Tests (10 tests)

Success / Edge-Case Parse Tests (3 tests)

Error Variant Coverage (5 tests)

Tests that exercise specific Error enum variants for code coverage:

Type Coercion Tests (2 tests)

Validates spec §2.5 best-effort numeric coercion during binary compilation:

Binary Reader Tests (4 tests)

Binary Corruption Tests (12 tests)

Tests that take valid binary output, corrupt specific bytes, and verify the reader does not panic:

All corruption tests assert no panic. Most also verify that Reader::from_bytes() or reader.get() either returns an error or handles the corruption gracefully.

Compression Stress Tests (4 tests)

Soak / Large-Corpus Tests (8 tests)

Stress tests for parser, writer, and reader with large inputs:

Memory-Mapped Reader Tests (10 tests)

Validates Reader::open_mmap() produces identical results to Reader::open() and Reader::from_bytes():

Directory Structure

adversarial-tests/
├── inputs/
│   ├── tl/              # 13 crafted .tl files (11 error + 2 success)
│   ├── json/            # 6 crafted .json files
│   └── tlbx/            # 4 .tlbx files (used by CLI scripts, not Rust tests)
├── core-harness/
│   ├── tests/
│   │   └── adversarial.rs   # 58 Rust integration tests
│   └── Cargo.toml
├── dotnet-harness/          # C# harness using TeaLeaf bindings
├── scripts/
│   ├── run_core_harness.ps1
│   ├── run_cli_adversarial.ps1
│   └── run_dotnet_harness.ps1
├── results/                 # CLI test logs and outputs
└── README.md

Adding New Tests

1. Add an Inline Test (preferred)

Most adversarial tests generate their inputs inline. This avoids stale fixture files and keeps the test self-contained:

#![allow(unused)]
fn main() {
#[test]
fn parse_new_error_case() {
    assert_parse_err("malformed: input here");
}
}

The assert_parse_err helper asserts that TeaLeaf::parse(input).is_err().

2. For Binary Tests

Use the make_valid_binary helper to produce valid bytes, then corrupt them:

#![allow(unused)]
fn main() {
#[test]
fn reader_new_corruption_case() {
    let mut data = make_valid_binary("val: 42", false);
    data[0] ^= 0xFF; // corrupt something
    let result = Reader::from_bytes(data);
    // Assert no panic; error or graceful handling OK
    if let Ok(r) = result {
        let _ = r.get("val");
    }
}
}

3. Input File Tests (for CLI scripts)

Place malformed input in the appropriate subdirectory for CLI adversarial testing:

adversarial-tests/inputs/tl/bad_new_case.tl

The CLI script run_cli_adversarial.ps1 exercises these files through the tealeaf CLI binary and logs results to results/cli/.