Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Schemas

Schemas are the foundation of TeaLeaf’s compact encoding. They define structure once so data can use positional encoding.

Defining Schemas

Use @struct to define a schema:

@struct point (x: int, y: int)

With multiple fields and types:

@struct user (
  id: int,
  name: string,
  email: string?,
  active: bool,
)

Optional Type Annotations

Field types can be omitted – they default to string:

@struct config (host, port: int, debug: bool)
# host defaults to string type

Using Schemas with @table

The @table directive binds an array of tuples to a schema:

@struct user (id: int, name: string, email: string)

users: @table user [
  (1, "Alice", "alice@example.com"),
  (2, "Bob", "bob@example.com"),
  (3, "Carol", "carol@example.com"),
]

Each tuple’s values are matched positionally to the schema fields.

Nested Structs

Structs can reference other structs. Nested tuples inherit schema binding from their parent field type:

@struct address (street: string, city: string, zip: string)

@struct person (
  name: string,
  home: address,
  work: address?,
)

people: @table person [
  (
    "Alice Smith",
    ("123 Main St", "Berlin", "10115"),     # Parsed as address
    ("456 Office Blvd", "Berlin", "10117"), # Parsed as address
  ),
]

Deep Nesting

Schemas can nest arbitrarily deep:

@struct method (type: string, last_four: string)
@struct payment (amount: float, method: method)
@struct order (id: int, customer: string, payment: payment)

orders: @table order [
  (1, "Alice", (99.99, ("credit", "4242"))),
  (2, "Bob", (49.50, ("debit", "1234"))),
]

Array Fields

Schema fields can be arrays of primitives or other structs:

@struct employee (
  id: int,
  name: string,
  skills: []string,
  scores: []int,
)

employees: @table employee [
  (1, "Alice", ["rust", "python"], [95, 88]),
  (2, "Bob", ["java"], [72]),
]

Nullable Fields

The ? modifier makes a field nullable:

@struct user (
  id: int,
  name: string,
  email: string?,   # can be ~
  phone: string?,   # can be ~
)

users: @table user [
  (1, "Alice", "alice@example.com", "+1-555-0100"),
  (2, "Bob", ~, ~),  # email and phone are null
]

Binary Encoding Benefits

Schemas enable significant binary compression:

  1. Positional storage – field names stored once in the schema table, not per row
  2. Null bitmaps – one bit per nullable field per row, instead of full null markers
  3. Type-homogeneous arrays – packed encoding when all elements match a schema
  4. String deduplication – repeated values like city names stored once in the string table

Example Size Savings

For 1,000 user records with 5 fields:

ApproachApproximate Size
JSON (field names repeated)~80KB
TeaLeaf text (schema + tuples)~35KB
TeaLeaf binary (compressed)~15KB

Schema Compatibility

Compatible Changes

ChangeNotes
Rename fieldData is positional; names are documentation only
Widen typeint8int64, float32float64 (automatic)

Incompatible Changes (Require Recompile)

ChangeResolution
Add fieldRecompile source .tl file
Remove fieldRecompile source .tl file
Reorder fieldsRecompile source .tl file
Narrow typeRecompile source .tl file

Recompilation Workflow

The .tl file is the master. When schemas change:

tealeaf compile data.tl -o data.tlbx

TeaLeaf prioritizes simplicity over automatic schema evolution:

  • No migration machinery – recompile when schemas change
  • No version negotiation – the embedded schema is the source of truth
  • Explicit over implicit – tuples require values for all fields