Data and representation

Implementation status: Canonical data and callable representations remain in current TTX. The Direct, Shared, Block and Fragment protocols below record the earlier transport design used by the Godot integration. Current TTX exposes whole-value Storage transfer while provider-specific access remains a design area.

An interface needs a description of the data crossing it. A color can be four fields, an array of four values, a view into a larger image, or something a provider generates when asked.

Data's job is to make the concrete agreement precise without deciding what the color means. We should be able to establish where the values belong and how to access them without pulling an image model, a source language or a type system into the transfer engine.

Compiling the description

A Schema is the description an owner authors. It contains primitives, structure, repetitions, pointers, callable signatures and the geometry needed to place them. The compiler turns that into a canonical Representation for consumers to use.

Consider these two C structures:

struct Separate {
    uint8_t r, g, b;
    uint32_t value;
};

struct Repeated {
    uint8_t rgb[3];
    uint32_t value;
};

If their offsets, extent, alignment and other admitted format facts agree, the three adjacent bytes can use the same compact description. The consumer has no reason to care whether the author spelled them as three fields or an array.

A nested struct boundary is a different matter. So are changes in primitive type, byte order or callable signature. Those facts can affect the agreement, so the compiler must preserve them.

Doing this work at publication gives the consumer a much simpler job. It can compare encoded lengths and bytes instead of reconstructing the producer's schema tree. The schema objects can even be released once the prepared description owns what it needs. C++'s Compiled<schema> can do the same preparation during constant evaluation, while a dynamically loaded provider can compile its description at runtime.

Preparing a representation still costs work and storage. That trade is useful when the prepared description can serve repeated calls or several consumers. Compiling it again for every small transfer would bring the work straight back into the path we were trying to simplify. Compilation and repeated comparison therefore need to be measured separately.

Keeping the result compact

The published stream contains fixed-width struct, element and callable blocks, encoded in little endian order. Every block in a publication uses the same width. The compiler chooses the smallest supported width that can hold all of the normalized fields, so two equivalent descriptions cannot differ merely because one producer preferred a wider encoding.

Struct headers describe extent, alignment and their direct element count. Element blocks describe a count, offset and distance, together with a primitive or referenced form. Repeated values and repeated structs can stay compact instead of expanding into one descriptor per occurrence.

A pattern with irregular gaps or changing element shapes can require more descriptors. The published description records those details so the consumer doesn't have to infer them during each transfer.

The full bit layout and normalization rules live beside the compiler, including reference ordering, pointer targets and invalid forms. That specification is what another encoder would need to follow.

Describing a call as data

An interface table contains data too. Its callable entries need descriptions of the calling convention, arguments and return type, including a receiver when the operation takes one. Pointer width alone would tell us very little about whether the stored function can be called safely.

Including those facts in the canonical representation lets ordinary agreement check the whole API record. Two participants can recognize the same semantic contract and still reject incompatible callable descriptions before using them. The description must include the pointer width and calling convention of the host on which the callable will run.

This establishes the physical call, while the UUID and the contract's behavior still matter. Matching descriptors won't prove that a convolution implements the right edge handling. They also won't make payloads equal. Padding positions are part of the format, but the values in those padding bytes need not be canonical.

Providing access to the result

Storage supplies bytes conforming to a Representation. A provider doesn't necessarily keep its private state in that form. It may lend a compatible pointer through Direct or Shared, populate caller storage through Block, or produce typed values individually through Fragment.

The provider is responsible for producing the requested form through the selected protocol. Data doesn't convert a private object by casting it to the consumer's type.

The higher-order Layout and Pack concepts have a different job: organizing semantic value flow. Keeping those apart from Schema, Representation and Storage lets the lower layer remain a concrete transport system while the conceptual owners decide which values should participate and why.