Data and representation

Before we can share an interface, we need a description of the data crossing it. That sounds like the simple part until we remember how many ways there are to lay out the same idea. A color can be four fields, an array of four values, a view into a larger image, or something a provider generates when asked.

Data's job is to make the concrete agreement precise without deciding what the color means. We should be able to establish where the values belong and how to access them without pulling an image model, a source language or a type system into the transfer engine.

Compiling the description

A Schema is the description an owner authors. It contains primitives, structure, repetitions, pointers, callable signatures and the geometry needed to place them. The compiler turns that into a canonical Representation for consumers to use.

Consider these two C structures:

struct Separate {
    uint8_t r, g, b;
    uint32_t value;
};

struct Repeated {
    uint8_t rgb[3];
    uint32_t value;
};

If their offsets, extent, alignment and other admitted format facts agree, the three adjacent bytes can use the same compact description. The consumer has no reason to care whether the author spelled them as three fields or an array.

A nested struct boundary is a different matter. So are changes in primitive type, byte order or callable signature. Those facts can affect the agreement, so the compiler must preserve them. Canonicalization is useful precisely because it has rules about which differences can disappear and which ones cannot.

Doing this work at publication gives the consumer a much simpler job. It can compare encoded lengths and bytes instead of reconstructing the producer's schema tree. The schema objects can even be released once the prepared description owns what it needs. C++'s Compiled<schema> can do the same preparation during constant evaluation, while a dynamically loaded provider can compile its description at runtime.

Publication still costs work and storage. That trade is useful when the prepared description can serve repeated calls or several consumers. Compiling it again for every small transfer would bring the work straight back into the path we were trying to simplify. Compilation and repeated comparison therefore need to be measured separately.

Keeping the result compact

The published stream contains fixed-width struct, element and callable blocks, encoded in little endian order. Every block in a publication uses the same width. The compiler chooses the smallest supported width that can hold all of the normalized fields, so two equivalent descriptions cannot differ merely because one producer preferred a wider encoding.

Struct headers describe extent, alignment and their direct element count. Element blocks describe a count, offset and distance, together with a primitive or referenced form. Repeated values and repeated structs can stay compact instead of expanding into one descriptor per occurrence.

There are limits to what that buys us. A pattern with irregular gaps or changing element shapes can require more descriptors. Compact repetition doesn't make an arbitrary arrangement free to describe. The important part is that the consumer receives a settled physical description rather than a new problem to infer during each transfer.

The full bit layout and normalization rules live beside the compiler, including reference ordering, pointer targets and invalid forms. That specification is what another encoder would need to follow.

Describing a call as data

An interface table contains data too. Its callable entries need descriptions of the calling convention, arguments and return type, including a receiver when the operation takes one. Pointer width alone would tell us very little about whether the stored function can be called safely.

Including those facts in the canonical representation lets ordinary agreement check the whole API record. Two participants can recognize the same semantic contract and still reject incompatible callable descriptions before using them. The initial native calling realization is System V AMD64; another ABI would need its own accurate description.

This establishes the physical call, while the UUID and the contract's behavior still matter. Matching descriptors won't prove that a convolution implements the right edge handling. They also won't make payloads equal. Padding positions are part of the format, but the values in those padding bytes need not be canonical.

Providing access to the result

Storage supplies bytes conforming to a Representation. A provider doesn't necessarily keep its private state in that form. It may lend a compatible pointer through Direct or Shared, populate caller storage through Block, or produce typed values individually through Fragment.

That freedom is what allows a GPU image and a generated value to take part in the same concrete agreement. The provider is responsible for satisfying the selected access contract. Data doesn't secretly convert an incompatible internal object by casting it to whatever the consumer wanted.

The higher-order Layout and Pack concepts have a different job: organizing semantic value flow. Keeping those apart from Schema, Representation and Storage lets the lower layer remain a concrete transport system while the conceptual owners decide which values should participate and why.