How the inspector reads an OCF
The parser requires the four-byte Obj\x01 magic, decodes the Avro map that carries header metadata, parses the required avro.schema JSON, and records the 16-byte file sync marker. Every data block is then bounded by its zigzag-varint record count, stored byte size, payload, and matching sync marker. Counts and offsets remain visible even when a codec cannot be decoded.
Codec and record support
The required null and raw deflate codecs are decoded in-browser. Optional codecs such as Snappy, Bzip2, XZ, and Zstandard remain structurally mapped but are not decompressed. Sample decoding supports the Avro binary primitive and complex schema types, including records, enums, fixed, arrays, maps, unions, and named references, with explicit depth, item, record, and block limits.
Schema comparison
An optional AVSC compares paths and structural definitions with the embedded writer schema. Documentation and aliases are ignored so descriptive edits do not obscure type, field, union, array, map, enum, or fixed changes. This is a structural diff, not a reader/writer compatibility decision: compatibility depends on direction, defaults, aliases, promotion rules, logical types, and the actual reader schema.
Important validation boundary
This is a bounded diagnostic, not an Apache Avro implementation certification or data-recovery guarantee. It does not validate every schema naming or logical-type rule, apply reader-schema resolution, verify application semantics, decode unsupported optional codecs, repair corrupt files, prove that recovered-looking bytes contain trustworthy records, or replace official libraries and production ingestion tests. Large or adversarial containers may reach the stated inspection limits.
Frequently asked questions
Why are declared and decoded record counts different?
The header can be readable while a codec is unsupported, a block is damaged, or the bounded decode limit is reached. Declared counts come from block framing; decoded counts require walking record values with the writer schema.
Can sync markers recover a corrupt file automatically?
No. A matching 16-byte marker is strong boundary evidence and can help split work, but it does not repair malformed bytes or prove the semantic integrity of neighboring records.
Why does an unsupported codec still show blocks?
Block counts, stored sizes, and sync markers live outside the compressed payload. They can be audited without claiming that the records were decoded.
Does the schema diff prove compatibility?
No. It is an inventory of structural differences. Test compatibility in the direction your producer and consumer actually use.
Authoritative reference
The Apache Avro 1.12.0 specification defines schemas, zigzag variable-length integers, binary record encoding, Object Container File metadata, codecs, blocks, and synchronization markers.
Browse data-engineering tools · Inspect NetCDF classic headers · Inspect SQLite WAL structure