Metadata load file · one record per document · the default index of an e-discovery production
A DAT is a delimited plain-text file that lists the documents of a production, one record per line, with a named column of metadata for each: Bates ranges, custodian, correspondence fields, dates, file paths and — in most productions — the extracted text. Every review platform on the market imports it, which is why it survives as the lingua franca of productions even as the review tools change.
It is structurally a CSV. What makes it look like gibberish in a text editor is the delimiter set: instead of commas, it uses characters chosen because they almost never occur in real text.
| Role | Character | Byte in Windows-1252 | Meaning |
|---|---|---|---|
| Field delimiter | DC4, usually written ¶ | 0x14 (ASCII 20); some shops use visible ¶ = 0xB6 | Separates fields; one record per physical line |
| Text qualifier | þ (thorn) | 0xFE | Wraps every field; a literal þ inside a value is written twice |
| In-value line break | ® | 0xAE | Stands in for CR/LF so a record never spans physical lines |
The characters are picked to be absent from ordinary documents, so a parser can tell structure from content without escaping gymnastics. The trade-off is the classic support ticket: open a DAT in Excel with default settings and you get one garbled column, because Excel's comma assumption does not match the file.
The first line is a header: the field names, each wrapped in the qualifier and separated by the delimiter. Every following line is one document, same field count, same order.
þBEGBATESþ¶þENDBATESþ¶þBEGATTACHþ¶þCUSTODIANþ¶þTEXTþ þABC0000001þ¶þABC0000004þ¶þABC0000001þ¶þSmith, Johnþ¶þDear team,®Please find the report.®Regardsþ þABC0000005þ¶þABC0000005þ¶þABC0000001þ¶þSmith, Johnþ¶þAttachment body textþ
Read one record aloud: qualifier, value, delimiter, qualifier, value … and the line ends after the final closing qualifier plus a CR/LF. A parser that splits on the delimiter without honoring the qualifier will shred any value containing a comma — custodian names ("Smith, John") are the usual casualty.
| Field | Meaning |
|---|---|
| BEGBATES / ENDBATES | First and last Bates number of the document; the page count is the inclusive span between them |
| BEGATTACH / ENDATTACH | Family range — every member of an email family shares the parent's values, which is how review tools keep attachments with the parent |
| PAGECOUNT | Number of imaged pages; should equal the Bates span and the OPT/LFP page run |
| CUSTODIAN, FROM, TO, SUBJECT, DATESENT … | Free-form metadata; names and subjects are where commas and qualifier-lookalikes live |
| TEXT | Extracted text, often the largest column; internal line breaks arrive as ® |
| NATIVE_FILE, OCR_FILE | Relative paths into the volume for natives and text files |
There is no enforced schema — the header names whatever the producing party chose. District-court default standards publish a common field list, but vendors add, rename and reorder freely, so a converter must read the header rather than assume positions.
A DAT has no in-band encoding declaration, which is the root of the most common corruption: the qualifier þ is one byte (0xFE) in Windows-1252 and two bytes (C3 BE) in UTF-8. If a writer emits UTF-8 and a reader assumes cp1252, every field sprouts a à before its þ. Reliable handling means: honor UTF-8/UTF-16 byte-order marks, verify UTF-8 strictly instead of assuming it, fall back to Windows-1252 for ANSI files, and detect GB18030 when the custodians are Chinese — by checking which candidate decoding keeps the delimiters and the text intact.
A checker that counts fields per line against the header, verifies each record is one physical line, and round-trips the delimiters catches all four before the volume goes out. The converter does this on load; the Bates checker adds cross-file checks against the OPT/LFP.