One dataset, two Croissant descriptions
One dataset on Hugging Face has two descriptions of itself.croissantminer/croissantminer is six Parquet files, about 1.1 MB, CC BY 4.0. Its authors did the work most dataset authors skip. They committed a Croissant 1.1 metadata file with Responsible AI fields: how the data was collected, how it was annotated, its known biases and limitations, and where it came from.
Hugging Face generated the second description. It is also valid Croissant 1.1. It carries none of the authors' RAI fields.
Same six files, two descriptions, and neither one records that anyone checked anything.
What does AI-ready data mean?
"AI-ready data" became a requirement before anyone defined it. Two bills introduced in the U.S. Senate this year, the AI-Ready Bio-Data Standards Act and the Artificial Intelligence Ready Data Act, would ask NIST to write the definition. DOE's Genesis Mission, launched by executive order in November 2025, calls for provenance tracking on the initial datasets it brings onto its platform. The requirement is arriving ahead of the definition.
Oak Ridge National Laboratory's data readiness work starts with a better question: readiness for what? Its answer depends on the use. A dataset prepared for training a climate foundation model is not ready for a protein folding model, and nobody expects it to be.
That answer has a consequence most discussions skip. If readiness depends on use, then "this dataset is AI-ready" is an incomplete sentence. It needs a profile: a named, versioned list of requirements, each with a check that decides whether it is met. A published standard can be a profile. So can your team's data checklist, once it is written down precisely enough for a program to evaluate. Name the profile and the sentence becomes something a program can check. Today it is a label in a catalog.
Why does an AI-ready label fall short?
The authors' Croissant file is a good label. It is also only a label.
It is a JSON document sitting next to the data, and nothing ties a decision about it to the bytes. Copy the Parquet files somewhere else and the label may or may not come along. Edit a file and the label still says the same thing. The catalog does not settle it either, because the catalog wrote its own.
Croissant is one label among several. CEOS publishes Analysis Ready Data specifications for Earth observation, with requirements split into threshold and higher levels. NIH's Bridge2AI program defines 28 AI-readiness criteria across seven dimensions. Four more exist: the ESIP checklist NOAA recommends to its grantees, ORNL's five readiness levels for scientific datasets, and frameworks from the UK government and the Open Data Institute.
These are careful documents written by people who know the problem. All of them describe what ready should mean. None of them records, for a specific set of bytes, that someone checked.
The gap is not the definition. It is the receipt.
How to attest that a dataset is AI-ready
Four steps turn that file into something a pipeline can rely on. You need an OCI registry, the kit CLI, cosign v3, and Python with the huggingface_hub and mlcroissant packages. Any registry works. The commands below assume a local one at localhost:5001 over plain HTTP. Create a signing key and a signing config with no transparency log:
cosign generate-key-pair
cosign signing-config create --out signing-config.jsonFirst, name a profile. This post uses its own, built from the Croissant specification and its Responsible AI extension. It asks four things of the metadata file. The official Croissant library has to load it without errors. It has to list every data file with a SHA-256 hash that matches the file actually packed. Eight Responsible AI fields have to be filled in: how the data was collected, what kind of collection it was, the annotation protocol, the annotation platform, known limitations, known biases, personal or sensitive information, and intended uses. And it has to say where the data came from, using the W3C provenance vocabulary.
The list is mine, loosely following what the NeurIPS 2026 datasets track asks authors to provide. Any of the frameworks above could be the profile instead. The predicate names its profile by ID, version, and URI, and the mechanism does not depend on which one it is.
Second, pack the Parquet files and the authors' croissant.json into a ModelKit. Six files and a metadata document need one digest covering the set, and a Kitfile produces it. A ModelKit is an OCI artifact, so it is addressed by digest and cannot change without getting a new one.
Download the dataset at a pinned revision into a data directory:
hf download croissantminer/croissantminer \
--repo-type dataset \
--revision f5fefc7c6b5f4ed67e789141317200e836208522 \
--include "*.parquet" --include "croissant.json" \
--local-dir dataThe Kitfile declares the data directory as a dataset:
manifestVersion: "1.0.0"
package:
name: croissantminer
datasets:
- name: croissantminer
path: datahf download leaves a .cache folder inside data. A one-line .kitignore keeps it out:
echo "data/.cache" > .kitignore
kit pack . -t localhost:5001/croissantminer:v1
kit push --plain-http localhost:5001/croissantminer:v1
# The manifest digest the claim will be about
DIGEST=$(curl -sI -H 'Accept: application/vnd.oci.image.manifest.v1+json' \
http://localhost:5001/v2/croissantminer/manifests/v1 \
| awk 'tolower($1) == "docker-content-digest:" { print $2 }' | tr -d '\r')Third, run a checker against the files you just packed. Any program that evaluates the files and records a result and the evidence for each requirement works. The mechanism depends only on what it outputs. Mine is a short Python script, check_croissant.py, that reads croissant.json and the Parquet files. Substitute your own:
python check_croissant.py data > predicate.jsonFourth, sign the result with cosign as an in-toto attestation whose subject is the ModelKit's digest. The predicate type is https://kitops.org/attestation/data-readiness/v1alpha1. I made that type up for this post, because the in-toto project's list has nothing for data readiness yet. The signed predicate looks like this, trimmed to the hash check, one of the eight RAI fields, and the provenance check:
{
"profile": {
"id": "croissant-rai",
"version": "1.1+rai-1.0",
"uri": "https://docs.mlcommons.org/croissant/docs/croissant-spec-1.1.html"
},
"level": "pass",
"checker": {
"name": "check_croissant.py",
"digest": "sha256:d0c7694088db3151d0cd6e71645e5c0372a85f28192a49b6bee8aa0b1518bf13"
},
"results": [
{
"requirement": "cr-files-match",
"status": "pass",
"evidence": {
"annotators.parquet": "match",
"calibration.parquet": "match",
"gold.parquet": "match",
"iaa.parquet": "match",
"ratings.parquet": "match",
"silver.parquet": "match"
}
},
{
"requirement": "rai-dataCollectionType",
"status": "pass",
"evidence": {
"rai:dataCollectionType": "Manual Human Curator. Annotators were instructed not to use AI tools while rating."
}
},
{
"requirement": "prov-provenance",
"status": "pass",
"evidence": {
"prov": "present"
}
}
]
}The signing command attests the digest, not the tag.
cosign attest --key cosign.key \
--type https://kitops.org/attestation/data-readiness/v1alpha1 \
--predicate predicate.json \
--signing-config signing-config.json \
--allow-http-registry \
localhost:5001/croissantminer@$DIGESTThe checker digest is the SHA-256 of the script itself. A different checker, or this one with a different list of requirements, produces a different claim.
The label said the data had RAI metadata. The claim says who checked it, against which profile, with which program, and which bytes it is about.
What happens when data changes?
A consumer verifies with cosign verify-attestation and a small CUE policy that requires this predicate type, this profile, and level pass. The --key flag pins the signer. This walkthrough signs with a local key and writes nothing to a transparency log, so cosign prints a warning on every verification and the checklist under it still claims the log was verified offline. Believe the warning, not the checklist.
The whole policy, saved as policy.cue:
predicateType: "https://kitops.org/attestation/data-readiness/v1alpha1"
predicate: {
profile: id: "croissant-rai"
level: "pass"
}cosign verify-attestation --key cosign.pub \
--type https://kitops.org/attestation/data-readiness/v1alpha1 \
--policy policy.cue \
--insecure-ignore-tlog \
--allow-http-registry \
localhost:5001/croissantminer:v1WARNING: Skipping tlog verification is an insecure practice that lacks transparency and auditability verification for the attestation.
will be validating against CUE policies: [policy.cue]
Verification for localhost:5001/croissantminer:v1 --
The following checks were performed on each of these signatures:
- The cosign claims were validated
- Existence of the claims in the transparency log was verified offline
- The signatures were verified against the specified public key
{"payload":"eyJfdHlwZSI6Imh0dHBzOi8vaW4tdG90by5pby9TdGF0…","payloadType":"application/vnd.in-toto+json","signatures":[{"sig":"MEQCIH/j0kEx…"}]}Now overwrite one byte of one Parquet file, repack with the same Kitfile, and push to the same tag.
# Overwrite one byte of gold.parquet in place. Any change to any packed file would do.
printf 'X' | dd of=data/gold.parquet bs=1 seek=1000 conv=notrunc
kit pack . -t localhost:5001/croissantminer:v1
kit push --plain-http localhost:5001/croissantminer:v1The croissant.json inside the modified ModelKit still lists every RAI field. It still carries a SHA-256 for gold.parquet, and that hash is now wrong. The data no longer meets the profile, and the label still says it does. Only a consumer who goes looking would see it, and the label gives them no reason to look.
The claim behaves differently. Same flags as before, by tag and then by the original digest:
cosign verify-attestation ... localhost:5001/croissantminer:v1
cosign verify-attestation ... localhost:5001/croissantminer@$DIGEST# By tag
WARNING: Skipping tlog verification is an insecure practice that lacks transparency and auditability verification for the attestation.
Error: no matching attestations:
error during command execution: no matching attestations:
# By the original digest
WARNING: Skipping tlog verification is an insecure practice that lacks transparency and auditability verification for the attestation.
will be validating against CUE policies: [policy.cue]
Verification for localhost:5001/croissantminer@sha256:31d378f1ee21747625ddd46a9a24d74efdc5e4cc689be25029f7a410771f159b --
The following checks were performed on each of these signatures:
- The cosign claims were validated
- Existence of the claims in the transparency log was verified offline
- The signatures were verified against the specified public key
{"payload":"eyJfdHlwZSI6Imh0dHBzOi8vaW4tdG90by5pby9TdGF0…","payloadType":"application/vnd.in-toto+json","signatures":[{"sig":"MEQCIH/j0kEx…"}]}The tag moved. The claim stayed with the digest it was made about. Pulling by tag now fails closed instead of passing silently.
Who signs a data readiness attestation?
The obvious objection is that nobody will do it. The dataset author has no incentive to sign, and the consumer who needs the claim cannot produce one.
That objection assumes the author has to sign. They do not. Everything above runs on the consumer's side. I pulled someone else's dataset, packed it, ran my own checker against my own profile, and signed with my own key. The claim is not "the authors swear this data is ready." It is "I checked these bytes against this profile with this program." That is the claim a platform team can make about data it did not produce, which is most data.
It also sets the trust boundary honestly. The walkthrough signs with a key file, so "who checked" means whoever holds that key. In practice, the checker runs in CI and signs with the pipeline's identity, the same way SLSA build provenance does. The claim is then as trustworthy as the pipeline that produced it.
What the claim does not prove is worth stating plainly. The checker confirms the RAI fields are present and nonempty. Whether "annotators were instructed not to use AI tools" is true is still the authors' word. Passing a profile also says nothing about whether the data suits your model. It says the agreed requirements were met, by someone, on a known set of bytes.
Will an official definition of AI-ready data be enough?
No matter how you define AI-ready, it will be a profile. The arguments will be about which requirements belong in it, and those arguments will be worth having. None of them answer the question that comes after: how does anyone downstream know this specific dataset was held to it?
A label tells you someone believed the data was ready. A claim tells you who checked, and it stops applying the moment the bytes change.
Package your own datasets as ModelKits with KitOps, and use Jozu Hub to scan them and check their attestations against policy before anything pulls or deploys them.