AI Dataset

Origin & evolution

The case for documenting datasets as governed assets was made in Datasheets for Datasets, proposed by Timnit Gebru and co-authors in 2018. Borrowing from the datasheets that accompany electronic components, they argued every dataset should ship with a record of its motivation, composition, collection process, and recommended uses. The paper was later published in Communications of the ACM in 2021, and the format now anchors dataset documentation across major model providers.

Three concerns have sharpened since. Provenance asks where the data came from and whether consent or licence covers its use. Contamination asks whether evaluation data has leaked into training, which inflates benchmark scores without improving real performance. Licensing has moved from a footnote to a live legal question as text and image corpora face copyright scrutiny.

How it works in practice

A team fine-tunes a classifier on 40,000 labelled support transcripts. Before training, they record the dataset's source (their own production logs), its licence (internal use only), and its date range. During evaluation, they discover that 600 transcripts in the test split also appear in training, so the reported accuracy is overstated. They deduplicate, re-split, and re-run. The honest score drops two points, and the datasheet now documents the contamination check so the next team does not repeat the mistake.

AI Dataset vs. its neighbours

• AI Model🤖AI ModelAI & Machine LearningAn AI or ML model used within the productView reference → is the trained output; the dataset is the input. The model inherits the dataset's biases and gaps, which is why the link between them is tracked rather than assumed.
• Data Source🛢️Data SourceData & AnalyticsA data source or integrationView reference → is the upstream system the data was pulled from, such as a CRM or a log store. A dataset is a curated, versioned snapshot derived from one or more sources.
• Data Classification🍃Data ClassificationSecurityA data classification levelView reference → labels how sensitive the contents are. A dataset carries a classification, and that label drives who may use it for training and where it may be stored.

In the graph

In the Unified Product Graph, an AI dataset sits in the AI and intelligence region as the governed input to training. Its primary edge is ai_model_trained_on_ai_datasetAI Modeltrained onAI Datasethierarchy, which records exactly which corpus produced which model. That single link supports the questions governance teams actually ask: if a dataset turns out to be unlicensed, which models are affected, and which products depend on those models.