AI readiness checklist: is your data ready?
A practical checklist for assessing whether your data is ready for an AI project: inventory, quality, access, ownership, and measurement.
Models are available by subscription; your documents, records and permissions are not. Before choosing a vendor or a use case, run this checklist with the people who own the systems. It shows where the first project should and should not go.
Why data readiness comes first
Retrieval-augmented generation, the most common way to point a model at company knowledge, works by retrieving passages from your documents and having the model answer from them [1]. If the right passage is missing, outdated or buried in an unreadable scan, the answer suffers. Research also links confident wrong answers to models guessing when uncertain [3], so supplying good source material and permitting "I don't know" are your levers.
The checklist
1. Scope
[ ] One specific use case with a named business owner.
[ ] A list of the questions or tasks the system must handle, written in the users' own words.
[ ] A definition of "good enough," such as the share of answers that must be correct and cited.
2. Inventory
[ ] Every system holding relevant data is listed: file shares, email, ticketing, ERP, CRM, spreadsheets.
[ ] Each source has an owner who can answer questions about it.
[ ] You know which sources are authoritative when two disagree.
3. Quality
[ ] Documents are current; superseded versions are labeled or removed.
[ ] Scans and images have been checked for legible text.
[ ] Duplicates and drafts are identified.
[ ] Key fields (dates, customer names, part numbers) are consistent across systems.
4. Access and sensitivity
[ ] You know where personal, financial or confidential data sits.
[ ] Access rules exist per source, and the AI system can respect them per user.
[ ] You have decided what must never leave your environment, and which vendors may see what.
[ ] A named owner has reviewed the use case, its data flows and its failure modes.
5. Structure and freshness
[ ] Metadata exists (document type, date, owner, product, customer) to help filter retrieval.
[ ] There is a plan for how new and changed documents reach the AI system, and how often.
[ ] Deleted documents are removed from the index.
6. Measurement
[ ] A test set of real questions with approved answers, built with the people who do the work.
[ ] A baseline: how long and how accurate the task is today, without AI.
[ ] Logging so outputs and sources can be reviewed later.
7. Governance
[ ] A named person accountable for the system after launch.
[ ] A review process for high-stakes outputs.
[ ] A framework to organize risk work. NIST's AI Risk Management Framework is intended for voluntary use, and NIST published a Generative AI Profile (NIST-AI-600-1) in July 2024 that covers risks specific to generative AI [2].
How to read your results
Result | What it suggests |
|---|---|
Mostly checked in sections 1-3 | Data is a reasonable starting point; proceed to a pilot |
Gaps in section 4 | Resolve access and sensitivity before any pilot touches real data |
Gaps in section 6 | Build the test set first; you cannot judge a pilot without it |
Gaps across sections 2-3 | Start with a smaller, cleaner data source |
Practical implication
Start with the narrowest use case whose data you can fully inventory, secure and test, then expand. A small pilot on clean, well-owned data teaches more than a broad one on data nobody can vouch for.
Sources
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks arXiv (Lewis et al.), NeurIPS 2020, 2020-05-22. arxiv.org/abs/2005.11401
- NIST AI Risk Management Framework NIST, 2023-01-26. www.nist.gov/itl/ai-risk-management-framework
- Why Language Models Hallucinate arXiv (Kalai, Nachum, Vempala, Zhang), 2025-09-04. arxiv.org/abs/2509.04664
.jpg){kind=link}