Data validation is the process of checking that data follows a set of rules before you store it or use it. The rules describe what good data should look like: the right type, the right format, a sensible range, no missing required fields and no duplicates.
For example, a rule might say that an email field must contain an "@" and a domain, or that age must be a whole number between 0 and 120. If a record breaks a rule, the system rejects it, flags it or sends it for review.
Validation answers one question: "Is this data in the shape and range we expect?" It does not answer "Is this data true?" That difference matters, and we come back to it later.
Validation also needs a defined structure to check against. A page of raw HTML or free text has no fields yet, so it must be parsed into fields first. Olostep's guide to structured vs. unstructured data explains that step.
Why Data Validation Matters
Data validation matters because one bad record rarely stays in one place. A wrong price or a missing ID can spread into dashboards, apps, reports and AI models before anyone notices.
Catching errors at the point where data enters a system is cheaper than tracing them later. According to IBM's overview of data quality dimensions, citing Forrester, over 25% of global data and analytics employees say poor data quality hinders data literacy, costing their organizations over USD 5 million annually.
AI projects feel this too. In Qlik's 2025 AI survey of 500 U.S. data and analytics professionals, 81% of AI professionals say their company still has significant data quality issues. A model can only be as reliable as the records it reads.
Common Types of Data Validation Checks
Most validation rules fall into a small set of check types. You combine them to describe what a valid record looks like.
| Check type | What it confirms | Example rule |
|---|---|---|
| Data type | The value has the expected type | price is a number, not text |
| Format | The value follows a pattern | date matches YYYY-MM-DD |
| Range | The value falls between limits | quantity is between 1 and 1,000 |
| Presence (required) | A required field is not empty | Every order has a customer_id |
| Uniqueness | No duplicate values where they are not allowed | No two users share the same email |
| Code or lookup | The value appears in an approved list | country is a valid ISO country code |
| Consistency (cross-field) | Related fields agree with each other | end_date is later than start_date |
When people ask about the "four types" of data validation, they usually mean type, format, range and consistency checks. The table adds presence, uniqueness and lookup checks because real datasets break those rules just as often.
Schema Validation for JSON and API Data
Schema validation checks a whole record against a schema, a written contract that lists every field, its type and whether it is required. Instead of writing many separate checks, you declare them once. JSON Schema is the common standard for data in JSON, the text format most APIs use.
A small schema for a product record looks like this:
{
"type": "object",
"required": ["name", "price", "currency", "url"],
"properties": {
"name": { "type": "string", "minLength": 1 },
"price": { "type": "number", "exclusiveMinimum": 0 },
"currency": { "type": "string", "enum": ["USD", "EUR", "GBP"] },
"url": { "type": "string", "format": "uri" },
"in_stock": { "type": "boolean" }
}
}This record fails it:
{
"name": "Trail Running Shoe",
"price": "89.99",
"currency": "usd",
"in_stock": "yes"
}A validator would report four problems. price is text instead of a number, currency is not in the allowed list, in_stock is not a boolean and the required url field is missing.
The same idea works before data exists. With schema-based extraction, you define field names and types up front, and the extractor fills that structure. Your validator then checks against the same contract the extractor used.
How the Data Validation Process Works
A data validation process turns rules into a repeatable routine. One common version has five steps.
- Define rules from how the data is used. Start with the fields your app, report or model depends on. Write a rule for each one.
- Validate at the boundary. Run checks where data enters your system, before it is saved or passed on.
- Decide what happens on failure. Reject the record, move it to a quarantine table or accept it with a flag. Pick one on purpose for each rule.
- Log every result. Record what failed, when and from which source. This makes errors traceable.
- Monitor over time. Track how many records fail each rule per run. A sudden jump usually means the source changed.
Validation can run at several points in a pipeline. Each point catches a different kind of problem.
| Where it runs | What it catches | Example |
|---|---|---|
| Form or user input | Typos and missing fields | A signup form rejects a blank email |
| API or data ingestion | Wrong shapes from outside sources | An incoming JSON payload fails its schema |
| ETL or ELT pipeline | Problems created by joins and transforms | Totals no longer match after a merge |
| Before an app or model uses it | Anything that slipped through earlier | A retrieval step drops records with empty text |
Olostep's guide to treating parser output as a JSON contract applies these steps to web data. It recommends locking the expected shape on one representative URL, storing provenance such as the URL and retrieval time, and tracking missing keys and null spikes per field.
Data Validation vs. Data Verification vs. Data Cleansing
These three terms are related but answer different questions. Validation checks rules, verification checks truth and cleansing fixes problems.
| Data validation | Data verification | Data cleansing | |
|---|---|---|---|
| Question it answers | Does the data follow our rules? | Does the data match reality or a trusted source? | How do we fix or remove bad data? |
| When it runs | On entry and on every pipeline run | After entry, often on a sample | After problems are found |
| Example | phone has 10 digits | Calling the number reaches the right person | Removing dashes and duplicate rows |
| Result | Pass, fail or flag | Confirmed or not confirmed | A corrected dataset |
A record can pass validation and still be wrong. A phone number with 10 digits is valid, but it may belong to someone else. That is why validation is one part of data quality, not all of it.
Data Validation Examples
Data validation looks different depending on where data comes from. These three examples move from simple to more technical.
Example: A Signup Form
A signup form is a familiar case. Before saving a new account, the system checks that the email field is filled in and has a valid format. It also checks that the password meets a minimum length and that the birth date gives an age within an allowed range.
If any check fails, the form shows an error and the record is never saved. The bad data stops at the door.
Example: Product Data Extracted From the Web
Web data is harder because the source can change without warning. A pipeline might start by converting HTML into structured JSON, then validate each product record. Rules might require price to be a positive number, currency to be an approved code and url to be present.
The tricky failure is a quiet one. After a site redesign, the request can still succeed and return valid JSON while price comes back empty on most pages. Checking each record catches the bad row, but tracking the null rate per field catches the pattern.
When you are running extraction in batches across thousands of URLs, validate every item, not just the job status. A batch that finished is not the same as a batch that returned usable data.
Example: AI-Generated Output
AI models that return JSON need validation too. The authors of the JSONSchemaBench benchmark note that the LM generation process is probabilistic and does not provide guarantees on the output's structure. In their tests of a small open model without constraints, outputs often failed real-world schemas, and failures grew as schemas got more complex.
A safe pattern is simple. Parse the model's output, validate it against your schema, then retry or flag the record if it fails. Constrained decoding, which limits a model to tokens that fit the schema, improved compliance in that benchmark, but results varied by tool, so the validation step still belongs in the pipeline.
Where Data Validation Falls Short
Data validation confirms that data follows rules. It cannot confirm that the data is correct. A well-formed record with the wrong price passes every check.
That gap matters more as AI systems act on data. In IBM's analysis of data quality issues, the company notes in a passage on agentic AI that nearly half (49%) of executives cited data inaccuracies and bias as a barrier to embracing the technology. Validation reduces that risk, but accuracy also needs verification against trusted sources.
Rules also go stale. When a source adds a field, renames one or changes a format, old rules may reject good data or miss new problems. For external sources such as websites and third-party APIs, plan to validate on every run, review rules when failure rates shift and keep provenance so you can trace any record back to where it came from.
Validated records are the starting point for useful AI inputs. Olostep's guide to preparing LLM-ready data covers the next steps of formatting and structuring that data for models.
FAQ
What are the four types of data validation?
The four most common types are data type, format, range and consistency checks. Many teams also add presence, uniqueness and lookup checks.
Which is an example of data validation?
Rejecting a signup when the email field has no "@" symbol is a simple example of data validation.
What is data validation in Excel?
In Excel, data validation is a feature that limits what users can type into a cell, such as a drop-down list or a number range. It applies the same idea to a spreadsheet instead of a data pipeline.
Is data validation the same as data quality?
No. Data validation is one part of data quality and checks rules, while data quality also covers accuracy, completeness, freshness and consistency over time.
When should data validation happen?
Data validation should happen wherever data enters or moves through your system, such as user input, API ingestion and pipeline steps. For external sources that change, it should run on every load.
Can AI output be trusted without validation?
AI output should be validated before use, because language models do not guarantee the structure of what they return. Parse it, check it against a schema and handle failures.