Trusted by the best startups startups in the world
All endpoints at a glance
Each API solves a different shape of problem. Follow the landing page for code samples, pricing context, and how it pairs with other endpoints.
Scrapes
Extract content from a URL you already know — markdown, HTML, text, JSON, screenshots, parsers, LLM extract.
Use when
You get
Per-URL content (markdown_content, html_content, json_content, etc.) ready for RAG, agents, or storage.
Use Batch when you have hundreds to thousands of URLs and want one job with predictable wall-clock time.
Crawls
Walk a site from a start URL with depth and page limits; poll until completed; list pages and retrieve content.
Use when
You get
Crawl status, per-page retrieve_ids, then markdown/HTML/JSON via /v1/retrieve — similar to batch but site-shaped.
Use Batch when your URLs come from anywhere (not one crawl frontier); use Crawl for hierarchical site walks.
Maps
Discover URLs for a single domain — sitemaps, links, filters, and cursor pagination.
Use when
You get
A search object with result.links (url, title, description) plus optional hosted JSON for the full result set.
Map discovers URLs only; Crawl fetches content across many pages of a site.
Batches
Process large URL lists in one job — predictable completion time, parsers, webhooks, metadata, cursor over items.
Use when
You get
Batch status, paginated items, then per-item retrieve — ideal for scale without hand-rolling concurrency.
Parallel Scrapes are often faster for tiny sets; Batch wins for large lists and operational simplicity.
Search
Semantic web search from a plain-English query — deduplicated links with titles and descriptions.
Use when
You get
A search object with result.links (url, title, description) plus optional hosted JSON for the full result set.
Search returns links across the web for a query; Map returns URLs discovered for one site you already chose.
Answers
Ask a question in natural language; get a source-backed answer and optional structured JSON.
Use when
You get
An answer object with validated fields, citations, and NOT_FOUND when the web cannot support a claim — not just raw page text.
Use Search when you need a list of links to explore; use Answers when you need a single synthesized response.
Monitors
Create persistent monitors from natural-language queries; run scheduled checks, detect page changes, and send alerts.
Use when
You get
Monitor status, schedule, tracked targets, snapshot events, change summaries, and run logs for recurring checks.
Use Scrapes for one-time extraction; use Monitors when the same source must be checked repeatedly.
Trusted by Amazing Teams Building the Future of AI
Olostep is the best!!! We automated entire data pipelines with just a prompt
Olostep has become the default Web Layer infrastructure for our company
Olostep works like a charm! And your customer service is exceptional
Olostep lets us turn any website into an API. Great product, great people
I highly recommend Olostep, great product!
We verify coupon codes at scale. Love Olostep. It works on any e-commerce
Olostep is the best API to search, extract, and structure data from the Web. Happy to be customers
We use /batches combined with parsers and it's magical how we can get structured data at large scale
Olostep allowed us to search and structure events data across the Web
Reliable and cost-effective API for working with data. Congrats on the cool product
Start scraping with Olostep
Get clean data for your AI from any website with Olostep
Most cost-effective API. Built for scale
Frequently asked questions
I have a single URL — which endpoint?
Use POST /v1/scrapes (Scrapes) to extract markdown, HTML, JSON, screenshots, or parser output from that page.
I need every URL on a website — which endpoint?
Use POST /v1/maps (Maps) for domain-level URL discovery with optional path filters and cursor pagination.
I need to walk a site and get many pages of content — which endpoint?
Use POST /v1/crawls (Crawls) when you want a multi-page crawl from a start URL, then list pages and retrieve content per page.
I have thousands of arbitrary URLs — which endpoint?
Use POST /v1/batches (Batches) for large lists with predictable job time, optional parsers, webhooks, and metadata.
I need a direct answer from the web — which endpoint?
Use POST /v1/answers (Answers) for agentic Q&A with validation and optional structured JSON — not just a link list.
I need ranked links for a question — which endpoint?
Use POST /v1/searches (Search) for semantic web search that returns deduplicated links with titles and descriptions; then scrape selected URLs if you need full page content.
When are parallel Scrapes better than a Batch?
For a small number of URLs, parallel /v1/scrapes calls often finish faster. Batches shine for large lists and operational patterns (webhooks, single job id, item cursoring).
