Click to try
Wait...

Choose the right API endpoint for your job.

Olostep exposes focused HTTP APIs: Answers and Search for questions and discovery; Scrapes for known URLs; Maps for site URL inventory; Crawls for multi-page site walks; Batches for large arbitrary URL lists.

Start for free Read the Docs
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

Trusted by the best startups startups in the world

All endpoints at a glance

Each API solves a different shape of problem. Follow the landing page for code samples, pricing context, and how it pairs with other endpoints.

Quick decisions to explore

I need all discoverable URLs for one domain

I need to crawl many pages on a site from a start URL

I have a large list of URLs to process in one job

I need semantic web search — ranked links for a natural-language query

Scrapes

Extract content from a URL you already know — markdown, HTML, text, JSON, screenshots, parsers, LLM extract.

Explore Read the Docs API reference

Use when

One page or a small set of known URLs
Rich formats and structured extraction per page

You get

Per-URL content (markdown_content, html_content, json_content, etc.) ready for RAG, agents, or storage.

Use Batch when you have hundreds to thousands of URLs and want one job with predictable wall-clock time.

Crawls

Walk a site from a start URL with depth and page limits; poll until completed; list pages and retrieve content.

Explore Read the Docs API reference

Use when

Multi-page documentation or blogs on one site
When you want exploration + content in one workflow

You get

Crawl status, per-page retrieve_ids, then markdown/HTML/JSON via /v1/retrieve — similar to batch but site-shaped.

Use Batch when your URLs come from anywhere (not one crawl frontier); use Crawl for hierarchical site walks.

Maps

Discover URLs for a single domain — sitemaps, links, filters, and cursor pagination.

Explore Read the Docs API reference

Use when

Discovery without maintaining your own SERP scrapers
Feeding candidate URLs into Scrapes or Batches

You get

A search object with result.links (url, title, description) plus optional hosted JSON for the full result set.

Map discovers URLs only; Crawl fetches content across many pages of a site.

Batches

Process large URL lists in one job — predictable completion time, parsers, webhooks, metadata, cursor over items.

Explore Read the Docs API reference

Use when

Thousands of arbitrary URLs (catalogs, SERPs, exports)
Pipelines that need webhooks and batch-level parsers

You get

Batch status, paginated items, then per-item retrieve — ideal for scale without hand-rolling concurrency.

Parallel Scrapes are often faster for tiny sets; Batch wins for large lists and operational simplicity.

Searches

Semantic web search from a plain-English query — deduplicated links with titles and descriptions.

Explore Read the Docs API reference

Use when

Discovery without maintaining your own SERP scrapers
Feeding candidate URLs into Scrapes or Batches

You get

A search object with result.links (url, title, description) plus optional hosted JSON for the full result set.

Search returns links across the web for a query; Map returns URLs discovered for one site you already chose.

Answers

Ask a question in natural language; get a source-backed answer and optional structured JSON.

Explore Read the Docs API reference

Use when

Grounding LLMs on live web data
Enrichment when the input is a question, not a URL list

You get

An answer object with validated fields, citations, and NOT_FOUND when the web cannot support a claim — not just raw page text.

Use Search when you need a list of links to explore; use Answers when you need a single synthesized response.

Monitors

Create persistent monitors from natural-language queries; run scheduled checks, detect page changes, and send alerts.

Explore Read the Docs API reference

Use when

Tracking pricing, changelogs, blogs, or status pages
Alerting teams by email, Slack, SMS, or webhook

You get

Monitor status, schedule, tracked targets, snapshot events, change summaries, and run logs for recurring checks.

Use Scrapes for one-time extraction; use Monitors when the same source must be checked repeatedly.

Trusted by Amazing Teams Building the Future of AI

  • Michelle Julia
    Co-founder & CEO Aurium

    Olostep is the best!!! We automated entire data pipelines with just a prompt

  • Richard He
    Co-founder & CEO Openmart

    Olostep has become the default Web Layer infrastructure for our company

  • Max Brodeur-Urbas
    Co-founder & CEO Gumloop

    Olostep works like a charm! And your customer service is exceptional

  • Rob Hayes
    Co-founder Merchkit

    Olostep lets us turn any website into an API. Great product, great people

  • Brandon Cohen
    Co-founder & CTO CivilGrid

    I highly recommend Olostep, great product!

  • Co-founder & CEO Gedd.it

    We verify coupon codes at scale. Love Olostep. It works on any e-commerce

  • Trevor West
    Co-founder & CEO Podqi

    Olostep is the best API to search, extract, and structure data from the Web. Happy to be customers

  • Rida Naveed
    Co-founder Zecento

    We use /batches combined with parsers and it's magical how we can get structured data at large scale

  • Kieran V.
    Growth PlotsEvents

    Olostep allowed us to search and structure events data across the Web

  • Paul Mit
    Founder Foundbase

    Reliable and cost-effective API for working with data. Congrats on the cool product

Start scraping with Olostep

Get clean data for your AI from any website with Olostep
Most cost-effective API. Built for scale

Are you an AI Agent? Get started here

Frequently asked questions

Product & Capabilities

  • I have a single URL — which endpoint?

    Use POST /v1/scrapes (Scrapes) to extract markdown, HTML, JSON, screenshots, or parser output from that page.

  • I need every URL on a website — which endpoint?

    Use POST /v1/maps (Maps) for domain-level URL discovery with optional path filters and cursor pagination.

  • I need to walk a site and get many pages of content — which endpoint?

    Use POST /v1/crawls (Crawls) when you want a multi-page crawl from a start URL, then list pages and retrieve content per page.

Usage & Automation

  • I have thousands of arbitrary URLs — which endpoint?

    Use POST /v1/batches (Batches) for large lists with predictable job time, optional parsers, webhooks, and metadata.

  • I need a direct answer from the web — which endpoint?

    Use POST /v1/answers (Answers) for agentic Q&A with validation and optional structured JSON — not just a link list.

Pricing & Plans

  • I need ranked links for a question — which endpoint?

    Use POST /v1/searches (Search) for semantic web search that returns deduplicated links with titles and descriptions; then scrape selected URLs if you need full page content.

  • When are parallel Scrapes better than a Batch?

    For a small number of URLs, parallel /v1/scrapes calls often finish faster. Batches shine for large lists and operational patterns (webhooks, single job id, item cursoring).