Public web data, ready for developers

Web Scraping API for reliable public web data extraction

Send a public URL and receive HTML, Markdown, text, metadata, links, screenshots, or structured JSON without building scraping infrastructure from scratch.

No credit card500 free requestsNo-code friendly

Web Scraping API playground

Scrape a public URL. Inspect the output.

Scrape summary

Example Domain

https://example.com

Status

200

Format

markdown

JavaScript

Standard

Links found

Interactive sample response. Connect an API key to run live requests against permitted public URLs.

# Example Domain

This domain is for use in illustrative examples in documents.

Trusted by the best startups in the world

The problem

Web scraping is powerful, but difficult to run at scale

Modern sites are dynamic, production infrastructure is expensive, and raw HTML rarely matches the data shape your product needs.

Dynamic websites

JavaScript, lazy loading, redirects, and layout changes complicate extraction.

Expensive infrastructure

Queues, retries, browsers, timeouts, monitoring, and validation require engineering.

Raw HTML is not enough

Teams need page text, metadata, links, products, prices, tables, or JSON.

The solution

Turn public web pages into usable data

Fetch, render, and extract public web content for AI enrichment, ecommerce monitoring, SEO audits, market intelligence, and automation.

Choose the output your workflow needs

Retrieve content and context through one endpoint.

HTML responses

LLM-friendly Markdown

Plain page text

Page metadata

Links on page

Hosted content URLs

Screenshots where configured

JSON via parsers or LLM extraction

response.json

{
  "id": "scrape_6h89o8u1kt",
  "url_to_scrape": "https://example.com",
  "result": {
    "markdown_content": "# Example Domain...",
    "html_content": "<html>...</html>",
    "page_metadata": {
      "status_code": 200,
      "title": "Example Domain"
    },
    "links_on_page": [
      "https://www.iana.org/domains/example"
    ]
  }
}

Capabilities

Built for scalable web scraping workflows

Public page fetching

Send a URL and receive page content, response metadata, status information, and discovered links.

JavaScript rendering

Wait for client-side scripts before extracting content from dynamic public pages.

Structured extraction

Use parsers or LLM extraction to transform page content into cleaner JSON fields.

Geo-targeted requests

Retrieve supported regional page variants, pricing, availability, and localized content.

Timeout and retry controls

Configure request behavior for slow or inconsistent pages and production monitoring.

Batch workflows

Run collection jobs across many URLs for enrichment, monitoring, and research pipelines.

Use cases

Use Web Scraping API across your data stack

AI products

Collect fresh public content for agents, RAG, research assistants, and enrichment.

Ecommerce intelligence

Monitor permitted product pages, prices, availability, catalog changes, and reviews.

SEO and growth

Extract titles, descriptions, headings, links, content blocks, and competitor signals.

Data teams

Automate public data collection, normalization, enrichment, monitoring, and reporting.

How it works

From a URL to usable data in three steps

Step 1

Send a public URL

Choose output formats, country, wait timing, and optional extraction settings.

Step 2

Olostep fetches and processes

The API handles retrieval, rendering, response processing, and optional extraction.

Step 3

Use the response

Send content, metadata, links, screenshots, or JSON into your product stack.

Developer experience

Simple for one page. Flexible for production.

Start with a URL and output formats, then add rendering waits, country targeting, parsers, or LLM extraction.

REST APIAPI key authenticationHTML, Markdown, and textCountry targetingTimeout controlsBatch workflows

Read the docs

cURL

curl --request POST \
  --url https://api.olostep.com/v1/scrapes \
  --header 'Authorization: Bearer YOUR_API_KEY' \
  --header 'Content-Type: application/json' \
  --data '{
    "url_to_scrape": "https://example.com",
    "formats": ["markdown", "html"],
    "country": "US",
    "wait_before_scraping": 0
  }'

Build or buy

Build scraper infrastructure or use Olostep?

Focus engineering time on data quality and product workflows instead of browsers, queues, retries, and monitoring.

Requirement

Build internally

Use Olostep

HTTP fetching

Build request infrastructure

Send an API request

JavaScript rendering

Operate browser workers

Configure page timing

Retry handling

Maintain failure logic

Use request controls

Geo-targeting

Run regional infrastructure

Use country parameters

Parsing

Write extraction logic

Return content or JSON

Scaling

Build queues and monitoring

Scale through API plans

Responsible public data workflows

Use scraping lawfully and review website terms, robots.txt guidance, privacy, copyright, rate limits, and applicable laws.

Collect only data you are allowed to access
Do not bypass login walls, paywalls, or access controls
Avoid sensitive personal data without authorization
Use reasonable rates, caching, and governance

Give AI workflows fresh public context

Retrieve current articles, documentation, product listings, and market pages for agents, RAG, enrichment, and research.

Research assistantsRAG collectionMarket researchProduct intelligenceContent enrichment

Pricing

Web Scraping API pricing

Start with 500 free requests, then scale your extraction workflow.

Starter

$9/ month

For testing, prototypes, and small workflows.

Everything in Free
5,000 successful requests
150 concurrent requests

Standard

$99/ month

For production monitoring and recurring extraction.

Everything in Starter
200K successful requests
500 concurrent requests

Scale

$399/ month

For high-volume platforms and enterprise workflows.

Everything in Standard
1 million successful requests
AI-powered browser automations

Confirm current pricing and limits in the dashboard before purchase.

Frequently asked questions

Product & Capabilities

What is a Web Scraping API?

A Web Scraping API lets developers send URLs programmatically and receive page data such as HTML, Markdown, text, screenshots, metadata, links, or structured JSON.

What is the difference between Web Scraping API and Web Search API?

A Web Search API discovers relevant URLs from a query. A Web Scraping API fetches and extracts data from URLs you already know.

What is the difference between Web Scraping API and SERP API?

A SERP API returns structured search-engine result data. A Web Scraping API works across many types of public web pages.

Can I use Web Scraping API for AI workflows?

Yes. It can collect current public content for agents, RAG pipelines, enrichment jobs, research, and summarization.

Usage & Automation

Can Web Scraping API render JavaScript?

Yes. You can configure a wait before extraction for pages where important public content loads after client-side scripts run.

What output formats are available?

The Scrapes endpoint supports formats including Markdown, HTML, text, screenshots, and structured JSON through parsers or LLM extraction.

Can I scrape ecommerce websites?

Yes, for lawful and permitted uses such as public pricing, availability, and catalog monitoring, subject to website terms and applicable laws.

Do I need to manage browsers or scraping infrastructure?

No. Olostep handles the retrieval infrastructure so your team can focus on extraction rules, data quality, and product workflows.

Pricing & Plans

Is web scraping legal?

It depends on the website, data, jurisdiction, terms, privacy rules, copyright, and intended use. Review applicable requirements before collecting or redistributing data.

Can I scrape private or login-protected data?

Only with proper authorization. Do not use this product to bypass login walls, paywalls, access controls, or security protections.

How do I start?

Create an API key, run a request for a permitted public URL, review the response, and connect the endpoint to your workflow.

Collect public web data without maintaining scraping infrastructure

Start with one permitted public URL and connect clean page data to your AI, SEO, ecommerce, research, or data workflow.