Click to try
Wait...

One API to Power
GEO, SEO, and AI Visibility Platforms

Everything you need to collect search, AI answer, and web data for modern GEO/SEO products. Trusted by Scaleups and Market Leaders.

Trusted by the best startups startups in the world

Collect Search, AI Answer, and Web Data From One API

Turn the open web into structured data for GEO, SEO, and AI visibility workflows.

Google Search

@olostep/google-search

Extract search results, AI Overviews, knowledge panels, etc. from search results

Brave Search

@olostep/brave-search

Extract search results, AI Overviews, and web page summaries from Brave results

Copilot Search

@olostep/copilot-search

Extract Copilot answers + Search Results from the Microsoft engine

Perplexity Search

@olostep/perplexity-search

Extract the full body answer, citations and sources from a Perplexity response

ChatGPT Search

@olostep/chatgpt-search

Extract the full body answer, citations and sources from a ChatGPT response

Gemini Search

@olostep/gemini-search

Extract the full body answer, citations and sources from a Gemini response

Meta AI

@olostep/meta-ai

Extract the full body answer, citations and sources from a Meta AI response

Grok Search

@olostep/grok-search

Extract the full body answer, citations and sources from a Grok response

Reddit Post

@olostep/reddit-post

Extract posts, comments, upvote counts, and user information from discussions

Or request a new data source - Contact Sales

Built for Developers Shipping GEO and SEO Products

Object-oriented API, native Python and NodeJS SDK clients,
metadata support, webhook events, easy to try and easy to scale

Get clean data from any URL

1# pip install olostep
2from olostep import Olostep
3
4client = Olostep(api_key="YOUR_REAL_KEY")
5
6result = client.scrapes.create(
7    url_to_scrape="https://en.wikipedia.org/wiki/Alexander_the_Great",
8    formats=["markdown", "html"],
9)
10
11print(result.markdown_content)
12print(result.html_content)
1// npm i olostep
2import Olostep from 'olostep'
3
4const client = new Olostep({ apiKey: 'YOUR_REAL_KEY' })
5
6const result = await client.scrapes.create({
7  url: 'https://en.wikipedia.org/wiki/Alexander_the_Great',
8  formats: ['markdown', 'html'],
9})
10
11console.log(result.markdown_content)
12console.log(result.html_content)
1curl -s -X POST "https://api.olostep.com/v1/scrapes" \
2  -H "Authorization: Bearer <YOUR_API_KEY>" \
3  -H "Content-Type: application/json" \
4  -d '{
5    "url_to_scrape": "https://en.wikipedia.org/wiki/Alexander_the_Great",
6    "formats": ["markdown", "html"]
7  }'

Crawl all the subpages

1# pip install olostep
2from olostep import Olostep
3
4client = Olostep(api_key="YOUR_REAL_KEY")
5
6crawl = client.crawls.create(
7    start_url="https://olostep.com",
8    max_pages=100,
9    include_urls=["/**"],
10    exclude_urls=["/collections/**"],
11    include_external=False,
12)
13
14print(crawl.id, crawl.status)
15
16# Wait for completion and iterate pages
17for page in crawl.pages():
18    print(page.url)
19    content = page.retrieve(["markdown"])
20    print(content.markdown_content[:200])
1// npm i olostep
2import Olostep from 'olostep'
3
4const client = new Olostep({ apiKey: 'YOUR_REAL_KEY' })
5
6const crawl = await client.crawls.create({
7  url: 'https://olostep.com',
8  maxPages: 100,
9  includeUrls: ['/**'],
10  excludeUrls: ['/collections/**'],
11  includeExternal: false,
12})
13
14console.log(crawl.id, crawl.status)
15
16// Wait for completion and iterate pages
17for await (const page of crawl.pages()) {
18  console.log(page.url)
19  const content = await client.retrieve({ retrieveId: page.retrieve_id, formats: ['markdown'] })
20  console.log(content.markdown_content.slice(0, 200))
21}
1# Start crawl
2curl -s -X POST "https://api.olostep.com/v1/crawls" \
3  -H "Authorization: Bearer <YOUR_API_KEY>" \
4  -H "Content-Type: application/json" \
5  -d '{
6    "start_url": "https://olostep.com",
7    "max_pages": 100,
8    "include_urls": ["/**"],
9    "exclude_urls": ["/collections/**"],
10    "include_external": false
11  }'
12
13# Check status (replace <CRAWL_ID>)
14curl -s "https://api.olostep.com/v1/crawls/<CRAWL_ID>" \
15  -H "Authorization: Bearer <YOUR_API_KEY>"
16
17# Get pages (replace <CRAWL_ID>)
18curl -s "https://api.olostep.com/v1/crawls/<CRAWL_ID>/pages" \
19  -H "Authorization: Bearer <YOUR_API_KEY>"
20
21# Retrieve content (replace <RETRIEVE_ID>)
22curl -s -G "https://api.olostep.com/v1/retrieve" \
23  -H "Authorization: Bearer <YOUR_API_KEY>" \
24  --data-urlencode "retrieve_id=<RETRIEVE_ID>" \
25  --data-urlencode "formats=markdown"

Get all the URLs on a website

1# pip install olostep
2from olostep import Olostep
3
4client = Olostep(api_key="YOUR_REAL_KEY")
5
6sitemap = client.maps.create(
7    url="https://docs.olostep.com",
8    include_urls=["/features/**"],
9    top_n=100,
10)
11
12print(f"Map ID: {sitemap.id}")
13
14# Iterate all URLs (handles pagination automatically)
15for url in sitemap.urls():
16    print(url)
1// npm i olostep
2import Olostep from 'olostep'
3
4const client = new Olostep({ apiKey: 'YOUR_REAL_KEY' })
5
6const map = await client.maps.create({
7  url: 'https://docs.olostep.com',
8  includeUrls: ['/features/**'],
9  topN: 100,
10})
11
12console.log(`Map ID: ${map.id}`)
13
14// Iterate all URLs (handles pagination automatically)
15for await (const url of map.urls()) {
16  console.log(url)
17}
1curl -s -X POST "https://api.olostep.com/v1/maps" \
2  -H "Authorization: Bearer <YOUR_API_KEY>" \
3  -H "Content-Type: application/json" \
4  -d '{
5    "url": "https://docs.olostep.com",
6    "include_urls": ["/features/**"],
7    "top_n": 100
8  }'

Process up to 10k URLs in one batch. Get results in 5-8 mins

1# pip install olostep
2from olostep import Olostep
3
4client = Olostep(api_key="YOUR_REAL_KEY")
5
6batch = client.batches.create(
7    urls=[
8        {"custom_id": "item-1", "url": "https://www.google.com/search?q=stripe&gl=us&hl=en"},
9        {"custom_id": "item-2", "url": "https://www.google.com/search?q=paddle&gl=us&hl=en"},
10    ],
11    parser="@olostep/google-search",
12)
13
14print(batch.id, batch.status)
15
16# Wait and iterate results (auto-waits for completion)
17for item in batch.items():
18    content = item.retrieve(["json"])
19    print(item.url, item.custom_id)
20    print(content.json_content)
1// npm i olostep
2import Olostep from 'olostep'
3
4const client = new Olostep({ apiKey: 'YOUR_REAL_KEY' })
5
6const batch = await client.batches.create([
7  { url: 'https://www.google.com/search?q=stripe&gl=us&hl=en', customId: 'item-1' },
8  { url: 'https://www.google.com/search?q=paddle&gl=us&hl=en', customId: 'item-2' },
9], {
10  parser: '@olostep/google-search',
11})
12
13console.log(batch.id, batch.total_urls)
14
15// Wait and iterate results (auto-waits for completion)
16for await (const item of batch.items()) {
17  const content = await item.retrieve(['json'])
18  console.log(item.url, item.custom_id)
19  console.log(content.json_content)
20}
1curl -s -X POST "https://api.olostep.com/v1/batches" \
2  -H "Authorization: Bearer <YOUR_API_KEY>" \
3  -H "Content-Type: application/json" \
4  -d '{
5    "items": [
6      {"custom_id": "item-1", "url": "https://www.google.com/search?q=stripe&gl=us&hl=en"},
7      {"custom_id": "item-2", "url": "https://www.google.com/search?q=paddle&gl=us&hl=en"}
8    ],
9    "parser": {"id": "@olostep/google-search"}
10  }'

Semantically search the Web

1# pip install olostep
2from olostep import Olostep
3
4client = Olostep(api_key="YOUR_REAL_KEY")
5
6search = client.searches.create("Latest updates with SpaceX")
7
8print(search.id, len(search.links))
1// npm i olostep
2import Olostep from 'olostep'
3
4const client = new Olostep({ apiKey: 'YOUR_REAL_KEY' })
5
6const search = await client.searches.create('Latest updates with SpaceX')
7
8console.log(search.id, search.links.length)
1curl -s -X POST "https://api.olostep.com/v1/searches" \
2  -H "Authorization: Bearer <YOUR_API_KEY>" \
3  -H "Content-Type: application/json" \
4  -d '{
5    "query": "Latest updates with SpaceX"
6  }'

Get answers from the Web

1# pip install olostep
2from olostep import Olostep
3
4client = Olostep(api_key="YOUR_REAL_KEY")
5
6answer = client.answers.create(
7    task="What does Olostep do and what is its core offering?",
8    json_format={"company": "", "what_it_does": "", "core_offering": ""},
9)
10
11print(answer.json_content)
12print(answer.sources)
1// npm i olostep
2import Olostep from 'olostep'
3
4const client = new Olostep({ apiKey: 'YOUR_REAL_KEY' })
5
6const answer = await client.answers.create({
7  task: 'What does Olostep do and what is its core offering?',
8  jsonFormat: { company: '', what_it_does: '', core_offering: '' },
9})
10
11console.log(answer.json_content)
12console.log(answer.sources)
1curl -s -X POST "https://api.olostep.com/v1/answers" \
2  -H "Authorization: Bearer <YOUR_API_KEY>" \
3  -H "Content-Type: application/json" \
4  -d '{
5    "task": "What does Olostep do and what is its core offering?",
6    "json": {"company": "", "what_it_does": "", "core_offering": ""}
7  }'

Monitor pages on a schedule and get change alerts

1import requests
2import json
3
4API_KEY = "<YOUR_API_KEY>"
5API_URL = "https://api.olostep.com/v1"
6
7# Create a monitor
8payload = {
9    "query": "Alert me when Tesla stock price is above $500",
10    "frequency": "every hour",
11    "email": "alerts@example.com"
12}
13
14headers = {
15    "Authorization": f"Bearer {API_KEY}",
16    "Content-Type": "application/json"
17}
18
19response = requests.post(f"{API_URL}/monitors", headers=headers, json=payload)
20monitor = response.json()
21monitor_id = monitor['id']
22
23print(f"Monitor created: {monitor_id}")
24print(f"Status: {monitor['status']}")
25
26# List all monitors
27monitors = requests.get(f"{API_URL}/monitors", headers=headers).json()
28for m in monitors['monitors']:
29    print(f"{m['id']}: {m['url']} ({m['frequency']})")
30
31# Get monitor details
32details = requests.get(f"{API_URL}/monitors/{monitor_id}", headers=headers).json()
33print(json.dumps(details, indent=2))
34
35# Delete a monitor
36requests.delete(f"{API_URL}/monitors/{monitor_id}", headers=headers)
37print(f"Monitor {monitor_id} deleted")
1const API_URL = 'https://api.olostep.com/v1'
2const headers = { 
3  'Authorization': 'Bearer <YOUR_API_KEY>', 
4  'Content-Type': 'application/json' 
5}
6
7// Create a monitor
8const res = await fetch(`${API_URL}/monitors`, {
9  method: 'POST',
10  headers,
11  body: JSON.stringify({
12    query: 'Alert me when Tesla stock price is above $500',
13    frequency: 'every hour',
14    email: 'alerts@example.com'
15  })
16})
17
18const monitor = await res.json()
19console.log(`Monitor created: ${monitor.id}`)
20console.log(`Status: ${monitor.status}`)
21
22// List all monitors
23const monitors = await fetch(`${API_URL}/monitors`, { headers }).then(r => r.json())
24monitors.monitors.forEach(m => console.log(`${m.id}: ${m.url} (${m.frequency})`))
25
26// Get monitor details
27const details = await fetch(`${API_URL}/monitors/${monitor.id}`, { headers }).then(r => r.json())
28console.log(details)
29
30// Delete a monitor
31await fetch(`${API_URL}/monitors/${monitor.id}`, { method: 'DELETE', headers })
32console.log(`Monitor ${monitor.id} deleted`)
1# Create a monitor
2curl -s -X POST "https://api.olostep.com/v1/monitors" \
3  -H "Authorization: Bearer <YOUR_API_KEY>" \
4  -H "Content-Type: application/json" \
5  -d '{
6    "query": "Track changes in product pricing and stock information",
7    "url": "https://example.com/products/widget-pro",
8    "frequency": "daily",
9    "email": "alerts@example.com"
10  }'
11
12# List all monitors
13curl -s "https://api.olostep.com/v1/monitors" \
14  -H "Authorization: Bearer <YOUR_API_KEY>"
15
16# Get monitor details (replace <MONITOR_ID>)
17curl -s "https://api.olostep.com/v1/monitors/<MONITOR_ID>" \
18  -H "Authorization: Bearer <YOUR_API_KEY>"
19
20# Delete a monitor (replace <MONITOR_ID>)
21curl -s -X DELETE "https://api.olostep.com/v1/monitors/<MONITOR_ID>" \
22  -H "Authorization: Bearer <YOUR_API_KEY>"

One API for Search, AI Answers, and Web Data

Get clean data for GEO, SEO, AI visibility, research, enrichment, and automation workflows without building and maintaining scraping infrastructure.

Reliable Web Data Collection

Get the content you need when you need it. Olostep handles rendering, request processing, proxies, retries, and scalable web access.

Parse PDFs Easily

Parse and output content from web-hosted PDFs, DOCX files, and other documents, converting them into structured, usable formats for analysis.

Automate Web Actions

Click, type, fill forms, scroll, wait, and interact with dynamic websites when static extraction is not enough, enabling complex automated workflows.

Crawls

Crawl websites at scale, collect content from subpages, control depth and URL patterns, and retrieve clean HTML or Markdown for indexing, enrichment, RAG, SEO, and AI workflows.

https://docs.olostep.com/
Depth 1
Depth 2
Depth 3
Crawled/ Included
Excluded by Rules

Maps

Discover every URL on a website using sitemaps and on-page links. Filter by path patterns, paginate large results, and prepare clean URL lists for SEO, crawls, and batches.

https://www.olostep.com/
/docs
/get-started/welcome
/features/maps
/integrations/n8n
....
/blog
/monitors-api
/parsers-vs-llm
/olostep-orthogonal
....
/store
/google-search
/brave-search
/bing-search
....
/api-ref
/scrapes/create
/batches/create
/crawls/create
....

Batches

Process up to 10k concurrent URLs in a single batch in 5-8 mins to get clean web data and aggregate content. Run many batches in parallel to scale to millions of concurrent requests.

Input: URLs
1,000,000
olostep.com/pg-1
olostep.com/pg-2
olostep.com/pg-3
olostep.com/pg-4
olostep.com/pg-5
olostep.com/pg-6
olostep.com/pg-n
Concurrent Batch Processing
Batch 1
Processing
Batch 2
Processing
Batch 3
Processing
Batch n
Processing
URLs Processed
1,000,000
Concurrent Batches
128
Throughput
48,752 /min
Total time
20m 14s

Pricing that Makes Sense

Most cost-effective web data API on the market

No credit card required

Trial

$0
Includes:
500 successful requests
All requests are JS rendered + utilizing residential IP addresses
Low rate limits
Get started
COST/1K $1.800

Starter

$9
/ month
Everything in Free, Plus:
5000 successful requests
150 concurrent requests
Purchase now
COST/1K $0.495

Standard

$99
/ month
Everything in Starter, Plus:
200K successful requests
500 concurrent requests
Purchase now
COST/1K $0.399

Scale

$399
/ month
Everything in Standard, Plus:
1 Million successful requests
AI-powered Browser Automations
Purchase now

Top-ups

Have spiky usage or don't like subscriptions?
You can buy credit packs. They are valid for 6 months.

Credit pack

10k credits

$20
Purchase Credit Pack
Credit pack

250k credits

$200
Purchase Credit Pack
Credit pack

2M credits

$1000
Purchase Credit Pack

Enterprise

Hundreds of millions of credits with enterprise-grade reliability. We offer custom discounts
Contact Sales

Trusted by Amazing Teams Building the Future of AI

Michelle Julia
Co-founder & CEO Aurium

Olostep is the best!!! We automated entire data pipelines with just a prompt

Richard He
Co-founder & CEO Openmart

Olostep has become the default Web Layer infrastructure for our company

Max Brodeur-Urbas
Co-founder & CEO Gumloop

Olostep works like a charm! And your customer service is exceptional

Rob Hayes
Co-founder Merchkit

Olostep lets us turn any website into an API. Great product, great people

Brandon Cohen
Co-founder & CTO CivilGrid

I highly recommend Olostep, great product!

Co-founder & CEO Gedd.it

We verify coupon codes at scale. Love Olostep. It works on any e-commerce

Trevor West
Co-founder & CEO Podqi

Olostep is the best API to search, extract, and structure data from the Web. Happy to be customers

Rida Naveed
Co-founder Zecento

We use /batches combined with parsers and it's magical how we can get structured data at large scale

Kieran V.
Growth PlotsEvents

Olostep allowed us to search and structure events data across the Web

Paul Mit
Founder Foundbase

Reliable and cost-effective API for working with data. Congrats on the cool product

Ready to start?

Get clean data for your AI from any website with Olostep
Most cost-effective API. Built for scale

Are you an AI Agent? Get started here

Frequently asked questions

Product & Capabilities

What is Olostep Web Data API and how does it work for GEO, SEO, and AI products?

Olostep is a developer-first Web Data API designed to power GEO platforms, SEO tools, and AI applications that rely on real-time web data. It combines search, crawling, scraping, monitoring, and structured extraction into a single API so teams can build data pipelines without managing infrastructure.

Instead of building scrapers, proxy layers, and parsers from scratch, you can use Olostep to turn any webpage, search result, or AI-generated answer into clean, structured outputs like JSON or Markdown.

It also includes an Agent layer that lets you automate multi-step workflows using natural language prompts, making it easier to go from manual research to scalable, production-ready pipelines.

What is a Web Data API and why is it important for AI and SEO tools?

A Web Data API is a service that lets developers programmatically collect and structure data from websites at scale. It handles complex tasks like JavaScript rendering, anti-bot bypassing, retries, and parsing so you can focus on building your product.

For SEO tools, GEO platforms, and AI applications, this means you can reliably extract SERP data, AI answers, competitor pages, and web content without maintaining scraping infrastructure.

Why should I use Olostep instead of building my own scraping stack?

Building a reliable web data pipeline requires managing proxies, handling anti-bot systems, maintaining crawlers, and writing parsers. Olostep replaces all of that with a single API that is ready to scale.

You get consistent outputs, built-in retries, structured data formats, and endpoints designed for real use cases like SERP tracking, AI answer extraction, competitor monitoring, and large-scale crawling.

It is also cost-efficient compared to maintaining your own infrastructure and lets you move faster from idea to production.

Who is Olostep built for?

Olostep is built for teams that need structured web data at scale, including AI startups, SEO platforms, GEO tools, data teams, and developers building automation or research workflows.

It is especially useful for use cases like AI visibility tracking, SERP monitoring, competitor intelligence, content extraction, data enrichment, and LLM grounding.

If your product depends on web data, Olostep helps you collect and structure it reliably through one API.

What kind of websites and data sources can Olostep access?

Olostep can access most publicly available websites, including those that require JavaScript rendering. This includes search engines, AI answer engines, content sites, e-commerce pages, PDFs, and discussion platforms.

For advanced use cases involving authentication, cookies, or logged-in sessions, you can contact the team to explore supported configurations.

Can Olostep support my high-volume requests?

Yes, Olostep is built to handle high-volume data extraction at scale, supporting up to billions of requests per month. With features like batch processing, distributed infrastructure and scalable workflows, it is designed for both growing teams and enterprise-level use cases.

Usage & Automation

How does a Web Data API work?

A Web Data API processes requests by rendering web pages, handling anti-bot protections, extracting structured data and returning it in formats such as JSON or Markdown. This removes the need to manage scraping infrastructure manually.

What is the difference between crawling and scraping?

Crawling refers to discovering and navigating multiple pages across a website, while scraping focuses on extracting data from a specific page. A Web Data API typically supports both processes in a unified workflow.

What formats does Olostep return results in?

Most Web Data APIs return data in structured formats such as JSON, as well as HTML, Markdown or raw content depending on the use case. Structured outputs are commonly used for automation and AI workflows.

Can Olostep automate my data pipelines?

Yes, Olostep is designed to support automated data pipelines and research workflows on the web. With capabilities for searching, crawling, scraping, structuring data and running repeatable workflows, it can support a wide range of business and AI use cases.

If you have a specific workflow in mind, contact the team at info@olostep.com or via the Contact Sales page to discuss the best setup for your use case.

Can I extract data with a prompt?

Yes, Olostep lets you extract data using natural language prompts. If you already know the exact page you want to process, you can use the /scrapes endpoint with LLM extraction to describe the data you want returned.

For high-volume or deterministic extraction, Olostep's parsers are the better option, as they return structured JSON more consistently at scale.

For more advanced workflows, such as searching for data, navigating across pages, handling pagination or validating results, the /agents endpoint can automatically carry out multi-step extraction based on your prompt.

What is counted as a request?

One request equals one webpage or one PDF processed. We do not charge separately for bandwidth, proxies, or data usage. All infrastructure costs are included in the price per request.

Does Olostep charge for failed requests?

No, Olostep does not charge for failed requests. You are only billed for successful requests, ensuring predictable and fair usage-based pricing.

For endpoints that involve LLM processing (such as the Answers API), any underlying model costs may still apply. However, Olostep itself only charges for requests that are successfully completed.

Is web scraping legal?

Web scraping is legal in many cases, but depends on how the data is accessed and used. It is important to follow website terms of service, data privacy regulations and applicable laws when extracting web data.

Pricing & Plans

Does Olostep offer a free trial?

Yes, Olostep includes a free plan with 500 requests to help you test the API before upgrading. Paid plans start from $9/month and include 5,000 credits per month.

This gives teams a low-risk way to evaluate Olostep's reliability, scalability and cost-effectiveness before moving to higher-volume usage.

Can I switch plans after signing up?

Yes, you can switch plans at any time. Plans are pro-rated, meaning any unused value from your current plan is carried over to your new plan.

This ensures you don't pay twice for usage you've already covered, giving you flexibility as your needs grow.

Can I ask for a refund if I don't use it?

Yes. If you're not satisfied with the Olostep API or it doesn't end up being useful for your use case, you can email info@olostep.com to request a refund.

If you cancel after a period of non-use, Olostep can also refund the unused portion of your plan where applicable.

How can I pay?

You can pay through Stripe Payment Links. To access billing, you must be logged in to your Olostep account. Go to your dashboard, navigate to Billing & Invoices, click Manage on Stripe, and add your card details. Once your card is added, you’re good to go.