Olostep Map endpoint
Get every URL on a website.
Trusted by teams worldwide
From domain to URL list
Discover structure before you scrape or crawl.
Step 1
Send the site URL
POST the homepage or any entry URL you want to discover paths from.
Step 2
Include or exclude paths
Use glob patterns like /blog/** to focus on the sections you care about.
Step 3
Get URLs + paginate
Receive up to large batches per response; use cursor when more URLs are available.
POST /v1/maps
{
"url": "https://docs.olostep.com"
}Built for discovery at scale
The /v1/maps API is tuned for large sites and flexible path rules.
Full-site discovery
Surface URLs beyond a single sitemap file — useful for messy or partial sitemaps.
Glob filters
include_urls and exclude_urls to narrow to products, docs, or blog sections.
top_n control
Cap how many URLs you pull when you only need a sample.
Batch & crawl prep
Feed discovered URLs into scrapes or batches at scale.
What you can build
Map once, then route URLs to scrapes, batches, or internal tools.
SEO audits
Inventory indexable URLs and compare against sitemaps.
Site migrations
Export URL lists for redirects and QA.
AI agents
Give agents a bounded list of pages to read or summarize.
Discover every path
Map a website in one call
Optional include_urls, exclude_urls, top_n, and cursor for pagination.
{
"url": "https://docs.olostep.com"
}{
"urls": [
"https://docs.olostep.com/",
"https://docs.olostep.com/features/scrapes/scrapes",
"…"
],
"url_count": 1240,
"cursor": null
}Frequently asked questions
Everything you need to know about the Map endpoint.