Website Crawler API for technical SEO

Inspect status, metadata, internal links and crawl checks across a defined site scope. Connect each issue to an affected URL and verify the result after remediation.

Start building Talk to sales

Technical SEO agent

my-app

Checking page metadata

Page checks · example.com

authjs.dev/category

Updated Example data

3 findings

auth.ts

Checks ready

Instant page analysis starts with a URL. A site crawl uses task-based collection with a target and crawl limits; confirm the scope before submitting a large site.

Crawl pages and inspect technical signals

Inspect response status, page title, description, canonical signals and available content checks. Investigate the affected URL before treating every warning as a ranking issue.

Code search docs

checkout.ts

1 const checkout = await polar. checkout s .create({

2 products: [productId],

3 })

Inspect URLs and crawl websites to investigate metadata, page status, links and technical checks. Use browser-based analysis only where the selected route and site require it.

SERP collection Group crawl issues by type and priority, then check affected URLs after a fix.

Instant page analysis starts with a URL. A site crawl uses task-based collection with a target and crawl limits; confirm the scope before submitting a large site. checkout.ts

example.com Create checkout

12 tests passed

Review metadata, links, and page checks

Use crawl results to review internal links and broken destinations. A blocked or unreachable page may be a crawl limitation rather than a deleted page.

Read workflow example

signup-form.tsx

1 "use client"

2 import { useFormState } from "react-dom"

+ import { useActionState } from "react"

3 export function SignupForm() {

SERP collection react useFormState current docs

react.dev/reference

Deprecated in React 19

useFormState is now useActionState, imported from react.

Turn crawl findings into a clear next step

Sites that rely on JavaScript may need browser-based collection. Rendering and Lighthouse routes have different setup and processing requirements from a basic HTML check.

Explore technical page checks

DataSEO highlights off

authjs.dev/…/providers/google DataSEO highlights

2,400 tokens

Set AUTH_GOOGLE_ID and AUTH_GOOGLE_SECRET.

Add Google to the providers array in auth.ts.

Callback URL: /api/auth/callback/google

Technical SEO grounded in page evidence

Lighthouse is a lab measurement for a particular run. It is not a substitute for real-user Core Web Vitals, and results vary with the page and test conditions.

Read workflow example

On-Page API Example — Illustrative research example. Figures demonstrate the display and

are not a measured benchmark or customer result.

Search, SEO, and market data together

Keep the original crawl settings and compare affected URLs after a fix. Record the status, check name and observation date so the report explains what changed.

Track a consistent keyword set

across locations

and devices.

Compare

current

observations with

dated records

to investigate search visibility and competing pages.

Bring audit data into your existing workflow

Instant page analysis starts with a URL. A site crawl uses task-based collection with a target and crawl limits; confirm the scope before submitting a large site.

See all integrations ( 55 )

Claude

Coffee and Claude time?

DataSEO tools

Inspect response status, page title, description, canonical signals and available content checks. Investigate the affected URL before treating every warning as a ranking issue.

· ↑

ChatGPT

What's on the agenda today?

DataSEO tools

Use crawl results to review internal links and broken destinations. A blocked or unreachable page may be a crawl limitation rather than a deleted page.

· ↑

Common questions

Inspect URLs and crawl websites to investigate metadata, page status, links and technical checks. Use browser-based analysis only where the selected route and site require it.

Instant page analysis starts with a URL. A site crawl uses task-based collection with a target and crawl limits; confirm the scope before submitting a large site.

Inspect response status, page title, description, canonical signals and available content checks. Investigate the affected URL before treating every warning as a ranking issue.

Use crawl results to review internal links and broken destinations. A blocked or unreachable page may be a crawl limitation rather than a deleted page.

Sites that rely on JavaScript may need browser-based collection. Rendering and Lighthouse routes have different setup and processing requirements from a basic HTML check.

Lighthouse is a lab measurement for a particular run. It is not a substitute for real-user Core Web Vitals, and results vary with the page and test conditions.

Keep the original crawl settings and compare affected URLs after a fix. Record the status, check name and observation date so the report explains what changed.

The website uses illustrative sample records. Production API calls use your account and the selected endpoint; example figures are not customer results.

Choose the relevant market, language, target set, required fields and collection cadence. Keep those settings with every response for repeatable analysis.

Product guides

Related use cases

Turn every library into context your agent can use

Request a demo Explore products

FROM IDEA TO FIRST REQUEST

Build your workflow with structured data.

Instant page analysis starts with a URL. A site crawl uses task-based collection with a target and crawl limits; confirm the scope before submitting a large site.

Read the docs ↗Explore On-Page API →

NODE.JS

// Server-side example; configure your deployment and credentials.
const url = new URL("/v3/on_page/instant_pages", process.env.DATASEO_API_ORIGIN);
const auth = Buffer.from(`${process.env.DATASEO_LOGIN}:${process.env.DATASEO_PASSWORD}`).toString("base64");
const response = await fetch(url, {
  method: "POST",
  headers: {Authorization: `Basic ${auth}`, "Content-Type": "application/json"},
  body: JSON.stringify([{"url": "https://example.com"}])
});
if (!response.ok) throw new Error(`HTTP ${response.status}`);
const data = await response.json();
for (const task of data.tasks ?? []) {
  if (task.status_code !== 20000) throw new Error(task.status_message);
}
// Use task.result only after checking task status.

Website Crawler API for technical SEO

Crawl pages and inspect technical signals

Review metadata, links, and page checks

Turn crawl findings into a clear next step

Technical SEO grounded in page evidence

Search, SEO, and market data together

Bring audit data into your existing workflow

Common questions

What does this product provide?

Choose a URL or a crawl?

Review status and metadata?

Inspect the link structure?

Choose rendering deliberately?

Separate lab and field performance?

Verify after remediation?

Are these live demonstration results?

How do I choose a collection scope?

Product guides

Related use cases

Turn every library into context your agent can use