/api/scrape
Returns structured text for the main article (or a plain-text fallback): title,
markdown, text_content. No object-storage upload.
Reference
The scrape endpoint uses POST with
Content-Type: application/json.
Pages are loaded in headless Chromium; HTML is parsed with Cheerio (ESM), noise is stripped, then content is converted to Markdown (Turndown).
POST /api/scrape requires
x-api-key (same value as in the Dashboard).
Header
x-api-key: <YOUR_API_KEY>
Never commit keys to public repositories; inject them from environment variables or a secrets manager in production.
/api/scrape
Returns structured text for the main article (or a plain-text fallback): title,
markdown, text_content. No object-storage upload.
Successful JSON response
{
"title": "…",
"markdown": "# …",
"text_content": "…"
}
JSON body for /api/scrape:
| Field | Type | Default | Description |
|---|---|---|---|
| url | string | — (required) | HTTP(S) page to load. |
Errors usually return JSON: { "error": "..." }.
Body failed validation (e.g. missing url, wrong types, or numeric out of range).
Missing x-api-key, or the key is not in the database.
Global rate limit (below) or monthly usage over max_limit (message similar to
Quota exceeded for current period).
Unhandled failure during fetch or extraction. Check server logs.
If the database is not configured, only the demo key may work; real keys need DATABASE_URL.
5 requests per IP per
1 minute (/health is excluded). Returns
429 when exceeded.
usage_count vs.
max_limit; overages return 429.
60s (override with
SCRAPE_GOTO_TIMEOUT_MS or
SCREENSHOT_GOTO_TIMEOUT_MS). Timeouts may surface as 500.
Minimal runnable examples — substitute your base URL and API key.
import axios from "axios";
const base = "https://snapapi.up.railway.app";
const apiKey = process.env.SNAPAPI_KEY;
async function scrape() {
const { data } = await axios.post(
`${base}/api/scrape`,
{ url: "https://example.com" },
{ headers: { "Content-Type": "application/json", "x-api-key": apiKey } }
);
console.log(data.title, data.markdown?.slice(0, 200));
}
scrape().catch(console.error);
import os
import requests
BASE = "https://snapapi.up.railway.app"
API_KEY = os.environ["SNAPAPI_KEY"]
def scrape():
r = requests.post(
f"{BASE}/api/scrape",
json={"url": "https://example.com"},
headers={"x-api-key": API_KEY},
timeout=120,
)
r.raise_for_status()
data = r.json()
print(data["title"], data["markdown"][:200])
if __name__ == "__main__":
scrape()