Skip to main content

Web Scraper API documentation

Get to know how Web Scraper API works and integrate it into your app.

Authentication​

All requests must include an X-API-Key header with a valid API key. For example:

X-API-Key: YOUR_SECRET_API_KEY

Scraper and API endpoint​

POST https://scrape.infatica.io/

  • Purpose: Fetches a webpage at a given URL, optionally using a specified country proxy and/or language headers.
  • Auth: Requires X-API-Key in the request header.
  • Requests: Each call consumes 1 request.

Features:

  • Ability to specify ISO 3166-1 country code (e.g. "US", "GB", "DE") to use a localized proxy.
  • Optionally set language (e.g., "en-US", "de-DE") to shape the Accept-Language header.
  • Return raw HTML (return_html = true) or a JSON structure (return_html = false, default).

Request body (JSON):​

FieldTypeRequired?Description
urlstringyesThe URL to fetch (e.g. "https://www.example.com")
countrystringnoISO 3166-1 alpha-2 code (e.g. "US", "GB", "DE"). Uses localized proxy
languagestringnoLanguage code in "xx-YY" format (e.g., "en-US", "de-DE")
return_htmlbooleannotrue returns raw HTML; false returns JSON with HTML sample (default)

Example cURL Request:​

curl -X POST "https://scrape.infatica.io/" \
  -H "Content-Type: application/json" \
  -H "X-API-Key: YOUR_SECRET_API_KEY" \
  -d '{
    "url": "https://www.example.com",
    "country": "US",
  }'

Success Response​

HTTP Status: 200 OK

If return_html = false (default):

{
  "status_code": 200,
  "requests_used": 1,
  "time_spent_ms": 123,
  "api_key": "YOUR_SECRET_API_KEY",
  "timestamp": "2025-03-25T12:00:00.123456",
  "country": "US",
  "language": "en-US",
  "data": {
    "html": "Base64 encoded HTML"
  }
}

If return_html = true, the response is raw HTML:

<!DOCTYPE html>
<html>
<head>...</head>
<body>Full HTML of the requested page</body>
</html>

Google SERP endpoint​

Advanced Google SERP scraping endpoint with auto-retry, parallel page fetching, and Google AI Overview support.

Endpoint: POST https://scrape.infatica.io/serp
Auth: X-API-Key header (Advanced plan required)

Parameters​

ParameterTypeDefaultDescription
urlstringrequiredGoogle search URL
resultsint10Total results to fetch (max 100)
pagesstringnullSpecific pages as CSV (e.g., "1,3,5", max 10 pages)
modestring"scrape""scrape" or "render" (forces browser + Google AI)
countrystringnullISO 3166-1 country code
return_htmlboolfalseReturn raw HTML instead of JSON

Note: pages takes precedence over results if both specified.

Response​

{
  "status_code": 200,
  "requests_used": 45,
  "elapsed_time": 3500,
  "html": "base64_encoded_html"
}

Behavior​

  1. First request: scrape mode (fast) → auto-switches to render if blocked
  2. Render mode: forces browser execution + enables scrape_google_ai=true for AI Overview
  3. Retry logic: up to 3 attempts on bans
  4. Additional pages: fetched in parallel (scrape mode, 1 request each)
  5. HTML: all pages concatenated

Examples​

Basic (10 results)​

curl -X POST "https://scrape.infatica.io/serp" \
  -H "X-API-Key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://www.google.com/search?q=python+scraping"}'

Multiple pages (50 results)​

curl -X POST "https://scrape.infatica.io/serp" \
  -H "X-API-Key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://www.google.com/search?q=machine+learning",
    "results": 50
  }'

Specific pages only​

curl -X POST "https://scrape.infatica.io/serp" \
  -H "X-API-Key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://www.google.com/search?q=ai",
    "pages": "1,3,5"
  }'

Force render + Google AI Overview​

curl -X POST "https://scrape.infatica.io/serp" \
  -H "X-API-Key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://www.google.com/search?q=climate+change",
    "mode": "render",
    "results": 20
  }'
curl -X POST "https://scrape.infatica.io/serp" \
  -H "X-API-Key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://www.google.com/search?q=restaurants",
    "country": "DE",
    "results": 30
  }'

Decode Base64 HTML​

Python:

import base64, json
html = base64.b64decode(response["html"]).decode("utf-8")

Node.js:

const html = Buffer.from(response.html, 'base64').toString('utf-8');

Bash:

echo "$RESPONSE_HTML" | base64 -d

Pricing​

  • scrape: 1 request per page
  • Additional pages: 1 request each (scrape mode)

Examples:

  • 10 results (1 page), scrape: ~1 request
  • 50 results (5 pages), scrape: ~5 requests

Error Codes​

CodeReasonSolution
400Invalid payloadCheck JSON format
403Advanced plan requiredUpgrade plan
429Rate limit exceededReduce request frequency
500All retries failedTry render mode or adjust params

Limits​

  • Max results: 100
  • Max pages: 10

Tips​

  • Use pages for selective scraping
  • Use render mode when you need Google AI Overview

AI Search Endpoints​

Query AI-powered search engines (Gemini and Perplexity) and receive HTML responses.

Endpoints:

  • POST https://scrape.infatica.io/gemini
  • POST https://scrape.infatica.io/perplexity

Auth: X-API-Key header (Advanced plan required)

Parameters​

ParameterTypeRequiredDescription
querystringyesThe search query to send to the AI service
return_htmlbooleannotrue returns raw HTML; false returns JSON (default)

Example: Gemini​

curl -X POST "https://scrape.infatica.io/gemini" \
  -H "Content-Type: application/json" \
  -H "X-API-Key: YOUR_API_KEY" \
  -d '{"query": "audi vs bmw"}'

Example: Perplexity​

curl -X POST "https://scrape.infatica.io/perplexity" \
  -H "Content-Type: application/json" \
  -H "X-API-Key: YOUR_API_KEY" \
  -d '{"query": "best programming languages 2026"}'

Example: Raw HTML Response​

curl -X POST "https://scrape.infatica.io/gemini" \
  -H "Content-Type: application/json" \
  -H "X-API-Key: YOUR_API_KEY" \
  -d '{"query": "climate change effects", "return_html": true}'

Success Response​

HTTP Status: 200 OK

If return_html = false (default):

{
  "api_key": "YOUR_API_KEY",
  "query": "audi vs bmw",
  "created_at": "2025-01-21T12:00:00.123456",
  "processed_at": "2025-01-21T12:00:02.456789",
  "time_spent_ms": 2333,
  "requests_used": 1,
  "html": "<html>...response content...</html>"
}

If return_html = true, the response is raw HTML from the AI service.

Error Codes​

CodeReasonSolution
403Advanced plan requiredUpgrade your plan
500No html_content in responseAI service issue, retry later

Notes​

  • You are charged only for successful responses
  • Response times may vary based on query complexity

Render JS endpoint​

POST https://scrape.infatica.io/render

  • Purpose: Fetches a webpage at a given URL and renders it via browser, optionally using a specified country proxy and/or language headers.
  • Auth: Requires X-API-Key in the request header.
  • Requests: Each call consumes 1 request.

Features:

  • Ability to specify ISO 3166-1 country code (e.g. "US", "GB", "DE") to use a localized proxy.
  • Optionally set language (e.g., "en-US", "de-DE") to shape the Accept-Language header.
  • Return raw HTML (return_html = true) or a JSON structure (return_html = false, default).

Request body (JSON):​

FieldTypeRequired?Description
urlstringyesThe URL to fetch (e.g. "https://www.example.com")
countrystringnoISO 3166-1 alpha-2 code (e.g. "US", "GB", "DE"). Uses localized proxy
languagestringnoLanguage code in "xx-YY" format (e.g., "en-US", "de-DE")
return_htmlbooleannotrue returns raw HTML; false returns JSON with HTML sample (default)

Example cURL Request:​

curl -X POST "https://scrape.infatica.io/render" \
  -H "Content-Type: application/json" \
  -H "X-API-Key: YOUR_SECRET_API_KEY" \
  -d '{
    "url": "https://www.example.com",
    "country": "US",
  }'

Success Response​

HTTP Status: 200 OK

If return_html = false (default):

{
  "status_code": 200,
  "requests_used": 1,
  "time_spent_ms": 123,
  "api_key": "YOUR_SECRET_API_KEY",
  "timestamp": "2025-03-25T12:00:00.123456",
  "country": "US",
  "language": "en-US",
  "data": {
    "html": "Base64 encoded HTML"
  }
}

If return_html = true, the response is raw HTML:

<!DOCTYPE html>
<html>
<head>...</head>
<body>Full HTML of the requested page</body>
</html>

Possible Error Responses​

HTTP Status Codes​

Status CodeCause
401Invalid API key; Not enough prepaid requests; Missing API key
403The account is suspended
400The provided country is not a valid ISO 3166-1 code
422The request body fails validation
500Server errors or unexpected issues

Country codes​

If you want to define the geolocation for your request, you may set the country code (string) parameter with one country code at the creation of the request.

Supported country codes can be found in the collapsible table below:

Where US is a two-letter uppercase country code according to ISO 3166-1:

Country codes (click here to expand)
CodeCountry nameCodeCountry nameCodeCountry name
ADAndorraGLGreenlandNONorway
AEUnited Arab EmiratesGMGambiaNPNepal
AFAfghanistanGNGuineaNRNauru
AGAntigua and BarbudaGPGuadeloupeNUNiue
AIAnguillaGQEquatorial GuineaNZNew Zealand
ALAlbaniaGRGreeceOMOman
AMArmeniaGSSouth Georgia and the South Sandwich IslandsPAPanama
AOAngolaGTGuatemalaPEPeru
AQAntarcticaGUGuamPFFrench Polynesia
ARArgentinaGWGuinea-BissauPGPapua New Guinea
ASAmerican SamoaGYGuyanaPHPhilippines
ATAustriaHKHong KongPKPakistan
AUAustraliaHMHeard Island and McDonald IslandsPLPoland
AWArubaHNHondurasPMSaint Pierre and Miquelon
AXÅland IslandsHRCroatiaPNPitcairn
AZAzerbaijanHTHaitiPRPuerto Rico
BABosnia and HerzegovinaHUHungaryPSPalestine, State of
BBBarbadosIDIndonesiaPTPortugal
BDBangladeshIEIrelandPWPalau
BEBelgiumILIsraelPYParaguay
BFBurkina FasoIMIsle of ManQAQatar
BGBulgariaINIndiaRERéunion
BHBahrainIOBritish Indian Ocean TerritoryRORomania
BIBurundiIQIraqRSSerbia
BJBeninIRIran (Islamic Republic of)RURussian Federation
BLSaint BarthélemyISIcelandRWRwanda
BMBermudaITItalySASaudi Arabia
BNBrunei DarussalamJEJerseySBSolomon Islands
BOBolivia (Plurinational State of)JMJamaicaSCSeychelles
BQBonaire, Sint Eustatius and SabaJOJordanSDSudan
BRBrazilJPJapanSESweden
BSBahamasKEKenyaSGSingapore
BTBhutanKGKyrgyzstanSHSaint Helena, Ascension and Tristan da Cunha
BVBouvet IslandKHCambodiaSISlovenia
BWBotswanaKIKiribatiSJSvalbard and Jan Mayen
BYBelarusKMComorosSKSlovakia
BZBelizeKNSaint Kitts and NevisSLSierra Leone
CACanadaKPKorea (Democratic People's Republic of)SMSan Marino
CCCocos (Keeling) IslandsKRKorea, Republic ofSNSenegal
CDCongo, Democratic Republic of theKWKuwaitSOSomalia
CFCentral African RepublicKYCayman IslandsSRSuriname
CGCongoKZKazakhstanSSSouth Sudan
CHSwitzerlandLALao People's Democratic RepublicSTSao Tome and Principe
CICôte d'IvoireLBLebanonSVEl Salvador
CKCook IslandsLCSaint LuciaSXSint Maarten (Dutch part)
CLChileLILiechtensteinSYSyrian Arab Republic
CMCameroonLKSri LankaSZEswatini
CNChinaLRLiberiaTCTurks and Caicos Islands
COColombiaLSLesothoTDChad
CRCosta RicaLTLithuaniaTFFrench Southern Territories
CUCubaLULuxembourgTGTogo
CVCabo VerdeLVLatviaTHThailand
CWCuraçaoLYLibyaTJTajikistan
CXChristmas IslandMAMoroccoTKTokelau
CYCyprusMCMonacoTLTimor-Leste
CZCzechiaMDMoldova, Republic ofTMTurkmenistan
DEGermanyMEMontenegroTNTunisia
DJDjiboutiMFSaint Martin (French part)TOTonga
DKDenmarkMGMadagascarTRTürkiye
DMDominicaMHMarshall IslandsTTTrinidad and Tobago
DODominican RepublicMKNorth MacedoniaTVTuvalu
DZAlgeriaMLMaliTWTaiwan, Province of China
ECEcuadorMMMyanmarTZTanzania, United Republic of
EEEstoniaMNMongoliaUAUkraine
EGEgyptMOMacaoUGUganda
EHWestern SaharaMPNorthern Mariana IslandsUMUnited States Minor Outlying Islands
EREritreaMQMartiniqueUSUnited States of America
ESSpainMRMauritaniaUYUruguay
ETEthiopiaMSMontserratUZUzbekistan
FIFinlandMTMaltaVAHoly See
FJFijiMUMauritiusVCSaint Vincent and the Grenadines
FKFalkland Islands (Malvinas)MVMaldivesVEVenezuela (Bolivarian Republic of)
FMMicronesia (Federated States of)MWMalawiVGVirgin Islands (British)
FOFaroe IslandsMXMexicoVIVirgin Islands (U.S.)
FRFranceMYMalaysiaVNViet Nam
GAGabonMZMozambiqueVUVanuatu
GBUnited Kingdom of Great Britain and Northern IrelandNANamibiaWFWallis and Futuna
GDGrenadaNCNew CaledoniaWSSamoa
GEGeorgiaNENigerYEYemen
GFFrench GuianaNFNorfolk IslandYTMayotte
GGGuernseyNGNigeriaZASouth Africa
GHGhanaNINicaraguaZMZambia
GIGibraltarNLNetherlandsZWZimbabwe

Language codes​

If you want to define the language in your request, you may set the language (string) parameter with one language tag at the creation of the request.

Language codes can be found in the collapsible table below:

All language codes/tags (click here to expand)
Language TagLanguageRegionDescription
ar-SAArabicSaudi ArabiaArabic (Saudi Arabia)
bn-BDBanglaBangladeshBangla (Bangladesh)
bn-INBanglaIndiaBangla (India)
cs-CZCzechCzech RepublicCzech (Czech Republic)
da-DKDanishDenmarkDanish (Denmark)
de-ATGermanAustriaAustrian German
de-CHGermanSwitzerland"Swiss" German
de-DEGermanGermanyStandard German (as spoken in Germany)
el-GRGreekGreeceModern Greek
en-AUEnglishAustraliaAustralian English
en-CAEnglishCanadaCanadian English
en-GBEnglishUnited KingdomBritish English
en-IEEnglishIrelandIrish English
en-INEnglishIndiaIndian English
en-NZEnglishNew ZealandNew Zealand English
en-USEnglishUnited StatesUS English
en-ZAEnglishSouth AfricaEnglish (South Africa)
es-ARSpanishArgentinaArgentine Spanish
es-CLSpanishChileChilean Spanish
es-COSpanishColumbiaColombian Spanish
es-ESSpanishSpainCastilian Spanish (as spoken in Central-Northern Spain)
es-MXSpanishMexicoMexican Spanish
es-USSpanishUnited StatesAmerican Spanish
fi-FIFinnishFinlandFinnish (Finland)
fr-BEFrenchBelgiumBelgian French
fr-CAFrenchCanadaCanadian French
fr-CHFrenchSwitzerland"Swiss" French
fr-FRFrenchFranceStandard French (especially in France)
he-ILHebrewIsraelHebrew (Israel)
hi-INHindiIndiaHindi (India)
hu-HUHungarianHungaryHungarian (Hungary)
id-IDIndonesianIndonesiaIndonesian (Indonesia)
it-CHItalianSwitzerland"Swiss" Italian
it-ITItalianItalyStandard Italian (as spoken in Italy)
ja-JPJapaneseJapanJapanese (Japan)
ko-KRKoreanRepublic of KoreaKorean (Republic of Korea)
nl-BEDutchBelgiumBelgian Dutch
nl-NLDutchThe NetherlandsStandard Dutch (as spoken in The Netherlands)
no-NONorwegianNorwayNorwegian (Norway)
pl-PLPolishPolandPolish (Poland)
pt-BRPortugeseBrazilBrazilian Portuguese
pt-PTPortugesePortugalEuropean Portuguese (as written and spoken in Portugal)
ro-RORomanianRomaniaRomanian (Romania)
ru-RURussianRussian FederationRussian (Russian Federation)
sk-SKSlovakSlovakiaSlovak (Slovakia)
sv-SESwedishSwedenSwedish (Sweden)
ta-INTamilIndiaIndian Tamil
ta-LKTamilSri LankaSri Lankan Tamil
th-THThaiThailandThai (Thailand)
tr-TRTurkishTurkeyTurkish (Turkey)
zh-CNChineseChinaMainland China, simplified characters
zh-HKChineseHond KongHong Kong, traditional characters
zh-TWChineseTaiwanTaiwan, traditional characters