Skip to main content

Web Scraper API documentation

Get to know how Web Scraper API works and integrate it into your app. Examples are provided in Curl, Javascript and Python.

Getting started

Data scraping and parsing endpoint.

https://scrape.infatica.io

Sync API

Sync API allows to receive immediate result using our proxy

If you intend to keep the same IP address across multiple requests, you can use the session_number (integer) parameter to proceed. Your session has a duration of 60 seconds.

If you want to define the geolocation of your session, you may set the country_code (string) parameter with one country code at the creation of the session.

Example: 'us', 'gb', 'fr', 'de', 'jp', 'cn', 'ru'

Supported country codes can be found in the collapsible table below:

All country codes (click here to expand)
CodeCountry nameCodeCountry nameCodeCountry name
adAndorraglGreenlandnoNorway
aeUnited Arab EmiratesgmGambianpNepal
afAfghanistangnGuineanrNauru
agAntigua and BarbudagpGuadeloupenuNiue
aiAnguillagqEquatorial GuineanzNew Zealand
alAlbaniagrGreeceomOman
amArmeniagsSouth Georgia and the South Sandwich IslandspaPanama
aoAngolagtGuatemalapePeru
aqAntarcticaguGuampfFrench Polynesia
arArgentinagwGuinea-BissaupgPapua New Guinea
asAmerican SamoagyGuyanaphPhilippines
atAustriahkHong KongpkPakistan
auAustraliahmHeard Island and McDonald IslandsplPoland
awArubahnHonduraspmSaint Pierre and Miquelon
axÅland IslandshrCroatiapnPitcairn
azAzerbaijanhtHaitiprPuerto Rico
baBosnia and HerzegovinahuHungarypsPalestine, State of
bbBarbadosidIndonesiaptPortugal
bdBangladeshieIrelandpwPalau
beBelgiumilIsraelpyParaguay
bfBurkina FasoimIsle of ManqaQatar
bgBulgariainIndiareRéunion
bhBahrainioBritish Indian Ocean TerritoryroRomania
biBurundiiqIraqrsSerbia
bjBeninirIran (Islamic Republic of)ruRussian Federation
blSaint BarthélemyisIcelandrwRwanda
bmBermudaitItalysaSaudi Arabia
bnBrunei DarussalamjeJerseysbSolomon Islands
boBolivia (Plurinational State of)jmJamaicascSeychelles
bqBonaire, Sint Eustatius and SabajoJordansdSudan
brBraziljpJapanseSweden
bsBahamaskeKenyasgSingapore
btBhutankgKyrgyzstanshSaint Helena, Ascension and Tristan da Cunha
bvBouvet IslandkhCambodiasiSlovenia
bwBotswanakiKiribatisjSvalbard and Jan Mayen
byBelaruskmComorosskSlovakia
bzBelizeknSaint Kitts and NevisslSierra Leone
caCanadakpKorea (Democratic People's Republic of)smSan Marino
ccCocos (Keeling) IslandskrKorea, Republic ofsnSenegal
cdCongo, Democratic Republic of thekwKuwaitsoSomalia
cfCentral African RepublickyCayman IslandssrSuriname
cgCongokzKazakhstanssSouth Sudan
chSwitzerlandlaLao People's Democratic RepublicstSao Tome and Principe
ciCôte d'IvoirelbLebanonsvEl Salvador
ckCook IslandslcSaint LuciasxSint Maarten (Dutch part)
clChileliLiechtensteinsySyrian Arab Republic
cmCameroonlkSri LankaszEswatini
cnChinalrLiberiatcTurks and Caicos Islands
coColombialsLesothotdChad
crCosta RicaltLithuaniatfFrench Southern Territories
cuCubaluLuxembourgtgTogo
cvCabo VerdelvLatviathThailand
cwCuraçaolyLibyatjTajikistan
cxChristmas IslandmaMoroccotkTokelau
cyCyprusmcMonacotlTimor-Leste
czCzechiamdMoldova, Republic oftmTurkmenistan
deGermanymeMontenegrotnTunisia
djDjiboutimfSaint Martin (French part)toTonga
dkDenmarkmgMadagascartrTürkiye
dmDominicamhMarshall IslandsttTrinidad and Tobago
doDominican RepublicmkNorth MacedoniatvTuvalu
dzAlgeriamlMalitwTaiwan, Province of China
ecEcuadormmMyanmartzTanzania, United Republic of
eeEstoniamnMongoliauaUkraine
egEgyptmoMacaougUganda
ehWestern SaharampNorthern Mariana IslandsumUnited States Minor Outlying Islands
erEritreamqMartiniqueusUnited States of America
esSpainmrMauritaniauyUruguay
etEthiopiamsMontserratuzUzbekistan
fiFinlandmtMaltavaHoly See
fjFijimuMauritiusvcSaint Vincent and the Grenadines
fkFalkland Islands (Malvinas)mvMaldivesveVenezuela (Bolivarian Republic of)
fmMicronesia (Federated States of)mwMalawivgVirgin Islands (British)
foFaroe IslandsmxMexicoviVirgin Islands (U.S.)
frFrancemyMalaysiavnViet Nam
gaGabonmzMozambiquevuVanuatu
gbUnited Kingdom of Great Britain and Northern IrelandnaNamibiawfWallis and Futuna
gdGrenadancNew CaledoniawsSamoa
geGeorgianeNigeryeYemen
gfFrench GuiananfNorfolk IslandytMayotte
ggGuernseyngNigeriazaSouth Africa
ghGhananiNicaraguazmZambia
giGibraltarnlNetherlandszwZimbabwe

You can use mobile flag (boolean 'true') to switch the user-agent to the mobile mode

You may also turn on following redirect links (301 code) with follow_redirect parameter and retrying URL not found (404) results using retry_404 parameter.

You're also free to pass your own headers set to the request

Query Parameters

NameDescriptionExampleOptions
url (string, required)Destination url to retrieve (url-encoded){"url":"google.com"}default=null
api_key (string, required)Web Scraper API key{"api_key":"0de32912321"}default=null
mobile (bool, optional)User-Agent type (true for mobile, false for desktop){"mobile":"true"}default=False
follow_redirect (bool, optional)Allow request to follow redirects (301 code){"follow_redirect":"true"}default=False
retry_404 (bool, optional)Retry with another proxy if 404 message returned{"retry_404":"true"}default=False
country_code (str, optional)Proxy country code (geolocation){"country_code":"fr"}default=null, options - us, gb, de, fr, cn, jp, ru
session_number (int, optional)Proxy session number{"session_number":"31"}default=0
render_js (bool, optional)Render JS on page{"render_js":"true"}default=false

Returns

Status codeDescriptionExample
200 (Success)Request successful. Returns JSON with headers and html fields{"headers":{}, 'html':""}
401 (Unauthorized)API key is missing or wrong{'error':'API key is missing or wrong'}
422 (Unprocessable Entity)Error in query parameters{'error':'Wrong query'}
504 (Timeout)Site returned timeout after 3 attempts to reach it{'error':'Timeout'}
API key:

'api_key': 'API_KEY', where API_KEY is your Infatica API key

Curl

curl -X POST "https://scrape.infatica.io/" -H "Content-Type: application/json" -d '{"api_key": "API_KEY", "url": "https://www.google.com"}'

Python

import requests
import json


req = requests.post('https://scrape.infatica.io/', data = json.dumps({
'url': 'TARGET_URL',
'api_key': 'API_KEY',
'mobile': True,
'country_code': 'us',
'session_number': 55
}), headers = {
'user_header_1': 'header1_value',
'user_header_2': 'header2_value'
})

content = json.loads(req.content)
print(content)

Javascript / NodeJS

const axios = require('axios')
const options = {
method: 'POST',
responseType: 'json',
data: {
url: 'TARGET_URL',
api_key: 'API_KEY',
mobile: true,
country_code: 'gb',
session_number: 55
},
url: 'https://scrape.infatica.io'
}

axios(options)
.then((result) => {
console.log(result)
})
.catch((err) => {
console.error(err)
})

Async API

Async API allows to put multiple time-consuming requests to the queue and receive the results as soon as they are getting ready

POST https://scrape.infatica.io/job

Payload parameters

NameDescriptionExampleOptions
url (string, required)Destination url to retrieve (url-encoded){"url":"google.com"}default=null
api_key (string, required)Web Scraper API key{"api_key":"0de32912321"}default=null
mobile (bool, optional)User-Agent type (true for mobile, false for desktop){"mobile":"true"}default=False
follow_redirect (bool, optional)Allow request to follow redirects (301 code){"follow_redirect":"true"}default=False
retry_404 (bool, optional)Retry with another proxy if 404 message returned{"retry_404":"true"}default=False
country_code (str, optional)Proxy country code (geolocation){"country_code":"fr"}default=null, options - us, gb, de, fr, cn, jp, ru
session_number (int, optional)Proxy session number{"session_number":"31"}default=0
render_js (bool, optional)Render JS on page{"render_js":"true"}default=false

Returns

Status codeDescriptionExample
200 (Success)Request successful. Returns JSON with headers and html fields{"id":'result_id'}
401 (Unauthorized)API key is missing or wrong{'error':'API key is missing or wrong'}
422 (Unprocessable Entity)Error in query parameters{'error':'Wrong query'}
504 (Timeout)Site returned timeout after 3 attempts to reach it{'error':'Timeout'}

Curl

curl -X POST -H "Content-Type: application/json" -d '{"api_key": "API_KEY", "url": "http://httpbin.org/ip"}' "https://scrape.infatica.io/job"

Python

import requests
import json

req = requests.post('https://scrape.infatica.io/job', data = json.dumps({
'url': 'TARGET_URL',
'api_key': 'API_KEY',
'mobile': True,
'country_code': 'gb',
'session_number': 55
}), headers = {
'user_header_1': 'header1_value',
'user_header_2': 'header2_value'
})

content = json.loads(req.content)
print(content)

Javascript / NodeJS

const axios = require('axios')
const options = {
method: 'POST',
responseType: 'json',
data: {
url: 'TARGET_URL',
api_key: 'API_KEY',
mobile: true,
country_code: 'gb',
session_number: 55
},
url: 'https://scrape.infatica.io/job'
}

axios(options)
.then((result) => {
console.log(result)
})
.catch((err) => {
console.error(err)
})

Async API - receiving results

POST https://scrape.infatica.io/job/<job_id>

Path Parameters

NameDescriptionExampleOptions
job_id (string, required)Job IDhttps://scrape.infatica.io/job/0de32912321default=null

Payload Parameters

NameDescriptionExampleOptions
api_key (string, required)Web Scraper API key{"api_key":"0de32912321"}default=null

Returns

Status codeDescriptionExample
200 (Success)Request successful. Returns JSON with headers and html fields{
"status":"running",
"statusUrl":"https://scrape.infatica.io/job/0962a8a0-5f1a-4e14-bf8c-5efcc18f1953",
"url":"http://httpbin.org/ip"
}
401 (Unauthorized)API key is missing or wrong{'error':'API key is missing or wrong'}
422 (Unprocessable Entity)Error in query parameters{'error':'Wrong query'}
504 (Timeout)Site returned timeout after 3 attempts to reach it{'error':'Timeout'}

Curl

curl -X POST -H "Content-Type: application/json" -d '{"api_key": "API_KEY"}' "https://scrape.infatica.io/job/<job_id>"

Python

import requests
import json

req = requests.post('https://scrape.infatica.io/job/<job_id>', data = json.dumps({
'api_key': 'API_KEY'}))
content = json.loads(req.content)
print(content)

Javascript / NodeJS

const axios = require('axios')
const options = {
method: 'POST',
responseType: 'json',
data: {
api_key: 'API_KEY',
},
url: 'https://scrape.infatica.io/job/<job_id>'
}

axios(options)
.then((result) => {
console.log(result)
})
.catch((err) => {
console.error(err)
})

Credits and Requests

Your plan determines how many credits you can use. Each request you make costs some credits. The number of credits you use varies based on the domain and parameters of your request. Geotargeting is included in these credit costs.

Domains

We have built special scrapers for some sites. These scrapers will run when you scrape those domains, changing the credit cost. Scraping other domains costs 1 credit (without additional parameters).

CategoryNormalE-commerceSERP
credit cost11010
render_js102020
caution

Normal - any other website if no additional parameters are added;

SERP - Google;

Ecommerce - Amazon;

List will be updated as functionality is added.

These parameters provide you with additional features for the parsing.

{"render_js":"true"} – requests cost 10 credits