Making API Calls in Python: Authentication, Errors, and Proxies

Denis Kryukov 18 Jun 2025 8 min read

Interacting with APIs in Python opens up countless possibilities – from accessing live market data to automating reports and building powerful applications. Whether you're running a one-off script or building an automated system, the key to success is understanding both the technical and operational aspects of API access – and planning for scale, limits, and edge cases. In this article, we’ll learn how APIs work, how to send requests with Python, and how to authenticate properly and handle errors.

Understanding APIs and HTTP Requests

APIs – short for Application Programming Interfaces – are the bridges that allow different software systems to talk to one another. Whether you're retrieving weather data, interacting with a payment gateway, or scraping product details from an e-commerce site, APIs provide structured ways to request and exchange data.

HTTP protocol is at the core of most APIs on the web; it defines how clients (like your Python script) interact with servers. These interactions revolve around standard HTTP methods:

• GET – Request data from a resource (most common).
• POST – Submit data to a server (e.g., form submission).
• PUT / PATCH – Update an existing resource.
• DELETE – Remove a resource.

Each API exposes a set of endpoints, typically URLs, that correspond to specific functions – like /users, /products, or /search. In Python, the most popular way to make these HTTP requests is with the requests library. It's simple, flexible, and widely supported. Here’s a quick example of a basic GET request:

import requests

response = requests.get("https://api.example.com/data")
print(response.status_code)      # Check the status
print(response.json())           # Parse JSON response

This response often includes more than just data – it comes with headers, status codes, and sometimes rate-limiting information, all of which are important for writing resilient code.

Making Your First API Call in Python

Let’s walk through making an actual API call using Python. We’ll use the requests library, which provides a simple interface for sending HTTP requests and handling responses. To start, make sure it's installed:

pip install requests

Let’s query a free, public API – for example, the JSONPlaceholder service, which mimics a REST API for testing.

import requests

url = "https://jsonplaceholder.typicode.com/posts/1"
response = requests.get(url)

print("Status code:", response.status_code)
print("Response JSON:", response.json())

This script sends a GET request to retrieve the post with ID 1. If the request is successful (status code 200), the API responds with a JSON object that looks like this:

{
  "userId": 1,
  "id": 1,
  "title": "sunt aut facere repellat provident occaecati excepturi optio reprehenderit",
  "body": "quia et suscipit..."
}

You can access individual parts of the response like this:

data = response.json()
print("Title:", data['title'])

Dealing With Authentication

Most production APIs aren’t entirely open – they require authentication to identify who’s making a request and whether they’re authorized. This ensures fair use, protects user data, and prevents abuse. There are several common authentication methods you'll encounter when using APIs:

1. API Keys (most common)

Many APIs provide a unique API key when you register. You typically include this key as a query parameter or in a request header. Here's a header-based example:

import requests

url = "https://api.example.com/data"
headers = {
    "Authorization": "Bearer YOUR_API_KEY"
}

response = requests.get(url, headers=headers)
print(response.json())

Some APIs use a custom header like X-API-Key instead of Authorization.

Query parameter example (less secure):

url = "https://api.example.com/data?api_key=YOUR_API_KEY"
response = requests.get(url)

⚠️ Passing keys in URLs is less secure because they can be logged or cached – headers are the preferred method.

Basic Authentication

This uses a username and password, typically for internal services or legacy APIs.

from requests.auth import HTTPBasicAuth

response = requests.get(
    "https://api.example.com/secure-data",
    auth=HTTPBasicAuth("username", "password")
)

OAuth 2.0 (used by Google, Twitter, etc.)

OAuth is more complex and usually involves several steps to obtain an access token, which is then used like a bearer token. This is common when accessing user data from large platforms. For example:

headers = {
    "Authorization": "Bearer ACCESS_TOKEN"
}
response = requests.get("https://api.platform.com/userinfo", headers=headers)

Security Tip

Never hardcode your API keys in scripts. Use environment variables or configuration files that are excluded from version control.

import os

api_key = os.getenv("MY_API_KEY")

Handling Errors and Rate Limits

When working with APIs, things don’t always go as planned. Servers may be down, requests may be malformed, or you might hit a rate limit. Writing resilient code means anticipating these situations and handling them gracefully.

Understanding HTTP Status Codes

APIs return status codes to indicate whether your request succeeded or failed. Here are some common ones:

• 200 OK – Success!
• 400 Bad Request – The server couldn't understand the request (check your parameters).
• 401 Unauthorized – Missing or invalid authentication.
• 403 Forbidden – You’re authenticated, but don’t have permission.
• 404 Not Found – The requested resource doesn’t exist.
• 429 Too Many Requests – You’ve hit the API’s rate limit.
• 500+ – Server errors; usually not your fault.

How to Solve Proxy Error Codes | Infatica

Errors make scraping unbearably difficult and, in most cases, impossible to execute. Often, HTTP errors happen due to the incorrectly rotated proxies. In this article, we will take a look at all HTTP error codes you can get when scraping.

Jan WiśniewskiInfatica Blog: Learn How to Harness the Power of Data

You can check these codes in your Python code:

if response.status_code == 200:
    data = response.json()
else:
    print(f"Error {response.status_code}: {response.text}")

Retry Logic

Sometimes, requests fail due to temporary issues – like network hiccups or server overloads. Rather than give up, it's best to retry with a short delay. Here’s a simple example with retries:

import time
import requests

url = "https://api.example.com/data"
retries = 3

for i in range(retries):
    response = requests.get(url)
    if response.status_code == 200:
        print("Success!")
        break
    else:
        print(f"Attempt {i+1} failed. Retrying...")
        time.sleep(2)

For production use, consider using libraries like tenacity or urllib3’s built-in retry adapter.

Respecting Rate Limits

Most APIs limit the number of requests you can send in a time period – for example, 100 requests per minute. Exceeding this can lead to throttling or temporary bans. Best practices:

• Check headers: APIs often return rate-limit information in response headers (e.g., X-RateLimit-Remaining).
• Introduce delays: Use time.sleep() or asyncio to pace your requests.
• Batch requests when possible.
• Use pagination wisely to avoid fetching too much data at once.

Using Proxies with Python Requests

Some APIs impose restrictions not just on how often you call them, but from where. Others limit the number of requests per IP, block non-local traffic, or apply stricter throttling for anonymous users. To overcome these challenges, proxies can be a powerful tool.

What Are Proxies?

A proxy server acts as an intermediary between your machine and the internet. Instead of sending a request directly to the API, your request is routed through a proxy, which makes the call on your behalf. The API sees the proxy’s IP, not yours. Proxies are useful for avoiding IP-based rate limits, accessing geo-restricted APIs, rotating identities across large request volumes, and enhancing privacy and testing from different locations.

How to Use a Proxy with requests

Python’s requests library supports proxies via the proxies argument. Here's a basic example using an HTTP proxy:

import requests

proxies = {
    "http": "http://username:password@proxy.example.com:8000",
    "https": "http://username:password@proxy.example.com:8000"
}

response = requests.get("https://api.example.com/data", proxies=proxies)
print(response.json())

If you're using an Infatica proxy, simply plug in the endpoint and authentication details you've received.

Rotating Proxies

Rotating proxies – like Infatica’s residential and datacenter proxy pools – can assign a new IP for each request or at fixed intervals. This helps avoid detection and blocks when scraping or testing high-volume endpoints. A simple rotation example (manual rotation shown below; automation is also possible with third-party libraries):

import random

proxy_list = [
    "http://user:pass@proxy1.example.com:8000",
    "http://user:pass@proxy2.example.com:8000",
    "http://user:pass@proxy3.example.com:8000"
]

proxy = random.choice(proxy_list)

response = requests.get("https://api.example.com/data", proxies={"http": proxy, "https": proxy})

Master Proxy Rotation with Python’s Requests Library for Efficient Web Scraping

Let’s explore the basics and advanced techniques of using proxies with Python requests – learn to rotate proxies, handle failures, and manage SOCKS5 proxies for web scraping!

Jan WiśniewskiInfatica Blog: Learn How to Harness the Power of Data

Tips for Using Proxies Successfully

• Choose the right type: Use datacenter proxies for speed, and residential proxies for higher trust levels.
• Set timeouts: Proxies may introduce delays – use timeout= in your requests.
• Monitor performance: Log failed requests and retry when needed.
• Respect API rules: Even with proxies, it’s best to follow usage guidelines and avoid overwhelming the server.

Best Practices for API Interactions

Once you're comfortable making API calls and handling authentication, errors, and proxies, it's time to optimize your workflow. Following best practices helps ensure your code is efficient, respectful of service limits, and ready for production.

Use Session Objects

Instead of making one-off requests, use a Session object to persist headers, cookies, and connection settings across multiple requests.

import requests

session = requests.Session()
session.headers.update({"Authorization": "Bearer YOUR_TOKEN"})

response = session.get("https://api.example.com/data")

This improves performance and simplifies repeat calls to the same API.

Respect Rate Limits and API Terms

Even with proxies, it's important to limit request frequency, pause between calls when required, and monitor for 429 Too Many Requests responses. If the API offers usage guidelines, follow them – especially for public or partner APIs.

Handle Failures Gracefully

Use retry logic, fallback behavior, and clear error logging. Consider exponential backoff, catching exceptions (e.g., requests.exceptions.RequestException), or retrying only on retryable errors (not on 400 or 401):

try:
    response = session.get(url, timeout=10)
    response.raise_for_status()
except requests.exceptions.HTTPError as errh:
    print("HTTP Error:", errh)
except requests.exceptions.ConnectionError as errc:
    print("Connection Error:", errc)

Sanitize and Validate Inputs

If your script accepts user input (e.g., from a form or CLI), always sanitize it before adding it to a request URL or body. This helps avoid malformed queries or potential injection attacks.

Secure Your API Keys

Never commit API keys or credentials to source control. Use environment variables or encrypted secret managers.

import os

api_key = os.getenv("API_KEY")

Frequently Asked Questions

Denis Kryukov

Denis Kryukov is using his data journalism skills to document how liberal arts and technology intertwine and change our society

• Tallinn, Estonia