Introduction
A Python application that calls an external API depends on another system’s availability and response time. Even reliable services can experience temporary network failures or periods of high latency.
Timeouts and retries help recover from transient failures. Used carelessly, however, they can increase load, delay responses, and duplicate operations.
Always Set Timeouts
Consider a request using the requests library:
import requestsresponse = requests.get( "https://api.example.com/products", timeout=(3.05, 10),)response.raise_for_status()products = response.json()The tuple configures connection and read timeouts. These are not necessarily a strict upper bound on the entire operation’s total duration.
Without timeouts, a request may occupy a worker much longer than intended. Choose values based on expected latency, caller deadlines, and user experience.
Retry Only Transient Failures
Some errors may resolve when a request is attempted again, such as temporary connection failures or selected server errors.
Other errors are usually permanent:
- Invalid credentials.
- Malformed requests.
- Missing resources.
- Validation failures.
Retrying permanent errors wastes time and can increase pressure on a struggling service.
Add Exponential Backoff
A retry loop can gradually increase the wait between attempts.
import randomimport timeimport requestsdef get_products(url: str, attempts: int = 4): for attempt in range(attempts): try: response = requests.get( url, timeout=(3.05, 10), ) if response.status_code in ( 429, 500, 502, 503, 504 ): response.raise_for_status() response.raise_for_status() return response.json() except requests.RequestException: if attempt == attempts - 1: raise delay = min(8.0, 0.5 * (2 ** attempt)) delay += random.uniform(0, 0.25) time.sleep(delay)This demonstrates exponential backoff with jitter, but it is simplified. It retries every RequestException, including errors that may not be transient, and does not honor the Retry-After header.
Production code should classify errors, respect provider retry guidance, cap total retry duration, and stay within the caller’s deadline.
Avoid Blind Retries for Operations With Side Effects
A GET request is generally expected to be safe to retry. Payment requests and order-creation operations require more care.
A payment provider might process a charge but lose the response. Retrying without protection could create a second charge.
For side-effecting operations, use provider-supported idempotency keys or another reliable deduplication mechanism. Keep the key stable across retries of the same logical operation.
Respect Rate Limits
When an API returns 429 Too Many Requests, it may include a Retry-After header.
Respect that guidance when applicable. Do not respond to rate limiting by immediately sending more requests.
Randomized backoff can also help prevent many workers from retrying simultaneously, a pattern known as a thundering herd.
Reuse HTTP Clients
Repeatedly creating HTTP clients can waste connections and resources. In synchronous applications, a reusable session is often appropriate. In asynchronous applications, use a shared asynchronous client with a clearly defined lifecycle.
HTTPX supports explicit timeout configuration:
import httpxdef fetch_products(): with httpx.Client( timeout=httpx.Timeout( connect=3.0, read=10.0, write=10.0, pool=3.0, ) ) as client: response = client.get( "https://api.example.com/products" ) response.raise_for_status() return response.json()In a long-running application, manage the client at the application or service level instead of recreating it for every request.
Test Failure Scenarios
Test connection timeouts, read timeouts, rate limiting, temporary server errors, malformed responses, exhausted retries, duplicate submissions, and cancellation.
Use mocks or a controlled test server to simulate these conditions without depending on an external provider.
Monitor Retry Behavior
Retries can hide an unhealthy dependency if only final success is measured. Track latency, error rates, retry counts, and exhausted retries.
If retry counts increase sharply, investigate the underlying dependency instead of simply increasing the retry limit.
Conclusion
Timeouts prevent requests from waiting indefinitely, while retries can recover from temporary failures. Neither technique replaces understanding the remote service’s behavior.
Use bounded timeouts, selective retries, exponential backoff with jitter, idempotency for side-effecting operations, and monitoring that reveals when recovery mechanisms are being used too often.
