Testing Code That Calls the X API Without Burning Quota
Testing Code That Calls the X API Without Burning Quota
There are two bad ways to test code that calls an external API. One is to call it for real in every test, which is slow, costs quota, and fails when someone's offline. The other is to mock so thoroughly that the tests pass while the integration is broken.
The way out is specific: record real responses once, replay them forever, and keep exactly one test that talks to the live API. That gives you fast tests that can't drift from reality without you noticing.
There's a free demo endpoint that needs no key, which makes it a good place to capture your first fixtures.
Record fixtures from real responses
Hand-written fixtures encode your assumptions, which is precisely what you're trying to test. Capture real ones instead:
# tools/record_fixtures.py — run this deliberately, not in CI
import json, pathlib, requests
OUT = pathlib.Path("tests/fixtures")
OUT.mkdir(parents=True, exist_ok=True)
CASES = {
"user_info_ok": ("/v1/demo/user/info", {"username": "elonmusk"}),
"user_info_401": ("/v1/user/info", {"username": "elonmusk"}), # no key
}
for name, (path, params) in CASES.items():
r = requests.get(f"https://api.socialapi.tech{path}", params=params, timeout=30)
(OUT / f"{name}.json").write_text(json.dumps({
"status_code": r.status_code,
"body": r.json(),
}, indent=2), encoding="utf-8")
print(f"{name}: {r.status_code}")
Record the status code alongside the body. A fixture that only has the body can't test your error handling, which is the part most likely to be wrong.
Keep the recorder out of CI. It's a tool you run when the API changes, not a test. If it runs automatically, your fixtures silently track the API and stop being a regression check.
Redact before committing. Replace your key with a placeholder in anything captured, and scan before the commit — recorded HTTP is a common way keys end up in repos.
Replay them in tests
import json, pathlib, pytest, requests
FIX = pathlib.Path(__file__).parent / "fixtures"
def load(name):
return json.loads((FIX / f"{name}.json").read_text(encoding="utf-8"))
class FakeResponse:
def __init__(self, status_code, body):
self.status_code, self._body = status_code, body
self.headers = {}
def json(self):
return self._body
def raise_for_status(self):
if self.status_code >= 400:
raise requests.HTTPError(f"{self.status_code}")
@pytest.fixture
def fake_api(monkeypatch):
"""Patch requests.get to return a chosen fixture."""
def use(name):
f = load(name)
monkeypatch.setattr(
requests, "get",
lambda *a, **k: FakeResponse(f["status_code"], f["body"]),
)
return use
def test_parses_follower_count(fake_api):
fake_api("user_info_ok")
user = my_client.get_user("elonmusk")
assert isinstance(user["followers_count"], int)
assert isinstance(user["id"], str) # ← IDs must stay strings
That last assertion is worth its own test. Tweet and user IDs are 19-digit values above the float safe-integer range, so a change that routes them through a numeric type corrupts the last digits silently. Asserting the type catches it in CI instead of in your database — the reasoning is in why IDs belong in TEXT columns.
Test the failures, because that's where the bugs are
Nearly everyone tests the happy path and stops. The bugs live in the other branches:
import pytest
@pytest.mark.parametrize("status,should_retry", [
(429, True), # rate limited — not billed, retry
(500, True),
(502, True),
(402, False), # out of credits — retrying is an infinite loop
(401, False),
(404, False),
(400, False),
])
def test_retry_policy(status, should_retry):
assert my_client.should_retry(status, attempt=0) is should_retry
def test_402_raises_distinctly(fake_api):
"""A 402 must not be swallowed by the generic error handler."""
fake_api("payment_required")
with pytest.raises(OutOfCredits):
my_client.get_user("someone")
def test_timeout_does_not_advance_cursor(monkeypatch):
"""The important one: ambiguous failure must not lose a page."""
monkeypatch.setattr(requests, "get",
lambda *a, **k: (_ for _ in ()).throw(requests.Timeout()))
state = {"cursor": "abc123"}
my_collector.fetch_page(state)
assert state["cursor"] == "abc123" # unchanged
That third test is the one that earns its keep. A timeout means you don't know whether the page was delivered, so advancing the cursor silently skips data — and a skipped page is invisible forever afterwards. It's hard to catch in review and trivial to catch in a test.
Parametrising the retry policy means adding a status code to your table forces a decision about it, rather than letting it fall into a default branch nobody considered. The full policy is in which errors to retry.
Test pagination with two pages, not one
This applies to the endpoints that actually page — followers, followings, retweeters, list members. Timelines return everything up to limit in one call, so there's nothing to walk. For the ones that do, a single-page fixture passes with completely broken pagination. You need at least two, and the second must have no cursor:
def test_pagination_stops_and_collects_all(monkeypatch):
pages = [
{"status_code": 200, "body": {"data": [{"id": "2"}, {"id": "1"}],
"meta": {"next_cursor": "c1"}}},
{"status_code": 200, "body": {"data": [{"id": "0"}],
"meta": {"next_cursor": None}}},
]
calls = []
def fake_get(url, params=None, **k):
calls.append((params or {}).get("cursor"))
p = pages[len(calls) - 1]
return FakeResponse(p["status_code"], p["body"])
monkeypatch.setattr(requests, "get", fake_get)
rows = my_client.paginate("/v1/user/followers", {"username": "x"})
assert len(rows) == 3 # didn't drop page 2
assert calls == [None, "c1"] # passed the cursor back
assert len({r["id"] for r in rows}) == 3 # didn't duplicate
Asserting on calls is what makes this a real test. Checking only the row count passes even if you ignored the cursor and fetched page one twice. Verifying the cursor was passed back is what proves pagination works.
Keep exactly one live test
Mocks can't tell you the API changed. One real call, run outside the normal suite, can:
import os, pytest, requests
@pytest.mark.live
def test_live_contract():
"""Catches upstream shape changes. Not part of the default run."""
r = requests.get("https://api.socialapi.tech/v1/demo/user/info",
params={"username": "elonmusk"}, timeout=30)
assert r.status_code == 200
d = r.json()["data"]
for field in ("id", "username", "followers_count", "created_at"):
assert field in d, f"missing {field}"
assert isinstance(d["id"], str)
assert isinstance(d["followers_count"], int)
# pytest.ini — excluded by default, run on a schedule
[pytest]
markers = live: hits the real API; run nightly, not on every commit
addopts = -m "not live"
Run it nightly rather than per-commit. An upstream change should wake up a scheduled job, not break the build of someone fixing an unrelated typo. And because the demo endpoint needs no key, this test works in CI without secrets at all.
Assert field presence and types, not values. followers_count == 241431332 fails tomorrow for a reason that isn't a bug. The contract you care about is that the field exists and is an integer.
What not to test
Worth saying, because over-testing an integration is its own trap:
- Don't test the provider's correctness. Whether a follower count is accurate isn't your test's job.
- Don't assert on exact counts or specific text. They change hourly.
- Don't test that
requestsworks. Mocking the HTTP library to verify you called it is a tautology. - Don't snapshot entire responses. A new field upstream shouldn't fail your suite; assert on the fields you use.
That last one matters more than it sounds. Full-response snapshots fail on every additive upstream change, so people get used to re-recording them without reading the diff — and then a real breaking change gets re-recorded too.
Our demo endpoint returns a stable real response without a key, which makes it usable as a CI contract check. Signing up with Google includes $0.20 in free credits and no card if you want fixtures from the authenticated endpoints too.
FAQ
How do I test code that calls the X API? Record real responses once as fixtures, replay them in unit tests, and keep a single live test on a schedule to catch upstream changes.
Should I call the real API in my tests? Once, in one test, outside the default run. Every test calling live is slow, costs quota, and breaks when the network does.
Why record fixtures instead of writing them? Because hand-written fixtures encode your assumptions, and your assumptions are what the test is supposed to be checking.
Should the fixture recorder run in CI? No. If it runs automatically your fixtures track the API silently and stop being a regression check. Run it deliberately when the API changes.
What should a fixture contain? Status code and body at minimum. Body-only fixtures can't exercise error handling, which is where most bugs are.
How do I avoid committing my API key in fixtures? Capture from the keyless demo endpoint where possible, replace keys with placeholders, and scan the diff before committing. Recorded HTTP is a common leak path.
What's the most valuable test to write first? That a timeout doesn't advance your cursor. Ambiguous failures that skip a page are invisible afterwards and nearly impossible to spot in review.
Why test that IDs are strings? Because they're 19-digit values above the float safe-integer limit. A refactor that routes them through a numeric type corrupts them with no error, and a type assertion catches it in CI.
How do I test retry logic? Parametrise over status codes with the expected retry decision. Adding a code to the table then forces a decision instead of letting it hit an unconsidered default.
Should a 402 raise a distinct exception? Yes, and test that it does. It needs human intervention, so it must not be caught by the handler that logs and continues.
How many pages do I need to test pagination? At least two, with the second having no cursor. A single-page fixture passes even if pagination is entirely broken.
What should the pagination test assert? That the cursor was passed back on the second call. Row count alone passes even if you fetched page one twice.
How do I mock requests in pytest?
monkeypatch.setattr(requests, "get", ...) with a small fake response class. responses and respx are fine too; the principle is the same.
Is VCR-style recording worth it? It works well and automates capture. The caveat is the same as any recording: scrub credentials, and re-record deliberately rather than whenever tests fail.
Why mark the live test instead of deleting it? Because mocks can't detect that the API changed. One scheduled live test is the only thing standing between you and silently stale fixtures.
What should the live test assert? Field presence and types, never values. Asserting a specific follower count fails tomorrow for a reason that isn't a bug.
How often should the live test run? Nightly is usually right. Per-commit means an upstream change breaks the build of whoever happened to push next.
Can I run a live test without secrets in CI? Yes, if the provider has a keyless demo endpoint. That makes a contract check possible on forks and PRs where secrets aren't available.
Should I snapshot whole responses? No. Additive upstream changes would fail your suite, and people learn to re-record without reading the diff — which eventually re-records a real break.
How do I test rate-limit handling without being rate limited? Return a 429 fixture. You're testing your reaction to the status code, which doesn't require actually hitting a limit.
Do I need a staging environment for this? Not usually. Fixtures plus one live contract test covers it for a read-only integration.
How do I test streaming or webhook code? For webhooks, POST a crafted payload at your handler and assert on signature verification, including a deliberately bad signature — the signing scheme is what you're exercising. For streams, feed your parser a recorded message sequence.
Should I test against the demo endpoint or a paid one? Demo for contract checks and first fixtures, since it's free and stable. Paid endpoints for fixtures of shapes the demo doesn't cover.
What's the right number of tests for an API client? Enough to cover each branch of your error handling plus pagination. That's usually ten to fifteen, and more value than fifty happy-path variations.
What's the most common testing mistake here? Testing only the success path. The integration code that handles 429, 402, 404 and timeouts is both the least tested and the most likely to be wrong.
Record once, replay always, and keep one live test on a schedule — that combination is what stops fixtures going quietly stale. A Google sign-up includes $0.20 in free credits and no card.