Your snscrape Script Stopped Working. Here's the Migration

November 8, 202613 min readSocialAPI Engineering

Your snscrape Script Stopped Working. Here's the Migration

It worked for months. You didn't touch it. Now it returns an empty list, or it throws somewhere deep inside the library, and nothing in your code changed.

This is the normal failure mode for snscrape, and it's worth understanding before you pick a replacement — because the same thing will happen to the replacement if you choose the wrong kind.

Below: why it breaks, what the actual options are, and a line-by-line migration of the three patterns nearly every snscrape script is built from.

If you just want the working version of your script, skip to the migration. There's a free demo endpoint you can test the shapes against without signing up.

Why it breaks, and why it isn't the maintainer's fault

snscrape reads X's internal web endpoints — the same ones the website itself calls. Those endpoints were never a public contract. They carry GraphQL query IDs that rotate, required header sets that change, and response shapes that get restructured without notice.

When X ships a frontend change, three things can break at once:

What changes	What you see
GraphQL query ID rotates	`404` on a call that worked yesterday
Required headers change	`401`, or an empty result with HTTP 200
Response shape restructured	Parser returns nothing, no error raised

That third row is the dangerous one. A silent empty result looks exactly like "this account has no tweets." If you're running a pipeline on a schedule, you can lose days before anyone notices the table stopped growing.

So the first decision isn't which library to use. It's who is on the hook when X changes something. With a self-maintained scraper, that's you, at whatever hour it breaks.

The realistic options

Four, honestly described:

1. Keep patching snscrape (or a fork). Free, you keep full control, and it works right up until it doesn't. Reasonable if X data is a side input to a hobby project and a few days of downtime costs nothing. Check the fork's commit activity before relying on it — abandoned forks look identical to healthy ones until you need a fix.

2. The official X API. The only legitimate route if you need to write — post, reply, follow, delete. For reads, it's priced per post and the free tier is small enough that most people hit it during testing. Worth pricing out properly before dismissing or adopting.

3. Build your own against the internal endpoints. You'll reimplement what snscrape already did: query IDs, header construction, request signing, pagination cursors, and the parsing. The code isn't hard. The maintenance is the entire cost, and it's recurring.

4. A managed read API. Someone else absorbs the breakage and you call a stable interface. You pay for that, and you give up the ability to fix things yourself. This is what we build, so treat the rest of this article as written by an interested party — the migration code below works regardless.

If you only write to X, none of the read options replace the official API. No third-party read service can post as you, and any claiming to is using someone's credentials.

The migration

Almost every snscrape script is one of three patterns. Here's each, before and after.

Pattern 1: search by query

The classic snscrape loop:

import snscrape.modules.twitter as sntwitter

tweets = []
for i, t in enumerate(sntwitter.TwitterSearchScraper("bitcoin").get_items()):
    if i >= 100:
        break
    tweets.append({"date": t.date, "user": t.user.username, "content": t.rawContent})

The replacement, using plain requests against a data API:

import requests

API = "https://api.socialapi.tech/v1"
HEADERS = {"X-API-Key": "your_key"}

def search(query, limit=100):
    r = requests.get(f"{API}/search/advanced",
                     params={"query": query, "limit": limit},
                     headers=HEADERS, timeout=30)
    r.raise_for_status()
    return r.json().get("data", [])

tweets = [
    {"date": t["created_at"], "user": t["author"]["username"], "content": t["text"]}
    for t in search("bitcoin", 100)
]

Field mapping for this pattern:

snscrape	Replacement	Note
`t.date`	`t["created_at"]`	String, not a datetime — parse with `"%a %b %d %H:%M:%S %z %Y"`
`t.rawContent`	`t["text"]`
`t.user.username`	`t["author"]["username"]`	Author is nested inline, no second call
`t.id`	`t["id"]`	String, not int — 19 digits overflow JS numbers
`t.likeCount`	`t["like_count"]`
`t.retweetCount`	`t["retweet_count"]`
`t.replyCount`	`t["reply_count"]`
`t.viewCount`	`t["view_count"]`
`t.lang`	`t["lang"]`
`t.sourceLabel`	`t["source"]`

The full response shape — every field, which ones go null and when — is in the tweet JSON walkthrough.

Pattern 2: a user's timeline

# snscrape
scraper = sntwitter.TwitterUserScraper("elonmusk")
items = list(itertools.islice(scraper.get_items(), 200))

# replacement
def user_tweets(username, limit=200):
    r = requests.get(f"{API}/user/last_tweets",
                     params={"username": username, "limit": limit},
                     headers=HEADERS, timeout=30)
    r.raise_for_status()
    return r.json().get("data", [])

items = user_tweets("elonmusk", 200)

Pattern 3: paginating past the first page

This is where most migrations go wrong, because snscrape's generator hid pagination entirely — you just kept iterating.

# snscrape: pagination was invisible
for t in sntwitter.TwitterSearchScraper(q).get_items():
    ...   # kept going until you broke out

With an HTTP API you page explicitly, using a cursor:

def search_all(query, want=1000, page=100):
    out, cursor = [], None
    while len(out) < want:
        params = {"query": query, "limit": min(page, want - len(out))}
        if cursor:
            params["cursor"] = cursor
        r = requests.get(f"{API}/search/advanced", params=params,
                         headers=HEADERS, timeout=30)
        r.raise_for_status()
        body = r.json()
        batch = body.get("data", [])
        if not batch:                    # no more results
            break
        out.extend(batch)
        cursor = (body.get("meta") or {}).get("next_cursor")
        if not cursor:                   # no further pages
            break
    return out

Two things to get right here. Break when the batch is empty and when there's no cursor — relying on only one of those is how you get an infinite loop or a truncated result set. And don't assume a full page means more pages exist; check the cursor.

If you're paging across many accounts rather than deep into one, fetching them in a single call instead of a loop is a different kind of saving — the arithmetic is in batching X API requests.

The error handling snscrape didn't make you write

snscrape raised library exceptions. An HTTP API gives you status codes, which are easier to act on correctly:

import time

def call(path, params, attempt=0):
    r = requests.get(f"{API}{path}", params=params, headers=HEADERS, timeout=30)
    if r.status_code == 429:                       # rate limited
        if attempt < 4:
            time.sleep(2 ** attempt)               # exponential backoff
            return call(path, params, attempt + 1)
        raise RuntimeError("rate limited after retries")
    if r.status_code == 401:
        raise RuntimeError("bad API key")
    if r.status_code == 402:
        raise RuntimeError("out of credits")
    r.raise_for_status()
    return r.json()

Back off on 429, fail fast on 401 and 402. Retrying a bad key just wastes time; retrying an empty balance never succeeds.

What you actually give up

Worth being straight about this, because the trade is real.

You lose the ability to fix it yourself. With snscrape, a breakage is annoying but yours to solve — you can read the source, patch the parser, move on. With a managed API you file a ticket and wait. If that's intolerable for your project, option 1 or 3 is genuinely the better choice.

You lose "free". snscrape costs nothing but your time. Any managed service costs money. The question is only whether your time is worth more than the bill.

You gain not being woken up by it. That's the whole proposition, stated plainly.

Testing the migration before you trust it

Run both for a day and compare. Not counts — IDs:

old_ids = {t.id for t in snscrape_results}
new_ids = {int(t["id"]) for t in api_results}

print("only in snscrape:", len(old_ids - new_ids))
print("only in new api :", len(new_ids - old_ids))
print("overlap         :", len(old_ids & new_ids))

Expect imperfect overlap. Search results genuinely differ between pulls — ranking shifts, deletions happen, and real-time results move. What you're checking for is systematic absence: an entire account missing, or nothing older than a certain date.

Our API returns the structure used throughout this article, so the migrated code runs unchanged. Signing up with Google includes $0.20 in credits, no card, which is enough to run the comparison above before deciding anything.

FAQ

Is snscrape dead? The code still exists and forks are actively used, but it reads undocumented endpoints that change without notice. Expect periodic breakage and plan for who fixes it. "Dead" is the wrong frame — "unmaintained relative to how fast X changes" is closer.

Why did snscrape suddenly stop working? Almost always a change on X's side: a rotated GraphQL query ID (you see 404), a changed header requirement (401 or a silent empty result), or a restructured response (your parser returns nothing with no error). Your code didn't change; the contract underneath it did.

Why does snscrape return an empty list instead of an error? Because the HTTP call succeeded and only the parsing found nothing. An endpoint can return 200 with a shape your parser doesn't recognise. This is the most dangerous failure mode — it looks exactly like "no results exist."

What is the best snscrape alternative in 2026? Depends on who should own the maintenance. Self-maintained fork if you want control and can absorb downtime; the official X API if you need write access; a managed read API if you want the breakage to be someone else's problem.

Can I still scrape tweets without an API key? Public data can be read without X credentials, which is what snscrape does. Third-party APIs need their own key, not an X one. What you cannot do without authenticating as a user is write, or read anything protected.

Is scraping Twitter legal? Reading public data is treated differently from automating actions with someone's credentials or accessing private content. The practical lines: no one else's credentials, nothing protected, no attack-like volume. What you do with the data afterwards carries its own obligations — this isn't legal advice.

How do I replace TwitterSearchScraper? A search endpoint call with the query as a parameter. The loop over get_items() becomes an explicit paginated loop using a cursor, since the generator was hiding pagination from you.

How do I replace TwitterUserScraper? A user-timeline endpoint taking a username. The same cursor pagination applies if you want more than one page.

What replaces t.rawContent? The text field. t.date becomes created_at (a string in Twitter's own format, not a datetime), and t.user.username becomes author.username on the nested author object.

Why is the tweet ID a string now? It's a 19-digit snowflake, above JavaScript's safe integer range. snscrape handed you a Python int, which is fine in Python but corrupts the moment it crosses into JSON or JS. Strings are the portable choice.

How do I paginate without snscrape's generator? Loop while you still want more, pass the cursor from the previous response, and break on both an empty batch and a missing cursor. Checking only one of those causes either a truncated result or an infinite loop.

Does snscrape still work for Telegram or Reddit? snscrape covers several platforms and their stability is independent — the X module breaking says nothing about the others. This article is only about the X side.

Is there a drop-in replacement for snscrape? No library-level drop-in, because the whole point of moving is to stop depending on undocumented endpoints. The migration is mechanical though: three call patterns and a field rename table, which is what this article is.

How much does it cost to replace snscrape? With per-call pricing you pay for what you pull, so the honest answer is "it depends on volume." Work out your calls per month and multiply — if it comes out cheaper than the hours you spend patching, the decision is made for you.

Can I run snscrape and an API side by side during migration? Yes, and you should. Run both for a day and diff the tweet IDs, not the counts. Perfect overlap isn't expected; systematic gaps are the thing to look for.

What's the difference between snscrape and tweepy? tweepy is a client for the official API and needs developer credentials. snscrape reads public web endpoints and needs none. Different access models entirely, which is why the migration path differs depending on which you were using.

Will a managed API break the same way? The underlying endpoints change for everyone — the difference is who fixes it. With a managed service, the fix ships without you doing anything. With your own code, you're the one reading the diff at 2am.

How do I handle rate limits after migrating? Back off exponentially on 429 and don't retry 401 or 402 — a bad key or an empty balance won't resolve by trying again. There's a fuller treatment in what rate limit exceeded actually means.

Can I get historical tweets after snscrape stops working? Depends on the replacement. Deep historical archives are the hardest thing to get cheaply from any source — if your use case needs years of backfill rather than recent data, price that specifically rather than assuming it's included.

Do I need to change my database schema when migrating? Usually only the ID column — if you stored tweet IDs as integers, move them to strings or a 64-bit type. The rest of the fields map one to one; the names change, the meanings don't.

How do I scrape Twitter data without the official API? Either run a scraping library yourself and maintain it as X changes, or call a hosted read API that does the maintenance. The tradeoff is maintenance burden versus per-request cost — there's no option where nobody absorbs the breakage.

Can I scrape all of a person's tweets? You can page back through a timeline, but not indefinitely — timelines are depth-limited regardless of tool. For genuinely complete history you need date-sliced search rather than one deep timeline walk.

How do I scrape tweets for a keyword rather than an account? A search endpoint instead of a timeline endpoint. The pagination pattern is identical; only the query parameter changes. Operators that narrow the query are worth learning first — they cut volume more than any code change.

Is scraping Twitter with Python still possible in 2026? Yes, but the libraries that worked unattended for years now break when X rotates its internals. What changed isn't feasibility, it's who maintains the breakage.

Why did my scraper work yesterday and fail today? Almost always an upstream change: a rotated query ID, a new required header, or a response shape that moved. Nothing in your code changed, which is exactly why this class of failure is expensive to own.

The migration is a few hours of work and the result is code that doesn't need you the next time X changes something. A Google sign-up includes $0.20 in free credits with no card, which is enough to port one script and compare the output against what snscrape used to give you.