Migrating off Tweepy: A Field-by-Field Mapping

12 min readSocialAPI Engineering

Migrating off Tweepy: A Field-by-Field Mapping

Tweepy is a good library. If you're moving off it, that's usually not because the library disappointed you — it's because the API underneath it changed terms, or because you only ever needed to read public data and the credentials, quotas and OAuth dance stopped being worth it.

This is the mechanical part: which call maps to which, which field names change, and what genuinely doesn't carry over. It's a few hours of work, and most of that is reading your own code to find out which calls you actually use.

Different library, different path: if you're coming from snscrape rather than Tweepy, that migration is a separate article — the failure modes aren't the same. There's a free demo endpoint if you want to compare field shapes before committing.


First: are you a read-only user?

This decides whether migrating is even possible, so check before you start.

# in your project root — find every write you do
grep -rnE '\.(create_tweet|delete_tweet|like|unlike|retweet|unretweet|follow|unfollow|send_direct_message|update_status|update_profile)\(' .

Any hits mean you need credentials that can act as an account, and a read-only data API cannot replace those. Keep Tweepy for the write paths — you can run both, with Tweepy doing the handful of writes and a data API doing the bulk of reads. That's a common and perfectly sensible end state.

No hits means the migration is straightforward. Everything below applies.


Method mapping

Tweepy (v2 Client) Equivalent request
get_user(username=...) GET /v1/user/info?username=
get_user(id=...) GET /v1/user/info_by_id?user_id=
get_users_tweets(id) GET /v1/user/last_tweets?username=
search_recent_tweets(query) GET /v1/search/advanced?query=
get_tweet(id) GET /v1/tweet/info?id=
get_users_followers(id) GET /v1/user/followers?username=
get_users_following(id) GET /v1/user/followings?username=
get_retweeters(id) GET /v1/tweet/retweeters?id=
get_quote_tweets(id) GET /v1/tweet/quotes?id=
Paginator(...) on timelines one call, limit up to 200 — no cursor
Paginator(...) on followers / lists cursor from the response meta

The biggest practical difference: handles work directly. Tweepy's timeline and follower methods take a numeric user ID, so the common pattern is a lookup call followed by the real call. Here the handle is accepted directly, which removes one request per target — on a list of a thousand accounts, that halves your call count.

# Tweepy: two calls
user = client.get_user(username="naval")
tweets = client.get_users_tweets(user.data.id, max_results=100)

# One call
r = requests.get(f"{API}/user/last_tweets",
                 params={"username": "naval", "limit": 100},
                 headers=HEADERS, timeout=30)
tweets = r.json()["data"]

Keep storing the numeric ID anyway. Handles change; IDs don't. Accepting a handle as input is a convenience, not a reason to key your database on it — the reasoning is in which fields make a safe primary key.


Field mapping

This is where quiet bugs come from, because several fields are the same concept under a different name.

Tweepy v2 field Here Note
public_metrics.like_count like_count Flat, not nested
public_metrics.retweet_count retweet_count
public_metrics.reply_count reply_count
public_metrics.quote_count quote_count
public_metrics.impression_count view_count Different name, same idea
user.public_metrics.followers_count followers_count Flat
user.description bio Renamed
user.name display_name Renamed
created_at (datetime) created_at (string) You parse it now
author_id author object, or author_id Often inline, no second call

Three of these will bite if you skim.

public_metrics is flat here. Tweepy nests the counters; a direct port that keeps t["public_metrics"]["like_count"] raises KeyError immediately. That one is loud, so it's fine.

description → bio and name → display_name are the dangerous pair. If your code does user.get("description", ""), you get an empty string rather than an error — and your output is silently blank for every record.

created_at arrives as a string, not a datetime. Tweepy parses it for you; here you do it yourself, and it isn't ISO 8601:

from datetime import datetime, timezone

def parse_created_at(s: str) -> datetime:
    # "Tue Jun 02 20:12:29 +0000 2009" — not ISO 8601
    return datetime.strptime(s, "%a %b %d %H:%M:%S %z %Y").astimezone(timezone.utc)

A naive datetime.fromisoformat() raises on this. Full field layout is in reading the tweet object.


Pagination: Paginator splits in two

Tweepy wraps every paged call in the same Paginator. Here the two kinds of endpoint behave differently, and porting the loop blindly gets one of them wrong.

Timelines don't page. last_tweets, media, photos and videos return up to limit items in a single call, and the response has no next_cursor. The maximum limit is 200, and some accounts return fewer however high you set it — X's own timeline depth varies by account. Where Tweepy needed a Paginator to walk back through a timeline, you replace the whole loop with one request and a larger limit.

Lists do page. Followers, followings, retweeters and list members return a next_cursor in meta, and you pass it back as cursor. You write that loop yourself — and it's fewer moving parts than the wrapper suggests:

def paginate(path, params, max_items=None):
    out, cursor = [], None
    while True:
        p = dict(params)
        if cursor:
            p["cursor"] = cursor
        r = requests.get(f"{API}{path}", params=p, headers=HEADERS, timeout=60)
        if r.status_code == 429:
            time.sleep(30)
            continue                      # not a failure; come back
        r.raise_for_status()
        body = r.json()

        batch = body.get("data") or []
        if not batch:
            break
        out.extend(batch)

        if max_items and len(out) >= max_items:
            return out[:max_items]

        cursor = (body.get("meta") or {}).get("next_cursor")
        if not cursor:
            break
    return out
followers = paginate("/user/followers", {"username": "naval", "limit": 200})

Break on both the empty batch and the missing cursor. Checking only one gives you either an infinite loop or a truncated result — and truncation looks exactly like "this account doesn't have many followers."


A compatibility shim, if you have a lot of call sites

If Tweepy objects are referenced in dozens of places, don't rewrite them all at once. Wrap the new responses in something attribute-accessible and migrate call sites gradually:

class Obj(dict):
    """Attribute access over a dict, so t.like_count keeps working."""
    def __getattr__(self, k):
        try:
            return self[k]
        except KeyError:
            raise AttributeError(k)

def adapt(d: dict) -> Obj:
    o = Obj(d)
    # bridge the renames so old call sites don't change yet
    if "bio" in d:
        o["description"] = d["bio"]
    if "display_name" in d:
        o["name"] = d["display_name"]
    return o

This is scaffolding, not a destination. It buys you a working system on day one and lets you delete the aliases as you touch each call site. Leaving it in permanently means carrying two names for every field forever, which is its own cost.


What you actually lose

Stated plainly, because discovering these mid-migration is unpleasant:

Write access. No posting, liking, following or DMs. If you need those, Tweepy stays for that part. This is the one that ends migrations, which is why the grep comes first.

Your own private analytics. X shows the author of a post metrics that aren't public — detailed impression breakdowns, profile visits from a post. No third-party API can see those, because they aren't public data. If your dashboard reports on your own account's private numbers, that part can't move.

OAuth-as-a-user flows. Anything where your app acts on behalf of a logged-in user requires the official API by definition.

What you gain, for balance: no developer account application, no per-user token handling, handles accepted directly, and a leaked key costs you quota rather than someone's account. That last one is worth more than it sounds — the blast radius of a read-only key is categorically smaller.


Verify before you trust it

Don't swap the implementation and assume parity. Run both against the same input and compare:

def compare(username, n=20):
    old = {str(t.id) for t in client.get_users_tweets(
        client.get_user(username=username).data.id, max_results=n).data or []}
    new = {t["id"] for t in requests.get(
        f"{API}/user/last_tweets",
        params={"username": username, "limit": n},
        headers=HEADERS, timeout=30).json().get("data", [])}

    print(f"old={len(old)} new={len(new)} overlap={len(old & new)}")
    print("only in old:", sorted(old - new)[:5])
    print("only in new:", sorted(new - old)[:5])

Expect near-overlap, not identity. The two can disagree legitimately: different default inclusion of replies and retweets, and new posts arriving between the two calls. Investigate a large gap; don't chase the last one or two. Check a handful of accounts including one that posts frequently, since that's where ordering and windowing differences show up.


Our API accepts handles directly and returns the flat field names above, so the mapping table is the whole migration for most code. Signing up with Google includes $0.20 in free credits and no card — enough to run the comparison script against your real targets before changing anything.


FAQ

Is Tweepy still maintained? Yes. The reason people migrate is usually the terms and quotas of the API underneath, or not needing write access, rather than any problem with the library.

What's the best Tweepy alternative for reading data? Any read-only REST API over plain HTTP. The things to compare are whether handles are accepted directly, how pagination is exposed, and whether a leaked key can touch an account.

Can I replace Tweepy entirely? Only if you never write. Grep for create_tweet, like, follow and friends first — any hit means Tweepy stays for that path.

Can I use both at once? Yes, and it's often the right answer: Tweepy for the few writes, a data API for the bulk of reads. They don't conflict.

Do I still need a developer account? Not for reading public data through a third-party API — you use that provider's key. You'd still need one for official API access and write operations.

Why don't I need a user lookup first? Because handles are accepted directly. Tweepy's timeline methods want a numeric ID, which forces a lookup call; removing it halves the request count on large target lists.

Should I key my database on handle or ID? ID, always. Handles change, and handle-keyed records silently vanish when an account renames — it looks like a deletion.

Where did public_metrics go? The counters are flat here: like_count rather than public_metrics.like_count. A direct port raises KeyError, which at least fails loudly.

Which field replaces description? bio. This is the rename most likely to cause a silent bug, because d.get("description", "") returns empty rather than erroring.

What about user.name? It's display_name. Same silent-failure risk as description if you read it with a default.

Is impression_count the same as view_count? They're the same idea under different names. Expect the magnitudes to be far above the other counters, and don't plot it on a shared axis.

Why doesn't created_at parse? It's not ISO 8601 — the format is "Tue Jun 02 20:12:29 +0000 2009". Use strptime with an explicit format string; fromisoformat raises.

How do I replace Paginator? Depends on the endpoint. For timelines, you don't — one call with limit up to 200 replaces the whole loop. For followers and lists, a while loop that passes next_cursor back as cursor, breaking when the batch is empty or the cursor is missing.

How do I handle rate limits without Tweepy's wait_on_rate_limit? Catch 429, sleep, and retry without advancing your cursor. A rate limit isn't a failure, and the request isn't billed.

Is there a drop-in compatible library? Not identical, but a small attribute-access wrapper plus two field aliases gets most existing call sites working unchanged. Treat it as scaffolding you delete incrementally.

How long does a migration take? A few hours for typical read-only code, and most of that is auditing which calls you actually use rather than writing anything new.

Will my results be identical? Near-identical, not identical. Defaults for including replies and retweets differ, and new posts land between calls. Compare sets and investigate large gaps only.

How do I verify the migration? Run both implementations against the same accounts and diff the ID sets. Include a high-frequency poster, which is where windowing differences surface.

What do I lose on private metrics? Anything X shows only to a post's author — detailed impression breakdowns, profile visits. That isn't public data, so no third-party API can provide it.

Can I still get more than 7 days of history? That limit belongs to recent-search on the official v2 API. Here a timeline call returns up to 200 of an account's latest posts — fewer for accounts whose timeline X serves shallowly. For older material, use date-bounded search.

What happens to my OAuth flow? It goes away for reads — you send an API key header instead. If part of your app acts on behalf of logged-in users, that part keeps OAuth and the official API.

Is a read-only key safer? Materially. A leaked user token means someone can post as you; a leaked read-only key means someone can spend your quota. Different kinds of bad day.

Should I migrate incrementally or all at once? Incrementally, behind a flag, with the comparison script running on both. All-at-once leaves you debugging the migration and the data at the same time.

Do I need to change my pagination page size? For timelines, set limit up to 200 — that's the whole result, in one call. For followers, limit is the page size the cursor walks through, so a larger value means fewer requests.

Can I keep using Tweepy's models with the new data? Not directly — they expect the nested v2 shape. The small attribute-access wrapper is the practical middle ground if your code passes those objects around.

What's the most common migration bug? Reading description or name with a .get(..., "") default. The code runs, the output is blank, and nothing in the logs says why.


Run the grep first, then the comparison script — those two steps decide whether this is a few hours or not possible at all. A Google sign-up includes $0.20 in free credits and no card, which covers both.