Tracking X Stats: Your Sampling Rate Is the Experiment
Tracking X Stats: Your Sampling Rate Is the Experiment
A stats tracker looks like a scheduling problem. Pick an interval, record the numbers, draw a chart.
It's actually a measurement problem, and the interval you pick silently decides which questions you can answer. Sample daily and you will never see that most engagement arrives in the first three hours — not because it didn't, but because your data can't show it. The chart will look smooth and reasonable and be wrong about the thing you most wanted to know.
This is about choosing that interval deliberately, and about which numbers survive comparison once you have them.
If you need the storage layer first, the schema decisions come before this. Here we're assuming rows are landing and asking what they can prove — there's a free demo endpoint if you want to see what one observation contains.
What cadence costs you, in numbers
Engagement on X is heavily front-loaded. The numbers below come from a saturating decay model, not from measured X data — treat the shape as the point and the exact percentages as illustrative. Model a post that ends at 1,000 interactions and you get:
| Hours after posting | Engagement accrued |
|---|---|
| 1 | 21% |
| 3 | 50% |
| 6 | 75% |
| 12 | 94% |
| 24 | ~100% |
Half the final number lands in the first three hours. Now sample that same curve at three different intervals and ask each one "what was the peak hourly rate?"
| Sampling cadence | Observed peak rate | Error |
|---|---|---|
| Hourly | 206 / h | — |
| Every 6 hours | 125 / h | −39% |
| Daily | 41 / h | −80% |
Daily sampling under-reports the peak by a factor of five. That ratio is a property of averaging, not of this particular curve: any front-loaded series sampled at 24-hour intervals loses its spike the same way. It isn't noise and it isn't fixable in analysis. If "how fast does my content take off?" is the question, a daily tracker cannot answer it at any sample size.
The inverse is also true, and worth saying because it saves money: if your question is "how is this account trending over months?", hourly sampling buys you nothing and costs 24x the requests. Match the cadence to the question.
| Your question | Cadence | Why |
|---|---|---|
| How fast does a post take off? | 15–60 min, first 24h only | The spike is in hours 0–6 |
| Which posts outperform? | Once at 48h | Engagement is ~settled |
| Is this account growing? | Daily | Follower counts move slowly |
| Competitive benchmarking | Daily | Comparability matters more than resolution |
The useful pattern is tiered: sample new posts aggressively for a day, then stop. A post from last month doesn't need hourly observation — it isn't changing.
Which numbers are actually comparable
Having the data doesn't make the comparisons valid. Three traps, in order of how often they're hit:
Raw counts across accounts are meaningless. An account with 2M followers getting 500 likes is underperforming; an account with 5k followers getting 500 likes is doing very well. Normalise before comparing, and be explicit about the denominator — the choice between followers and views changes the answer, and what impressions actually mean matters here.
view_count isn't the same kind of number as the others. It's typically two to three orders of magnitude larger, it's counted differently, and it isn't available for every post. Putting it on a shared axis with likes produces a chart where likes are a flat line at zero.
Age dominates everything. A post from this morning and one from last week aren't comparable on raw engagement at all. Compare like-for-like on age, or only compare posts that have settled.
def comparable_engagement(post, min_age_hours=48):
"""Return None rather than a misleading number."""
age_h = (now() - parse_created_at(post["created_at"])).total_seconds() / 3600
if age_h < min_age_hours:
return None # too young — say so, don't guess
views = post.get("view_count") or 0
if not views:
return None # no denominator, no rate
inter = (post.get("like_count", 0) + post.get("retweet_count", 0)
+ post.get("reply_count", 0) + post.get("quote_count", 0))
return inter / views
Returning None beats returning a number you don't believe. A tracker that silently emits engagement rates for three-hour-old posts will show you a trend that's purely an artefact of post age, and it's very hard to spot once it's in a chart.
Telling movement from noise
Counts on X go down as well as up. Unlikes happen, accounts get suspended, posts get deleted. A tracker that treats every decrease as an event will page you constantly.
def is_real_change(prev, curr, min_abs=5, min_rel=0.02):
"""Require both an absolute and a relative threshold."""
delta = abs(curr - prev)
if delta < min_abs:
return False # 3 likes on anything = noise
if prev and delta / prev < min_rel:
return False # 50 on 10,000 = noise
return True
Both conditions, not either. An absolute threshold alone floods you on large accounts, where 50 likes is nothing. A relative threshold alone floods you on small ones, where going from 2 to 4 is a 100% increase. Requiring both is what makes the signal usable across account sizes.
A sudden drop to zero is usually not engagement loss — it's a deleted post or a suspended account. Treat it as a state change to investigate, not a metric to chart.
Follower counts are cached
This catches people building growth dashboards: the follower count on a profile is not a live number. It's cached, it updates on its own schedule, and it will not match a count of the follower list you just walked.
Two consequences:
Don't compute daily growth from two snapshots taken minutes apart — you're measuring cache refresh, not follows. Space your samples at least a day for follower growth to mean anything.
Small discrepancies against a list export are expected. A fraction of a percent on a large account is normal, as exporting followers covers. If your dashboard flags that as a bug, you'll chase it forever.
A tiered collector
import time
from datetime import datetime, timedelta, timezone
HOT_WINDOW_H = 24 # aggressive sampling
HOT_INTERVAL_S = 900 # every 15 min
COLD_INTERVAL_S = 86400 # daily, for account-level stats
def due_for_sample(post, last_sampled):
age = datetime.now(timezone.utc) - post["created_at"]
if age < timedelta(hours=HOT_WINDOW_H):
return (time.time() - last_sampled) >= HOT_INTERVAL_S
return False # settled: stop sampling individual posts
def collect(api, username, state):
posts = api.user_last_tweets(username, limit=50)
hot = [p for p in posts if due_for_sample(p, state.last.get(p["id"], 0))]
for p in hot:
record_metrics(p) # append an observation
state.last[p["id"]] = time.time()
# account-level numbers on a slower clock
if time.time() - state.last_profile >= COLD_INTERVAL_S:
record_profile(api.user_info(username))
state.last_profile = time.time()
The return False for old posts is the cost control. Without it, every post ever published stays in the sampling rotation forever, and your request count grows without bound while telling you nothing new. A tracker that samples 10,000 historical posts hourly is spending almost all of its budget re-reading numbers that haven't moved.
One request returns up to 50 posts with their counts attached, so the hot window is cheap: one call per account per interval covers every recent post at once. The arithmetic of that is in batching requests.
Our API returns engagement counts inline with each post, so a single timeline call is a complete observation rather than N follow-up requests. Signing up with Google includes $0.20 in free credits and no card — enough to sample a real account for a day and see the shape of the curve before you decide on a cadence.
FAQ
How often should I sample X stats? By question, not by habit. Takeoff speed needs 15–60 minute sampling in the first 24 hours; account growth needs daily; anything older than two days barely needs sampling at all.
Does daily sampling really lose that much? Yes. Against an hourly baseline on a front-loaded curve, daily snapshots understate the peak rate by roughly 80% — a 5x error, built into the method rather than the data.
Why is engagement so front-loaded? Distribution happens early, so roughly half of final engagement arrives in the first three hours and about 94% within twelve. That's why the sampling window that matters is day one.
When has a post settled? Around 48 hours for most posts. After that the counts move slowly enough that further sampling costs requests without adding information.
Can I fix coarse sampling in analysis? No. Averaging across a 24-hour gap flattens the spike by construction. The resolution you didn't collect can't be recovered.
Should I sample every post I've ever collected? No, and this is the main cost mistake. Stop sampling individual posts once they've settled, or your request count grows forever while the numbers don't change.
Are raw engagement counts comparable between accounts? No. 500 likes is weak for a 2M-follower account and excellent for a 5k one. Normalise, and state your denominator.
Should I normalise by followers or views? Views for per-post performance, since it's the actual audience reached. Followers for account-level comparison. They answer different questions and will disagree.
Why does view_count break my charts? Because it's two to three orders of magnitude above the other counters. On a shared axis everything else becomes a flat line — give it its own axis or its own chart.
Is view_count available on every post? No, and that's why rate calculations need a guard. Dividing by a missing or zero view count is the most common way a tracker produces nonsense.
Why return None instead of 0 for an engagement rate?
Because 0 is a claim and None is an absence. A tracker that emits rates for three-hour-old posts will show a trend that's purely an artefact of post age.
Can engagement counts go down? Yes — unlikes, deletions and suspensions all reduce them. Don't build alerts that assume monotonic growth.
What's a sensible threshold for a change alert? Both an absolute and a relative floor. Absolute alone floods you on big accounts; relative alone floods you on small ones. Requiring both works across sizes.
What does a sudden drop to zero mean? Usually a deleted post or a suspended account, not lost engagement. Handle it as a state change to look into rather than a data point to plot.
Is the follower count on a profile live? No, it's cached and refreshes on its own schedule. Two samples minutes apart measure cache behaviour, not follows.
Why doesn't my follower count match my exported list? The profile number is cached and the export is a live walk through a changing list. A fraction of a percent of difference on a large account is expected.
How do I measure follower growth properly? Daily snapshots at a fixed time of day. Shorter intervals mostly capture cache noise; a fixed hour removes time-of-day effects from the comparison.
How much data does a tracker accumulate? It's driven by observations, not posts. One post sampled every 15 minutes for a day is 96 rows. Multiply by posts and accounts before choosing a cadence.
Should I store every observation or just the latest? Every observation if you'll ever ask about change over time, which is usually the reason for building a tracker. Latest-only turns the tracker into a cache.
How do I compare two accounts fairly? Same metric, same normalisation, same post-age window, same sampling cadence. Any one of those differing makes the comparison decorative.
What's the cheapest useful tracker? One timeline call per account per day, storing all returned posts with their counts. That gives trends and relative performance; it just won't give takeoff curves.
Can I get historical stats for posts I didn't track? Only the current numbers, not the history. Engagement history exists solely in whatever you recorded, which is why starting collection early matters more than collecting densely.
Does one request include the engagement counts? Yes — counts come inline with each post, so a timeline call is a complete observation. You don't need a follow-up request per post.
How do I avoid rate limits in a tracker? Tier your cadence and batch where possible. Most rate-limit problems in trackers come from sampling settled posts, not from sampling new ones.
Should I track my own account or competitors? Both, but note the asymmetry: for your own posts X gives you private metrics an API can't see. Public counts are what makes cross-account comparison possible at all.
What's the single biggest mistake in stats trackers? Choosing the interval by convenience and then asking questions the interval can't answer. The cadence is the experiment design, not a scheduling detail.
Decide the question first, then the cadence follows from it — that ordering is most of the value here. A Google sign-up includes $0.20 in free credits and no card if you want to sample a real account before committing to an interval.