Monid Finds the Tweet. Volumn Tells You Who Sent It.

I ran the same X search three ways to find people asking for a scraping tool. The naive phrasing returned twenty results, twelve of which were about the topic and most of which were viral content that happened to share a word. Tightening it with search operators pushed that to nineteen of twenty.

Then I read all twenty by hand, and the number that mattered was different:

20 results
19 mention the topic
 5 are the same two tweets posted repeatedly
 2 are a human being asking for a recommendation

Nineteen out of twenty on keywords. Two out of twenty on intent. That gap is the entire job, and no amount of query tuning closes it, because the thing you are filtering for is not in the text.

Fair disclosure. You are on the Monid blog, Monid sells the X search endpoint used below by the call, and Volumn.ai is a content partner whose product sits on the judgment side of that gap. Volumn is not in the Monid catalogue. The section near the end says where X is the wrong place to be looking at all.

Why does a twitter scraper hand you noise instead of buyers?

Because search ranks text, and buying intent is a property of the person, not the sentence.

Here is what came back from the tightened query, which is the good one. Two examples out of the twenty:

@mfranz_on      15,747 followers
  "I am looking for a web scraper library in Python. Any suggestion?"

@DBurzynski18488     13 followers
  "Has anyone used Leads Sniper, especially the Google Maps Scraper. Is it..."

Both are real. Someone is in market and asking. That is what you came for.

Now the other eighteen. Five are content marketing threads of the "10 GitHub repos that replace tools costing $30,000 a year" variety. Three are the identical promo tweet from one account, posted three times inside a single page of results. Two more are another account's tweet, twice. One is Y Combinator promoting a portfolio company. Four are builders describing what they are making, which reads like interest and is the opposite of it: they are building the thing you sell. One is a bare link with no text.

Every one of those matched the keywords. Most matched them better than the two that mattered, because marketing copy is written to contain the words.

Three structural problems fall out, and it is worth naming them separately because they need different fixes.

Duplicates are a quarter of the page. Five of twenty were repeats of two tweets. Accounts repost promos on a schedule and the search returns each instance. Dedupe on normalised text, not on tweet ID, because the IDs differ.

Follower counts span five orders of magnitude. From 13 to 1,641,319 inside one result set. The 13 follower account was one of the two genuine buyers. Whether that is your best lead or your worst depends entirely on what you sell, and the search will not decide it for you.

Builders read like buyers. The single hardest class to filter. "I'm building a web scraper agent where you can..." matches every keyword a scraping vendor would target and represents zero pipeline.

Which endpoint actually searches X, and what does it return?

tikhub /api/v1/twitter/web/fetch_search_timeline, reachable through X on Monid. It takes a keyword, a search_type of Top, Latest, Media, People or Lists, and a cursor for paging. It bills a flat rate per call regardless of how many results come back, which matters more than it sounds and is the subject of a section below.

For agents

Grab an API key at app.monid.ai, then paste this to your agent and hand it the key:

set up https://monid.ai/SKILL.md

It learns the discover, inspect and run workflow itself. More detail in the agent quickstart.

For humans

npm install -g @monid-ai/cli
monid keys add --label main --key <your-api-key>

More detail in the CLI quickstart.

The call

monid run -p tikhub -e /api/v1/twitter/web/fetch_search_timeline \
  --query '{"keyword":"\"looking for\" (scraper OR \"scraping api\") -filter:retweets","search_type":"Latest"}' \
  -w -o x.json

--query and not -i, because this endpoint takes query parameters. Use Latest rather than Top for intent work: Top is ranked by engagement, which is precisely the ranking that surfaces marketing threads over a quiet question from someone with thirteen followers.

What comes back

Results arrive under timeline, twenty per page, with twenty fields each. The ones that carry the work are text, favorites, views, replies, created_at, conversation_id, and user_info.

user_info is the field people miss. It arrives nested on every tweet and carries followers_count, friends_count, description, location, verified and created_at for the author. That means you can filter by account size, bio text and account age without a second call and without a second charge, which is the difference between one flat call and a per profile bill on top.

conversation_id is the other one worth reading. It groups a tweet with the thread it belongs to, which is how you tell a standalone question from the fourth reply in an argument.

How do you write a query that finds intent?

Operators, and then knowing when to stop.

The naive version, looking for a tool to scrape twitter as a plain phrase, returned twenty results of which twelve were on topic and the top hits were a Milwaukee police chief and a rescue dog. X treats a long conversational string as a bag of words.

The tightened version does three things:

"looking for" (scraper OR "scraping api") -filter:retweets

Quote the intent phrase. "looking for" as an exact phrase, not as two words that may appear anywhere. This is what pins the grammar of asking.

Group the subject with OR. People do not use your product category name. Give the query the two or three phrasings they actually type.

Exclude retweets. -filter:retweets removes amplification, which is pure noise for this job since a retweet is not the retweeter asking.

That took keyword relevance from twelve of twenty to nineteen of twenty. It also cost nothing extra to find out, because the endpoint bills per call rather than per result: three query variations is three charges whatever comes back, so iterating on the query is close to free and there is no reason to guess.

Then the third variation, "any recommendations for" "data api" -filter:retweets, returned zero results. Two exact phrases both matching is rare, and X returns an empty timeline rather than relaxing the query for you. An empty result set is a query problem, not a market signal, and the fix is to loosen one clause and re-run rather than to conclude nobody is asking.

Widening this into a scheduled job is a different piece of work, and we have written it up: a keyword-to-tweets scraper you can build in an afternoon, running it at volume, and tracking what rises and falls over time. The n8n version of the same shape, for people who want the whole enrichment chain, is in the n8n data layer guide.

How do you tell a buyer from a marketer?

By reading the account, which is a second question the search does not answer.

The two genuine buyers in my twenty had 15,747 and 13 followers. Nothing in the tweet text distinguished them from the promotional posts, and both would have failed a naive follower threshold in one direction or the other.

What actually separates them is account level: does this person post like a practitioner or like a channel, are they consistent or dormant, is the bio a job or a pitch, and does their engagement look earned. Those are all readable from public data, and none of them are in the tweet.

This is the point where the honest answer is that you are now building a product rather than running a query. You can do it: the user_info block gives you follower count, bio, account age and verification for free in the same response, and a couple of thresholds on those will remove most of the obvious noise. Beyond that, pulling an account's history tells you about consistency, and comparing accounts at scale tells you about relative quality.

Or you can use something built for it. Volumn.ai is an AI growth platform for X that packages this judgment layer: their X profile audit scores a public account with engagement diagnostics and growth recommendations, and their use case flows for B2B social selling on X and founder lead generation on X wrap the whole loop from finding the conversation to engaging it. They also run creator discovery and posting consistency tools on the same data.

The division is clean, which is why the partnership makes sense to write about. Monid sells the raw search by the call, one key, no seat. Volumn sells the judgment and the workflow on top. If you are building the loop yourself you want the first. If you want the loop to exist by Thursday you want the second.

What does running this continuously cost?

Less than the reading, which is the real constraint.

The endpoint bills per call at a flat rate, whatever comes back. That shape has a specific consequence for this job: the cost scales with how often you ask, not with how much you find. Checking six query variations every hour, all day, is a rounding error, and the same sweep across ten competitor keywords barely moves it. Magnitudes for anything in the catalogue live on monid.ai/tools, because a number in an article goes stale quietly.

Compare that with a per result endpoint, where a wide sweep is exactly what you pay for. Both shapes exist in our catalogue and the difference decides your architecture rather than your invoice, which is the argument in pay per call against a subscription and the cross platform comparison.

The cost that does bite is human. Twenty results with two worth acting on means somebody reads eighteen dead ends, and at ten searches a day that is the whole job. Every dollar of value here is in narrowing what reaches a person.

When is X the wrong place to look?

When your buyers are not there. X skews heavily toward software, crypto, media and marketing. If you sell to dentists or plant managers, a beautifully tuned intent query will return an empty timeline and it will be telling you the truth.

When the purchase is not discussed publicly. Nobody tweets "looking for a payroll provider." Categories with real switching costs get researched quietly. Intent search works where people crowdsource opinions in the open, and that is a narrower band than it looks.

When you need coverage rather than a sample. Search returns a ranked page, not a census. For anything that has to be complete, this is the wrong instrument.

When two of twenty is not worth the reading. Be honest about the arithmetic. If your deal size cannot fund somebody reading eighteen irrelevant tweets to find two conversations, then either the filtering has to get much better or the channel is not for you.

We sell the endpoint, so weigh that list accordingly. The strongest version of the case against us is that most teams should not build this: they should run one manual search first, read twenty results, count how many are real, and only then decide whether to automate anything.

Conclusion

A twitter scraper is a solved problem and it is not the problem. Getting the tweets takes one call and one flag; getting from twenty tweets to two conversations takes dedupe, an account level read, and a judgment about who is worth a reply.

The rule worth keeping: before you automate an intent search, run it once and read every result by hand. Count how many are genuine. If it is two in twenty, as it was here, you now know exactly what your filtering has to accomplish, and you know it before you have built anything.

FAQ

Last updated August 2026.