2026-09-15, updated 2026-11-11, Fundamentals · GEO

Build vs buy for AI visibility tracking: what in-house collection costs

The cost of collecting AI answers in-house is rarely the bill you avoid. It is the engineering time to reach production quality on each engine, plus the maintenance that follows for as long as you run it. Build when collecting answers is your product or you need data no provider returns. Buy when the answers are an input to something else you sell: a visibility dashboard, an agency report, a research dataset, an alerting feature. The framework below fills in the numbers behind that call.

Estimate the steady state, not the first engine. A prototype that captures one assistant’s answer on a laptop says little about a collector that captures several assistants and Google, in many markets, every day, with parsers that don’t silently lose fields. That collector is an operations function.

Definitions

• Collection layer: everything between “we have a prompt” and “we have a structured record of what the engine answered”: execution, parsing, retries and storage.
• Analysis layer: what you build on top: share of voice, citation tracking, competitor reports, alerts. This is where customers compare products.
• Run: one prompt, on one engine, in one market, collected once.

What building involves

Consumer AI products such as ChatGPT, Perplexity, Gemini, Copilot, Grok and Google AI Mode show answers that include web search, citations, product cards and local results. Those surfaces live in the consumer applications, not in the model providers’ developer APIs. Each of the components below exists once per engine.

1. Collection runs

You run automated collection against each product and keep throughput inside what each product allows. That is infrastructure to provision, scale, patch and monitor, sized for your peak run. Each assistant behaves differently.

2. Parsing that keeps up with interface changes

Extracting the answer text is the easy part. The fields that make the data useful are harder: which sources were cited, which inline citation supports which claim, product cards, places, the searches the assistant ran. Each engine renders them differently and changes them without notice.

You need fixtures (saved examples) per engine, tests that run your parser against them, and a way to notice when production output stops matching. The expensive failure is silent: the run succeeds and a field such as sources comes back empty. Catching that takes data-quality monitoring per field, per engine.

3. Market coverage

Answers differ by country, and for local intent by region. If your report says “answers in Germany”, each answer has to be the one a user there would see, with matching language settings. That means a way to place requests in each market, a way to verify it took effect, and a plan for markets where an engine isn’t available.

4. Queueing, retries and bounds

A daily run is prompts × engines × markets. Some runs fail transiently, some engines are slow, some prompts time out. You need a queue with priorities, retries with backoff, a deadline after which a run is abandoned, idempotency so a retried submission doesn’t double-count, and reporting on what didn’t complete. On AnswerLine, a queued task waits until it can start and has ten minutes from its first start to finish (async tasks).

5. Delivery and storage

Results go somewhere: a warehouse, an application database, a webhook. Receivers need to verify origin, deduplicate, and handle out-of-order arrival. This is the same whether you build or buy, so don’t count it as a saving on either side.

6. Terms, compliance and review

Each consumer product has its own terms of service. Collecting from them at volume is a decision for your legal and compliance review, per engine, before engineering starts, and again when terms change.

7. On-call

Interfaces change without notice and on no schedule. Somebody owns the pager for each engine. Alert on field-level drift, such as a jump in the share of answers with zero sources, as well as on errors.

A cost framework you can fill in

Team cost, engine count and quality bar vary too much for universal numbers. Fill in this table with your own.

Then:

• Build, cost over T months = E × B × C + E × M × C × T + I × T + D × T + R
• Buy, cost over T months = the monthly cost of your request volume (below) × T, plus the time to map the response shapes to your storage

Count shared costs once, on neither side: the prompt set, storage, metrics and the receiver exist either way. Estimate maintenance from your prototype’s history. If the build side only wins when maintenance is assumed near zero, it doesn’t win.

Adjust for three effects the formulas miss:

1. Build is paid before value. E × B months pass before the first reliable dataset; for a product that competes on time to market, that delay has its own cost.
2. Maintenance scales with engines, not volume. A fifth engine adds a fifth parser and a fifth pager rotation, even if you run few prompts on it.
3. Opportunity cost. Every engineer-month in the collection layer is one not spent on the analysis layer.

Computing the buy side

Buying is priced per request, charged from a prepaid USD balance. The base price per request at the Starter tier is:

Some options add to the price. ChatGPT’s include flags and extra Google News pages are free. On Google Search, each extra page adds $0.00071, and the AI Overview adds $0.00071 only when the response holds one. The maximum is reserved and the actual cost charged on success. Failed requests are charged nothing, and queued tasks you clear are never charged. Larger tiers, reached by 30-day spend, cost less per request. The details are in the pricing docs.

Worked example 1: a large program

20,000 prompts on ChatGPT and Gemini in the US, plus 2,000 Google News queries in 10 markets:

• Runs per sweep: 20,000 × 2 + 2,000 × 10 = 60,000.
• Cost of the assistants: 20,000 × ($0.0015 + $0.0015) = $60.00.
• Cost of Google News: 2,000 × 10 × $0.001 = $20.00.
• Cost per full sweep, no add-ons: $60.00 + $20.00 = $80.00.

A weekly sweep (4 a month) is $320 a month at Starter prices, a 30-day spend that reaches the Pro tier. A daily sweep is $2,400 a month at Starter prices and reaches the Max tier. Once a tier is reached its lower prices apply, so the real bill is lower. Every tier’s prices are on the pricing page.

Worked example 2: a focused program

200 prompts on ChatGPT and Gemini in the US, plus 50 Google News queries in 3 markets, weekly (4 runs a month):

• Cost per prompt: $0.0015 + $0.0015 = $0.003.
• Per sweep: 200 × $0.003 + 50 × 3 × $0.001 = $0.60 + $0.15 = $0.75.
• Per month: $0.75 × 4 = $3.00.

Tiers follow your last 30 days of spend, so estimate from your steady cadence, not your peak. The cost planning guide covers sampling and cadence choices that change these numbers, and sampling vs census covers when you can run fewer prompts without losing the signal.

What the buy side returns

Compare the output as well as the price. Each engine returns structured JSON under result, with fields that differ by engine:

• All AI engines: text, links[], sources[], and markdown with include.markdown.
• ChatGPT: citationPills[], entities[], map[], images[], and with include.shopping inlineProducts[]: the products from its shopping search, with price, link and merchant offers. The searchQueries[], shoppingCards[] and ads[] fields exist but come back empty on the answers served today.
• Gemini: citationPills[], and shoppingCards[] for the comparison tables in an answer.
• Perplexity: citationPills[], places[] and images[].
• Copilot: citationPills[].
• Grok: searchQueries[] (its web and X searches), relatedQueries[] and model.
• Google AI Mode: citationPills[], places[], map[], shoppingCards[], videos[] and ads.

For volume, tasks go through POST /v1/async/task/batch (up to 500 per request) with idempotency keys and webhooks. You maintain no queue or retry code; see async tasks.

What you still build when you buy

Buying the collection layer moves engineering to the parts that differ between products:

• Prompt set and schedule. Which prompts, engines and markets, and how often. Prompt set design and AI answer volatility cover the choices.
• Submission and retries. Batching, idempotency keys derived from each task’s meaning, and a retry policy that never duplicates work; see idempotency keys.
• A webhook receiver. Signature verification over the raw body, deduplication by task.id, and storage; see verifying webhook signatures.
• Normalization. Field names differ by engine (inlineProducts[].product.title on ChatGPT, inlineProducts[].title on AI Mode; citation ids 0-based on ChatGPT and 1-based elsewhere), so map them into your own schema once.
• Metrics and the analysis layer. Mentions, citations, share of voice and reports, which is what customers pay for; see mentions vs citations.
• Failure policy. Whether to resubmit failed tasks, and how to report gaps.

A minimal receiver in TypeScript, the one piece of infrastructure you add:

import express from "express";
import { verifyWebhook } from "@answerline/sdk";

const app = express();
const seen = new Set<string>(); // use your database's unique key in production

app.post("/hooks/answers", express.raw({ type: "application/json" }), async (req, res) => {
  if (!(await verifyWebhook(req.body, req.get("webhook-signature"), process.env.WEBHOOK_SECRET!))) {
    return res.sendStatus(400);
  }
  const { task, response } = JSON.parse(req.body.toString("utf8"));
  if (!seen.has(task.id)) {
    seen.add(task.id);
    await store(task, response); // COMPLETED: response.result; FAILED: response holds the error
  }
  res.sendStatus(204);
});

SDKs and workflow integrations are listed under integrations; for AI agents, the MCP server exposes each engine as a tool.

Hybrid: buy most engines, build the gap

A workable sequence:

1. Buy first for coverage. Run your prompt set on the engines a provider covers and ship the report.
2. Measure gaps. Note which engines, markets or fields your customers ask for that you don’t have.
3. Build only the gap, writing results into the same schema. Using the provider’s field names (text, sources[], url, position) keeps downstream code identical.
4. Re-evaluate yearly. If the provider adds the gap, retire your collector; if your in-house engine proves stable and cheap, consider moving more.

A batch submission for step 1, in Python:

import os, requests

prompts = ["best heat pump brands", "is a heat pump worth it in a cold climate?"]
engines = ["CHATGPT", "GEMINI", "PERPLEXITY", "COPILOT", "GROK"]

tasks = [
    {
        "taskType": e,
        "payload": {"prompt": p, "country": "US"},
        "idempotencyKey": f"hp-{i}-{e}-US-2026-11-11",
        "webhook": {"url": "https://your-app.com/hooks/answers"},
    }
    for i, p in enumerate(prompts)
    for e in engines
]
r = requests.post(
    "https://api.answerline.dev/v1/async/task/batch",
    headers={"Authorization": f"Bearer {os.environ['ANSWERLINE_API_KEY']}"},
    json=tasks,
    timeout=60,
)
print(r.json()["summary"])

Decision criteria

Signals that the decision should change

• Maintenance creep. Your in-house collector needs attention every week for interface changes, so the steady-state cost is higher than planned.
• Coverage requests. Customers ask for engines or markets you don’t collect. Adding them in-house repeats the whole build; check whether a provider covers them first.
• Data quality incidents. A field went empty for days before anyone noticed. You need that monitoring either way, but a provider with a published schema and failure semantics makes it visible sooner.
• Provider gaps. If your customers need a field or engine your provider doesn’t return, building that one piece may be justified.
• Scale changes. Per-request pricing and fixed infrastructure cross at some volume. Re-run the costing with this year’s numbers.

How to evaluate in a week

1. Pick your ten hardest prompts: the ones with shopping, local or citation-heavy answers that break parsers.
2. Run them on every engine you need. A new account’s 100 free requests (once, no card) work on any live engine, so you can evaluate the assistants before paying: 10 prompts on ChatGPT and Gemini are 20 requests. Paid, they would cost 10 × $0.003 = $0.03.
3. Check the contract: every field you plan to store is defined in the API reference, and it holds data for your prompts.
4. Compare field by field against what your pipeline or manual checks show: citations present, ordering, product cards, places.
5. Check the failure path: send a malformed request and a burst of requests, and confirm your code handles 422, 429 and batch item errors as described in the errors post. Check that failures are uncharged and bounded in time.
6. Check operational fit: idempotency keys, signed webhooks, and rate limits that match your schedule.
7. Price your volume with the per-request math above and your cadence, then fill in the build table with your own team’s estimates and compare.

Pitfalls in the comparison

• Pricing the prototype. The first engine working on a laptop says little about the fifth engine in production.
• Comparing one engine built to several engines bought. Price the scope you need.
• Ignoring maintenance. First-version cost is the smaller term over any horizon longer than a few months.
• Ignoring silent failures. Count answers with zero sources and fields that went empty, not only errors.
• Double-counting shared costs. The receiver, storage and metrics exist either way.
• Treating developer model APIs as equivalent. A completion from a model API does not include the consumer app’s web search, citations or product cards; see provider APIs.
• Locking your schema to one provider’s quirks. Normalize to a small core (answer, sources, market, engine, time) so either path can fill it.
• Pricing at peak cadence. Tiers follow 30-day spend; estimate the steady state.
• Skipping the terms review. It belongs at the start of a build plan, not after launch.

Related: web scraping vs an API. To run your first prompts, start with the quickstart.

Keep reading