What AI Agents Need from News Retrieval

I want the LLMs working with news to spend their context on reporting that matters, rather than on cleaning up the delivery. Better signal-to-noise ratios can reduce downstream processing costs, whether you are a developer consuming an external news feed or a publisher building research tools over your own archive.

My background is in structuring, storing, and processing data. At AI Forum Paris, I spoke to publishers about what agents need from news, but the engineering problem applies to anyone consuming it: prepare useful information upstream so that each downstream agent has less work to repeat. I approach that problem through discoverability, efficiency, and traceability.

Every research loop can repeat the same unnecessary work

A deep research workflow searches, reads, reasons, and then searches again. Each result can introduce an entity, a relationship, or a time period that changes the next question. A single research task can also delegate work to other agents, each following its own loop.

That is why I care about the shape of the data at the point of retrieval. If every result includes navigation text, subscription prompts, irrelevant fields, or awkward wrappers, the model repeatedly receives material that contributes little to the question. If each agent also has to extract the same relationships before it can investigate them, preparation becomes part of every investigation.

Consider an illustrative question about companies affected by a policy change. A developer might investigate it using reporting from an external provider. A publisher's internal research assistant might investigate it using the newsroom's own archive. Both need to find the affected organizations, follow relevant connections, and check what was reported when. Owning the articles does not remove those processing steps.

I would begin with a trace from a representative question. Mark which steps discover new evidence and which merely repair, reformat, or re-extract material already encountered. That separates useful investigation from work the data layer could handle for subsequent requests.

Prepare reusable information before the agent arrives

One point I emphasized in the talk was doing processing earlier so agents downstream do not have to repeat it. Extracting a primary-source reference or a reported relationship before retrieval can make that information available to subsequent searches. The agent can investigate the connection instead of first reconstructing it from prose.

This is a data-design decision. A developer evaluating a provider should ask which preparation is already included and which steps remain in the application. A publisher building over its own material should ask which extractions belong in a shared ingestion layer rather than inside each assistant. In either case, the reusable record needs a link back to its supporting text.

I also want to know whether I can use the data in my intended workflow—not just whether I can fetch it. Public availability does not by itself establish permission, while paywalled reporting raises questions about authorized access and licensing. For developers building on external feeds and publishers using their archives internally, those checks belong alongside data quality, before the content reaches an LLM.

An illustrative implementation might extract organization references and source links when an article enters the system, store them with the article version, and reuse them when relevant. That does not mean extracting every conceivable label from every article. Upfront processing has a cost, and rarely used enrichment may cost more than it saves.

The same trade-off applies to freshness. Corrections and changed extraction methods can invalidate stored results. Reuse needs versioning and a refresh policy; otherwise a cheaper repeated lookup can return a stale judgment. Identify recurring work, measure how often its results are reused, and retain the original evidence so mistakes remain inspectable.

Spend tokens on evidence, not packaging

A context window is finite, even when it is large. The question, previous results, tool instructions, and retrieved reporting all compete for space. The useful target is the amount of relevant, verifiable information delivered within that space.

In the presentation, I contrasted noisy responses with prompt-optimized text that removes tags and subscription clutter. The cost mechanism is straightforward: token-priced models charge for input, and unnecessary text contributes tokens whenever it is included. Characters and tokens are not interchangeable units. Formatting overhead varies with the tokenizer, and caching and provider pricing affect the bill.

Structured output still has a purpose. Labels, identifiers, and delimiters can prevent ambiguity. Removing every bracket is a poor objective if the result makes sources or dates harder to distinguish. I want a compact record whose meaning survives the cleanup: the relevant passage, its source, its time information, and the context needed to interpret it.

Duplicate content deserves the same inspection. As an evaluation example, repeated copies of a passage can consume context without adding evidence. But similar-looking articles may contain an independent confirmation, a correction, or a changed detail. Deduplication should preserve those distinctions and their provenance rather than silently treating all coverage of an event as interchangeable.

Compare the noisy and cleaned representations on the same task. Count the tokens actually submitted to the model, then check whether the cleaned version still supports the answer. A shorter response that forces another retrieval to recover a missing caveat can erase the saving.

Find the relevant context before asking the model to process it

Cleaning text helps after retrieval. Better discovery helps decide which text reaches the model at all.

I find semantic search useful as a starting point in a large collection. It can surface reporting whose wording differs from the question. From there, explicit relationships can lead to relevant information that is not semantically similar to the initial query. In the talk, I described that combination as using vectors for discovery and a graph to find hidden links.

Entity disambiguation makes those paths more useful. Different references to the same person or organization should lead to the relevant evidence, while similarly named entities should remain distinct. I also stressed that disambiguation is a difficult, requirement-dependent problem. An incorrect merge can send a research workflow down the wrong path.

For an external feed, inspect how entity identities and extracted relationships are represented. For an internal archive, test the same behavior across your own naming conventions and historical coverage. Neither a vector match nor a graph edge establishes that a claim is correct.

I would test a question with a known, indirectly related document. Does the retrieval path find it? Can I inspect the passage supporting the relationship? Does the date fit the question? Better targeting can reduce irrelevant context, but it can also enable a deeper investigation with more searches. Measure the whole task before calling that a cost reduction.

Keep the evidence trail intact

Traceability is a client requirement in my work, not simply a request for publisher attribution. A consequential answer needs an explanation of where its information came from. The agent also needs enough of a trail to revisit earlier evidence when a later result changes its interpretation.

That applies just as strongly inside a publisher. An internal assistant should let a colleague open the reporting behind a claim and distinguish the original article from a later correction. A developer consuming external news needs the same ability across the sources in the feed.

I want each retrieval to give the agent useful evidence and a way back to the reporting, with less unnecessary work in between.

For an evaluation, I would retain the cited passage, document identifier, attribution, timestamps, and relevant query or filter settings. Those fields consume space, but they serve a purpose: a reviewer can inspect both the document and the decision that brought it into the analysis. Efficiency should remove overhead without removing accountability.

Historical questions need another boundary. In the talk, I discussed backtests that simulate what could have been known in the past. Publication dates alone may not capture later corrections or enrichment. Ask what historical versions the data layer can reconstruct, and document gaps. Restricting retrieved evidence also does not erase knowledge already present in the model. A cheaper test is of little value if its evidence boundary makes the result misleading.

Give the agent a usable task

There is overhead on the request side too. An agent may need to interpret a large tool description, choose among confusing parameters, search for identifiers, fetch the documents, and request another representation before it can reason over the news.

My recommendation is a small set of clear, action-oriented tools. Where a task requires several internal calls, a service can coordinate them and return usable evidence. Keep controls that affect the question, such as time boundaries and source selection, visible.

Model Context Protocol (MCP) can provide that interface, alongside REST, SDKs, or a CLI suited to the consuming workflow. The protocol does not clean the underlying data by itself. A wrapper around a noisy response still leaves the agent with a noisy response.

The same design test works for a provider's retrieval tool and a publisher's internal archive tool: can the agent complete a useful task without learning the service's internal architecture? Check the description it actually receives and the calls it actually makes. Tool-loading behavior and caching vary, so a long schema is not necessarily billed afresh at every step.

Compare the cost of reaching a supported answer

I would evaluate these changes on representative research questions, with the same model settings and evidence requirements. Separate the cost of preparing data from the cost of using it, then inspect both:

• Preparation and reuse: What does cleaning, extraction, indexing, storage, and refreshing cost? How often is the prepared information reused?
• Model input: How many relevant and irrelevant tokens reach the model across the full run, including repeat context and tool instructions?
• Calls and recovery: Which searches advance the investigation, and which repeat work because the previous response was incomplete or difficult to use?
• Evidence quality: Are important details, independent sources, corrections, and attribution preserved? Are inferred labels supported well enough for the filters using them?
• Total task cost: Include data access, preprocessing, retrieval, model input and output, and any retries. Compare runs that meet the same answer-quality standard.

For a developer, a higher-priced prepared feed can be worth evaluating if it removes enough application-side processing. For a publisher, investment in a shared preparation layer can be worth evaluating if multiple internal workflows reuse it. Neither conclusion follows from a smaller response alone. Low reuse, expensive enrichment, or lost evidence can reverse the result.

My takeaway is to make the news easier to discover, cheaper to pass where unnecessary input can be removed, and possible to trace. Start with a recurring question, inspect where the work repeats, and improve that part of the data path. Then measure whether you reach a supported answer with less total processing.

Explore docs for the documented AskNews MCP interface.