Products & Tools

Confluent launches Streamhouse and Lightning Tables for AI agents

Confluent says Lightning Tables can serve new data in about 100 milliseconds and exceed 1,000 queries per second as Streamhouse targets governed, real-time AI context.

Confluent unveils Streamhouse & Lightning Tables for AI
Confluent unveils Streamhouse & Lightning Tables for AIBru-nO / pixabay
By Elena Vasquez6 min read

Why it matters

  • Lightning Tables became generally available on AWS on November 9, 2026
  • Internal tests showed about 100-millisecond data serving and 10-to-100-millisecond query latency
  • Confluent reported throughput of more than 1,000 queries per second
  • The platform includes more than 150 managed and custom connectors
  • Accenture said time-to-insight fell from months to days for an organization serving 800,000 employees

By Elena Vasquez

Confluent said internal testing showed that Lightning Tables can serve newly created data in about 100 milliseconds, with query latency of 10 to 100 milliseconds and throughput of more than 1,000 queries per second. The company said actual performance depends on configuration and workload.

On November 9, 2026, Confluent introduced a Streamhouse architecture and new query tools for its Data Streaming Platform. Confluent's release includes general availability of Lightning Tables on AWS.

Confluent, which sits within IBM, said Streamhouse gives AI agents, applications, and analytics systems a governed view of a business's current state. It brings streaming, connection, processing, governance, and query functions into one platform.

Confluent says the release addresses a gap between older data stacks and AI systems. Its argument: older architectures were built mainly for reporting and batch processing, while AI increasingly needs current operational data from multiple systems at the moment decisions are made.

What does Streamhouse change?

Confluent positions Streamhouse as a layer alongside data warehouses, data lakes, and lakehouses. The architecture combines updates from source systems into a shared, continuously updated operational picture. Data remains where it already resides instead of being consolidated first.

Confluent calls that operational picture current, mission-critical, and decentralised. The design shifts the emphasis from historical analysis toward live context for agents and applications.

Confluent now groups the platform into five areas:

  • Stream: Confluent bases this function on Apache Kafka and the company's Kora engine.
  • Connect: Confluent says it includes more than 150 managed and custom connectors for moving data into and out of enterprise systems.
  • Process: Confluent uses Apache Flink to combine and enrich data close to its source.
  • Govern: Confluent places schemas, lineage, and access policies across the data path from capture through serving.
  • Query: Confluent places Tableflow and the Real-Time Context Engine here. Tableflow makes streaming data available to lakehouses and warehouses in open table formats. The Real-Time Context Engine serves business context to AI agents through the Model Context Protocol.

Together, the five functions let Confluent present one governed data path from capture through serving. Confluent says that path can supply agents with the live business state older batch systems often lack. The combination matters because it joins streaming speed with controls for schemas, lineage, and access.

How does Lightning Tables serve live data?

Lightning Tables is a read-only serving layer in Confluent Cloud. It stores and materializes Apache Kafka data into tables and keeps those tables updated as source data changes. Customers can query the result through SQL or a direct API.

According to Confluent, that design reduces the need to move streaming data into separate databases just to serve applications. It also carries governance controls, including schemas, lineage, and access policies, from data capture through serving.

Confluent said the service can also support machine-learning use cases. It can store features computed in Flink for tasks such as real-time personalization. That connects the Query and Process functions within the same platform.

Why does the airline case matter?

Confluent illustrated the business case with an airline disruption scenario. Separate systems for flight operations, scheduling, and reservations hold different versions of events. In that setup, an automated rebooking system could propose an itinerary that looks valid in isolation.

Under Confluent's scenario, the itinerary can still fail once aircraft location, departure time, and seat availability are considered together. Confluent said Streamhouse is intended to prevent that mismatch by combining updates from source systems into a shared, continuously updated picture. It leaves the underlying data distributed while giving decision systems a common view.

For AI agents, the distinction is material. A proposal can depend on facts held in several systems at once rather than on one isolated record.

What do early customers report?

Accenture said it has adopted a Streamhouse architecture based on Confluent across its global business.

“Operating across more than 120 countries, our global enterprise was previously constrained by legacy batch and point-to-point architectures where duplicative data extractions drove up operational overhead and analytics data lagged.

Moving to a Streamhouse architecture powered by Confluent decouples data producers from consumers, transforming high-volume event streams into governed, reusable real-time data products that flow seamlessly across operational and analytical systems.

With Confluent handling infrastructure automation and providing seamless enterprise connectivity, our developers operate in a standardised environment without the friction of building custom integrations or micro-managing data movement.

This continuous data foundation has dropped our time-to-insight from months to days and slashed reporting times, empowering our teams to focus on delivering high-value AI for our 800,000 employees,” said Thomas Mahaffey, SAI Chief Data Strategist, Accenture.

Mahaffey put two measurable outcomes on record. Accenture reported time-to-insight falling from months to days while serving 800,000 employees. It also cited lower operational overhead from replacing duplicative extractions and point-to-point connections.

Another customer, Extend, said it moved away from traditional message queues and static data lakes while building systems for pricing, fraud detection, and merchant onboarding.

“As an AI-native commerce platform delivering personalised shopper experience and operational and financial advantage to merchants, Extend requires exceptional adaptability across rapid merchant onboarding, real-time pricing adjustments, and instant fraud detection.

We initially struggled with traditional message queues and static data lakes that lacked event replay capabilities, persistence, and strict schema enforcement at scale.

Implementing a 100% serverless Streamhouse architecture anchored by Confluent Cloud turns Kafka into our operational source of truth, enabling downstream microservices to asynchronously consume and query real-time data without breaking contracts.

This inherently decoupled streaming foundation directly provides the flexibility and responsiveness required to scale our operations, adapt to live market signals, and power real-time agentic AI workflows,” said Jonathan Kropp, Director of AI and Architecture, Extend.

Kropp described a 100% serverless Streamhouse architecture anchored by Confluent Cloud. He said it makes Kafka the operational source of truth and lets downstream services consume and query real-time data without breaking contracts.

Shaun Clowes, General Manager, Confluent and Data, IBM, framed the commercial pitch plainly:

“The next era of enterprise AI will be won by the agents that know what's happening in the business the moment they act. That's what a Streamhouse can deliver.

With Lightning Tables, our Data Streaming Platform gives each agent and application a live, governed view of the business. This means companies can act on fraud signals sooner, serve customers in the moment and run operations on what's true now versus what was true yesterday,” said Shaun Clowes, General Manager, Confluent and Data, IBM.

Clowes links the serving layer to three practical outcomes: earlier fraud action, in-the-moment customer service, and decisions based on current state. Confluent supplied the serving figures from internal tests; Clowes described the customer outcomes as intended benefits.

What has to hold in production?

General availability on AWS now puts those internal-test figures against production traffic. Confluent says actual performance depends on configuration and workload, making production validation the next checkpoint for teams evaluating the service.

Confluent's wider bet extends beyond query speed. Streamhouse aims to turn streaming data into governed, reusable context for agents, applications, and analytics systems without first centralizing every source.

If Confluent's internal-test figures hold under real workloads, the company will have narrowed a practical gap between enterprise data and real-time AI decisions.

Source: itbrief.com.au

Share this article:

More from Elena Vasquez

Elena Vasquez

Show full bio

Market editor covering technology and product news in artificial intelligence and machine learning at AI In Context.

446 articles

Related articles

  1. Cloudflare Brings OpenAI's GPT-5.4 to Agent Cloud for Enterprises
  2. IBM Targets Mainframe Modernization With Z Atlas AI Package
  3. OpenAI Ships GPT-5.3-Codex-Spark, a Coding Model Built for Real-Time Work
  4. OpenAI built an internal data agent on GPT-5.2 and Codex
  5. OpenAI Posts First Benchmark Results for Jalapeño Custom Inference Chip

« Previous articleNext article »