Products & Tools

AWS folds synthetic data, auto-labeling and human review into SageMaker AI

AWS added synthetic data generation, quality evaluation, transformation and auto-labeling to SageMaker AI on November 17, 2026, with jury-based confidence routing rows to human reviewers.

AWS Adds AI Data Preparation Tools to Amazon SageMaker AI
AWS Adds AI Data Preparation Tools to Amazon SageMaker AIThisisEngineering / unsplash
By Sophie Lindqvist5 min read

Why it matters

  • AWS announced the SageMaker AI data preparation tools on November 17, 2026, available in US East (N. Virginia) and US West (Oregon).
  • Auto-labeling uses agreement across several models as a confidence score, routing low-agreement rows to a review queue based on Label Studio Community Edition.
  • The tools require SageMaker Python SDK version 3.30.0 or later and a DataPreparer class with generate(), evaluate() and transform() methods.
  • Data-producing jobs register outputs in an account hub, with datasets passed between stages by ARN.
  • Splunk, a Cisco company, used the synthetic generation tools for its Agent Observability evaluation models.

AWS announced new AI data preparation capabilities for Amazon SageMaker AI on November 17, 2026, adding synthetic data generation, quality evaluation, transformation, auto-labeling and human review directly to model development workflows. The tools are generally available now in the US East (N. Virginia) and US West (Oregon) regions.

The release matters because data preparation usually sits scattered across separate labeling tools, generation scripts and validation jobs. SageMaker AI now consolidates generation, evaluation, transformation and labeling into one managed dataset path, where a team can use a single capability inside an existing process or chain several together. Specifications and code stay available to inspect and change as the dataset moves toward training or evaluation.

The stakes are real for anyone paying for GPU time. Validation now runs ahead of training compute, so teams can catch defective records before a training run burns budget.

What does synthetic generation actually produce?

Synthetic data generation creates training, evaluation and test records from a specification, with optional grounding in seed data and documents. It doesn't just emit rows. It also produces a code-based pipeline that users can review, modify, reuse and scale, which makes the generation process itself open to inspection rather than a black box.

Quality evaluation works alongside it. It can apply a customer's own business rules, built-in metrics, and LLM-as-a-judge metrics, where a language model assesses the data. It finds duplicates and near-duplicates, and it surfaces label noise or answer leakage — the failure modes that quietly degrade a model's training signal.

Takeaway: AWS is treating the dataset as code, something you can read, audit and rerun.

How does the cleanup-to-training chain work?

Transformation reshapes records through generated code, built-in deduplication, merging, splitting and filtering, or user-supplied Python running at scale. AWS's example training chain formats the resulting data for supervised fine-tuning, direct preference optimization or reinforcement learning, checks it for quality, and then hands it to SageMaker AI model customization.

The sequence puts dataset validation before training compute. That ordering is the quiet point of the release: teams get a chance to fix bad records before they spend money on a training run.

Takeaway: check the data first, train second — the pipeline now enforces that order.

How do confidence scores route work to humans?

Auto-labeling can start from three positions: an unlabeled dataset, a small labeled seed set, or no labels at all. Built-in tasks include classification, named entity recognition, extractive question answering, ordinal rating and summarization. Custom tasks can also be defined.

The mechanism is a jury. For tasks with a fixed set of possible answers, several models label each row, and their level of agreement becomes that row's confidence score. A short labeling specification is validated against the user's own examples before the full dataset is processed.

Rows below the selected threshold drop into a SageMaker Studio review queue built on the open-source Label Studio Community Edition. Outside subject-matter experts can correct them in the review app. Those corrections feed back into auto-labeling, and teams can generate consensus metrics and labeling reports from the loop.

Takeaway: model agreement acts as a cheap triage signal, and humans only see the rows where the models disagree.

What changed for developers?

Developers can call these stages through the SageMaker Python SDK, or hand the work to a coding agent equipped with the AWS AI-ML skill, which can select and run the stages from SageMaker Studio JupyterLab, Kiro or a local IDE. The SageMaker AI documentation specifies SDK version 3.30.0 or later and a DataPreparer class with generate(), evaluate() and transform() methods. The generate() method covers both synthetic data generation and auto-labeling.

Synthetic generation, auto-labeling, quality evaluation and transformation all run as SageMaker-managed jobs. The handoff mechanics matter here:

  • Data-producing jobs write records to Amazon S3 and register them in an account hub as DataSet content.
  • The SDK's DataSet object carries an Amazon Resource Name (ARN) that can be passed to the next stage.
  • Transformation publishes to the same hub as its input, usually as a new version of that dataset.
  • Quality evaluation writes results to S3 without registering another dataset.

The coding assistant drafts specifications, checks the environment and requests approval before running a job or creating resources. It can also save job runs as a notebook or script, giving teams a reusable artifact they can run again.

Takeaway: datasets move between stages by ARN, so every step leaves a managed, addressable trace.

Where does the data live?

The workflow uses a customer-controlled AWS Identity and Access Management execution role. Source data, generated datasets and labels stay in the customer's own S3 buckets in the selected AWS Region, keeping storage and access within the permissions and location the customer selects.

That design will matter to regulated customers, who can point to where every record sits and who touched it.

Who is already using it?

Splunk, a Cisco company, used synthetic data generation while building evaluation models for its Agent Observability product. Yash Sheth, director of engineering for Splunk Agent Observability, said the tools produced statistically representative datasets quickly while preserving patterns its models depend on. He also said the SageMaker integration reduced data wrangling for its data scientists and customers.

For developers, the operational change is that generation, checking and correction can now feed model development through one managed dataset path. AWS has paired that automation with inspectable artifacts, approval requests and customer-held S3 records, so control over how data is produced remains part of the preparation workflow itself — a signal of where AWS expects the data layer of AI development to settle.

Source: unite.ai

Share this article:

More from Sophie Lindqvist

Sophie Lindqvist

Show full bio

Staff writer covering regulation, policy and their impact on artificial intelligence and machine learning at AI In Context.

749 articles

Related articles

  1. Islo Raises $15 Million for Controlled AI Software Factories
  2. Hedra Launches Model Factory, Cuts a 2K Image Edit From 22 Seconds to 3.6
  3. Industrial AI Moves From Demos to Tightly Controlled Deployment
  4. Rockwell Brings NVIDIA Omniverse Into Emulate3D for Factory-Scale Simulation
  5. Adobe's EditAgent Makes AI Judge Its Own Photo Edits With Hard Numbers

« Previous articleNext article »