The factory inspection problem that clean benchmark images miss

MVTec AD 2 makes lighting changes and difficult defects visible in anomaly-detection evaluation.

An inspection model can become very good at recognizing the photographs it was given and remain unreliable at recognizing the parts a factory actually makes. Lighting, coatings, camera position and ordinary supplier variation all alter the image. An anomaly detector must decide which changes matter to quality and which are harmless. That distinction is harder than producing an attractive colored heat map.

MVTec AD 2, first described in a March 2025 research paper, is designed to challenge industrial anomaly detection beyond easier benchmark conditions. Its eight scenarios include more than 8,000 high-resolution images. The work examines difficult objects and lighting shifts, and part of its evaluation uses withheld ground truth. The dataset carries a noncommercial license. It is a research benchmark, not a free substitute for a manufacturer's licensed production data. [Dataset](https://www.mvtec.com/research-teaching/datasets/mvtec-ad-2), [research paper](https://arxiv.org/abs/2503.21622).

Anomaly detection is appealing when defective examples are scarce. A model learns patterns from acceptable parts and flags departures. But an unusual image is not necessarily a defective product. A changed background can dominate the score while a tiny crack remains unnoticed. Conversely, a legitimate surface finish can trigger so many alerts that operators stop trusting the system. The operating problem therefore begins with a quality definition, not with a network architecture.

For an illustrative connector-inspection pilot, the quality team might define bent pins, missing components and contamination as separate defect categories. It should also identify acceptable changes such as different approved molding shades. Photograph both groups across relevant shifts, cameras and supplier batches. Preserve the original images, inspection decisions and uncertainty; avoid silently treating every operator decision as perfect ground truth.

Separate data by production batch or collection period. Neighboring frames of the same component belong together. Otherwise a training set and a test set may contain almost the same scene, exaggerating the model's ability to generalize. Reserve a later production period for acceptance testing, and include a planned lighting change to examine how the detector behaves when the imaging setup drifts.

Start with a transparent baseline. A simple measurement or conventional vision rule may solve a clearly defined dimension check more predictably than a learned anomaly score. Use machine learning where texture, appearance or variability makes the baseline inadequate. Anomalib provides implementations and deployment tools for anomaly-detection research; it can shorten experimentation, but using its export workflow does not certify the resulting inspection station. [Anomalib documentation](https://anomalib.readthedocs.io/en/stable/).

The acceptance report should show missed defects and unnecessary rejects separately. An overall image score can hide a failure on the defect the factory cares about most. Record results by defect type, material batch and operating condition, then examine the troublesome images with quality engineers. If the system locates an anomaly, check whether that location corresponds to the actual fault rather than a nearby label or reflection.

Threshold selection is an economic and quality decision. Tightening a threshold may reduce missed faults but increase rejected good parts and manual inspection. Define who can approve the tradeoff and what happens to uncertain cases. For safety-related components, an unexplained score must not casually replace established acceptance requirements. A reviewed hold-and-inspect route is often more useful than forcing every image into pass or fail.

Budget for stable illumination, lens maintenance, triggering, part presentation and integration with the reject mechanism. Those physical details can determine performance more than a change of model. Include storage and labeling effort, software maintenance and periodic revalidation. The cited sources do not establish a universal camera price, deployment fee or savings percentage, so a purchasing decision needs an actual quote and a measured local baseline.

A useful system also detects when it should stop making decisions. A disconnected camera, dirty lens or radically different product should create an understandable inspection-unavailable state and route material to the agreed fallback. Tracking those events prevents an apparent reduction in rejects from concealing periods when inspection was ineffective.

The contribution of harder benchmarks is to expose weaknesses before they become production surprises. A factory earns confidence through separate data, representative defects, understandable rejection decisions and observed performance after integration. That process turns anomaly detection from a promising experiment into an accountable part of quality control.