AI vision models are improving fast, but construction photos still challenge them. To understand where things truly stand, we ran six leading multimodal models across three AEC domains: MEP, structural engineering, and building science/envelope. We see how well they identify components from raw field images.

The task is simple to describe but hard to execute:
Given a photo, can the model name the primary component and/or condition and map it to a fixed construction vocabulary?

Below are the updated results.

TL;DR

MEP / HVAC

• GPT-5 leads at 70%

• Gemini 3.5 Pro at 60%

• GPT-4.1 at 60%

• Gemini 2.5 Pro at 55%

• Grok 4 at 40%

• Claude Opus 4.1 at 40%

Structural

• Grok 4 tops at 38.89%

• Gemini 3.5 Pro at 38.89%

• Gemini 2.5 Pro at 33.33%

• GPT-5 at 27.78%

• GPT-4.1 at 22.22%

• Claude Opus 4.1 at 16.67%

Building Science / Envelope

• Gemini 3.5 Pro leads at 72.50%

• GPT-5 and GPT-4.1 tied at 60%

• Gemini 2.5 Pro at 59.48%

• Grok 4 at 47.50%

• Claude Opus 4.1 at 33.16%

How to Interpret These Numbers

Component identification means identifying the dominant element in an image and mapping it into a construction taxonomy. Examples include:

• MEP: air handler, VAV box, condensing unit, circulator pump

• Structural: W-section beam, rebar cage, embed plate, tendon anchor

• Envelope: AVB membrane, TPO roofing, mineral wool insulation, veneer ties

The three domains test different aspects of a model’s ability to see, classify, and reason.

MEP / HVAC Performance

MEP tends to be the friendliest domain for general-purpose models. Equipment has distinct forms and often includes visible:

• Data plates

• Manufacturer logos

• Tags and labels

• Gauges and controllers

Models that can fuse text recognition with pattern recognition perform the strongest.

Why GPT-5 leads (70%)

It combines reliable OCR, recognition of repeated manufacturer geometries, and stronger grounding on mechanical systems. GPT-5 is good at distinguishing between visually similar units (e.g., RTU vs AHU, VFD vs pump).

Gemini 3.5 Pro and GPT-4.1 at 60%

Both are competent on front-facing shots and readable plates but less consistent when text is obscured or partially visible. It may be due to the text in plates.

Gemini 2.5 Pro at 55%

Performs respectably but drops more sharply when equipment is obstructed or when the scene contains multiple competing elements.

Grok 4 and Claude Opus 4.1 (40%)

Both models struggle when there isn’t a clear silhouette or readable text, and often misclassify peripheral hardware as the primary component.

Structural Engineering Performance

Structural components remain the hardest area for AI vision.

Images often include:

• Dust-covered reinforcement

• Overexposed concrete

• Complex assemblies with subtle geometry

• Unlabeled embeds, welds, and hardware

This reduces reliance on text and forces models to interpret pure shape and context.

Why Grok 4 is the top performer (38.89%)

Grok 4 appears more sensitive to fine geometric cues. It detects plates, studs, bar alignments, and connection details more reliably than the others, although still far from perfect.

Gemini 3.5 Pro matches Grok 4 (38.89%)

It performs well when structural elements are cleanly lit or have distinct profiles. However, accuracy declines when reinforcement or connections are partially covered.

Lower performance across the board

Even the best general-purpose model stays below 40%, underscoring how challenging structural imagery is for today’s vision systems.

Building Science / Envelope Performance

Building science sits between MEP and structural. Many envelope assemblies include:

• Distinct layer sequences

• Consistent material textures

• Regular shapes (rigid insulation, cladding, ties)

• Annotations or labels in detail drawings

This makes the domain more “taxonomically compressible.”

Gemini 3.5 Pro leads comfortably (72.5%)

It excels when images show clear membranes, insulation layers, or labeled details. Gemini handles text and hatch interpretation well, giving it an edge.

GPT-5 and GPT-4.1 at 60%

Both models are reliable but slightly less consistent on fine-grained distinctions, especially between similar membranes or insulation products.

Gemini 2.5 Pro at 59.48%

Only slightly behind GPT-5, performing well on standard envelope setups.

Grok 4 at 47.5%

Better at geometry than texture, resulting in lower performance on membranes and surface treatments.

Model Behavior Patterns

Across domains, several consistent error types appear:

• Taxonomy drift: “fan coil” vs “unit heater,” “beam clamp” vs “hanger clip”

• Over-generalization: calling all reinforcement “rebar”

• Context misses: parapet membranes mistaken for below-grade waterproofing

• Hardware confusion: small connection components are frequently misidentified

• Multiple components in frame: models focus on the dominant object and ignore secondary ones

A Note on Gemini 3 Models

The Gemini 3 family (3.5 Pro & 2.5 Pro) shows interesting behavior trends:

• Gemini 3.5 Pro
  Strong in building science and solid in MEP. Performs well when components are labeled or visually consistent. More sensitive to lighting and obstructions compared to GPT-5.

• Gemini 2.5 Pro
  Capable but less robust. Performs best on textbook-like images and less well on messy field conditions.

Takeaway

General-purpose multimodal models are improving, but their accuracy varies widely by domain:

• MEP: Models can do reasonably well using text + silhouette

• Structural: Still extremely challenging; the ceiling remains low

• Building science: Models perform strongest where assemblies follow predictable layer logic

For real-world construction workflows, these results show that relying on generic AI still leaves large accuracy gaps—especially in structural engineering—while highlighting promising performance in envelope and MEP identification.

If you want, I can also turn this into a polished LinkedIn post or generate consistent charts for all domains.