TL;DR: Reality capture means building your 3D content by scanning a real place or object instead of modeling it by hand. There are now three main ways to do it: photogrammetry, laser or LiDAR scanning, and Gaussian splatting, the newest, which can look close to a photograph from any angle. Scanning is the right call when the place itself is the point and it already exists. Modeling is the right call when people need to touch, change, relight or measure what they see. Most serious XR projects end up using both: a scan for what people look at, and a simple modeled layer underneath for what the app has to calculate. Here's how to tell which you need, what each method costs you in practice, and what to ask before you pay for a capture.
If you have seen a Gaussian splat demo, you probably understand why buyers are asking about it. Someone walks around a room with a phone, and an hour later there is a 3D version of it that looks like video. Glass, foliage, hair and shiny surfaces, the things that used to break photogrammetry, come through looking real.
That is a genuine step forward. It is also easy to commission the wrong thing on the back of a demo, because a splat that looks perfect in a viewer can still be the wrong asset for the app you are actually building. This article is about making that call before the capture day, not after it.
What "reality capture" covers
Three families of technique, each producing a different kind of 3D data.
Photogrammetry
Software finds matching points across many overlapping photos, works out where each camera was, and reconstructs a textured mesh: a surface made of triangles with images painted on it. It has been the standard way to scan objects and sites for years.
Meshes are what game engines and XR runtimes are built around, so a photogrammetry result slots into the normal pipeline. The weaknesses are well known. Reflective, transparent, thin and repetitive surfaces confuse it. Raw output is heavy and full of holes. Lighting from the day of the shoot is baked into the textures. Getting from a raw scan to something a headset can draw smoothly is real cleanup work, which we cover in the 3D content pipeline behind an XR app.
Laser and LiDAR scanning
A scanner measures distances directly and produces a point cloud: millions of measured points in space. Survey-grade terrestrial scanners are about accuracy. The LiDAR sensors on recent phones and tablets are much less precise but fast and convenient for a quick room scan.
Point clouds are the right source when dimensions matter: engineering, construction, facilities and anything that will be compared against drawings. They are not something you put in front of a user as-is. They usually get converted into meshes or combined with photos.
Gaussian splatting
The newest of the three, published as a research method in 2023 and adopted quickly. Instead of a surface, the scene is represented as a very large number of small, soft, semi-transparent blobs, each with a position, a shape and a color that can change depending on the angle you look from. The blobs are fitted to photos or video of the scene, and then drawn very quickly on a modern graphics chip.
That view-dependent color is why splats look so convincing. Reflections, sheen and fine fuzzy detail are captured as they appeared, rather than approximated by a material that has to be built by hand.
It is also why they behave differently from everything else in an XR project. A splat has no clean surface. It is closer to a photograph that happens to be 3D than to a model.
The decision: scan it or model it
Start with what the user will do with the content, not with which technique looks best in a demo.
Scan when the place is the point
Capture wins when the value is that this is the real place, as it really is today:
- Pre-visiting a site before going there in person, whether for a site survey, a property viewing or pre-planning a response to a specific building.
- Heritage and documentation, where the record of the place matters in itself. XR for museums and cultural heritage covers how institutions handle scan archives.
- Training in a specific environment, where trainees need to recognize the real space they will work in. See VR training for high-risk teams.
- Backdrops and context, where the scene is something to look at around the part of the app people actually interact with.
Model when people need to act on it
Hand-built or CAD-derived content wins when the app has to do things with the geometry:
- Physics and collision. Users walking, grabbing, opening and bumping into things need clean, closed surfaces.
- Change. Anything that has to show a future state, an alternative design or a step in a process. A scan only knows what was there on the day.
- Relighting. Time-of-day studies, emergency lighting, a darkened room. Captured lighting is fixed.
- Measurement. Exact dimensions that someone will rely on. VR architectural visualization is mostly a modeled-content story for this reason.
- Things that do not exist yet. Obvious, but it settles a lot of debates early.
Most real projects are hybrid
The common pattern is a captured visual layer for realism, with a simplified modeled layer underneath that the user never sees: floors to walk on, walls to collide with, hotspots, doors and equipment that respond. The scan is what people look at. The model is what the software reasons about.
Plan both from the start. Retrofitting collision and interaction onto a splat after the fact is much harder than scoping it in.
What Gaussian splats are good at, and where they bite
Because splats are the reason most buyers are asking, it is worth being specific.
Where they shine
- Visual fidelity for the effort. A capture with a decent camera and a careful walk-through can produce a result that would take a 3D artist a long time to approach by hand.
- Hard surfaces for older methods. Glass, polished floors, vegetation, fabric and clutter, which photogrammetry tends to mangle.
- Speed of capture. A shoot is mostly about coverage and consistency, not specialist equipment, which makes repeat captures of a changing site practical.
Where they bite
- No clean geometry. Collision, physics, accurate occlusion against the real world in AR, and navigation all need a separate mesh or proxy.
- Lighting is baked in. You get the light of the capture day, shadows and all. Virtual objects you add will not cast shadows into the splat or receive light from it without extra work.
- Editing is awkward. Removing a person who walked through the shot, a parked car or a stray artifact floating in the air is possible but not as straightforward as editing a mesh.
- Performance on standalone headsets. Drawing a large number of overlapping transparent blobs, twice per eye, at a frame rate that does not cause discomfort, is demanding. Scenes usually need to be trimmed, compressed and tested on the actual target device. A splat that runs smoothly on a desktop graphics card may not on a standalone headset. Choosing an XR headset and VR motion sickness explain why frame rate is not optional.
- File size and delivery. Raw splat files can be large. That matters for app size, download over site Wi-Fi and especially for WebAR in the browser, where every megabyte delays the first view.
- A moving ecosystem. Engine support, file formats and compression approaches are still settling. Build so the splat is a replaceable content layer, not something the rest of the app depends on. XR app maintenance covers building for change.
None of these are reasons to avoid splats. They are reasons to prototype on the target device before committing a whole project to them.
Planning a capture that you will not have to redo
Most scan problems are created on the capture day and discovered weeks later. A few things decide whether you get usable data.
Coverage
Every surface the user will see needs to be photographed or scanned from enough angles. Gaps become holes in a mesh or blurry, smeared areas in a splat. Decide in advance where the user can stand and look, because that defines what has to be covered well and what can be rough.
Consistency
Changing light, moving people, opening doors and passing vehicles all confuse reconstruction. Overcast days and closed sites help. If a building has to be captured while in use, plan shoots around quiet periods.
Scale and alignment
A scan has no idea how big anything is or where it sits in the world unless you tell it. If the content must line up with a real building in AR, or with other datasets, include reference measurements or survey control points. This is the same registration problem that sits at the heart of location-based AR.
Permissions and privacy
Capturing a real place can capture people, screens, documents and security-sensitive layouts. Agree what is in scope, blur or remove what should not be there, and decide who may access the data. A detailed scan of a building interior is sensitive in its own right. XR data and privacy covers the wider picture.
Ownership
Agree in writing who owns the raw photos, the processed scan and any derived assets, and in what formats they will be handed over. Raw captures are worth keeping, because processing methods keep improving and the same photos may produce a better result later.
What drives the cost
We do not quote figures without a scope. The main drivers are:
- Size and complexity of the area, and how much of it the user can get close to.
- Accuracy required. Looks right versus measures right are different budgets, and the second often needs survey-grade equipment.
- Number of captures. A one-time record differs from a site that has to be re-scanned as it changes.
- The interaction layer. Collision proxies, hotspots and modeled interactive objects layered on the scan.
- Target devices. Standalone headset, phone, browser or desktop, each with different limits on what a scene can contain.
- Cleanup and optimization, which is usually larger than the capture itself.
What an XR app costs breaks down the rest of an XR budget. For anything above a small scene, an XR proof of concept is the cheapest insurance: capture one room or one object, put it on the real target device, and check that it looks right and runs smoothly before scaling up.
Where Null Studio fits
We are an AI-native software studio building across AI automation, custom software and XR. Our XR work is where the scan-or-model question comes up in practice. ERIS XR, built for ARCortex, is an XR platform that helps firefighters pre-plan and train, blending digital and physical worlds in real time, the kind of product where a real building, the information about it and the interactions on top all have to work together. Planes XR, also for ARCortex, places live flight simulations into the real world using AR and live maps, which is the alignment problem every capture project eventually meets. MR Camera is a multiplayer mixed reality environment where several people place and interact with 3D models in shared space, and Nystag is clinical VR eye-tracking on the Vive Focus 3, where frame rate and device limits are not negotiable.
We will tell you when a capture is the right answer and when a modeled scene, or a hybrid, will serve the app better. If you are comparing studios, choosing an XR development company lists the questions worth asking us and anyone else. If your capture is feeding a live operational view rather than an experience, digital twin development is the better starting point.
Questions to answer before capture day
- Is the value that this is the real place, or that users can act on it?
- What will users do with the content: look, walk, grab, change or measure?
- Which device must it run on, and has a sample been tested there?
- Do you need it to look right, or to measure right?
- What simple geometry sits under the scan for collision and interaction?
- Does it need to line up with the real world, and against what reference?
- What is out of scope for privacy or security, and who may access the data?
- Who owns the raw captures and processed assets, and in what formats?
Considering a scanned environment, a Gaussian splat or a hybrid scene for your XR app? Book a demo and we'll help you decide whether to capture or model, and scope a test scene on your target device before you commit. See our XR work: ERIS XR and Planes XR for ARCortex, MR Camera shared mixed reality, and Nystag clinical VR.