TL;DR: A headset is a room full of sensors strapped to someone's face. An XR deployment touches four separate layers of data: a 3D scan of your building, camera frames from wherever the wearer is standing, a continuous stream of body movement that is more identifying than people assume, and the application data you actually meant to collect. Most privacy reviews only ask about the last one. Here is what each layer really contains, what a security team will ask, and how to design the collection down before it becomes a procurement problem.
The XR project that stalls does not usually stall on price. It stalls on a question nobody prepared an answer for, asked late, by someone who was not in the kickoff.
What exactly is that thing recording?
It comes from information security, from legal, from a works council, from a hospital privacy officer, or from a customer who has just realised your pilot involves scanning their site. It is a fair question, and the honest answer is longer than most vendors expect. We are not lawyers and nothing here is legal advice, but almost all of the pain in these reviews comes from discovering the answer after the build rather than before it.
Four layers of data, and reviews only ask about one
It helps to separate what a headset touches into four categories, because they have completely different sensitivities and completely different owners.
The space. The geometry of the room the wearer is in: a scene mesh, a floor plan, a play boundary, and on passthrough devices, camera frames of the real environment.
The body. Head position and rotation many times a second, hand and finger joints, eye gaze where the device tracks it, voice, and physical attributes like height and the distance between someone's pupils.
The session. Which scenario, how long, which steps were completed, which mistakes were made, plus any recording or spectator stream of it.
The application data. The work order, the patient identifier, the customer record the headset pulled from your systems.
Nearly every privacy review opens on the fourth layer, because it is the one that looks like normal software, and it is usually the least novel of the four. The first three do not exist in a web app, and they are the ones nobody has a policy for yet.
The room is data, and it may not be your room
Spatial mapping is not a picture of a space, it is a usable model of it. That is the point, and it is also the concern. A scene mesh of a hospital ward, a substation or a customer's factory is a site document, and some organisations treat it exactly as they treat photographs of the same place, which is to say they do not allow them.
Passthrough makes it sharper. On a mixed reality device the cameras look at the real environment continuously in order to work at all, and the reasonable question from a security team is what happens to those frames. The answer differs by platform and by what your application asks for, which is why it has to be written down for the specific device and the specific build rather than answered in general terms. Applications rarely need to store camera frames, and the ones that genuinely do should say so loudly.
The value can be worth it. ERIS XR, the emergency response platform we build for ARCortex, helps firefighters pre-plan and train against real buildings, and that only works because the digital version of the building is accurate. But an accurate model of a real building is exactly the artifact a facilities team will want a policy for, and getting that agreement early is part of the project rather than an obstacle to it. Location is the quieter version of the same issue: a build like Planes XR, our AR project connected to live OpenSky data, is anchored to where the user physically is, and location history does not stop being sensitive because the app is immersive.
The body is the category people underestimate
This is where XR genuinely differs from the rest of software, and where the conversation gets ahead of most companies' internal policies.
A headset produces a continuous stream of body movement: how you hold your head, how you reach, how fast you turn, how tall you are. Motion data of this kind carries more identifying signal than people intuitively expect, which makes "the session log is anonymous" a weaker claim in XR than it would be elsewhere, and one worth scrutinising before you rely on it.
Eye tracking raises the bar again, because gaze reveals attention. In several jurisdictions, data derived from someone's body or behaviour this way can fall under biometric or special category rules, and the answer varies by where you operate and what you do with it. That is a question for your counsel, and it is far cheaper to ask before the build than after.
Sometimes the body data is the entire point. Nystag, our VR eye-tracking diagnostics build on the Vive Focus 3, exists to measure eye movement precisely, and in a clinic that measurement is patient data with everything that follows from it. We treat the handling as part of the minimum build there rather than a later phase, which is the same argument we make about clinical grade VR generally.
The useful principle in both cases is the distinction between raw and derived. A raw gaze stream is a heavy, sensitive artifact; the measurement you actually needed is usually a small number. Where the maths can happen on the device and only the result is stored, take that option, because it removes the risk instead of managing it.
Recordings, shared sessions and the people who did not opt in
Session recording is where XR privacy stops being abstract and becomes a conversation with a specific employee.
Recording a VR training session is not equivalent to logging a form submission. You are capturing where a named person looked, hesitated and fumbled, and in most workplaces that lands somewhere between performance monitoring and surveillance depending on how it was introduced. It is often legitimate and genuinely useful for debriefing a scenario. But it needs a stated purpose, a stated retention period and an answer to the obvious question of whether it will be used to evaluate the person, and in some countries it needs an agreement with employee representatives before you switch it on.
Shared sessions add people who never agreed to anything. In a multiplayer mixed reality environment like MR Camera, where several users place and interact with 3D models in a shared space, a spectator view or a recording may include another participant's real surroundings, someone walking through the room, or a whiteboard nobody thought about. The same applies to remote assistance, where a technician's camera feed shows a colleague at the next bench. Co-located deployments need a house rule about bystanders, and the technical part is usually simple once someone decides what the rule should be. We cover the mechanics in mixed reality collaboration; this part belongs in the rollout plan.
Write down where the data actually goes
The most useful deliverable in an XR privacy review is not a policy document. It is a one page map of every place data can end up, because the honest picture involves more parties than the buyer expects: your application and its backend; the device platform, which collects its own telemetry under its own terms and which you cannot answer for, so do not pretend to; the device management layer any real deployment needs, which knows which device was used, by whom and when; any analytics or crash reporting library you embedded, which is the one people forget; and the destination system, the LMS, the health record, the maintenance platform, that results are pushed into.
Two details on that map cause most of the trouble. The first is identity on shared hardware. A headset passed between fifteen people is anonymous by default, and the moment you want per person training records you have created a link between a body movement stream and a named employee. That is a deliberate step, not an implementation detail. We deal with the practical side of it, including badge and PIN sign-in, in enterprise XR rollouts.
The second is retention and deletion. Someone has to be able to say how long each layer is kept and what happens when an employee leaves, a patient asks, or a customer ends the contract. If nobody can answer, the default is forever, and forever is the answer that fails audits. Ownership of the data and the accounts it sits in belongs in the contract, which is the same argument we make about what you should own when someone else builds your software.
The cheapest fix is collecting less of it
Almost every problem above shrinks if the build collects less in the first place, and this costs nothing decided early and a great deal retrofitted.
Derive on device and store the result rather than the stream. Store the outcome of a scenario, not a video of it, unless someone has named the reason for the video. Keep raw signal separate from the training record, so the useful part is retained long term while the sensitive part expires quickly. Do not turn on a sensor the application does not use, and be able to say which ones you did. Ask for consent as the headset goes on rather than in an onboarding email, because in XR you can genuinely tell someone what the device is doing while they wear it. And check what your hardware supports before committing, since data controls and enterprise terms vary by device class and belong in choosing an XR headset rather than being discovered afterwards.
Questions worth asking before you commission a build
- Which of the four layers does this build actually touch, and which sensors are switched on? A vendor who has not thought about it will answer with application data only.
- Does anything leave the device, and where does it land? Ask for the one page map, including analytics libraries and device management.
- Are we recording sessions, why, for how long, and who can see them? If the answer involves a named employee's performance, that is a decision for HR and legal, not for engineering.
- How is a session linked to a person on shared hardware, and did we choose that on purpose?
- Is any of this biometric or special category data where we operate? Your counsel answers this, but the vendor has to give them an accurate description of what is captured.
- What is the deletion path? For an employee who leaves, a patient who asks, a site that ends the contract.
- Who at the customer site has to approve a spatial scan of the building, and have we asked them yet?
The bottom line
XR privacy problems are rarely caused by anyone being reckless. They are caused by a category of data that did not exist in the buyer's previous software, arriving in a security review written for web applications, three weeks before a go live date. The room is data. The body is data, and it is more identifying than it looks. Recordings involve people who did not sign up. None of that blocks a project when it is scoped at the start, written down honestly and reduced to the minimum the outcome actually needs. Cost and scope follow from those decisions too, which is why they belong in the same conversation as what an XR app costs.
Planning an XR build that will have to survive a security review? Book a demo and we will map what your specific build would capture, honestly, including the parts we would advise you not to collect. See our XR work: ERIS XR and Planes XR for ARCortex, Nystag clinical VR eye-tracking on the Vive Focus 3, and MR Camera shared mixed reality.