Why do users find our VR app hard to use?

Almost always because screen conventions were carried into a headset where none of them apply. There is no cursor, so selection happens with a ray from a controller, a hand or a gaze direction, and targets sized for a mouse are genuinely hard to hit. Panels have a distance, size and angle now, and text that looked crisp in the editor can be unreadable through a lens across a room. Anything that requires holding an arm out in front of the body is pleasant for the ninety seconds of a demo and exhausting in a twenty-minute session, so frequent controls belong low, near the body or attached to a hand. Density is the other common mistake, because a dashboard's worth of controls becomes a wall in front of someone's face. There is usually a second cause underneath all of that: first-time users do not know they can turn around, how far to reach, or that an object can be picked up at all, and nothing in the app told them. Onboarding in XR is a designed scene that teaches one mechanic at a time inside the real task, not a tooltip layer added at the end.

Should an XR app use hand tracking or controllers?

It depends on who is using it, how often, and what their hands are doing. Controllers are precise and full of buttons, which matters more than it sounds, because a physical press is unambiguous and the haptic bump when something is grabbed removes a whole category of confusion. That makes them the right default for training and simulation where realistic manipulation carries the value, at the cost of objects to hand out, charge, pair, lose and clean. Hand tracking removes all of that hardware and is strong for occasional users, public-facing experiences and shared settings, but there are no buttons, so every command has to be a gesture people can discover and repeat, there is no haptic feedback, precision is lower, and tracking degrades in poor lighting, when one hand blocks the other, or when the user is wearing gloves. Gloves are not an edge case in the field, since the firefighters ERIS XR was built for wear them routinely. Gaze and dwell, often paired with voice, suits hands-busy overlay work where the person's real hands are on real equipment, with the caveat that voice fails in noisy environments. Frequent trained users can be taught an efficient input model; one-time users need one they already know.

How do people log in and type inside a VR headset?

The honest answer is that they should not have to. Pecking at a floating keyboard with two rays is slow, error-prone and unpleasant in front of colleagues, and the fix is not a better virtual keyboard, it is designing the typing out of the product. For sign-in, a badge scan, a QR code on a lanyard, a short numeric PIN or pairing with a phone app all beat entering an email address and password in mid-air, and on shared devices that is a rollout requirement rather than a nicety. For everything else, replace free text with pick-lists, presets and scanned codes, pre-populate from the system of record so an existing work order arrives in the headset instead of being retyped, use voice dictation where genuine free text is unavoidable and the room is quiet enough, and move long-form work such as reviews, reports and annotations to a companion phone or web view. Text entry is the requirement teams discover late, and it is far cheaper to design around at the start than to retrofit once the flows exist.

XR Interaction Design: Why Immersive Apps Feel Wrong, and What Fixes It

TL;DR: Most XR pilots that quietly die are not ugly. They are awkward. The screen UI got ported into a headset, the input model was inherited from whatever shipped in the box, typing turned out to be impossible, and first-time users were dropped into the scene with no idea they were allowed to reach for anything. Interaction design is the layer that decides whether people use an immersive app twice, and it is the layer most proposals skip. Here's what actually breaks, how we choose an input model, and how to test any of it before the budget is spent.

There is a specific piece of feedback that means an XR project is in trouble, and it does not sound like trouble. It sounds like "that was cool."

Said once, by someone taking the headset off, without asking when they can use it again. The graphics were fine. What was not fine was the twenty seconds where they could not work out how to select anything, the menu that made them hold an arm up like they were hailing a taxi, and the login screen with a floating keyboard.

None of that is a rendering problem. It is interaction design, and in XR it is a much larger share of the build than teams expect, because almost none of the conventions you rely on in software exist here.

Your screen UI does not survive the trip into a headset

On a screen there is a cursor, a click, a keyboard and a flat plane at a fixed distance from someone's eyes. In a headset every one of those assumptions is gone.

There is no cursor, so the pointer is a body part. Selection happens with a ray from a controller, a hand, a gaze direction or a fingertip in space, and each has different precision. Targets that are trivial to hit with a mouse become genuinely difficult, so the control sizes and spacing from your web app are simply wrong.

Depth is now a design decision. Text that looked crisp in the editor can be unreadable through a lens at three meters, and a panel placed too close makes people cross-eyed. Every panel needs a deliberate distance, size and angle.

Arms get tired faster than anyone plans for. A menu that requires holding a hand out in front of you is pleasant for the ninety seconds of a demo and exhausting in a twenty-minute session. Frequent controls belong low, near the body, or attached to a hand rather than floating in the middle of the view.

Density does not translate. A dashboard's worth of controls becomes a wall in front of someone's face. The move is almost never to reproduce the layout, it is to work out which three things a person actually does in the headset and give the rest back to the desktop. Port the outcome, not the interface.

The input model is a design decision, not a hardware default

Teams tend to inherit whatever input the device shipped with. It deserves an actual decision, because it changes the scope of the build and who can use the result.

Controllers

Precise, reliable, and full of buttons, which matters more than it sounds. A physical press is unambiguous, and the haptic bump when something is grabbed removes a whole category of confusion. Controllers are the right default for training and simulation, where realistic manipulation carries the value. The costs are logistical and human: they are objects to hand out, charge, pair, lose and clean, and a first-time user has to be taught the buttons before anything else can happen.

Hand tracking

Nothing to hold, nothing to charge, and pointing and grabbing are things people already know how to do, which makes it strong for occasional users, public-facing experiences and settings where passing a shared object between people is awkward.

The trade is real. With no buttons, every command has to be a gesture someone can discover and repeat, and with no haptics the app has to confirm actions some other way. Precision is lower, and tracking degrades with poor lighting, when one hand blocks the other, or when the user is wearing gloves. That last one is not a footnote in the field: the firefighters ERIS XR was built for wear gloves as a matter of course.

Gaze, dwell and voice

Look at a thing, pinch or dwell to select it. Low effort, no equipment, and a good fit for hands-busy overlay work where the person's real hands are on real equipment. Voice commands complement it well and fail badly in noise, which describes most of the industrial environments people want them in.

The questions that decide between these are not technical. How often will this person use the app, once ever or every shift? Are their hands free, and are they gloved? How precise does manipulation need to be? What is the lighting and noise like where this actually runs? Frequent trained users can be taught an efficient input model, while one-time users need one they already know. It is entangled with the hardware too, which is why we work through it alongside choosing an XR headset rather than after.

Text input is the wall everyone hits late

Typing in a headset is miserable. Pecking at a floating keyboard with two rays is slow, error-prone and faintly humiliating in front of colleagues, and it shows up in the least avoidable places: signing in, filling a form, adding a note, searching a catalogue.

The answer is not a better virtual keyboard. It is designing the typing out.

Authenticate without typing. A badge scan, a QR code on a lanyard, a short numeric PIN or pairing with a phone app all beat an email address and a password entered in mid-air. On shared devices this is a rollout requirement, not a nicety, and we cover the identity side of it in enterprise XR rollouts.
Replace free text with choices. Pick-lists, presets and scanned codes cover most of what people would otherwise type.
Pre-populate from the system of record. If the work order already exists in your back office, the headset should receive it rather than ask someone to retype it.
Use voice for genuine free text, with the caveat that it needs a quiet room and a way to correct mistakes.
Move long-form work off the device. Reviews, reports and annotations are usually better on a companion phone or web view.

Comfort, movement and the users who quietly opt out

How a person moves through the space is one of the earliest scoping questions, because it is tied to the physical room. Seated, standing in place, or free room-scale movement each imply different content, different safety assumptions and different site requirements. A scenario designed for people walking around a training bay does not survive deployment to an office with a two meter square of clear floor.

Where the virtual space is bigger than the real one, movement has to be artificial, and this is where comfort gets decided. Smooth continuous movement feels natural to people used to games and makes a meaningful share of everyone else feel ill. Teleporting or snap turning is less elegant and far more tolerable. Offer both, default to the comfortable option, and let confident users switch.

Underneath all of it sits frame rate. A headset draws the scene twice, dozens of times a second, and when the image stops keeping up with the head people feel it physically, which is why the 3D content pipeline and the frame budget are not separate conversations from usability.

Treat comfort as accessibility rather than polish, because the failure mode is silent. The person who felt queasy rarely complains. They just stop signing up for sessions, and your adoption number slowly explains itself.

The first ninety seconds decide the rest

Experienced XR users are a small minority of any workforce. Everyone else arrives not knowing that they can turn around, that they are allowed to walk, how far to reach, or that the object in front of them can be picked up at all.

Onboarding in XR is a designed scene, not a tooltip. The pattern that works is teaching one mechanic at a time inside the real task, in the order the task needs them, with something forgiving to practise on. A trainee's first session should be spent on the procedure you bought the system for, not on discovering the menu.

The same applies to making the world legible once they are in it. Anything grabbable should announce itself when a hand or ray comes near, every action needs an immediate response visually and with sound, and nothing should fail silently, because a grab that did not register with no feedback reads as a broken app rather than a missed target. In shared sessions, other people need to be visible as people: where they are standing, what they are looking at, what they are pointing to. That presence layer is a large part of why mixed-reality collaboration feels collaborative rather than haunted, and it is the problem MR Camera, our multiplayer mixed-reality environment, exists to solve.

Non-technical users deserve a mention of their own. Nystag, our VR eye-tracking diagnostics build on the Vive Focus 3, puts patients in a headset in a clinic. They did not choose to be there, they may be unwell, and the session has to work the first time with no learning curve at all. When that is your audience, simplicity is not a preference, it is the requirement.

You cannot evaluate any of this in a document

Interaction quality is not visible in a wireframe, a storyboard or a rendered video. It exists only in the headset, in the room, on the actual hardware.

So build a rough version of the riskiest interaction in the first week, before the content and before the scenario library. Then test it with people from the real user group in the real conditions: the gloves, the noise, the lighting, the cramped floor space, the person who has never worn a headset. Five real users surface more than a room full of stakeholders reviewing a video, and what you are measuring is whether they finished the task, how long it took and where they got stuck, not whether they enjoyed it. This is the same instinct behind prototyping the risky interaction first when we scope what an XR app costs, and it is the cheapest insurance in the project.

Questions worth asking before you commission a build

What are the three things a person actually does in the headset? If the answer is a feature list, the interface has not been designed yet.
Which input model, and why that one for these users? The answer should reference gloves, hygiene, frequency of use or precision, not the device's default.
Where does someone have to type, and how have you removed it? Especially sign-in.
Seated, standing or room-scale, and does every site have that space?
What are the comfort options, and what is the default?
What does the first ninety seconds look like for someone who has never worn a headset?
When will we try the core interaction on real hardware with real users? If the answer is not in the first two weeks, the risk is being deferred, not managed.

The bottom line

XR projects rarely fail on visuals. They fail on the gap between an experience that demos well and one a person will put on their head again tomorrow, and that gap is almost entirely interaction design. Port the outcome rather than the screen layout. Choose the input model against your real users and their real environment. Design typing out of the product. Treat comfort and onboarding as requirements rather than polish. Then prove the riskiest interaction in a headset in week one, because that is the only place the answer exists.

Have an XR concept and want to know whether the interaction actually works before you fund the build? Book a demo and we'll pressure-test it honestly, including telling you when a phone or a plain web app gets you there faster. See our XR work: ERIS XR and Planes XR for ARCortex, Nystag clinical VR eye-tracking on the Vive Focus 3, and MR Camera shared mixed reality.