← Back to selected work
Case Study 04 / UX in VR · MSc dissertation

When a gesture feels natural.

Investigating how people expect to use their hands in virtual reality — and how feedback can shape a smaller, more understandable set of gesture interactions. A completed 2023 MSc research project, presented as a design investigation rather than a shipped headset feature.

Portfolio reconstruction · not a working VR prototype
Project
Socially Acceptable Gestures in VR
Context
MSc UX Engineering · Goldsmiths, University of London
Period
2022–2023 · Dissertation submitted September 2023
Research prototype
Meta Quest 2 reference · Figma and Adobe Premiere Pro video

Natural does not always mean intentional.

The familiar act of pointing at something can also trigger an unintended selection. This project explored the tension between intuitive hand movements, deliberate feedback and whether a gesture feels appropriate to use around other people.

01 / Research questionsGoldsmiths MSc research · 2023

Why pinch? And what else?

The dissertation investigated two connected questions: why pinching had become a primary gesture for many VR headset interactions, and whether a proposed alternative gesture vocabulary could be socially acceptable to users. The work used Meta Quest 2 as its main practical reference, alongside a comparison of headset interaction approaches.

Question one

Why does pinch remain such a common selection gesture?

  • Deliberate selection
  • Tactile feedback: thumb meets finger
A literature-review question; not a claim to have invented or improved pinch.
??
Question two

What alternative gestures could people understand, remember and feel comfortable performing for everyday VR functions?

A research question tested through elicitation, a simulated prototype and feedback, not a deployed interaction system.

The literature review considered the Midas touch problem — accidental activation when an interface interprets movement as intention — alongside the tactile contact between thumb and finger that accompanies a pinch. This helped frame controlled, recognisable gestures as a design consideration rather than assuming the most familiar movement is always the safest interaction.

  1. 01

    A hand moves through the interface

  2. 02

    Movement is read as intent

  3. 03

    An unintended selection

  4. Pinch

    A deliberate moment: thumb meets finger before anything is selected

The Midas touch problem as an interaction sequence. Portfolio reconstruction.
02 / Research methodologySeparate stages · Not a combined participant total

From the landscape to lived preferences.

The study combined a comparative review, an online questionnaire, gesture-elicitation interviews and a later simulated interaction evaluation. Each stage answered a different question; the numbers below describe those stages separately and should not be added together as unique participants.

  1. Stage 01Desk review

    Existing interaction models

    • Meta Quest 2
    • HTC Vive Focus 3
    • Apple Vision Pro (announced)

    Compared Meta Quest 2, HTC Vive Focus 3 and the then-announced Apple Vision Pro as a 2023 snapshot of hand-tracking and interaction approaches. This was a desk-based comparison, not a hardware benchmark.

  2. Stage 02Questionnaire

    38 responses

    An online Google Forms questionnaire explored headset experience, awareness of hand tracking, perceived importance and preferred interaction methods. Half the respondents reported prior VR headset experience.

  3. Stage 03Gesture elicitation

    15 interviews

    Participants acted out the gestures they expected to use for familiar functions such as selection, scrolling, playback and switching applications. The exercise explored mental models, rather than recognition performance.

  4. Stage 04Simulated evaluation

    14 participants

    Four task videos supported Wizard-of-Oz evaluation of a proposed gesture set. Feedback from UX/VR expert reviewers also informed a later simplification. No working native gesture-recognition prototype was tested.

Each stage is a separate sample. The numbers are not added together and are not a measure of reach or impact.

Source: MSc dissertation (2023), methodology and evaluation, pp. 15–38; accompanying academic presentation. These were modest research samples, not a representative cross-cultural population study.

03 / Gesture elicitationParticipant expectations · Before the prototype

The obvious gesture wasn't always the right one.

Interview participants were asked to perform gestures for hypothetical VR functions, without being directed to the intended answer. The resulting suggestions created a useful tension: gestures that feel familiar on phones or touchscreens can behave differently in an open, camera-tracked VR interface.

Selection

A pointing or nudging motion felt intuitive to many participants.

The concept retained pinch for selection because deliberate contact provides a clearer confirmation action in the absence of a physical surface.
Scrolling and switching

Vertical hand movement felt aligned with scrolling. A horizontal wave initially appeared useful for changing applications.

Later evaluation revealed that changing applications and opening recent applications were too similar as separate actions.

  1. Pinch Selection Existing reference
  2. Vertical wave Scrolling Initial candidate
  3. Open palm Play / pause Initial candidate
  4. Horizontal wave Jumping between applications Initial candidate · later removed
  5. Two-handed L-frame Recent applications Initial candidate
Initial gesture-to-function mapping proposed after the elicitation interviews. Portfolio reconstruction: hand poses drawn for explanation, not traced from study footage.

The questionnaire also allowed multiple interaction preferences. Its reported 32 of 38 selections for hand gestures should not be described as 32 people choosing gestures as their only or primary interaction mode.

04 / Prototype strategyAn honest technical pivot

When an engine prototype stalled, the research continued.

The original plan was to explore gesture recognition in Unreal Engine with Meta Quest 2. An early attempt produced inverted hand skeleton meshes in the headset; time and support constraints prevented a reliable native prototype. Rather than present that attempt as a functional implementation, the project shifted to a video-based simulation of how the gesture interactions could work.

Attempted Native hand tracking · not completed

  1. Plan A

    Unreal Engine + Quest 2

    Exploratory hand-mesh setup, discontinued when an unresolved inversion issue blocked the planned interactive implementation.

  2. Blocked

    Inverted hand mesh

    An early attempt produced inverted hand skeleton meshes in the headset.

Change in approach

Completed Video simulation for evaluation

  1. Plan B

    Figma interaction flow

    Created a conceptual VR interface and task flow using publicly available visionOS design resources as a visual starting point.

  2. Simulation

    Adobe Premiere Pro

    Combined interface screen recording with recorded hand movements, using compositing to demonstrate each action for participant feedback.

  3. Evaluation

    Wizard-of-Oz tasks

    Participants watched the composited task videos and gave feedback on each gesture.

Layer 1Screen-recorded interface (Figma)

Layer 2Filmed hand gesture

ResultComposited demonstration video

Portfolio reconstruction: an illustration of the compositing approach, not a frame from the original video. No recorded participants or third-party interface assets are shown.

Prototype disclosure: this was a Figma/Premiere video simulation evaluated through Wizard-of-Oz tasks. It was not a deployable Quest application, functional hand-tracking engine, or implemented visionOS feature.

05 / Evaluation and iterationWizard-of-Oz feedback · 14 participants

One less gesture. A clearer system.

The four simulated tasks tested whether the gesture ideas felt understandable in context. The main design issue was redundancy: switching directly between applications and bringing up recent applications appeared too close in meaning. The dissertation records 11 of 14 evaluation participants expressing confusion about these overlapping functions; expert feedback reinforced the concern.

11/ 14

evaluation participants expressed confusion between “jump to other” and “recent applications”.

Wizard-of-Oz video evaluation · 14 participants · simulated study

Initial proposal: four new gestures + existing pinch

  • Vertical waveScroll · kept
  • Open palmPlay / pause · kept
  • Horizontal waveJump between applications · removed: overlapped with recent applications
  • Two-handed L-frameRecent applications · kept
  • PinchSelection · existing gesture retained
Design decision after evaluation. Portfolio reconstruction.

Within the simulated evaluation table

  • Vertical wave14 of 14 marked acceptable
  • Open palm14 of 14 marked acceptable
  • Two-handed L-frame14 of 14 marked acceptable

Limits: one simulated Wizard-of-Oz video study with 14 participants. Participants judged composited videos, not live hand tracking. This is not universal social acceptance or a product success rate.

Before evaluation

Four new gesture candidates, plus the existing pinch action.

Vertical wave · Open palm · Horizontal wave · L-frame; pinch retained for selection.
After evaluation

Remove horizontal wave as a separate app-switching action.

Keep the L-frame route to recent applications and use pinch for a deliberate selection. The final design proposed three new gestures, not five new ones.

This was a change in the concept based on observed confusion, not proof of universal social acceptance. The dissertation contains an inconsistent aggregate acceptability calculation, so this case study deliberately does not reproduce a headline acceptance percentage.

06 / Final gesture proposalThree new gestures + existing pinch

Three gestures, each with a job.

The final proposal is a small interaction vocabulary for generic VR functions, developed for this dissertation. These mappings are proposed designs, not shipped Meta controls or certified accessible interactions.

Proposed gesture · Academic concept

Vertical wave

Scroll

A controlled, step-based movement through a list

Physical movement
An open hand moves up or down in short, deliberate steps.
Intended interaction
Scroll through an application list in controlled steps rather than an unrestricted continuous flow.
Why it remained
Vertical movement matched how interview participants expected to scroll, and all 14 evaluations in the simulated study marked it acceptable.
Proposed gesture · Academic concept

Open palm

Play / Pause

One recognisable gesture for two related playback states

Physical movement
An open hand is held up with the fingers extended.
Intended interaction
Play and pause media as one related toggle, rather than asking users to remember two separate gestures.
Why it remained
One gesture covers two related states, so there is less to remember; all 14 evaluations in the simulated study marked it acceptable.
Proposed gesture · Academic concept

Two-handed L-frame

Recent applications

Bring up the application overview

Physical movement
Both hands form an L with the thumb and index finger, together framing a rectangle.
Intended interaction
Open recent applications. The separate horizontal-wave app-switching gesture was removed during iteration.
Why it remained
It became the single route to recent applications once the overlapping horizontal wave was removed; all 14 evaluations in the simulated study marked it acceptable.

Existing pinch gesture retained for selection.

Existing gesture retained: pinch for selecting or confirming an item. The original video prototype and task flow are documented in the private research materials; this content-stage page intentionally does not publish recorded participants or third-party interface assets.

07 / Reflection and limitsA research proposal · Not a product launch

The lesson was not to add more. It was to remove the overlap.

The most valuable outcome of the project was a clearer set of proposed interactions and a practical example of evaluating a gesture concept before engineering it. Feedback challenged an initial assumption that horizontal and vertical swipes could both translate neatly into VR. Keeping recent applications as a distinct L-frame action made the proposal easier to explain.

There were meaningful limitations: the samples were small and not representative; the prototype was a video simulation rather than functioning gesture recognition; and confidence, comfort, cultural suitability and accessibility would require more rigorous testing across users, contexts and devices. Native implementation in a headset and further exploration of pointing-based selection were identified as future work, not completed outcomes.

08 / Evidence and attributionSource-grounded portfolio summary

What this page does — and doesn't — claim.

This case study is based on the author's September 2023 MSc UX Engineering dissertation, Investigation on hand gestures in VR environment and a proposal of socially acceptable gestures, and its academic presentation. The original research documents remain private because they contain identifying details, third-party illustrations and research material.

Study-stage sample sizes and the final three-gesture proposal are drawn from the dissertation. This page does not claim that the prototype ran as an interactive Quest app, that the proposed gestures were implemented by Meta or Apple, or that a small exploratory sample establishes broad social acceptance. Any later diagrams or hand depictions built for the portfolio must be labelled as reconstructions rather than recovered originals.

Portfolio reconstruction

All hand illustrations, interface panels and diagrams on this page were drawn for the portfolio. No dissertation pages, original video frames, participant recordings or third-party headset interface assets are reproduced.

Source sections: dissertation pp. 15–29 (methods and elicitation), 30–35 (gesture selection and prototype), 36–38 (Wizard-of-Oz evaluation and revised proposal), and 39 (conclusion). Supporting academic presentation: methods, prototype and evaluation slides. No identifying participant responses are reproduced.