Why does pinch remain such a common selection gesture?
- Deliberate selection
- Tactile feedback: thumb meets finger
Investigating how people expect to use their hands in virtual reality — and how feedback can shape a smaller, more understandable set of gesture interactions. A completed 2023 MSc research project, presented as a design investigation rather than a shipped headset feature.
The dissertation investigated two connected questions: why pinching had become a primary gesture for many VR headset interactions, and whether a proposed alternative gesture vocabulary could be socially acceptable to users. The work used Meta Quest 2 as its main practical reference, alongside a comparison of headset interaction approaches.
Why does pinch remain such a common selection gesture?
What alternative gestures could people understand, remember and feel comfortable performing for everyday VR functions?
A research question tested through elicitation, a simulated prototype and feedback, not a deployed interaction system.
The literature review considered the Midas touch problem — accidental activation when an interface interprets movement as intention — alongside the tactile contact between thumb and finger that accompanies a pinch. This helped frame controlled, recognisable gestures as a design consideration rather than assuming the most familiar movement is always the safest interaction.
A hand moves through the interface
Movement is read as intent
An unintended selection
A deliberate moment: thumb meets finger before anything is selected
The study combined a comparative review, an online questionnaire, gesture-elicitation interviews and a later simulated interaction evaluation. Each stage answered a different question; the numbers below describe those stages separately and should not be added together as unique participants.
Compared Meta Quest 2, HTC Vive Focus 3 and the then-announced Apple Vision Pro as a 2023 snapshot of hand-tracking and interaction approaches. This was a desk-based comparison, not a hardware benchmark.
38 responses
An online Google Forms questionnaire explored headset experience, awareness of hand tracking, perceived importance and preferred interaction methods. Half the respondents reported prior VR headset experience.
15 interviews
Participants acted out the gestures they expected to use for familiar functions such as selection, scrolling, playback and switching applications. The exercise explored mental models, rather than recognition performance.
14 participants
Four task videos supported Wizard-of-Oz evaluation of a proposed gesture set. Feedback from UX/VR expert reviewers also informed a later simplification. No working native gesture-recognition prototype was tested.
Each stage is a separate sample. The numbers are not added together and are not a measure of reach or impact.
Source: MSc dissertation (2023), methodology and evaluation, pp. 15–38; accompanying academic presentation. These were modest research samples, not a representative cross-cultural population study.
Interview participants were asked to perform gestures for hypothetical VR functions, without being directed to the intended answer. The resulting suggestions created a useful tension: gestures that feel familiar on phones or touchscreens can behave differently in an open, camera-tracked VR interface.
A pointing or nudging motion felt intuitive to many participants.
The concept retained pinch for selection because deliberate contact provides a clearer confirmation action in the absence of a physical surface.Vertical hand movement felt aligned with scrolling. A horizontal wave initially appeared useful for changing applications.
Later evaluation revealed that changing applications and opening recent applications were too similar as separate actions.
The questionnaire also allowed multiple interaction preferences. Its reported 32 of 38 selections for hand gestures should not be described as 32 people choosing gestures as their only or primary interaction mode.
The original plan was to explore gesture recognition in Unreal Engine with Meta Quest 2. An early attempt produced inverted hand skeleton meshes in the headset; time and support constraints prevented a reliable native prototype. Rather than present that attempt as a functional implementation, the project shifted to a video-based simulation of how the gesture interactions could work.
Attempted Native hand tracking · not completed
Exploratory hand-mesh setup, discontinued when an unresolved inversion issue blocked the planned interactive implementation.
An early attempt produced inverted hand skeleton meshes in the headset.
Change in approach
Completed Video simulation for evaluation
Created a conceptual VR interface and task flow using publicly available visionOS design resources as a visual starting point.
Combined interface screen recording with recorded hand movements, using compositing to demonstrate each action for participant feedback.
Participants watched the composited task videos and gave feedback on each gesture.
Layer 1Screen-recorded interface (Figma)
Layer 2Filmed hand gesture
ResultComposited demonstration video
Prototype disclosure: this was a Figma/Premiere video simulation evaluated through Wizard-of-Oz tasks. It was not a deployable Quest application, functional hand-tracking engine, or implemented visionOS feature.
The four simulated tasks tested whether the gesture ideas felt understandable in context. The main design issue was redundancy: switching directly between applications and bringing up recent applications appeared too close in meaning. The dissertation records 11 of 14 evaluation participants expressing confusion about these overlapping functions; expert feedback reinforced the concern.
11/ 14
evaluation participants expressed confusion between “jump to other” and “recent applications”.
Initial proposal: four new gestures + existing pinch
Within the simulated evaluation table
Limits: one simulated Wizard-of-Oz video study with 14 participants. Participants judged composited videos, not live hand tracking. This is not universal social acceptance or a product success rate.
Four new gesture candidates, plus the existing pinch action.
Vertical wave · Open palm · Horizontal wave · L-frame; pinch retained for selection.Remove horizontal wave as a separate app-switching action.
Keep the L-frame route to recent applications and use pinch for a deliberate selection. The final design proposed three new gestures, not five new ones.
This was a change in the concept based on observed confusion, not proof of universal social acceptance. The dissertation contains an inconsistent aggregate acceptability calculation, so this case study deliberately does not reproduce a headline acceptance percentage.
The final proposal is a small interaction vocabulary for generic VR functions, developed for this dissertation. These mappings are proposed designs, not shipped Meta controls or certified accessible interactions.
Vertical wave
A controlled, step-based movement through a list
Open palm
One recognisable gesture for two related playback states
Two-handed L-frame
Bring up the application overview
Existing pinch gesture retained for selection.
Existing gesture retained: pinch for selecting or confirming an item. The original video prototype and task flow are documented in the private research materials; this content-stage page intentionally does not publish recorded participants or third-party interface assets.
The most valuable outcome of the project was a clearer set of proposed interactions and a practical example of evaluating a gesture concept before engineering it. Feedback challenged an initial assumption that horizontal and vertical swipes could both translate neatly into VR. Keeping recent applications as a distinct L-frame action made the proposal easier to explain.
Four new gesture candidates became three after evaluation, leaving one clear route to recent applications.
A video simulation surfaced the overlap before any native gesture recognition was built.
Separate stages of 38, 15 and 14; findings reflect one study context, not a population.
Participants judged composited videos rather than live hand tracking in a headset.
Comfort, cultural suitability and accessibility need wider testing across users, contexts and devices.
There were meaningful limitations: the samples were small and not representative; the prototype was a video simulation rather than functioning gesture recognition; and confidence, comfort, cultural suitability and accessibility would require more rigorous testing across users, contexts and devices. Native implementation in a headset and further exploration of pointing-based selection were identified as future work, not completed outcomes.
This case study is based on the author's September 2023 MSc UX Engineering dissertation, Investigation on hand gestures in VR environment and a proposal of socially acceptable gestures, and its academic presentation. The original research documents remain private because they contain identifying details, third-party illustrations and research material.
Study-stage sample sizes and the final three-gesture proposal are drawn from the dissertation. This page does not claim that the prototype ran as an interactive Quest app, that the proposed gestures were implemented by Meta or Apple, or that a small exploratory sample establishes broad social acceptance. Any later diagrams or hand depictions built for the portfolio must be labelled as reconstructions rather than recovered originals.
Portfolio reconstruction
All hand illustrations, interface panels and diagrams on this page were drawn for the portfolio. No dissertation pages, original video frames, participant recordings or third-party headset interface assets are reproduced.
Source sections: dissertation pp. 15–29 (methods and elicitation), 30–35 (gesture selection and prototype), 36–38 (Wizard-of-Oz evaluation and revised proposal), and 39 (conclusion). Supporting academic presentation: methods, prototype and evaluation slides. No identifying participant responses are reproduced.