[Award-winning] Discerning User Activity in XR Through Biometric Side-Channels

I wrote this scientific paper with Jonathan Roscoe and Max Smith-Creasey. In this early work, we aim to highlight an important issue for those seeking to utilise XR (virtual reality, mixed reality, etc.) for secure applications.

Whilst a lot of public attention is on the gaming aspect of XR, there is an increasing population of enterprise users. This makes the security of those working in XR paramount. In our experimentation, we found that a simple wearable accelerometer (such as found in a smartwatch or phone) is sufficient to differentiate between different activities (in this case Half-Life: Alyx, Beat Saber or Pistol Whip).

With sufficient training data, we anticipate that it will be feasible to identify specific actions carried out by a user (such as entering credentials or interacting with applications in a specific way) which would enable eavesdropping on a user in VR.

Abstract

Extended reality technologies such as virtual reality are becoming increasingly common for enterprise applications. They have the potential to create secure multi-user environment in previously less-secure spaces, without the need for privacy filters or secure rooms. In this pilot paper we explore how malicious actors may be able to eavesdrop on a virtual reality session, by tracking the physical movements of a user. This is carried out using a third-party accelerometer, attached to the user. Through initial experimentation, we observe that specific actions and session types can be identified through visual analysis of the accelerometer. We posit there is substantial potential for sophisticated and automatic classification of user activity in VR. We discuss how this may enable eavesdropping by malicious actors, or could serve as a mechanism for improved security.

Best Presentation Award

This paper received the Best Presentation award at the 18th Annual IEEE International Conference on Intelligence and Security Informatics.