Software needs a user-friendly interface. There are various ways of evaluating whether this is the case. Most of these relate to the usability of desktop applications. The testing of mobile applications (apps) remains largely unnoticed in this respect. In most cases, the usability of applications is tested in the usability lab.

The usability lab has some disadvantages. For example, the test environment is very present, as the test subject operates the app while he/she is in an unfamiliar room and is observed by the test supervisor. Furthermore, his/her complete behavior is recorded. Only the predefined tasks are performed by the test subject and the collection of suitable test subjects is time-consuming [1].

An alternative, which is already common for websites in a non-medical context, is remote testing [1, 2]. In this case, the test subject visits the website from their own device, performs tasks, is possibly observed through the webcam and maintains virtual contact with the test administrator. The quantity and quality of findings in the usability lab and in remote testing are comparable (2, 4).

Remote usability testing for apps is currently uncommon. It has various advantages, such as extending the test phase [3]. In addition, the app can be tested under everyday conditions and all performance characteristics can still be recorded [5].

Disadvantages, on the other hand, are the local separation of the two parties. When using a smartphone, observation by the front camera to interpret facial expressions makes little sense and data protection is more complex. This also eliminates the often used eye tracking. A way must therefore be found to ensure that the data quality is appropriate. This quality refers not only to the test data, but also to the data input after the test phase in order to check reliability.

Hectic entries, which are usually made in stressful situations, are often prone to errors. Poor usability, e.g. searching for a button, also leads to stress.

Two physiological measures that are known to correlate with stress are skin conductance and heart rate [5]. Skin conductance (more moisture / sweat, increases conductance) is measured with an additional device. It is therefore unsuitable for remote testing as the test scenario becomes very present and it is unsuitable in everyday life.

The increase in heart rate and heart rate variability can be measured without additional hardware. The built-in smartphone camera can act as a pulse oximeter. Although this is also uncomfortable and unusual, depending on the smartphone, the data input can be put into context.

The duration of the input is also helpful. As various (button) events are provided with a timestamp during remote testing, this is easy to record [3]. This allows the expected duration of an input to be compared with the actual input duration.

The built-in accelerometer measures the acceleration of the smartphone. This makes it possible to determine whether the input was made at rest or in motion.

Asking the test subject about the current stress level is subjective and requires additional input, which in turn is not subject to quality control.

The "think-aloud method" is only rarely suitable for remote testing, as it is not suitable for everyday use [1]. If it is nevertheless used, there are various software solutions that can make a statement about the stress based on the voice.

With the help of various measuring points, statements can be made about the data quality. An increase in the pulse and heart rate variability, which are measured by the smartphone camera, as well as hectic movement of the smartphone, which is recorded by the accelerometer, and the comparison of the input duration with the expected values can determine a stress level and thus put the input into context.

This should be as automated as possible. Checking these inputs in addition to the actual testing would be a considerable additional effort.

Bibliography:

[1] Sauer, J, Muenzberg, A, Siewert, L, Hein, &, Roesch, N. (2019). Remote Testing of Usability in Medical Apps. 8th EAI International Conference, MobiHealth 2019, Dublin, Ireland, November 14-15, 2019, Proceedings, 3-17. ISSN: 1867-8211

[2] Brush, B, Ames, M & Davis, J (2004). A Comparison of Remote and Local Usability Studies for an Expert Interface, CHI Vienna, 1179-1182. doi: 10.1145/985921.986018

[3] Sauer, J, Muenzberg, A, Hein, &, Roesch, N. (2019). Simplify Testing of Mobile Medical Applications by Using Timestamps for Remote, Automated Evaluation. 2019 International Conference on Wireless and Mobile Computing, Networking and Communications (WiMob), 203-206. doi: 10.1109/WiMOB.2019.8923241

[4] Thompson, K, Rozanskim E & Haake, A. (2004) Here, There, Anywhere: Remote Usability-Testing That Works. Proceedings of the 5th Conference on Information Technology Education, SIGITE Salt Lake City, UT, USA, October 28-30, 2004, 132-127. doi: 10.1145/1029533.1029567

[5] Tullis, T, Albert ,B. (2008). Measuring the User Experience. Burlington, United States: Elsevier.