Usability Assessment of Delivery Applications for Visually Impaired People: A Case from Saudi Arabia

Abstract

Technology has become an essential part of our lives, and many of our daily tasks have become entirely dependent on it. For example, routine chores such as shopping for household necessities, booking travel tickets, going to places using all different kinds of transportations etc., are quickly done through mobile phones. And because of how it is easy to use mobile phones, we may forget that others, such as the visually impaired, may face many difficulties when using them.

In this research two of the most widely used delivery applications in Saudi Arabia, namely, Hungerstation and Mrsool were studied and evaluated in order to assess their usability for people with visual impairments. Evaluation results show that both applications have usability problems. Nonetheless, the results of the standard ISO usability metrics (Effectiveness, Efficiency and satisfaction) showed that Hungerstation is more usable than Mrsool.

Keywords

Visually impaired, Application usability, Usability metrics, Delivery applications.

Introduction

Over the past years, the development of mobile devices was accompanied by many tools and applications that help visually impaired people use them. People who have blindness can use technology today to do many things such as sending emails, surfing the Internet, making purchases, and much more. Applications such as screen readers and Braille keyboards allowed them to use various electronic devices independently, which solved many of the accessibility obstacles for blind individuals.

According to [1], it is estimated that 43 million people are blind worldwide in 2020, and around 295 million people will suffer in the future from moderate to severe visual impairment. As for Saudi Arabia, nearly one million people in the Kingdom have a visual impairment [2].

Due to the high percentage of people with visual impairments in Saudi Arabia, our aim in this research is to evaluate the usability of two popular and widely used local delivery applications namely: Hungerstation (https://hungerstation.com) and Mrsool (https://mrsool.co/). These two applications are specialized in delivering food from restaurants, cafes and grocery stores, as well as other types of delivery such as delivering necessities from one place to another.

Targeting the two delivery applications was based on their popularity by both sighted persons as well as visually impaired persons, based on interviews we conducted with visually impaired people. Therefore, this research aims to measure the usability of the selected delivery applications for visually impaired people, find the issues and problems in each application and finally give some recommendations to improve the usability of the applications to be used effectively by people with visual impairments.

Methodology

Our research has gone through several stages of data collection and analysis as follows:

Preliminary stage: Consists of two steps: (1) distributing a questionnaire to visually impaired people to find out the most commonly used applications and websites. Then (2) interviewing five blind people to find the most widely used delivery applications.

Testing stage: Consists of two steps: (1) pilot testing and (2) user testing. In the pilot testing, we tested three delivery applications (Hungerstation, Mrsool and Jahez) with four visually impaired people. We tested Jahez because it was among the top chart of the Apple store for food and drinks category. But it was excluded from the study due to its very poor usability.

While in the user testing step, ten visually impaired people (5 females and 5 males) used both applications, following our test protocol while observing and recording the whole experiment. Table 1 summarizes the participants’ demographics.

Table 1: Participants’ information.

All the participants in the study were users of iOS devices of various models from iPhone 6 to iPhone 12 pro. An overview of the required tasks was given for each of them before the start of the testing process. A Total number of twelve tasks were tested and were divided into the following types: the search and selection process, including (choosing the restaurant, choosing the required products), the checkout process, including (choosing the payment method, adding notes, choosing the delivery location, and finally order). Tasks were arranged to correspond to the sequence of their appearance in both applications, and were done entirely depending only on the VoiceOver of the iPhone.

Quantitative methods were used in this research through the use of ISO usability metrics [3] which are: effectiveness, efficiency and satisfaction. Effectiveness is defined as the ability of a user to execute a specific task in a given setting. It can be calculated by measuring the completion rate of the task or a stage in completing a task. Efficiency is the user’s ability to complete a given task quickly and accurately or time on task. Efficiency can be calculated by how long each task takes to complete. Both Effectiveness and Efficiency can be calculated in a laboratory setting or by observation. While the level of comfort and enjoyment experienced or able to accept the expectations and requirements by a user is referred to as user satisfaction, Satisfaction is a subjective concept that can be measured through a survey, such as a Likert-scale rating [4].

Results and Discussion

The results of the usability evaluation showed that the most popular local delivery applications need to be improved, as some problems were found that hinder usability for people with visual impairments. Also, usability testing should be conducted by blind people before and after the applications are released to their potential users to avoid any future problems.

To measure the usability of the two applications, the following usability metrics were used: effectiveness, efficiency and satisfaction. The previous metrics showed that Hungerstation is more usable than Mrsool, with an overall effectiveness rate of 92.27%, average time for each task equals 1:03 minutes, and satisfaction equal to 81.95.

While Mrsool application got an overall effectiveness rate equals to 90.83%, an average time for each task of 1:13 minutes and a satisfaction score equals to 50.25.

The study of these applications allowed us to know the problems in each application and elicited some suggestions to improve the usability of delivery applications used by visually impaired people. We also hope that our research becomes the beginning of conducting similar research on other local applications of all kinds.

Design Recommendation

Here are some suggestions to improve the usability of delivery applications based on our research findings:

1. Navigation with VoiceOver and gesture: the blind person relies on the movement of his/her hand to move between the content of the page while hearing VoiceOver to understand each element’s location, which must be taken into account when designing the page layout.
2. Adherence to the standard design recommendations provided by iOS and Android, such as placing the back button at the top left of the page in the English interface and its top-right in the Arabic interface.
3. Describe and label the page elements, one of the most important points that must be taken into account during the design, as the absence of a label for a button or an option makes the use of the blind impossible.
4. Reduce notifications and link them with a sound indicating their appearance if necessary; because the blind cannot know if a notification is appearing on the screen.
5. Use the correct terms for the buttons; for example, use the word “close” instead of “OK” to refer to the need to close a specific notification before returning to navigate through the page. This mistake was noticed on Mrsool select location page.
6. Do not use layering, the appearance of parts of another page or an option as a layer above the main page is a bad and distracting factor for the blind, as the VoiceOver reads the contents for the two layers at the same time.
7. Do not use drop-down lists; as the blind user could not guess the type of a list and often clicks on it to be read by VoiceOver, which leads to its closure if it was a drop-down list.
8. Translate all buttons and page contents of the User Interface (UI), the lack of translation of the screen content may hinder the blind from accessing many options and features.
9. Separate the options that should allow the blind to perform operations on them, such as deleting and modifying a specific product in the cart. As product quantity reduction in Hungerstation application could not be tested in this research since the VoiceOver was reading all products together and does not separate them, and also the use of hand gestures to indicate and select only one product to delete did not work either, which indicates both a major accessibility and usability problems.
10. Do not separate a field from its description, such as the writing field, as the blind person is going to press on the description of the field repeatedly, thinking that the keyboard will appear like what happened while testing Mrsool application.
11. Add a search field to the map page is also necessary if the blind person wants to make a request for a location other than his current location.
12. Make sure the application update does not change the usability of the application; it was noticed when doing the experiments in this research that some of the old updates had higher usability level than the new ones.

References

[1] M. Burton et al., “The Lancet Global Health Commission on Global Eye Health: vision beyond 2020,” The Lancet Global Health, vol. 9, no. 4, pp. e489–e551, Apr. 2021, doi: 10.1016/S2214-109X(20)30488-5.

[2] “APD – هيئة رعاية الأشخاص ذوي الإعاقة.” https://apd.gov.sa (accessed July 1, 2022).

[3] “ISO 9241-11:2018(en), Ergonomics of human-system interaction — Part 11: Usability: Definitions and concepts.” https://www.iso.org/obp/ui/#iso:std:iso:9241:-11:ed-2:v1:en (accessed July 1, 2022).

[4] Sunardi, G. Desak, and Gintoro, “List of Most Usability Evaluation in Mobile Application: A Systematic Literature Review,” in 2020 International Conference on Information Management and Technology (ICIMTech), Bandung, Indonesia, Aug. 2020, pp. 283–287. doi: 10.1109/ICIMTech50083.2020.9211160.