A Learn App: Mobile augmented reality vocabulary learning application

1. Introduction

In cooperation with its strategic partners, Mada Center works to identify the needs of persons with disabilities for assistive technology and digital access solutions in Arabic. The center has supported start-ups and entrepreneurs to localize technological devices and solutions through the Mada Innovation Program, with the aim of improving digital accessibility for PWDs at the local, regional and international levels. This year, Mada and HBKU worked together to develop a mobile app using augmented reality technology and for the first time three dimensional AAC Symbols for learning purpose. The app is named A-Learn developed by Dr. Kamran Khowaja, Dr. Dena Al-Thani, and Dr. Siti Salwah Salim from Hamad Bin Khalifa University.

2. Ideation

The idea of Al-Learn is to transform the learning of vocabulary through mobile augmented reality. The advantage of using mobile augmented reality is its ubiquity i.e., learning anything supported through an app, anytime, anywhere. The use of mobile augmented reality is expected to dominate the market by 2022 in comparison to virtual reality. A-Learn allows children to learn letters and vocabulary through augmented reality 3D objects.

3. Mada’s Support

The 3D models, influenced by Tawasol symbols (Tawasol Symbols, 2018), are specially designed for children with autism by a team of experts in special education, language and speech therapy, and human-computer interaction. To the best of our knowledge, the A-Learn would be the first of its kind for the research community as well as the children with ASD in Qatar, their caregivers, and teachers. This would allow them to become an independent individual and live a better life. The app would support English and Arabic languages.

Each child with ASD is different; it is possible that if one technology-based solution works for one child, it may not work for another child. Therefore, the researchers have started to use different technologies in the interventions for children with ASD to identify the best possible technologies that suites an individual with ASD. Furthermore, the buying cost of each technology may vary drastically; thus evidence-based research on the use of technology can be useful for the parents, caregivers, school, centre among others to decide based on their need and availability of the budget to buy the required quantity of the technology.

4. 3D AAC Pictograms from Tawasol Symbols

Mada provided the 3D Tawasol Symbols for the A-Learn application under the direct grant stream of the Mada Innovation Program (Al-Thani et al., 2019). Tawasol Symbols is a bilingual Arabic / English symbol dictionary of frequently used words in spoken and written communication (in Arabic) represented by images and pictograms . It is a freely available symbol. Symbols are verbal or visual representations of concepts and ideas.

Augmented and Alternative Communication (AAC) methods use visual symbols in the form of graphics such as a picture or object as a tool in A-Learn. When choosing a symbol system, the iconicity of the symbols needs to be considered. Iconicity is the amount that a visual symbol relates to its referent. In other words, it’s how much the symbol resembles the intended message in the A-learn application. Iteratively develop and evaluate a mix-reality online interactive learning platform.

5. A-Learn App

A-Learn is an educational augmented reality game. You can use the game to scan letter cards to show the letters in an augmented reality environment, then you can combine these letters to form words and display 3D representations of these words and interact with them.

Figure 1. QR Code to download the app A-Learn

Figure 2. Some UI From the App

Figure 3. AR Technology displaying a 3D Tawsol Symbol through the app A-Learn

References

1. Al-Thani, D., Al Tamimi, A., Othman, A., Habib, A., Lahiri, A., & Ahmed, S. (2019, December). Mada Innovation Program: A Go-to-Market ecosystem for Arabic Accessibility Solutions. In 2019 7th International conference on ICT & Accessibility (ICTA) (pp. 1-3). IEEE.
2. Elsheikh, A., & Zeinon, N. (2019, December). Mada Tawasol Symbols & Mobile App. In 2019 7th International conference on ICT & Accessibility (ICTA) (pp. 1-5). IEEE.
3. Khowaja, K., Banire, B., Al-Thani, D., Sqalli, M. T., Aqle, A., Shah, A., & Salim, S. S. (2020). Augmented reality for learning of children and adolescents with autism spectrum disorder (ASD): A systematic review. IEEE Access, 8, 78779-78807.
4. Mada Center. (2018, February 19). Tawasol Symbols. Tawasol Symbols | Arabic Symbol Dictionary. https://tawasolsymbols.madaportal.org/tawasol/en/home/
5. Othman, A., & Al-Sinani, A. (2021). Tawasol Symbols: Alternative Augmented Communication Pictograms to Support the Inclusion During Pandemics. In Radical Solutions for Education in a Crisis Context (pp. 225-239). Springer, Singapore.