Experiments by Google Greater Access through Creatability

Launched in 2009, Google has brought together coders to experiment with the company’s marquee web browser Chrome, its operating system Android, and with Artificial Intelligence (AI), Web Virtual Reality, Augmented Reality (AR) and other emerging technologies.

The small projects that coders from around the world have developed have yielded new and innovative ways in which Google can be a platform for people to play, create and communicate with the outer world. The program, named “Experiments With Google,” recently launched a stream dedicated to exploring how creative tools can be made more accessible to people with disabilities.

“Creatability,” is a set of experiments made conducted by creators with the participation of allies in the digital accessibility community, including people with disabilities themselves. They explore how creative tools that enable drawing, music, and more, can be made more accessible using web and AI technology.

The results of these experiments are shared online alongside open-source code and tutorials to empower others to make their own projects. Here are a few of the projects that have emerged from the Creatability platform.

Keyboard

This is a virtual musical keyboard someone can play with the movement of their face, body, mouse, or through a keyboard. Movements are tracked and mapped using a webcam. Users can also customize the scale and number of notes on the screen. For more advanced musicians, they can also control other instruments using MIDI technology. Like all other projects on the platform, this project was developed in coordination with The Viscardi Center, a network of nonprofit organizations that provides a lifespan of services for children and adults with disabilities with an emphasis on digital accessibility. Tools such as this can be liberating, even life changing, for musicians with physical disabilities, or for anyone that wants to enjoy the brilliance of music despite their disability.

Sound canvas

Here, coders created a simple drawing tool that works through both sight and sound. A person can control a virtual pen or brush to draw with their mouse or keyboard. Alternatively, a webcam can be used to track a point on their body, like a nose. This is an ideal way to draw for people that are unable to use their hands to control traditional input methods. To support artists with visual disabilities, the whole canvas space can be turned into sound. For example, a rising line makes a rising sound. A line drawn from left to right will be heard moving from your left to right ear.

Word Synth

In this experiment, users are afforded a fun way to play with speech and music. People can combine speech and music by typing some words, then setting them to a melody. The music can change through the mouse or keyboard, and users can also explore different voices, scales, and more. The technology was built with Google Cloud Text-to-Speech API, which enables developers to synthesize natural-sounding speech. This is a great tool for people with learning disabilities who want to leverage an interactive tool to improve their literacy skills.

As can be seen, all the experiments made on the Google Creatability platform explore a diverse set of inputs, from mouse, keyboard, body, wrist, nose, or voice. They allow people with disabilities to make music by moving their face, or draw using sight or sound, and to experience music visually, amongst other ways to be more expressive.

Google has also worked hard to make sure that the experiments work with many screen readers across different platforms, with a commitment to improve compatibility. They are also actively inviting blind users to submit feedback based on their experiences with the tools created.

Whereas many of the tools developed as part of this initiative are still in their embryonic stage, and still require a lot of work to prepare them for widespread implementation, the spirit of collaboration and inclusion of people with disabilities is setting the standards for similar projects.

There is no good design without user testing, and to avoid the need to retroactively make changes, a process that can prove to be very expensive, people with disabilities need to be consulted at the design and development stage. The result will be an online tool, piece of hardware or physical space that will be user centered and inclusive.

To learn more about Google Creatability, visit https://experiments.withgoogle.com/ collection/creatability