AI can already write text and code, and generate images and video, but it’s being put to even more unconventional uses.
Google has unveiled SignGemma, a model that can translate sign language. This upcoming model, part of the Gemma family, is designed to translate various sign languages into spoken language text, with an initial focus and robust testing on American Sign Language (ASL) and English. The announcement underscores a broader trend in AI: the remarkable evolution of technologies like the Transformer model, which, while originally conceived for language translation, now powers a diverse array of applications, from understanding animal communication to generating complex visual media.
“We’re thrilled to announce SignGemma, our most capable model for translating sign language into spoken text,” Google stated, highlighting its potential to open up “new possibilities for inclusive tech.” The company further described SignGemma as a “groundbreaking open model for sign language understanding,” emphasizing its design for multilingual capabilities despite its current proficiency primarily with ASL.
A crucial aspect of SignGemma’s development is Google’s commitment to collaboration. Recognizing the importance of lived experiences and specific needs, Google is actively seeking input from developers, researchers, and particularly the Deaf and Hard of Hearing communities worldwide. “As we prepare for launch and beyond, we’re eager to collaborate… to make SignGemma as useful and impactful as possible. Your unique experiences, insights, and needs are crucial,” the announcement appealed, inviting interested parties to share their thoughts with the SignGemma team.
The development of SignGemma is a testament to the incredible journey of the Transformer architecture, which was first introduced in a 2017 Google paper titled “Attention Is All You Need.” Initially, its primary application was machine translation, revolutionizing the field by allowing models to weigh the importance of different parts of the input data. However, the fundamental principles of the Transformer – its ability to process sequences and understand context through attention mechanisms – have proven to be far more versatile.
Today, Transformer models are the backbone of a vast spectrum of AI applications. They are not only adept at understanding and generating human language but have been successfully adapted for tasks that were once considered distinct domains. This includes generating photorealistic images from text prompts (as seen in models like Imagen and Stable Diffusion), creating video content, and even composing music. The architecture’s scalability and adaptability have made it a cornerstone of modern AI research and development.
Google’s own explorations into novel communication domains further illustrate this versatility. Before SignGemma, the company had also worked on projects like DolphinGemma, an initiative aimed at understanding the complex vocalizations of dolphins. While distinct in its application, DolphinGemma shares the underlying theme of using advanced AI to decode and interpret forms of communication previously opaque to machines.
The advent of SignGemma, therefore, is more than just a new translation tool. It represents a convergence of AI advancement, a commitment to open-source principles, and a drive towards greater inclusivity. By leveraging the power of mature architectures like the Transformer and fostering community collaboration, Google aims to break down communication barriers and create technology that is more accessible and beneficial for everyone. As AI continues to evolve, the ability of models like SignGemma to understand and interact with the diverse ways humans (and potentially other species) communicate will undoubtedly lead to even more profound innovations.