The pace of AI progress is often overwhelming to non-technical people, but it seems that AI scientists themselves are surprised too.
Amazon CEO Andy Jassy has highlighted the astonishing speed of generative AI development and the impact it’s having even on those closest to the technology. He emphasized the iterative nature of generative AI, contrasting it with traditional software development.
“Because the reality is, even more so than software development, generative AI is very iterative,” Jassy said in an interview with CNBC. “It’s not like software development where you can get on a whiteboard with a team of architects and design something and, maybe it doesn’t work exactly as you designed it, but largely you know,” he said.
“Whereas in generative AI,” Jassy continued, “the models, they get better at kind of disproportionate rates sometimes. I think that a lot of times when you’re building models, and the model gets so much better, and you talk to the scientists and the team, they just can’t believe how much better it got because the model is learning itself,” he added.
This sentiment echoes recent discussions in the AI community about emergent properties of large language models – capabilities that arise unexpectedly as the models grow in size and complexity. These emergent properties are a testament to the unpredictable nature of generative AI development, making it a field rife with both exciting possibilities and significant unknowns.
This rapid pace of improvement, Jassy warned, creates significant pressure for companies to invest in generative AI now. Falling behind, even for a short period, can result in a substantial disadvantage. “So I think that if you actually aren’t investing in generative AI, you are going to be behind by even the amount of time that you waited,” he cautioned, “because there are so many lessons you get from iterating and building applications today.” This urgency reflects the broader trend of companies across various sectors scrambling to integrate generative AI into their products and services, driven by the fear of missing out on a transformative technology.