Pulkit Agrawal, an assistant professor at MIT specializing in AI and robotics, commends the recent advancements in multimodal AI models by Google and OpenAI. OpenAI’s GPT-4V, capable of analyzing images, and Gemini, which can interpret live video in real time, have especially impressed Agrawal. He notes that the new versions of ChatGPT show similar potential.
Agrawal emphasizes that the interaction with these AI models in real-world scenarios could offer valuable training data for companies, but questions remain about their practical applications. Google’s Project Astra, set to launch through Gemini Live later this year, may pave the way for a revival of their previously unsuccessful Glass smart glasses.
Despite the impressive demos, the current capabilities of multimodal models have limitations in fully understanding the physical world. Brenden Lake, an associate professor at NYU, stresses the importance of creating AI models that can build a mental model of the physical world, like humans do naturally through interaction.
DeepMind’s work on game-playing AI programs and advancements in robotics are seen as key frontiers in advancing AI. Demis Hassabis, from Google DeepMind, believes that enhancing AI models with a deeper understanding of the physical world will be crucial for future progress.
While the development of a multimodal universal agent assistant shows promise towards artificial general intelligence, Hassabis acknowledges that it is only the beginning of a journey towards achieving capabilities comparable to human intelligence.
