At the Augmented World Expo, Snap showcased a preview of its real-time, on-device image diffusion model, which can generate rich AR experiences. The company also introduced generative AI tools for AR creators. Snap’s co-founder and CTO, Bobby Murphy, emphasized that the model is compact enough to run on a smartphone and fast enough to re-render frames in real-time, guided by a text prompt.
Murphy highlighted that while the emergence of generative AI image diffusion models is exciting, they need to be significantly faster for them to have a significant impact on augmented reality. To address this, Snap has been working to accelerate machine learning models. Snap plans to roll out this technology to Snapchat users in the coming months and make it available to creators by the end of the year.
The introduction of this generative model marks a new direction for augmented reality, allowing for more dynamic and immersive experiences. Murphy stated, “This and future real-time on-device generative ML models speak to an exciting new direction for augmented reality, and is giving us space to reconsider how we imagine rendering and creating AR experiences altogether.”
Snap has also launched Lens Studio 5.0, which provides developers with access to new generative AI tools. These tools will enable them to create AR effects faster and more efficiently, saving them weeks and months of development time. With these new tools, AR creators can generate selfie Lenses with highly realistic ML face effects, custom stylization effects, and 3D assets. They can also generate characters, face masks, textures, and materials using text or image prompts and Snap’s Face Mesh technology.
Lens Studio 5.0 also features an AI assistant that can answer creators’ questions and provide guidance on using the generative AI tools.
