Nearly ten years ago, Google introduced a feature called Now on Tap in Android Marshmallow. This feature allowed users to tap and hold the home button to access helpful contextual information related to the content on their screens. For example, if you were discussing a movie with a friend over text, Now on Tap could provide details about the film without leaving the messaging app. Similarly, when looking at a restaurant on Yelp, the phone could surface OpenTable recommendations with just a tap.
At the time, these advancements felt thrilling and futuristic, with the ability to understand on-screen content and predict user actions. While Now on Tap eventually evolved into Google Assistant, it wasn’t quite the same. Fast forward to today, at Google’s I/O developer conference in Mountain View, California, and the latest features in the Android operating system feel reminiscent of the original Now on Tap, using advancements in large language models to provide contextual information and make phone usage easier.
According to Dave Burke, vice president of engineering on Android, the technology now exists to create powerful assistants that can understand and predict user needs based on screen content. The new features introduced at the conference leverage years of development in this area.
The updated Circle to Search feature, reminiscent of Now on Tap, offers a new way to interact with Google Search on mobile devices. Users can circle items on their screens to initiate searches, offering a more interactive and engaging experience. This feature has been well received by consumers, with a specific enhancement based on student feedback. Circle to Search now provides step-by-step instructions for solving physics and math problems without leaving the app.
Gemini, Google’s AI assistant, is gaining traction and is seen as surpassing Google Assistant. Users now have the option to replace Google Assistant with Gemini on many Android phones. While some may wonder about the future of Assistant, Sameer Samat, president of the Android ecosystem at Google, emphasizes that Gemini is an opt-in experience for users. The assistant continues to evolve and improve, with no concrete plans to phase out Assistant at this time. The focus is on providing consumers with a choice and receiving valuable feedback on the new AI-powered assistant.
