Apple Intelligence, Apple’s new generative AI offering, isn’t limited to consumer use; developers can also harness the latest technology. During Apple’s keynote address at WWDC 2024 on Monday, the company revealed that developers could integrate Apple Intelligence-powered experiences into their own apps.
Apple’s SDKs (software development kits) have been upgraded with new APIs and frameworks, enabling app developers to easily integrate features like the Image Playground for generative AI image creation with just a few lines of code. Apple showcased how an app like Craft could enhance users’ documents visually by adding AI-generated images.
AI-driven writing tools will also be automatically available in any app utilizing the standard editable text view. Apple demonstrated this with an app like Bear Notes, showing that users could automatically rewrite, proofread, and summarize their notes.
Moreover, Apple is developing additional ways for developers to enable actions in their apps using Siri.
Developers who have already integrated SiriKit can benefit from many of Siri’s new capabilities instantly without extra effort, Apple stated. This includes areas such as Lists, Notes, Media, Messaging, Payments, Restaurant reservations, VoIP calling, and Workouts.
In its Developer keynote, Apple introduced two new Siri capabilities that developers can utilize without extra work. First, Siri can now invoke any item from an app’s menus. For example, users could say “show my presenter notes” when viewing a slide deck or use more conversational commands like “I need to see my speaker notes.”
Secondly, Siri can access any text displayed on the page using Apple’s standard text systems, allowing users to reference and act on on-screen text. For instance, if you had a note that says “wish grandpa a happy birthday,” you could simply say “FaceTime him” to take action on that note.
Additionally, Apple’s App Intents framework, which allows for lightweight app-like interactions without installing the app, will also incorporate Apple Intelligence.
Apple is defining new intents and making them available to developers across various categories, starting with Books, Browsers, Cameras, Document Readers, File Management, Journals, Mail, Photos, Presentations, Spreadsheets, Whiteboards, and Word Processors.
These intents are defined and tested to be easier for developers to adopt, according to Apple.
With these intents, a photo-editing app like Darkroom could utilize the Apply Filter intent, letting users say “Apply a cinematic preset to the photo I took of Ian yesterday” to automatically apply the filter. More domains will be included over time.
Initially, users will be able to employ a develop with the Shortcuts app, but eventually, Siri will gain the ability to call the app intents in supported domains.
Additionally, as shared during the keynote, apps fitting an existing SiriKit domain will benefit from Siri’s enhanced conversational capabilities. Siri can respond accurately even if users stumble over their words or reference an earlier part of the conversation.
Siri will also be able to search app data using a new Spotlight API, allowing app entities to be included in its index. These entities are part of Apple Intelligence’s semantic index, which includes photos, messages, files, calendar events, and more.
Also, on Monday, Apple announced its own password manager app, AI-generated Bitmoji, and Calculator for the iPad.
