A Guide to Apple Intelligence
AI for the rest of us.
While other big tech giants have largely put their AI cards on the table, Apple has notably stayed on the sidelines. Just last week, the WSJ published an article title “How Apple Fell Behind in the AI Arms Race”. However, many have observed that this was a “very Apple take on AI”: approach new technology with intention, develop a unique vision, conduct intensive testing and iteration to achieve a polished final product. Earlier this week, Apple finally dropped their big AI update. Since then, Apple’s stock ($AAPL) reached a new all-time high, reclaiming its position as the world’s most valuable company.
Whether you were excited or underwhelmed by WWDC, several interesting trends hint at where AI might be headed.
The Big Picture
Siri 🤝 ChatGPT = Apple intelligence
Math Notes
AI Writing Tools
Other
Apple & OpenAI
Privacy
The Big Picture: What Does This All Point To?
AI is a feature, not a product: these demos underscore AI’s power as an integrated, holistic experience rather than a standalone app or chatbot. A lot of the features (summarization, image generation, etc) aren’t new. Apple’s pitch is less about innovation than it is convenience - about what AI can achieve when it operates directly on your device and routes data around the operating system for you.
Apple’s core advantage is what it already knows about you: Apple is grounded in your personal information and context. It can retrieve data from your email, your message, your contacts, and countless other apps. It can reference content on your screen. Goodbye copy-and-pasting between apps! This is a much more seamless experience that further locks users in the Apple ecosystem.
ChatGPT wants to be your personal assistant, but can’t unless it’s integrated into the flow of your day-to-day. Microsoft CoPilot has similar aspirations, but currently can only plug into the MSFT apps used at work - nothing from your personal life.
LLMs commoditized: Apple’s partnership with OpenAI is not exclusive - as we know, they very rarely rely on third parties for critical product components. They've hinted at future support for other model providers, possibly including Gemini. By not developing its own LLM from scratch, Apple observed several rounds of GPT capability advancements and saved $$$ and resources on model training. Their current focus seems to be on small on-device models and ultimately acting as an AI aggregator.
What does this mean for the app economy?: When Google released AI Overviews (AI-generated replies vs. “10 blue links”), many feared it would be the “end of Google search as we know it”. This shift raised concerns around reduced web traffic to human published content and creators. Gartner predicts traffic to the web from search engines will fall 25% by 2026. “Google will now do the Googling for you, and everyone who benefited from humans doing the Googling will very soon need to come up with a Plan B,” says Casey Newton. Will we see a similar dynamic in the app economy?
Bad news for AI gadgets: A new wave of AI consumer hardware companies have quickly flooded the startup world - Rabbit R1, Humane AI Pin, Meta smart glasses. Everyone has been trying to figure out what an “AI device” might look like. However, many features promised in their visions are now integrated into the iPhone, which already boasts powerful processors and connectivity. Why carry two devices when your iPhone can handle it all?
What did they announce?
A lot of things - from Genmoji to customizable home screens to scheduled texts, Apple is essentially integrating AI into all of its core apps and features. I won’t be covering them all, but TechCrunch has a comprehensive overview here. Some of the key releases:
Siri 🤝 ChatGPT = Apple intelligence
“This is AI for the rest of us, personal intelligence you can rely on at work, at home, and everywhere in between.” - Craig Federighi, SVP of Software Engineering
Apple Intelligence is Apple’s catchall name for its AI integrations across the Apple ecosystem, marking a “new era for Siri”. An even more capable, integrated, and personal Siri.
Siri’s scope has dramatically expanded from engaging in a robotic conversations to actually being able to take actions and coordinate between different apps on your behalf - agentic capabilities! It can now:
Retrieve data and act across apps
Understand your personal context
Communicate multi-modally: through voice and text, and on-screen awareness
Examples:
“How long will it take to get to my daughter’s play” - Apple Intelligence can understand who your daughter is, identify the play details from her email a few days ago, the time and location of your current meeting, and the predicted traffic between your house and the theater.
“Show me all photos of my mom and my dog at the beach”
Math Notes
The iPad finally gets a Calculator app! There’s now a third calculator option (in addition to the Basic and Scientific layout) called Math Notes:
Handwrite mathematical expressions and see the answer upon writing an “=” sign
Adjust equations in real-time and watch the results change dynamically in front of your eyes
Tackle more complex math and physics problems - change variables (e.g., speed or angle, as demonstrated) or ask Math Notes to generate a graph.
AI Writing Tools
Proof read your text
Rewrite different versions until tone and wording are just right (Friendly, Professional, or Concise)
Reformat into a Summary, Key Points, Table or List
Summarize an entire lecture or get a short version of a long group thread
Writing Tools will be available anywhere you write on your iPhone, iPad or Mac - including in third-party apps (eg. Slack, Instagram, Twitter, LinkedIn)
Other
iPhone Mirroring: your iPhone display can appear in a window on your mac, allowing you to control your iPhone using your trackpad and keyboard. Helpful if you want to quickly check notifications or tasks while your phone is across the room (or check on your child’s device from your computer).
Hide an App: you can now hide apps - their icons and names will not be visible on the Home Screen. They will only be visible in Settings and in an obscured folder in your App Library. Face ID will be required to reveal or use them. Parents of younger kids with iPhones, beware!
Genmoji: create your own emoji. Such a fun feature but poses risks for misuse. When Meta introduced AI-generated 'stickers', users created inappropriate images, such as heavily armed Mario characters. We can expect similar attempts to push features like Genmoji to misbehave.
Apple & OpenAI
Apple is not paying OpenAI any cash as part of their partnership. “Apple believes pushing OpenAI’s brand and technology to hundreds of millions of its devices is of equal or greater value than monetary payments," says Mark Gurman. Apple’s control over its product ecosystem provides a massive distribution channel - in 2022, Google reportedly paid Apple $20B to become the default search engine in the Safari web browser.
This partnership is non-exclusive, and Apple is already in talks to provide other chatbot options. In the future, Apple plans to establish revenue-sharing agreements with various AI partners and has hinted at allowing users to choose their preferred models.
Privacy
There are widespread (and extremely justified) concerns about sharing personal data with LLMs, and Apple is trying to reinforce its reputation as a privacy-conscious tech company.
The AI will keep what it learns about you from your apps strictly between you and your device. Apple will not have privileged access to user data, and its cloud will not retain user data. If a user without an OpenAI account queries ChatGPT, their IP address will be masked.
For most questions, Siri uses an AI that runs on-device, ensuring that your questions never leave your phone or tap into the cloud. Apple aims to have as much generation done on-device as possible.
For complex queries/workloads that require more processing power, Siri use a more advanced AI model that runs on Apple’s servers. They call this approach Private Cloud Compute. If Siri does send your question to Apple’s servers, this data is anonymized.
If a question is best suited for ChatGPT, Siri will ask if you’d like to use ChatGPT.
Independent researchers will be able to inspect the infrastructure code to verify these claims and identify potential flaws. Marques Brownlee has a great recap on Apple Intelligence and Privacy (14:48) and The Verge has a helpful technical breakdown of Private Cloud Compute.
When can I use this?
Apple Intelligence and its generative AI features will come out with iOS 18 later this year (expected to be released around September 2024). It will require an iPhone 15 Pro or any iPad/Mac with M1 Chips.
The new capabilities are opt-in, so Apple won’t force users to adopt them if they choose not to.
As with any big tech release, the real test will come later this year when we see Apple Intelligence in action. Although Apple has been fashionably late to the generative AI party, we should not underestimate their potential.
Apple is uniquely positioned to dominate on-device AI, potentially building the most intuitive agent controllers on the market. By leveraging the vast amount of personalized data on their devices, Apple aims to make its AI-powered ecosystem infinitely more valuable. “AI for the rest of us” is coming.