The Big Picture: September 2024
Thoughts on NotebookLM, Meta's Orion AR, Incoming AI Regulation, and Big Tech's AI Education Ambitions
🚨 New format alert! (Don't worry, the weekly roundups aren't going anywhere). Each month, we’ll be cutting through the noise to bring you the signal around the intersection of AI and education.
So without further ado, here’s what caught my eye in September 2024:
Has Google created one of the most viral AI education apps?
Modality matters (and our collective chatbot fatigue…)
Building in an era of information excess
AR + AI: a new computing paradigm
A global AI regulatory divide?
Big Tech doubles down on AI Education
Or listen to a podcast version of the piece (thanks to NotebookLM!):
📚 NotebookLM: From Google Experiment to AI Education Sensation
What is NotebookLM? At its core, NotebookLM is designed to ingest and process various content types (documents, PDFs, and audio files) and generate grounded responses with citations. Its standout feature, "Deep Dives," creates an AI-generated conversation between two AI hosts discussing input material, offering a novel way to passively consume information. It's clear why it delivers immediate value to students, researchers, and anyone involved in knowledge work or information-heavy tasks. More great deep dives on the NotebookLM product here, here, and here.
NotebookLM can handle up to 1,500 pages at once, and leverages techniques like Retrieval-Augmented Generation (RAG), Text-to-Speech (TTS), and sophisticated prompt engineering. Google is now working on adding more customization options (e.g. changing the length, format, voices, and languages).
Initially marketed as a study and research tool, it’s taken a life of its own - users are “podcastifying” their code repos, summarizing Reddit threads, making study guides, and generating unexpected conversations (like this meta conversation of NotebookLM AI voices realizing they’re AI themselves).
Although the underlying tech behind NotebookLM is not entirely groundbreaking, its seamless UX/UI has propelled it to viral status as a consumer product. In fact, its drawing praise from leading AI researchers on X and students on TikTok alike.
Observations:
Has Google created one of the most viral AI education apps?: Initially developed as 'Tailwind' in Google Labs, the project was rebranded and launched as NotebookLM. It’s another AI win for incumbents like Google, who can capitalize on their vast consumer distribution network. Their key advantage lies in offering a unified workspace where users can seamlessly interact with uploaded content—eliminating the need to juggle multiple study apps or AI chatbots, each with fragmented experiences, inconsistent performance, and poor integration. This is a classic case of platform lock-in: Google can leverage its massive ecosystem to entrench users - why use separate apps when NotebookLM integrates directly with the platforms people already rely on? The more users rely on Google for not just search, but as their workspace for AI-powered knowledge management, the stickier the platform becomes. This lock-in potential underscores the value of owning the platform, not just the product.
Modality matters (and our collective chatbot fatigue…): Along with OpenAI’s roll out of Advanced Voice Mode, it’s clear AI is having its “audio moment”. While early AI tools have focused on chat interfaces, the turn-taking nature of chat isn't always the most enjoyable or intuitive form of communication, especially for younger students or those unfamiliar with AI, who may struggle to formulate specific questions. Audio interfaces and features like NotebookLM's Deep Dives help overcome the "blank canvas problem" by providing a more passive and accessible way to engage with content.
NotebookLM’s audio also employs "disfluency injection" - transforming a sterile AI-generated script by incorporating natural banter and pauses. According to Google Labs Editorial Director Steven Johnson, this seemingly small detail is crucial — people don't want to listen to robotic-sounding conversations. By adding an extra layer of human-like pauses, filler words, and speech patterns, the dialogue feels more realistic and engaging.
An era of information excess: In the age of AI, we've shifted from a problem of content scarcity to one of content overload. The challenge now lies in efficiently navigating the sheer volume of content and quickly accessing relevant knowledge. This emphasizes the importance of reducing "time to value" – how quickly users can extract meaningful insights from large volumes of content. For product builders, how can we create tools that not only generate or process information but also help users make sense of it more effectively?
That's what I think is ultimately so compelling about the 2-person podcast format as a UIUX exploration. It lifts two major "barriers to enjoyment" of LLMs. 1. Chat is hard. You don't know what to say or ask. In the 2-person podcast format, the question asking is also delegated to an AI so you get a lot more chill experience instead of being a synchronous constraint in the generating process. 2. Reading is hard and it's much easier to just lean back and listen. - Andrej Karpathy
👓 XR + AI: dual pillars of the next computing paradigm
At Meta Connect, AI and XR took center stage. Alongside updates on their AI models, the event showcased two significant XR developments: an updated Quest VR headset and a prototype of full holographic AR glasses called 'Orion'.
Orion represents a leap forward in AR technology. It integrates mini built-in projectors into the glasses' temples to create a seamless heads-up display. The device incorporates advanced interaction features including voice AI, hand and eye tracking, and a groundbreaking wrist-based neural interface.
These technologies converge to enable effortless control of AR environments using natural commands — voice, eye movements, and hand gestures — for a highly intuitive experience. Use cases include real-time translation, scanning QR codes, and even AI assistance to help you remember where you parked. More on the device here.
While impressive, Orion is still in its prototype phase and several years from being consumer-ready. With current production costs hovering around $10,000, along with a limited two-hour battery life, it faces significant hurdles before hitting the mainstream.
Though still early, it's clear Meta is betting on AR glasses as the next major AI-powered form factor:
Smart glasses are going to become the next major computing platform. They will gradually replace phones by 2030, much like mobile devices surpassed computers without fully replacing them - Mark Zuckerberg
Amazon, Google, and Snap are all either officially working on similar technologies (or rumored to be doing so). As AR evolves into the dominant platform for human-computer interaction, AI will play a critical role in making these interfaces more intuitive and powerful.
Observations:
AR + AI: a new computing paradigm: Next generation AR devices will leverage AI for real-time environmental understanding and personalized content delivery, enabling more intelligent and context-appropriate interactions. These devices will combine visual, auditory, and potentially other sensory inputs to provide more comprehensive and useful assistance. For example, Orion might understand that you’re at dinner with family, and decide not to notify you with a work-related message. Future AR devices will likely feature more sophisticated, context-aware AI assistants capable of ongoing conversations and complex tasks.
Neural interfaces: Meta is also exploring neural interfaces that allow users to control the glasses with thoughts and gestures. However, these brain-computer interfaces require surgical procedures and are therefore unlikely to reach consumer markets any time soon. Meta did announce that the company is developing a new brain-computer interface for the next version.
Spatial intelligence: AI is learning to see the world the way we do. Notably, Fei-Fei Li, often referred to as the “godmother of AI,” recently emerged from stealth with $230M in funding to launch World Labs. The company is developing AI systems that push beyond the constraints of 2D models, enabling machines to understand and interact with the 3D physical world. They are pioneering “large world models” (LWMs) that mimic human-like spacial intelligence. What new streams of spatial data from AR glasses could fuel this next generation of AI models?
Timing: Tech companies have been pushing and investing in smart eyewear for years. In 2013, Google introduced the now-infamous Google Glass. Although ahead of its time, by 2015, Google had shelved its vision of Glass as a mainstream consumer product. It was a case of the right idea but the wrong timing. Today’s AR glasses represent a more measured approach, recognizing that consumers may be more inclined to adopt augmented reality in the form of familiar, everyday accessories (like sunglasses that seamlessly record video) rather than fully immersing themselves in VR headsets. Are AR glasses finally ready to become a mainstream success?
⚖️ AI Regulation is Coming
California's recent AI safety bill, SB 1047 sought to establish a framework for accountability in AI development, reflecting a growing public demand for regulation. Polls showed that 73% of voters believed AI companies should be held responsible for harms caused by their technologies. The bill was approved by California's Legislature in August. However, the bill faced strong resistance from tech leaders who argued that such regulations could stifle innovation and slow the development of beneficial AI applications.
SB 1047 proposed making tech companies legally liable for damages caused by AI models and mandated the implementation of a “kill switch” to shut down AI systems in cases of misuse or rogue behavior.
With just one day left to decide, Governor Newsom ultimately vetoed SB 1047. He cited multiple factors in this decision: the burden the bill would have placed on AI companies, CA’s lead in the space, and a critique that the bill may be too broad (only penalizing larger models, while smaller models in high-risk contexts wouldn't be covered).
SB 1047 does not take into account whether an AI system is deployed in high-risk environments, involves critical decision-making or the use of sensitive data … Instead, the bill applies stringent standards to even the most basic functions — so long as a large system deploys it. - Gavin Newsom
Despite the veto, Newsom did sign 18 other AI regulations into law, forming one of the most comprehensive legislative packages to date aimed at curbing the misuse of generative AI.
Observations:
An emerging global AI regulatory divide: The EU AI Act establishes a comprehensive regulatory framework that categorizes AI systems by risk level, with strict regulations on high-risk applications like healthcare and law enforcement. In contrast, California's AI legislation takes a lighter approach, focusing on transparency, such as requiring generative AI developers to disclose dataset information, without imposing stringent safety protocols. We may be witnessing a global divide in AI regulation, with some countries aligning with the EU’s comprehensive risk-based legal framework, which includes market surveillance and enforcement mechanisms, while others opt for a more decentralized approach like the US, relying on self-regulatory initiatives and a patchwork of fragmented rules.
So what was signed and how do they impact AI Education?:
AB 2876 mandates California’s State Board of Education to integrate AI literacy into math, science, and history curricula, covering AI basics, limitations, and ethical considerations. SB 1288 requires CA superintendents to form working groups on AI’s role in public education.
AB 2013, effective in 2026, will require AI providers to disclose key details about their datasets, including the sources, how the data is used, the number of data points, the inclusion of copyrighted or licensed data, and the time period of collection.
AB 1008 extends California’s existing privacy laws to generative AI systems, ensuring that personal data exposed by AI models, such as names or biometric information, is protected under current privacy regulations.
SB 942 requires systems to include a disclosure in their content metadata indicating that the content was AI-generated (eg. all images created by OpenAI’s Dall-E will require a watermark in their metadata saying that they’re AI generated).
🚀 Big Tech’s AI Education Ambitions
Education is emerging as a critical priority for leading tech companies. In September alone, several major players have made significant moves to expand their reach in the education and workforce learning space. It seems Big Tech is doubling down on importance of education in shaping the future of AI!
OpenAI appointed Leah Belsky as its first general manager of education to lead ChatGPT integration into K-12, higer education, and continuing education settings. They're also partnering with universities (Oxford, ASU, Columbia, and more) to discuss AI's role in teaching and research.
Perplexity recently launched a Campus Strategist Program, engaging US university students to drive AI growth and awareness on campuses. Participants manage marketing budgets and lead initiatives, shaping AI knowledge sharing in higher education.
NVIDIA partnered with California's community colleges and state agencies to bring AI education to underserved communities. This initiative aims to democratize AI through curriculum integration, workforce development, and faculty training.
Salesforce pledged an additional $23 million toward AI education, totaling $150 million over 12 years. Their funding focuses on preparing students for an AI-driven future through various programs and nonprofit support.
Google CEO Sundar Pichai announced a $120 million fund for global AI education. The initiative will support curriculum development, teacher training, and inclusive AI learning experiences worldwide.
OMG NotebookLM! Love this switcheroo!