Manton.org: Last Week

I know social networks aren’t Apple’s thing, but I still think it would be cool for them to have a platform for sharing personal spacial videos (and photos) for visionOS shot on iPhones. Would help the content problem.

Math Notes is super impressive. No mention of AI or machine learning but it’s gotta be all over this.

iPhone mirroring seems… really strange. Can’t wait to try it, though.

So far so good with Apple Intelligence. I have a lot of questions but the framing seems right, with the private cloud compute and working with app data. LLMs will make Siri so much better at interpreting your questions.

Apple’s little custom icon to indicate what’s an AI summary is nice. I’ve been using a robot icon for summaries in Micro.blog, but maybe Apple’s icon will catch on.

Vaguely keeping up with the Platform State of the Union in the hotel lobby. Thinking I’ll update to macOS Sequoia later, but not on this wi-fi.

Lots of details in this Apple blog post about the new AI models. Looks like the on-device models are similar to other small-ish LLMs, and the server models are roughly comparable to GPT-3.5. GPT-4o is still bigger, which is why the OpenAI partnership fits. I think Apple’s strategy here makes sense.

Random shot of San Jose light rail train, interrupted by a bus.

Video 2024-06-10

Nice live blog at The Verge of the interview session with iJustine. Nilay Patel comments:

I think Apple really wants to seem transparent and open about their AI plans, especially around privacy, and having Federighi and Giannandrea this much on the record in basically every publication is a good way to do it.

This text without any vending machines is like a cruel mirage in what feels like the middle of the desert of southern California this afternoon, outside Boron.

2024-06-11

Yesterday was a blur. I was planning to camp at Lake Havasu but it's just too hot, so I continued driving to Flagstaff. Really long day. Caught up on nearly all the WWDC-related podcasts plus progress on audiobooks.

A couple days after the WWDC keynote, starting to see a couple cracks in Apple's AI strategy. I'll blog more later. But I think the architecture for small on-device models and larger models in the cloud that are extremely locked down is very smart. This is flexible and can scale, if users accept it.

Freight trains in the middle of nowhere, Arizona.

Finished reading: Sunbringer by Hannah Kaner. This was excellent. Loved the way it revealed more of who the characters are and built up toward the end, setting up what I assume will be a third and final book. 📚

Wind turbines as the sun sets in New Mexico.

In 1979 my uncle wrote a book about solar power — or more specifically, although I haven't read it, about how the fossil fuel industry had too much influence on the future. 45 years later, solar panels are now cheap and widely deployed. Nearly free energy, falling like rain. It can change so much.

WWDC initial takeaway

One thing I didn't appreciate before this year's WWDC is how limited Apple's on-device models could be. Apple is going for easy wins and generally not biting off more than they can chew. Summarizing or rewriting text is something LLMs are great at, with almost no risk for getting derailed with hallucinations. So it shouldn't have been surprising that Apple is doing so much themselves with their own models, and punting to ChatGPT for what Craig Federighi called "broad world knowledge" that is beyond Apple's own models.

The only thing that struck me as strange in the WWDC keynote was image generation. I didn't expect Apple to do that and I still don't see why they needed to. It opens up a can of worms, something that was discussed well on this week's episode of Upgrade. See the chapter on "AI feelings".

The rest of the strategy is really good, though. The on-device models are small, but they can be supplemented with cloud models for more advanced tasks. And because it will be transparent to the user whether a local or cloud model is used, Apple can add bigger models to newer iPhones as RAM increases, for example, and the user won't know the difference. Tasks will just become faster and more sophisticated.

This does require the user's buy-in on Apple's premise: that "private cloud compute" is just as secure and private as on-device data. On first glance this doesn't seem technically true. As soon as the data leaves the device, you're in a different world for things to go wrong. But Apple has built up a lot of trust. If user's accept the private cloud — and, importantly, if users even realize that Apple's cloud is completely different than OpenAI's cloud — it gives Apple a new strength that others don't have, even if that strength is propped up mostly on goodwill.

Personally I have no concern with the cloud approach for my own personal data. I expect Apple's solution to be robust, likely bordering on over-engineered for what it actually needs to do, but that builds confidence.

Ben Thompson is optimistic about Apple's AI strategy too. From a daily update on whether other companies could displace iOS:

I’m unconvinced that large language models are going to be sufficiently powerful enough to displace iOS, and that Apple’s approach to productize LLM capability around data that only they have access to, while Aggregating more powerful models, is going to be successful, but time will tell. Relatedly, the speed with which a wide array of model builders delivered useful models both gives me confidence that Apple’s models will be good enough, and that there isn’t some sort of special sauce that will lead to one model breaking away from the pack.

I'm not sure. There is no telling whether there will be another GPT-level advance in a couple years. Already OpenAI has some technologies like the voice matching that are so powerful that OpenAI almost seems scared to even release them. If there is a breakthrough, it may be difficult for other companies to replicate it right away, giving a single player a years-long advantage.

At the same time, there is just enough friction in Apple Intelligence that even with the improvements to Siri, it may feel slightly crippled compared to a hypothetical new voice assistant. As I wrote in a blog post before WWDC:

While it’s true that the iPhone will continue to dominate any potential non-phone competition, I think there is a narrow window where a truly new device could be disruptive to the smartphone if Apple doesn’t make Siri more universal and seamless across devices. This universality might sound subtle but I think it’s key.

It's unlikely for Apple to be displaced. People love their phones. I think there is still an opening for something new — a universal assistant that works everywhere, can do nearly everything, and is a joy to use. But we may never get there, or "good enough" may be fine, in which case Apple is really well-positioned.

Stopped overnight in Roswell, New Mexico to visit with the aliens. Queueing up the audiobook The Terraformers for the final leg back to Austin. I had started with the e-book a couple weeks ago. 🛸

Stellar Coffee in Roswell. ☕️

Gandy’s Creamery in Brady, Texas. No trespassing.

Good interview from Kara Swisher with Mira Murati, CTO of OpenAI. She led the GPT-4o announcement and in an alternate timeline was CEO for more than a few days. They talk data deals, the voice controversy, disinformation, and the promise of AI in education.

Now that I’m back home from WWDC, my excitement about installing the betas has evaporated. Will probably wait until Apple Intelligence is enabled. Doesn’t seem much for new APIs I can use it.

What if personal domain name registrations could essentially renew on auto-pilot, regardless of changing or expired payment, for decades? It has always bothered me that blogs are like self-published books that self-destruct when expired. After lots of waffling, I think I’m ready to tackle this.

Jason Snell paints a picture of Apple as sort of irritatedly getting on board the AI hype train. On the OpenAI deal, I don't think either side has clearly won. It brings to mind Draft Day: "This is a good deal... This is a good deal for both of us."

Last month I got sucked into an auction for Disney artwork and came away with a sequence of drawings of Scrooge in Mickey’s Christmas Carol, from 1983. They arrived! Need to figure out how to frame them.