The dream, which some call a nightmare, of the metaverse has a foundational restriction: hardware, specifically optics.

These optics are not a metaphor, e.g., the optics of whether data centers are good or bad for society. These optics must exist in a physical device worn on a person’s face at an affordable price. Many scoff that people will never wear augmented reality glasses, much less wear VR goggles. Those who scoff seem to forget that more than 70% of adults in the US wear sunglasses, and billions across the globe wear eyeglasses for corrective vision. I’ve worn glasses since the fifth grade. If eyewear is useful, you’ll wear it.

When the value and affordability reaches a mass market, we will be in a different era of computing. Zuckerberg is right that another platform will replace the smartphone. (Maybe it won’t be eyewear, but our devices will get smaller and more portable. Or maybe as neural implants. I’m not even ready for that last one.)

Unlike software products, the materials and engineering of these reality adjusting eyewear, whether glasses or VR goggles determine when the products come to market. The tech is maturing, but the price point is not yet there.

Meanwhile, as Zuckerberg was starting his move towards the metaverse, OpenAI had the audacity to pour massive amounts of money into a purpose that came to consumer’s as software via their existing web browsers. I know that Sam Altman claims that they didn’t have a product in mind when starting OpenAI. But he’s an entrepreneur. Surely, he knew where OpenAI was going.

Maybe Zuckerberg should have made the bet on AI instead of optical hardware. But, as we’re seeing, AI is becoming a commodity. And we’ve also seen that a premium headset finds an audience: Apple Vision Pro.

The audience for Apple Vision Pro is small. Many who have the Apple device report excellent experiences. Whereas, many who have the Quest have it piled in the back of a closet. The Quest was intended as a gateway to VR and never as the epitome of the technology.

A market report published on March 13, 2026 by treeview, a spatial computing company, estimates that more 475,000 units of the Apple Vision Pro has been sold.

My advice to any following the tech market always has been to follow the reports coming out of the research labs. Every day I get a feed from scholar-inbox with a dozen papers detailing advances in the ecosystem that will encompass our digital lives ahead of us. All the work in 3D and world models are directly applicable to the products that we will experience through smart glasses and headsets, even if these articles do not mention virtual reality.

A few examples from just today, April 3, 2026:

• Lifting Unlabeled Internet-level Data for 3D Scene Understanding (arXiv link)

• Reflection Generation for Composite Image Using Diffusion Model (arXiv link)

• Omni123: Exploring 3D Native Foundation Models with Limited 3D Data by Unifying Text to 2D and 3D Generation (arXiv link)

• LivingWorld: Interactive 4D World Generation with Environmental Dynamics (arXiv link)

• Anime-Ready: Controllable 3D Anime Character Generation with Body-Aligned Component-Wise Garment Modeling (arXiv link)

• ActionParty: Multi-Subject Action Binding in Generative Video Games (arXiv link)

• Director: Instance-aware Gaussian Splatting for Dynamic Scene Modeling and Understanding (arXiv link)

• Large-scale Codec Avatars: The Unreasonable Effectiveness of Large-scale Avatar Pretraining (arXiv link) Note: this one comes from the Code Avatar Lab at Meta.

• Generative World Renderer (arXiv link)

And that’s just from one day.

I don’t want to be misleading. Many of those papers were submitted to conferences and are only now being published on arXiv. But it’s still an astonishing amount of research being produced.

For those of you keeping count and concerned about the US vs China dominance in AI: six of those nine research papers were produced by Chinese institutions. The one by Meta was the only US representative in the lot. There also was one from Korea, another from a joint Canada/UK effort. One of the China-produced papers was in collaboration with a Japanese studio. This sample is not scientific, just a random set from one day. The whole US/China discussion is extensive and, I think, misleading.

What’s most difficult to track: when and which parts of research will come to market as products.

Let’s set aside the word metaverse, which came from a barely readable novel. With this newsletter, I admittedly hopped onto that train that went seemingly went nowhere. To extend that railroad analogy: that train carrying the metaverse relied on tracks not yet laid. It’s not the end of the line for extended reality, and that concept—extended reality—is a mood more encompassing than the metaverse.

I’ll be renaming this newsletter. It won’t be called extended reality is open. My next post will reveal the name. I’ll be using a term that has stuck with me now for nearly 25 years.

I will continue to write about what I’ve always written about: how we tell and experience stories, whether fictional or factual, in digital media. And that term digital media is so vague that I rarely use it anymore. (It has a late 1990s feel.) Digital media is software. But we only have experience software through our devices.

Software is the easier part, the software based on open standards and open source. The future is open for us to make an endless hybrid of experiences.

A prime problem with the metaverse is that the term was too often associated with blockchain, crypto, and grifters trying to cash in on emerging technology. Others were skeptical of Zuckerberg’s vision of the future and the rebranding of Facebook to Meta.

Here’s Matthew Ball’s extended definition of the metaverse (Ball, The Metaverse: And How It Will Revolutionize Everything, p29, 2022):

“A massively scaled and interoperable network of real-time rendered 3D virtual worlds that can be experienced synchronously and persistently by an effectively unlimited number of users with an individual sense of presence, and with continuity of data, such as identity, history, entitlements, objects, communications, and payments.”

Yeah, that’s a mouthful but it’s easier to say one word: metaverse. Yet, I do feel a certain ick when using that word. I’m wondering if I should rename this Substack.

My own working definition of the metaverse is a 3D world that allows you to step into the story.

That doesn’t capture all the uses wrapped into Ball’s lengthy definition. But I’m not focused on every aspect of real-time 3D.

My interests are in the experiences made possible through open standards of mixed and virtual reality. The emphasis on experiences is because I’m more fascinated by our imaginary lives than the enabling tools.

These tools are projecting images in front of our field of view. (Sound may also be a part of the experience.) We can reach out and touch the aspects of our view in ways not enabled through keyboard and mouse. What impact does this imaginary presence have on how we think?

Where are all the technical developments heading, where are they converging? My X feed is filled with news mostly about world models, Gaussian splats, AI films, and robotics. The future of our imagination is embedded in all that’s happening.

The challenge with any analysis is not explaining the technical developments but the unknowns in when a company can bring a technology-based product to market.

Tools only exist to create experiences that will, in some way, be monetized by providing value to people.

Tools are not uniquely tied to the concept of the metaverse. But many of the tools and approaches (like world models, AI video generation, procedural content generation, and scene graphs) are essential for stepping into a dynamic imaginary storyworld.

The 21st Century is becoming more chaotic than we imagined. What we do know is that changes in technology, driven by AI, are not stopping. Will the metaverse be a part of that? Advances in computer vision, world models, and optical engineering indicate that those fundamental technologies will continue to produce tools that to come to market as products and experiences. Maybe those experiences will be niche and not the next computing platform that replaces smartphones. But niche experiences have value and a market.

Unless you work within a sector of an industry, you might not think about segments of an industry. Let’s use the hospitality industry as an example. In the 21st Century, are motels (as distinct from hotels, mostly for their drive-up-to-your-room feature and lower costs) still relevant?

The Super 8 hotel in the Virginia town where I lived is being razed to make way for a Sheetz. (And, wow, convenience stores have changed in the last couple of decades.) The older Motel 6 down the road is likely not long for this world, either.

But hotels remain very relevant and used by millions of people every night. The metaverse may go the way of motels, but the enabling functionality will thrive in products used by millions.

I could just have ChatGPT or Claude spit out an essay but I wouldn’t learn anything that way. I spend hours every day with those tools but sometimes you need to slow down.

A slow-paced literature review allows for ideas to turn over and settle in your mind. These thoughts form mental relationships, revealing connections that might find expression as insights in an essay.

With world models in 2026, I’m interested in two questions:

• will world models replace game engines?

• how can world models simulate social dynamics?

An excellent starting point is an article updated in December 2025: Ding, Jingtao, et al. “Understanding world or predicting future? a comprehensive survey of world models.” ACM Computing Surveys 58.3 (2025): 1-38. That survey also has a GitHub repo: https://github.com/tsinghua-fib-lab/World-Model with graphics and pointers to other repos of code and publications.

Not all of the 300+ citations tracked in that survey are of interest to me. While a major use use of world models is in robotics, I’m deliberately excluding robotics and embodied AI from the scope of my interests.

My reading process lately is to eyeball each page of an article, which is a preliminary habit I’ve always employed before reading anything that’s non-fiction. Then, I have Michael Caine read the article to me.

Did that cause you to pause?

Take a look at ElevenReader and watch this:

It’s a bit disconcerting to hear the 92-year old actor talk so articulately about AI. (I’m assuming ElevenLabs paid a load of cash to license that voice.) Anyway, hearing a quality voice reading a complicated text to me is another level of absorbing the information.

In the Ding, et al, survey, I identified a few dozen out of the hundreds of sources covered that might be of interest to me. That survey also has been cited more than 100 times in other publications. The survey was first released in December 2024, with updates in mid-2025 and the end of 2025.

From the above diagram, you see how they define the two aspects of world models as

• implicit representation of the external world

• future predictions of the physical world.

From the applications section of the diagram (the bottom part), my interests are in the virtual and societal applications.

At this point, I still have dozens of papers on world models that I want to review. I won’t read most of them closely. I’m looking for things that really stand out relevant to my interests.

I’ve been talking a lot with my high school kid about the process of research and am trying to model that process for her; on one of the 3 desks in my home office, I have colored-coded index cards scribbled with notes. (All 3 desks are for my use, though I let her use one when she’s studying with me.)

Once I have a grasp on the topic, I will then turn to AI tools for another perspective. But I don’t want to do that too early in this process. Before working with Claude and NotebookLM on this topic, I’ll review what has come across my X feed lately.

X is the best way to stay current with very recent developments in AI. It seems that most AI researchers post regularly on X (with the exception of Yann LeCun, who seems to have had enough of Elon, and primarily posts on LinkedIn). If you’re getting started on X with the intention of tracking X developments, the best approach is to follow some of the lists curated by Robert Scoble. Be aware of the many fraudsters on X posting about their one-shot marvels or their undisclosed shilling for specific companies. (I almost wrote “unscrupulous companies” but that would seemingly entail most of capitalism at this point.)

Context: On game engines and world models

Here are some key insights recently from X.

Tim Sweeney, founder and CEO of Epic Games, maker of Unreal Engine:

As providers of world models are highlighting their ability to generate games, the evolving nature of world models have a direct impact on Epic’s future. Epic’s Unreal Engine is the single most complex (and fascinating) software I’ve ever used. AI assisted tools will make developing in UE easier. The integration of world models with game engines will unlock significant advances in not only gameplay but also any form of simulation. Game engines are used for much more than creating games.

That clip from his post cuts off the 2nd and 3rd paragraphs:

What would the ideal kind of memory be? It would be essentially a scene graph (James Clark, 1976!) hierarchically describing all of the objects in a world together with their properties, in a way that’s easy to query and operate on.

This is the reason for the core thesis that the ideal evolutionary path sees a merging of world models and engines, the AI side providing vast amounts of loosely organized audiovisual and textual knowledge, and the engine providing consistent reproducible data representation and simulation.

I have another upcoming post in the works about scene graphs.

Jacob Sansbury, developer/entrepreneur:

Click through on that post. He goes on to write about a point that I think is on target:

Medium-term is where it gets interesting. Add sound generation, longer context, more control and you have something Netflix should be terrified of. Imagine exploring Westeros between seasons. Wandering the Stranger Things universe. That’s a real product, and it’s coming.

But that’s interactive storytelling.

Gamers play because it’s fun to get better at something. Progression systems. Mechanical mastery. Nostalgia, where things work exactly how they always worked. They sink months into a single title. Years. And here’s the thing: they mostly don’t care about graphics or narrative.

Every single one of these motivations sits at the exact weak spot of world models.

Games require determinism. Multiplayer needs every client to agree on physics, every frame. Speedrunners need frame-perfect consistency across thousands of attempts. Competitive play needs rules that don’t drift. You can’t have ranked when reality is probabilistic.

World models are competing with passive media.

Long-term, they’ll probably eat the renderer. Generating pixels instead of rasterizing triangles. But game logic, systems, authored constraints? That’s a different problem entirely.

He’s wrong about the renderer part but it’s all comes down to how the pixels are generated.

The Marketplace: World Models are Coming Everywhere

Everybody seems to be releasing world models in 2026. I’m not going to list every announcement. World Labs, because of the quality of their team, is a company I’m keeping an eye on. Here’s a good essay from Fei-Fei Li on spatial intelligence.

Martin Casado is posting a lot about world models, 3D scene generation and games:

But he’s also a general partner at Andreessen Horowitz, an investor in World Labs. In the last decade, I noticed that VCs have become adept marketers of the companies in their portfolio.

The first time I ever learned about venture capital was in high school in the early 1980s. I lived near Nashville and the newspaper ran a profile on a local (Jack Massey), who was actually quite a major figure in venture capital at that time. But I always got the feeling then that these guys (Massey and Lucious Burch III) kept a low profile compared to today’s VC firms that are all over the media with their dedicated YouTube channels and podcasts. But this is a different era.

David Baszucki is the founder and CEO of Roblox, which is a company I first learned about a while back when my daughter was about 8 years old. She told me about Roblox.

Roblox is a company to watch. Hook the kids on your product; those kids eventually become adults with a disposable income of their own. The music industry has known that for years.

Runway is the company in this space that has the best aesthetic sensibility. Co-founder and CEO Cristóbal Valenzuela maintains an excellent X account. Runway’s focus, for now, is on AI films. But their use of video generation is an essential element of how I’m thinking about world models.

Take a look at Valenzuela’s essay from last year on It's All About The Pixel Economy (Or Why You Might Be Optimizing for the Wrong Future)

And the thing about everybody’s doing world models. Here’s a post from VP for AI at AMD:

Research: The Technical View behind World Models

But what are these world models really doing?

Jim Fan, NVIDIA Director of Robotics and Distinguished Scientist published an “article” on X where he describes world modeling as the next paradigm shift:

Very few understand how far-reaching this shift is, because unfortunately, the most hyped use case of world models right now is AI video slop (and coming up, game slop). I bet with full confidence that 2026 will mark the first year that Large World Models lay real foundations for robotics, and for multimodal AI more broadly.

That article is an excellent short introduction to world models in 2026 and explains how all this video generation relates to robotics.

He notes the key question:

That comprehensive survey by Ding, et al, points to articles on the technical nature of world models.

That’s why you also need to follow Yann LeCun on LinkedIn, who explains “that a proper world model is not a video generation system.”

That link goes to an 8-hour video. LeCun’s talk starts at 5:42.47

That workshop starts with a talk and Q&A session with Jürgen Schmidhuber, who co-wrote a foundational paper on world models. I wrote about Schmidhuber in my earlier post on world models. You can’t talk about world models without talking about Schmidhuber. (And Schmidhuber will remind you of that if you do so.)

On Feb 7, 2026 on X, Yann LeCun posted: “Anyone with an interest in world models should watch this”

“This” is this:

Malik’s presentation starts at the 1 hour, 4 minute mark.

Next steps

Some of that material I’ve already looked at closely. Other parts I’m going to look at more closely. Some I will put aside. Some of you may wonder why don’t I just drop that all into something like NotebookLM? That will be interesting, and I will do that at some point. A big question for this process is whether you start with an analysis done by an LLM or if you engage with an LLM while you read (or watch) or afterwards?

There’s not a best answer to that. It depends upon your goals, i.e., why are you handling this material in the first place? How deep into certain areas of knowledge do you want to go and for what purposes?

I probably take the middle ground and utilize Claude as well as NotebookLM as a reading partner to help me track what I’m learning. Ultimately, the only way I’m going to know if I have absorbed this topic is if I can comfortably express the concepts in my own words, connect those concepts to my own work, and articulate the insights I have gained from those connections.

Go back to the photo of my messy office. Those colored index cards are where I write (by hand) my thoughts that will (by reassembling the index cards) form an essay.

That’s a very different process than the one I used to write this long, and a bit rambling, post on the sources I’m gathering. This post is more than 2,500 words of free association that I typed directly into Substack at my stand-up desk over four hours on a Sunday. Is this a waste of time? Not for me. This is my life.

AI can be very valuable in writing, especially for editing your prose. But it’s too easy to lose your own writing voice and adopt the manner in which AI expresses a topic. LLMs often come up with a resounding phrase, which is tempting to insert here and there. I do use LLMs extensively everyday for brainstorming, research, and coding. Claude has become my daily driver, followed by Grok, and then Gemini. I’m cancelling my Pro subscription to ChatGPT but staying on the lower paid plan. Let’s get on with this month’s update.

I’m trying to stay focused on my one story. See my last post.

But what is Night Listener, you might ask. You thought I said I was working on Never Fear Tokyo??

Night Listener is a self-contained companion game to Never Fear Tokyo. Limited in scope, Night Listener is a project that I should be able to get done without getting sucked into the messy complexity of a larger environment.

This game takes place in one room over one night with only one character.

The character is Alex, the same photojournalist protagonist of Never Fear Tokyo, who has arrived to cover the 1964 Olympics. During this rainy night in his rented room, he overhears voices that he initially assumes are his neighbors. As the night progresses, the phrasing and tone of what he hears intensify to the point where he doubts where the voices are coming from, whether from neighbors or the supernatural. It’s a rough night for Alex as he visualizes shapes on the wall. Outside his window is a neon sign, typical of Tokyo; the neon casts color into the room. Throughout the night, the colors change and the sign changes to reflect words he’s hearing. Or, at least that’s what he imagines.

As the player, you have to figure out, along with Alex, what’s happening in that room that night, and what type of shape he (and you) will be in when the sun rises in the morning.

I have a set of game mechanics that I won’t go into at this point. In subsequent devlogs, I’ll label those as spoilers in case you’re interested in experiencing Night Listener.

The mechanics are documented in a game design document that’s roughly over 5k words. I did use ChatGPT and Claude to help brainstorm the game. But most of the key elements were ideas that I came up with on my own. For example, a couple of the key concepts I added that the LLMs didn’t come up with for the game:

• Night Listener is an auditory experience. The voices, heard only in Japanese, are the key to unlocking what’s happening. Game mechanics help a non-Japanese speaker, which includes Alex, grasp the impact. But even if you know Japanese, the voices arrive in fragments that need assembling. The essence of Night Listener is an audio experience about not understanding and listening without context.

• The player, like Alex, is a listener constrained to the specific space of a room during one night.

Obviously, I don’t expect this to be a blockbuster game! The appeal will be limited but if only a hundred people find this intriguing then I will have accomplished what I set out to with this small game. The more comprehensive Never Fear Tokyo is designed for a broader audience interested in narrative-driven games.

A defining aspect of both games is that they’re presented as a stage play rather than a full 3D world. That also was my own idea and not one provided by AI. You’ll initially view Night Listener as if you’re sitting in the audience of a theater and watching a play. Take a look at the concept art for this post, and recognize how it resembles a stage. Night Listener could work equally as well as a play dramatized on stage or as a short story. I’m highly interested in transmedia. But the game provides capabilities not possible through other media. As you engage with Night Listener the game, you’ll realize that you can switch to Alex’s first-person POV that unlocks capabilities not seen from the audience’s third-person POV.

Voice as Pedagogy & Research Testbed

Which of my three “big ideas” does this relate to? Voice as pedagogy. But I’m being very careful that Night Listener is not an educational game for learning Japanese. That would be a bit boring. But it is helping me learn the language.

The expressions and utterances that Alex hears over the night form a cohesive story in itself, though not told in order. The story is one I wrote nearly 20 years ago while living in Buenos Aires. I’m translating the text, which is mostly dialogue, into Japanese.

For the voices in Japanese, I’ve been experimenting with ElevenLabs, which has cloned voices of native speakers from Japan. There are also other software more finetuned for Japanese. But I probably will end up contracting with voice actors. The expressiveness of AI-voices is still lacking. And that’s an area I’m tracking with my research.

Night Listener provides a benchmark for measuring acting expressiveness through AI voices. With significant effort, one probably could modify AI voices to the extent that they would be suitable for my purposes. But this is a case where it would be easier and quicker (thought not cheaper) to hire voice actors. Based on my experiences and knowledge of generative AI for audio, we should have fully expressive generative voices in the major languages by the end of 2027. That’s a very conservative estimate but note the emphasis on fully expressive. For many purposes, you can do that now. And I will be using Night Listener, even after I finish this game, as an evaluation tool for benchmarking the acting quality of generative audio.

Voice acting is not a good career option for anyone in the mid-21st century. (Is any career a good option going forward?) People who want to pursue voice acting will need to start authoring their own stories and bringing their own characters to life while also figuring out how to monetize their efforts. There will be audience for that! But voice actors should not expect that their future income will come from publishers and for-profit companies that currently contract voice actors. Yet, there will be opportunities for publishers and studios that state they only use human voice actors; but I expect that segment to be very small since businesses will seek to optimize their expenses.

Working with Unreal Engine

I already mentioned the concept art for Night Listener.

I made the first pass at a gray blockout in UE. Before I settled on the stage-like aspect for Night Listener, I thought about a realistic room for 1964 Japan with tatami mats. Rooms in Japan are sized based on tatami mats. A 6-mat tatami room would be about 10 sqm.

The sketching was a useful exercise, especially for thinking about how the character moves within a constrained space, but a game environment doesn’t need to be realistic. It needs to be plausible.

I’m developing in Unreal Engine 5.7. I’ve been working with UE for more than 5 years, and I still learn something new every time. I’ve been diving into Substrate, which is UE’s latest approach to materials.

If you’re on X, you’ve might have seen short clips of people creating games with world models, or people creating games for themselves with tools like Claude. Will we continue to need game engines like Unreal Engine? I think so, and I’m writing an essay to explore that topic. Eventually, there will be an integration of world models and game engines.

Language learning

Some might wonder if there’s a need to learn a language if AI can translate for us via earbuds or smart glasses. While that tech will be useful, many people will still embrace learning a language in order to fully integrate into a community with native speakers of wherever they’re at. I’ve used Google Translate quite effectively in Vietnam while conversing with Vietnamese who don’t speak English. But, after a while, you run into limitations, especially with the difficulty of ‘small talk’, the casual banter that form a friendship. Developing a deep relationship through a translation app is impossible. Besides, language learning has tremendous cognitive benefits.

There are a massive number of YouTube channels dedicated to language learning. Combined, those channels have an audience in the millions. AI translation has its place but people will still continue learning languages. I do question whether teaching a foreign language in a classroom setting has a role in future years. When language teachers start using words like preterite and imperfect subjunctive, they’re already on the path to failing as a teacher. Instead, teach grammar as patterns to express what you’re trying to say. No one is standing on a street corner attempting to talk to someone and thinking, “Oh, in this moment, I need to use an intransitive verb.” Students struggle enough to learn the language; they don’t need to be confused with academic grammatical terminology.

AI offers some really great methods for language learning in a personalized context.

Learned some Japanese. As for my own progress in Japanese, I’ve started to recognize some words when watching a Japanese-language show. I’m understanding the structure of the language and sentence patterns. I feel that I’m on the path and have built confidence that I can learn Japanese.

Isn’t learning a language with a different character set challenging? Yes. Russian was the first foreign language I learned. (That’s an unusual choice for an American who knew nothing other than English; my high school offered Spanish but my speech impediment made me very insecure about taking an elective Spanish class though I wanted to do so.) My college required a third-year foreign language course for the bachelor’s degree.

I expected at the time to go into a career in international relations, which I wished I had done but I lost focus and got sidetracked. It was the mid-1980s and the cold war was still in full bloom. On the very first day of class, my Russian professor introduced Cyrillic. That was our focus for the first week, and it only took a week to learn Cyrillic. (I ended up taking 8 courses in Russian before graduating.)

So, if you want to learn Japanese, start with learning Hiragana and Katakana. That’s 92 characters with some additional variations on those characters for pronouncing certain sounds. Some sounds in the scripts overlap to produce 71 standard sounds in the Japanese language. (It’s still a lot easier than English despite Japanese having more sounds to learn.) What about the thousands of Kanji? Focus first on Hiragana and Katakana, especially Hiragana. Phonetically, everything in Japanese can be written in Hiragana, though it’s not from a practical perspective. The latter is where the concept-based Kanji comes into use.

I found these letter blocks on Amazon. It’s really helpful for assembling words in a tactile manner for learning the character sets. One side of the block has Hiragana, another side Katana, another side has a Kanji character, etc. (The photo below only shows Kanji.)

I’m immersing myself as much as possible in hearing Japanese; Netflix has a lot of Japanese shows. Japanese reality dating shows are very good, much better than their counterparts in other countries. The Japanese express a kindness and courtesy in their dating shows that are absent from American dating shows. In American dating shows, there are always these buff guys (talking loudly) looking for the chance to strip off their shirt at every moment. That doesn’t happen on Japanese dating shows. I highly recommend Offline Love, which is set in Nice, France. The participants travel from Japan to France. The concept of the show is that they must lock away their digital devices and rely only on handwritten letters and communicating in person. It’s a throwback to life before the 1990s when you couldn’t tell someone you would be late for lunch because there was no texting or calling on mobile phones.

About 5 other Japanese dating shows consumed my viewing time in January. I also found time for two movies: Pale Flower and Crazed Fruit. These are directly related to my theme of 1960s Tokyo.

I strongly recommend Pale Flower. That’s a great movie.

I also found two books on eBay that will help tremendously with developing Never Fear Tokyo .

Japan on $5 a Day (1964 edition) and This Week in Tokyo, which covers the week of November 2, 1964. The latter is a remarkable find! Never Fear Tokyo is set during the 1964 Tokyo Olympics, which happened in October. I think I’ll shift the storyline to be the week just after the Olympics ended, which is the week of November 2, now that I have a detailed magazine on events and places during that week in 1964.

Big picture time

I’ve restructured my life to explore the new genre of storytelling that is filmic and interactive. Note the emphasis on ‘new genre’. This is not a replacement for movies, though many movies will use generative AI in the process just as movies have used special effects for a long time now. This genre is a hybrid between movies and games. Generative AI provides capabilities not possible in either medium. Extend that experience into virtual reality and you’ll arrive in the metaverse. This metaverse concept, which is still relevant, will not be broadly adapted for another decade. The tech still needs to mature, and we’ll need to wait for better devices than Meta’s Quest to come to market.

Will that ever happen? Yes. While the market evolves, major developments are happening in research labs across the world. Technical reports and research papers are constantly being published that describe new techniques in generative AI for scene development and 3D. We’re also seeing a lot of venture capital flowing into world models, which offer an interactive experience similar to games (though with a lot more limitations).

One of the things pushing off developments in VR hardware is an emphasis in the market on robotics. Honestly, I’m not ready for humanoid robots. But I would appreciate a robotic lawnmower or a device to sweep the cobwebs from the corners of my ceiling every so often. We’ve had robotic vacuum cleaners for a while now, but I would drop some serious cash for a robot that can deep clean my house and fix any maintenance problems that come up.

From a research perspective, I’ve chosen to ignore robotics. I can’t go down every rabbit hole. But I will go deep into world models, which are used to train robots. Of course, I’m more interested in the entertainment aspect of world models.

Among the questions I’m examining in February: what does interactive video, developed via world models, mean for game development?

It’s easy to get distracted chasing the latest announcement, the newest AI model, the shiniest object. I’m not here to report AI news.

This year, I’m going all in on one project: creating an interactive story that lets me explore both the promise and the failure of generative AI for creativity.

The Story

Never Fear is my story about the absence of fathers and the obsessions their sons never name. In fact, I’ve already been working on this for 20 years. First as a novel, and now I’m adding new aspects as an interactive narrative. Maybe I’ll work on this for the next 20 years.

It spans three cities: Paris in the 1920s, Tokyo in the 1960s, Buenos Aires in the early 2000s. At its center is the artist Mateo Monti, who vanished from Paris in 1926. I wrote about the Paris portion of this story in October 2024 on this newsletter in a post titled "Crafting an Interactive Mystery”.

In 2026, I’m focused on the Tokyo chapter (Never Fear Tokyo). You play as Alejandro “Alex” Monti, an Argentine photographer who arrives in Tokyo to cover the 1964 Olympics. Beneath his assignment lies a personal mission: his father Mateo was sighted in Tokyo, decades after disappearing.

This isn’t just a mystery. It’s also an experiment in language learning: you and I are learning Japanese along with Alex as he learns. I’ll be using AI-generated voices as our guides through the story.

What I’m Actually Building

Let me be clear about what I’m trying to do. I don’t want to automate creativity. I want to learn how AI can help generate conditions that enable the audience to feel:

• dread

• recognition

• obsession

• tenderness

• distortion.

These traits signal the absence of those you love. That matters to me.

So when I look at AI tools, I’m asking three questions:

• Does this help me shape perception? I’m thinking about how the audience experiences something through attention, tone, uncertainty, and interpretation.

• Does this help me maintain consistency across different media, different tools, different sessions?

• What’s the actual workflow? Not for a 4 second clip, but the real process of making a complex story world.

Through this year, I’m documenting where AI helps and where it fails. Both matter.

Three Big Ideas

I’m organizing my work around three big ideas. Of course, these are not unique to me. These are concepts to explore.

I use the term big ideas, intentionally, based on the Understanding by Design framework for teaching. Throughout the year, I’ll break each one down into essential questions, learning objectives, and the practical skills and knowledge that emerge from the work.

Here’s the landscape:

1. Stylistic Consistency

Can AI generate coherent aesthetic identity across media?

For Never Fear Tokyo, I need grainy, 16mm-style visuals depicting rain-slick streets, flickering neon, figures appearing in photographs. I need period-accurate audio, e.g., jazz drifting through a bar.

For a chapter within the project, I need soundscapes, specifically whispers through walls in fragmented Japanese and Spanish, environmental sounds that shift, evoking voices from a future you’ll never see.

The challenge isn’t just generating these things. It’s generating them consistently. The same character aging across photographs. The same unsettling tone across audio scenes.

I’m watching developments in multimodal AI (models that work across text, image, audio, and video) because consistency across media is where so much falls apart right now.

2. Voice as Pedagogy

Can AI-generated voices that teach while they also tell a story?

Never Fear Tokyo isn’t just a mystery. It’s an experiment in language learning. You learn Japanese alongside Alex as he navigates Tokyo.

This is harder than it sounds. Voices for language learning need to be accurate; getting AI voices to pronounce Japanese correctly is not easy. Plus, the voices also need to be alive. A voice that teaches you to ask for directions needs different energy than a voice whispering through a séance. The same phrase repeated for shadowing (a language learning technique) shouldn’t sound robotic; it should have subtle variation, the way a patient teacher might say something three slightly different ways.

I’m building what I call the Japanese Sentence Studio as a tool for tracking generated performance variants. The same sentence, but with different emotional shadings. Hesitation. Confidence. The flattened affect of someone hiding something. (Maybe I’ll make the code for that tool available; I don’t know, yet. For me, it’s simply a tool to develop the project.)

The goal is to create AI-generated voices that feel like characters not language tapes.

Emi, your guide in Tokyo, should sound like a person with her own relationship to what she’s teaching you and not a disembodied pronunciation engine.

This is where AI voice generation gets interesting: not just mimicking human speech, but creating voices that serve both story and learning at the same time.

3. Context Engineering for Interactive Stories

What should an AI know, and when, to generate a meaningful experience?

Here’s the core challenge: what does an AI need to know to generate the next moment of your experience within an interactive story?

That’s context engineering: deciding what information to feed the system and when. For interactive fiction, this gets complicated fast. It needs a game engine.

I’m developing this in Unreal Engine because I don’t think any generative AI tool can maintain that level of knowledge about each player’s experience. To clarify: Unreal Engine is a game engine that handles many factors; there still need to be generative AI tools that handle other factors in the gameplay. (This is analogous to how you build an AI-enabled app by scaffolding a lot of Python code around an API to an LLM model, except in this case, the game engine is the app and code scaffolding.)

Unreal Engine is designed for tracking player state, managing persistent variables, handling branching logic, saving and loading progress. It’s a mature system built for complex interactive experiences, especially in 3D graphics.

AI tools, even with long context windows, don’t have the same capabilities for session persistence or player-state tracking.

Tools that simulate world models and generate video are good for prototyping and visualizing scenes. But they’re generative, not interactive in the game-engine sense, though there’s increasing interactivity. People are developing small game environments through world models. But these systems can’t track that you lingered on a photograph in session three and adjust how that photograph manifests in session seven. Or, at least, they cannot yet do that. I’ll be extensively using tools build around world models to test how far they can go in 2026. I expect there will be significant progress on world models and video generation by 2027.

The system needs to know the world: Tokyo in 1964, the visual language of the story, the mysterious woman who haunts these photographs, and other aspects of the storyworld. It needs to know the player: what they’ve noticed, what they’ve resisted, what patterns they’ve fallen into. And it needs to know the current moment: where we are in the narrative, what just happened, what’s possible next.

Context windows are getting longer. Memory systems are getting more sophisticated. But the real work isn’t technical, the real work is storytelling: deciding what should persist and what should fade. Or, if you’re coming from a game design background, the real work is in the gameplay.

Closing

I’m not trying to keep up with everything. I’m trying to go deep on one project and document what I learn.

These three big concepts—Stylistic Consistency, Voice as Pedagogy, Context Engineering for Interactive Storytelling—will organize the work that I release throughout this year. Each concept raises essential questions I don’t yet know how to answer. That’s the point.

The hotel was established in 1880; that’s 145 years ago. The place has been updated a bit but still carries a lot of French colonial vibes. A plaque celebrates the hand-carved grand staircase. Yet, we all know the dark history of French colonialism in Vietnam and the decades of war that followed. Choices made more than a century ago reverberate through time. We live in choices made by others.

Ordinary life goes on outside. Across the street, stylish young Vietnamese women pose for photos in front of the Opera house. Presence lingers. Someday, their grandchildren will find those digital portraits, snapshots of a life.

Graham Greene likely sat many days at a table on the veranda of the hotel overlooking the Opera house. In the movie, a violent scene of explosions takes place on the very spot that is now an Instagram hotspot. Greene died in 1991 but his framing of place, conflict, and morality continue to influence conversations he never foresaw.

This hotel: a century and a half, 150 years: it’s not an abstraction. Many places in the world, particularly here in Vietnam, are witnesses to the lived reality of a turbulent, transformative time.

The Saigon that Greene experienced is not today’s Ho Chi Minh City, which now (through governmental redistricting) stretches 100km to the South China Sea. In the 70 years since Greene wrote The Quiet American in this hotel: wars came and ended, the city was renamed, the nation remade. 150 years is transformation stacked upon transformation.

If a hotel can last 150 years, if a novel can still shape thought decades after its author’s death, then what about us? What about the digital traces we leave behind? What about the ghostly avatars of ourselves that may linger in classrooms, homes, and public spaces long after we’re gone?

Responsibility Horizon

Last year, when I was teaching an undergraduate course on Responsible AI, I asked a simple question: how far into the future are we responsible?

At first, the answers clustered around the immediate: years, maybe decades, centuries. Then the conversation shifted toward family and the children that my students might have someday. I didn’t have a child until later in life, at age 45. For many years, I thought I would never have a child but I always recognized a responsibility for the next generation.

My students started to think not only about the lives of their future children, but the children of their children, the grandchildren of today’s college students. My students came to the point that the most concrete measure of the future is not abstract centuries, but the lives of their own grandchildren.

Let’s do the math. Assume that a 20-year-old college student today was born in 2005. Some will have children earlier or later, but let’s assume in ten years, age 30, that they will start a family with their first child born in 2035. And that child will have a child thirty years later in 2065. The child born in 2065 would be the grandchild of today’s college student. With advances in healthcare and longevity, that grandchild might live more than 90 years to around the year 2155. That’s 130 years from now. Adjust for various lifespans and years of birth and longevity, and we’re easily looking at a time horizon that the grandchild of today’s college student could be alive 150 years from now: the year 2175.

150 years. Not infinite time, not eternity, but a human horizon, the span of this moment today to the lifetime of a yet unborn grandchild.

This span of 150 years, using the frame of a grandchild’s life, makes responsibility felt. Our responsibility is not for strangers, but for our descendants, for the descendants of today’s youth.

If my students’ grandchildren are going to live with the traces of what we build, then we cannot measure responsibility by the length of a funding cycle, a product roadmap, or even our own lifespans. The horizon stretches 150 years. The challenge: how do we build with that future in mind?

What does that mean in practice? I’m trying to imagine my students’ grandchildren, logging into a class and finding me there: not in the flesh, but as a lifelike avatar, stitched together from today’s words, code, and videos.

Imagine if you can leave a lifelike avatar for your own grandchildren and great-grandchildren?

Ghosts in the Data

There are legitimate questions as to whether it’s even desirable for lifelike versions of ourselves to persist beyond us. But, undoubtedly, many will choose to leave a virtual legacy for the future. Like Greene’s novels, my avatar might still speak long after I am gone. But unlike Greene, my presence would be interactive, persuasive, and unbounded in reach. The question is not whether such ghosts will exist, but whether we can design them to guide rather than mislead, to bear wisdom rather than distortion.

Graham Greene, if he were writing in our century, might have given such a story a title like The Human Error: the tragedy not of machines gone rogue, but of humans failing to anticipate how their voices, their biases, their ghosts would echo far beyond their own lifetimes.

Writers know that they are writing for a future generation. But I often get the impression that business people know that their business is likely not to exist beyond their own lifetime. Not all businesses are made to last forever. They are made to make a profit and secure a financial foundation (often through being acquired) for the founders, their families, and investors. That’s why avatars must be built around open standards that surpass the control and limits of any one company.

Let’s enter the classroom of the future. (I use the word classroom, though I think it’s highly unlikely that the type of educational environment I’m about to describe will happen in a physical room filled with students.) Imagine the year 2098. A group of students situated anywhere in the world logs into a class. Their teacher materializes as a lifelike avatar of myself: stitched together from the posts, code, and videos I left behind in the early 21st century. (This thought horrifies me; I really need to up my game if my presence is going to linger around.)

An easy critique of this argument is that there’s no way we can know what 2098 will be like. We barely have an idea what 2028 will be like. So, let’s ignore the long-term and focus on building the present, the next few years. But building the present is building a presence that lingers year-by-year.

The avatar speaks with confidence, even when wrong. (Many professors, and most politicians, already do that.) This avatar (let’s just call it Professor) carries my voice, my mannerisms, my blind spots. To those students, the Professor feels present, alive, authoritative. But it is also a ghost trained on a version of me that never had the chance to answer their questions, adapt to their world, or learn from their mistakes.

What, then, should we be doing now to account for such ghosts trained on versions of ourselves that never had the chance to answer future questions, adapt to future worlds, or learn from future mistakes?

Perhaps one response is contextualization.

We often read literary texts, like Greene’s The Quiet American, with introductions and footnotes that remind us of the era in which it was written. Perhaps our digital presences need something similar: layers of metadata and framing so that no one mistakes these avatars for timeless oracles.

Design for Contextualization

But what happens when our digital ghosts speak without footnotes? An avatar trained on my words might answer a question in 2098 with total conviction. Yet the answer would be grounded in assumptions from 2025, a time of very different debates, blind spots, and horizons of knowledge. To a future student, though, the avatar’s confidence might look like authority.

If books require footnotes, perhaps avatars require overlays: metadata that marks the time and place of their training; caveats that declare what they could not know; context that helps future audiences interpret rather than simply absorb. Without this, digital presences risk becoming timeless in the worst way: voices speaking as though they are universal, when in fact they are provincial, contingent, and bound to their own moment.

The ethical design question: how do we embed annotations into presence itself? Not just for scholars, but for anyone who will live with the legacies of our voices long after we are gone.

Another response is impermanence. A hotel might stand for more than a century, but a digital avatar could be designed with off-ramps: expiration dates, kill-switches, or archival transitions. Not all businesses are started to last forever. Not everything we create should speak forever.

Build Sunset Clauses & Off-Ramps

The Continental Saigon has lasted 145 years, but that is precisely the problem. Its persistence is not neutral; it carries the architecture of colonialism into the present. Digital presences may prove even more enduring, but unlike physical structures, avatars can replicate infinitely and seep into every corner of daily life. If left unchecked, these digital voices may speak forever with a confidence unmoored from reality.

That is why we must ask: should every presence persist?

In most of our infrastructures, we already accept the necessity of endings. Bridges have lifespans. Medications expire. Hotels need renovating. (You better change those mattresses often.) We plan for renewal, revision, or retirement. Yet when it comes to digital avatars and AI systems, permanence is often treated as the default with immortality coded in by design.

What would it mean to embed off-ramps? To give our avatars expiration dates, graceful shutdowns, or archival transitions that allow them to persist as history but not as authority? Could an avatar be designed to “retire” after 50 or 100 years, shifting from a living presence to a documented artifact? Is this a new job for librarians: the avatar archivist?

The ethical horizon of 150 years demands humility: not everything we create should endure. Some presences should end. The challenge is deciding not only when but also how, so that the traces we leave are inherited as history.

Equally important is documentation. An avatar built from today’s words and code will carry today’s blind spots. We can begin to name those limits now by recording not only what we know, but what we don’t, and what we suspect we may have wrong. But nobody likes writing documentation. How can generative AI embed the needed documentation into the metadata that resides with our avatars?

Document Bias and Limits

Every voice is partial. Greene’s The Quiet American was not a neutral account of Vietnam; it was steeped in his perspective, his disillusionment, his politics, his religion. Today, scholars and readers approach it with that in mind. The text carries his bias, but we can locate it, contextualize it, and argue with it.

An avatar, however, carries bias in a different form. Trained on a finite body of words, code, and images, it speaks as though that slice of a life were the whole truth.

If we accept that these avatars will exist, then one responsibility we bear now is to make our partiality visible. This means documenting not just the content we produce but also its conditions: what sources we relied on, what perspectives we ignored, what we misunderstood. It means recording our doubts alongside our convictions, so that future interpreters have more to work with than a seamless illusion of certainty. Who is going to do that work?

Imagine if every avatar carried with it a ledger of limits, a record of the assumptions baked into its training. Future generations could treat our digital ghosts not as oracles, but as situated voices.

To acknowledge bias is not to weaken presence. It is to leave behind a trace that is more honest, more interpretable, and ultimately more humane.

There is also the matter of governance. The decision to keep or retire an avatar should not belong only to the individual or to the companies that host them. We may need intergenerational mechanisms (trusts, councils, or institutions) that can audit and curate the presences that persist.

Intergenerational Governance

Who decides which presences persist? With books, there are editors, publishers, and libraries. But those are gatekeepers whose presence is increasingly shown to be unneeded. With monuments, there are city councils and preservation boards. But those can be weaponized for political ends. Even bridges and buildings are inspected, regulated, and often torn down and replaced with something more appropriate for the era. Is presence ever left entirely to chance?

Our digital avatars risk slipping through those filters. Once created, how do we prevent these avatars from being copied endlessly, hosted anywhere, and repurposed without oversight? My students’ grandchildren may not just inherit one version of me: they may inherit dozens, some faithful, some distorted, all speaking with authority I never authorized.

This is not only a technical question; it is a political one. Who has the right to retire a ghost? Who can audit its accuracy, revoke its license, or determine whether it is still fit to speak? If we leave those decisions solely to corporations, the answer will be guided by profit. If we leave them to the dead, the answer will be silence.

We may need new forms of governance that look forward across generations rather than back to individual rights. Imagine an intergenerational trust that holds digital legacies in stewardship, auditing their use every few decades. (Yikes, that’s rather concerning.) Imagine councils charged with deciding whether an avatar still serves the living or whether it should be archived. Honestly, I don’t see how any of that bureaucracy will work in practicality without being used for political advantage to influence public opinion.

But without some mechanisms, the danger is not simply that ghosts will persist, but that these avatars will persist without care. Intergenerational governance is the recognition that responsibility cannot stop at death. It must be carried forward, not just by families, but by institutions designed to mediate between the living, the dead, and the not-yet-born.

Most of all, we need to shift the frame. The goal is not digital immortality. It is stewardship.

Shift the Frame from Immortality to Stewardship

Behind many visions of digital legacy lies a hidden desire: immortality. The fantasy that we can keep speaking, keep teaching, keep being, long after our bodies fail. But immortality is not the same as responsibility. An avatar that never ends may not honor us; it may burden those who inherit it.

What if we stopped asking how long our presences can last, and instead asked how well they can be stewarded? Stewardship implies curation, care, and responsibility. It means pruning what no longer serves, preserving what still holds value, and handing down legacies as gifts for future generations.

In this light, the measure of a digital ghost is not its duration but its quality of care. Who tends to it, updates it, retires it, or reframes it for a world its maker could never imagine? A stewarded presence could be a guide, a companion, or a historical artifact. An un-stewarded presence could become noise, distortion, or ruin.

The ethical shift we need is simple: from the pursuit of endless life to the practice of responsible inheritance. Our task is not to outlast ourselves, but to ensure that what endures is worthy of being lived with.

The Continuum of Presence

If the Hotel Continental Saigon reminds us that colonial legacies can linger in structure, and Graham Greene that words can linger on the page, then our responsibility is to recognize that digital presences will linger in ways harder to erase.

Questions to think about:

• What traces of yourself are you leaving today that could still be speaking 150 years from now?

• If future generations encounter your voice, will they inherit wisdom or your blind spots amplified by confidence?

• Should every digital presence be designed to persist, or do some deserve an ending?

• Who gets to decide which ghosts endure: you, your descendants, corporations, or non-profits that take on archival responsibility?

• When presence lingers long after intention has faded, what does responsibility really mean?

Will what we leave behind be inheritance, scar, or error?

When an AI generates an image, does it see a picture or a world?
Consider whether AI understands spatial relationships or merely assembles pixels.

Is a simulated character alive beyond the camera’s frame?
Think about whether virtual beings persist in meaningful ways when they're not directly observed. (That sounds intuitively ridiculous but consider the role of persistent characters in an ongoing game-like experience.)

What if creativity isn’t about generating content but about simulating possibilities?
Reflect on shifting our focus from AI as a content factory to an imagination engine.

How can we distinguish between a compelling illusion and genuine understanding in generative AI?
Probe the difference between superficial memorization and internal comprehension of a simulated reality.

Does the AI know it broke the window (and does it care)?
Explore the significance of cause-and-effect relationships, agency, and consequences within AI-generated worlds.

In my recent course on Data for Deep Learning, we grappled with a deceptively simple question without realizing it:

What makes a simulated world feel coherent, interactive, and alive?

It’s a question with implications far beyond gaming. The systems we design for storytelling, immersive education, urban planning, self-driving cars, and robotics all rely on a concept that many creators and researchers treat as invisible: the world model.

What Is a World Model?

In the most basic sense, a world model is an internal representation of an environment. It’s how a system (whether a game engine, an AI, or even a simulation of a city) understands what exists, what happens, and what could happen next. It’s the architecture beneath the surface, the logic that holds a world together.

But world models are not just about pixels, coordinates, or geometry. These models are about relationships, causality, and possibility. These models provide structure to both immersive storytelling and machine learning, serving as the hidden foundation upon which all interactivity rests.

As of August 2025, we are seeing a lot of exciting releases that advance the use of AI based on world models. This post isn’t about those releases but about the concept of world models.

Beyond the Pretty Picture: A Creator’s Guide to Evaluating the New Wave of Generative AI

For creators in gaming, film, and storytelling, this new wave of generative AI is undeniably powerful, but it's also a source of profound uncertainty. How do we distinguish a dazzling tech demo from a genuinely transformative creative tool? How do we look past the immediate spectacle to gauge an AI’s true potential for our craft?

The answer doesn't lie in the polish of the pixels or the slickness of the short clip. It lies in a deeper, more fundamental capability: the AI's ability to understand and simulate a world.

As we navigate the hype, the single most important question to ask of any new generative tool is not "What can it generate?" but "What kind of world does it understand?" The shift from a mere content generator to a world simulator is the leap that will redefine creative collaboration.

The Pitfall of the Static Image

We’ve seen how generative AI has excelled at creating beautiful but shallow artifacts. Like a Hollywood set, they are all façade. A diffusion model can render a stunning jazz scene in a Paris nightclub, but it has no concept of the space behind the stage or the weight of the people sitting around the table. A short, text-to-video clip can show a character walking down a street, but the AI often struggles to maintain that character's appearance or the street's layout in the very next shot.

This is the core limitation of models that lack a robust internal simulation engine. They are pattern matchers, not world builders. For a storyteller, this is the difference between being handed a single, static photograph and being given a location to film in. The photograph is a prompt; the location is a canvas. True creative work, whether it’s interactive game design, dynamic cinematography, or emergent narrative requires a canvas.

The Engine of Imagination: What is a "World Model"?

This is where the concept of a world model becomes the critical lens for evaluation.

Pioneered in research and now powering the most advanced systems, a world model is an AI’s internal, compressed, and predictive understanding of an environment. It's a learned simulation engine that allows the AI to "dream" or "imagine" what will happen next based on a coherent set of rules.

A sophisticated world model doesn't just know what pixels go next to each other; it learns an intuitive grasp of the principles governing a world:

• Spatial and Temporal Coherence: It understands object permanence: that a character who walks behind a pillar still exists. It maintains consistency over time, so that a shattered window remains broken in a later scene.

• Cause and Effect: It learns that flipping a switch illuminates a room, pushing a box makes it move, and an action taken now will have logical consequences later. This is the foundation of agency.

• Intuitive Physics and Affordances: It develops a feel for gravity, momentum, and how objects can be used. A chair affords sitting; a lever affords pulling. This is crucial for creating interactive environments that feel natural and responsive.

These aspects are fundamental with game engines like Unreal Engines. But that’s a tool where the game developer directs everything.

Yet, when an AI tool generates a playable video game from a single image, its true breakthrough isn't the pixel generation but the latent action model: an embedded world model that understands what "jump" or "move right" means within the context of that 2D world's physics.

A Creator's Checklist: What to Look For

With this understanding, we can formulate a checklist to cut through the hype of the next big AI announcement. Instead of being passive spectators, we can become active, critical examiners.

1. Does it Demonstrate Consistency and Coherence? Look past the 3-second clip. Ask for the 30-second version. Does the character’s clothing remain the same? Does the environment’s layout stay consistent when viewed from a different angle? Systems built on strong world models need to track relationships between different parts of a scene over time in order to exhibit far greater temporal and spatial stability.

2. Does it Enable Agency and Interaction? This is the litmus test that separates a passive video generator from a virtual world. Can a user act within the generated space? Is the environment playable? The world model provides a framework for user actions to have meaningful consequences. When examining a new tool, ask: Is this a movie, or is it a place?

3. Does it Support Emergent Behavior? The ultimate goal is not just a simulated world, but one that can surprise us. A world model with a robust understanding of its own rules can facilitate emergent narrative and gameplay. An NPC, guided by the world model's logic, might react to a player's unforeseen action in a way that is not explicitly scripted, but is consistent with the world's rules. This moves the AI from a simple executor of commands to an improvisation partner.

4. How Controllable is the Story and Art Direction in the World? A powerful simulation is useless if the creator cannot steer it. The most promising tools will not be black boxes. Look for evidence of controllability. Can we provide high-level textual guidance that designs game mechanics from stories? Can we define the world's physics, its aesthetic, or its core rules? The goal is not automation, but augmented creation.

As we move forward, the most transformative AI tools will not be those that simply offer to do our work for us. The most transformative tools will provide us with new worlds to work within. These tools will function less like a content factory and more like a physics engine for reality itself: a malleable, explorable, and interactive canvas for our imagination. The next time a new model makes headlines, look for signs of a world behind the curtain. That is where the future of storytelling truly lies.

From Hand-Crafted Worlds to AI-Generated Simulations

When we talk about world models in virtual reality or mixed reality, we often focus on designed systems. Whether it’s a digital twin of a city, a historical VR simulation, or an interactive museum, these world models are authored to deliver particular experiences.

But there’s another side to the story that involves systems that learn their own models of the world.

In 2018, David Ha and Jürgen Schmidhuber published a paper titled “World Models.”

Their central idea: instead of training an AI by forcing it to explore the real world over millions of iterations, why not let it learn a simplified model of the world and train itself inside that model?

They called this process “learning inside a dream.”

“Can we train our agent to learn inside its own dream, and transfer this policy back to the actual environment?

If our world model is sufficiently accurate for its purpose, and complete enough for the problem at hand, we should be able to substitute the actual environment with this world model. After all, our agent does not directly observe the reality, but only sees what the world model lets it see”

How Does It Work? (A Friendly Breakdown)

Ha and Schmidhuber’s architecture has three components:

• Vision Model: Compresses raw input from the environment (like video frames) into a simplified, abstract representation known as a latent space.

• Memory Model: A neural network that learns how the world changes over time, predicting future states based on past states and actions.

• Controller: A decision-making model that learns how to act, using the abstracted, predicted states from the world model.

The key insight? The agent learns to act entirely inside its own dream world, a simplified simulation built from its own learned representations.

Why This Matters

The implications of Ha and Schmidhuber’s work extend far beyond AI research. When I teach my students about world models, I’m encouraging them to think about how systems understand and simulate reality.

World models are foundational architecture:

• Digital twins of factories or cities, where the system must track, simulate, and optimize complex processes.

• Interactive storytelling, where NPCs adapt and respond based on their internal models of the world.

• Virtual education, where the system monitors and adjusts to how students interact within a learning environment.

• Generative agents, which create realistic social simulations by modeling memory, motivation, and interaction dynamics.

What Ha and Schmidhuber introduced was a method of training AI inside its own imagination. It’s an approach that echoes how human creativity works by reflecting, simulating, and refining ideas internally.

Connecting World Models to Mixed Reality and VR

Whether designing virtual museums, historical simulations, or AI-driven learning environments, there are fundamental questions:

• What parts of a world model can be learned from data, and what parts must be designed manually?

• How can an AI agent be embedded within a virtual world so that it dreams and plans just as humans do?

• What kinds of data architectures support these simulations?

By bridging these questions to AI research, we see that world models are more than tools for simulation. World models are tools for imagination.

Next Steps: Modeling a Simulated Reality

World models are the foundation for simulating reality, not just for games, but for any interactive system that models a world to make decisions, support presence, or generate outcomes.

World models represent the convergence of perception, structure, dynamics, and learning.

Are we talking about the same thing? Digging Deeper into the Definition of a World Model

When people talk about world models, the term lacks precision. An excellent paper attempts to provide a theoretical foundation for clarifying the terminology: Li, Kenneth, Fernanda Viégas, and Martin Wattenberg. What Does it Mean for a Neural Network to Learn a" World Model"?. arXiv preprint arXiv:2507.21513 (2025).

That essay is a great example of clear academic writing that provides a rigorous operational definition of world model. Li et al. provide a clear, mathematically precise definition of a 'world model' based on the idea of linear probing. According to their definition, a neural network contains a world model if its internal computations can be simplified into an interpretable representation, directly reflecting key structures or patterns from the external world.

In interactive storytelling, a well-defined world model might let an AI reliably 'remember' that a character left an object in a specific location, influencing future narrative interactions. Li et al.’s criteria provide a way to empirically test whether an AI truly builds and uses such representations.

For game developers using AI-generated environments, Li’s distinction between a 'learned' world model and trivial memorization helps clarify if a neural network has genuinely grasped game physics, level logic, or player behavior.

When is an AI’s World Model "Real"? Three Key Criteria

How can we be sure an AI is genuinely modeling a world internally, rather than simply memorizing superficial details?

According to their research, a genuine world model must satisfy three important conditions:

• Nontrivial: The internal representation genuinely captures meaningful, structured aspects of the external world. It doesn't merely replicate or trivially repackage the input data.

• Learned: The representation is not directly contained in the input data itself. Instead, it's actively constructed by the AI’s internal computations.

• Emergent: The internal representation appears naturally within the AI, without being explicitly demanded by the AI’s primary tasks or outputs. In other words, it emerges organically from the AI’s attempt to solve problems.

Practical Ways to Experimentally Identify World Models

One of the strongest aspects of Li et al.’s essay is their emphasis on practical experimentation. Specifically, they suggest using "probing" techniques, like linear probes, to test whether a neural network’s internal representations correspond meaningfully to interpretable aspects of the external world.

Five Takeaways about World Models

1. World models are invisible yet essential.
   They form the hidden architecture enabling coherent, interactive, and dynamic simulated realities, whether in storytelling, gaming, or AI-driven simulations.

2. Authentic creativity in AI comes from internal simulation, not surface-level generation.
   Genuine AI creativity involves understanding relationships, possibilities, and consequences, not simply generating isolated images or scripted scenes.

3. Evaluate generative AI by the depth of its world model.
   Use practical criteria like consistency, agency, emergent behavior, and art-direction to distinguish powerful world simulators from content generators.

4. Advanced AI systems learn by imagining.
   Techniques like Ha and Schmidhuber’s “learning inside a dream” highlight how AI can internally simulate possibilities, much like human creativity does.

5. Clear definitions and empirical methods, such as those provided by Li et al., are crucial.
   Operational criteria and experimental probes allow us to rigorously verify whether AI systems genuinely understand the worlds they simulate, paving the way toward richer, more meaningful interactions.

Our memory fools us

If you’ve read much Faulkner, you’ll know that he writes a lot about the mistrustful nature of memory. Evidently, his own memory of writing As I Lay Dying was not as faithful as he remembered, or he strove to create a mystique about the inspiration that poured out of him by claiming that “he wrote the novel in 6 weeks, and without changing word.” The Digital Yoknapatawpha project at The University of Virginia details that claim as not entirely true.

Faulkner did change some words but not nearly to the extent that other writers revise their first drafts. But, hey, he’s Faulkner.

As writers increasingly co-author and/or edit with AI, how much will we know about an author’s writing process in the future?

Let’s set aside the argument of whether writers should or should use the aid of AI. And let’s not compare anyone to Faulkner. I use this example by Faulkner because it came across my X feed the day that I was thinking about this topic.

Even without the use of AI, the process of literary analysis based on manuscript collections has been impacted by handwritten (or hand edited) drafts replaced by word processors and written correspondences replaced by email. A lot of literary history has been lost in the last few decades, though there are researchers working on archiving digital files and versions.

Lots of writings are toss away works. John Lennon referred to many of his songs as throwaways: songs that were crafted quickly or without significant emotional depth. In the 1980 Playboy interview with David Sheff, Lennon goes through the catalogue of Beatles songs and his own while describing many as “throwaways”; I was disappointed that one of my favorite Lennon solo numbers was described that way. Maybe he was simply being self-deprecating.

But a true example of a throwaway work where we don’t need a historical analysis would be Ballard, an adaptation by Amazon of the Michael Connelly novels featuring his female detective Renée Ballard.

Last night, I was bored. I had finished watching Squid Game, season 3, the night before and wanted another series. I had the feeling that Ballard wouldn’t be good since it was a spinoff of the Bosch series, which started out good but I gave up on watching eventually. I got halfway through the first episode of Ballard and decided that I was wasting my life by watching it. I was a fan of Connelly’s Bosch novels, particularly the early ones but never cared for the Ballard series. I’ve stopped reading Connelly because I couldn’t remember which novels I’ve read before. Likewise, I’ve given up on watching the Mission Impossible movies because they all seem the same, and I can’t remember which ones I’ve watched. Sometimes I like mindless entertainment as a brain break from my Criterion Collection.

Despite what Hollywood writers say, a lot of the stuff on streaming could be written by AI, or made better with the aid of AI.

AI Filmmaking is here. Runway has a lot of momentum. Some incredible independent films will be made with AI. (More bad films will be made with AI.)

But here’s where things get interesting. AI doesn’t have to be a shortcut for cranking out slop to fill streaming channels, but a tool for building a new kind of narrative experience altogether.

Generative AI starts to shine not by replacing writers or directors, but by enabling entirely new kinds of stories that unfold through interaction, exploration, and presence.

If you follow my work, you know I’m interested in branching narratives and narrative-driven games.

Branching narratives: stories with multiple possible paths and outcomes.

Branching narratives are an area where AI brings something to the table that isn’t nearly as possible with humans writing out and filming every possibility.

Interactive narrative design

That brings me to Never Fear Paris, an interactive historical mystery I've been developing for the past year. Set in 1920s Paris, the story begins in a smoky nightclub with a surrealist dancer named Elodie and an artist named Marcel. You play as a character who steps into the story as a photographer, an observer of human nature, who witnesses something strange, maybe even sinister, and becomes entangled in a mystery that unfolds across the city.

Dialogue in the Dark, which you’ll find below, is a preface to that story told from today’s perspective. It’s a conversation between an archivist and a filmmaker, spoken over still images and archival footage. In the interactive version of the story, this dialogue becomes the root of a branching narrative where players can pursue multiple paths, make choices, and uncover different truths depending on their perspective and decisions.

The script for Dialogue in the Dark is designed to serve as a foundation for future interactivity, where AI will dynamically generate variations, continuations, and responses as players push deeper into the story world.

For the most part, I consider Dialogue in the Dark to be a throwaway (to use Lennon’s phrase). The larger vision I’m pursuing is Never Fear Paris, whereas I consider this Dialogue to be a slice of that larger work, a possible entry point.

It’s a throwaway also in the sense that this prelude isn’t needed for the full Never Fear Paris experience. This dialogue will serve as a testbed for generating branching narratives with AI.

Script: Dialogue in the Dark

[I’m leaving out the visual descriptions of the scene.]

Filmmaker: An empty chair in a room full of dreams and a canvas holding unfinished thoughts. Everything vanishes.

Archivist: Ah, the empty studio. A space filled with more questions than answers, as if he stepped entirely out of reality. The last place Matteo Monti was seen. You know, the art scene back then was like jazz filled with ideas and conversations.

Filmmaker: A perfect setting, and yet he left all that.

Archivist: Left? vanished? I've always thought something more sinister was behind his disappearance.

Filmmaker: This headline, it's haunting.

Archivist: Many. Some thought it was a publicity stunt. Others believed he got lost in his surreal world. Others think he simply boarded a boat back to Buenos Aires. But there's no trace of him in Argentina. I've spent years in Buenos Aires trying to find anything about him. I did find these letters he wrote from Paris to his former mentor, but they're all dated before his disappearance. These letters offer a glimpse into his mind. He often spoke of blending reality and dreams.

Filmmaker: Could his disappearance be his ultimate masterpiece?

Archivist: More intriguing are these scraps of paper found in the nightclub.

Filmmaker: Wait, what are these?

Archivist: Evidently, they’re thoughts Monti would scribble on scraps of papers at the club and then toss aside. He did that night after night until he vanished. His friend Marcel secretly collected these scraps and kept them. I think, at first, Marcel was seeking insight into Monti's creativity. Marcel was seeking insight into Monti's creativity. Then, after the disappearance, Marcel became obsessed and a bit of a detective.

Filmmaker: Yes, Marcel, I know about him.

Archivist: Then, you know...

Filmmaker: That they were lovers and enemies?

Archivist: Competitors. Everybody in that scene were lovers and enemies, competing in life, love and art.

Filmmaker: This unfinished painting, it's like an open-ended question.

Archivist: Or a clue, some say he disappeared to complete it in solitude.

Filmmaker: What happened to the painting?

Archivist: It turned up years later in a catalogue from an art gallery. Though some scholars believe it was not finished by Monti. The provenance is unclear. It got sold to a private collector who eventually got tired of people coming around asking about it. I have the collector's name here somewhere, but he started refusing all requests to see the painting or even talk about it. I've heard he even rejected some big offers to buy it.

Filmmaker: So many trails to follow.

Archivist: Answers are scarce.

Filmmaker: That's what makes it a compelling story, even after all these years.

Archivist: And that radio show...

Filmmaker: I love that series from … What's his name? Edward Thomas?

Archivist: Thompson. Edward Thompson from the BBC.

Filmmaker: He got fired from the BBC for going down this rabbit hole night after night.

Archivist: He was quite a character from what I've heard.

Filmmaker: Definitely I have all the recordings and his notebook.

Archivist: Oh, really?

Filmmaker: In the end, Matteo Monti may have achieved what all artists aspire to. Immortality through mystery.

Archivist: Or perhaps the mystery is the art itself.

Filmmaker: A life shrouded in ambiguity.

Archivist: That's everyone's life when looked from the outside, when looking back by others.

Filmmaker: We all become phantoms, phantoms of ourselves, ghosts reappearing as fragments.

Archivist: Intriguing.

Filmmaker: I view what I do as a resurrection of sorts, crafting the very specific, almost ritualistic conditions that bring my audience into an enigma.

Archivist: Will you find an ending to this film of yours?

Filmmaker: An ending? What's that? The story lives on and on.

After the dialogue is over, the user will be presented with a screen containing a set of choices that will lead the user down varying paths that will keep evolving.

The process of writing Dialogue in the Dark

Okay, so what parts of that script did I write myself? What parts were written by AI? I’ll detail all that in an upcoming post.

At the time, I laughed, appreciating his playful reply, but looking back now, his words were more insightful than he probably intended. The laptop isn't merely the ubiquitous computer so many people use everyday. The laptop has become a versatile musical instrument through software generically known as Digital Audio Workstations (DAWs) that are powered by an ecosystem built around Virtual Studio Technology (VST).

Over the past three decades, these technologies have enabled musicians (from professionals scoring films to teenagers making beats in their bedrooms) to compose without physical instruments and, often, without knowledge of music theory. This ecosystem provides a powerful model for how generative storytelling, with its emerging AI-driven narratives, might build its own open framework, empowering visual storytellers to create interactive worlds.

Setting the Scene

A new genre of storytelling is evolving. Narratives are not limited to linear, predetermined outcomes. Instead, stories can adapt to real-time user interactions and algorithmic inputs. Generative storytelling is a process where AI algorithms help shape plot twists, dialogue, visual environments, and character behaviors.

Yet, as promising as generative storytelling appears, it faces a critical challenge: the absence of an open, interoperable ecosystem. Currently, creators navigating AI-driven narratives find themselves caught between competing proprietary platforms, each attempting to dominate the landscape and control the creative tools. Without open standards, storytellers risk vendor lock-in, limiting creative flexibility and collaboration. If generative storytelling is to realize its full potential, in the same way as digital music production, we must establish common technical standards and frameworks. This essay explores how the ecosystem around DAWs and VSTs overcame similar challenges, ultimately building an enduring and thriving creative environment that could guide the way forward for generative films, animations, and interactive narratives.

What the Digital Music Ecosystem Got Right

At the heart of digital music production is a powerful but straightforward concept: software-based tools that enable musicians to record, edit, mix, and produce complete musical compositions. The central software environment for this process is called a Digital Audio Workstation (DAW).

The choice of which DAW to use is a topic that elicits as much divergent opinion as politics and religion. Personally, I use FL Studio. My choice is not based on anything other than I really find the interface of FL Studio to be crisp and appealing. (Some other DAWs have UIs that feel very dated and give me eyestrain.) FL Studio’s UI is a place where I want to spend time.

I do most of my music sessions at night, when I’m relaxing after a day of coding and thinking about AI. As much as I personally like FL Studio’s interface, I’m sure there are others who hate it or dislike the workflow that FL Studio forces you into utilizing. But this essay is about DAW as software and not about which DAW to choose for yourself.

Earlier this year in my Responsible AI class, I raised the topic of DAWs. Only one student had heard of a DAW. It’s easy to forget that vibrant technologies exist that most people have never heard about.

DAWs have the capabilities of supporting specialized plug-ins known as Virtual Studio Technology (VST). VSTs are software-based musical instruments and audio effects (virtual pianos, synthesizers, drum machines, reverbs, compressors) that integrate with a DAW.

An excellent research-oriented overview of music production system, specifically the development of VSTs, is the 2025 paper: B. Andrei, M. V. Merino and I. Malavolta, "The Ecosystem of Open-Source Music Production Software – A Mining Study on the Development Practices of VST Plugins on GitHub"

Crucially, a VST operates as a standard developed by the Steinberg company in 1996, enabling interoperability: a VST created by one vendor can function inside any compatible DAW, regardless of who built it. (Steinberg has been a huge force in digital music production since the 1980s.)

Most VSTs are coded using JUCE, which is an open source application framework in C++. (For simplicity, I’m focusing on VST but there are other audio plugin formats.) This open plugin architecture means creators are not locked into a single vendor's ecosystem, fostering innovation, competition, and creative freedom.

But it’s not all roses and unicorns. A lot of virtual instruments are designed specifically for the Kontakt player, which is a product from the company Native Instruments. That company also has a very expensive suite of instruments and effects that are bundled into a set known as Komplete.

To understand what a thriving ecosystem might look like for generative storytelling, it's helpful to examine one of digital music’s most successful ecosystems: Native Instruments' Kontakt and its companion bundles known as Komplete. Kontakt, at its core, functions as a software environment within a DAW that manages a library of virtual instruments. Each virtual instrument replicates the sound and behaviors of real-world instruments such as pianos, violins, synthesizers, and drum kits, or creates entirely new sounds.

The advantage of Kontakt is its openness: third-party developers can build and distribute high-quality instruments that function seamlessly within the Kontakt player. For example, the acclaimed Noire piano, created by Galaxy Instruments in collaboration with Native Instruments, offers nuanced, expressive sound shaping controls accessible directly within Kontakt. Such collaborations allow small, specialized teams to leverage Kontakt’s infrastructure to reach audiences far beyond their individual capacities.

The Noire piano (retails for $149) sounds wonderful. Follow this link to hear the sounds and view some cool videos of how Noire was developed in collaboration with Nils Frahm: https://www.native-instruments.com/en/products/komplete/keys/noire

Native Instruments business models profits from this ecosystem through Komplete bundles: curated packages that group Kontakt-based instruments, effects, and sound libraries into cohesive collections. These are pricey. (Tip: Native Instrument regularly offers sales with deep discounts.)

Komplete serves as both entry point and creative powerhouse, attracting users ranging from hobbyist beat-makers to award-winning film composers. This carefully balanced ecosystem demonstrates how an open, standardized architecture combined with curated content can enable diverse creators to flourish and scale their creative ambitions far beyond the limits of a single vendor’s vision or toolset. Native Instruments also has a set of hardware that integrates with their musical plugins.

Just as I’m writing this post, Native Instruments released an 80-minute video on how to build an instrument for their Kontakt player.

Native Instruments is not without its share of people who detest the company, as any quick browsing of Reddit will reveal. There are many companies providing VSTs. Spitfire Audio is another that I often use. There’s also EastWest Sounds and an increasing number of subscription-based services like Splice. Plus, there are smaller sites like Mondo Loops offering sample packs, presets, and plugins.

Making music from samples is a small hobby of mine but I’ve already spent a significant amount of money on software, sound libraries, and gear. And that’s the point of this essay: there are business models that are built around creativity, business models that are built around offering tools that enhances one’s own creativity.

But, if you’re not familiar with this musical environment, you might be asking: is that really creativity? Here’s a take on opening up to the world of music composition from Guy Michelmore, who has one of the most informative and entertaining channels on music education. (The clip starts at the 20:30 mark. Watch the full video if you want a deeper overview of the latest editions of Komplete and Kontakt):

The reality of working with any of these tools by Native Instruments (or by competitors like EastWest Sounds) is that the learning curve is still significant compared to the ease of generating images and video with AI. With these enhanced tools in VSTs, you can create some pretty sounds with one finger. But it’s a very different effort to pull together a 4-minute song with layered instruments. Likewise, in Runway, you can easily create a 4-second video that is pretty cool. But pulling together a 4-minute film that conveys a story is a different level of creativity and self-expression.

At the 22:55 time mark in the above video, Guy Michelmore explains,

“There’s a continuum between originality and self-expression on one-side and painting-by-numbers on the other. And presets are definitely on the painting-by-numbers side. If you really want self-expression and originality, it comes from understanding what you’re playing…It’s the difference between going to a restaurant and making your own food…It’s not a path to self-expression and originality.”

Patterns & Techniques

The success of DAWs and VSTs is grounded on musical principles: scales, chord progressions, rhythmic patterns, and song structures. These foundational elements provide a universal language that plugins and DAWs can reliably build upon.

Similarly, visual storytelling and interactive narratives also rely on foundational narrative structures though stories lack the mathematical precision of music: the hero’s journey, three-act structure. Could these patterns serve as the universal frameworks necessary to establish an interoperable generative storytelling ecosystem? By learning how digital music harnessed these shared standards, we can better understand the potential for generative storytelling to similarly flourish.

“Listen I know it must sound absurd”

So, what does digital music production built on DAWs, VST plugins, open standards, and curated ecosystems offer as a lesson for generative storytelling? It demonstrates clearly how an open yet structured creative environment empowers innovation. In music, even dominant vendors like Steinberg and Native Instruments operate within ecosystems shaped by interoperability and agreed-upon standards. This openness has given creators at every skill level the ability to experiment freely, to mix and match tools, and to scale from casual experimentation to professional mastery.

Generative storytelling, whether through interactive films, animations, or branching narrative games, needs a similar foundation: technical standards to ensure compatibility, open marketplaces that encourage diversity of content, and flexible authoring environments that offer control without restricting creative possibilities. Imagine storytelling plugins as intuitive and interoperable as Kontakt instruments, AI tools as expressive as Noire’s generative note engine, and creative environments as engaging as your favorite DAW interface. If we get this right, creators won't just tell stories: they’ll orchestrate entire narrative experiences, shaping the future of storytelling itself.

“Listen I know it must sound absurd” is a line from this song recorded in 1984. Here’s a clip from that Nick Cave show in 2017 in Brooklyn.

I’m not visible but I’m in the crowd just beyond Nick’s right leg. I can tell you that the fabric of his pants is very nice.

There’s a tweet going around that we’re closer to 2050 than to the year 2000. That means nothing to my 14 y.o. kid. It signifies everything to me. Twenty-five years ago, I had moved to Miami Beach and started a great job at the University of Miami. I always refer to those years as the prime of my life (though I know I’m lying to myself).

What about 25 years from now? Will I even still be alive in my mid-80s? I don’t worry about that. I have never worried about dying. Likely that’s because my father died suddenly when he was 37. Since my childhood, I’ve always known that death could come at any moment.

I use ChatGPT throughout the day. (Shout out to Gemini Deep Research. If you’re not using Gemini for deep research, then you’re missing out.) I use ChatGPT more often in a day than I have ever used a search engine or almost any social media site. I must admit that I’m big consumer of YouTube video. I love YouTube! There’s something extremely relevant about Google that their products that are core to my life: gmail, Gemini Deep Research, YouTube, NotebookLM.

Yet, despite my use of Google products, I feel locked into the OpenAI ecosystem for most of my chats due to setting up projects in ChatGPT and that it has memory of the essential work I’m developing.

I usually have a tab in my browser opened to ChatGPT where I can jump to any moment for an insight or for help in thinking through an issue or for a quick overview of a topic or to answer a question.

The tech giants have cornered the market on generalized use of AI.

In the last few weeks two essays have caught my attention. The first is Sam Altman’s The Gentle Singularity . The second essay, published earlier in 2025, is Stop treating ‘AGI’ as the north-star goal of AI research by a team of 16 AI researchers. (The primary authors are Borhane Blili-Hamelin and Christopher Graziul.) I recommend that you read both essays in light of each other since they represent fundamentally different philosophies about the future of AI.

Over the last year, I often told the students in my college courses that technology seeps into our lives. As Altman casually points out, “In some big sense, ChatGPT is already more powerful than any human who has ever lived.” We now take for granted that we have multiple systems (ChatGPT, Gemini, Claude, Perplexity, etc.) that are more powerful machines than anything we’ve ever encountered. Just a few years ago, those capabilities would have been doubted by most and shocking to almost everyone. Now, we just shrug our shoulders.

If you’re paying attention to AI developments, you’ll know 2025 is the year of AI agents, though the term AI agent has been so corrupted by marketing speak. A recent video with Andrew Ng has him stressing that we should stop dissecting the term AI agent and refer to agentic systems. It’s an excellent video. Watch it if you’re interested in agents, uh, agentic AI. Even if you’re not interested in agentic AI, you should watch the video. Andrew Ng always has insights.

Lots of companies are working on robots. I’m trying not to think about robots. I admit that I’m not ready for humanoid robots. But when I look at any parking lot and see the amount of money that Americans spend on their cars, then (well, hell yeah) Americans will be buying humanoid robots (on credit). There’s likely to be robot dealerships (like car dealers) showcasing all the latest models. But these robot retailers will be more snazzy like the Apple Store than the Chevrolet dealer. (Inevitably, some dude will wonder if his humanoid robot could drive his Ford F-150 someday; there’s an odd type of American who might be more comfortable with a humanoid robot driving a truck than with self-driving vehicles. Sigh.)

The 2030s

Sam Altman gives us a reassuring truth: “In the most important ways, the 2030s may not be wildly different. People will still love their families, express their creativity, play games, and swim in lakes.”

I avoid the word singularity. It means nothing to me, and means nothing to most people. It’s jargon.

Of course, the 2030s will be different from the past, just as the 1930s were different from the preceding decades. It’s easy to see change in hindsight, but very difficult when you’re leaving through it.

The concern is that the next decade will be radically different than anything we have known. We’ve been here before. In the 1990s, those of us working deeply on the Internet and the emerging Web knew that huge changes were coming, though no one knew quite what shape would emerge in the 2000s.

Jobs did disappear. New types of jobs were created. Will that pattern hold? I’m not so sure.

Sam Altman:

There will be very hard parts like whole classes of jobs going away, but on the other hand the world will be getting so much richer so quickly that we’ll be able to seriously entertain new policy ideas we never could before.

The word abundance is getting tossed around a lot. I’ve recently started listening to the Ezra Klein book of that same word.

Part of me loves Altman’s vision of the gentle singularity. But here are some essential questions that we must consider:

• Who guides abundant intelligence toward meaningful ends?

• How do we ensure powerful cognitive tools align with genuine human flourishing?

• In what ways might these tools subtly diminish critical human faculties such as reflection and deliberate thought?

These questions directly shape my current collaborative effort with a business partner. (More on that in a future post.) Together, we envision AI as a thought partner designed not to shortcut thinking, but to encourage slow, reflective inquiry. We believe that how we think shapes who we become.

AI Safety & Alignment

AI safety gets a disparaging reaction from some people on X. Admittedly, some proponents of AI are so overly cautious. And then there are the anti-AI folks who pop up in any conversation about AI art. Altman gives AI safety some credence, “We do need to solve the safety issues, technically and societally”. Let’s look at the essay, Stop treating ‘AGI’ as the north-star goal of AI research. (See earlier link above.)

Is the focus on AGI not only counterproductive but also dangerous?

The illusion of consensus. The terms AGI and superintelligence create a false sense (among the uninformed public) that everyone in the AI industry agrees on what society should be trying to accomplish with AI. Of course, if you’re involved with AI, then you know there’s little agreement on even what those terms mean other than an AI that is more capable in every domain than any human. That’s a really broad statement.

Specificity over Generality

We need more AI-enabled specialized tools that have clear, measurable, and context-aware goals. The future of AI cannot be built in silos controlled by a very small number of corporations. The greatest risk of a single "north-star" goal is the homogenization of research and the consolidation of power. OpenAI, Anthropic, and others play a very important role in the foundation of what we do with AI. In many cases, they will provide the optimal experience for the majority of people. In many other specialized cases, smaller players will focus more deeply on providing a better experience. How do we reconcile visionary optimism for rapid technological progress with disciplined caution to ensure AI genuinely enhances human lives? That requires a commitment to open standards and decentralized architectures that ensures domain experts are not just passive recipients but active participants in shaping how AI is integrated into our digital lives.

The divergent viewpoints table was created with the assistance of Gemini 2.5 Pro. Formatting was created with the assistance of Datawrapper.

The next time you find yourself talking about AGI, take a breath. Think about a very specific use case of AI that does not yet exist. Think about how you would measure how well AI accomplishes that goal. Find a specific use case of AI that really energizes you. Or, identify an aspect of society or our lives that you want to be sure does not get lost as AI advances. Those are the things you work on. That’s how you craft a responsible path to AI.

If you’re working on something you’ve carried too long, I see you. I’m right there, too.

Before moving onto my project about AI and interactive storytelling, I need to finish the novel that I started 20 years ago.

A couple of years ago, my teen-age daughter asked me, “How long have you been writing that novel?”

I replied, “18 years.”

She remarked, “Your novel is now an adult.”

It’s approaching 20 years. My kid will occasionally ask me, “How’s your adult novel going?”

Of course, there were many months, which turned into years, where I never touched the novel. Most of the writing happened in 2009 when I would stay up all night writing a scene, night after night. It often took about half-an-hour before I got into a trance-like writing flow. But, still, it would take hours to write 1,500 words. By that point, I was exhausted but I considered it a good writing session.

The work originated in 2005 after I had quit a six-figure job in Miami and moved to Buenos Aires without a plan. That was the best decision of my life.

A few months after settling into Buenos Aires, which turned into 8 years of my life, I decided to write a novel. Why not? But I didn’t know how to write fiction. I should have started with a short story. Later, I stopped the novel and wrote some flash fiction and a short story that did get published (though the online magazine is no longer online).

Returning to the page

I’m finally finishing it, over 70,000 words that I wrote entirely on my own. All those years ago, this story about forgery emerged within me.

The novel is not only about forgery but about creativity and the value of art. These themes have lived with me for 20 years, and have become even more meaningful now that I am immersed in the impact of artificial intelligence on creativity and art.

Letting Go

Editing a novel this old is emotional. I’m doing something unexpected: I’m letting parts of it go.

There were entire scenes I loved that I had to cut to make the story stronger. I had written over 100,000 words. Letting go is one of the hardest parts of editing. But it’s also where clarity begins.

A big mistake was telling this story from multiple points-of-view. In the editing, I eliminated the point of view of a major character. But the absence of his perspective makes his presence on the page stronger.

That POV of a man named Federico was originally the main character, and he still plays a critical role but we only see him from the alternating POV of two others: Andrea and Daniel.

Letting go of Federico’s POV unlocked the entire story. In rereading, I realized that I had almost finished this novel years ago.

I want to note the character Daniel, who is the forger in the story. His character did not even exist in the first couple of drafts. Then he emerged in one scene, and then in another. Slowly, I discovered that the novel is his story. I should rewrite the novel entirely from only his POV. But, after 20 years, I don’t have the energy for that. I need to finish this story.

Listen to your own story

I also listened to the entire novel using ElevenLabs. I used the voice of “Eleanor” in ElevenLabs. When I release the audiobook, I’m not sure if I’ll use that voice or not. At this stage, I’m only thinking about how I’m hearing a voice read the text to me.

Hearing the story out loud changed how I understood it. I could feel the rhythm. I could hear where a phrase did not work or where a line surprised me in a good way.

A Research Lab for Narrative Transmedia

An outcome of this novel is that I will have a completed work for which I own the copyright and can use it in anyway I want. After the print version is published, I’ll be using the text of Forgery Lessons as a foundation for exploring variations in the story through AI. I also will be benchmarking how well AI handles the editing of fiction.

Plus, there’s a tie-in to my interactive story Never Fear Paris, which is set in the 1920s. The central mystery of Never Fear Paris is the disappearance of an artist. That missing artist turns out to be the father of Federico in Forgery Lessons.

Will I write another novel? I don’t know. Maybe. But I’m more pulled towards the questions of how we tell and read stories through visuals, whether it’s 2d or 3d graphics, VR, open-world games, and examining where AI enables each person to experience a unique narrative.

In so many aspects of my life, like retiring from academia, I’m letting go of an old version of myself so I can build something new. You’re never too old to re-invent yourself.

I started writing this post in one of the many picturesque coffee shops in Saigon. I had journeyed to that city for the celebrations marking the 50th anniversary of the end of the Vietnam War. The trip also marked an ending for me as well: retiring from academia. But I hate the word retire, especially at my age (59). I’m leaving academia because there are questions I want to explore that do not fit into the role I had as a librarian and as non-tenure track faculty.

First, there’s the core question that I have realized is my life’s work. This question has followed me for nearly 25 years since I first recognized it as a thread that runs through what I attempt to do with my work:

Understanding how we convey and experience stories, both fictional and factual, through digital media. The future of my child’s life is the lens through which I view the changing shape of creativity, learning, and leisure in the twenty-first century.

Now, I can devote myself to that full-time. But what does that mean in practice?

I use the word practice deliberately. Now that my time is no longer structured by the rhythm of semesters and courses or any type of a regular job, I have the freedom to structure my own time. But that is so deceptive, so tantalizingly easy to drift into doing almost nothing. (With my mental health issues, that’s a real concern.) I need structure but am very glad not to have someone else impose structure on me.

The first structure I must impose on myself is health. Without my health, my retirement will be very short if I sit on the sofa, watch Netflix, and experience my body rapidly declining. That’s not an option. I have an exercise routine and keep a daily fitness log in a notebook. I’m not always as consistent as I need to be each day. But I’m working on the challenge, which is really a matter of commitment, of committing to start specific exercises each day. That commitment is a mental process. (There’s also the matter of nutrition. I know what I need to eat and not eat. It’s a matter of committing to an enjoyable pattern of eating healthy, natural foods.)

As for my mental health: I have a practice of meditation and journaling. My mindfulness practice over the last six years has saved my life.

The value of meaningful work

I view the work that I have chosen for myself as meaningful. If I had done what I needed to do earlier in my life, like pursuing a PhD and obtaining a tenured faculty position, then I probably would not be retiring from academia. But I didn’t make those choices. Now, I am embarking on my own path of study, research, and development.

I had a lot of time to think about this on my recent journey:

• 10-hour flight from Washington, DC to Istanbul

• 19-hour layover in Istanbul

• 10-days in Saigon

• Got rerouted by Turkish Airlines to a flight on ANA

  • 6-hours Saigon to Tokyo

  • 3-hour layover at Tokyo-Haneda

  • 12-hours Tokyo to Washington, DC (Dulles)

The most annoying part of the entire trip was the slow-moving immigration line for US citizens at Dulles and the 3+ hour drive back to Lexington, Virginia.

Without an institutional affiliation, I fall into the category of independent researcher. I have these projects, these interests of mine, that all overlap to some extent on the concept of immersive narratives. (Let’s set aside for a moment that all well constructed narratives are immersive.) Considering my skills in software engineering, I would not be fair to myself or others if I focused only on theory.

Rather than wander aimlessly from topic to topic, I’m forming a research agenda and methodology that grounds my creative and scholarly efforts, possibly for the next 25 years. (At my age, I strongly embrace the notion that I have practically a lifetime of intellectual productivity ahead of me.)

I’m anchoring everything in an interactive story/game that I’ve titled Never Fear Paris, not as a side project but as the embodied research instrument for my efforts. This move grounds my inquiry in material practice. Never Fear Paris becomes the lab where my theories are tested, refined, and made public. (See my post on LLMs in interactive mysteries.)

Never Fear Paris is an interactive mystery set in 1920s Paris. You play the role of a photographer caught up in the disappearance of a surrealist artist. As the player-photographer you uncover clues by exploring the streets, nightclubs, and art galleries of Paris. You interact with non-player characters (NPCs) to piece together the story of the missing artist.

In my next post, I’ll further explore Never Fear Paris as a research lab.

The major AI tools have broad capabilities for implementing chatbots through APIs. ElevenLabs has nearly perfected the product delivery of voice-oriented AI tools.

Voice interactions with our devices are becoming normalized, but it’s like talking on the phone or participating in interactive radio. Over the last year, we’ve been hearing a lot of talk about the imminent rise of AGI that surpass humans capabilities in every domain. But is the AGI revolution faceless?

Throughout the past six months, we’ve started hearing more talk about humanoid robots from Tesla, Nvidia, and a range of venture-backed startups. Most of the talk is marketing spin telling us that every home will have a humanoid assistant to do the chores. Why do those devices need to be humanoid? They don’t. We already have machines automating many of the chores. But these companies have to sell us something that’s really expensive, nearly the price of a car. I have no doubt that, over the next decade, that humanoid robots will be technically feasible. But the big barrier to these robots will not be technical but regulatory. While AI is seemingly getting a pass on regulation for the next few years, robots are tangible products. The idea of a rogue robot suddenly physically harming young children is no one’s idea of an acceptable risk.

Meanwhile, developments continue in 3D graphics. Let’s talk about the digital human technology stack. I’m not talking about robotics but extended reality and virtual reality. It remains to be seen whether there will be a platform shift from smart phones, desktops, and laptop computing and toward broad adoption of headsets and advanced eyeglasses with augmented reality capabilities. Even if extended reality does not become the dominant paradigm in 5 years, though I think there’s a high likelihood of that possibility, realistic avatars will certainly be a part of many people’s lives, even if it’s only for gaming, media and entertainment.

What’s the essential technology stack required to create lifelike, interactive digital humans with conversational AI capabilities?

For the open-world interactive stories in VR that we want to develop, we need a structured framework that identifies core components for helping us think through the challenges for implementation and the possible innovations needed to bring these experiences to reality.

Start from the user perception and traverse down to the backend infrastructure.

Perceptual. Everything leads to this layer. We know it as the uncanny valley. If we don’t perceive digital humans looking and acting with natural facial expressions, realistic body movements, and clear speech, then the illusion doesn’t exist. (Let’s set aside, for a moment, that there will be many scenarios where we don’t want characters that are truly human-looking. Maybe the story calls for cartoon-like people or even animals.)

The Conversational layer brings us believable dialogue. Here is where a lot of progress has been made in the last year. We have the AI-enabled tools to understand what we say. We are increasingly getting tools that respond naturally with realistic speech and a bit of emotion. The ability of AI to adapt conversations by recognizing and expressing empathy is a matter of finetuning that will rapidly advance.

The Context Awareness layer remembers previous conversations, recognizes our preferences, and understands our virtual environment to respond naturally over time.

The Multimodal layer is about integration: combining speech, facial expressions, gestures, and emotional cues into unified, context-aware interactions.

The Infrastructure layer supports all these interactions by providing the cloud and edge computing resources, ensures privacy and ethical handling of user data, and continuously optimizes digital human interactions through performance monitoring and adaptive learning.

As you think about these technologies, keep the future in mind. A decade is not that far away. I know that’s hard to comprehend for many people, especially if you’re younger than 30. But in 10 - 15 years, you will understand that. The progress of time is why you want to work on problems that may take 5 or 10 years or more.

These key technologies all have interdependencies. The mechanics of specific technical implementations might change but establishing the concepts help us document, categorize, and analyze state-of-the art advancements and future developmental trajectories.

What is the big question, for you, that keeps coming back year after year?

Artificial intelligence is only a disruption to your work if you have not realized what your life’s work is all about.

There are, have been, and will be, far greater disruptions and transitions in your life than AI.

Death of those we love, major illnesses, and other calamities send a shock to our system. Yet, we must find a way to keep going.

The transitory milestones of life: graduating into adulthood, relationships, failure after failure until we learn who we are as our own person, alone and afraid in this world until we muster the confidence that binds us to others in friendship and love.

Our responsibility to the world shines through our children, nieces, nephews, students, and every young face we encounter.

Friends will come and go, jobs will come and go, politicians will come and go, but we are still here. Every day offers the potential for hundreds of beautiful moments, one after another.

As a college professor, I’m surrounded by hundreds of young people embarking on their careers. Few, I suspect, have a developed a notion of what drives their work through life. I certainly did not at age 22 or at age 32.

I encounter many students whose life’s work seemingly appears to be to help people with more money than themselves to make more money. After all, we all must find a a way to support ourselves and our families.

We may have careers that bring us satisfaction in many forms. Only in my forties, while living in Buenos Aires, after having a child, and thinking about the age of my child at the midpoint of this century, did I recognize a common thread that ran through every aspect of my life. That day, I sat in a café and scribbled thoughts into my notebook. Over the last 13 years, I revisit these words—with a tweak here and there—as a touchstone as to why I do what I do.

Fundamental to my professional interests—as a professor, as a software developer, as a writer, as a reader—is understanding how we tell and read stories (both fiction and non-fiction) in digital media. The future of my child’s life is the lens through which I view the changing shape of creativity, learning, and leisure in the twenty-first century.

At age 59, as I retire from this university, I’m entering another transitory phrase of life where I go away from these comfortable surroundings that have nurtured (and traumatized) me over the years. By going away from this place, my life’s work crystallizes into guiding thoughts and actions that I will pursue for the rest of my life.

I am healthy today and hope for thirty more years of intellectual engagement. We are never guaranteed time. I know that as well as anyone. My father died in a car wreck when he was 37. Unless I succumb to an accident in the next twenty years, the probabilities are high that I will have more time than my father had in his entire adult life.

Waste not a moment of life.

I could go sit on a beach in Thailand and never work again. But, the enjoyment of life and work are not mutually exclusive.

I am simply one person working on a problem. Reminding yourself that you are one person working on a challenging problem provides a frame for your endeavors year after year.

As artificial intelligence decimates what we have known as knowledge work, we must recover our individuality. Our contributions to society will be augmented by AI, our capacity to accomplish more will be expanded by AI. Do not race to keep track of every new development in AI. Do not fear missing out. What you are seeking is within yourself. Let the questions of your life emerge. Build your work around your unstoppable ideas.

In the words of Theodore Roethke:

That’s an excerpt from LeCun’s talk to the UN Security Council. This 8-minute talk (starting at the 12-minute mark in the video), linked in the description, is packed with insights. LeCun is not some fringe scientist. He’s a leading figure in AI research. When he makes a prediction like this, it’s worth paying attention.

What’s striking is that LeCun comes right out and says that this need not be a frightening scenario.

“By amplifying human intelligence, AI may bring not just a new industrial revolution, but a new renaissance, a new period of enlightenment for humanity.”

Training and fine-tuning

Let's break down a core concept in machine learning that is usually not realized by the average person.

“AI systems are produced in two phases. One is training a foundation model, and the second phase is fine-tuning it for a particular application.”

The distinction between training and fine-tuning is very important. My data for deep learning class is focused on the training phase. My responsible AI class is focused on the applications that arise from fine-tuning foundation models for specific uses. I’ll go further into the distinction between training and fine-tuning in another post.

LeCun gets specific about the steps that are needed for ensuring that the benefits of AI are shared equitably across the globe.

“Foundation models must be trained on all the world's cultural materials in all languages if we want them to be accessible and useful to everyone around the world.

Since all of our digital diet will eventually be mediated by AI systems, fine-tuned systems need to be numerous, diverse, to represent all cultures and value systems around the world.

Two conditions are necessary for this to happen. Foundation models must be free and open source. And second, training must be performed in a collaborative and distributed fashion in multiple data centers around the world.”

Let’s repeat that. Make sure you understand these points:

• Foundation models must be trained on all the world's cultural materials in all languages.

• Fine-tuned systems, i.e., AI-enabled apps, must represent all cultures and value systems around the world

Two necessary conditions to accomplish global AI development:

• Foundation models must be free and open source

• Training must be collaborative, distributed across multiple data centers around the world.

This is going to require global collaboration among governments and the private sector.

“Governments and the private sector must work together to ensure this global network of infrastructure exists to support the development of AI, enabling people all over the world to participate in the creation of a common resource.”

LeCun is a strong advocate for open source foundation models. We have seen how successful the Internet and the Web have been over the last 35 years because of open source developments. If you are new to the concept of open source, be sure that you understand it.

“The future of AI is inevitably one in which free and open source foundation model will dominate. History shows that infrastructure software platforms always end up being open source. For example, the software infrastructure of the internet and the mobile communication networks are entirely open source.”

Two extraordinary examples of how open source collaboration have advanced global development through infrastructure are the work of the IETF and the W3C. Plus, there are all sorts of standards developed in collaboration by organizations throughout the world that we take for granted. The IETF is the Internet Engineering Task Force that defines the protocols that enable the Internet to function. The W3C, the World Wide Web Consortium, “develops standards and guidelines to help everyone build a web based on the principles of accessibility, internationalization, privacy and security”. These organizations have been operational for decades and produce work that is instrumental in our daily lives. Yet, very few people know about these groups. The remarkable thing is that anyone can join and participate in the process to have your voice heard.

LeCun stresses that AI cannot reflect only the values and perspectives of a small part of the planet. We all must advocate for training foundation models on data that represents the full spectrum of human languages and cultures.

A key aspect is regulation, which may be more challenging than any technical development.

“unifying the regulatory landscape so that the development and deployment of open source foundation model is not hindered.”

Think about the potential economic and social benefits of making AI accessible to everyone. We need some kind of global consensus on how to govern these powerful technologies. It's about finding that balance between encouraging progress and safeguarding our values.

LeCun's timeline that AI will surpass human intelligence within twenty years really drives home the point that we need to be proactive in shaping this future. And some leaders in the AI industry are saying this will happen within the next few years, though I think that’s mostly hype to drive up the valuation of their companies.

LeCun's talk underscores a critical message. The future of AI is not predetermined. It's being shaped right now by the choices we make, the projects we pursue, and the values we prioritize. There's a real sense of urgency about using AI to make a positive impact on the world.

Safety

But what about the risks? We hear a lot about the potential dangers of AI. Lecun addresses it head on.

“Foundation models must go through rigorous testing and red teaming.”

Red teaming is a process, usually internal to a company, where they try to identify vulnerabilities through prompting.

You might be wondering: couldn’t someone with malicious intent just ignore those safety checks and develop AI for harmful purposes anyway?

Yes, I think so. We need to figure out methods for combating that. But LeCun argues that open source platforms with their transparency and the scrutiny of the global community tend to be more secure.

You also might be wondering: if that's really going to happen within the next decade or two, shouldn't we be focusing all our energy on figuring out how to control something that could potentially be more intelligent than us?

Those are valid concerns and LeCun doesn't shy away from them. He believes that we can develop safety guardrails to guide their behavior and prevent harm.

“Those super intelligence systems will do our bidding and remain under our control. They will accomplish tasks that we give them, subject to safety guardrails. Guardrails will shape their behavior similarly to how inviolable laws would shape human behavior.”

Of course, even with societal norms and international law, we still see genocide and horrible acts of violence. But, with AI, we must do as much as we can with safety. It’s naïve to think that AI is not here to stay.

And this is where your interest in responsible AI becomes absolutely crucial. As we develop these incredibly powerful AI systems, we need to be thinking deeply about the ethical implications, the potential risks and the safeguards we need to put in place. This isn't just some abstract philosophical debate. It's a field that will be in high demand as AI continues to advance.

Our work & lives

LeCun paints a picture where we'll each have a team of AI assistants working for us.

I’m a former librarian. For several decades, we have envisioned the Internet as the universal repository of all human knowledge. But what does it mean for global development when you can have conversations with an expert on any topic imaginable, in any language?

The dissemination of knowledge

LeCun's comparison of AI's impact to the invention of the printing press.

“It is often said that AI is enabling the next industrial revolution. I think the effect of AI on society may be more akin to the invention of the printing press and the wide dissemination of knowledge through printed material.”

Librarians, since the early 1990s, have been comparing the Internet to the invention of the printing press. Over the last 35 years, most scholarly information is now available online, though much of it still exists behind paywalls. The publishing process also has changed during this period. All books are designed mostly with Adobe InDesign software and produced as PDFs for printing, even those that are sent to high-end printing facilities for hardcover and paperback books. That has enabled a boom in self-publishing, though the quality of content varies greatly. More importantly, the digital process of book production means that all books published in the last few decades exist as ebooks in one form or another.

Libraries also have been digitizing archival materials that only existed as handwritten manuscripts and as documents never published. Likewise, in the early 2000s, Google embarked on a massive digitizing project in cooperation with major libraries to scan more than 40 million books.

“By amplifying human intelligence, AI may bring not just a new industrial revolution, but a new renaissance, a new period of enlightenment for humanity.”

But to get there, we need to adjust the challenges of global collaboration, ethical development, and ensuring that the benefits of AI reach everyone, not just a privileged few.

Call to action

Let's talk about what this means for you, someone who's just beginning to explore AI.

There will be a huge demand for people who can bridge the gap between the technical and the social, ensuring that AI is developed and deployed in ways that are both sustainable and equitable.

AI is complex and moving fast. It can feel overwhelming. But the key is to break it down and focus on what truly excites you.

• What aspects of AI resonate most strongly with you?

• Are you drawn to the technical challenges of developing safe and reliable AI systems?

• Are you more interested in the ethical and societal implications of these technologies?

How do we ensure that AI, even at a super intelligent level, remains aligned with human values? How do we build in safeguards to prevent unintended consequences? These are some of the biggest challenges facing us today. And they're challenges that you have the opportunity to tackle.

Remember, you don't have to figure it all out today. AI is constantly evolving and there are many ways to get involved. It's more about finding your niche within this evolving landscape.

Actually, I’ve never completed this novel. But I’m getting back into it. I have a first draft, second draft, third draft. I’ve lost count. I need to finish the damn thing. I started it in 2005, not long after I moved to Buenos Aires, where I lived for 8 years and where I might live again. Buenos Aires became my adopted home town. I’ve written over 270 pages. This draft has just over 80,000 words. It needs expanding. It needs finishing touches. I’ve put that off so long. But listening to this podcast review of my uncompleted novel is energizing.

I need to stress that I’ve written every word of this novel by hand and typed every word. Writing fiction is a sheer pleasure. As much as I value artificial intelligence, I have not and will not use AI to write a single sentence of this novel. I will use AI as a tool to help me think about the novel. I have used AI as a reader of the novel to help me think through the story. I have a video on that on my channel.

A brief overview of the novel before listening to the podcast:

The novel follows the interconnected lives of several characters in Buenos Aires. The narrative focuses on the art world, specifically the forgery of paintings, and explores themes of desire, deception, and the pursuit of beauty. Daniel, a talented artist, becomes entangled with Mirtha, a ruthless art dealer, and Federico, an enigmatic teacher, both of whom manipulate people for their own gain. Andrea, an art conservator at a local museum, is caught in the middle of their schemes and struggles with her own moral compass as she witnesses the consequences of their actions. The story unfolds as these characters navigate the complexities of love, betrayal, and the pursuit of personal ambition in the captivating and chaotic setting of Buenos Aires.

The podcast does a good job of conveying the essence of the story. Caution: there are spoilers in the podcast, though they don’t give everything away. But they do reveal a lot. Indeed, that’s why I never read book reviews before reading a novel. The reviews always give away too much.

The images in this video are all generated in Midjourney. I’ve taken hundreds of photographs of Buenos Aires, which I have posted online over the years. I’m sure my own photos were used to train Midjourney. I’m okay with that.

Here’s the podcast. There are two speakers. You can listen to the podcast on my YouTube channel.

Welcome back everyone. Ready for another deep dive? This time we're going to explore the world of art forgery.

Oh yeah. Definitely not your typical art heist story.

Nope, not at all. It's a, it's a novel called Lessons Learned Through Forgery. And it takes place in Buenos Aires, which has this whole other layer.

So setting is so important. So, uh, are you familiar with the book?

Oh yeah, absolutely. We both read it before we got together today. I was immediately drawn to the characters, especially Daniel. He's this queerly talented artist. I think a lot of us can relate to that struggle of, you know, trying to make a living from your art.

Definitely. And Mirtha, the gallery owner, she's a force of nature. I mean, she's charming and sophisticated, but there's also this darkness to her.

Yeah. She's definitely got a secret or two. And then there's Andrea, the art historian. She's trying to advocate this world where the lines between art and commerce are constantly blurred.

Right. And she's pulled in so many different directions. By Daniel, by Mirtha, and then there's this whole other character.

Oh, Federico. He's the mysterious artist who becomes Andrea's mentor.

He's, uh, he's definitely got his own agenda. But one thing that struck me about the novel is how it uses forgery as a metaphor for so much more than just, you know, copying a painting.

Exactly. It makes you question what authenticity even means, not just in art, but in life in general.

It's like, Daniel, he's so good at what he does. He can basically reproduce any masterpiece, but he's also struggling to survive, to make ends meet. So when Mirtha presents him with this opportunity, he's tempted.

I mean, who wouldn't be

right? And she's very persuasive. She knows exactly how to manipulate him, how to appeal to his ambition and his desperation. And the way she targets lesser known works for the forgery market.

It's smart, less risk of getting caught.

Exactly. But the novel doesn't shy away from the moral implications.

Now it definitely makes you think about the ethical dilemmas involved. Like where does ambition cross the line into deceit?

It's a question we see Andrea wrestling with throughout the book. She's drawn to both Daniel and Federico, but she also sees the damage they're causing. She's constantly questioning her own moral compass.

Her relationship with Mirtha is particularly interesting. Andrea used to work at the gallery, but she left because she felt like the museum world was more pure. But when they meet for lunch, you could feel the tension between them.

Especially when Mirtha starts asking about the provenance of a particular painting.

It's like a power play. And it reveals a lot about both of their characters. Like Mirtha, she's still trying to control Andrea, even though Andrea supposedly moved on.

And Andrea, she's trying to hold on to her principles, but it's hard when she's constantly surrounded by these morally ambiguous characters.

And then there's the setting of Buenos Aires. It's almost like another character in the story.

Oh, absolutely. The descriptions of the bustling art scene, the hidden workshops, the evocative streets, it all adds to the atmosphere.

And there are all these little details that become symbolic as you read, like that novias sign near Federico's building. At first it seems random, but as you get to know his character.

His past relationships, his almost predatory nature.

It takes on this whole other meaning. It's like a symbol of the fleeting, almost transactional nature of love and desire in this world.

The way the novel is structured also adds to the suspense. We keep getting these glimpses into the characters past their motivations, like Daniel's experiment living on the streets or Andrea's childhood memories in the botanical garden.

Right. It's like breadcrumbs leading you deeper into their psyches, making you question everything you thought you knew about them. And I think that's what keeps you hooked. You're trying to piece together these lessons learned alongside the characters. But the novel doesn't give you easy answers. It throws you curveballs and leaves you grappling with those questions long after you finish reading.

It makes you think about the choices we make and the consequences that follow and how sometimes the most important lessons are the ones we learn through our mistakes. It's like the novel throws this huge curveball, right? I mean, up until this point, the stakes are high, but Mirtha's death, it just changes everything.

Oh, yeah. Like you think you know where the story is going and then bam, it just takes this dark turn. And for Daniel, who's already kind of teetering on the edge,

it pushes him over. He goes into this complete downward spiral.

He's drowning in guilt, isn't he? Not just for the forgeries, but for everything. The choices he's made, the people he's heard.

And then he does something really surprising. He confesses to his sister that he killed Mirtha.

Even though we know he didn't.

It's like, why would he do that? It's such a fascinating psychological moment. He's carrying so much guilt, so much self-loathing. It's almost as if he wants to be punished, even if it's for something he didn't do.

Like he needs to take responsibility for the whole mess, the whole tangled web of lies and deceit.

Exactly. And in the midst of all this turmoil, he finds himself drawn back to the archives.

Which is so interesting, right? Because it's like the opposite of the world he's been living in, this world of forgery and secrecy.

The archives represent order, history, a sense of permanence.

And it's where he has this incredible conversation with Señora Cianbiano, the archivist.

She's amazing. So wise and insightful.

She tells him, you don't have to be born in a place to be of that place, which I mean, that just hit me hard.

It's such a profound statement. It's like she's challenging his whole idea of belonging, of identity. And it's a theme that runs throughout the novel, this idea of where we belong, who we are.

Right. And for Daniel, who's been so lost and adrift, it's like a light bulb moment. He starts to see the value in the ordinary, in the lives of people who might not have fame or fortune, but who have stories worth telling.

It ties back to the novel's title, Lessons Learned Through Forgery, because it's not just about art, it's about life, about finding meaning in the unexpected.

It's about those everyday moments, those small connections that make us human.

And that leads him to uncover the truth about the Minkowski painting, the one that Andreas deals.

Oh, yeah, I was wondering when we were going to get to that.

It's all connected. Daniel stumbles upon the story of Minkowski, this Jewish artist who escaped the pogroms in Europe and found refuge in Buenos Aires. But his story has been largely forgotten, his work overshadowed.

It's like a microcosm of the themes we've been talking about, authenticity, forgotten history, the way value is a sign.

Exactly. And Daniel feels this incredible connection to Minkowski, almost like a responsibility to honor his legacy, to bring his story to light.

He wants to make things right, to use his talent for something good.

And he realizes that art can be a powerful tool for remembrance, for keeping those lost stories alive. It's not just about creating beautiful objects, it's about connecting with something bigger than yourself.

So he decides to recreate the Minkowski painting, not as a forgery this time, but as a tribute.

It's a powerful moment. He talks to Nudlebaum, a Holocaust survivor, and he says, "Forgetting is a form of death. Remembering is a form of life." That idea really hits home for Daniel.

It gives him a purpose.

And he makes this incredibly bold decision. He incorporates the recreated Minkowski into a mural he's creating on the wall of the museum.

Talk about a statement.

It's a challenge to the museum's authority, a way for him to reclaim his artistic identity, to move beyond the shadows of forgery.

The scene where he's painting the mural is just so vivid. You can practically feel the energy, the passion he's pouring into it.

He's not just painting a picture, he's telling a story.

He's making a statement about art, about history, about the power of memory.

And it all culminates in this incredibly intense moment. Daniel, Andrea, Ramiro, they're all at the museum for the unveiling of the mural.

It's a reckoning, a confrontation with the past.

And for Andrea, it's a moment of profound realization. She's been through so much. She's questioned everything. And seeing the mural, especially the recreated Minkowski, it's like it gives her a sense of peace, a sense of close love.

She even refers to it as the Minkowski or what it was.

Right. Like she's finally letting go of those rigid definitions of authenticity.

Embracing more fluid understanding of art and its meaning.

And then Daniel adds this final touch to the mural, an image of a couple arriving at Constitution Station, their lives stripped bare.

It's such a powerful image. Yeah. So full of emotion.

It represents all those countless individuals who have shaped the city but whose stories are often overlooked. It's a tribute to the ordinary, to the everyday heroes.

The ending of the novel is open ended, isn't it?

It is. It leaves you wondering what happens next for these characters, how their choices will continue to shape their lives.

But it's clear that they've all been deeply transformed by their experiences.

They've faced their demons, grappled with the consequences of their actions and come out the other side with a deeper understanding of themselves and the world around them.

Those final lines of the novel are so powerful.

“Mistaking as worthless the dull everyday swirl surrounding us, we forget our prime roles as reluctant witnesses ceaselessly tracing the void after the story vanishes.”

It's a reminder to pay attention, to value the stories around us.

And to recognize our own role in preserving them.

It's like the author wanted to show us all these different facets of the art world. You know, the creative side, the business side, the historical and ethical side. They all come together in this story.

Yeah. And they clash. I mean, it's not exactly a harmonious picture.

Not at all. But it feels real, doesn't it? Like you can see how these different forces are constantly at play.

Right. And then you have a character like Federico, who's almost like a puppet master pulling strings in the background.

He's so hard to pin down. Like one minute he's this brilliant artist and mentor and the next

he's taking advantage of people, manipulating them to get what he wants.

Exactly. And it makes you think about the dark side of mentorship, especially in a field like art, where it's so subjective. So much about personal connections.

It's like a cautionary tale about the potential for abuse of power.

And the way the novel handles his downfall, it's so well done. It's not this big dramatic showdown. It's more like

the slow unraveling.

Yeah. Like he's finally caught in his own web of lies. And that scene where dos Santos, the enforcer for the Brazilian syndicate, confronts him in his apartment.

Oh, man. That was chilling.

It's like the whole atmosphere changes. You suddenly realize the danger that lurks beneath the surface of this world.

It's a reminder that actions have consequences.

And sometimes those consequences can be brutal.

But amidst all that darkness, there's this glimmer of hope in Daniel's journey. He's made some terrible mistakes, but you can see that he genuinely wants to redeem himself.

Yeah, he's not just trying to escape punishment. He's actually trying to make amends.

And his decision to recreate the Minkowski painting, to incorporate it into his mural. It's such a powerful statement. It's not just about honoring a forgotten artist. It's about

confronting the wrongs he's committed.

Exactly. And it's about reclaiming his art, using it for something good, something meaningful.

It's also a reminder that even in the darkest of times, there's always the possibility for redemption.

And that's what makes this novel so compelling. It's not just a story about art forgery. It's a story about human nature, about the choices we make and the consequences that follow.

And the power of art to connect us to something larger than ourselves.

Absolutely. The way Daniel uses his art to tell stories, to preserve memories, to make a statement about the world. It's inspiring.

And that final image he adds to the mural, the couple arriving at Constitution Station, it's just so poignant.

It speaks to the universality of the human experience, doesn't it? Like we all come from somewhere. We all have a story to tell.

And those stories are worth remembering, even the ones that seem ordinary or insignificant.

Because they make up the tapestry of life. And it's our responsibility to bear witness to them, to honor them, to keep them alive.

And that's exactly what this novel does. It reminds us of the power of storytelling, of the importance of remembering and of the enduring beauty that can be found in the most unexpected places.

So to our listeners out there, we leave you with this question. What lessons resonated with you most from lessons learned through forgery? We'd love to hear your thoughts. Share your insights on social media using the hashtag, hashtag deep dive forgery. Until next time, keep exploring, keep questioning and keep diving deep.

Imagine Paris in the 1920s.

You play the role of a photographer caught up in the mystery of a surrealist artist, who has suddenly disappeared. For more on the backstory, check out my video: AI-Crafted Radio Mystery: A Surrealist Artist Vanishes in Paris.

As the player photographer you uncover clues by exploring the streets, nightclubs and art galleries of Paris. You interact with non-player characters (NPCs) to piece together the story of the missing artist. A working title for this game is Never Fear Paris. (An alternate title is Never Fear a Nightclub Dancer.)

A large language model helps bring these characters to life by creating dialogue that feels authentic. Indeed, the language model is able to shape unique conversations each time you play. Replayability is an important factor in game design. Who likes a game that can only be played once, especially if it’s in two hours?

In this lecture, we’ll explore at a high level how the language model enables realistic NPC dialogue and provides context-aware responses that shape the player's choices and narrative path.

A language model is essentially a complex probability distribution over sequences of words, determining the likelihood that certain words will follow others to form coherent sentences and responses. This probability-based structure allows it to generate text that aligns with the context, character, and setting.

Imagine the scene: You’re in a smoky Montmartre bar, seeking clues to the whereabouts of a missing artist. As you speak to the bartender, the language model calculates which words he’s most likely to say based on historical knowledge, the mystery’s tone, and how he might typically respond to questions about an artist.

First, a disclaimer: this interactive mystery is in English. In real life, our bartender would be speaking French. For the examples of this lecture, we are going to have all the dialogue in English.

Example: If our player asks, “Have you seen this artist?” the model calculates the likelihood for the bartender to respond, “I saw him with that peculiar art dealer near the Seine…” over less relevant responses.

But, already, we have a problem. A barebones large language model will respond with the most likely word to that question, which is “no”. A language model is exactly that: a model of the language. A language model has been trained on vast amounts of text to understand the syntax of language and the probability of word distributions.

Training a large language model has two stages: pre-training and post-training. You also will see this described as training (in other words, pre-training) and fine-tuning (in other words, post-training). I prefer the training and fine-tuning description and will use that going forward.

To understand how a large language model can play a performative role in our interactive mystery, let’s examine training and fine-tuning.

Natural language is sequential.

We all learn grammar in elementary school (and middle school and high school). Through studying grammar we are learning syntax: the rules of what words go in which order to form a coherent sentence. Then, in college, you learn writing essays, which is learning to craft sentences into paragraphs. A paragraph is a sequence of sentences that present a point. An essay is a sequence of paragraphs that make a persuasive argument. Writing a good essay is challenging. It forces you to think carefully about what you are trying to say.

But, even without an education, as very young children, we learn language by emulating what we hear around us.

We learn intuitively that language is a sequence of utterances that have meaning. To know a language, like English or French, means that we have internalized meaning to those sounds, which are represented as letters and words. That internalized (or learned) meaning of words are known as semantic properties.

Large language models, through the architectures of neural networks, have been trained to learn those same properties of semantics.

The types of large language models that are popular today are known as autoregressive language models.

Why autoregressive?

We know that the language model takes a sequence of words as inputs, and then generates the next predicted word. It then takes that most recently generated word and automatically returns it back into the model to generate the next word in the sequence, and so forth. The feedback loop is the essence of autoregression.

This process allows the model to generate coherent sequences word by word, with each new word prediction informed by the previously generated words, continuously building a sentence or dialogue in context.

But, we still haven’t explained why the model would respond to the question, “Have you seen this artist?” with the response, “I saw him with that peculiar art dealer near the Seine.”

Context-specific responses, like those in my interactive mystery, are achieved through fine-tuning. Fine-tuning is an additional training step where the model is adapted to a specific domain (in this case, a 1920s Parisian mystery).

Here’s how it fits into the overall training process:

Initial Training: Large Language Models are first trained on a broad corpus of diverse text, often scraped from the internet. This training process teaches the model general language patterns, syntax, and broad contextual associations.

Fine-Tuning: After this general training, the model undergoes fine-tuning to adapt its responses for a specific application, such as my interactive mystery set in 1920s Paris. During fine-tuning, the model is provided with examples of context-appropriate responses, such as conversational snippets and phrases that fit the storyline, setting, and character roles.

Fine-tuning helps the model associate questions like, “Have you seen this artist?” with historically and contextually plausible responses, such as “I saw him with that peculiar art dealer near the Seine,” instead of irrelevant or anachronistic replies. Fine-tuning helps establish the narrative tone, historical setting, and character-specific language.

During training, the LLM learns probabilities for word sequences based on vast amounts of text data. In fine-tuning, we refine this process by exposing the model to targeted examples. By fine-tuning it on curated text relevant to our interactive mystery—dialogues, scene descriptions, and possible interactions—the model learns patterns specific to our setting, enhancing its ability to generate responses that feel authentic to 1920s Paris.

The fine-tuning gives the nuance the LLM needs to deliver responses fitting the game's unique world.

Fine-tuning is a step done by the developers of a software application that is enabled by AI. For instance, ChatGPT is a software application enabled by a large language model. OpenAI, the makers of ChatGPT fine-tuned their LLM with a lot of additional instructions and information so that ChatGPT would function as a robust conversational assistant.

Likewise, if you are developing an AI-enabled app, you would fine-tuned the LLM that you are using with a lot of specific information about the context of your app.

As for my interactive mystery, remember that a video game is a software application. An LLM can be fine-tuned to function as a conversational agent within a game.

One important takeaway is that most people do not know that anyone can fine-tune an LLM to serve specific purposes.

Think about that. If you think about that deeply, you will understand the possibilities of large language models.

We’re barely 100 years since the first radio broadcast by the BBC (in 1922). I’m enchanted by the early twentieth century: the changes in the social and world order. And there are the parallels to the development of technology.

Earlier this year, I imagined a radio show that introduced the mystery of a surrealist artist that vanished in Paris in 1926. I created the concept, edited the script, and edited the final video. The images are from MidJourney, the dialogue voice from ElevenLabs, the music and sound effects from WavJourney.

The particular purpose of this radio show is to serve as a prologue to a larger interactive mystery, Never Fear Paris.

Story Design Dossier

A game design document (GDD) guides the development of a video game. One could argue easily that an interactive story is a game. But I’m not entirely comfortable with that framing of the experience. I’m using the term ‘story design dossier’.

The story design dossier guides the development of the interactive story. The dossier details narrative elements, character profiles, plot points, and interactive mechanics, providing a structured framework to ensure consistency and coherence throughout the process. A story design dossier aligns the vision with the implementation.

High-level Concept

Never Fear Paris: A Photographic Mystery is an interactive murder mystery immersing players in the artistically vibrant world of 1920s Paris. The year is 1926. You play a photographer observing life in the city. You step into the shoes of a photographer, capturing images revealing hidden clues and engaging with enigmatic characters: Elodie, a nightclub dancer; Marcel, an artist.

In an early scene, you sit and watch Elodie dancing and interacting with Marcel. You overhear a person in the club say, “The killer is in the room.” You see Elodie suddenly leave the club. Why did she leave? Who are these people, and who was killed?

As the player, you’ve entered this mystery. You task yourself with finding the killer, though everyone seems a suspect. With no detective training, as a photographer and observer of human nature, you learn about 1920s Paris life while seeking the killer.

You photograph the streets, the neighborhood around the club, hoping to see the dancer again. You encounter others—artists, musicians, and patrons who frequent the club. As you photograph them and the area, you collect photographic clues. In your studio, developing the film reveals details not evident on first sight.

The game blends history with AI-driven experiences exploring the nature of conversations among strangers, some leading to friendship, others leading down a deadly path. One one level, “Never Fear” is simply a murder mystery with the backdrop of a city recovering from the shadows of World War I. But it’s also a serious game wrapped in a story that wrapped in revealing the limits of perception, misinformation, and technology.

The Disappearing Artist

Through the clip from the BBC radio broadcast, we learn that a fellow surrealist artist and close friend of Marcel's disappeared without a trace several months ago. The last place he was seen was at Le Chat Noir, sketching a mysterious figure in the corner of the club. Marcel has been haunted by the disappearance, convinced that the answer lies within the walls of the club or perhaps even with Élodie herself, who joined the club around the same time.

Interactive Story & Gameplay

Interactive story and gameplay are used almost interchangeably, reflecting their intertwined nature in the context of narrative-driven experiences. An interactive story extends beyond traditional storytelling by allowing the reader, or player, to make choices that influence the narrative’s direction and outcome. This interactive element is what we refer to as gameplay in the context of a narrative-driven game.

Gameplay, traditionally associated with player actions and strategies in game environments, emphasizes the player’s engagement with the story through decisions, explorations, and interaction with characters and objects. These gameplay mechanics are integrated into the narrative structure to create an immersive experience where the player’s choices drive the story forward, resulting in unique playthroughs. Understanding gameplay in this context recognizes how player interactions contribute to the unfolding of the story, blurring lines between playing a game and experiencing a story.

Interactive Story Elements

The gameplay is designed for each player to shape the course of their experience where choices have impact on the story’s outcome. This AI-driven feature enhances the depth of the game’s world and plays a critical role in unraveling the mystery, as the information gathered by the players directly influences their understanding of the unfolding story.

The Cast: artists, musicians, and their admirers. Each person has their own story and potential secrets.

Branching Narratives: the story employs AI-driven dynamics to create dynamic story paths, enabling different experiences for each player. Each decision made by the player not only alters the immediate response from the story world but also branches the storyline in significant ways.

Conversations with Non-Player Characters (NPCs): Engaging conversation is powered by AI. Players can question NPCs and gather perspectives. Each dialogue dynamically influences the NPC’s reactions and attitudes.

First-Person Perspective: Experience the story through the eyes of the photographer, capturing the essence of 1920s Paris and its inhabitants.

Photographic Clue Collection: Photograph scenes, people, and events around the club and Paris. Developing these photos in your studio reveals hidden details and clues.

Non-linear Storytelling: Explore the streets of Montmartre, uncovering clues at your own pace and following leads based on your discoveries.

Puzzle Solving: Use your skills as a photographer to piece together the mystery, analyzing photographs for hidden messages and connections.

Moral Choices and Consequences: Decisions made throughout the story affect how characters interact with you and can lead to different endings, some not so pleasant.

Notable Qualities

Several notable qualities set it apart as a distinctive interactive narrative experience. These qualities offer players a lens through which to explore 1920s Paris. From the historical setting to the use of photography as a gameplay mechanism, each element has been crafted to engage players into a deeper storyline that encourages multiple playthroughs.

Historical. The setting brings to life the sights, sounds, and atmosphere of 1920s Paris.

Photography as a Gameplay Mechanism. Use of photography in gameplay allows players to uncover clues and piece together the story in visual ways.

Deep Narrative. The multi-layered story presents a compelling mystery and dives into the themes of perception, misinformation, and the impact of technology on society.

Character Interactions. Engage with complex characters, each with their own backstory and relationship to the mystery.

Replayability. Multiple endings based on player choices, encouraging the exploration of different narrative paths.

Target Audience

• Fans of murder mysteries, historical movies, and novels.

• Players who enjoy narrative-driven games.

• Visual novel enthusiasts interested in gameplay mechanics and non-linear storytelling.

Player Character

The Photographer: the player character. You see the game from the photographer’s point-of-view. You never see your own face in the story. (In your own life, do you see ever see your own face, except in a mirror?)

Main Non-Player Characters

Élodie: a nightclub dancer

Marcel: a tormented artist

Mateo (Teo) Monti: missing artist. Last seen: June 13, 1926 at Le Chat Noir nightclub. DOB: January 30, 1893. Buenos Aires, Argentina. Age at disappearance: 33. Monti came to Paris at age 29 in 1922.

Erik. Frenchman, age 59, composer of music, in the last year of his life. Knowledgeable character in the style of Erik Satie.

Edward Thompson: BBC reporter living in Paris.

Secondary Non-Player Characters

Jazz musician. African-American saxophone player living in Paris.

Jazz singer. African-American woman in the style of Josephine Baker or Nora Holt.

Dancer. French female friend of Elodie

Expat writer. American in the style of Hemingway.

Ballet dancer. Man who frequents the club.

Assorted people in the nightclub and on the streets of Paris.

Clayton: modern-day archivist maintaining the collection of photographs taken in 1920s Paris.

Becca Hayes: a filmmaker in 2024 making a documentary about the disappearance of Monti.

Never Fear Paris: A Photographic Mystery is more than a game; it’s an immersive journey into the past, a challenge to the lines between reality and illusion. As a player, you bring this world to a life.

Insights

• Meta Movie Gen (link):

  • Description: Meta teases a few video examples.

  • Insights: The 92-page research paper “Movie Gen: A Cast of Media Foundation Models” Here’s all the videos from the paper. This is likely to be required reading for my Data for Deep Learning course next term.

• Wispr Flow [pro account: $15/mo] (link) and superwhisper [$8.49/month] (link):

  • Description: Voice-to-text. At those prices, it’s obvious which one I will try first. Looks like educators also get the student discount to Wispr Flow. So, that’s good. Developer roadmap for superwhisper.

  • Insights: I’m really into conversational AI, but I don’t want to speak to every app. I should try dictation for writing, as a way of thinking aloud. But, for serious writing, I’m still a pen on paper guy.

• OpenAI Canvas [link]:

  • Description: Enhanced UX (badly needed) within ChatGPT for writing and coding.

  • Insights: Coding with the help of ChatGPT was painful, e.g., copying/pasting code from ChatGPT into VS Code. When I had a GitHub Copilot license, it was much better to get AI help directly within VS Code. I’ve not tried Cursor [link] but plan to listen to the Lex Fridman podcast with the Cursor team. But, has OpenAI Canvas obliterated the need for Cursor? The Cursor website is really ugly. The site design alone discourages me from giving it a try.

  • As for writing prose, I already have a solid routine with Scrivener. I use ChatGPT mainly for providing thoughtful questions that help me revise my writing.

• More than 15k prompts [link]:

  • Description: From the ACM 2024 paper “Do Anything Now: Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models”

  • Insights: Includes more than 1,400 jailbreak prompts. I’m not really interested in that. But I always learn a thing or two from looking at other people’s prompts.

OpenAI ChatGPT 4o system prompts

Found via @elder_plinius and @jasonkneen

ChatGPT 4o Canvas system prompt:

Code conversion:

Code review:

Bug fixes:

Question Needing My Attention

How can analyzing responses from a diverse set of prompts enhance my understanding of AI's proficiency and limitations in generating coherent and dynamic branching narratives?

This question encourages exploration into how AI manages complexity in storytelling, particularly how it handles narrative forks, maintains plot cohesion, and adapts to user inputs within interactive frameworks. This could be instrumental in assessing the depth, variability, and engagement of AI-generated stories.

[The previous two paragraphs were generated by ChatGPT. I find it helpful framing.]

Thanks for reading!

Jeff

On my walk today, I kept trying to find a better means of expressing the spatial experiences that will be offered through future display devices.

When I Zoom with a friend, an image of my friend appears on my computer monitor. She is talking, listening, and responding to me. Are we not experiencing reality?

I would think most people will say that Zooming with someone is reality despite the lack of physical presence where we can touch another, or even smell their scent.

More than a decade ago, when I lived in Buenos Aires, I would Skype with my mother who lived in Tennessee. She has since died, yet I still have memories of talking with her.

Those memories enter my mind as thoughts, sometimes at the most inopportune times. I know she’s gone but the memory is real. Or, perhaps I should say, my experience of a thought as a memory is a real experience for me. I know that memories are not accurate. Some memories are of a reality that never occurred but have become, over time, conflated in our minds with other situations. Yet, experiencing those thoughts are our reality. Is that a form of mixed reality? I think so. Mixed reality does not require a digital device. Mixed reality is a play upon the mind.

As we move further into the mid-century, we are creating mechanisms through advances in display technologies that are distorted mirrors projecting scenes onto our field of view. (See Meta’s prototype of Orion eyeglasses.)

With advances in all technology, comes responsibility to those wielding the capacity to create the worlds we experience. Likewise, each person stepping into this new form of mixed reality has responsibility for recognizing falsehood from reality. Educators often talk of digital literacy. The aim of all education should be a literacy of reality.

Connecting to My Life’s Work

I’m not studying artificial intelligence merely to know more. I want to teach more engaging courses on AI. I want people to make the connection between “science” and their lives, specifically with what they view as their life’s work.

In connecting AI to my own life’s work, I need more focus on embodied systems, e.g., avatars that play the role of characters in stories. Or in another context, how AI-enabled avatars serve as teachers, conveyors not only of information but aiding us in developing meaning and understanding of new topics.

The work of Murray Shanahan offers many insights for thinking about LLMs as components embodied (i.e., integrated) within an application. In my courses, I asked the students to write their answer to this yes/no question: is ChatGPT a LLM? In both courses, over 90% of the students said, ‘Yes.’ We then proceeded to talk about how ChatGPT is an application with functionality wrapped around the LLMs created by OpenAI.

Quote to Think About

"Our life is a faint tracing on the surface of mystery." Annie Dillard

A memory

In college, one of the books we read for my Religion class (a required course at Sewanee) was Pilgrim at Tinker Creek. What I remember about that book is nothing from the book itself but a place, a specific reading room in the library, where I sat while reading the book. One day, sitting near me was a guy from the same class. I’ve forgotten his name. I could vaguely describe him. We both sat there reading the same book. We started talking; he surely initiated the conversation since I was too shy. I remember his enthusiasm for the book. Why has the image of that brief conversation stayed with me all these decades? Whenever Annie Dillard’s name comes up, that moment in the library in 1985 is what comes to my mind.

I should reread that book, particularly since I live in a valley by the Blue Ridge Mountains near where Dillard wrote her book. This is a book I should give to my kid.

Helpful thoughts on “seeing” (by Maria Popova) from reading Pilgrim at Tinker Creek.

Why do we remember?

A book I purchased last week and started reading: Why We Remember: Unlocking Memory's Power to Hold on to What Matters by Charan Ranganath, which I learned about from the Finding Mastery podcast.

Reading a book is a passive experience. I love reading. The feeling conveyed through a love of reading, for me, is the feeling of being immersed in some other world. But, even after reading a book I really enjoyed, the finely crafted sentences in that book leave no more than “a faint tracing”.

My vivid memory of The Pilgrim at Tinker Creek was formed through an active association located in a specific physical environment that held meaning for me. How do I create virtual environments and experiences that serve as containers for stimulating the memory?

Why I learn what I learn

The other morning, I did a review of my year, so far, and what I’ve accomplished or, more appropriately, what I did not. There’s the life I want to return to next year: location independent while pursuing my life’s work in creating interactive stories with narrative paths and dialogue enhanced by AI.

This new genre of spatial experiences lack a decent name, though agonizing over what to call a genre is less important than creating works in that medium.

When it comes to my life's work in understanding how we tell and read stories (fiction and factual) in digital media (i.e., via hardware and software) in the mid-21st century, I want to form a lens through which to examine the matter. (Honestly, I just want a short phrase to tell myself every morning as a reminder about why I’m doing what I’m doing.) I want to articulate my lens in a simple way, e.g., if this tech exists, what experiences can be created? By "tech", I mean the AR/VR that is becoming possible over the next decade due to advances in AI, GPUs, computer graphics and related tech.

This lens focuses on the transformative potential of AR/VR, AI, and advanced graphics technologies to push beyond traditional boundaries of storytelling.

The core questions to guide my exploration:

Experience Design: What new types of immersive experiences become possible with the integration of AR/VR and AI-driven storytelling?

Emotional and Cognitive Impact: What emotional and cognitive experiences can be uniquely crafted with this technology that were not possible before?

By using this lens, I want to maintain a focus on the possibilities and implications of these technologies on storytelling and user experience, always asking: "Given this tech, what new kinds of stories can we tell, and how will they change the way we experience and understand narratives?"

I really need a strong sentence that I tell myself everyday and that I can convey to others as to what my life's work is all about.

What new types of immersive experiences become possible with the integration of AR/VR and AI-driven storytelling?

Experience Design:

Spatial Experiences are stories that unfold within immersive, interactive environments, allowing participants to move through and engage with the narrative as if it were a tangible world. These experiences blur the boundaries between the physical and the imagined, transforming how we perceive, explore, and connect with stories by placing us directly within them. They create a dynamic interplay between the environment and the participant, offering a profound and personal journey that reshapes our understanding of narrative and reality.

On my walk, I wanted to toss away the term spatial experiences as the term for defining the genre. Is not everything we do in the world a spatial experience? Perhaps “spatial interactive experience”, which could be called SIX. Catchy, no? But it makes no sense. I’m having a spatial interactive experience as I sit here typing on a keyboard.

I return to mixed reality as an intrinsic part of the human experience.

With these technologies, how can we craft stories that reshape our perception and engagement with the world?"

At the end of the day, while I’m wildly intrigued by the philosophy of these experiences, I’m more excited about the craft of spatial experiences.