In summer 2023 I’ll have a 3-month research leave from my university position. I’m on a 12-month faculty non-tenure track appointment, so I have to take my sabbatical during the summer months. This post is an excerpt from my sabbatical proposal.
My research intends to identify a roadmap to conversational artificial intelligence (AI).

The computational capacity in graphic processing units (GPUs) that powers computer gaming also has led to massive breakthroughs in AI. Not only are the visuals becoming more photorealistic, the behaviors enabled by AI are accommodating a nearly endless range of interactions. This convergence with a real-time 3D virtual environment produces digital avatars capable of engaging in unscripted conversation. Embedded throughout this research is a concern for the ethics of this new reality we are entering.
In 2022 it’s difficult to grasp the impact that 3D virtual environments will have on our lives and work by the end of this decade. In 1992, as I finished graduate school, the Internet had barely emerged beyond academia, and “the Web” had yet to even enter the everyday lexicon of hardly anyone. By the year 2000, the Web was everywhere. Likewise, a century ago in 1922, everyday air travel and talking movies in full color seemed fanciful flights of imagination.
Our students will witness changes more profound than those we have encountered in our lifetimes. These students are the ones that get to define that future and shape the course of governance, society, culture, and industry in this century. My research leave focuses on one specific change impacting that generation.
I recognize that we can never know the actual implementations and rare events that shape the times ahead. Yet, as a librarian, I have followed the trajectory of published research studies and proceedings from many disciplines on technology transfer to industry. Likewise, analyzing the job postings in the tech industry highlights the significant hiring in artificial intelligence, optical engineering, and related fields.
Looking ahead over the next thirty years, into the mid-21st century, reveals the next evolutionary phase of the Web: real-time 3D virtual environments. For many people that phrase evokes a head scratching, “Huh?”
Let’s break that scenario down into essential components:
Virtual environments: this is not just VR. Already we live, work, and socialize in virtual environments mediated in 2D by flat screens.
3D: our flat screens will be supplemented by devices that display photorealistic models of the natural environment, buildings, objects, animals, and people.
Real-time: a 3D environment that responds to your actions within milliseconds.
Sounds fanciful? In 1922 the thought of color TV was barely known, and let’s not even consider the 1922 perspective on today’s digital culture. Tumultuous and transformative changes are ahead even if we don’t consider the digital. (People must, first, survive climate change, potential nuclear Armageddon, and authoritarian regimes.)
Industries such as automotive, architecture, construction, and engineering are utilizing real- time 3D to create digital twin models to plan and study physical infrastructures. These digital models often are built with software commonly known as game engines since video games are defined by virtual environments that respond to a player’s actions in real-time as the player presses a controller or keyboard.
The conversion of real and imagined physical landscapes and buildings into realistic 3D models represent worlds for exploration. (For examples of photorealistic 3D, see https://quixel.com/.) (For examples of digital humans, see https://www.unrealengine.com/en-US/metahuman ) Populating these virtual environments are digital animals and digital humans: avatars of our imaginary selves.
For the past two years, my research agenda focused on a topic I described as the intersection of game engines, creativity, and business. I have gained significant skills in programming Unreal Engine, the most advanced 3D simulation tool on the market. That skill development has led to hands-on experience with these concepts.
In Fall 2021 as part of guiding an independent study in Data Science, I started examining how the major providers of cloud data infrastructure (e.g., Amazon, Microsoft) are providing prebuilt AI models and an application programming interface (API) ecosystem for startups to utilize AI-enhanced products and services without the massive investment in hardware infrastructure. The productization of AI by Amazon, Microsoft, and others significantly lowers the cost of entry. (For expanding my skills in this area, I have a side project utilizing Microsoft Cognitive Services to explore speech synthesis and generative text creation for language learning and storytelling.)
The aim of my leave is to consolidate my direct experiences in the technology with a study of current research to produce a guide to conversational AI with digital humans. I have chosen that focus due to its impact on how we tell stories, how we experience stories, and how we learn.
Within two decades, many of our tasks and encounters present possibilities for replacement by conversational AI with digital humans.
Obviously, serious ethical matters with these technologies require addressing by a generation of thoughtful people. In Fall Term 2021, I taught Writing 100: Virtual Reality in which the students explored the positive and negative impacts of virtual environments through writing essays about a novel (Tomorrow and Tomorrow by Tom Sweterlitsch) that presented the scenario (in a cautionary tale) of digitally re-creating the lives of those in Pittsburgh after the city was destroyed by a nuclear weapon. In the novel, people could don VR gear to interact with their departed loved ones and wander the streets of the vanished city. The foundational technologies for such a virtual recreation exist today, though the graphic fidelity and scaling to market are still ten to fifteen years out.
Our students grasp the implications, particularly the need to establish societal guard rails for technology with the recognition that unstable participants (rogue corporations, as in Ready Player One , or even rogue nations) would wreak havoc. My work in this area aims to equip a rising generation with an understanding of the possibilities inherent in the complex challenges presented by this technology.
Perhaps the biggest obstacle is getting in front of these challenges before the proverbial train has left the station. Based on my technical knowledge of this area, the train already has departed. Technology seeps into our lives. What we once took as fanciful earlier in our own lifetimes (e.g., video calls) are normalized with little fanfare.