Immersive Media 101

The world of academia now boasts a number of newly-minted programs focused on immersive reality. For this week’s post we reached out to Joseph Amodei, a professor of Immersive Media at Chatham University in Pittsburgh and asked them to share some insights and highlight student work.

Thanks for joining us, Joseph! What have you been working on lately?

At the moment I am working on projects that tackle gerrymandering through game-based performance (Packing and Cracking), Vaccine advocacy and Access through Free the Vaccine and other local Pittsburgh-based groups, and stitching together archives of care in the AIDS crisis and the COVID-19 pandemic through a project called the dance floor, the hospital room and the kitchen table. The last project is still in early redevelopment and will take the final form of a browser-based VR archive and an immersive theater performance.

I teach in the Immersive Media program at Chatham University in Pittsburgh which is an interdisciplinary program equipping students with knowledge of virtual reality and augmented reality technology, architecture, 3D modeling, game development engines, writing, and design thinking to prepare them for careers in immersive media creation.

At Chatham I teach how to work with the above technologies and, just as importantly, how to implement them in many fields with a critical eye for how they can best be leveraged to produce a more justice-oriented world. I also collaborate with students for my research and arts practice to give them a lens into how I specifically interface with immersive media in the expanded field.

That all sounds fascinating. How did you originally get interested in exploring the world through new media?

I never did anything in the art, performance, or technology world until my undergraduate years at UNC. There, through the help of some wonderful mentors, I learned to connect my ideas to creative and technologically driven artist-activist modes of production. The activist part has only become increasingly more important to me as time has gone on. I come from a low-income background and, as a result, ‘access’ in a broad sense has always been important to how I work with technology. I like working at the forefront of emerging technology because I can freshly explore these mediums before they become codified into the alienating capitalist cycles of commodification. I believe there is an often-overlooked opportunity to employ these technologies to create hope and joy via mediated surprises, outlets, and reformulations of tech within everyday life.

What are your students most interested in exploring these days?

My students are interested in all sorts of things! They are interested in using art, design, and technology to work toward social justice ends, and in thinking about where the field of immersive media can keep expanding into. They are also just excited about learning. The field is broad and we are always trying out new toolkits, techniques and modes of making across AR/VR/XR spaces.

What are your thoughts about the increased adoption of VR/AR environments for education, vocation and recreational usage?

I think it’s great! There have long been flight simulators used for pilots to learn to fly and I think AR/VR tech can eventually be used to help others learn to do anything. There is a lot of excitement around thinking about how safety can be improved in workplace training. For example, underwater welding will always be dangerous, but wouldn’t it be great if you could log a couple of hundred virtual hours before you had to go and give it a try in a dangerous real life situation. There is also great possibility for immersive media in healthcare. My insurance company already offers wellness training in a gamified application and it makes it much more engaging and fun! Although I never thought I would end up spending time playing a game about sleep hygiene I actually did and ended up walking away with some common-sense tips I may not have retained from simply reading an article. Of course these technologies are already being used in helping surgeons train, too and I think moving forward we will see AR/VR become more integrated into all sorts of new ways. On the personal health front we are even seeing AR/VR move into the recreational fitness space. Of course, there are fun workout/dance games like Beat Saber, but you also see exercise machines increasingly having something of an XR component. Here’s an example of an aerobic machine that was donated to the Immersive Media program, and it is a self-styled VR flying as exercise machine. It’s fun and also makes you engage your core.

What are some of the most interesting applications of emerging immersive technologies you have recently come across?

I have long been interested in depth-capture technologies and photogrammetry. The new Ipads have a Lidar sensor that can capture reality in 3D in remarkably accurate and quick ways. While Ipads are still expensive it does mean I no longer have to develop a custom capture backpack that had a battery supply, waterproofing, a laptop, and a kinect to do the same thing. And before that I was able to use the front-facing depth cameras on iphones (often used as part of the security feature face-ID) to make collages of depth data. Like in my artwork Point Cloud Gardens.

Super cool! Can you drill down a bit on the participatory part of your new project and introduce any collaborators?

Do we choose our politicians, or do our politicians choose us? Packing and Cracking is an interactive mapmaking event about gerrymandering: the pervasive practice of politicians choosing their voters rather than the other way around. Through participatory drawing and map-drawing games, Packing and Cracking uses critical cartography, gerrymandering history, and interviews with politicians and reformers today to show how easy and disenfranchising gerrymandering can be and ask what, if anything, we should do about it.

This was an in-person drawing game based theater performance that when the pandemic hit, my collaborators and I shifted it to an online performance. We translated all the in person games into digital ones through a combination of Zoom, a custom “game page” website, and projection design software. This piece and others are in this exhibition of all of our work: Creativity vs. COVID: Ending the Pandemic for Good

These are quite civically engaged projects. What are your thoughts on the creative technologist's role when it comes to commentary and critique?

The creative technologists role ought to be one of responsibility. This means working in ideas of intersectionality to help audiences consider structures of power while actively trying to dismantle these established structures towards more justice-oriented ends. I think a lot about race, gender, and class disparities in tech and I think the patterns are established early. For example, it is important who exactly has access to these tools and platforms and who is encouraged to explore them. As a white, genderqueer person I try to understand where I have privileges and where I do not and do my best to listen, keep learning, and apply my knowledge to help those around me and in my various communities. So, in certain ways, the role of the creative technologist is not that different than any other responsible citizen, it just means I often work with unique skills and tools in varied contexts.

Thank you so much for speaking with us today Joseph!

Joseph Amodei (they/them) is a new media artist, theater designer, educator, and activist. Joseph conceives of art as a powerful epistemic and emotional tool for examining assumed realities. Their work combines innovative technology, extensive research, and hope for alternate futures to invite audiences into a communal process of debriefing and re-learning. Joseph grew up in North Carolina, where they received a BFA in Studio Art from UNC-Chapel Hill. Joseph completed their MFA in Video and Media Design at Carnegie Mellon's School of Drama. Currently, they are a professor of Immersive Media at Chatham University in Pittsburgh. www.jamodei.com