Media, Technology, and Entertainment Extended Reality (MTEn-XR) MFA Lab Courses & Projects at Florida Atlantic University

Extended Reality (XR) courses I teach at FAU utilize motion capture animation, game engines, and the latest VR-AR headsets, and include Preproduction, Prototyping and Previsualization (DIG6546), 3D Production for Interactivity (DIG6547), and Interactive Interface Design (DIG6605). My grad courses are often cross-listed with undergrad Film, Video, and New Media BA courses, such as Visual Design for Film, Animation, and Games (FIL4703), Advanced 3D Computer Animation (DIG3306C), and Special Topics: 3D Video Game Design (DIG4930). Our labs are equipped with Noitom Perception Neuron Pro mocap suits, Rokoko Virtual Production Studio, HTC Vive, Oculus, and Magic Leap headsets, and Unity, Unreal, Autodesk Maya, and Adobe Creative Suite software.

Projects, Courses, and Events

Mitchelville AR Tour Header

NEH Funded Historical Site AR Project: Mitchelville Stories of Emancipation & Freedom in Beaufort County SC

We received a National Endowment for the Humanities (NEH) Digital Projects for the Public “Discovery” grant, an Epic Games Megagrant, and a Walter & Lalita Janke Emerging Technologies Fund seed grant to begin design work on an augmented reality (AR) tour of Mitchelville, a historic site on Hilton Head Island SC which was the first Freedman’s town in the US during the Civil War, and a Gullah-Geechee heritage site today. We will be creating a 360 experience of the story of America’s first efforts at Civil Rights for African Americans during the Reconstruction period as it relates to the Port Royal Experiment, with life-sized historical figures like Harriet Tubman, who will be experienced on site with Magic Leap headsets and mobile phones.


We have also applied to an NEH Digital Projects for the Public grant to fund development, and are collaborating with The Mitchelville Preservation Project, Reconstruction Beaufort, Penn CenterFort Lauderdale Museum of Discovery & Science, Daruma Tech, and researchers from University of South Carolina Beaufort, North Carolina State University, Coastal Carolina University, and Columbus State University. This project is supported by educational partnerships with Magic Leap and the FAU Center for Body, Mind, and Culture. Read the full proposal: NEH Mitchelville Grant Submission 2019 Maraffi.

Fall 2019 we started Mitchelville pre-production work in my Preproduction, Prototyping, and Previs (DIG 6546) MTEn grad course that is cross-listed with my undergrad Visual Design for Film, Animation, and Games (FIL 4703/DIG 4122C) course, with both grads and undergrads working together on visualizing several stories related to Mitchelville and Reconstruction in Beaufort County. Students created 3D models of historic personalities like Harriet Tubman, Robert Smalls, and Frederick Douglass, and contemporary Gullah-Geechee storytellers like Aunt Pearlie Sue as performed by Anita Singleton-Prather:

Harriet Tubman Combahee River Raids research poster by undergraduate Dawn Davidson:

Mitchelville Project Harriet River Raids Undergrad Poster by Dawn Davidson, Fall 2019

We started virtual pre-production to create 3D cinematics for the Harriet Tubman Combahee River Raids storyboard using our Rokoko motion capture suit, HTC Vive base stations and trackers, and Unreal game engine. Graduate student Ledis Molina performed Harriet, assisted by graduate students Alberto Alvarez, Brandon Martinez, Camilo Morales, and undergraduate students Dawn Davidson, Mike Gonzalez, Juan Aragon, Tifffany Fedak, Alexandra Lusaka, Brandon Cela, and Robert Barclay:

Here is a render test in Unreal game engine of some shots edited by undergraduate student Tiffany Fedak, 3D model by graduate student Ledis Molina, virtual production by MFA students Alberto Alvarez, Brandon Martinez, and Camilo Morales:

Museum Research & Pedagogy to Enhance Exhibits With XR

We have submitted an IMLS grant to expand our summer App-titude collaboration with Fort Lauderdale Museum of Discovery and Science (MODS) to do year-round research and pedagogy on using XR to enhance museum exhibits. Over three weeks in the summer we taught App-titude high-school interns how to create AR enhanced research posters on STEM topics featured in MODS exhibits.

Fall 2019 graduate student Richie Christian started pre-production on a Prehistoric Florida AR project in my 3D Production for Interactivity (DIG 6547) MFA course. Here is a video of the initial design:

VR MFA Project: Homebound


MTen MFA students Alberto Alvarez and Brandon Martinez created the initial designs and prototype for their VR thesis project on an undocumented immigrant’s journey across the Southern US border in my spring 2019 Interactive Interface Design (DIG6605) course. Their research poster (click to see details):


Some image samples from their design document:

Spring 2019 “The experience will convey the hardship of a life changing journey for those that may never have this experience and promote a spirit of empathy and understanding towards our neighbors while exploring how these stories impact our lives in south Florida and create an open dialogue about the current state of immigration.”

Funded Interdisciplinary Driving Game Study


Spring 2019 We received a Dorothy F. Schmidt College of Arts and Letters seed grant to begin working on an interdisciplinary project to develop a driving game simulation using the Magic Leap One headset that will be used to inform autonomous vehicle design (in collaboration with Computer Engineering faculty Dani Raviv, Hari Kalva, and Aleks Stevanovic). Our game simulation will track what highly ranked drivers do with their eyes and bodies to successfully maneuver a vehicle in an urban environment. MTEn MFA students Alberto Alvarez and Brandon Martinez will be developing this project throughout the 2019-2020 academic year. Here is a video of a smaller VR car project they created for my spring 2019 Interactive Interface Design course:

Performatology Research Project: MRder Mystery Theatre


We are designing a spatial computing app for the Magic Leap One headset that is a Mixed Reality (MR) version of the classic murder mystery live improv-theatre game. A virtual body is found in the environment, with clues to the killer that can only be seen when wearing the headset. Animated avatar faces and costumes are superimposed on the players to hide the headsets and create a masked-theatre aesthetic of classic Clue-style characters reminiscent of commedia dell’arte archetypes. Suggested lines are generated in the HUD to drive the narrative of the play in both the game and theatrical sense. This project is part of our lab’s Performatology research, which applies performing arts and animation theory to interactive media.

Somaesthetics VR Locomotion Study

Somaesthetics VR walkabout

This project seeks to simulate mindfulness or meditative walking practices where practitioners gain heightened awareness of their body as they move through a natural landscape, by focusing their gaze on the area where their feet hit the ground in front of their body while walking. I am collaborating with FAU Philosophy Professor Dr. Richard Shusterman to apply his Somaesthetic concepts to human computer interaction (HCI) design of VR avatar feedback, so that players perceive more agency and presence as an avatar, and to relieve the problem of VR motion sickness.

Art Gallery Research Project: Climate Reality in South Florida

I have applied to the Knight Foundation Art Challenge 2019 for Miami area to create a virtual gallery experience that uses Magic Leap headsets to better understand the impacts of Climate Change in South Florida:

Description: “The Miami art gallery appears empty, with spotlights on blank walls and a sign reading “Climate Reality in SoFlo”. Three pedestals contain Magic Leap headsets. When wearing a headset, the walls reveal spherical images, virtual “tiny world” photographs of South Florida sites. You hear water sounds as you approach an image of downtown Miami, when it suddenly surrounds you as a 360-degree video. Water rises in the 3D environment until it reaches your waist, and as the vision fades, text informs you of projected sea level rise at this location. Along with other people wearing headsets, you notice virtual figures in the gallery. One of the shadowy figures reaches out towards you, and a climate related question appears. This art experience connects people to place and people to people through the issue of Climate Change using augmented reality headsets that communicate between a gallery in Miami and on FAU campus.”

FAU MTEn-XR Lab Grad Research Day Posters


MFA students in my Interactive Interface Design (DIG6605) course presented research posters at the 10th annual FAU Graduate and Professional Student Association (GPSA) Research Day. Pictured are Camilo Morales (above), clockwise: Brandon Martinez and Alberto Alvarez; Raven Doyle; Monisha Selvaraj; Alex Henry; Raven, Monisha, and John Horvath:

DIG3306C: Mocap Animation Reel


Spring 2019 My Advanced 3D Computer Animation (DIG3306C) students created animation sequences from motion capture data in online libraries and custom movements we captured with the Noitom Perception Neuron suit. Here is a selection of their work:

MTEn-XR Lab Open House

Spring 2019 We had an open house in our MTEn-XR lab for potential students where we demo’d our Magic Leap, Oculus Rift, HTC Vive, and Noitom Perception Neuron mocap technology:

DIG4930/DIG6547: 3D Game Design Projects


Fall 2018 Students in my Special Topics: 3D Video Game Design course used Unity game engine and Autodesk Maya to create seek-and-find 3D island adventure games. They applied the Mechanics, Dynamics, Aesthetics (MDA) game design framework to design the look and interaction, and then coded in C# to develop the operational logics. Here are some screen shots from some of their games:

Students also created research posters, and Marshall Fornataro-Minnig  showed a demo of his game at FAU’s Undergraduate Research Day:

PowerPoint Presentation

FIL4703/DIG6546: Visual Design & Pre-production Projects


Fall 2018 My Visual Design for Film, Animation, and Games undergrad course is cross-listed with my Preproduction, Prototyping and Previsualization grad course in the fall semester. In this course students go through the entire preproduction pipeline, including planning a media concept, designing aesthetics, painting concept art, drawing storyboards, and creating animatics. Here are samples of student concept art:

DIG4703 student Shannell Tebeau presented a research poster of her history inspired pre-production work at FAU’s Undergraduate Research Day (click poster for details):

PowerPoint Presentation

Students use a tablet and stylus to digitally visualize their narrative by drawing a storyboard of their scenes:


Categories: Uncategorized


I am an Assistant Professor of Multimedia Production at Florida Atlantic University (FAU), where I teach game design, 3D animation, and extended reality courses in the Media, Technology, and Entertainment MFA degree program. I also teach undergrad design courses in the School of Communication and Multimedia Studies BA concentration in Film, Video, and New Media.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s