Mitchelville AR Tour

FAU Media, Technology, and Entertainment faculty and students have designed an immersive and interactive augmented reality tour experience of Reconstruction-era (1861-1900) narratives for Historic Mitchelville Freedom Park on Hilton Head Island, South Carolina. We used historical photography and period maps to visualize the Mitchelville structures and Gullah Geechee residents in the Unity and Unreal game engines. This project is funded by a Walter and Lalita Janke Emerging Technologies Fund seed grant, an NEH Digital Projects for the Public Discovery grant, and an Epic Games Megagrant.

We finished the preproduction design phase over the 2019-2020 academic year. The main deliverable for the “Discovery” phase of the project was a design document for the NEH, which can be viewed here as a PDF… >Mitchelville AR Tour Design Doc 2020<

2019-2020 Video Demo of Virtual Preproduction and Headset Tests:

Spring 2020 we used historical photography taken by Henry P Moore in 1862 of Mitchelville inhabitants and houses to create a 3D simulation of the town in the Unity game engine:

Mitchelville 1862 3D Game Engine Simulation
Mitchelville game engine simulation of 1862 historical photograph (Library of Congress). 3D models by Ledis Molina and James Jean-Pierre, Spring 2020.

We have designed a multimodal production pipeline to distribute 3D scenes developed in a game engine (Unity and/or Unreal) to multiple XR platforms, including augmented reality (AR- below left) mobile devices and mixed reality (MR – below middle) headsets in the park, a downloadable app for virtual reality (VR below right) headsets, and a web site portal to a 360 interactive web experience (below next image).

Mitchelville AR-MR-VR Multi-modal Apps
Mitchelville AR-MR-VR Multi-modal Apps

The prototype will also include a web site that features portals to 360 web pages of our AR tour experiences, as well as additional educational content and mini-documentaries:

Mitchelville AR Tour Web Design 2020
Mitchelville AR Tour Web Design by Topher Maraffi (2020), 3D models by Ledis Molina and James Jean-Pierre, historical images courtesy of the Library of Congress.

Here is a 360 web browser test of the Harriet Tubman animation set at the Prayer House in Mitchelville (left mouse click and drag to look around the 360 rendered scene):

  • 360 Demo by FAU MFA student Ledis Molina. Music by Anita Singleton-Prather and the Gullah Kinfolk, recorded live at the 2018 Original Gullah Festival in Beaufort, SC.

Fall 2019 we started Mitchelville pre-production work in my Preproduction, Prototyping, and Previs (DIG 6546) MTEn grad course that is cross-listed with my undergrad Visual Design for Film, Animation, and Games (FIL 4703/DIG 4122C) course, with both grads and undergrads working together on visualizing several stories related to Mitchelville and Reconstruction in Beaufort County. Students created 3D models of historic personalities like Harriet Tubman, Robert Smalls, and Frederick Douglass, and contemporary Gullah-Geechee storytellers like Aunt Pearlie Sue as performed by Anita Singleton-Prather:

Harriet Tubman Combahee River Raids research poster by undergraduate Dawn Davidson:

Mitchelville Project Harriet River Raids Undergrad Poster by Dawn Davidson, Fall 2019

We started virtual pre-production to create 3D cinematics for the Harriet Tubman Combahee River Raids storyboard using our Rokoko motion capture suit, HTC Vive base stations and trackers, and Unreal game engine. Graduate student Ledis Molina performed Harriet, assisted by graduate students Alberto Alvarez, Brandon Martinez, Camilo Morales, and undergraduate students Dawn Davidson, Mike Gonzalez, Juan Aragon, Tifffany Fedak, Alexandra Lusaka, Brandon Cela, and Robert Barclay:

Here is a render test in Unreal game engine of some shots edited by undergraduate student Tiffany Fedak, 3D model by graduate student Ledis Molina, virtual production by MFA students Alberto Alvarez, Brandon Martinez, and Camilo Morales:

See more FAU Media, Technology, and Entertainment research projects at: FAU MTEn-XR Lab

Categories: Research, Uncategorized

tophermaraffi

I am an Assistant Professor of Multimedia Production at Florida Atlantic University (FAU), where I teach game design, 3D animation, and extended reality courses in the Media, Technology, and Entertainment MFA degree program. I also teach undergrad design courses in the School of Communication and Multimedia Studies BA concentration in Film, Video, and New Media.