FAU Media, Technology, and Entertainment faculty and students have designed an immersive and interactive extended reality (XR) tour experience of Reconstruction-era (1861-1900) narratives for Historic Mitchelville Freedom Park on Hilton Head Island, South Carolina. We used historical photography and period maps to visualize the Mitchelville structures and Gullah Geechee residents in the Unity and Unreal game engines. This project is funded by a Walter and Lalita Janke Emerging Technologies Fund seed grant, an NEH Digital Projects for the Public Discovery grant, and an Epic Games Megagrant.
We finished the preproduction design phase over the 2019-2020 academic year. The main deliverable for the “Discovery” phase of the project was a design document for the NEH, which can be viewed here as a PDF… >Mitchelville XR Tour Design Doc 2020<
2019-2020 Video Demo of Virtual Preproduction and Headset Tests:
Spring 2020 we used historical photography taken by Henry P Moore in 1862 of Mitchelville inhabitants and houses to create a 3D simulation of the town in the Unity game engine:
We have designed a multimodal production pipeline to distribute 3D scenes developed in a game engine (Unity and/or Unreal) to multiple XR platforms, including augmented reality (AR- below left) mobile devices and mixed reality (MR – below middle) headsets in the park, a downloadable app for virtual reality (VR below right) headsets, and a web site portal to a 360 interactive web experience (below next image).
The prototype will also include a web site that features portals to 360 web pages of our AR tour experiences, as well as additional educational content and mini-documentaries:
Here is a 360 web browser test of the Harriet Tubman animation set at the Prayer House in Mitchelville (left mouse click and drag to look around the 360 rendered scene):
- 360 Demo by FAU MFA student Ledis Molina. Music by Anita Singleton-Prather and the Gullah Kinfolk, recorded live at the 2018 Original Gullah Festival in Beaufort, SC.
Fall 2020 we started developing an AI 3D character design pipeline that uses Generative Adversarial Networks (GANs) and Neuro-evolutionary AI breeding tools to design realistic versions of historical characters, by inputting source photographs from the National Archives to learn features. Here are some examples of Harriet Tubman and Frederick Douglass that were done at Artbreeder.com:
Fall 2019 we started Mitchelville pre-production work in my Preproduction, Prototyping, and Previs (DIG 6546) MTEn grad course that is cross-listed with my undergrad Visual Design for Film, Animation, and Games (FIL 4703/DIG 4122C) course, with both grads and undergrads working together on visualizing several stories related to Mitchelville and Reconstruction in Beaufort County. Students created 3D models of historic personalities like Harriet Tubman, Robert Smalls, and Frederick Douglass, and contemporary Gullah-Geechee storytellers like Aunt Pearlie Sue as performed by Anita Singleton-Prather:
Harriet Tubman Combahee River Raids research poster by undergraduate Dawn Davidson:
We started virtual pre-production to create 3D cinematics for the Harriet Tubman Combahee River Raids storyboard using our Rokoko motion capture suit, HTC Vive base stations and trackers, and Unreal game engine. Graduate student Ledis Molina performed Harriet, assisted by graduate students Alberto Alvarez, Brandon Martinez, Camilo Morales, and undergraduate students Dawn Davidson, Mike Gonzalez, Juan Aragon, Tifffany Fedak, Alexandra Lusaka, Brandon Cela, and Robert Barclay:
Here is a render test in Unreal game engine of some shots edited by undergraduate student Tiffany Fedak, 3D model by graduate student Ledis Molina, virtual production by MFA students Alberto Alvarez, Brandon Martinez, and Camilo Morales:
See more FAU Media, Technology, and Entertainment research projects at: FAU MTEn-XR Lab
Categories: Research, Uncategorized
Topher Maraffi 