Spring 2024: Student Metahuman Monologues
I began teaching NCSU MADTech students the Metahuman Animator pipeline in my Special Topics: Motion Capture and Advanced Animation course (ADN492/592), which involved face capture with an iPhone and then developing the performance in Unreal Engine 5.2 for custom Metahuman characters using keyframing on a control rig. Fourteen students were taught how to apply Disney’s animation principles and performing arts techniques to motion capture in our virtual production studio, and then how to develop their real-time scenes in UE5’s sequencer for cinematic rendering. The resulting Metahuman Monologues have a liveness and personality that comes directly from the student’s embodied performance, with improvised expressions that would not be likely in a traditional keyframing process:
Fall 2023: Student Metahuman Cinematics for Creative Tech Studio 2
MADTech students in my Creative Technology Studio 2 course (ADN460) created Metahuman cinematics using photogrammetry and performance capture. Hardware/software they used were Unreal Engine 5.2, Reality Capture, Autodesk Maya, Quixel Megascans, Blockade Labs AI Skybox, Noitom Perception Neuron Studio and Livelink Face. Here are a couple of the best by Laura Reeve and Maleah Seaman:
Fall 2023: Creative Coding Student Game Projects
NCSU MADTech student game design reel for my Creative Coding course (ADN311/502). Students were introduced to three coding platforms: Fortnite Creative, Unreal Engine Blueprints, and Unity C# scripting. They applied game design concepts like MDA, Flow and Narrative Immersion to create similar 3D games in all three platforms, and then designed a research poster to summarize their design process and choices.
2023 NCSU MADTech Open House: Live Mocap Performance with our LED Wall
I performed my Metahuman double for a live audience in a Halloween-themed theatrical performance and mocap demo at our NCSU MADTech Open House in October. This recording shows me in our Noitom motion capture suit performing in front of our LED wall with an Unreal game engine scene that triggers dancing NPC zombies and visual effects like virtual explosions and fire.
2022-23: NCSU MADTech Metahuman UE5 Demo & Projects
Metahuman-Topher was born dancing on a virtual beach! Some stills from a demo project for my NCSU ADN-460 course on using game engine technologies to create a hyper-real self portrait for Unreal Engine 5 cinematics.
I taught students how to use Reality Capture to photogrammetry their head, so that they can make a Metahuman of themselves in UE5. They then exported the model to Autodesk Maya for animation using mocap from our Noitom suit, and created custom clothes in Marvelous Designer. The final animation is rendered in a UE5 themed environment that they build using Quixel Megascans.
Here is a video of my class demo and some of the final student projects for fall and spring:
……
Fall 2022: NCSU Media Arts, Design and Technology (MADTech) AR Projects in Unity
Undergraduate students in my ADN 460 Interactive Media studio course created augmented reality projects using Vuforia AR in Unity game engine, with animation assets created in Steam Fuse, Mixamo, and Autodesk Maya:
……
Fall 2022: NCSU VR Ethics Book Chapter Talk
November 17th 2022, I gave a lecture on “The Ethics of Playing Black Bodies in VR” based on my recent book chapter in The Changing Face of VR (Vernon Press, 2022). This was part of the NCSU Art & Design Fall Lecture Series titled Emergence. Here are my slides and a Zoom recording of the talk:
……
Summer 2022: New Book Chapter “VR Storytelling for Social Justice and the Ethics of Playing Black Bodies”
July 22nd 2022, I have a chapter in the new peer-reviewed book The changing face of VR: Pushing the boundaries of experience across multiple industries, edited by Dr Jordan Frith and Dr Michael Saker (Vernon Press, 2022):
Abstract: Using Augusto Boal’s Theatre of the Oppressed role play principles of a “spectactor” and joker figure as a performative lens, I analyse VR storytelling experiences by artists of colour that confront social justice issues and educate on Civil Rights history using a variety of platforms from 360 video to room-scale VR installations. I argue that narrative framing to cast a spect-actor into the body illusion of a protagonist of colour can be done ethically, and role play with NPCs designed as joker figures representing the oppressed community can promote empathic understanding.
Keywords: Ethical VR, Experiential Storytelling, Serious Play, Empathy Machine
Read an abbreviated sample of the chapter…
Errata (Chpt 2, Pg 28, Para 1, Sent 3): “Performing an avatar role focuses player attention on the character’s experience and can lead to investing in their relationships with others in the scene, which helps to prevent ego-oriented distractions and both immersion and flow for character actors (Maraffi, 2021).” Should be: “Performing an avatar role focuses player attention on the character’s experience and can lead to investing in their relationships with others in the scene, which helps to prevent ego-oriented distractions, and encourages both immersion and flow for character actors (Maraffi, 2021).
Summer 2022: USCB-NEH Reconstruction-era Institute Presentation “Mitchelville XR Tour”
July 14th 2022, I presented a talk and slides named Using AI and 3D Tools to Visualize History, which featured design work on our NEH-funded Mitchelville XR Tour research project.
Summer 2022: DIGRA 2022 Conference Paper “Multimodal Framework for Enhancing RPG Playfulness through Avatar Acting Affordances”
July 9th 2022, I presented a peer-reviewed extended abstract of my work-in-progress SGRplay framework for analyzing RPGs and XR storytelling experiences to the 2022 Digital Games Research Association conference proceedings. Here are a PDF of the slides for my talk.
Summer 2021: Cinergie Journal Article “Stage Magic as a Performative Design Principle for VR Storytelling”
August 4th 2021, I published an article in a special VR Storytelling issue of the international journal Cinergie:
Abstract: This article examines The VOID’s Star Wars: Secrets of the Empire (2017) VR arcade attraction, and analyzes the intermedial magic principles employed by co-founder and magician Curtis Hickman to create the illusion of a fictive world with impossible space and liveness. I argue that The VOID (Vision of Infinite Dimensions) functioned like nineteenth century magic theaters run by Georges Méliès and others, by employing magic principles of misdirection that directed player attention towards the aesthetics of an illusion, and away from the mechanics of the effects generating technology. Narrative framing and performative role play transported multiple players into a believable Star Wars immersive experience, creating an aesthetics of the impossible that reflected the goal of many stage magic tricks, and was foundational to trick films in the cinema of attractions of the early twentieth century. Using game studies concepts like Huizinga’s magic circle and theatre arts concepts like Craig’s über-marionette, this article suggests that The VOID and other stage magic approaches to VR, like Derren Brown’s Ghost Train (2017), are a new medium for participatory theatre that incorporate immersive features from both cinema and games.
Keywords: VR Magic Circle; Impossible Aesthetics; Immersion; Space in VR; Liveness.
Citation: Maraffi, C. (2021). Stage Magic as a Performative Design Principle for VR Storytelling. Cinergie – Il Cinema E Le Altre Arti, 10(19), 93–104. https://doi.org/10.6092/issn.2280-9481/12234
Download the full article from Cinergie…
Fall 2020: Artificial Creativity 2020 Conference Presentation
Nov 19th 2020, Before popular generative AI art tools like Midjourney there was a GAN-based online tool called Art Breeder that was used to “breed” images with “gene” features. This 2020 Artificial Creativity Conference (hosted by Malmo University, Sweden) presentation titled “Sherlock Frankenstein: Transmedia Character Design with AI Breeding Tools” talks about previous work using Art Breeder to generate preproduction character designs that incorporate human actor style features and touches on some of the ethical implications:
Fall 2020: Mitchelville and MODS projects Featured in FAU Research Magazine
Our National Endowment for the Humanities and Epic Games Megagrant funded research project Mitchelville AR Tour, as well as our Prehistoric Florida project for the Museum of Discovery and Science, were featured in FAU’s Owl Research & Innovation Fall 2020 issue, on pages 19-21:
Spring 2020: Ethical XR Symposium
On Feb 21st we hosted an Ethical XR Symposium with SCMS Multimedia Studies faculty at Living Room Theaters on the FAU Boca Raton campus. Magic Leap participated in the demos with MTEn MFA students, and talks were given by media scholars from around the country, including Harvard, NYU, and Columbia (photos by Topher Maraffi and Nicole Morse):
Spring 2020: Research Encounter Event at FAU Boca Campus
FAU Media, Technology, and Entertainment students in my XR Immersive Design course showed their research project prototypes at Research Encounter 2020 on Feb 12th. Hosted by the FAU Division of Research, we conducted demos of several projects on the Magic Leap and Oculus Quest headsets. FAU President John Kelly stopped by to see what we have been working on, and there were many attendees from the university and Boca community (photos by Brandon Martinez and Topher Maraffi):
Summer 2019: FAU-MODS Apptitude AR Course
FAU SCMS and Computer Engineering faculty taught a STEAM (STEM+Art) summer school course for high school students that focused on designing and developing augmented reality (AR) apps for research posters. This course is part of the Fort Lauderdale Museum of Discovery and Science (MODS) Apptitude internship program, and runs for three weeks in the summer at FAU Boca campus. Students learned how to create an AR app in Unity game engine that displayed green-screen video and animated 3D models on their research posters for museum exhibit topics.
Students working in class and the Freshwater habitat group poster:
A video of MODS project demos (by grad TA Richie Christian) and final App-titude AR summer school poster presentations:
Spring 2019: FAU MTEn Grad Research Day Posters
Media, Technology, and Entertainment (MTEn) MFA students in my Interactive Interface Design (DIG6605) course presented research posters at the 10th annual FAU Graduate and Professional Student Association (GPSA) Research Day. Pictured are clockwise: Raven, Monisha, and John Horvath: Alex Henry; Monisha Selvaraj; Camilo Morales; Brandon Martinez and Alberto Alvarez; Alex; Raven Doyle.
Spring 2019: FAU MTEn-XR Lab Open House
We had an open house in our Media, Technology, and Entertainment (MTEn) Extended Reality (XR) lab for potential MFA students. We demo’d our Magic Leap, Oculus Rift, HTC Vive, and Noitom Perception Neuron mocap technology:
Spring 2018: USCB Film & Digital Media Symposium
We had the second USCB Film and Digital Media Symposium in partnership with the Beaufort International Film Festival, with a Sea Island Center gallery reception, lightning round talks from USCB faculty and BIFF filmmakers, and a keynote talk by The Simpson’s director Mark Kirkland.
“Hopscotch Joiner” (55”x24”), Digital Imaging course compositing demo of applying David Hockney’s joiner compositing technique using digital media to create a seamless panorama scene, Spring 2018.
“Self Portrait” (32”x24”), Digital composite print on canvas, and then layered with acrylic paint, Spring 2018.
Fall 2017: USCB Faculty Exhibition
In November, see the latest artworks by USCB Fine Arts faculty at Sea Island Center gallery on the historic Beaufort campus. Here are some pictures of our opening reception on Thursday November 2nd. The show featured two of my mixed-media canvases and digital imaging studies, as well as a poster and teaser video on my documentary project.
Artist Statement
This year I am showing three creative works and related practice-based media arts research that began as class demonstrations in my 2017 USCB courses.
“Persistence of Vision: Coasting Ahead, Hitch Throws Hopper the POV Sign” (42”x26”), Media Design digital compositing demo with samples of Alfred Hitchcock and Edward Hopper imagery, printed on canvas and layered with acrylic paint glazes, Fall 2017.
The piece displayed in the main gallery entitled Persistence of Vision: Coasting Ahead, Hitch Throws Hopper the POV Sign was started as a compositing demo in Adobe Photoshop for my Fall 2017 MART B102 Media Design course. The POV theme refers to the nineteenth century concept of the same name that gave birth to cinema. The term described the visual phenomenon that a sequence of images played at a sufficiently fast rate creates the illusion of continuous motion. More conceptually, POV is being used to reference the evolution of a visual aesthetic that was first introduced by Renaissance painters experimenting with optical devices, evolving to the techniques used in chemical photography and cinematography, and very much employed by digital artists today. The compositing process I demonstrate in my course uses the same techniques Renaissance painters first developed in the Fifteenth century; staging a scenic narrative from a variety of image sources, modeling the curvature of forms with high contrast lighting, and using perspective lines to create the illusion of depth.
In addition to exploring concepts related to the POV theme, students were required to research at least one cinema artists and one fine arts painter that relate to each other, such as Alfred Hitchcock and Edward Hopper, and then digitally compose an image that incorporated visual and style references from both. My demo piece referenced how Hopper was inspired by early cinema, often going to the movies for weeks at a time, and how Hitchcock staged scenes that echoed the aesthetics of a Hopper painting. Source elements were significantly processed through digital filters to create a unique aesthetic, and the Hunting Island lighthouse was incorporated to relate the scene to Beaufort. Then I printed the digital composite on canvas so that I could hand paint acrylic glazes onto the surface, adding a final layer of texture and color depth to the piece.
“Lab Monitor” (36”x28”), digital “joiner” collage printed on canvas and enhanced with ink, colored pencil, and acrylic paint, Summer 2017.
The mixed-media piece in the hallway entitled Lab Monitor, with related studies and research poster, was also started as a class demo inside of Adobe Photoshop. However, the work went through a more complex process of alternating between digital and physical media in my MART B145 Digital Imaging course. We initially used digital photography to crowd-source images of a scene in our lab where I did a simple action of rotating in my chair. Students used the photos to explore David Hockney’s “joiner” collage technique, where photos are treated like brush strokes to create a Cubist representation of motion in time and space, but inside of Photoshop. This allowed us to process source photography through paint filters to make them more compatible with physical media, and then to print the final result on matte paper to enhance with ink, colored pencils, and paint. The next stage of the project involved using the prints to create texture maps in Blender 3D software and model a digital representation of the original scene. We then imported the scene into Unity game engine to make a “joiner” game where we could interactively create Hockney style collages through a photography play metaphor.
“Study for Lab Monitor” (16″x20″), digital animated “joiner” in Blender 3D software.
I composited several of these game-generated collages to create the final image that was printed on canvas, and then layered with physical media to add the hand of the artist back into the work.
“Jumpin’: SC Roots of Swing Dance Documentary Teaser” (30” Mac), Digital video loop, Summer-Fall 2017.
The Jumpin’: SC Roots of Swing Dance video teaser and research poster shows work being done related to my MART B250 Broadcast Design course. This project involves using the latest digital filmmaking techniques, including visual effects and motion graphics, to create a PBS-style feature documentary on how swing dancing in the early twentieth century originated from the Gullah-Geechee culture in Beaufort County. The pre-production research displayed in this work was conducted over summer and fall 2017 with a Sea Island Institute grant, in partnership with SCETV, including storyline and visual development for motion graphics, backgrounds for green-screen interviews, 2.5D visual effects, and lower-third graphic designs.
“Research Poster for Jumpin’ Documentary” (48”x36”), Digital print on paper, Summer-Fall 2017.
Categories: Event
Topher Maraffi 