Sam Rolfes. Interaction in the Metaverse
New York-based artist Sam Rolfes is one of those personalities who best represent the concepts of worlding, combining a personal database of influences with an incredible application of new technologies and aesthetics. Moving from graphic design, experimental music, and 3D animation, he merges everything into his experimental VR performances, where spatial choreography plays a fundamental role. Alongside his team, Team Rolfes ⎯ founded with his brother Andy ⎯ he realized projects and collaborations with Danny L Harle, the collective House of Kenzo, Lady Gaga and A.G. Cook, often portrayed in festivals, galleries and experimental events all over the world. I had the pleasure to talk with Sam about his creative process, his unique set of compositional/composite elements, and his latest project.
In the last few years, the increasing interest of the world of visual and sonic culture in the creation of a collective amalgam of disparate and affective elements brought the art scene to an unprecedented point of awareness. As I explained in previous articles, the construction of complex systems of memories, theories, and visual reminders, that doesn’t belong to a unique narration but rather as a deconstructed collective structure of lore, has become the root of contemporary artistic practice. Worlding finds the perfect field of experimentation in virtual space, in which affective objects collide in a sandbox for new worlds. Artists converge theoretical aspects, pop-cultural references and digital experimentations in hybrid and all-encompassing ambiences, rather than classical artworks.
My first approach to the work of Rolfes was through the video for the track AS Chingy, by experimental electronic music duo Amnesia Scanner, where an uncanny club-like scenario melts over syncopated lights and distorted surfaces collapse as the beats kick in. The eerie spaces, filled with overly saturated creatures and textures whose movements are liquid and unwieldy at the same time, become the core of Rolfes’s practice. The camera moves in the space like an FPS game, and it deconstructs and reshapes the materiality of objects through its explorations. Coming from a background as a painting artist, he has been able to translate his personal graphic trait into a series of twisted, organic shapes that spread all over most of the elements present in his virtual worlds.
However, Sam Rolfes’ distinctiveness lies in the approach to VR performance and the use of the bodies’ materiality. The process focuses on the relation between virtual and real objects, where spatial choreography and movements work as narrative elements. Performers are mutated into digital puppets with visceral features by using motion capture suits and instruments, while interaction with real elements translates into surreal contortions on screen. Everything becomes narration in Sam Rolfes’ projects, whether the final result is a video or a live event.
While talking about his workflow for creating music videos, he explains that he starts by sculpting a virtual space that could evolve in a narrative arc. The use of editing is kept to a minimum, in favour of a one-shot approach. In this way, he is able to focus on the movement throughout the space itself in relation to the music. Sound plays a fundamental role in his works. A notable example is the video for the track Bunny’s Dream, by artist Matthew Dear, in which Rolfes manipulated a dismembered bird-like creature and its prey, by treating them as virtual puppets through the motion capture of his hands. As the music proceeds, the hunt/play between the two beasts becomes more and more intense, also driven by the shifting scenery and the floating camera performed in first person.
Speaking of performances, the live experience brought by Sam Rolfes works on a parallel but unique level at the same time. Every performance is divided into different acts, formed by individual scenes usually interspersed with DJ sets; the main aspect that is taken into consideration is the role of the bodies related to space ⎯ both virtual and real. Each scene has its own purpose and belongs to its own narrative arc in the created virtual space. The performers work not only as mere actors but more importantly as elements of reconfiguration of such space. The movements and choreography are always unique and independent in every show, and thanks to the motion capture puppetry adopted by Rolfes, they are capable of reshaping the world in which they act. In this way, the combination between the real performative space and the virtual set created by Rolfes is in perpetual evolution. However, when it comes to the theme of worlding, these micro-universes and the creatures that inhabit them become a core feature. The virtual framework in which everything happens is sometimes an amalgam of concepts, models, and textures taken from previous assets made by Sam Rolfes and his team. Some of the same elements are used multiple times, helping to create a sort of leitmotif in the different performances. The worlds could change their shapes, but the objects might remain the same ⎯ even with some modifications and reworkings. This is at the heart of world-building and the concept of database, in which single elements are as valuable as the world that they inhabit. Sam Rolfes’ peculiarity is moreover represented by the vastness and undefined nature of his creative output. Every medium melds with another, in a vibrant and viscid result of dancing flesh, mechanical bodies, astonishing sounds, and saturated visuals. In addition, the interactive and collective nature of the performative medium represents the perfect example of the all-encompassing component of world-building.
For Sam Rolfes, creating a dialogue between pre-existent elements coming from different fields is more intriguing than creating them from scratch. This collage practice comes from a turntablist-like approach, explains Rolfes. It follows the modus operandi of game studios that use 3D models multiple times for different projects. It is not hard to see common traits in his performances: during his residency for FACT Magazine in 2020, Sam realized a series of virtual tours ⎯ using a chibi avatar with his feature rendered on a Majora’s Mask-like head ⎯ throughout the worlds he created for different collaborative projects, in which he highlighted common features found in such spaces. In the Morbid Angel performance, realized with the multidisciplinary collective House of Kenzo and artist Rabit for the “Day For Night” Festival in 2017, an enormous, gnarled angelic creature walks stubbornly in front of a warped version of a record shop, accompanied by a giant moth. The decayed virtual world is filled with reused assets and models previously created by Rolfes for other projects of various nature: artworks for electronic music albums, shots from fashion films, and music videos running on screens scattered around the area. The consolidation of a personal aesthetic is central to Sam Rolfes’ work. Among his influences, he quotes set designer Robert Wilson, praising his ability in creating an immediately recognizable stage in almost all of his productions.
Speaking with Sam helped me to delve into his universe and better understand what moves him into experimenting with performance and digital technologies. He also quotes the traditional Japanese puppet theatre bunraku as a form of inspiration in his use of virtual model doppelgangers. The way he manipulates the creatures on-screen with movements that are barely human draws from the puppet handling operated by the performers in the theatre practice. What is usually hidden behind the stage, away from the audience, is the hand that moves the marionette. Rolfes overturns this concept by mimicking the bunraku theatre practice of showing the puppeteers directly on the scene and exposing the motion capture process in real-time. Sam’s interest lies in the possibilities given by a body movement translated into a virtual space. Every action and surface could be mapped in a completely abstract way, allowing for an infinite set of moves in which the body could exceed its own capabilities. The movement of the knee could be mapped to the extension of the neck of a performer, or the turn of an ankle could morph the entire body into a different creature. Games also play a fundamental part in Rolfes’s creative process, starting from the basic use of game engines to create his worlds to the principles of game design to build the narrative arc of his projects. The Unreal engine has been used in several works by Sam and his team, starting from the AS Chingy video to the most disparate outputs ⎯ like the talk Absurd Arms at Sonic Arts festival in 2018, in which he combines a presentation with a performative act.
Moreover, during our interview, he emphasized the influence that some games had on the way in which he designs the exploration of a virtual space. In particular, he quotes the sci-fi FPS Half-Life 2 as a major inspiration for the camera activity in his works: the possibilities of rotating the first-person point of view of the playable character drove Sam to create an analogue system for the investigation of his metaverse.
But the apex of Rolfes’s interest in game design is reached by his latest and upcoming project, the realization of an experimental music physics game through a fundraising campaign. He explains that the game will be a combination of the well-known physics sandbox Garry’s Mod and the Nintendo minigame series WarioWare. The project is to be intended as an attempt to translate Sam Rolfes’ distinctive performance acts, enhancing the collective feature of his work in the medium of gaming as well. Thinking of Garry’s Mod, the final project could be understood as a free space of creation based on the physics of the models present in it ⎯ models certainly based on Team Rolfes’ aesthetic. Nonetheless, the real deal of this project belongs to the interaction with the audience: in fact, the purpose of the game is to share a substantial part of its development with the community through the fundraising campaign based on the selling of digital art pieces. This will allow the donors to own elements from the game and to contribute to the creation of sceneries, lore, and characters through a continuous exchange with Rolfes and his team, in an attempt to make players part of the final work.
Sam Rolfes represents one of the most intriguing personalities emerging in today’s contemporary art world. His ability to melt different media ⎯ from sonic arts, choreography, graphic design, to mocap performances ⎯ makes him capable of creating unique metaverses. The distorted imagery, twisted aesthetic, and broken space design brought by him and his studio in their interactive performances best represent the concepts of world-building, not only on a practical and visual level but also on a conceptually. Attending one of these experiences is an overwhelming and unique event, leading the audience to become an integrated part of what Sam is building in real-time, once again erasing the border between artist and public. An alpha version of his game is coming in the next few months, so we will just have to keep an eye on Rolfes and his astonishing work.
