THE PHYSIOGNOMIC (UN)GENRE: Challenges of Automated Facial Expression Analysis-Based Media Art to both the Art and Science of Face

Abstract

As 9/11, Facebook, and WikiLeaks level events forever change the media climate, the past-perfect promise of a newest harnessed methodology for face datafication and emotions computability inspires in the cultural imaginary an unparalleled–if not unprecedented–‘physiognomic frenzy.’ To tactically critique this ‘face of the age,’ media artists increasingly utilize as both medium and subject Automated Facial Expression Analysis (AFEA). Yet, these proprietary closed-source algorithms, introduced by technological industry and expert science, are black box frameworks that veil program functionality input from available data output. That is, neither artist nor audience can see the way they work, including the very ‘electronic mugbooks’ and Facial Action Coding System that train the algorithm. Consequently, each media artist invents their method ex novo, uninformed of its intermedial genealogies from the contemporary science of facial expression as well as the historical art of physiognomy. Thus, these artists often significantly misrepresent the very science about face they explicitly claim to question. Historians of media art fail to trace essential correspondences in operationalized aesthetic and visual rhetoric between AFEA-based media artworks. And data affordances from media art to face science are artificially constrained. Through a scientific connoisseurship of in-the-field training and targeted interviews, I probe artworks by Julius von Bismarck, Paolo Cirio and Ludovico, Ishiguro, Lev Manovich, and Marnix de Nijs. Proposing what I call a "physiognomic genre," that bridges database, hactivist, installation, net, and robotics art, I problematize how media artists reflect today’s face literacies and emotional competencies–how we 'think' about what we 'feel.'