Rhythmic Trajectories - Visualizing Cinematic Rhythm in Film Sequences

Rhythmic Trajectories - Visualizing Cinematic Rhythm in Film Sequences

Szilvia Ruszev
Rhythmic Trajectories is a series of short études set to accentuate, visually and sonically, rhythmic elements of specific film sequences. As a film editor and researcher, I am interested in revealing and visually expressing cinematic rhythm. I am following hereby Karen Pearlman’s idea stating that “the functions of rhythm are to create cycles of tension and release and to synchronize the spectator’s physical, emotional, and cognitive fluctuations with the rhythms of the film.” (Pearlman 2009: 61) These are the rhythmic trajectories that constitute a sensuous, kinesthetic knowledge about a film. In my work, I am interested in visualizing these trajectories, adding a layer of a visual ‘close reading’ to the given film sequence while preserving the original sequence. My research project is situated at the intersection of information visualization, digital humanities, and artistic practice and follows a mixed method approach. The essay consists of two visualizations and a written part that is meant to highlight the research questions, to situate the research within the different fields and to provide insight into the creative process of developing the first two visualizations. The first étude uses a sequence from the short documentary Wagah (Supriyo Sen, 2009, Germany/India), edited by the author, showing the bizarre choreography of the flag-lowering ceremony on the Indian-Pakistani border. The video essay examines how rhythm can be built out of different elements such as sound and movement, colour, graphical structures, cuts, etc. The rhythmical structure has been deconstructed and formalized to reach an abstract notation and to represent a movement. The second étude focuses on the rhythm of gestures using a scene from the film A Woman Under the Influence (John Cassavetes, 1974, USA), edited by David Armstrong, showing the main lead, Gena Rowlands, wildly gesticulating during a fight with her husband in the film, Peter Falk. This is a ‘close-up’ of the gestures that are defining and leading the conversation. Using a similar visualization method, gestures are transposed into colourful meandering lines.
John Cassavetes; Szilvia Ruszev; Supriyo Sen; Editing; rhythm; movement; gesture; visualization; artistic research.



Biomechanics by Vsevolod Meyerhold: “The performance of ‘The Arch’” (http://pismowidok.org/index.php/one/article/view/196/355).

Rhythmic Trajectories is an artistic research project designed to develop visual and sonic representations of cinematic rhythm. At the basis of this exploration is the understanding of movement as the inherent specificity of cinema — the illusion of coherent movement based on the projected sequence of distinct frames; the illusion of a coherent cinematic time and space assembling separate shots. The synergy of the composed audiovisual elements of cinema results in a rhythmic structure that I call cinematic rhythm. This research aims to examine the elements forming this cinematic rhythm.

This project uses visualization to deconstruct and formalize the elements of cinematic rhythm. Rhythm and, specifically, cinematic rhythm has been addressed by different authors in the context of film studies – Ken Dancyger, Karel Reisz, David Bordwell and Kristin Thompson – just to mention a few. It has been generally tied to the temporal specificity of cinema and especially to the formal categories of film editing such as timing, pacing and frequency discussed and reviewed in depth in Karen Pearlman’s book Cutting Rhythm, Shaping the Film Edit. In the words of another theoretician: “... rhythm is a kind of dialectic of time rather than a continuity whose intermittent variations distort for us the normal flow of time.” (Mitry 2000: 104)

For the purpose of this essay, I will define cinematic rhythm as the formal characteristic of a complex and systematic, temporally and spatially structured audio-visual sequence that contains meaning. Cinematic rhythm and its visualization concerns meaning which can be sensed beyond (but not instead of) the narrative and linguistic realm of a film. Cinematic rhythm contains what we can call embodied, kinesthetic knowledge as an indispensable part of a film experience.

The practical part of the research consists of a series of short études relating to different aspects of cinematic rhythm. I design and add these aspects as a visual and/or auditive layer to the original film sequence. This artistic research project builds on my knowledge as a film editor. It reflects and visually contextualizes the mostly intuitive work done by a film editor, while also tying that work to the cognitive process of experiencing an edited film. The research interest of my project is to define what cinematic rhythm consists of; to identify the formal elements to which cinematic rhythm can be disassembled and finally to develop a visual representation to each of these distinct elements. In order to do so, it is important to note that there are different levels on which cinematic rhythm can be discussed – inside one shot; on the cut, at the juxtaposition of two shots; in a sequence as the connection of several shots; or on the narrative level of the film, where the structural element is a scene (or character). Karen Pearlman talks about physical, emotional and event rhythm (Pearlman 2009: 84) These categories are pointing toward a typologisation of rhythm based on the content of the film sequence and how these different types are experienced by the spectator. (A more detailed discussion follows.) The concept of energy, the ideas of tension and release, the structural meaning of repetition and iteration are all an important aspect of the work of visualization to be presented. It ties back to the questions of what cinematic rhythm consists of, and how movement (be it physical, emotional or event) as the inherent specificity of cinema is able to create a sensuous knowledge.

The études are short experimental studies set to develop and iterate formal tools and aesthetics (such as animation using Adobe After Effects or computational approach using the software Processing) to visualize cinematic rhythm as part of the research process. Using the methodological approach of the étude understood as a simple exercise set to develop in its iteration a complex idea situates it in the tradition and constructivist context of Meyerhold’s biomechanics (see Fig. 1) and the montage experiments of Kuleshov. The études are non-destructive, close readings of film sequences (these concepts will be explored in depth later in the text) with formal relevance in the context of cinematic rhythm. Interestingly, the method of adding a visual and/or auditory layer to the original film has opened another field of inquiry — keeping the original sequence intact turns the added layer into an interpretation that interacts in turn with the original sequence. A friction evolves from the relationship between the original and the added layer that is of special interest for my artistic research. One possible question that arises from the études is: What happens at the contact point of friction between those two layers, what kind of sensuous knowledge arises from the interpretation that originates at the edge of the friction?

As a research project, Rhythmic Trajectories can be situated at the intersection of three loosely related fields that shed light on its methodology and its research question. In the following part, I will provide a short overview of the fields that constitute the theoretical background for my artistic research: 1) information visualization, and more specifically film annotation and visualization, where I will refer to Lev Manovich’s definition of this term; 2) the theory and practice of film editing with a special focus on cinematic rhythm, especially the influential work of Karen Pearlman, Cutting Rhythms, Shaping the Film Edit, which deals with the different forms of rhythm within film that are created and shaped by editing; and 3) visual arts and especially artistic research, pointing toward examples and different approaches in both fields.

Film Visualization

A visual fingerprint of the film Soliaris / Solaris (Andrei Tarkovskii, 1972, USSR) by Frederic Brodbeck, Cinemetrics (http://cinemetrics.fredericbrodbeck.de/).

The primary goal of Rhythmic Trajectories is to find visual and aural representation for cinematic elements (within the frame and on an intra-frame level as well). This is done to emphasise, formalize and visualize data present in the film sequence. In this context, my research can be situated in the field of information visualization and more specifically film visualization. Lev Manovich defines information visualization as “a mapping between discrete data and a visual representation. […] if we believe that a brain uses a number of distinct representational and cognitive modalities, we can define infovis as a mapping from other cognitive modalities (such as mathematical and propositional) to an image modality” (Manovich 2010). Cognitive modalities can be understood as different models of how we make sense of the world. Each modality has its own structures of representational ‘tools’ and affordances. The term ‘mapping’ used by Lev Manovich points beyond the meaning of the term ‘translation’ where both objects (the translated and the original) stay in the realm of the same representational mode of, for example, language. In the context of visualization, mapping can be used as meaning transposition – moving from one representational structure to another (such as from language to visual, or mathematical to visual). The correspondence between the two modes, as opposed to translation, can be much more complex and indirect.

Film itself is multimodal – it is visual, aural and linguistic all at the same time. Looking at the history of visualizing films, its primary goal is an analytic one and connected to digital humanities, film studies, and film-historical approaches. As these studies have always been text-based at their core, this approach has been limited by its own boundaries. Language turns out to be insufficient when it has to represent complex audiovisual entities. The mere description of categories such as shot size, shot length, content, and sound can serve as a transcript but it is never capable of representing temporality and the complexity of the ways audiovisual entities generate meaning. The necessity of visual correspondence to be used alongside the argument in these studies is not questioned anymore. The work of authors such as Catherine Grant1 and Kogonada2 has legitimised the form of the video essay in academic discourse. Nevertheless, there is still a discussion about the legitimacy of visualizations as a stand-alone (or non-linguistic) argument.

Visualisation of shot lengths of the film Odinnadtsatyi / The Eleventh Year (Dziga Vertov, 1928, USSR), created by Lev Manovich. https://www.flickr.com/photos/culturevis/8349178594/in/album-72157632441192048/)

On the other hand, there has been a multitude of analytic approaches that are focusing on visual modes without building on the linguistic meaning in the way the video essay does. The project Cinemetrics3, for example, initiated by Yuri Tsivian is based on a measurement theory relating to shot length and using this as the data point for a further analysis. Frederic Brodbeck’s visualisation project4, titled Cinemetrics as well, utilizes quantitative measurement theory in order to create a so-called visual “fingerprint” of a film. In her forthcoming book Digital Humanities and Film Studies. Visualising Dziga Vertov's Work, Adelheid Heftberger (2019) discusses the research project conducted in cooperation with Lev Manovich, situating it at the intersection of digital humanities, film studies and information visualisation. The visualization combines quantitative and qualitative approaches in visualizing such attributes as shot length on the one hand and conducting a visual shot analysis on the other hand.