This blog post is about one of the three scores from my trio ||: trouble letting go :|| – ]HoldingOn[ – 4 Echoes: whistle, whisper, gasp, silence composed for violin and objects. The trio is a collection of three pieces that are intended to be simultaneously performed together and was written for the London-based concert series 840 for their ‘New Music for Violin and Objects’ programme. In this blog I reflect on some developments in my work with video scores in the piece ]HoldingOn[. 

---

video score for ]HoldingOn[ / instructions for reading the score

Background
]HoldingOn[ is my second self-published video score and extends upon some of the previous work I was doing in this is not natural, a trio for French hornist, pianist, and contrabassist.[1]

To briefly recap the work I did in this is not natural: the score for this is not natural is made/composed from edited video-documentation of pre-recorded performances of a choreographed sequence of movements lasting fifteen seconds. Inspired by Bill Viola’s Quintet of the Astonished, large portions of the original performance documentation were extremely slowed down to fill the duration of nine minutes. Frames from the original footage are spread out across larger distances, breaking the illusion of apparent motion, and through a process of interpolation, new frames are generated to fill the space in-between original frames and render the movement/footage smooth.

For a live performance of this is not natural, performers are instructed to watch a video of their technologically dilated bodies and attempt to reproduce ‘their’ movements.[2] Because the score/video has been rendered in such a way that the interpolation between each original frame is (considering the degree of stretching) relatively very smooth, movements subtly morph into each other and locating the exact originating points of movements can sometimes be difficult. In search for minor shifts in movement that the performer can register and translate from sight into a felt movement in their body, the eyes are rapidly, and crucially, continuously scanning across and around the surface area of their screenicly displayed bodies. As a reading experience, a performer’s eyes are captivated by — always looking for difference in — the video.

Expanding Modes of Engagement with Video Scores
Aside from compositionally making decisions about the degree to which video is stretched, changing color footage to black and white to visually sharpen the contrast of the image and reduce visual distractions (perhaps also enhancing the “reality effect”[3] of the transfigured documentation), and occasional overlaying a tint of transparent light blue on the horn part to indicate air passing through the mouth of the performer, the score for this is not natural is basically unchanging in the way that a performer engages in the process of reading their bodies. Another way of putting that is, the eyes are effectively always moving around the screenic display/image in the same way, looking for and processing the same kind of information. In ]HoldingOn[ I expand the ways that the eyes are directed around the screen/image and the kinds of information required to process.

]HoldingOn[ continues to predominantly explore the use of variable playback speed, still focusing on slow motion movement, but expands the dimensions of the video score in the following ways:

• Videos from different sorts of ‘performances’ are layered,
• A more intuitive use of background tints is employed to ambiguously signify information about changes in the speed of performance/video-playback, granularly indicate local-level information about points of contact between the performer’s body and the instrument along with information about the amount of pressure/force applied to the instrument, and cause the eyes to shift focus from one part of the screen/image to another,
• Minimal amounts of graphic and textual information are included,
• The overall trajectory of movement is broken up by jump cuts,
• And instead of interpolating newly fabricated frames between original frames post-time-remapping to create smoothly morphing movements, the performer is instructed to interpolate the movements between variably spaced frames of movement.

Spaces of Documentation and Performance
In addition to these expansions of representation, I tried a different methodological approach towards documentation. Instead of producing the video documentation myself, enacting a kind of performance with the camera, producing a theatrical document,[4] I partially ceded the responsibility of documentation to the performer. With this is not natural, the space of performance was insignificant, but in ]HoldingOn[ I was interested in the spatial/personal significance of the video documentation and the affect it may have on the performer during the act of reading the score. For this reason it was important that the performer have a personal connection with the process of producing their recordings.

To guide this process I emailed Ruben Zilberstein, the violinist I was working with to make this piece/score, and prompted him to make his own video recordings by following the instructions listed below. The videos were made with the help of Alex Nikiporinko and can be viewed by clicking on the link before each prompt:

Hi Ruben,

Could you send me some video of you tuning?

I’d be interested in having three videos, all shot at the highest frame-rate you possibly can. 

The audio quality is not important, but please do record audio with the videos.

Video One: A short close-up video of the side of your heels on the ground as you tune in an unfamiliar (preferably large) space. The duration is dictated by the amount of time it takes between the beginning of tuning and the point at which you feel your heels being lifted up (or rather: you feel lighter and “picked up” by the resonance with the space).

Video Two: A 5 minute full-body video of you tuning in a familiar place/space.

Video Three: A 5-7 minute close-up video of the entire span of your violin’s strings, and both hands while tuning in an extremely small and claustrophobic-inducing space unlike any one that you have previously tuned in for these videos. During the course of this video, be sure to play some full bodied/force 3- and 4-string chords. Also, be sure to make contact between your left-hand fingers and the fingerboard on occasion during this video. Finger contact onsets should range from delicate to extremely aggressive in force/pressure. 

Happy New Year,
Michael

For ||: trouble letting go :|| – ]HoldingOn[ – 4 Echoes: whistle, whisper, gasp, silence I was interested in the idea of superimposing multiple spaces of performance. Effectively, each piece of the trio is withdrawn into its own world, only on occasion incidentally relating to each other. Prior to the correspondence above, Ruben and I were in conversation with each other. We talked a bit about the experience of being in performance, isolation and the interior space of performance, and how he ‘felt’ physical spaces through tuning in/to them. For ]HoldingOn[ this interest in multiple space is manifest in my instructions to make videos of tuning the violin in different spaces, and to, inside an entirely different space, watch the video score and reproduce the bodily responses to those tunings post-video-editing/processing.

Ideally, the spaces of performance from the videos transport Ruben to another combined/multiple/virtual space during performance and fragment his body, simultaneously projecting qualities of openness and tightness. To achieve this, portions of videos one and three from above were used. Video two ended up serving more as a reference of Ruben’s body in general as I made the score. For me, it didn’t carry enough significant spatial significance to warrant inclusion in the score.

Hearts that Eyes Pulse / Feeling what Eyes Hear – Listening, Interpolation, and Pulsation
Between making this is not natural and starting work on ]HoldingOn[ I conducted a workshop with Franc Chamberlain, Hilary Elliott and Eilon Morris – three members from the drama department at The University of Huddersfield – to devise another performance of the this is not natural by working with residual documentation from the production of the piece and its first performance.[5] Afterwards, I interviewed Franc, Hilary and Eilon to gain some insight into their experiences from the workshop. Responding to a question about whether or not they felt as though the act of listening was driving their movements, Hilary remarked:

“…although I listened with my ears, I ‘listened’ more with my whole body, by which I mean I let the field of my attention spread out; listening became a whole-body receptivity to Eilon, Franc and my own physical/sonic presence. Listening in this way is a kind of sinking in through the skin to the entire surrounding.”[6]

While I was working with the video documentation that Ruben and Alex made, I was more sensitive to this idea that listening, as a mode of attention/receptivity that spreads outwards from, and then back into, the skin, wasn’t exclusively limited to organ of the inner ear, or more specifically, the cochlea. The body could listen, and importantly for my development with video scores, the eyes could listen.

Where this became most significant for me was when I was experimenting with variable speeds of playback. At certain points, the time stretching I was doing would be so extreme that instead of smoothly interpolating new frames between the original frames, an original frame would simply freeze before moving onto the next original frames. I’m not exactly sure why this was happening, but nevertheless I was intrigued by the almost pulsating results. Curious to know what it might be like to copy movements without smooth playback, I tried to imitate the movements and fill in the gaps myself. While doing this I noticed a few surprising things.

The first was that it seemed easier to read the score – it felt more like the act of reading a notated score. I believe this was because my eyes were able to, generally speaking, anticipate how quickly there would be a change to a new position/frame and how far apart in physical space each position would be from one another. The eyes were not engaged in a process of continuously scanning the entirety of the image – they were not performing in the now, looking for miniscule differences in movement, but were instead reading ahead and playing catch-up. This is a lag in processing that allows the reader to perceive a noticeable change, quickly scan the image to take in important changes in positional information, and move to the position last seen. The performer interpolates the missing frames because they are retrospectively inferable.

The other peculiar thing that I noticed while I was imitating these freeze-frame movements was that my wrists, fingers, and arms – my limbs – had the sensation that they were pulsating. It felt as though there were hearts in my limbs. And this is because the freeze-frames were occurring periodically, much like heart pulses. These bumps or beats were divorced from bars, instead co-existing within trajectories of movement. My eyes were hearing pulses… hearing rhythm.

Throughout the score for ]HoldingOn[ the rate of pulsation changes subtly, and it’s as though each shift in speed can be felt as a shift in pulsation of hidden hearts. The full ramifications of this personal discovery are not developed very far in this score. This is because I did not find a compelling relationship between the video documentation I was working with as content and this sensation of feeling beats inside containers of movement. However, I believe it could be an interesting area to consider for future video scores.

Most scores I know that use performance documentation of the videography variety as content and screens as the display-medium for that content maintain smooth quasi-realistic movement of the bodies.[7] In some scores, movements are repeated and/or looped, there are jump cuts, or points of reversal create visual seams or breaks in the flow from one positional point to another. Whereas these seams, even if they have a pulsating recurrence, only create heard rhythmic iterations realised in performance, what I am proposing I have discovered with my most recent video score is: a way of feeling, through eyes that hear, rhythmic subdivisions of movement.

More Precise – Points of Contact, and Abstraction in Video Scores
The ability to precisely specify intricate, local-level micro/ancillary gestures is greatly compromised in video scores. This is, at least, for two reasons. The first is the translation of a three dimensional performance into a two dimensional representation. Although I am highly skeptical of rhetoric which dictates that documentation categorically entails lose, reduction or degradation relative to some ontologically original reality or iteration, I am willing to concede that a direct transferal of information from one medium (live performance) to another (video documentation) results in difference. This difference requires thoughtful consideration if video scores are to be taken seriously as an alternative form of notation. Considerations of what and how to enhance or marginalize specific aspects of the captured performance need to be taken into account.

The second reason that specificity and precision are difficult to convey when reading video as a prescriptive notation is because of the fact that, at least in my own scores, the process of reading is at least partially contingent upon having already previously produced the movements seen. I’ve described this reading contingency from the performer’s perspective, saying they are “…always in dialogue with how they know to move, how they are being shown to move, and how they remember moving.”[8] I think one solution to this problem resides in not attempting to strive for a realistic representation/presentation of what has been recorded and documented. The documentation can be used as abstract material. This is another reason why I think it may be easier to ‘read’ the score for ]HoldingOn[; the precise movements of the body are not as important as the general points/positions along a trajectory of motion. A performer is able to fill in the gaps and make the realization their own. Written forms of notation are incomplete, and this impartial nature of trans/in-scription is in part what gives notation is signifying power.[9]

But what options are there for enhancing the videographic image to more precisely indicate the specificities of the movements made in-between indicated points/positions of motion? And, how can those enhancements be made in a way that is easy to read in conjunction with the videographic image and do not rely on some prior experiential relationship with the body in the video score? I suspect there are several options, and as a part of my work in this field, it is my intention to continue to experiment and find ways of signifying information that a musician would find meaningful and useful when performing from a video score. For ]HoldingOn[ one approach was a variable use of semi-transparent background tints. Background tints play a prominent role in indicating where and when the eyes are focused on different parts of the screen, where points of contact between the performer’s body and the instrumental body are made, how much pressure/force is applied to the instrument by the performer’s body, and what the general speed of movement is at any given moment.

Out of these varied uses, the indication of contact and force/pressure seems the most interesting to me. Because the background-tints fade-in and -out from the background of the image, they have a smooth and continuous quality that contrasts the periodic pulsation of the freeze-frames. In a way, they work in tandem with the pulsating freeze-frames to help guide the quality of a performer’s interpolated movements. In order to allow the performer to focus on feeling their way through the score, I have attempted to make the varied uses of background-tints intuitively understandable. Rather than being swept up and away in the act of reading, I have attempted to design the score in such a way that they are able to interpret the score rather than solely reproduce the score; I’ve attempted to enhance a highly prescriptive form of notation with descriptive forms of signification.

I believe there is still a lot of ground to cover within this area of compositional activity, and am excited to continue and see how others are using video as material for their scores. Please see the full video score for ]HoldingOn[ above to see how all of the developments I have discussed above have been manifest.

The complete trio, ||: trouble letting go :|| – ]HoldingOn[ – 4 Echoes: whistle, whisper, gasp, silence, receives its premiere performance on March 5th, 2016 in London.

More information is available here: http://www.eightforty.co.uk/events/050316/

---

Endnotes
[1] Read my guest article for Tim-Rutherford Johnson’s blog, The Rambler, for background information on how this is not natural was made. Michael Baldwin, “Contemporary Notation Project: Michael Baldwin,” last accessed 29 February 2016, https://johnsonsrambler.wordpress.com/2014/07/18/contemporary-notation-project-michael-baldwin/.

[2] Perhaps interesting to note here is the writing of Carrie Noland in her article Motor Intentionality: Gestural Meaning in Bill Viola and Merleau-Ponty. In her article she discusses how she would physically (re)-enact the manipulated movements of the figures/performers in Bill Viola’s Quintet of the Astonished. She imitated their blocking, learned their roles, and copied large body movements (shoulders, hands, mouths) Yet, she notes, she was unable to produce through sheer will the “twitch of a facial muscle, or the trembling of a cheek.” She goes on to elaborate by saying that “[…Mark] Hanson is correct when he insists that new media technologies can expose to sight elements of aliveness… that are performed without volition.” Although there is an inability to completely reproduce the “elements of aliveness” exposed by the technological dilation, it seems interesting to me that something in the work of Viola has prompted another scholar and performer to set about imitating the movements of these videotaped performers/figures. It makes me wonder if there is something about extreme slow-motion movement that prompts mimicry. Carrie Noland, “Motor Intentionality: Gestural Meaning in Bill Viola and Merleu-Ponty,” Postmodern Culture 17, no. 3 (2007), last accessed 29 February 2016, http://search.proquest.com.libaccess.hud.ac.uk/docview/1429123697?accountid=11526. See also “‘The Quintet of the Astonished’ by Bill Viola (excerpt),” Vimeo video, 02:00, posted by “Urban Video Project,” https://vimeo.com/15130088.

[3] Philip Auslander, “The Performativity of Performance Documentation,” in Perform, Repeat, Record: Live Art in History, ed. by Amelia Jones and Adrian Heathfield (Bristol: Intellect, 2012), 49.

[4] Although ultimately rejecting these categorical distinctions, on the discussion of photographic documentation of performance art, Philip Auslander proposes  “that performance documentation has been understood to encompass two categories: the documentary and the theatrical.” (original emphasis) The documentary category ontologically connects performance and document, the documentation “provid[ing] both a record of [performance] through which it can be reconstructed and evidence that it actually occurred. [In the documentary] the event preced[es] and authoriz[es] its documentation.” The theatrical category encompasses documentation of performances that “were staged solely to be photographed or filmed and had no meaningful prior existence as autonomous events presented to audiences.” Auslander, “The Performativity of Performance Documentation,” 47-49.

[5] “this is not natural [transfiguration] – devised in workshop with Franc / Hilary / Eilon,” Vimeo video, 12:22, posted by “Michael Baldwin,” last accessed 29 February 2016, http://vimeo.com/110993970.

[6] Michael Baldwin, “Performers’ Insights into into is not natural [transfiguration],” CeReNeM Journal 5 (2015), last accessed 29 February 2016, http://cerenem.ricercata.org/articles/questions_performers/page06.html.

[7] An incomplete list of works I have come across so far that use video as a prompt for performance in some way include: Mário Del Nunzio’s Serenata Arquicúbica, Cleste Oram’s 8 x ∞, XEROX ROCK, rupture | rapture, and third person, Daniel Portelli’s Animal and Mapping Australia, Lee Chie Tsang’s Yu Moi, and to some extent, Andy Ingamells’ Bowmanship and Packaged Pleasure. See:

• “Serenata Arquicúbica (Mário Del Nunzio) [score – part 1],” Youtube video, 10:35, posted by Mario Del nunzio,” last accessed o29 February 2016, https://www.youtube.com/watch?v=XloUECmzzOw.
• Celeste Oram, “8 x ∞ (2015),” last accessed 29 February 2016, http://celesteoram.com/Works/8-x-2015-for-mixed-octet.
• “XEROX ROCK (2015) : a video score,” Vimeo video, 08:48, posted by “Celeste Oram,” last accessed 29 February 2016, https://vimeo.com/129306394.
• “rupture | rapture [score extract],” Vimeo video, 01:22, posted by “Celeste Oram,” last accessed 29 February 2016, https://vimeo.com/129306394.
• “third person : a video score (performance),” Vimeo video, 08:52, posted by “Celeste Oram,” last accessed 29 February 2016, https://vimeo.com/101925022.
• “60 sec video score preview,” Vimeo video, 01:00, posted by “Daniel Portelli,” last accessed 29 February 2016, https://vimeo.com/122970091.
• “Video Score (2014),” Vimeo video, 07:05, posted by Daniel Portelli,” last accessed 29 February 2016, https://vimeo.com/99456843.
• “Packaged Pleasure,” Youtube video, 05:26, posted by Andy Ingamells,” last accessed 29 February 2016, https://www.youtube.com/watch?v=ZXsJmPJPcVw.
• “Bowmanship,” Youtube video, 03:47, posted by “Andy Ingamells,” last accessed 29 February 2016, https://www.youtube.com/watch?v=Bkxwm6Ie_B4.
• “‘Yu Moi’ (2014),” Youtube video, 10:22, posted by “Chie Tsang LEE,” last accessed 29 February 2016, https://www.youtube.com/watch?v=V1HaV5JPSrY.

[8] Baldwin, “Contemporary Notation Project: Michael Baldwin.”

[9] It may seem from this move towards abstraction that I am contradicting my self-professed interest in the personal connection that a performer has with their documentation and the spatial significance of the performance documentation. And the truth is, I am. I am interested in the personal and spatial significance that Ruben has with the score for ]HoldingOn[, but I am also interested in the notion that the score could still be accurately read and interpreted by another performer without having any personal connection with space of documentation or the body being documented. This tension/conflict of interests is something that I am stilling thinking about and hope to find a resolution to in future work.