Kate Sicchio and Alex McLean

Over several years of discussing, collaborating, making and reflecting, Kate Sicchio and Alex McLean have worked on highlighting the relationships between code and bodies through sound and dance. The discussion below draws on their previous research, as well as their collaborative performance Sound Choreographer <> Body Code. In this work, a feedback loop between McLean’s live coding of sound and Sicchio’s movement based on generative dance score is created through the use of motion tracking and sound frequency analysis.

Embedded here is a version of the code, which will generate choreographic instructions based upon sound analysis through your computer’s built-in mic (Chrome browser recommended). The score begins slowly but accumulates complexity over time:

Sicchio’s order and number of repetition of gestures and direction of movement are all interpretations of the changing connections of the nodes and the spanning tree as seen below.

For Sicchio and McLean, this performance has led to many questions, which they discuss here: what makes code a score, what is notation, how is code interpreted within performance, and where does human agency lie in code?

Kate Sicchio: One thing that stands out to me from our early thoughts on coding and performance is the idea of interpretation of code. Code can be used as a score for movement, but also the movement can change the score as it’s being interpreted.

The Sound Choreographer score (half of the Sound Choreographer <> Body Code piece) reflects one of Dick Higgins’ scores from the 1960s and the ideas that the individual performer devises the system for interpretation.

Alex McLean: Ah this is very nice:

All markings on the face of the notation are to be considered as notation, and all elements of these markings are to be used or deliberately ignored for the production of movements and sounds according to any consistent system which the individual performer devises for the notation he (sic) is using. ¹

All focus is on the notation, and all markings must be paid attention to (even if ignored), but interpretation is left to the human performer. This is so different to computer programming but I think helps us imagine what a more ‘natural’ computer programming language might be – one that is more alive and open to change through use.

I’d like to bring up an article I wrote about live coding early on, in 2004. It begins

I’ve found the experiences of dancing and programming to have a great deal in common. With both I am immersed in an abstract world of animated structures, building up and breaking down many times before finally reaching a conclusion. Indeed, when the operation of even the dullest data-munging computer program is visualized, for example in a debugger, it does seem to be dancing around its loops and conditions — moving in patterns through time.²

Reading back, I’m surprised at the extent to which dance was really a central pre-occupation, thinking through the feel of languages, dancing to algorithms, movement in language. Even recounting how I began a software development process by dancing to Speedy J. I still remember that experience well, it was an incredible live set and I just ‘felt’ a system in my head that I was later able to explore through code. That period of my life was incredibly productive for live coding, but thanks to our belief in liveness we have almost no documentation. But you can see the interface on my screen on the right here from around 40 seconds in:

I think we’ve talked before about how you’ve come to programming through dance, and by collaborating with you, I’ve come to dance through programming, and to a large extent that’s true. But I suppose in some ways it’s not true, because really it is part of human nature to dance from birth. Would you agree? Perhaps there are also ways in which it is human nature to code from birth — we just have an over-constrained view of what coding is.

KS: I had similar feelings around movement and code as well. I presented this for a DRHA paper in 2010 where I shared a similar sediment: “Both choreographer and programmer working with real-time video systems are charged with organizing movement, whether they are working with bodies or pixels. They both create tasks in which this movement is filtered and then viewed by an audience.”³

I think the connect that we feel between dance and code is movement. We move from birth and we organize movement inherently. The idea that we constrain what code means is interesting. I think we do that with the meaning of the word choreography too. We are very quick to put things into their disciplines, rather than see how they apply to other fields and how they intersect. The movement, the patterns, the process of creating movement is ubiquitous. So if we can code dance, and dance can code, do we find new possibilities in the differences dance and code bring?

AM: Yes I agree breaking down artificial barriers and finding things between disciplines is what we should be doing. You remind me here of Malke Rosenfeld, a teacher who has developed a maths curriculum based on percussive dance.

Students learn to view themselves as both choreographers and problem solvers as they create their own dance patterns. Through this process, students begin to think mathematically as they choose from movement variables, combine smaller patterns into larger ones, and transform their patterns using different types of symmetry.⁴

It’s a lovely thought that you could learn maths by jumping about in patterns. All I got from maths lessons was a rushed sense of anxiety.

KS: I also see a lot in this Rosenfeld quote. I feel like that “choose from movement variables, combine smaller patterns into larger ones, and transform” could be a definition for choreographic composition. There are functions and there are ways to perform them in variation. This makes patterns. This makes a time-based composition.

AM: Your earlier question is about the possibilities in the differences between dance and code. I think you might be alluding to what we’ve talked about before – how human dancers get to choose how they follow choreographic instructions, perhaps changing their system of interpretation through a performance. Computers in general do not have agency. But now you’re working with machine learning and choreography. Are you looking for ways of giving computers creative agency?

KS: This is an interesting question. I think I’m just interested in using machine learning to discover possibilities that I might not think of because I am human. Thus far, the way that I have used machine learning has not given the computers more creative agency. In fact, it has only given them organisational tasks. I have layered these algorithms with human interactions in such a way that the human is prioritized and perhaps the human has to be more creative. There is a new term for this coming out of game design, called “mixed initiative creative interface”, where humans use machine learning to support them in making work, but it is still very much the human that makes the final decisions. Your rhythm script sounds like this – it is designed to support or inspire decisions, not make them.

AM: If you don’t mind me picking through my archive a bit longer, I remember probably fifteen years ago wanting to write a script that simulated movements in order to generate rhythms from them. I didn’t get very far but did end up with a script that drew a stick figure dancing. Here’s an excerpt of the code that describes the animations:

my $bodyparts =
  {
   head => {default => [['','   O   ']],
            leftright => [['','   O   '],
                          ['','   o   '],
                         ]
           },
   body => {
            default => [['','','   Z   ']],
            spin => [
                     ['','','   Z   '],
                     ['','','   z   '],
                     ['','','   Z   '],
                     ['','','   z   '],
                     ['','','   Z   '],
                     ['','','   Z   '],
                     ['','','   Z   '],
                     ['','','   Z   '],
                    ]
           },
   arms => {
            default => [['',
                         '       ',
                         ' ./ \. '
                        ]
                       ],
            flap => [
                     ['',
                      '       ',
                      ' ./ \. '
                              ],
                     ['','.__ __. ',
                     ],
                    ],
            feelie => [
                       ['',
                        '       ',
                        ' ./ \. '
                       ],
                       ['',
                        '       ',
                        ' ,/ \, '
                       ],
                    ],
            curl => [
                     ['       ',
                      '       ',
                      ' ./ \. '
                     ],
                     ['       ',
                      '.__    ',
                      '    \. '
                     ],
                     ['   .   ',
                      '  (    ',
                      '    \. '
                     ],
                     ['       ',
                      '.__    ',
                      '    \. '
                     ],
                     ['       ',
                      '       ',
                      ' ./ \. '
                     ],
                     ['',
                      '    __.',
                      ' ./    '
                     ],
                     ['   .   ',
                      '    )  ',
                      ' ./    '
                     ],
                     ['',
                      '    __.',
                      ' ./    '
                     ],
                    ],
            wave => [
                     ['   .   ',
                      '.__ )  ',
                     ],
                     ['     . ',
                      '.__ /  ',
                     ],
                     [' .     ',
                      '  \ __.',
                     ],
                     ['   .   ',
                      '  ( __.',
                     ],
                     [' .     ',
                      '  \ __.',
                     ],
                     ['     . ',
                      '.__ /  ',
                     ],
                    ]
           },
   legs  => {
             default => [['','','','  / \  ']],
             wobble => [['','','','  / \ '],
                        ['','','','  ( )  '],
                        ['','','','  / \  '],
                        ['','','','  ) (  ']
                       ],
             strut => [
                       ['','','','  ( \  '],
                       ['','','','  / )  ']
                      ]
            }
};

Source code: https://gist.github.com/yaxu/2b73d29600b683fe0f32749eb0c3950c

KS: A main difference between computers and humans performing patterns or instruction is that the human can choose how to interrupt this with agency. So for example, a dancer may choose not to follow a specific instruction as a response without that being an explicit instruction to begin with. This is still a valid choreographic choice. It is much harder for a computer to execute this without it being made a choice ahead of time. Computers need humans to help them make choices on-the-fly. This is where live coding comes into play in an interesting way.

So, where does this leave dance and code? Choreography may be made using code because of patterns and compositional possibilities are easily made with both. However, the interpretation of code in new ways requires the creative agency of a human. Code shows us possibilities; humans create the possibilities.

Dr. Kate Sicchio works at the interface of technology and performance. Most recently she has been working with wearable technology, live coding and technology as an intervention in the choreographic process. Her work has been shown at Banff New Media Institute (CAN), FoAM (BE), V&A Museum Digital Futures (UK) and Times Up (AT) amongst other venues. Her edited volume with Camille Baker Art and Technology: Techne/Technique/Technology was recently published by Routledge. She is currently Visiting Assistant Professor in Integrated Digital Media at New York University.

Dr Alex McLean is active across the digital arts, developing a creative practice centring on live coding and improvised music since the year 2000, co-founding the TOPLAP and Algorave movements, and the International Conferences on Live Coding and Live Interfaces. He applies live coding techniques in research into ancient textiles, as part of the European PENELOPE project, hosted by the Research Institute of the Deutsches Museum in Munich. He is based in Sheffield, where he curates the annual festival of Algorithmic and Mechanical Movement, and is trustee of the Access Space charity. He is also works as generalist with FoAM Kernow. He is currently co-editing the Oxford Handbook on Algorithmic Music with Prof Roger Dean, and working on a solo album with Sheffield label Computer Club. http://slab.org/