Hack the Space was a 24 hour hackathon set in the Tate Modern's turbine hall organised by the awesome 3 Beards. It's an amazing space to work in and it was a privilege to be part of the first ever hackathon there.
Helen had the idea of creating a synaesthetic artwork from an essay by Richard Lloyd Parry called Ghosts of the Tsunami published earlier this year.
The essay describes his visit following the 2011 tsunami in Japan to a priest who had been exorcising the ghosts of the dead from survivors.
Synestheisia is a condition were senses become entwined and some people perceive letters or sounds to have colours.
Helen wanted to take the text of the essay and represent each word with a colour. She and Imeh looked through the Tate’s collection and found five images that were connected with the sea and Japan including a Kandinsky and a Hockney. They used these images to generate a colour palette for the words.
I took the text of the essay directly from the LRB website and processed it using Ruby to generate a list of the unique words, with very short one like 'I', 'a' and 'it' removed. These were then assigned colours by Helen and Imeh based on their importance and their starting letter.
I then built an app using openFrameworks that took the text and the colours and drew vertical bars to represent the words. The word length affects the bar width, with sentence and paragraph breaks shown as gaps.
We tried recording some ambient sounds in the turbine hall but these didn't really work. So it was fortunate that we then got introduced to Peter Gregson who had recently done some data sonification. He suggest we use the G minor scale to go with the piece and even created the long and short notes we used in Ableton Live.
Each word plays a sound based on it's colour and length. The colours used were grouped by hue, sorted by frequency and then assigned to the notes in the scale. Peter gave us some invaluable tips on this and the end result came out much better than i was expecting.
After a very tiring, but rewarding 24 hours, here's a screen recording of the piece.