Music in your head

It’s official: music created by a brainwave monitoring device goes mainstream. As a part of the marketing campaign for the new sci-fi series „Sense8”, Netflix has recently released „Brainwave Symphony”, an electronic musical piece recorded with using a headband that monitors alpha, beta, delta, gamma and theta brainwaves of eight volunteers.

The song promoting the new Wachowskis’ series available in the popular streaming service is neither the first nor the last example of using brainwaves sensors to compose music. Similar experiments were carried out for several decades. But this time the song is no longer a niche artistic project targeted to a small audience. Due to the fact that it is part of a marketing campaign run by the Internet giant and it concerns a series with a growing amount of fans, „Brainwave Symphony”, which is available on Spotify and on the video embedded below, it has a chance to hit a much wider audience.

https://www.youtube.com/watch?v=HMH4VFY6cFk

Just a few weeks before this piece of music was released, I visited DMY 2015 festival (it was held from 11 to 14 June in Berlin) where I met Bob van Luijt and Renate Roze. They were showing the project titled „control(human, data, sound)” that comprises brainwaves while creating music. Bob applied in his work the Muse Brain Sensing Headband that is being sold and advertised on the commercial market as a device facilitating daily meditation. In January this year I had an opportunity to test this device at CES fairs (despite my initial skepticism about its effectiveness, Muse really helped me to relax for a while), but at beginning of this year I did not know that the headband can be also reprogrammed into a kind of musical instrument. Such an idea came to Bob’s mind. And he is not alone – Muse was also used somewhat later by studio Tool  in a production process of „Brainwave Symphony” for Netflix. And I have a hunch that this is not the last word. As Bob van Luijt said, in few years from now brainwaves sensors could be built into virtual reality headsets.

Kubrickology Bob van Luijt Renate Roze DMY credits Trend Nomad
Bob van Luijt (on the right): creator, composer, founder of Kubrickology – a strategic design company that focuses on technology, music, art, games and urban design. Renate Roze: a co-producer of the film about „control(human, data, sound)” project; a freelancer, an interviewer and editor for international film festivals. Photography: Trend Nomad

Trend Nomad: You are a musician by background, but instead of writing notes, you create music compositions by coding.
Bob van Luijt: Few years ago, when I was visiting a technology conference in San Francisco devoted to augmented reality issues, I saw Muse – a wearable device that measures brain activity. I started thinking what kind of project could I do with data collected from a human brain. I decided to use it for an artistic purpose. I created a music composition based on variables that I got from a brain sensing headband. The composition itself has its form, structure and fixed instruments, but many elements such as keys, tempo, duration, note length and panning are determined by variables prescribed by data coming live from Muse device that the dancer wears on his or her head while dancing.

https://vimeo.com/114393301

Your project is much more about data than about music.
I definitely agree, but please note that data stored in a computer means nothing. Only a human being can contextualize data, and only a context gives data its value. Nowadays more and more algorithms analyze and try to give a context to data. Sometimes that context makes sense, but usually it doesn’t make sense at all. The bigger the pile of data gets, the more meaningless it becomes. I wonder what will happen if one-day data would be harvested directly from our brain. My project called „control(human, data, sound)” is an abstraction of that idea.

How do you translate brainwaves into music?
I used Muse Brain Sensing Headband’s developer kit and I made a function that browse through data, downloads some data, and turns it into a number – so called Integer – that affects the composition. I created the composition using Node.js development environment with integrated MIDI library, as well as Logic Pro X recording studio software. I chose 14 instruments from three libraries, which were strings and string section effects from ProjectSAM’s „Symphobia 2” series, Eduardo Tarilonte’s „Epic World”, and Best Service’s „Synth Werk”.

When the software was ready, you put Muse headband on a dancer’s head and ask him to start dancing. Does his movement make any impact on the music?
Sometimes yes, but and sometimes no. The software itself determines where to place information and make music out of it. It always sounds different when someone else wears it, or when you record it twice with the same person. A dancer is not able to change the music deliberately.
Renate Roze: Please note that music is also made when a dancer stands or sits still. His brain is always active, he does not have to move to make music. A melody will not become more dynamic when a dancer starts dancing faster, that is not the case. It depends on the state of his mind.
B.v.L.: If I show to the dancer a picture that would calm or scare him, the sensor will notice the change on his brain activity and the music will reflect it.
R.R.: But the change would be very subtle. As Bob has already mentioned, synthesisers are preselected. If you repeat the same experiment again, and the same dancer would think of something completely different while dancing, or would make other movements, then an outcome would be modified in a subtle way. We would hear the same kind of sound, but maybe a little bit faster, softer or louder, but it would not be a completely different piece of music.
B.v.L.: But it is never exactly the same. The wearable device sends six data points every millisecond or every two milliseconds. That means every second I get about 4500 data points. That’s a lot of data to be translated into music.

Kubrickology data
Columns of the player in the console made by Bob van Luijt.

Have you ever considered that some people could be afraid of putting such a device on their head?
B.v.L.: If someone is wondering if his or her thoughts could be „hacked” by a wearable device, I can say that the answer is: „No”. It is impossible to check what you think of when you sit, stand, walk or dance with Muse on your forehead. All I can see is that your brain activity changes when you are stressed or relaxed.

Besides music and wearable devices, you are interested in virtual reality issues. Do you think that one-day devices such as Oculus Rift will be equipped with brainwaves sensors?
B.v.L.: I think that in the next 15 years this kind or rather more advanced sensors will become common parts of VR devices. I am sure that people will wear more and more sensors around their bodies, including a head. Facebook, among others, will be delighted.

Do you believe that in 2020 VR devices will be as popular as smartphones are today?
B.v.L.: I think it will be more like a 3D television right now – not everyone has 3D TV set at home. You need a lot of stuff to watch VR content. People can also – literally – get sick of it [but don’t worry, there is a VR motion sickness relief capsule – ed.]. In my opinion, virtual reality is being commercialized to soon. Few years ago, when I went to a virtual reality conference for the very first time, I heard a lot of different talks about VR. Now it’s completely taken over by commercial issues. Devices that are launched to the market as end products, in fact, are just prototypes. But maybe I’m wrong. Some people said in the past that the internet will never go mainstream.

If you are interested in more technical details of the „control(human, data, sound)” project, please watch the movie below and read Bob’s article: „A story about how I created music out of data”.

https://vimeo.com/114440097

If you have any questions regarding „control(human, data, sound)” project, please contact Bob van Luijt at bob@kubrickolo.gy.

Kubrickolo.gy

P.S. If you need more information about the Muse headband, you can watch the movie embedded below and visit the Choosemuse.com website. In case you would like to use this device for composing music with your brainwaves, please download adequate code from the Bob van Luijt’s library available on Github.

https://www.youtube.com/watch?v=YI5uXlTnNms

Do you like the interview? Then buy me a coffee! You can donate a small sum of money using your PayPal account or credit card. All donations will finance my journeys to fairs, festivals and conferences devoted to design and new technology – this is where I find news for my blog. Just click the button below to perform a secure transaction. Thank you for your support, it will help me to take a step forward and write new posts.

More articles