Helena Nikonole

Credits:

-Veronica Samotskaya

ornithologist, scientist-biologist, science popularizer, journalist

-Natalia Soboleva

AI expert

-Konstantin Yakovlev

PhD in Physics and Mathematics, AI expert

-Nikita Prudnikov

musician, developer, AI expert

Helena Nikonole is a media artist, curator and educator whose work explores intersections of art and technologies such as Artificial Intelligence or Internet of Things. According to Helena Nikonole, an artistic exploration of the potential possibilities (and also potential risks) of technologies like Machine Learning is necessary to understand the context of modern technologically and media determined world. She presents lectures and workshops in the field of Art & Science, Neural Networks in Arts and new media art at different institutions including Rodchenko Art School (Moscow), Art Laboratory (Berlin), Mutek Festival (Montreal and Tokyo), Institute of Philosophy (Russian Academy of Sciences), Innopolis University (Innopolis), National Centre for Contemporary Arts (Moscow) and many others. Exhibitions include “Out of the Box” at Ars Electronica 2019, “Persisting Realities” in the frame of CTM Festival (Kunstraum Kreuzberg, Berlin, Germany), “Open Codes” ZKM Center for Art and Media in Karlsruhe, Germany, “The Wrong – New Digital Art Biennial”, “New Codes of Art”, ELECTROMUSEUM (VI Moscow Biennial of Contemporary Art), IAM at GARAGE Museum Moscow, “Earth Lab” (The Polytechnic Museum and Ars Electronica Center collaborative project), “101” Mediapoetry Festival (St. Petersburg) and many others.

We created a universal tool to understand the structure of bird language. We can translate from human language to bird language and vice versa, in other words we have translator and text-to-speech for bird language.

The motivation was the paradigm of non-anthropocentric world, where we can understand non-human others, like animals and birds. The inspiration came from Jakob Johann von Uexküll’s concept of Umwelt – the experiential world that a being inhabits, its perceptual surrounds. I’m inspired by the idea, that Artificial Intelligence as a un-human agent, can help us to understand birds, also un-human agents. In this case AI is not only a mediator or interface between human beings and birds but rather an organ or full partner, semiotically active. It helps to understand bird’s subjectivity through the language.

We started to work with the Great Tit, one of the most widespread species from Europe to Asia. The first machine learning approach we used, was XGBoost – boosted decision tree algorithms. While it classified bird signs in two groups: calls and songs with an accuracy of 83%, it was not a universal tool to understand the structure of the language. Now we are developing the second machine learning approach based on auto encoders. First, we take the waveforms from the audio of the bird language and process it with an auto-encoder, in order to extract the shape of the bird phonemes. Once we have these shapes, we then take a second auto-encoder, and feed it with the shapes of the phonemes, which allows us to see clusters. These clusters reveal the language structure and let us deconstruct the bird language into a series of phonemes, which we are using to build an AI-translator for interspecies communication.

http://nikonole.com/birds