Joana Fernandes Gomes

Pictures from my Phd presentation - Behavioural  interfaces  in  the  mediation between image  and sound 

This is a 15minutes presentation to show the research I developed during the last year 

RAMA is a web application for visualizing and interacting with networks of music artists. It uses data of roughly 600000 artists and 3 million user tags, collected from Last.fm’s API. Data includes artists similarities, associated tags and popularity.
RAMA provides two simultaneous layers of information:

1. A map built from artist similarity data, modeled as a physical system using the framework Arbor.js
2. Overlaid labels containing user tags.

RAMA emphasizes commonalities as well as main differences between artists, users can interact with the graph in different ways, e.g. changing the graph complexity. Users can also edit graphs manually, removing some artists and expanding artist’s neighbors.
Through the use of YouTube’s API users can also listen to music while exploring the map. It is also possible to save songs into playlists that can be edited and exported to text files that can be loaded again in future uses of RAMA, or used in other applications.

System requirements

You will need access to the Internet and JavaScript enabled in your web browser.
So far, RAMA has been tested successfully with the following web browsers: Firefox 4.0 or greater, and Google Chrome. It also works on Safari with the exception of the playlist upload.

Contributors

• Fabien Gouyon
• Luis Sarmento
• Nuno Cruz
• Joana Fernandes Gomes
• Diogo Costa
• Bruno Gustavo Costa

Related papers

• Costa D., Gouyon F., Sarmento L. “RAMA: An Interactive Artist Network Visualization Tool.” Late-Breaking Demo Session, International Conference on Music Information Retrieval, Kobe, 2009.
• Sarmento L., Gouyon F., Costa B., Oliveira E. “Visualizing Networks of Music Artists with RAMA.” International Conference on Web Information Systems and Technologies, Lisbon, 2009.
• Costa B., Gouyon F., Sarmento L. “A Prototype for Visualizing Music Artist Networks.” International Conference on Digital Arts, Porto, 2008.

UNTITLED*  

In generative art, the artist sets tasks to the machine, and establishes in the machine an extension of him/ herself. Those extensions can be biological or psychological [1] [2]. It provides a semi-autonomous system [3] [4] where the artist can be the agent that selects or gives a program the ability to execute a selection through the rules he builds-in.

The relationship between art and science is very close. Generative art is a discipline that values and brings together art and science. From principles of biology, where we can understand evolutionary concepts, selection principles [5], and the acquisition of the external process of the human comprehension, it allows the creation of artificial replicating structures that don’t belong to the human domain. Generative art is a branch of artistic practice that uses resources from biology, mathematics, physics and other scientific fields for its simulations such that are able to generate new paradigms that until then were beyond the artist’s reach.

New characteristics such as learning, adaptation and mutations are typical of those systems. Normally, the most adapted ones perpetuate the skills more valuable and desired for the system in that moment. [6] [7]

The rules are the algorithms generated by the artists and the rules applied are the parameters that shape the behaviour of a certain individual, population and habitat. But like in the living beings those rules can be transgressed and the process reacts in a lot of different ways. This unpredictability, typical of complex systems [7], gives the artist the possibility of action and results that are beyond the ones he is capable of comprehending through his natural systems of perception: vision, touch, smell, etc.

Thus, the processes and the relations between humans and machines become closer. The interactions become more fluid and adapted. The intelligence of some of those systems allows that each individual gets better responses to his/her/its and more evolved and optimised actions.

Untitled was developed during my internship at the Music Technology Group (MTG), at the Universitat Pompeu Fabra, in Barcelona. Based on a Tangible User Interface (TUI), this work is a geometric generative system that is controlled by the movement on the surface. Each object on the table has a different functionality. Some objects produce forms and others act like tools to shape the visual composition.

[1] Mcluhan, H.M. (1964). Understanding Media: The Extensions of Man. New York: The New American Library

[2]Walter, B. (2008). “The Work of Art in the Age of Technical Reproduction.” Belknap Press

[3] Todd, S. & Latham, W. (1992). Evolutionary Art and Computers. Academic Press.

[4] Whitelaw, M. (2004). Metacreation, Art and Ar- tificial Life. Cambridge: MIT Press

[5]Dawkins, R. (2006). The Selfish Gene. New York: Oxford University Press

[6] Holland, J. H. (1992). Adaptation in Natural and Artificial Systems. MIT Press

[7] Principe, J.C. Keynote speaker. Gouyon, F., Barbosa, A., Serra, X. (Ed.), Proceedings of the 6th Sound and Music Computing Conference

[8] Galanter, P. (2003). What is generative art? Com- plexity Theory as a Context for Art Theory. New York: Interactive Telecommunications Program, New York University.

Concert of “2+n” @Culturgest in Lisbon

my generative visual work (with sound input from the Symphony Orchestra of Porto) - Casa da Música 

////Português////

Com o aproximar de mais uma época natalícia reunimos os sininhos, as luzes e o azevinho e celebramos o espírito natalício num almoço que se quer recheado de sobremesas musicais, entre delays de rabanadas, streamings de azevias, loops de sonhos e reverbs de coscorões. A Formação Variável de Laptops do CITAR apresenta-se em formato telemático numa performance em rede com Praga e Palo Alto. Sons que se constroem no espaço, traçando texturas e definindo camadas perceptivas conjugados com visuais gerados em tempo-real a partir da informação musical. Silent Lunch reúne a súmula de um trabalho que tem vindo a ser desenvolvido no CITAR entre as ciências, as tecnologias e muita arte.

A Formação Variável de Laptops do CITAR encontra-se dia 15 de Dezembro na Sala de Coro pelas 13.30 para nos empanturrar de cultura digital durante 25 minutos em solo ou em banda (entrada livre)

Artistas: André Baltazar, Nicolas Makelberge (deslocado - Praga), Vasco Carvalho, Maílis Rodrigues, Vitor Joaquim, João Cordeiro, Joana Fernandes Gomes (visuais) e Álvaro Barbosa (deslocado - USA).

///English////

With the approach of another holiday season met the bells, lights and mistletoe and celebrate the Christmas spirit at a lunch that is filled with music desserts, between rabanadas of delays, streams of azevias, loops of Sonhos and reverbs of Coscorões. The “Formação variável de Laptops do CITAR” presented in a telematic format in a network performance with Prague and Palo Alto. Sounds that are constructed in space, texture mapping and defining in conjunction with visual perceptual layers generated in real-time from musical information. Silent Lunch meets the epitome of a work that has been developed in CITAR between science, technology and art.

The “Formação variável de Laptops do CITAR” happened in December 15th in the Choir Room at 13:30 

Artists: André Baltazar, Nicolas Makelberge (displaced - Praga), Vasco Carvalho, Maílis Rodrigues, Vitor Joaquim, João Cordeiro, Joana Fernandes Gomes (visuais) e Álvaro Barbosa (displaced - USA).

///André Baltazar///

///Joana Fernandes Gomes///



///João Cordeiro///


///Mailis Rodrigues///



///Vasco Carvalho///


///Vítor Joaquim///


///Álvaro Barbosa + Nicolas Makelberg///


///Rehearsal///




(PICTURES BY VÍTOR JOAQUIM)

Article in which I’m mentioned from Jornal de Letras, Artes e Ideias - special issue (30 years anniversary)

Artigo em que sou citada no Jornal de Letras, Artes e Ideias numa edição especial de comemoração de 30 anos do Jornal

VAI EMBORA (2006)

Is an interactive video installation that reacts to the heartbeat of the visitor. The character is provocative and the visitant can only see the whole video if there is a “cardio-response” to it. The video gets more provocative at times, and the visitants have to manage their heartbit to be able to undergo the hole “experience”.

This work was presented in Maus Hábitos (Porto, Portugal)


Keywords: Interactive video installation, provocative, body

VAI EMBORA VIDEO

CONFRONT (2007)

Confront is an interactive video installation controlled by the movement of the spectator. Each time someone enters the room (one person at a time) one of the two videos is selected in random function. In one video we have Maria a young woman with liberal political views and on the other António a young guy from a very traditional and conservative background.

Depending on the way the person stands and moves towards the projection the character on the video will “respond”. If the person is really still or moving very slowly, the character responds aggressively. The amplitude and the velocity of the movements are captured by a webcam and received by Eyesweb and Processing.

The main purpose of this project is to provide an interactive communication of the type character-visitant, destroying the barrier between these two “worlds”.

This work was selected and presented in Culturgest (Lisbon, Portugal )during a conference called TeDance.

keywords: video, installation, interactive, motion tracking, real X virtual

LIVE Generative Visuals for B&W closing ceremony. Carlos Zíngaro + formação variável de laptops do citar

Demo from the concert at Centro Cultural de Belém

PROGRAMA

NICOLAS BERNIER Writing machines (2005)
JOSÉ LUÍS FERREIRA Limits / Capacities (2008)
RITA TORRES O mostrengo (2008) video e música
CORNELIUS CARDEW Treatise (pag 21 & 22) (1963-67)
GILLES GOBEIL E RENÉ LUSSIER La gageure - excerto da obra - Le Contrat (1996-2003)
PAULO FERREIRA-LOPES Três peças do livro da escuridão (2007)
OBRA COLECTIVA Skypping #2 (Luz e Música)
JOSÉ JÚLIO LOPES Verschwinden (2008)


JOANA FERNANDES GOMES media-artist e realização vídeo

Solistas
MIGUEL CARDOSO laptop
PEDRO BITTENCOURT saxofones
PEDRO PATRÍCIO laptop
RICARDO GUERREIRO laptop
VINCENT WIKSTRÖM IPulseGuitar
VITOR JOAQUIM laptop

Concepção e direcção artística do projecto PAULO FERREIRA LOPES | JOSÉ JÚLIO LOPES
Direcção PAULO FERREIRA-LOPES
Produção executiva ORCHESTRUTOPICA
Coordenação Técnica VITOR JOAQUIM
Desenho de luzes JOSÉ RAMALHO

Em colaboração com
CITAR (UNIVERSIDADE CATÓLICA PORTUGUESA)
UNIVERSIDADE DE KARLSRUHE DEPARTAMENTO DE MÚSICA E INFORMÁTICA


CO-PRODUÇÃO
CCB/ORCHESTRUTOPICA EM RESIDÊNCIA
Location: CCB, Lisboa

A few videos from the database in a small video mix

The piece “HOME” is a video art installation, where the materiality is questioned and where the real and the virtual are confronted, opening windows to a completely abstract and digital world.

Generated by an artificial life system, this piece communicates with the spectator in two ways: one abstract and another actual (concrete). Each one shows one side of this algorithmic world.

Outside of the installation room there is a screen; on it is shown to the visitor an artificial life system in its more abstract form; the digital world as it really is: mathematical and schematic. The individuals have shapes that remand the visitor to a digital/virtual world, and transmit the mathematics and “deshumanization” inherent to such world. Is a community built by computers and represented trough graphics.

On another side, inside the installation there are three screens that represent the “construction” of that same world. This system provides its own actions and it modifies itself during time by means of parameters set by the algorithm. During the time that such assembly of individuals exists, they feed themselves, reproduce themselves, die and so on. The result of those actions is assessed and creates a set of parameters, which selects at each minute an assembly of three videos from a database. Triggered in a synchronized way and displayed on each of the installation walls, such assembly of videos represents the “status” of the world. The sequence of those choices results in the presentation of the “landscape” of that world.
The videos are composed of characters that interact with each other in a wide white immenseness and that move along the various projections. The goal is to create a world that communicates with the spectator in those two ways, being at the same time abstract and actual.

The layout of those projections is aimed at producing in the visitor the sensation that he is looking trough windows. The videos give the visitors access to this “world”. The white immenseness baffles the visitor taking and hinders him to perceive the extension of that world. The characters “humanize” themselves losing slowly and progressively the virtual and abstract appearance they present on the screens outside the installation. It is a community that lives “on the other side”.

In this installation the videos are an essential part of the construction of the identity of this society/world. These videos have very special characteristics as it is the way such community communicates itself with us, the spectators. It is where the community reveals itself and transcends the virtual.

The white immenseness of the videos (which in technical terms means that the filming was done in Cyclorama) is an aesthetic normally related to Sci-Fi movies, where the absence of spatial references and where the minimalist aspect of that environment is valued. The Cyclorama offer in this piece, videos that at the same time seam naked and that on another side suggest the existence of a technological society that spreads over an almost infinite space. The absence of parameter warrants that all the attention of the visitors be directed to the gestures of the characters. With this technique, the lack of references makes it impossible to have the full comprehension of the relations between characters and the real size of everything that is shown. In a totally amorphous space we have the possibility of building whatever we wish. The world built by means of actions. A Plantation exists solely when an individual makes movements that are in some way referred to a plantation space.

For clear comprehension of the actions performed by the characters, it is necessary to define something that establishes that relationship between the action and the totally white background. The sound is essential for that to happen. The selection and the edition of references sound transformed and inserted in the videos allow the gestures to transform into actions. The sound is also a way to emphasize the wrapping of that “world” around the spectator as each screen emits its own sound resulting in a surround effect.

Another important aspect is that the main objective of this installation is to enhance the attitudes of the characters, that is, to build a digital society, that expresses itself by means of the videos, and that communicates by means of actions. In this case the white immenseness is a way to emphasize the actions of characters without showing other distractions to the spectators.  As the objective is to create in the spectator the sensation that he/she/it sees the community through glass windows, synchronization of the screens is important. That is, the presented videos in each screen will have two other videos, which will be being presented in the other two screens. The objective is to create a liaison among them, which makes the idea of a construction of a world beyond the walls feasible. That liaison is going to happen mainly through the movement of characters from a glass window to another. The result is the construction of a sensation of wrapping the spectator with the “digital world”.

The concept of a “window” is really important for this piece. The spectator that enters this installation will be a mere observer, without any type of action on the society. He/She receives information through the glass windows that “frame” HOME. In spite of the illusion of proximity, we will be far and separated from the true reality of this digital world.
We can say that in spite of these videos being the representation of the “world”, for the spectator, they are the summit of choices that result in one controlled representation of certain characteristics and that implicate a manipulation on the vision offered to the spectator. The glass windows have the function to create the illusion of clarity and transparency that are not real.

The fixed camera reinforces even further that feeling. In this project, the user does not travel or walk in the space. Restricted information is supplied to him/her/it. Depending on the status of the inhabitants of HOME, the video is a restrict information. Depending on the HOME state the videos may become solely the white immenseness.
The algorithm was developed in actionScript 2.0 and the videos are controlled by use of Max MSP/Jitter. The sound design was done in Pró-Tools.

The installation space:

(3D model by Maria João Neves)