Music in Images

[arjungupta]

Hi Guys,

I am new here, and I have a project that I have developed as part of my Masters thesis that I wanted to show and request some feedback.

Its called Music in Images

This project presents a new approach towards algorithmic music composition where the music is composed using the pixel values of digital images.

Application Info:
The application makes use of image data, such as: pixel Color, Saturation, Brightness, and qualities like: Overall Image Brightness, Color Variation and Prominent Colors.

Each of the above properties contribute towards composition of the music. The song attributes controlled by them include: Melody generation, Durations, Cord Generation, Chord Progression, Tempo,Scale and Transposition.

The theory behind the translation of image attributes to music attributes is based on previous works done in the field of psychology, in the topic: perceptual associations of audio frequency and music modes with hue, saturation and brightness. In addition to this, there are certain translation techniques which are based on the my artistic sense.


The analogy for Image to Music translation used here is:

Pixels : Similar Color regions : Overall Image
Notes : Chords : Scale

You might find it interesting. Please do check it out here:

http://www.musicinimages.com

This application works best for abstract images. It tends to produce calmer music with smoother images, or with less color/brightness variation, and more complex music with high variation images. The overall brightness also adds an effect on the music.


I would really appreciate if you could give my your feedback on it.

These are a few main questions I need answers on:

1. You can try out different images and play them using the given instrument options. What are your impressions on the quality of the music produced ?

2. If you are a musician, can this application be useful to you ?

3. If you area a psychologist, do you think this application can be used for exploring relationships between images, computer generated music and human emotional interpretation?

Feel free to add any other questions/ comments that you might like.


Thanks!
Arjun

 

[Sparrow]

Oh, it's already here...Awesome application dude. I love you.

 

[EyeSeeCold]

It's pretty interesting, but I think it needs more work.

 

[IndigoSensor]

This is AWESOME! I really love love this idea.

I am not so sure how well this would relate to musical therapy, but I think this could be useful for people in therapy in the sense it might allow them to "relate" images and music and use that as a springboard for discussion. Sort of a rorschach type thing.

As far as the music itself. I wouldn't really call it "music" but more of ambient sounds. It's really an argument semantics with that, but all the same it is a wonderful idea.

One thing I did notice, is the sounds become very "meaningless" if you don't watch the little marker trace the image. When you watch it move and follow it, the sounds have more meaning, interest, and "strength" to it.

It's pretty interesting, but I think it needs more work.

I agree but I am at a loss to explain how it could be improved. I think making the algorithim more complex and allowing for more instruments and patterns might be useful. I have no idea how the program works though so I don't know if that is even possible or how it could be done.


ETA: also, the more complex the image, the better the sounds, but there comes a point where the image is too complex (such as photographs) because there are too sharp of transistions, and it becomes just a mess of sound. I gave this picture a try and I think this one really highlights the potential this has. Set with the strings and bells, it's very fitting. And I did find you still need to watch the icon trace or it becomes less "meaningful"

 

[arjungupta]

Thanks guys!
I agree some images sound better than the others.
IndigoSensor, your image sounds great! thanks for sharing.

Its a work in progress and I am improving the application to make it even better. Feedback certainly helps! Thanks again.

 

[nrenniks]

Algorithmic music production is actually one of my interests. I wanna do my eventual thesis on it!

 

[420MuNkEy]

This is a very interesting, exciting, and ambitious concept, and I'm pretty surprised how well it works so far.


It needs to be able to produce better music chaos when an image is complex. I'd try achieving this by having it identify and process highly complex regions in a different way that would produce something more musical, but just as chaotic. Also, it doesn't sound to me like there's anything going on with the velocity (ie, how "hard" each key is hit), though I could be wrong. I think that would add a nice depth that would sound more like music.

It would be nice if, on the UI, the melody and chord were represented as different colors on the keys, and if the keys stopped glowing (or faded out at the same rate as the sound) when the key is no longer pressed. Not very essential, but could be useful to someone. Another thing it might be helpful to add to the UI is a mute button and/or volume knob for each "instrument" being used. It would also help if there was an option to retrieve an image from a url (e.g., if I wanted to try that image IndigoSensor linked, I could just enter the URL somewhere instead of having to download it locally, then upload it to the application).

Lastly, embedded examples of images it processes well would be very nice.



P.S.
Any chance it's open source?

 

[Kokoro]

It would also help if there was an option to retrieve an image from a url (e.g., if I wanted to try that image IndigoSensor linked, I could just enter the URL somewhere instead of having to download it locally, then upload it to the application).

Last time I did it, putting in the URL worked just fine. Maybe you should try again.

 

[Saeros]

Very cool. I like listening to these ones.

 

[DesertSmeagle]

This is cool. Im goin to college for psychology.This is really cool. If it was perfected and studied..so much potentilal, it could inspire a while new group of art, so when people go look at art they can say " well it looks beautiful, but how does it sound. Just needs some improvement. like some pictures might have big spaces.. but if an artist were to make a pience of art that sounds good they wouldnt include spaces..cool

 

[Saeros]

An alternative to this might be to convert music into a picture, as well as a picture into music. I think that would be interesting.