Music from our surroundings

During a school course called Experience Prototyping, we were introduced to Teachable Machine, a Machine learning tool. The purpose of this was to enable us, designers, to work with Machine learning from the perspective of prototyping to envision and test our ideas.

For a 3 day exploration, I decided to develop a tool that enables you to create music from your surroundings.

THE CONCEPT

The goal of the concept was to make music from our surroundings using the pixel colours of objects. I used Teachable Machine to teach it three colours i.e.Red, Blue and Yellow. The trained model was then linked to P5 where each of those colours was then assigned to a different musical tone. In order to make sure it could recognise different tones and textures of the three colours I had to make sure to provide the model with many colour samples and textures. . Ultimately through the camera input the program read colours around me and play sounds for the detected colours of objects around me.

To view how this turned out, watch the video below.

TRAINING THE MODEL

To train the image model to classify the images into different colours i.e. Red, Blue and Yellow. Teachable Machine was used to teach the model to classify images into three colour categories. I made sure you use objects that had different tones of blue and different textures this would enable the model to recognise different versions of the same colour and result in a better-trained model.

Training the model to classify objects based on these three colours. A series of textures and tones were used to train the model.

Testing the model that was trained to recognise Red, Yellow and Blue on Teachable Machine.