During a school course called Experience Prototyping, we were introduced to Teachable Machine, a Machine learning tool. The purpose of this was to enable us, designers, to work with Machine learning from the perspective of prototyping to envision and test our ideas.

For a 3 day exploration, I decided to develop a tool that enables you to create music from your surroundings.

THE CONCEPT

The goal of the concept was to make music from our surroundings using the pixel colours of objects. I used Teachable Machine to teach it three colours i.e.Red, Blue and Yellow. The trained model was then linked to P5 where each of those colours was then assigned to a different musical tone. In order to make sure it could recognise different tones and textures of the three colours I had to make sure to provide the model with many colour samples and textures. . Ultimately through the camera input the program read colours around me and play sounds for the detected colours of objects around me.

To view how this turned out, watch the video below.

TRAINING THE MODEL

To train the image model to classify the images into different colours i.e. Red, Blue and Yellow. Teachable Machine was used to teach the model to classify images into three colour categories. I made sure you use objects that had different tones of blue and different textures this would enable the model to recognise different versions of the same colour and result in a better-trained model.

Training the model to classify objects based on these three colours. A series of textures and tones were used to train the model.

Testing the model that was trained to recognise Red, Yellow and Blue on Teachable Machine.

What is Form in IxD?

There is a classic notion of “Form“ is a shape or 3D object, which has the ability to express emotion and character. But “Form” can also be many elements coming together to create a whole. In the case of Interaction Design it is that particular sequence of elements which together can give the idea of an emotion. For me Form is about giving shape to an overall experience which I have tried to do in the exercise below.

For this exercise I chose to express the term Humiliation.

On diving deeper into the emotion I realised that humiliation has two acting parties. Therefore for this purpose I chose to express a person being humiliated.

Image credits: solmonstar.live

Using ‘Curves’ to visually represent Humiliation

Humiliation is a series of intense feelings which could be embarrassment, shame, discomfort etc that lead to the person feeling completely destroyed in the end. I visualised the emotion in a curve pattern to better understand the interaction and effect that I want to create.

The curve of humiliation.

No person feels humiliated on their own. It is a feeling that is triggered another person or event.

When choosing to express this emotion it was critical that it was expressed only once triggered by a person which is the key element when trying to express Humiliation.

Code a New Expressive Language

I choose to express this emotion using light and programming a reaction on pressing a force sensor. Code is a new subject to me and to achieve what I had in mind I realised that I need to first break down the sequence that I had in mind as this would enable me to know the code that I would require. This also made it easier for me as it formed a checklist of feedback that I want to receive. Overall it was fun to experiment with Arduino but I realise its a muscle to develop only with more practice

.

The purpose of this exercise was to develop an understanding to the dynamic nature of forms in movement. Kinematics is the study of a form in motion. During this exercise we chose to experiment with concepts of elasticity in a various materials through motion .

This was a group project with 5 team members

Our inspiration was a Parquet Deformation which is a regular pattern of tiles that transforms as you go from left to right whilst maintaining the regularity of the tiling . Taking the inspiration of a grid metamorphosis we thought to explore how movement, texture and light would affect the grid to further create interesting patterns.

The Parquet Deformation Grid that we used when developing tessellate.

Tessellate is a kinetic installation built with minimalistic movements. It uses tensile materials that are layered and overlapped to create patterns. We saw this piece fitting as an ambient wall installation without drawing away from the actual space. Each unit consists of sections of tensile cloth which are tied to acrylic rods and nylon string. The acrylic rods light up in synchronisation with the movement and the music for greater visual effect. The nylon string carries out the movement between the tensile layers which produces the different fold, curves and overlapping patterns.

The details of the mechanisms and working system to create a working mock up of tessellate.

Various explorations of form and material.