Sutro Ke Anusaar
In eng: According to Sources
An experience design project
Domain
Experience Design Installation,
Staging, New Media
Technology Used
AI, Python, Machine Learning, Arduino, Basic Sensors
Duration
6 weeks
Keywords
Disinformation, Experience Design, Staging, New Media Design, Interactive Art installation, Voices, Stories, Voice Cloning, Machine Learning, Python, Artificial Intelligence, Material Exploration, Spaces, Luminance, Electronics
सूत्रों के अनुसार
Sutro ke Anusaar (eng: According to Sources) | Experience Design Installation
“Can you trust yourself?”
‘If it is on the internet, it must be true.’ In this well-oiled mechanism of a fact-guzzling world, both information and the informant somehow end up becoming either a travesty or a collective truth. *DING*, here comes another WhatsApp forward: a promising candidate maybe, in this transtemporal competition of information. The time or the source through which information goes through seems to sufficiently modify one reality and make-believe in another. Speaking of source, let’s throw in individual biases in the mix. Where does that lead? This almost radical blurring of the limits of fiction and reality is where this installation stems from.
‘Sutro ke Anusaar’ (English: According to Sources) is an experiential installation in which the ever-changing fundamentals of truth have been looked at closely. The judgement of whether a piece of information is simply fascinating fiction or just an excuse for a very lazy ‘Alternative Fact’, is left for the participant to decide. What if your voice starts betraying you? The sense of mystery has thus been kept shrouded within the setting of this installation. Now, I must ask you again: “Can you trust yourself?”
Participant interaction video
Keep sound on | Headphones recommended.
Every message we read, every post we scroll, every newspaper we read with our hot cup of chai every morning, every phone call we receive, every person we meet, everywhere we go, everything we do bombards us with information in some form or the other. We are in a pool of saturated information. It just rests on us if we drown or survive by knowing what to ingest and what to not.
Built with multiple thoughtful iterations and through the usage of Machine Learning, 'Sutro ke Anusaar' delves on information and the power it holds via its own source. Let's look at how this came into being.
Process and Iterations
I first started with two very broader themes first where I conducted certain experiments. The first one was about "How do we trust?".
Keywords Selected: Trust, Belief, Confirmation Bias, Cognitive Dissonance
1
Trust Fall 2.0
Multiple personas were created with physical attributes, gender pronouns, occupation, and what they said about themselves and also their hometown and mother tongue either of which were revealed after the second round. These were selected after careful scrutiny. Few participants were brought in for a trust fall exercise but with a twist. They were asked to read through the details and select 3 amongst 6 and they had to tell us the reason.
Then for the second round, they could look into either their hometown or mother tongue. Then finally they had to select that one person whom they would trust enough to do the trust fall with and provide their reason for preference.
Findings: Very interesting findings came through. Most of these people were selected on the basis of whom they could have a stronger bonding and deep conversation. Some people were judging based on the empathy level that they 'might' have. Few people were judging them on the basis of their occupation and gender pronouns. Very few people were judging them on the basis of their physical attributes, language, or their hometown. But, interestingly, all of them were preparing these vague more detailed personas in their head with whatever information they could gather according to their assumptions.
2
How do you trust?
Trust Fall 3.0
Participants read the writeups and made their choice
A few willing participants were enrolled in and asked to write a few lines vaguely describing themselves of what kind of a person they think they are. Then a random person was called in and asked to come in and read aloud these writeups and tell us loudly their thoughts on what the person might be like according to their own assumptions and select the person that they wish to do the trustfall with.
Participants got to meet up and share their personal thoughts
Findings: Very interesting findings came through here as well. Although these volunteers rightly put up the best version of themselves without their shortcomings, it was very fascinating that the unsuspecting participants who were asked to make the judgements were very critical of their stand on what the person might be like. Some of them even started guessing genders which most of the time came out wrong. They were all looking at angles of empathy from these writeups. After doing the trust fall, they were surprised to meet the people they made the harshest judgements about because realised some of their comments might be totally false.
3
Cat's got your tongue?
Taking the power of speech for granted.
We do not understand the privilege of speech until speech eludes us when it is absolutely necessary to express. This was a small quick experiment on how people react when they realize they cannot express themselves properly.
Chatting with the bot
Excerpts from a chat
The participant was asked to chat with a bot to place a complaint but the keyboards were all deliberately re-mapped, and well, spacebar, backspace, delete and enter buttons were specifically made to either correspond to another key or not at all. All of this but within a time limit and interrupting texts from the bot. Oh, and the mouse was also disconnected. I know it was kind of cruel. Sorry for that.
Findings: The feeling of frustration of not being able to portray your issues or thoughts with a medium we are so very comfortable with these days. Interestingly, one of them even tried to use emojis to portray their thoughts. This experiment brought in a very crucial angle of voices of people from different strata be it social, political, geographic, linguistic, etc. who are not so very privileged to bring forward their thoughts to the mainstream to be discussed and acted upon further.
These two experiments brought me to a third larger area which keywords were taken from both of these experiments
4
Redacted | Manipulated Stories
Story, Voices, Manipulations, Society, Equality
These two experiments brought me to a third larger area which keywords were taken from both of these experiments
The participant walks through an elongated corridor. Each character on the podium talks about a different issue. The space becomes much more cramped as they move along. Each podium here stands for a different issue.
Top view of the space
There are also these 'censorship' membranes that are a result of our biases. It redacts and distorts certain information that the character is trying to tell you.
If one is eager enough, they can go behind these membranes and listen to the actual story that is being narrated.
As they move along, if they decide to not listen to the entire story, the characters stop their narration and have an upset emotion on their face. The biases get stronger as they move along and the video and sound quality and the lights start getting worse.
This last character will have the story which has been neglected all this long or failed to gain traction for discussion in mainstream media. The bias is strongest here with regular glitches in sound and video and a dimmed flickering light.
Then it was time to further narrow the scope.
5
Sutro ke Anusaar | Version 1.0
Story, Voices, Manipulations
The time or the source through which information goes through seems to sufficiently modify one reality and make-believe in another. Speaking of source, let’s throw in individual biases in the mix. Where does that lead?
Teleprompters in action
The participant comes in and starts narrating from the teleprompter. As soon as they are done speaking, they start going out via the narrow corridor. As they go ahead, a new teleprompter turns on and the text is spoken automatically through a preloaded voice.
But the most important aspect is that the story changes with every new teleprompter. The story either ends up being either a very sordid manipulation or a sarcastic take depending on the tone.
Teleprompters in action
Getting the voice
To increase the 'skin in the game' factor for the participants, I decided to have the manipulated stories be narrated through the participant's voice but because Machine Learning is a relatively new field for me, so it was a challenge to get the
Flow of voice cloning with the three main components: encoder, decoder and vocoder
Building the first voice cloning platform
SV2TTS GUI - By Corentin Jemine
SV2TTS is a deep learning framework in three stages. In the first stage, one creates a digital representation of a voice from a few seconds of audio. In the second and third stages, this representation is used as a reference to generate speech given arbitrary text.
The speaker encoder’s job is to take some input audio (encoded as Mel spectrogram frames), of a given speaker, and output an embedding that captures “how the speaker sounds.” SV2TTS is built on Tacotron 2. Tacotron 2 is said to be an amalgamation of the best features of Google’s WaveNet, a deep generative model of raw audio waveforms, and Tacotron, its earlier speech recognition project.
But the problem was although SV2TTS was able to replicate a form of the voice, the narration through the vocoder was in an American accent which was not very relatable for the participants.
Recording voice samples to train the cloning model
Then I got to know about a platform called Descript which helped me to train the models and get an almost accurate replication of the recorded samples.
Let the making begin
Laser cutting the models and making the stands
Testing out placements with cardboard cutouts in one of the presentation halls.
Making of the final models
Building the circuit
Second setup in the design shop space
Testing out different lights and placements with carboard cutout models.
Experimenting with lights in the space
The final setup in the Design Gallery Space
Fibers were added to the models to make them more realistic. Smoke was used to concentrate the light.
Final shoot in progress
“How do you believe in what you believe?” In this transtemporal competition of information, how do you determine the fine line between fiction and reality? ‘Sutro ke Anusaar’ (English: According to Sources) is an experiential installation which lets the participant be the judge. As a master manipulator (more so an evil one), the Nazi propaganda minister, Joseph Goebbels said, “A lie, repeated a thousand times, becomes a truth.” What if your own voice deceives you? In this decisive shift of ‘Alternative Fact’ regime, this installation seeks to explore the relation between “truth” and “personal biases/interests”. ‘Sutro ke Anusaar’ wants to ask you personally, “Can you trust yourself?”
