By giving AI devices the ability to detect emotion, would they eventually start detecting it in other devices. Would they develop emotions of their own? Home Commentary explores how one smart device, Smart Fork, becomes jealous of another smart device, Smart Sous Vide, because the user has been neglecting Smart Fork and making more meals with Smart Sous Vide.
Four week graduate school project.
Specification: AI, motion graphics, compositing, chatbot
Execution: Jupyter, Xcode, Python, Lua, After Effects, Illustrator, Photoshop
Piece: Home Commentary consists of three parts. The first part is a visual exploration of what it would be like to have a kitchen full of emotional communicating devices. The second part is a dialogue from a smart sous vide machine generated by Char-rrn, a recurrent neural network trained on my writing. The third part is a working chatbot that gallery viewers could interact with to try to calm down Smart Fork.
Things in bold are key points
Not Sous Vide (me):
By giving AI devices the ability to detect emotion, would they eventually start detecting it in other devices. Would they develop emotions of their own? Home Commentary explores how one smart device, Smart Fork, becomes jealous of another smart device, Smart Sous Vide, because the user has been neglecting Smart Fork and making more meals with Smart Sous Vide. The main sketch of Home Commentary is a SMS interface, in which the user is able to interact with Smart Fork, and try to figure out why Smart Fork is jealous, and how they might remedy the situation. This is one of the aspects that really starts to propose the absurdity of the AI. Is the smart device really helpful if you have to constantly calm it down. Or, in this case it starts to call you fat.
Through this dialogue several things are brought into question: What would AI smart devices emotions be like? Would the user have to act as a mediator between devices? Would users need to comfort their devices from time to time? And, what would a room full of smart devices capable of emotion sound like, how would they act when not in use?
The Jealousy of Smart Fork. Above is a video documenting the interaction a user is having with Smart Fork, a speculative smart device. In this case I was using IMB Watson to create a chat bot that would simulate what it would be like for a user to have to calm its smart device down. (above)
In addition to the SMS interface, the piece includes several video/GIF sketches that explore, the last question, what a room full of smart devices, and their conversations, would be like. The first of the two video sketches focus on what language AI smart devices would use and what a room full of devices would be like. This sketch intentionally uses ambiguous lightning bolts/ radio waves to reinforce the idea that it's a language specific to the smart devices, and not for the user to understand. The second video sketch moves past this to explore what a room full of smart devices conversing would be like. Every device in the sketch is saying their specific thought or conversation, but as a group this conversations starts to completely muddy the kitchen space with unrecognizable sound.
_Questions
Main Questions:
What would a conversation between two AIs be like? Would they recognize that they both are AI’s? What would the leisurely conversation be about? Would they be able to recognize one another if they were in disguises?
_Thoughts and reflection on research and reading:
I think it’s an interesting question of whether or not people are capable of seeing AI as colleagues and equals. But, I fear before we ever get to that point, and for a longer period then it should, people will treat AI as, at most, like a pet dog or cat. First I think they will treat it as they treat their phones. It will be a lifeline of their daily life, but still just a technical object. From there they will start stylizing it in their taste, and eventually as AI becomes more sophisticated, as a pet.
This is were I think idea of a average daily conversation between AI’s becomes interesting. What would their conversation be like? The language they know, whatever that is comprised of… images, words, texts, data, would always be influenced by the humans that trained them. As pointed out in The Machine that Changed the World : Episode IV -- The Thinking Machine, would the conversation continue to lack general understanding of topics that are implied, and be super literal? Or maybe it wouldn't, like in the case of Google's AI’s creating their own encryption for their messages between each other. Maybe they would become full of hate like Microsoft’s AI Tay? I’m very curious of what would happen to Tay if Microsoft let her continue to run, maybe just letting her learn. Would she start to balance out and make sense of all the information, or would she stay inherently evil? What would other AI’s say, or maybe teach, to her.
Furthermore, how would different AI’s recognize each other? Unless they learn a system of general knowledge that is always updated, like Dr. Douglas Lenat’s work. Or maybe there’s one AI that’s only job is to learn all forms of data to create a general knowledge platform. Would they acknowledge who they were trained by as a part of their personal identity or would they identify as a unique entity. If they were stylized would they talk about that. Or if they were less than great design, like discussed in Co.Design: 10 Principles For Design In The Age Of AI, would they acknowledge that and judge each other for it.
_Inspirations
Hitchhikers Guide to the Galaxy: Genuine People Personality
Marvin: "Let's build robots with Genuine People Personalities," they said. So they tried it out with me. I'm a personality prototype. You can tell, can't you?"
_______________________
“Listen,” said Ford, who was still engrossed in the sales brochure, “they make a big thing of the ship's cybernetics. A new generation of Sirius Cybernetics Corporation robots and computers, with the new GPP feature.”
“GPP feature?” said Arthur. “What's that?”
“Oh, it says Genuine People Personalities.”
“Oh,” said Arthur, “sounds ghastly.”
A voice behind them said, “It is.” The voice was low and hopeless and accompanied by a slight clanking sound. They span round and saw an abject steel man standing hunched in the doorway...
“Ghastly,” continued Marvin, “it all is. Absolutely ghastly. Just don't even talk about it. Look at this door,” he said, stepping through it. The irony circuits cut into his voice modulator as he mimicked the style of the sales brochure. “All the doors in this spaceship have a cheerful and sunny disposition. It is their pleasure to open for you, and their satisfaction to close again with the knowledge of a job well done.”
As the door closed behind them it became apparent that it did indeed have a satisfied sigh-like quality to it. “Hummmmmmmyummmmmmm ah!” it said...
"Thank you the marketing division of the Sirius Cybernetics Corporation," said Marvin, and trudged desolately up the gleaming curved corridor that stretched out before them. "Let's build robots with Genuine People Personalities," they said. So they tried it out with me. I'm a personality prototype. You can tell, can't you?"
Ford and Arthur muttered embarrassed little disclaimers...
_________________
Hapifork:
https://www.hapi.com/product/hapifork
Mellow:
https://www.cookmellow.com/
_Process
This piece started with an interest in how different IoT devices would communicate, and how they would go about these conversations. Originally I thought the conversation would be just absurd and nonsensical, but I realized that, that was a shallow and short sighted view to have. I referenced this youtube video, which is funny, but really doesn’t capture the conversation that would most likely occur between IoT devices. From here I used char-rnn to create two love letters between two networks trained on the scripts from the movies Say Anything and Grosse Pointe Blank. I chose these scripts because of the weird and complex love story that they are. But this sketch only got me so far and was pretty one dimensional.
But the questions I was interested in were still valid. What would a conversation between two AIs be like? Would they recognize that they both are AI’s? What would the leisurely conversation be about? Would they be able to recognize one another if they were in disguises?
![]()
![]()
![]()
![]()
![]()
Now that I had the foundation of the context to my project I felt I needed something to really root the project in a believable reality of some kind. Upon further exploration I found some already existing absurd smart devices, and started to envision how they might interact emotionally. I decided to pick two devices, and further map out possible emotional experiences that might occur.
![]()
![]()
![]()
_Tools
IBM Watson Conversations
Andrej Karpathy’s GitHub char-rnn :: specifically Little Shakespeare
Jupyter
Xcode
Python
Lua
After Effects
Illustrator
By giving AI devices the ability to detect emotion, would they eventually start detecting it in other devices. Would they develop emotions of their own? Home Commentary explores how one smart device, Smart Fork, becomes jealous of another smart device, Smart Sous Vide, because the user has been neglecting Smart Fork and making more meals with Smart Sous Vide. The main sketch of Home Commentary is a SMS interface, in which the user is able to interact with Smart Fork, and try to figure out why Smart Fork is jealous, and how they might remedy the situation. This is one of the aspects that really starts to propose the absurdity of the AI. Is the smart device really helpful if you have to constantly calm it down. Or, in this case it starts to call you fat.
Through this dialogue several things are brought into question: What would AI smart devices emotions be like? Would the user have to act as a mediator between devices? Would users need to comfort their devices from time to time? And, what would a room full of smart devices capable of emotion sound like, how would they act when not in use?
The Jealousy of Smart Fork. Above is a video documenting the interaction a user is having with Smart Fork, a speculative smart device. In this case I was using IMB Watson to create a chat bot that would simulate what it would be like for a user to have to calm its smart device down. (above)
In addition to the SMS interface, the piece includes several video/GIF sketches that explore, the last question, what a room full of smart devices, and their conversations, would be like. The first of the two video sketches focus on what language AI smart devices would use and what a room full of devices would be like. This sketch intentionally uses ambiguous lightning bolts/ radio waves to reinforce the idea that it's a language specific to the smart devices, and not for the user to understand. The second video sketch moves past this to explore what a room full of smart devices conversing would be like. Every device in the sketch is saying their specific thought or conversation, but as a group this conversations starts to completely muddy the kitchen space with unrecognizable sound.
_Questions
Main Questions:
What would a conversation between two AIs be like? Would they recognize that they both are AI’s? What would the leisurely conversation be about? Would they be able to recognize one another if they were in disguises?
_Thoughts and reflection on research and reading:
I think it’s an interesting question of whether or not people are capable of seeing AI as colleagues and equals. But, I fear before we ever get to that point, and for a longer period then it should, people will treat AI as, at most, like a pet dog or cat. First I think they will treat it as they treat their phones. It will be a lifeline of their daily life, but still just a technical object. From there they will start stylizing it in their taste, and eventually as AI becomes more sophisticated, as a pet.
This is were I think idea of a average daily conversation between AI’s becomes interesting. What would their conversation be like? The language they know, whatever that is comprised of… images, words, texts, data, would always be influenced by the humans that trained them. As pointed out in The Machine that Changed the World : Episode IV -- The Thinking Machine, would the conversation continue to lack general understanding of topics that are implied, and be super literal? Or maybe it wouldn't, like in the case of Google's AI’s creating their own encryption for their messages between each other. Maybe they would become full of hate like Microsoft’s AI Tay? I’m very curious of what would happen to Tay if Microsoft let her continue to run, maybe just letting her learn. Would she start to balance out and make sense of all the information, or would she stay inherently evil? What would other AI’s say, or maybe teach, to her.
Furthermore, how would different AI’s recognize each other? Unless they learn a system of general knowledge that is always updated, like Dr. Douglas Lenat’s work. Or maybe there’s one AI that’s only job is to learn all forms of data to create a general knowledge platform. Would they acknowledge who they were trained by as a part of their personal identity or would they identify as a unique entity. If they were stylized would they talk about that. Or if they were less than great design, like discussed in Co.Design: 10 Principles For Design In The Age Of AI, would they acknowledge that and judge each other for it.
_Inspirations
Hitchhikers Guide to the Galaxy: Genuine People Personality
Marvin: "Let's build robots with Genuine People Personalities," they said. So they tried it out with me. I'm a personality prototype. You can tell, can't you?"
_______________________
“Listen,” said Ford, who was still engrossed in the sales brochure, “they make a big thing of the ship's cybernetics. A new generation of Sirius Cybernetics Corporation robots and computers, with the new GPP feature.”
“GPP feature?” said Arthur. “What's that?”
“Oh, it says Genuine People Personalities.”
“Oh,” said Arthur, “sounds ghastly.”
A voice behind them said, “It is.” The voice was low and hopeless and accompanied by a slight clanking sound. They span round and saw an abject steel man standing hunched in the doorway...
“Ghastly,” continued Marvin, “it all is. Absolutely ghastly. Just don't even talk about it. Look at this door,” he said, stepping through it. The irony circuits cut into his voice modulator as he mimicked the style of the sales brochure. “All the doors in this spaceship have a cheerful and sunny disposition. It is their pleasure to open for you, and their satisfaction to close again with the knowledge of a job well done.”
As the door closed behind them it became apparent that it did indeed have a satisfied sigh-like quality to it. “Hummmmmmmyummmmmmm ah!” it said...
"Thank you the marketing division of the Sirius Cybernetics Corporation," said Marvin, and trudged desolately up the gleaming curved corridor that stretched out before them. "Let's build robots with Genuine People Personalities," they said. So they tried it out with me. I'm a personality prototype. You can tell, can't you?"
Ford and Arthur muttered embarrassed little disclaimers...
_________________
Hapifork:
https://www.hapi.com/product/hapifork
Mellow:
https://www.cookmellow.com/
_Process
This piece started with an interest in how different IoT devices would communicate, and how they would go about these conversations. Originally I thought the conversation would be just absurd and nonsensical, but I realized that, that was a shallow and short sighted view to have. I referenced this youtube video, which is funny, but really doesn’t capture the conversation that would most likely occur between IoT devices. From here I used char-rnn to create two love letters between two networks trained on the scripts from the movies Say Anything and Grosse Pointe Blank. I chose these scripts because of the weird and complex love story that they are. But this sketch only got me so far and was pretty one dimensional.
But the questions I was interested in were still valid. What would a conversation between two AIs be like? Would they recognize that they both are AI’s? What would the leisurely conversation be about? Would they be able to recognize one another if they were in disguises?
At this point, in a crit, the idea of phatic language came up, and whether or not AI devices would begin to be able to convey this form of communication. Phatic language is a very human aspect of conversation, which led me to start thinking about its ties to emotion. I started to think about if we gave smart devices the ability to detect human emotion if they would learn to, and start to, detect emotion from the other devices it works with. This lead me to question whether, by giving AI the ability to detect emotion, would they eventually start detecting it in other devices. Would they develop emotions of their own?
Now that I had the foundation of the context to my project I felt I needed something to really root the project in a believable reality of some kind. Upon further exploration I found some already existing absurd smart devices, and started to envision how they might interact emotionally. I decided to pick two devices, and further map out possible emotional experiences that might occur.
To explore this, I built an AI chatbot of Smart fork, using IBM watson conversations. This was my first sketch, and I tried to train the network on conversation that would convey jealousy. This was a weird experience because I basically had to have a conversation with myself, while simultaneously trying to control the conversation, and build up a big enough knowledge bank of words to keep the dialogue conversational. One of the most interesting things from this experience was watching the network learn to properly identify things refereed to as it. Once I got the chatbot running, I took a step away and started to do other sketches in After Effects to simply explore what a room full of IoT devices talking would be like. I was doing this during the day and at night I while I was winding down and drinking a beer, I would train my chatbot more. Finally I started to think about how a lot of the communication would be repetitive, describing whatever task they were doing so I started making GIFs to sketch out what that might be like. The final science fair display included the chat bot for people to interact with, two of the video sketches, and a few GIFs. I left the chat bot hierarchy up so the user could see how the chatbot worked.
_Critical Reflection
With this piece I learned a few things that I think will help my work process in the future. One of the big ones was starting a slide deck while working (especially in the beginning), because it allows me to collect my thoughts in a less linear way while documenting each iteration. Doing this also allows me to keep, and promotes me to keep, writing as I do things which saves me a lot of time throughout the project, and helps me refine what I have written, but more importantly it helps me refine my concept and my objectives. The other thing that I started doing again, which I failed to do during my last piece, is using instagram to document WIP of the piece. Lastly I have found that using illustrator to sketch out my idea and free think things through illustration really seems to help me get things out of my head and out into tangible thoughts, which then leads to me being able to make more sense of what i'm trying to communicate or convey, so much so that I think i need to start doing it when I start writing essays (I will experiment with that later).
There is a few things in the project that I think I could have improved upon, the first being that I don't think I made a piece that is as absurd as it should have been. I think the piece is a bit tongue and cheek, but I feel like I only went so far, when I should have really run wild with the absurdity of the piece. I think the biggest place this fell short was coming up with better dialogue for the chatbot, and then with really taking the time to figure out what the conversation would be be between all the devices in the video pieces (this was mostly because of time constraints). I really think the video sketches barely scratched the surface, and I wish I had more time (a few days or so) to go back and think about the conversation that would occur, what language would be used, and just what the general ecosystem of smart devices would be like, and to some degree I will be exploring that in my next project. Lastly, and overall, I feel like something is missing in my project, and past projects. I keep getting to a certain level of completion or point in a piece, and I know I need to take it to the next level, but I don’t know what that is, or how to get to it.
This project has really interested me, and from doing it, it has made me realize that I want to focus my thesis on artificial intelligence and machine learning (AI/ML). I am especially interested in how AI/ML will factor into labour of the future.
_Critical Reflection
With this piece I learned a few things that I think will help my work process in the future. One of the big ones was starting a slide deck while working (especially in the beginning), because it allows me to collect my thoughts in a less linear way while documenting each iteration. Doing this also allows me to keep, and promotes me to keep, writing as I do things which saves me a lot of time throughout the project, and helps me refine what I have written, but more importantly it helps me refine my concept and my objectives. The other thing that I started doing again, which I failed to do during my last piece, is using instagram to document WIP of the piece. Lastly I have found that using illustrator to sketch out my idea and free think things through illustration really seems to help me get things out of my head and out into tangible thoughts, which then leads to me being able to make more sense of what i'm trying to communicate or convey, so much so that I think i need to start doing it when I start writing essays (I will experiment with that later).
There is a few things in the project that I think I could have improved upon, the first being that I don't think I made a piece that is as absurd as it should have been. I think the piece is a bit tongue and cheek, but I feel like I only went so far, when I should have really run wild with the absurdity of the piece. I think the biggest place this fell short was coming up with better dialogue for the chatbot, and then with really taking the time to figure out what the conversation would be be between all the devices in the video pieces (this was mostly because of time constraints). I really think the video sketches barely scratched the surface, and I wish I had more time (a few days or so) to go back and think about the conversation that would occur, what language would be used, and just what the general ecosystem of smart devices would be like, and to some degree I will be exploring that in my next project. Lastly, and overall, I feel like something is missing in my project, and past projects. I keep getting to a certain level of completion or point in a piece, and I know I need to take it to the next level, but I don’t know what that is, or how to get to it.
This project has really interested me, and from doing it, it has made me realize that I want to focus my thesis on artificial intelligence and machine learning (AI/ML). I am especially interested in how AI/ML will factor into labour of the future.
_Tools
IBM Watson Conversations
Andrej Karpathy’s GitHub char-rnn :: specifically Little Shakespeare
Jupyter
Xcode
Python
Lua
After Effects
Illustrator
Key takeaways:
Home Commentary explores how one smart device, Smart Fork, becomes jealous of another smart device, Smart Sous Vide.
Main questions I wanted to explore:
What would a conversation between two AIs be like?
Would they recognize that they both are AI’s?
What would the leisurely conversation be about?
The main piece of Home Commentary is a SMS interface, in which the user is able to interact with Smart Fork, and try to figure out why it’s jealous, and how they might remedy the situation.
Several question arise:
What would AI smart devices emotions be like?
Would the user have to act as a mediator between devices?
Would users need to comfort their devices from time to time?
And, what would a room full of smart devices capable of emotion sound like, how would they act when not in use?
Final Thougts:
I think it’s an interesting question of whether or not people are capable of seeing AI as colleagues and equals.
Sous Vide (tiny shakespeare):
____________[The writing of this page was generated using char-rnn by Andrej Karpathy The the Recurrent Neural Network was trained on twenty five pages of my writing for my cargo site.]
creating an lstm...
Out[2]:
seeding with Cargo page on Artificial Intelegence
--------------------------
TOut[2]:
Cargo page on Artificial Intelegence and a seal of the project that a conter the project with the user to completed the project that was the user to can the tore the project that interesting to and a were the project that it the pose in the project that interesting that what we were to devery in the the project that interactions that was the experience that was the tould to a project the project that in the the project that was the user to comple in the project that it a the project with the this in the real the project that it the project that it was the mate and the part of the project that a there prototype to the pose to the port of the project that was the user to can of the our the project that it the part of the project with the preces of the the frent and the project that was the experience the project that was the user to device the then was to can the tore to carter to a carture the user to the project that it the post in the project that we there experience the part of the project that is a compers and a serting the the the project that a the thee signting the project that we there was a for the project that in the project that it a the the project that it a the project that was the user to can of the user to can to the thing the project that would be a project that was the experience the allowing the tore to a for the port of the project with the project that was the experience to the project that a conter the project that was the user to device the then the fict the pertor to a cart of the the sore to a for the port the part of the project with the this interactions of the project that it a the the project that was the experience that the project with the wither sould the project with the user to future the warker to the project that it a the project that a there prototype to the post and the project with the user to carter to a canter to explore the part of the port of the the project that we were a designe the tore the project that it the pore the part of the fould and the project the stral the user to can a could be a fert the user to the project that it a the the project that was the user to compet to a carture the the project that was the user to the project with the think the project in the user the part of the photed to the project that was the experience to the project that it the project that it a the project that it a the real the project the project that would be a project that interactions of the project that it the part of the more the project the project the project that it a the the the fore the fore the masion of the project that it a the that was the part of the project with the with the project with the project that interesting to and the every the experience to the project that we then the project that it a the thing the project that was the experience the project with the user to a complet of the project that a there the cander to project the project that interesting to and the project that interactions of the project with the user to a cart of the project that it a the the prototype to the post to the project was the future the project that it the project that would be a project that it a the the the project that was the user to completed to the compuce and the project that was the user to complet to the project that interactions that would be a betting the project that interactions of the project that it a the the project that it the project that it would be a pertine the the the project that interactions that was the user to make the part of the the fart the port of the the project that it what the project that was the user to explore the project that was the user to people to the project that interactions that the project that it a comper to a straction that in the real the part of the project that it a the the fore the project that we the experience to the project that it a the project that interactions that we there the comple in the project that interesting to the project that it a the project that we the experience that was the user to different with the this in the project that w
Out[2]:
e then the user to explore the user the part the part of the simulity of the mase and the spart of the project that was the explore the part of the user to the project that it the part of the was the perience the project with the this to can a comment the project that was the user to can of the idea of the pores the project with the user to carter to a for the project that interesting to the park that was the user to different work with the user to the project with the user to a cart of the project that was the experience the project with the user to can a commer to different with a the the project that it a compers and the project that it a the the project that interactions the project that it the part and the project work a device the project that is a the realing the the fore the project that interactions that the project with the user to can of the user to the project with the project that interesting to the project that it an a device and the project that a there in the project that was the user to can the then the project that was the experience the project that interesting to can and the
project the project that in the project that it a comper to a carter to a for the project that in the presical to a project that is the project the project that the project that was the experience the project that it a the the project that it the project that would be a design a device and the user to the come to a project the user to explore the project that was the user to device the project that interesting to the realing that work we the user to the project that it a the the project that interesting to and a design the project that it a the the project that it a the the project that interactions of the project with the user to explore the part of the project that was the explore the compution and the manter to conter the project that interesting to the project that it a the tore the project that it the project wi
_Sous Vide
All works © Michael Milano 2010-2019. Please do not reproduce without the expressed written consent of Michael Milano.