a day at the museum
context
Screen-based interactions exist in a physical and temporal context, requiring the UX designer to attend not to usability and efficiency issues, but also embodied and situational aspects of an interactive experience. So far this semester, you have focused your attention primarily on screen-based interactions without an explicit indication of how users might interact with these technologies in a specific time and place. To maximize a patron’s aesthetic experience (cf., John Dewey), you must attend to multiple points of contact (i.e., touchpoints) that might exist in a technology-rich environment.
goal
In this project, your goal is to design a museum experience with multiple digital and physical touchpoints. As a group, you will choose a period of design history that will serve as a theme for your museum exhibit. Based on this theme, you will identify appropriate technologically-rich and meaningful interactions that allow patrons a sense of embodied participation in the exhibit, enabling them to learn about the period of design history through immersion and participation. At minimum, you should develop an integrated set of three touchpoints that vary from large-scale to small-scale screen interactions, alongside physical or print media within a built environment. These interactions should work together in a symbiotic way to facilitate the museum’s learning goals, while also serving as compelling aesthetic experiences in their own right.
process
project overview and scope
In this project, we were tasked with creating a museum experience that includes multiple touchpoints, digital and physical, that encompasses a specific period of design history.
We spent a few days researching different periods of design history, and ultimately elected one to move forward with.
Scoping blob
To the left you will see a large blob; this represents our design area: a museum experience. We briefly researched within each identified area and thereafter chose one to proceed.
Minimalism — Frank Stella, 1940s
Photography — Bob Adelman, Civil Rights Movement
Op Art — Bridget Riley, 1960s
Lettrism — Isidore Isou, 1940s
We decided to proceed with Lettrism, for its multi-sense experience that could be translated into a technologically-rich, meaningful museum experience.
As a group, we were intrigued by Lettrism’s visual and especially audible components, and how we could bring these together to create an immersive museum experience.
secondary research
After choosing a final focus, we began researching about the design period.
Goal: to learn more about Lettrism and its visual and audible components, as well as its main contributors.
What is Lettrism?
Lettrism was created by Isidore Isou in the mid-1940s to rebel against Surrealist poetry. He believed that poetry should be deconstructed down to its most basic elements: letters and symbols. This expressive form of poetry was created using the Lexique Des Lettres Nouvelles, a sonic alphabet of 130 sounds that crossed letters and symbols with distinct sounds.
Who contributed to Lettrism?
Isidore Isou began the Lettrism design period. He was disappointed by surrealist poetry so he invented a new type of poetry using only letters, writing symphonies with “lettries” — essentially a new, avant-garde art form where sound is created from letters.
Examples of Lettrism
.jpg)
.jpg)
.jpg)
The examples above are visual compositions from “lettries,” a language of symbols, each with a distinct sound.
An audible experience
What intrigued our group about Lettrism in particular was the audible experience evoked by these works. Because each symbol corresponded to a given sound, a combination of these symbols would create a distinct piece of music, or “symphony,” according to Isidore Isou.
Hear a Lettrism symphony using the QR code to the left.
Navigate to timestamp: 31:20
early ideation
Goal: to identify potential contents, touchpoints, and interactive components of our exhibit utilizing visuals and audio.
After choosing a final focus, we brainstormed a variety of ideas for our museum exhibit:
Contents of exhibit
- Paintings from the 40’s-60’s
- Poetry and film projection from the era
- Definition of Lettrism
- Video of Isidore Isou explaining lettrism
- Sounds of Isidore singing his art while entering the exhibit
- Examples of “lettries”
- Tunnel at beginning with symbols and sound playing
Touchpoints
- Creating your own art digitally on an iPad
- Launchpad/piano on ground where a sound is triggered by a letter and you can build your own symphony/song using lettrism
- Type letters, symbols, or a phrase and convert into a song
- Xbox Kinect sensor — drag letters together to create sounds and visual art at the same time
- Lettrism work on the walls, with a button & headphones where you can hear what each work sounds like
- Magnets — drag around to create lettrism art
Probe activities
- Series of symbols — see what sounds can be associated with each symbol
- Stamps on images
- Covering words with lines or paint
- Listen to a lettrism poem and draw a piece of art based on what is heard
- Show a lettrism piece and ask for the participant to write down their emotions
- Group letters/icons into groups of emotions
- Before/after — what you think vs. actual lettrism/hypergraphy symbols
- Lettrism fridge magnets
probes
A main component of this project was the incorporation of probes. Upon getting informed about Lettrism, we decided we could move forward with the creation of this.
Goal: to better understand the emotion provoked by Lettrism poetry sound pieces through user engagement.
Our vision for probe utilization was to set up a journey to learn about Lettrism throughout a few days of short activities. Each activity would build upon the previous to expand the probe receiver’s knowledge of the art style in small doses. Each day had a specific purpose, and the order of activities was strategically thought out based on the goals we set in place. The four goals for the four activities were as follows:
Day 1: Exposure
User is exposed to the an initial dose of Lettrism. This initial dose is a full fledged showcase of what Lettrism sounds and looks like. There is no explicit explanation of what it actually is until Day 2, so at the end of this day we ask the participants to reflect on their initial thoughts and emotions on Lettrism so far. We also ask them to see if they have ever heard of these sounds anywhere else in their personal life.
Day 2: Education
User is properly educated on the history of Lettrism and what exactly it is. We show common art pieces to give them an extra taste of Lettrism. There is no reflection on this day as we have given them a good amount of information.
Day 3: Creation
User is prompted through an activity of creating their own Lettrism. We provide them images to place their Lettrism on top of. These images are supposed to invoke different emotions with the hopes to see how it affects their Lettrism creation.
Day 4: Reflection
User is given some final questions to reflect on over the whole process on learning Lettrism sand their experience.
contents of probe
before beginning
Welcome
The goal of our probe was to understand the emotional journey a user might go through when encountering Lettrism for the first time. We are aware that not many people know much, if anything, about Lettrism, and that it is more non-traditional than what someone might think when they’re talking about a museum exhibit. Thus, we wanted to gain insight into what emotions people experience when they are engaging with the art.
day 1
Exposure
On Day One, we had our participants listen to a portion of an audio clip of Isidore Isou sounding out a “symphony” to introduce them to the sonic aspect of Lettrism. After they listened to about a minute of the audio, we had them reflect on what they had heard with questions surrounding thoughts, feelings, emotions, and real life.
day 2
Education
On Day Two, we had our participants watch another video, this time of Isidore Isou explaining Lettrism. We also provided them with multiple pieces of Lettrism artwork to start exposing them to what the visual side of Lettrism looks like. We also included a paragraph introducing them to the history of Lettrism to help them understand more about what they were seeing.
day 3
Creation
On Day 3, we had our participants create their own artwork. We provided them with four different images to choose from to add symbols to, but also invited them to use their own image if they wanted. We instructed them to use whatever symbols they like - whether that be symbols from the examples in Day 2, or their own symbols if they wanted to create some. After drawing the symbols, we encouraged them to affiliate sounds with some of the symbols they used. We asked them to include what it might sound like on the back of the card.
day 4
Reflection
For our final day, we had our participants reflect on the experiences they had throughout the different probe tasks. We asked them what they found interesting and what they found confusing, and we also asked if they found this as a valuable learning experience. We also asked them to include any questions they still had about Lettrism to understand confusions our patrons may have after learning a bit about the movement and art.
analyzing probes
We sent three probe kits out to three, unique individuals. Each individual was a student at Purdue University between the age of 18 and 21. We chose these individuals as they were easily accessible, and so we could receive our probe back in a timely manner.
takeways
Lettrism is very out of the ordinary - it isn’t similar to a lot of what people have experienced before as far as art goes
The symphonies are very startling at first exposure - it involves sounds that users do not typically hear
Relating the symbols to the sounds they would make was very difficult. Typically in language, we learn the sounds that are associated with symbols (letters) at a very early age. When we are left to create sounds that would be associated with new symbols, it’s difficult for us to do.
The language of calling the sonic aspect of Lettrism “music” or “symphonies” is confusing because it does not line up with users’ mental model of what “music” or “symphonies” are.
The users were confused on the meaning of the art - visual and sonic - and didn’t know how to interpret it.
visualizing our ideas
Goal: to visualize our internal ideas as a full mockup of the museum.
Takeaways
To begin this process of sketching and ideation, we decided to each come up with a mockup our intended museum experience. We did not focus on the specific touchpoints that we would include, but the journey that one would have to go through to experience each of the touchpoints.
From each of our sketches we noticed three key similarities. We all imagined including a hallway of some sort to lead the way into the Lettrism experience, a section that was dedicated to just the history of Lettrism, and a larger section that contained our most prominent touchpoint.
annotated portfolio
Goal: based on our probe results, we compiled an annotated portfolio to better represent the emotion we aim to evoke during our sound-oriented museum experience.
Takeaways
Emotion is a large factor in our museum exhibit, especially after considering the outcomes of our cultural probes, where individuals experienced a wide array of emotions due to exposure to Lettrism. With our space visualized and our initial ideas for the contents of the exhibit, we decided to further explore some of the emotions that could be caused by them. By isolating the emotions that each part of the exhibit evokes, we can better understand the emotions of the museum patrons in the form of an experience map.
mapping the experience
Goal: To understand what we, as designers, want patrons to think, feel, and how we want them to behave, while navigating our museum experience. We also want to be taking into considerations potential pain points and the various environments and touchpoints a patron might enter.
Takeaways
The above emotion map is a representation of our intended visual, audible, behavioral, and emotional components of our patrons. The accuracy of these will be determined through future testing, where we can identify how we can better convey the experience above in our exhibit.
museum narrative
Goal: To create a meaningful narrative for our museum-goers to travel through, utilizing multiple senses along the journey to deliver an immersive and interactive experience.
We wanted to guide the patrons through our exhibit the same way we walked them through our probe: exposure, education, creation, reflection. This provided insights into a wide variety of aspects cognitively, audibly, and visually.
Part 1 | Exposure
The journey begins with a walk through a dark hallway, illuminated by glowing Lettrism symbols on the ground, guiding the patrons through the narrow space. On the walls are mounted glowing buttons, spaced a few feet apart. Pressing these buttons returns a short audible segment of a Lettrism song, one to two seconds in length. The purpose of these short sounds is to prime the patrons for the rest of the exhibit through what is known as systematic desensitization. This graduated exposure technique is used to slowly expose people to uncomfortable experiences, such as fears. We thought that beginning with full Lettrism audio pieces might be overwhelming for patrons, so we are introducing the art form incrementally.
Part 2 | Education
This next space would answer the question: “What is Lettrism?” through a text segment on the wall, a video of the creator, and a plaque under the video with a short biography of Isidore Isou. An important part of this educational phase is to explain the history of Lettrism; for example, why it was created in the 1940s and what influences is has had in the following decades.
Following the brief history lesson, we wanted the patrons to experience Lettrism to its full extent. The following hallway would include booths showing various pieces of Isidore Isou’s art, as well as Lettrism audio playing through headphones. This immersive activity would appeal to multiple senses and leave an emotional impact. After listening and visualizing a piece of Lettrism, the patron would enter an emotion to add to a larger emotion word cloud. On the opposite side of the booths is a room focused on Isidore himself, with visual works and personal stories. The mode of entry into this room is the IsiDOOR.
Part 3 | Creation
Following the education phase, patrons would move into a more open space for a hands-on activity. This creative space would feature several kiosks where people can create their own Lettrism piece. The patrons would first select a background image, then they would overlay Lettrism symbols from a vast library. This activity is highly immersive and puts the patrons in the shoes of the Lettrism creator himself, Isidore Isou.
Part 4 | Reflection
After creating their own art piece, patrons would walk through a gift shop where they can purchase a print of their Lettrism as a memento of their experience. Following the gift shop, the patrons would walk through a final hallway with quotes from Isidore Isou, leaving the patrons with something to think about. We wished to leave the patrons with the takeaway that they can still be expressive and creative with no context or semantics behind the art form — this is the true meaning of Lettrism.
mapping the museum experience
Goal: To create a meaningful narrative for our museum-goers to travel through, utilizing multiple senses along the journey to deliver an immersive and interactive experience.
An Initial Sketch
As a team, we whiteboarded the layout of our exhibit, keeping in mind how physical space, lighting, and architecture can affect the patron’s experience. We wanted the initial phase to evoke feelings of confusion, curiosity, and uncomfortability. For this reason, the main hallway has been designed to be relatively narrow, and the exhibit slowly opens up at each touchpoint. The largest section is the third touchpoint, providing breathing room for the patrons to create their own unique piece of Lettrism on their own digital kiosk.
Digitizing the Layout
After we finalized our layout on the whiteboard, we created a digital 3D render in HomeByMe. We chose this specific website over SketchUp because it provided the most customization and flexibility. HomeByMe also allowed us to view our architecture from the interior at any location and at any angle, from the perspective of a patron.
visualizing museum emotion
We hope, through understanding these emotions, we can better balance them throughout the exhibit. Having too much of one emotion, or not enough of any, will lead to an overwhelming and unenjoyable experience, or an underwhelming and bland experience. We wanted to create a truly aesthetic experience, full of a balance of emotions.
visualizing touchpoints
Goal: to sketch our three interactive touchpoints patrons would engage with in our museum exhibit so we can share these during our class critique and test them.
Touchpoint 1 | Lettrism Hallway
In this hallway, patrons are met with a dark hallway. On the floor, they will see Lettrism symbols, and on the walls, nondescript buttons. These buttons, when pressed, will play a sound that represents a symbol. This is to give the patron a gradual exposure to the jarring sounds that are associated with Lettrism.
Touchpoint 2 | Emotion Cloud
Patrons will choose a visual piece of Lettrism, then hear its associated symphony via a sound shower. The touchpoint visitor would then type in an emotion that would be added to a collaborative word cloud.
Touchpoint 3 | Create Your Own Lettrism
Here patrons will finish off their Lettrism experience by doing the visual part of it. This is where the sounds patrons made in the Sonic Studio will be added.
class feedback on touchpoints
Goal: to become aware and learn from potential holes in our proposed touchpoints.
Touchpoint 2 seemed detached because the entire point of Lettrism is that there is no meaning in the piece
We need to make sure that we are conveying the lack of semantic meaning so people understand the movement of Lettrism
Our exhibit seemed to be focused on including touchpoints only on the walls and did not incorporate the space effectively into our experience
Our exhibit seemed to be too individualistic. There needed to be something to engage more than one patron at a time with collaboration
The museum experience felt too educational. There needed to be something that took the patrons out of the educational space
The word cloud did not seem to fit in with the experience too well. Audience was unsure of what its purpose was
comic book testing our touchpoints
Goal: to observe a user’s path through a given touchpoint by presenting the user with a series of images.
Overview
The main idea with this testing was to present users with an image (the first image of each row below) then the user would “click” with their finger to navigate through the touchpoint. The “comic book” name of this testing comes from the idea that these images, when presented sequentially, create a comic and tell a story, like each of our touchpoints.
tested on 3 participants
Male; 20; Computer Science Student
Female; 20; Engineering Student
Female; 19; Public Health Student
touchpoint 1: Lettrism Hallway
touchpoint 2: Word Cloud Booth
touchpoint 3: Lettrism Kiosk
feedback on our touchpoints
touchpoint 1: Lettrism Hallway
When presented with the first touchpoint, users did not fully understand what the hallway was, which sparked their curiosity as they wanted to continue to investigate.
The most prominent and intriguing part of this touchpoint was the Lettrism on the floor of the hallway.
touchpoint 2: Word Cloud Booth
One participant felt that this touchpoint had few interactive elements. The primary function of the sound booth was to expose patrons to Lettrism visuals and audible samples. It would be most meaningful to have them physically create something.
This touchpoint was disconnected from the other two touchpoints in the exhibit. Our goal was to tell a story through a journey of steps that build upon each other, and this was a missed opportunity with the sound booth as it stands.
touchpoint 3: Lettrism Kiosk
When presented with the kiosk prompt, users were interested to see what creating their own Lettrism really meant, and had the curiosity to explore more deeply.
The circles on the bottom of the “draw your own” symbol section was confusing to users. These were meant to represent colors, and would be made clearer with a higher fidelity, colored, mockup.
teachback testing the sonic studio
Goal: to test users’ comprehension and sensemaking abilities of our digital touchpoint to the extent that they are able to teach the concept back to the designer
Overview
The main idea with this testing was to allow users to provide their understanding of our proposed design. Users were asked to explain how certain parts of the exhibit work.
tested on 3 participants
Male; 20; Computer Science Student
Female; 20; Engineering Student
Female; 19; Public Health Student
Class Critique Feedback
Through our class critique, we one of the main pieces of feedback we received was regarding our second touchpoint: the word cloud. We decided to transform this touchpoint into a Lettrism Sonic Studio, where patrons would enter a studio-like environment with a microphone and headphones, and create their own Lettrism sounds, that they would then use to create their final piece of Lettrism.
Studio
The patron records individual Lettrism sounds, names them, then submits them to be used at the end of the exhibit.
Sounds Submitted
This page signals to the patron that their sounds have been submitted to the kiosks at the end of the exhibit.
feedback on the sonic studio
One of the testing participants struggled to use this interface with a mouse. This interface would be better on a tablet like an iPad (something that is activated by touch, rather than a mouse).
Both participants thought that the system did not return proper feedback after recording an audio piece. They weren’t sure if their sound was successfully recorded. A check mark or color alteration would be a meaningful signifier.
After recording a sound, one participant realized that there was no way to play back or retake the sound after the initial take. A couple more functions represented by icons would allow for greater flexibility.
There was not enough context and instruction within the interface to tell the user the purpose of this interface. Adding a page before the main interactive portion would give the user more understanding before engaging in the main portion.
One participant was unsure what type of audio sample to record. She verbalized the question: “Do I record my reaction to the art?” An instruction page would help alleviate this confusion.
iteration on the sonic studio
Goal: to take the feedback we received via comic book and teachback testing and iterate upon our previous design.
We iterated upon our second touchpoint, the Sonic Studio, utilizing the feedback from our comic book and teachback testing.
Splash Page
This page is presented to the user before they enter the studio, giving them an initial idea of what they will be doing in the Sonic Studio.
This was added from feedback from testing, where users signaled that they did not have enough direction into what they were supposed to do.
Studio
In the studio, users record individual Lettrism sounds to be used at the final step of the exhibit.
We changed the wording on this page to signal to users that they can re-hold a button to re-record a sound.
Sounds Submitted
This page signals to the user that their sounds have been submitted to the kiosks at the final part of the exhibit.
We adjusted the wording on this page, as well, to clarify that these sounds will appear at the kiosks at the end of the exhibit.
We also decided to put this interface on an iPad rather than a computer. During testing, users had trouble clicking and holding with a mouse so we decided that an iPad would be easier — to record via touch rather than via a computer mouse.
mocking up our exhibit
Goal: to be able to visualize our museum exhibit concept without the need to build the physical architecture
Lettrism Hallway
Patrons encounter this mysterious, dark hallway upon entering the exhibit to both lure them into the exhibit and spark their curiosity. They can click buttons on the walls and hear small snippets of Lettrism sounds.
After the Hallway
This is the view when a patron exits the dark, Lettrism Hallway and enters the Learning Hallway.
.png)
.png)
Learning Hallway
Here, patrons can learn about the history of Lettrism, see a video of the creator of Lettrism, Isidore Isou, and see examples of Lettrism art.
Isidore Room
History of the creator of Lettrism, Isidore Isou, will be contained in this room. It is meant to inform patrons of where Lettrism came from and how it was formed.
.png)
.png)
Sonic Studio
This is an interactive touchpoint where visitors can create their own sounds and combine them to form the audio portion of Lettrism. They are able to record sounds of their choice and them combine them together to listen to them played back on the Lettrism piece at the Create your own Lettrism touchpoint.
Create your own Lettrism
This is another highly interactive touchpoint where patrons will finish off their Lettrism experience by doing the visual part of it. This is where the sounds patrons made in the Sonic Studio will be added.
.png)
.png)
Exit Hallway
This final hallway leads the patron to the exit. On the walls of this hallway are a variety of quotes from Isidore Isou - “No word is capable of sending the impulses one wants to send with it.” and “Feelings demand living space.” are a few examples. These quotes from Isou are meant to leave the patron with something to think about and reflect on as they leave the exhibit with their own Lettrism piece.
a video to sum it up
museum goer learning outcomes
#1 | To have the courage to create art without fear of a lack of meaning
One of the main aspects that sets apart Lettrism from other forms of art is that it is devoid from all semantic meaning, yet was still an impactful experience. At the end of our exhibit, we want users to be more comfortable in creating art, and to know that it does not need to possess semantic meaning to be art. We accomplish this by giving patrons the opportunity to create their own visual and audible Lettrism.
#2 | Learn about an unusual form of art
The art movement that our art focused on was Lettrism. Through interaction and applications in our exhibit, patrons should be able to explain what this is and where it came from. By learning about this unusual form of art, we want users to be more open in thinking of how art is constructed.
#3 | Understand that art is not restricted to the styles you learn about in school
Lettrism was founded in the mid 1900’s in retaliation to surrealism. Not every piece of art fits into the categories people learn about in classes, and Lettrism is a prime example of that. We want our patrons to understand that with art and design, the possibilities are limitless.
going foreward
With more time, we knew we wanted to make a few adjustments and additions to our exhibit:
1. Refine our coded touchpoint providing more signifiers
- Add a check mark or green highlight after recording a sound
- Implement post-record options to allow playback and retake of audio clips
2. More testing with our 3D mock ups to give the people we were testing a more realistic experience
- This would give allow us to potentially get more of an authentic feedback.
3. Coding of the final touchpoint interface (creating Lettrism)
- Testing to see how users interpret a high-fidelity interface and create their own Lettrism
final thoughts
This project was a long one — at least looking back now. From the beginning we aimed to create an immersive and enjoyable museum experience. We did encounter some rough patches along the way, which we worked through as a team.
1. Scoping down
At the beginning of the project, our team knew we had to scope down quickly and be decisive, to give us more time to develop a solution. However, this was not the case despite our effort. We spent nearly an entire week moving from one scope to another, until we landed on Lettrism. We think the primary reason for this was the critique sessions where we received feedback that gave us the impression that changing our scope would balance out that feedback; however, we soon realized that changing our scope was not the solution, but rather changing our thinking within that scope to align with the feedback we received during critique sessions.
2. Storytelling
As soon as we came up with a general idea for our museum exhibit, we were anxious to start creating. However, we realized that, in order to provide the patrons with the best experience possible, we needed to tell a narrative. Turns out, storytelling was crucial to creating a more cohesive museum exhibit. Thus, we spent a meeting creating a narrative for our exhibit, with which we could eventually create an experience map.
3. Delegating tasks
This project had many moving parts, especially toward the end where we were putting everything together: documentation, media component, presentation — the list goes on. At this point we knew delegating tasks and working on them individually would be the best option. Thus, we each championed individual tasks and worked on them, while asking questions in-between. This made for a much smoother finish of the project, and brought us more confidence in our process.