Meaning Modes in Design - Fluxible 2016

MARSHA HAVERTY
@mjane_h
Meaning Modes in Design
A Visual Tour
Fluxible 2016

Invisible things about meaning…
We’re going to take a visual tour of some invisible things about meaning, some properties of meaning, that matter with the kinds of designs we
are bringing into the world these days.

Two JJGs
Jessie James Garrett James Jerome Gibson
1950s -70s
Image credit: @UXweek Twitter account Image credit: Wikipedia
Before we visualize these properties, I must mention two JJGs. One JJG is Jessie James Garrett, our contemporary in experience design. The
other is James Jerome Gibson, the founder of the field of ecological psychology, working half a century ago.

Two JJGs
Jessie James Garrett James Jerome Gibson
2002 1950s -70s
In 2002, Jessie James Garrett gave us an exploded planes view of the elements of user experience, in which the structure, skeleton, and
surface planes treat how we use information as raw material in terms of its architecture, visualization, and perception.

Two JJGs
Jessie James Garrett J.J. Gibson
2002 1979
J.J. Gibson gave us a naturalized view of how we perceive information, how we pick it up directly as we act in the environment. (For present
work using the ecological framework developed by Gibson, see Golanka S., and Wilson A. (2013). Embodied cognition is not what you think it
is, Frontiers in Psychology )

Two JJGs
Jessie James Garrett J.J. Gibson
MEANING
And both of them worry about how information influences meaning.

Experience is impossible to model and fully design
Rocket contrail by Seay Photography via Creative Commons
Emergent
Ineffable
Intangible
Subjective
Jessie James Garrett
Closing Plenary, IA Summit 2016
In his closing plenary in the 2016 Information Architecture Summit, Jessie James Garrett lamented with us that experience is impossible to
model and fully design. It is emergent, subjective, intangible, ineffable.

Shifting patterns of awareness
Frames of meaning
An unfolding as a human engages a design Jessie James Garrett
Experience is an unfolding as a human engages a design that creates shifting patterns of awareness, and frames of meaning that a human
assigns to how that particular design in that particular moment-to-moment shifts her patterns of awareness.

How far can we go? Jessie James Garrett
Shifting patterns of awareness
Frames of meaning
He challenges us as designers. How far can we go in understanding these shifting patterns of awareness and frames of meaning and apply
that to our craft?

How we think about meaning affects
what we can see…
But, how we think about meaning affects what we can see.

Photo credit: Barcroft Media via Daily Mail, 3D image of human brain connections
That these beautiful brains we walk around with have somehow become disconnected from the surrounding environment, the very thing
that, over hundreds of thousands of years of human evolution, tuned us as tribal hunter-gatherer poets, is…a little odd.

Photo credit: Pasos Adalente
That we somehow take stuff from our senses, process that so we can recreate it inside our brains, process it some more so we can
manipulate it into calculated decisions which are then output as behaviors, is a very unnatural way to think about meaning.

From The world is the screen: elements of information environments, Andrew Hinton, 2013 IA Summit
I’m ever grateful to Andrew Hinton for showing this slide in a talk he gave in 2013 giving us another way to think about meaning, from our
midcentury JJ Gibson, that puts us and our brains back on the ground, immersed in our environment. Meaning doesn’t happen as an isolated
thing only inside our brains; we’re soaking in it.

Meaning is ecological
J.J. Gibson
1979
From our midcentury J.J. Gibson, we see that meaning is ecological because information is ecological.

Informationis ecological
J.J. Gibson
1979

Visual Information is ecological

Visual Information is ecological
Chair image credit: Yoichi Yamamoto
surfaces
edges
textures
We don’t have to have seen a chair from every possible perspective in every possible light to recognize the chair. It’s the relationships among
the surfaces, edges, and textures that we recognize, that we pick up as invariant, that let us recognize the chair.

MechanicalInformation is ecological
balance
sound
*
touch
cold
warm
pain

All Information is ecological
Perceptual information
*
Language
words on
a surface
words through the air
Sabrina Golanka
All perceptual information is ecological. And present-day embodied cognitive psychologists, in particular, Sabrina Golanka, are drawing on
Gibson’s ecological psychology to frame how language, or conceptual information, though different from perceptual information, is ecological
too. See Sabrina Golonka (2015) Laws and Conventions in Language-Related Behaviors, Ecological Psychology, 27:3, 236-250, DOI:
10.1080/10407413.2015.1068654

All Information is ecological
Information

Meaning emerges
InformationGoal-Directed Actions
Meaning
Meaning emerges when we act with the information in the world around us.

Meaning is dynamic
InformationGoal-Directed Actions
But this is not a static thing. Our goals change, our actions change, and the information in the environment changes. These things must adapt
together. Meaning is dynamic, meaning is an event that unfolds over time.

Meaningful action feels different
depending on
the properties of its flow

Meaning Modes
As designers, if we make deliberate decisions about these properties, then we can facilitate particular meaning modes to suit our design needs.

Impact of a drop of water on a water surface by Roger McLassus https://commons.wikimedia.org/wiki/File:2006-01-28_Drop- impact.jpg
Is information like water?
To get at our first property of meaning modes, we ask the question: is information like water?

temperature
pressure
solid
gas
liquid
The phase-space of water
If we run all the permutations on the different relative amounts of temperature and pressure, we end up with what we know as the phase-space
of water. The nature of water is drastically different as a solid vs. liquid or gas.

Language
The phase-space of information
words on
a surface
words through the air
*
What if we do the same thing for the two types of information: perceptual and linguistic? What is it like to interact with different combinations of
perceptual and linguistic information? Do we find neighborhoods of combinations in which it is drastically different to engage with the meaning
of information?

Perception flows easily Tacit
Reflexive
photo credit: Jason Pratt via Creative Commons
Perceptual information flows easily. It is tacit and reflexive. Once we’ve learned to detect it, we don’t have think about it to engage with it.
Perception flows easily. It has a low viscosity, like water.

Language is viscous, more work to flow Awareness
Associativity
Requires attention
photo credit: edibleoffice via Creative Commons
Language is viscous and takes more work to flow. It is laden with awareness and associativity, and requires active attention for us to engage it.

intense
concentration
intense
coordination
conceptual
visceral
Language
Ease of flow of meaning
Generally, the area dominated by perceptual information is reflexive. The area dominated by linguistic information is more attentive. But we can
get more granular than that. If we have little or no language, all perceptual information, we are engaging with meaning in a very visceral manner.
If we look at the area with little perceptual information, we are operating in the world of concepts. We are thinking conceptually to engage
meaning. If we have a lot of language, especially if it’s abstract, we need intense concentration in this highest viscosity state. If we have a lot of
perceptual information to deal with, we must engage with intense coordination. And when we are faced with a lot of information of either type, we
may become overwhelmed.

intense
concentration
intense
coordination
conceptual
visceral
Language
Before images After images
Remember Twitter before images? This is a waybackmachine snapshot from 2011 [left]. Tweets were made of words and, because we are
context jumping from one unrelated tweet to the next, it is a highly viscous way to engage with meaning. When Twitter introduced images to
tweets, that disrupted the behaviors we had honed to engage with the meaning of twitter. Suddenly there were these perceptual swaths mixed in
with the text. We got used to it, we developed new behavior for engaging Twitter, and we probably don’t think about it at all anymore, but Twitter
changed. It phase-shifted. The nature of engaging with meaning on Twitter is not the same.

intense
concentration
intense
coordination
conceptual
visceral
Language
Palette Gear
The speaker yesterday showed us Palette Gear, physical controls that can be mapped to actions on tools like Light Room. This design phase-
shifts what it’s like to engage with meaningful action in Light Room: it adds more perceptual information. It requires more coordination to engage,
but lets photographers engage multiple simultaneous actions and do adjust the qualities of photos faster.

intense
concentration
intense
coordination
conceptual
visceral
Language
VISCOSITY TOLERANCE
PERSISTENCE PRESENCE
Behavior precision
That’s our first property of meaning modes: viscosity, or the ease of flow of meaning. Next, we’ll look at tolerance, or how precise our behavior
must be to engage the information in a design to maintain the flow of meaning.

WIDE
Tolerance
Forgiving
Information
NARROW
Tolerance
Precise
Breaks the flow of meaning
Information
Tolerance is about precision
A design has wide tolerance if we can veer our behavior a bit and still maintain our engagement with the information. It’s forgiving. If a design
has narrow tolerance, we must be very precise with our behavior and if we veer too much, we break the engagement with the information and
break the flow of meaning.

It’s dark outside. You’re asleep.

You are drawn from sleep by a telltale hum…

You know what to do; you reach in the direction of the thing, tapping around on the glassy surface, trying to make the sound stop. The sound is
still going, so you have to you to lift your head a little, open your eyes a little to see onto the bedside table and finally manage to hit the spot.

That snooze button on your phone may be plenty big to tap with your finger when you’re awake and have your vision light adjusted and are looking
at the face of the phone at a direct angle. But, the intent of this snooze button, the meaningful engagement with this snooze button, is to extend the
continuity of your dream state 8 minutes and 59 seconds longer. Struggling to find a button in the dark, lifting your head a bit, opening your eyes
some to try to see, breaks the flow of meaning of extending your dream state. Or worse, in your tapping around, you may accidentally hit stop and
break capped nature of the flow of meaning of snoozing.

NARROWTolerance
Precise
Imprecision as a design value
WIDE Tolerance
Forgiving
Well-lit,
Wide-awake behavior
Night-blind,
Dream state behavior
The point of snooze is to keep us in our night-blind, dream state. But that requires a design that is forgiving of the kind of behavior we are capable
of in that state: imprecise.

car UI concept and image from: http://matthaeuskrenn.com/new-car-ui/
Let’s look at a different example where imprecision can be a design value. The concept for a car dashboard control panel is just a blank screen.
The human puts a finger anywhere on the screen and the UI comes to that spot. Depending on the number of fingers on the screen, the human
can control different things: temperature, volume, etc. The human moves her finger generally up or down to make adjustments to the ambient
comfort of the car.

Imprecision as a design value
NARROWTolerance
Precise
WIDE Tolerance
Forgiving
This design has a wide tolerance and is very forgiving of the distracted nature of a driver of a car.

Let’s look at another example of a the common button layout design. You may have some perceptual memory that the climate control button is
in the lower right…

But you still have to glance in order to see exactly where you need to press to hit the button and get to the climate controls.

You may also have perceptual anticipation that the temp buttons are in the upper left and right…

But you still must glance to bring your finger to the exact button location. And, if you want to adjust quite a bit, you must lift your finger off the button
and return it to the exact spot to press several degrees, at the same skewed angle with respect to where you are sitting and facing in the driver’s
seat.

Viscosity & Tolerance work together
Tolerance
Viscosity
Tolerance
Viscosity
The viscosity is quite low for both of these designs, but the difference in tolerance makes or breaks the flow of meaning for driving a car.

Flow of meaning of driving a car
Swath of attention
Periphery
of distraction
Wayfinding
Path Events
Music/News
Passengers
The flow of meaning of driving a car involves wayfinding, continuously watching our path; watching out for events in our path that we must react
to; we may be listening to music or news, and that can waft in and out of our attention; and we may be chatting with out passengers.

Swath of attention
Periphery
of distraction
Wayfinding
Path Events
Music/News
Passengers
Comfort
The need to glance at the buttons to engage their precision takes our vision off of watching for wayfinding cues and path events and breaks the
flow of meaning of driving a car.

Swath of attention
Periphery
of distraction
Wayfinding
Path Events
Music/News
Passengers
Comfort
Voice commands
What about forgetting the screen altogether and using voice commands? That uses different type of information, entirely different phase-space
neighborhood (conceptual instead of visceral) and allows us to focus on the visual information we need for wayfinding and reacting to path
events. But, it’s a higher viscosity and requires our active attention, higher still because we have to remember the precise words and speech
syntax to verbally engage the environment controls of the system. So, we have to break our attention to listening to news/music and chatting
with our passengers. We’re still breaking the flow of meaning of driving a car. We’re breaking less critical parts of that flow, but still.

Swath of attention
Periphery
of distraction
Wayfinding
Path Events
Music/News
Passengers
Comfort
The concept control panel precision and low viscosity allows us to engage it in the periphery of our distraction, and maintain the flow of
meaning of driving a car.

Two sides to tolerance
HUMAN
BEHAVIOR
SYSTEM
BEHAVIOR
WIDE NARROW
TEMP VOL FAN
But, there are two sides to tolerance. We’ve looked at the precision required for the human’s behavior. The design also has behavior and that has
a tolerance too. We see that human engagement has a tolerance and system response has a tolerance. It’s not one or the other alone that
accounts for the the tolerance of the meaning mode, but both feeding off of each other. Even though the human can veer a bit in how she
engages the car control panel, the system doesn’t make wild changes in temp or volume. Temp still goes up and down by degree, and volume
changes by a fine increment, and the fan speeds shift specifically. So we see that, the system has a precise behavior even though the human is
allowed to have imprecise behavior to engage with it.

Two sides to tolerance
HUMAN
BEHAVIOR
SYSTEM
BEHAVIOR
WIDE-NARROW
Forgiving-Precise
TEMP VOL FAN
We’d call the overall human-system behavioral tolerance a forgiving-precise dynamic.

HUMAN BEHAVIOR—SYSTEM BEHAVIOR
WIDE-NARROW
Forgiving-Precise
Adjusting while distracted
Multi-tasking
NARROW-NARROW NARROW-WIDE
WIDE-WIDE
Comfort
The forgiving-precise dynamic is just one of the combinations.

Forgiving-Precise
Multi-tasking
NARROW-NARROW
Precise-Precise
Fine-tuning
Range of complex actions
WIDE-NARROW
CODING
NARROW-WIDE
WIDE-WIDE
An example of a precise-precise, human-system dynamic is writing computer code. If we think about the flow of meaning of writing code, it’s a
very high viscosity activity requiring lots of intense concentration to wield all of the abstractions; encode them in the precise syntax conventions
of the particular language

HUMAN BEHAVIOR—DESIGN BEHAVIOR
Forgiving-Precise
Multi-tasking
NARROW-NARROW
Precise-Precise
Fine-tuning
WIDE-NARROW
CODING
NARROW-WIDE
WIDE-WIDE
Forgiving-Fuzzy
Long-lead discovery
Play/explore multiple possible worlds
Code Review image from GitHub
It’s mostly made of concepts and very persnickety syntax. There is some perceptual information in the nested commands, and some color
coding for code reviews in some cases. The human must be extraordinarily precise with this language; the system behaves exactly as dictated
in the code.

Forgiving-Precise
Multi-tasking
NARROW-NARROW
Precise-Precise
Fine-tuning
WIDE-NARROW
NARROW-WIDE
WIDE-WIDE
3D MODELING
Mouse image credit: 3D Connecxion Model Image credit: MaTane Tutorials, Autodesk Inventor Mirroring, via YouTube
Where I work at Autodesk, we make tools that help engineers and designers create things in the world, products, machinery, architecture, by
way of 3D modeling.

Forgiving-Precise
Multi-tasking
NARROW-NARROW
Precise-Precise
Fine-tuning
WIDE-NARROW
NARROW-WIDE
WIDE-WIDE
Forgiving-Fuzzy
Long-lead discovery
3D MODELING
Mouse image credit: 3D Connecxion Model Image credit: MaTane Tutorials, Autodesk Inventor Mirroring, via YouTube
Direct 3D modeling requires high precision behavior to select particular edges and faces and points on the geometry of the thing being
made, and then make very precise adjustments to them. The system updates the visualization very precisely with each action. A lot of our
customers use an exquisitely precise 3D mouse. This requires a lot of perceptual coordination and is a very visceral way of engaging with
information.

Forgiving-Precise
Multi-tasking
NARROW-NARROW
Precise-Precise
Fine-tuning
WIDE-NARROW
NARROW-WIDE
WIDE-WIDE
Forgiving-Fuzzy
Long-lead discovery
3D MODELING
Mouse image credit: 3D Connecxion
Digital Agent
What about when we start offloading some of the modeling work to digital agents? The word agent implies agency, or the ability to act with
some degree of freedom.

Forgiving-Precise
Multi-tasking
NARROW-NARROW
Precise-Precise
Fine-tuning
WIDE-NARROW
NARROW-WIDE
WIDE-WIDE
Forgiving-Fuzzy
Long-lead discovery
Digital Agent
Constraints
Instead of the human doing all the intricate geometry sculpting, she sets some constraints on what we’re making, and then the digital
agent runs through all the different possibilities within those constraints and brings them back to the human to refine further.

Generative design image credit: Autodesk project Dreamcatcher
The digital agent can then visualize all the possibilities within the human’s set constraints, and the human can make further adjustments on
constraints. If the designer wants to explore options for chair design, the constraints may be a particular type of material, amount of material,
requirements for weight that should be supported, that it should still have 4 legs. And the the agent comes back with the range of possibilities
that the designer can filter further.

Generative design image credit: Autodesk Airbus partition project
Or in designing a partition for a jet plane that is much lighter, uses less material, but still meets very important stress criteria.

NARROW-WIDE
Precise-Fuzzy
Quick-hit discovery
Unknown possibilities
GENERATIVE DESIGN
Forgiving-Precise
Multi-tasking
NARROW-NARROW
Precise-Precise
Fine-tuning
WIDE-NARROW
NARROW-WIDE
WIDE-WIDE
3D MODELING
This type of generative design phase shifts our customers from the highly perceptual engagement with the direct geometry of the design that
requires intense coordination in a precise-precise style of engagement between human and design tool, over to a conceptual mode of
engaging in which our customers are no longer directly conduct the actual geometry of the design itself. Instead they precisely constrain the
conceptual facets of the design and offload the geometry decisions to the design tool as digital agent. The tool becomes an agent that has
agency to run through permutations and come up with possibilities for the designer to then refine further. These two different styles of modeling
have different meaning modes in terms of viscosity and tolerance.

NARROW-WIDE
Precise-Fuzzy
Quick-hit discovery
GENERATIVE DESIGN
Forgiving-Precise
Multi-tasking
NARROW-NARROW
Precise-Precise
Fine-tuning
WIDE-NARROW
NARROW-WIDE
WIDE-WIDE
Forgiving-Fuzzy
Long-lead discovery
3D MODELING
MACHINE LEARNING
If we add some machine learning to the digital agent’s agency, then we open up the wide-wide tolerance, or forgiving-fuzzy dynamic for 3D
modeling. The agent can tolerate less precision on our part because it is learning from other things like usage patterns, material properties, all
sorts of things that can expand into feedback loops leaving the human to task of making tweaks as desired based on aesthetics or higher-level
requirements.

intense
concentration
intense
coordination
conceptual
visceral
Language
NARROW-WIDE
Precise-Fuzzy
Forgiving-Precise
NARROW-NARROW
Precise-Precise
WIDE-NARROW
NARROW-WIDE
WIDE-WIDE
Forgiving-Fuzzy
VISCOSITY TOLERANCE
Coordinating vs. Adjusting
Our next property is persistence: are we relying on the information in the design to help coordinate our behavior, or are we using it to make
adjustments or decisions? (This property is based on: Sabrina Golonka (2015) Laws and Conventions in Language-Related Behaviors,
Ecological Psychology, 27:3, 236-250, DOI: 10.1080/10407413.2015.1068654 )

COORDINATING BEHAVIOR ADJUSTING
Forgiving-Precise
Multi-tasking
NARROW-NARROW
Precise-Precise
Fine-tuning
NARROW-WIDE
WIDE-NARROW
3D MODELING
Precise-Fuzzy
Quick-hit discovery
GENERATIVE DESIGN
WIDE-WIDE
Forgiving-Fuzzy
Long-lead discovery
Notice that in 3D modeling, the human is using the design tool to coordinate behavior over time as she directly sculpts the geometry. In generative
design, the human is simply making adjustments, decisions about which of the possibilities best suit the situation and constraints. This difference
between relying on information to coordinate our behavior vs. to make adjustments is our next property of meaning modes. (Notice that when
precision is about conceptual information (or the language-dominated region of the phase-space), precision is not about usability of the interface
itself; the literal act of engaging, but in the conceptual act of engaging. How conceptually precise must the human be to engage the information and
make adjustments (in the case of generative design). In the case of controlling the car dashboard panel, the questions of tolerance (precision)
involved how precise the human had to be in physical behavior to engage the information. This is an important distinction.)

COORDINATING BEHAVIOR ADJUSTING
Three Dancers, pen and ink by Chris Carter
When we think about gestural input, which one is that suited to? Which style of persistence?

ADJUSTING
Hololens gesture image credit: Microsoft
Gesture as CONCEPT
Forgiving-Precise
When I make gesture with my arm or hand or eye gaze, I’m not able to do it exactly the same angle and speed, etc., every time; there’s too
many degrees of freedom. There has to be some invariant structure the system picks up to get the gesture. That makes them a lot like
concepts. Directions; adjustments.

COORDINATING
BEHAVIOR
Sonar haptics image credit: MIT Media Lab
Gesture as SCULPTING
Precise-Precise
MIT and other groups are starting to use focused sound ways to create a mechanical, haptic effect that we can feel. We could actually touch and
feel holograms. This is just proof of concept now, but we could imagine that it could then use our behavior, our gestures, to directly change it. We
could actually use gestures to coordinate our behavior along with the haptic feedback we get from the focused sound waves changing as our
motions change the hologram. That shifts what we can do with gestures from just adjusting by evoking concepts, to relying on the system to
coordinate our behavior over time.

intense
concentration
intense
coordination
conceptual
visceral
Language
NARROW-WIDE
Precise-Fuzzy
Forgiving-Precise
NARROW-NARROW
Precise-Precise
WIDE-NARROW
NARROW-WIDE
WIDE-WIDE
Forgiving-Fuzzy
COORDINATING
BEHAVIOR ADJUSTING
VISCOSITY TOLERANCE
Sense of Place
Persistence is about whether we are relying on the information in the system to help coordinate our behavior over time, or to make
adjustments. Our next property is Presence or sense of place.

Creative Commons free usage forest panorama image
From the ecological point of view, place is a collection of affordances that support a meaningful activity (NOTE: see Heft, The Participatory
Character of Landscape https://www.researchgate.net/publication/253950787_The_participatory_character_of_landscape ). This forest can be
many different places, depending on the activity. If I’m looking to set up camp, place becomes a particular layout of the forest that offers a
clearing, a reasonably unbumpy ground, and perhaps a break in the trees to watch the stars.

Creative Commons free usage forest panorama image
When I’m ready to make a camp fire, place becomes an extent of the forest floor within a walkable radius from my camp. The affordances of
this place become objects that appear twig-ish and stick-ish, with surfaces that appear dry and burnable. A forest contains nested places made
of physical information, depending on my activity. We also have places made of digital information. For more, see Information Architecture for
the Web and Beyond, Morville, Rosenfeld, Arango, and Pervasive Information Architecture, Resmini and Rosati, and Understanding Context,
Andrew Hinton.

Precise-Fuzzy
NARROW-WIDE
ADJUSTING
VISCOSITY TOLERANCE PERSISTENCE
One place made of digital information is Amazon. The Amazon website has a viscosity that is a combination of concepts in terms of the categories
of products and their facets, and the perceptual cues of the products themselves displayed as images. If you are browsing boardgames or books,
you may lean more on the labels, titles, and recommendation text for the flow of meaning; if you are looking for a new shirt, you may engage more
heavily on the perceptual information about the surfaces, edges, textures of the shirts portrayed in the images. Amazon has a precise-fuzzy human-
system dynamic in that the human must select categories or enter well-formed search terms, but the system is fuzzy in offering “customers also
bought” and customer reviews and rec’s for other things, and search term structuring with each additional character typed. The human relys on the
information in Amazon to adjust, and make decisions about things to purchase.

MARKETPLACE
If we zoom out, this set of activities we engage in Amazon gives it a sense of place. We recognize Amazon as a marketplace.

DIGITALPHYSICAL
MARKETPLACE
Emergence
Image credit: RD Riccoboni
We know there is some emergent nature to engaging a digital place like Amazon in whatever our present physical surroundings may be, and
the designers of Amazon have to worry about its meaning shape shifting across contexts of devices, and languages, and accessibility modes,
but the physicality of the surrounding environment, the objects and layout of the surrounding physical environment, do not participate directly in
the meaning of engaging amazon as marketplace. As far as meaningful engagement with the activities in the marketplace that is amazon, our
physical surroundings may as well be a painting.

DIGITALPHYSICAL
MARKETPLACE
Except for the beige boxes that show up on doorsteps.

DIGITALPHYSICAL
There is a designed connection between the physical box that contains the thing you bought, and the digital object that is order tracking of the thing
you bought in the marketplace. Tracking your box location is part of the meaningful activities in Amazon as Marketplace, and participates in the
place that is amazon.

DIGITALPHYSICAL
Correspondence
--David Benyon, Blended Spaces
PRESENCE
These designed connections among physical and digital objects and relationships are what David Benyon, working on Blended Spaces in HCI, calls
correspondences. This extends the presence of Amazon marketplace to our physical world. Although the meaningful activities we engage in this
marketplace are mostly digital, this connection extends the meaningful activities, our sense of presence, our sense of place of Amazon, into the
physical world. These designed connections among physical and digital objects and relationships are what David Benyon calls correspondences. See
his research including, http://iwc.oxfordjournals.org/content/24/4/219.short (Also, keep an eye on the work of Andrea Resmini, bringing these concepts
to his Pervasive Information Architecture work.)

DIGITALPHYSICAL
Room image credit: Autodesk Revit rendering; VR game credit: Sony Project Morpheus
VR
What about virtual reality?

DIGITALPHYSICAL
VR
Not only does virtual reality deliberately not offer correspondences to our physical surroundings, but deliberately tries to obliterate our ability to shift
our attention from the digital place to our physical surroundings.

DIGITALPHYSICAL
Correspondence
Wall alert
VR
But, we like to move around. And because we can’t perceive our physical surroundings, we might run into walls or furniture. So, we have to let a
little physicality seep into our “virtual” reality. These “wall alerts” take the form of beeping sounds or maybe a visual cue that appears like an aurora
in this other place that we are inhabiting. This wall alert is a designed correspondence too, but rather than participating directly in the flow of
meaning in the digital place, it disrupts that presence and for a moment asks our presence to co-exist in two parallel places. It’s a very unnatural
kind of presence.

DIGITALPHYSICAL
BALANCE
LOCOMOTION
Missing Correspondences
VR

Image credit: Microsoft, Hololens
What about holograms?

DIGITALPHYSICAL
Holograms
BALANCE
LOCOMOTION
OBJECTS
No Correspondences
LAYOUT
Now we can see through to the physical surroundings. Balance and locomotion, and visual information about the layout of physical
surroundings is available, but the layout and objects in the physical surroundings don’t participate in the meaning of engaging with the
hologram. There’s no correspondence there. Although we can shift attention between the digital place that is the hologram and its meaningful
activities, and my physical surroundings, they are not both participating in the same sense of presence.

Image credit: Microsoft, Hololens game Fragments
In the game Fragments for Hololens, the system first 3D scans your entire room. The objects in the room actually participate in the game. A
detective game where you find clues. So a hologram character can sit on your actual chair, because the chair is recognized, and has
affordances for the holograms. Theoretically, the system could place digital things inside physical drawers and cabinets because they could be
recognized in terms of the dispositions of the kinds of activities they support.

DIGITALPHYSICAL
3D Room Scanning
Mixed Reality
Correspondence
OBJECTS
LAYOUT
PRESENCE spans both
In the case of 3D room scanning mixed reality (or whatever fancy name gets assigned to that), the physical objects and layout of the room
participate directly in the meaning of the place, and there is a designed correspondence between the physical place and digital place, and our
sense of presence spans both.

DIGITALPHYSICAL
Degree of
Physical-Digital
Correspondences
PRESENCE
Nouns & Verbs
Affirmations
extending
Ecosystem
-Alaine Mackenzie -Dan Brown
OBJECTS + INTENT
So, one important consideration for presence is recognizing the degree to which our physical surroundings participate in the meaningful activities of
the designed place. Are our physical surroundings (objects and layout) participating in the place we’re engaging, in the meaningful activities we’re
engaging, or are they just our surroundings? Two of the speakers yesterday, Alaine Mackenzie’s “nouns and verbs” and Dan Brown’s affirmations
extending the ecosystem of the domain, are great ways to help us define what are our meaningful correspondences among physical objects/layout
and digital objects/layout? Or, if they need not be present. Defining the ontology of our design domain is a critical step in understanding the scope of
place (and therefore of presence when our humans engage them). (For more on ontology, Dan Klyn on ontology-taxonomy-choreography, and
Abby Covert’s book, How to Make Sense of Any Mess.)

“One sees the environment not just with
the eyes but with the eyes in the head on
the shoulders of a body that gets about”
Image credit: Designboom (Alison Brooks Architects ‘The Smile’)
J.J. Gibson
We humans want to explore! We don’t just look at our surroundings, we explore them to meaningfully engage them. From an ecological point of
view, we see the environment by moving through it. J.J. Gibson quote from, An Ecological Approach to Visual Perception, p.222 1979. According to
JJ Gibson, our legs (or whatever we use to physically move around) are part of our visual systems. It’s part of how we see. See also Heft (2010)
“Affordances and the perception of landscape: an inquiry into environmental perception and aesthetics,” in Innovative Approaches to Researching
Landscape and Health. An ecological point of view makes a strong distinction between looking at an image and engaging the environment.

Are we just looking at this playground, or can we explore it?

DIGITALPHYSICAL
Degree of
Physical-Digital
Correspondences
PRESENCE
Explorability
of Objects
& Layout
ACTING WITH
LOOKING AT
We can ask in our designs, to what degree are there correspondences between the objects and events in our digital place and physical
surroundings? How explorable are the objects in our designs? If your augmented reality design is just an informational overlay on an object in a
specific geographic location, that’s just a map; a distributed map, but still just a map; it doesn’t have materiality because it isn’t explorable.

WRITING GROUP
MARKETPLACE
Explorability is not just for physical, “virtual” places. Around 2006, new kinds of digital places popped up, nested within the digital place that is
Amazon the marketplace. Some shoppers found humor in some of the products Amazon offers, like Tuscan whole milk and microwave for one.
These shoppers re-purposed the product reviews to write humorous literary works, short stories, poems, about the products. Others would riff
on these works and add their own creative writing. Shoppers became writers. The product reviews of these products became writing groups,
and nested places with distinct meaningful activities very different from Amazon as marketplace, yet within.

*'t?'t?'t?'t? Paradise Redredged
By Dr. D. v. Simmental on August 11, 2006
Timeless works often suffer at the hands of translators. One thinks of the numerous and continuing attempts to render
Dante's "Divina Commedia" (another early vernacular Italian masterpiece and contemporary to the justifiably obscure
Tuscana Latte series) and the struggle with both terza rima and meaning. No so for Tuscan Whole Milk ("La tutta
latte"). Few works are better left
translation of the redredged thir
librettist for two of the earliest p
Tuscan series and "Mechanicae
*****Milk for a Better Tomorrow
By C.A. Little on August 29, 2006
opening nights sometime in the I feel compelled to relay my story with Tuscan Whole Milk. My husband is a middle manager
store. In an attempt to move him up the ranks of paper pushers (pardon the pun), we decide
That Stilton translated the Tus and invite some of his bosses. Not being much of a homemaker, I was frantic to come upwi1
Emmenthaler, present at the Gr I discovered the response people seem to have to Tuscan Whole Milk.
Reisen", returning to his rooms I
the Arno, but had been clogged. I ordered a gallon - 128 fluid ounces of pure dairy goodness. I was quite concerned about he
massive s
complete
work.He
and he fle
� One Friday, Without the Milk, October 30, 2006
By Amazon Customer
This review Is from: Tuscan Dairy Whole Vitamin D Milk, Gallon, 128 oz (Grocery)
He always brought home milk on Friday.
After a long hard week full of days he would burst through the door, his fatigue hidden behind a
smile. There was an icy jug of Tuscan Whole Milk, 1 Gallon, 128 fl oz in his right hand. With his left
hand he would grip my waist - I was always cooking dinner - and press the cold frostiness of the jug
against my arm as he kissed my cheek. I would jump, mostly to gratify him after a time, and smile
lovingly at him. He was a good man, a wonderful husband who always brought the milk on Friday,
Tuscan Whole Milk, 1 Gallon, 128 fl oz.
ed to se,
using Tu
greet o
the kitct
edel Crys
Iwrought
Milk. lwa,
ilk! I COUit
sticated an,
*'Clf:rtrC:i With apologies to Emily
By S. Mayo on February 1, 2011
Because I could not stop for milk,
It kindly came to me.
I ordered it through Amazon,
Whole milk; sweet Tuscany.
The driver, though, he knew no haste,
No Expediter, he!
The wait: Four days, and weekend, too.
It seemed eternity.
My milk, it passed through NYC
Where traffic snarled in rings,
Slow past the fields of grazing cows
Who nodded knowingly.
Its expiration date passed, too;
And though the truck was chill,
Vile germs digested lactose sweet
And had their gruesome fill.
Rarely mi
read (if ne
diatribes
infamous
Grasso")
by some s
Dickinson'
Proust's "
Neal; "El
Peck and
Then there was that Friday, the terrible Friday that would ruin every Friday for the rest of my life. The It paused before my house at last,
door opened, but there was no bouyant greeting - no cold jug against the back of my arm. There was can Whole Milk. 1 had to place my
This Stilto
the flavor
spoke littl
is not cert
One start
• 3comm
no TuscanWhole Milk in his right hand, nor his left. There came no kiss. I watched as he sat down in (By USPS Ground)
a kitchen chair to remove his shoes. He wore no fatigue, but also no smile. I didn't spe'!fo��e!!:!:�=:::!::s
:::o:::f:::th=e=fi=m=e=d
=a
=i
�s
=a
=v
=a
=n
=ts
=a
=t
=T
=u
=s
=ca
=n=::::!:=======�===================i
back to the beans I had been stirring. I stirred until most of their little shrivelled skins
surface of the cloudy water. Something was wrong, but it was vague wrongness that no ar
hard thought could give shape to. •
Over dinner that night I casually inserted,"What happened to the milk?"
"Oh,"he smiled sheepishly, glancing aside,"! guess I forgot today."
That was when I knew. He was tired of this life with me, tired of bringing home the Tus r
Milk, 1 Gallon, 128 fl oz. He was probably shoveling funds into a secret bank account, I I
228
votes
Question:
Answer:
If I spill it, can I cry?
If you do it is best to cry either next to it or below it. Crying over it is useless.
By Marcus M. on February 25, 2013
� See more answers (7)
apartments in town, casting furtive glances at cashiers and secretaries and waitresses. l,'h,'-��---=--=��=�-----��------�--------------------'
knew it was over. Some time later he moved in with a cashier from the Food Mart down the street. Tuscan Whole Milk, tomorrow a
And me? Well, I've gone soy.
Help other customers find the most helpful reviews
Was this review helpful to you? GisJ� Yes � Report abuse
-S&J
• 7 comments 602 people found this helpful. J
The Tuscan whole milk writing group has its own activities embedded within the product reviews. The act of writing a review, the act of selecting
stars, rating a review as ‘helpful’ are all activities with very particular meaning in this nested place. It has its own meaning mode that is very
distinct from the meaning mode of Amazon as marketplace.

MARKETPLACE
TUSCAN MILK
WRITING GROUP
ADJUSTING
WIDE-WIDE
Forgiving-Fuzzy
ACTING WITHDIGITALPHYSICAL
No Correspondences Explorable
The Tuscan milk writing group has its own meaning mode. It has a high viscosity: highly conceptual, made of language, creative writing, playing with literary genres and tropes, and
riffing off of the works of others with the dual purpose of literary form and humor. We’d call this a forgiving-fuzzy human-system dynamic because a review writer participating in this
writing group can introduce a new trope, a new way of riffing on others that other writers can then riff off of in turn. The forgiving-fuzzy dynamic has vast generative power to
associate on the work of others and weave in new literary styles. Ten years later, people are still writing Tuscan milk reviews. Engaging the information in this place involves writers
relying on the information to adjust their behavior, responding creatively to what is already there, what just appeared, how other writers comment on their work. There are no
designed correspondences between the digital place and physical surroundings: the meaning, the set of activities that is this place all occur in the digital layer. This place is highly
explorable: writers in this place are acting with the information they engage.

SELECT STARS
COMMENT ON OTHERS
MARK REVIEWS ‘HELPFUL’
ANSWER QUESTIONS
WRITE REVIEWS AS SHORT STORIES, PARABLES, POEMS
Writers engage in a variety of meaningful activities that make this an explorable place.

MARKETPLACE
TUSCAN MILK
WRITING GROUP
ADJUSTING
WIDE-WIDE
Forgiving-Fuzzy
LOOKING ATDIGITALPHYSICAL
No Correspondences
Photostatic
NOT SIGNED IN
Notice that the explorability changes if the writer is not signed in. Only signed in writers can comment on the writings of others, mark reviews
’helpful,’ and write their own reviews. When a writer visits this place but is not signed in, the place loses its materiality; the writer can only look at
this place, she cannot engage the meaningful activities. It’s photostatic, like a painting. This changes the nature of how a writer finds presence in
this place.

intense
concentration
intense
coordination
conceptual
visceral
Language
NARROW-WIDE
Precise-Fuzzy
Forgiving-Precise
NARROW-NARROW
Precise-Precise
WIDE-NARROW
NARROW-WIDE
WIDE-WIDE
Forgiving-Fuzzy
COORDINATING
BEHAVIOR ADJUSTING
Physical
-Digital
Correspondences ACTING WITH
LOOKING AT
Explorability
of Objects &
Layout
VISCOSITY TOLERANCE
DIGITALPHYSICAL
These are four properties of the invisible things about meaning we can recognize in our designs. We can use viscosity, or ease of the flow of meaning, like a design dial to
speed up an interaction or deliberately slow it down and draw awareness and attention when something important needs it, or something engaging that we don’t want to miss.
Tolerance is about considering the precision of the human-system behavior dynamic, and which situations those different dynamics are good for. Persistence is about whether
we are relying on the information to help coordinate our behavior as we act over time, or if we are using it to make adjustments or decisions. Presence is about our sense of
place, and recognizing when we need designed correspondences among our physical surroundings and our digital places and what objects and layouts are participating in
meaningful ways in our place. Presence is also about how explorable the objects and layout of digital and physical surroundings are in allowing us to act with them, or just look
at them (photostatic). All of these together facilitate very different meaning modes for the humans that engage them.

Meaning Modes
These properties of meaning are really abstract things. Why should designers add this to all the other things we already have to worry about in
our designs?

J.J. Gibson’s Observer is not in 1979 any more
J.J. Gibson
1979
JJ Gibson’s observer is not in 1979 any more.

We’ve opened many more wormholes to information with our designs. [Watch icon via bluetip design; VR headset via Michael Beno; iphone
icon via Edward Boatman; shoe icon via romzicon; egg carton via Blaise Sewell; toaster icon via Creative Stall; band icon via Nate Eul]

Information wormholes are getting all over us
And they’re getting all over us; in our clothes, on our kids, and all over the objects around us.

Robots minding all the wormholes
Same degree of attention
In the same way
We don’t want to have to act like robots minding all the information wormholes with the same degree of attention, in the same way.

We are tribal hunter-gatherer poets
And we want to act like it

MARSHA HAVERTY
@mjane_h
Thank You

Meaning Modes in Design - Fluxible 2016

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (18)

En vedette

En vedette (20)

Similaire à Meaning Modes in Design - Fluxible 2016

Similaire à Meaning Modes in Design - Fluxible 2016 (20)

Dernier

Dernier (20)

Meaning Modes in Design - Fluxible 2016