Co-Designing with Machines
The machine was, and still is, my constant partner. I need her in order to translate the creative thoughts in my head into tangible ideas I can share with the world. Transitioning to design from a modern dance career in my twenties, I never thought a machine would be my accomplice for innovation.
Machines have rapidly developed intelligence in this generation and their capabilities are changing the products we design. The process in which they are designed will also need to evolve. This article is the start of a conversation about co-designing with machines and what I’m calling Invisible Design— a process and design language for product designers working with artificial intelligence and technologies like machine learning. I believe these processes and tools are seeds for the future of product design.
Invisible Design–a process and design language for product designers working with AI and technologies like machine learning.
Math and science are invisible forces that reveal themselves in more discernable ways when we take the time to observe and analyze them. Take, for example, an English gentleman strolling through his garden in the 18th century. He observes an apple fall from a tree and wonders why it didn’t fall sideways or upwards from the ground. How is this possible? What are the forces at play? What are they made of? Does the same effect apply to something as small as an apple and as large as a wagon? Sir Isaac Newton continued to grapple with these questions for over twenty years in what would become his law of universal gravitation. He was able to describe an invisible force that has tangible effects in our everyday lives.
The influence invisible forces have on our lives can be unexpected. I was recently perusing my Facebook feed when I noticed that several of my friends liked Simply Framed, a company that allows you to create and order custom frames for posters and artwork online. I started to think about all the unframed work in my closet and tapped through to check it out. What made me want to try that recommendation? What caught my eye? What kind of information was needed in order to personalize that post? The invisible forces of science and math here are not gravity, but Facebook’s algorithms. Advertising software is just a zygote when it comes to the power of machine learning and where these products are headed in the next five to ten years. Machines will increasingly be making decisions within user experiences, and co-designing with them is an essential partnership for the future of product design.
As in any craft, there are individual components to the creation process—understanding, tools and interpretation. I began to get the idea for developing Invisible Design while going through this process at Airbnb on a couple of data intensive product launches. I want to share some of my thoughts on what I have observed in my own work.
You have to truly understand a thing to design a thing.
You have to truly understand a thing to design a thing. Imagine trying to design a plane, but not knowing anything about aerodynamics, or designing a glove without knowing what environment it will be used in or the anatomy of a hand. You have to understand what something can do in order to design a product well.
Last year, my product partnership team was having a conversation about a machine learning model for a new pricing tool we wanted to build for our hosts. We were trying to create a model that would answer the question, “What will the booked price of a listing be on any given day in the future?” Answering this question was no small feat. I was trying to keep up as my data science partner described the regression model they were building. The words he was using were alphas and betas, and while he was showing me charts that I could follow, the language was foreign to my design background. I sat down with him afterwards and asked him to sketch a diagram of the model and talk me through it. This was an eye-opening experience. When he started speaking the language I knew— sketches and diagramming— I understood the model and what it was trying to achieve immediately. This was my light bulb moment. I understood what the machine could do for the product and how to integrate the information into the experience. We both were excited by this understanding, and once our language barrier was broken and we could speak fluidly about where the product could go, we could really begin to take the product thinking to the next level.
I realized that this conversation did not have to be an isolated incident, but could have a larger impact on our teams. The discussion we had was a bite-size form of storytelling, just like what designers do when they quickly sketch out screens in a notebook. I learned from my colleague that the story of a product isn’t limited to the screens that the user can touch and see, it can also describe what’s happening behind the scenes. In the initial phases of product creation, an overarching story of how the experience will impact the end user is often created to help everyone understand what the product will look and feel like. These can take many forms, from storyboards to prototypes, strategy decks and diagrams. These presentations are created for many reasons, and one very important reason is to create a shared understanding of a product vision.
Understanding empowers teams. Building a shared knowledge allows innovation to happen as a step change instead of in micro steps. Visualizing the roles that data and the machine play in the discovery process is the first part of Invisible Design.
Understanding empowers teams and building a shared knowledge allows innovation to happen as a step change instead of in micro steps.
I’m continuing to work with my teams to build data visualizations that tell stories along with the interfaces our users interact with. These visualizations tend to vary as much as the products we’re creating, but the outcome is always that they help to motivate, inspire and educate the broader product team.
After understanding what we’re designing and how it works, we can start building the product with a variety of tools. A carpenter has a hammer. A photographer, a camera. A product designer, sketch. A software engineer, code. What’s interesting about all of the examples above is only one of them has a tool with the ability to learn, change and grow over time. Most product designers today sculpt UI with reactive tools–shapes and pixels are drawn on screen input directly from a designer. We also use these tools for designing outputs that are controlled programmatically in systems like responsive platforms and components. Our data partners in product are adept with tools that evolve over time. Physical systems, economic models and algorithms organically grow as variables shape their outcomes. Technologies based on these factors can learn and determine their own paths. In conjunction, the tools that designers, data scientists and engineers use are advantageous to each other throughout the entire product process, not just in building the final user interface. This is the next step in the evolution of product design.
Invisible Design adds in data sets and algorithmic decisions into the initial stages of design– wireframing and user flows– to bring dimensionality into a typically flat and static part of the process.
Take, for example, a holiday campaign for pricing tips, which was the first iteration of our Smart Pricing product. We knew from past holiday seasons that there is typically low traveler demand during the last couple weeks of December and a spike around New Year’s when folks travel a lot for the festivities. We wanted to let our host community know that if they lowered their prices during December, they could attract more travelers. In our wireframe process, we had a one size fits all module to communicate this message. What we learned from the data model is markets have varying down seasons and need differing messages and visualizations. For example, Sydney’s low season starts in November and Miami doesn’t experience a low season due to the consistent demand from vacation travelers. Our user flows and wireframes could show how the market trends and data would have an impact on the product.
At Airbnb, we design for a global audience with diverse needs and on multiple platforms. We’re constantly looking for opportunities within our process and product for systemic patterns that help simplify and create understanding out of complexity. The use cases from the market demand, when visualized, show a communication system and not just a few modules. Even though we quickly adjusted our visual designs before launching the new features, our retrospective uncovered missed opportunities in some markets by not getting the down season message out earlier than December as originally planned.
Everyone can be empowered to understand something and taught how to use a tool, but true craftsmanship is developed over time through experience and the development of a personal style. This quality is truly human, and no machine has yet learned to express individual creative thought and artistic expression. I’ve been privileged to work alongside profoundly talented designers who are top craftsman in the field. I’m humbled to say I work with many such designers today at Airbnb. Still, creative insight is not limited to designers. The insight that data scientists bring is an art form unto itself. Creating models and interpreting data sets into hypotheses about human behavior has shown me one important thing: that human behavior is complex, and product experiences cannot be designed in isolation by one team.
For years I worked at an agency where disciplines are segregated— designers work in one department and software engineers in another. This was also the case when I joined Airbnb. We were only 10 designers in a company of over a hundred engineers, and there simply weren’t enough of us to go around. As we grew, our VP of product formed a leadership group that consisted of a design manager (myself), product lead, data scientist lead, engineering manager and financial manager. My world perspective began to shift. I was exposed to conversations that I hadn’t been before, and we collectively made decisions with each other’s disciplines in mind. I learned how the other worlds operate and how to leverage the expertise and capabilities of my partners to build something better.
Product teams should be structured with experts from each discipline who make key decisions as a single unit.
Product teams should be structured with experts from each discipline who make key decisions as a single unit. This structure is important for the process of Invisible Design to work, and is not a typical model for all tech companies in Silicon Valley. Some companies are engineering led; At some the product manager is king; In others designers run the show. But it’s very rare the major disciplines work side by side in a true partnership, calling the shots together and respecting the decisions that fall into the court of the other’s expertise. Sometimes a bit of healthy sparring helps make a damn good product.
Let’s take an example with Smart Pricing where the team structure moved the product forward. The model predicted what the booked price for a host could be on any given day in the future. The product would allow our hosts to turn pricing tips on and their prices would be automatically adjusted for them. We thought this would be great for our hosts in terms of task management because they wouldn’t have to adjust their prices on a daily basis. In user research, however, we heard that some hosts wanted the ability to set maximum prices, no matter the demand. This was surprising since these hosts could potentially make more money, but this particular use case had a personal definition of price for their homes they wanted to set for travelers.
The model was working as expected but the product also needed to take into account the qualitative feedback from our hosts. The cross-discipline leaders on the team discussed the findings and the necessary updates to the experience. By having research, design, product management, engineering and data science in one room, the team was able to pivot the product strategy and experience quickly to balance the user and data needs into a highly successful product. More control was integrated for our hosts, allowing the ability to set a minimum and maximum price, as well as the desired frequency of hosting. You can see below the progression from our version 1 of the product, pricing tips, to where we are today with Smart Pricing.
Pricing tips were a simple interpretation of the model where adjusting price shows the likelihood of getting booked. If a host turned on pricing tips for a month, there was no flexibility in setting the ceiling for tips, except to override individual days. Our most recent version, Smart Pricing, allows for up to 4 months of tips and has more granular tools for the overall min and max prices, responding to the controls and functionality that our hosts find valuable. Smart Pricing is loved by many hosts, and is the result of ensuring cross-disciplinary team members were in place to have the conversations that were needed at crucial moments.
These thoughts are just the beginning of the conversation about Invisible Design. I will continue to explore and write about my discoveries in understanding, tools and interpretation as I dive more deeply into my own path in product design. To see how Invisible Design is developing in practice, you can watch a more in-depth use case on Smart Pricing at The Design Scientist, a talk I did at IxDA in Helsinki earlier this year.