Concerns the digital infrastructure, operational modalities, and information flows that collectively underlie AI. Helping users navigate this complexity requires thoughtful interaction design that maps the journeys taken by both users and data streams through a product.


Architecture is the first principle in our design language—not because it is the most important, but because it is the aspect of AI that most of us tend to focus on. Architecture is a catch-all term for the infrastructure needed to make AI work: the data collection processes, the systems that train models on such data, and the interfaces that allow users to interact with those models. Contrary to some expectations, architecture is only one piece of an AI’s design. In fact, single-mindedly focusing on architecture (speeding up servers, improving data collection, and so-forth) will invariably detract from the overall user experience because valuable features get removed for efficiency. Designing an AI system requires more than technical architecture expertise, it requires many stakeholders, many voices, and most of all the voices of whom it intends to serve.

Given this framing, what then are the important considerations for architecting AI? How does one go about building a high-performance, high-accuracy, complex analytical system? The answer, is to start with understanding data.

Learning by Example

Data is to machines what experiences are to humans. We use experiences to inform our future decisions—without them, we have no basis for identifying trends and patterns. High-quality data-gathering and data-labeling allows an AI system to do the same. A well-designed architecture for AI requires a deeply embedded theory of data, in which data sources and data collection strategies actually mirror the eventual decision strategies of the AI. For example, if you want to build an app to recommend healthy restaurants, you may need to start by identifying what constitutes healthy eating to different people, so that not everyone is shoehorned into the same cultural notion of healthy. You may need to start with a data collection strategy that uses several scores and dimensions per each restaurant catalogued.

Lingua Franca: Artificial Intelligence (AI) fitness based filtering

Investigate your theory of data by mapping the journey of a given piece of data all the way from collection through processing, augmenting, transforming, aggregating, and storing, all the way up its inclusion in a model (see Data & Insights). Many times, techniques to filter out irrelevant data can unintentionally skew the resulting dataset and mangle your model’s effectiveness. Other times, you may realize that you need to augment your data with an additional key feature in order to make meaningful correlations.

Context and Purpose

Badly designed AI often fails for the simple reason that the data used to train the model is from a different context than the one where the AI makes decisions. An AI trained on stock photos will likely fail when presented with user-provided photos. Stock photos are from a different context, since those photos have excellent lighting with subjects near the center of the frame and a clear distinction between foreground and background. In contrast, a user-provided photo may contain bad lighting, with subjects out of focus and extraneous objects in the frame. Consider whether your AI’s architecture does enough to select and process data before it ever touches an algorithm.

How Much Accuracy is Enough?

When asked to design an AI architecture, most engineers will attempt to build one that maximizes ‘accuracy’. Keep in mind that accuracy used in this context refers to the AI model’s ability to accurately complete a given task. However, most user needs do not fit into this narrow formulation. For example, there is no metric of ‘accuracy’ to recommend new music to someone. Is a recommendation accurate if the user plays the song? Or is an accurate recommendation one where the user shares the song with a friend? Or perhaps, a recommendation is only accurate if a user still likes the song several years later. Furthermore, people’s preferences are shaped by society and trends, such as popular artists or whether a given song was mentioned by a celebrity. Be careful in architecting your AI around a narrow view of ‘accuracy’, but instead look at how the AI can work in harmony with the overall user experience to provide value to your user. Instead of attempting to maximize an arbitrary number, consider what that number may mean to your users.

The Netflix Prize

In 2007, Netflix offered a large cash prize to whoever could design a better movie recommendation algorithm. However, despite awarding a winner in 2009, Netflix never used the resulting algorithm. This is partly because the algorithm was, in a sense, too accurate. It predicted movies that a user had likely already seen either in theaters or on another platform. Accurately predicting a movie rating turned out to have very little to do with creating a great experience.

Design Questions

Map a user’s journey through your system, including times where the user disagrees or points where the user needs to escalate the system’s decision to a higher authority.
Does your system handle these exceptions well?
Is there more information that your system needs as input or output to make this overall journey better?
Document your data collection process in as much detail as possible, including storing, labeling, cleaning, formatting, and validating.
How can you ensure that this data is high-enough quality for your needs?
Can you identify any contextual assumptions made by this process?
Re-consider your original assumptions about accuracy in your system.
Roughly, what is the lowest accuracy level that is good enough per use case?
Are there ways of making the system more useful without making it necessarily more accurate?


Interface Mitigates Data

Clarity in a UI can affect user experience much more than manipulating the underlying data, and has a lower risk of unintended consequences.

Consider the value of how your information is currently presented, rather than assuming that problems with your product stem from ‘bad data’. For example, if the result of your AI is a ranked list of recommendations, but you display the recommendations as bullet points, people may end up clicking on the second or third result “just because”, since the first result is seen as too obvious. Bringing attention to the rank may lead users to trust the first result more.

Expensive, Stale, or Inaccurate

Consider the trade-off between three different architectural choices: slow & stale, fast & inaccurate, or expensive & accurate.

The first option, while affordable, will tend to have more ‘stale’ elements, which are stuck loading or displaying old data. The second option will sacrifice accuracy for speed. The third option is best for when cost is not an issue. All three options are beneficial in certain circumstances.

Designing for Labelers

Designing a better experience for labeling data will affect downstream results, and therefore be mindful of UI innovations for both labelers and users.

Data labelers have the often difficult job of giving data accurate labels when there may be fundamental ambiguity in the data itself. Certain issues may cause your data to become highly mislabeled, such as a case where labelers don’t know whether “bat” should refer to the animal or the sports equipment.

Local vs Global

With more and more ‘edge’ computation, AI can often give different answers depending on where the processing is happening.

Sometimes the system can only use local (on device) information, sometimes the system is using an account, and sometimes the system has global access to data. Displaying this context can enhance the experience greatly by setting expectations for the user.

Branded Datasets

Branding your dataset allows users to more clearly interpret the information powering your AI.

Branding your dataset allows you to bring attention to the quality of your data. It gives users a clear identity that can be attached to your AI products, especially where there is no ‘embodied’ agent to represent your service. Furthermore, it allows you to transfer a positive brand image from one product that users the data to another.

Further Resources