Overview
Architecture is the first principle in our design language—not because it is the most important, but because it is the aspect of AI that most of us tend to focus on. Architecture is a catch-all term for the infrastructure needed to make AI work: the data collection processes, the systems that train models on such data, and the interfaces that allow users to interact with those models. Contrary to some expectations, architecture is only one piece of an AI’s design. In fact, single-mindedly focusing on architecture (speeding up servers, improving data collection, and so-forth) will invariably detract from the overall user experience because valuable features get removed for efficiency. Designing an AI system requires more than technical architecture expertise, it requires many stakeholders, many voices, and most of all the voices of whom it intends to serve.
Given this framing, what then are the important considerations for architecting AI? How does one go about building a high-performance, high-accuracy, complex analytical system? The answer, is to start with understanding data.
Learning by Example
Data is to machines what experiences are to humans. We use experiences to inform our future decisions—without them, we have no basis for identifying trends and patterns. High-quality data-gathering and data-labeling allows an AI system to do the same. A well-designed architecture for AI requires a deeply embedded theory of data, in which data sources and data collection strategies actually mirror the eventual decision strategies of the AI. For example, if you want to build an app to recommend healthy restaurants, you may need to start by identifying what constitutes healthy eating to different people, so that not everyone is shoehorned into the same cultural notion of healthy. You may need to start with a data collection strategy that uses several scores and dimensions per each restaurant catalogued.
Investigate your theory of data by mapping the journey of a given piece of data all the way from collection through processing, augmenting, transforming, aggregating, and storing, all the way up its inclusion in a model (see Data & Insights). Many times, techniques to filter out irrelevant data can unintentionally skew the resulting dataset and mangle your model’s effectiveness. Other times, you may realize that you need to augment your data with an additional key feature in order to make meaningful correlations.
Context and Purpose
Badly designed AI often fails for the simple reason that the data used to train the model is from a different context than the one where the AI makes decisions. An AI trained on stock photos will likely fail when presented with user-provided photos. Stock photos are from a different context, since those photos have excellent lighting with subjects near the center of the frame and a clear distinction between foreground and background. In contrast, a user-provided photo may contain bad lighting, with subjects out of focus and extraneous objects in the frame. Consider whether your AI’s architecture does enough to select and process data before it ever touches an algorithm.
How Much Accuracy is Enough?
When asked to design an AI architecture, most engineers will attempt to build one that maximizes ‘accuracy’. Keep in mind that accuracy used in this context refers to the AI model’s ability to accurately complete a given task. However, most user needs do not fit into this narrow formulation. For example, there is no metric of ‘accuracy’ to recommend new music to someone. Is a recommendation accurate if the user plays the song? Or is an accurate recommendation one where the user shares the song with a friend? Or perhaps, a recommendation is only accurate if a user still likes the song several years later. Furthermore, people’s preferences are shaped by society and trends, such as popular artists or whether a given song was mentioned by a celebrity. Be careful in architecting your AI around a narrow view of ‘accuracy’, but instead look at how the AI can work in harmony with the overall user experience to provide value to your user. Instead of attempting to maximize an arbitrary number, consider what that number may mean to your users.
The Netflix Prize
Design Questions
Considerations
Interface Mitigates Data
Clarity in a UI can affect user experience much more than manipulating the underlying data, and has a lower risk of unintended consequences.
Expensive, Stale, or Inaccurate
Consider the trade-off between three different architectural choices: slow & stale, fast & inaccurate, or expensive & accurate.
Designing for Labelers
Designing a better experience for labeling data will affect downstream results, and therefore be mindful of UI innovations for both labelers and users.
Local vs Global
With more and more ‘edge’ computation, AI can often give different answers depending on where the processing is happening.
Branded Datasets
Branding your dataset allows users to more clearly interpret the information powering your AI.
Further Resources
- Anatomy of an AI System by Kate Crawford & Vladan Joler