Systems

From Monolithic to Distributed Architectures

| Jorge Arango

Amazon CTO Werner Vogels on how the company transitioned from a monolithic application architecture to a distributed one:

We created a blueprint for change with our “Distributed Computing Manifesto.” This was an internal document describing a new architecture. With this manifesto, we began restructuring our application into smaller pieces called “services” that enabled us to scale Amazon dramatically.

But changing our application architecture was only half the story. Back in 1998, every Amazon development team worked on the same application, and every release of that application had to be coordinated across every team.

To support this new approach to architecture, we broke down our functional hierarchies and restructured our organization into small, autonomous teams, small enough that we could feed each team with only two pizzas. We focused each of these “two-pizza teams” on a specific product, service, or feature set, giving them more authority over a specific portion of the app. This turned our developers into product owners who could quickly make decisions that affected their individual products.

Breaking down our organization and application structures was a bold idea, but it worked. We were able to innovate for our customers at a much faster rate, and we’ve gone from deploying dozens of feature deployments each year to millions, as Amazon has grown. Perhaps more dramatically, our success in building highly scalable infrastructure ultimately led to the development of new core competencies and resulted in the founding of AWS in 2006.

Technological change requires new ways of working — especially when the change is happening at the structural level. Decentralizing the implementation at the technical level isn’t enough; decision-making must be decentralized as well. I read Amazon’s transition to two-pizza teams as a push towards bottom-up systemic interventions.

This strikes me as a more appropriate response to today’s complex challenges than the top-down hierarchies of the past. Alas, many designers and product managers are still operating within organizational structures that emerged during the industrial revolution, and which don’t easily accommodate bottom-up decision-making.

Modern applications at AWS – All Things Distributed

TAOI: Personalized Yelp Results

| Jorge Arango

The architecture of information:

Per TechCrunch, Yelp announced earlier this week that it will allow users to personalize search results:

Once you’ve made your selections, those preferences will start affecting the search results you see. The personalization should be obvious because the results will be identified as having “many vegetarian options” or “because you like Chinese food.” The homepage will also start highlighting locations that it thinks you would like.

Seems like an obvious feature, especially for a system like Yelp that aims to connect users with places they will like. A short video explains how it works:

A baseline 21st Century tech literacy skill: Training the algorithms that personalize your search results. (For designers: Watch for emerging user interface standards for such training mechanisms. I was intrigued by Yelp’s use of the heart icon to signify personalization.)

Yelp will let users personalize their homepage and search results

Touchscreens and the Loss of Nuanced Control

| Jorge Arango

From a report in The Verge about the aftermath of the collision of the USS John S. McCain, which killed ten sailors and injured many more:

The US Navy will replace the touchscreen throttle and helm controls currently installed in its destroyers with mechanical ones starting in 2020, says USNI News. The move comes after the National Transportation Safety Board released an accident report from a 2017 collision, which cites the design of the ship’s controls as a factor in the accident.

Not the only factor, to be sure; fatigue and lack of training also played a role. Still:

Specifically, the board points to the touchscreens on the bridge, noting that mechanical throttles are generally preferred because “they provide both immediate and tactile feedback to the operator.” The report notes that had mechanical controls been present, the helmsmen would have likely been alerted that there was an issue early on, and recommends that the Navy better adhere to better design standards.

There are systems in which humans are expected to play key control roles. A destroyer is one such system; as far as I know, fully autonomous vessels of this size don’t yet exist. User interfaces aren’t only means for users to send commands to systems; UIs also provide feedback to users. Some of this feedback is visual and auditory. But humans are creatures with more than two senses, and traditional mechanical controls can provide richer interactions than touchscreens, most of which rely primarily on sight.

Continue reading →

Artificial Intransigence

| Jorge Arango

Me: Ooh, X looks interesting. I wonder if I can find a short video about X. [Finds a video on X and watches to the end.]

Recommendation algorithm: Oh, s/he watched X! I know what s/he likes. X! Like? Nay! X is the bread on his/her table, the air s/he breathes, his/her raison d’être. S/he has a visible X tattoo on his/her body. His/her firstborn will be/is named after X. X in continuous rotation, 24 x 7! More X! More X! MORE X!

Me: Whoa, whoa! [Looks around for a way to say “no more X.” Finds a link to hide video about X. Clicks it. The video disappears from the recommendations feed.]

TIME PASSES

Me: [Idly visits video site.]

Recommendation algorithm: New X video! Oh, and here are three others you may have missed. And these two are kinda like X.

Me: Hmmm. I thought I said no more X. How does this thing work? [Clicks on hide links for three other videos about X. Reloads page.]

Recommendation algorithm: New X video! Oh, and here are three others you may have missed. And these two are kinda like X. Oh, and here are some about Y and Z, just in case.

Me: Really?! [Clicks on hide link for another X video. Reloads page.]

Recommendation algorithm: New X video! Oh, and here are three others you may have missed. And these two are kinda like X. Oh, and here are some Ys and Zs, just in case.

Me: Sigh. [Clicks on video about Z. Watches to the end.]

Recommendation algorithm: Oh, s/he watched Z! I know what s/he likes. Z! Like? Nay! Z is the bread on his/her table, the air s/he breathes, his/her raison d’être. S/he has a visible Z tattoo on his/her body. His/her firstborn will be/is named after Z. Z in continuous rotation, 24 x 7. More Z! More Z! MORE Z!

…

TAOI: Disneyland App

| Jorge Arango

The architecture of information:

Digital experiences are changing our understanding of physical environments. Google Maps gives you the ability to walk around a new city as though you’d known it for a long time. And should you develop a sudden hankering for ice cream, Yelp allows you to locate the nearest gelateria. The most noticeable change comes from layering information on the environment. For example, when trying to decide between two neighboring restaurants you’re no longer constrained to judging them solely by their appearance; you can also peruse their reviews in Yelp. Restaurant A has four-and-a-half stars, whereas restaurant B has three — A it is!

The number of stars is information about the place. You won’t find it in the physical place itself, but in its representation in an information environment which you access through your magical pocket-sized slab of glass. We’ve grown used to these augmented interactions with physical space, and mostly take them for granted. But recently I had one such interaction with an app I hadn’t used before, and which stood out to me for 1) its clarity of purpose and 2) the degree to which that purpose changed the experience of the place. I’m referring to the Disneyland app.

My family and I visited Disneyland a few weeks ago. We hadn’t been in five years, and the Disneyland app was one of the novelties since our last visit. The app presents a map of the Disney theme parks. As such it mostly replaced the parks’ old (and sometimes beautiful) paper-based maps. Thanks to the phone’s sensors, the Disneyland app makes it easy to figure out where you are, where to go next, and how to get there. But the app adds an additional key piece of information to the experience that can’t be had with paper-based maps: attraction wait times. Over every representation of an attraction in the park, you see a little callout that indicates how long you’ll have to wait in line to experience that ride or show:

Disneyland app

This piece of information is always available at all levels of zoom in the map. It’s the definitive element of the experience: in these maps, attraction wait times have the highest visual priority. As a result, wait times become the defining factor in sequencing the exploration of the park. The apps preferred answer to the question “What should we do next?” is always “Whatever is closest that has the shortest lines.”

This is an interesting choice that recalls the park’s old ticket levels. A long time ago, each Disneyland attraction required a separate ticket. Not all attractions used the same tickets; there were several levels ranging from A to E. “E-tickets,” such as the Haunted Mansion, were the most popular and desirable. These were considered the park’s premium attractions; their tickets were worth more than the others. This economic scheme influenced how visitors experienced the park. Ticket “coupon books” only included a limited number of E-tickets as compared to the lower denominations. Guests could buy more tickets inside the park, but having a limited number of the various level tickets affected choices. (I remember visiting Walt Disney World when it had a similar scheme, and hearing things like, “let’s visit this ride next, we have to use up our C-tickets.”)

The Disneyland app creates a similar economy by making attraction wait times the key informational element of the experience. When you’re trying to decide between two rides, knowing you’ll have to wait 65 minutes in line in one versus 15 minutes in another could be the key factor in your choice. (It was for my wife and me. Children get very cranky after waiting in long lines all day!) Our choosing to go on the ride with the lower wait times would contribute to slightly increasing that ride’s wait times and lowering the wait times for the more popular rides. I don’t have data, but my expectation is that this would help even out wait times throughout the park.

That is, of course, if all other things are equal — which they aren’t. The Haunted Mansion is a much more elaborate and compelling experience than Dumbo the Flying Elephant. Also, some rides have higher throughput than others. So the choice of riding one rather than the other doesn’t come down solely to which has the shortest waits.

That said, for someone like myself, who knows Disneyland very well, having this extra bit of information made the experience of visiting the park much better. In our two days at Disneyland, my family and I experienced more of the park than we’d ever been able to before. We also had more fun, since we spent a lower percentage of our time there in queues. But I wonder about the effect on folks who are less familiar with the parks. Will the emphasis on wait times drive them to prioritize less popular attractions over the park’s highlights? Adding feedback mechanisms to a system influences the way the system works. In what unexpected ways does this app change the experience of visiting Disneyland?

Data in Conceptual Models

| Jorge Arango

When designing interactive systems, it behooves you to define a conceptual model before you start thinking about the user interface. A great resource to learn to do this is Austin Henderson and Jeff Johnson’s Conceptual Models: Core to Good Design. In this book, they describe the system’s conceptual model in terms of “how the user would ideally think about the application and its use in supporting tasks.”

This requires you first understand the tasks (and “task domains”) the system is meant to accommodate. As you analyze the ways users understand tasks, you can define what (conceptual) objects the system must present to help users accomplish those tasks. These objects have certain attributes that are essential to their roles in the system, and they allow users to carry out particular operations that help them accomplish their tasks.

The book includes the following example, which defines a conceptual model for a simple office calendar application:

Objects Attributes Operations
Calendar owner, current focus examine, print, create, add event, delete event
Event name, description, date, time, duration, location, repeat, type (e.g., meeting) examine, print, edit (attributes)
To-Do item name, description, deadline, priority, status view, print, edit (attributes)
Person name, job-description, office, phone send email, view details

It’s obvious why it’s useful to define such a structure before you design a user interface: a conceptual model gives you a high-level picture of the system’s user-facing components. Rather than adding affordances for features piecemeal, here you have the opportunity to think about them as a whole. This allows you to think holistically about the system’s interaction mechanisms and language. Having such a conceptual model​ before working on the UI saves a lot of headaches and leads to a much more coherent experience.

However, there’s another good reason to start by defining a conceptual model: doing so allows you to better understand opportunities for leveraging data that may go unacknowledged when you begin by sketching out screen-level artifacts. Thinking about the conceptual objects in the system, their attributes, and the actions they accommodate helps surface opportunities inherent in the data these objects, attributes, and actions require and generate.

For example, when interacting the model above, the user will generate data points. Adding an event to a calendar is an action that will be captured by the system. We know events include a “type” attribute, so over time, this system will have data about various types of events. We can use this data to spot usage patterns and predict behavior, information we can use for the benefit of users and the business.

While conceptual models focus on user-facing concepts, defining them upfront also sheds light on these “invisible” aspects of the system. Doing so allows you to spot opportunities for improvement that may otherwise become apparent only further down the line (or not at all.)

TAOI: YouTube Subscriber Counts

| Jorge Arango

The architecture of information:

A headline on The Verge: YouTube is changing how subscriber counts are displayed, possibly shifting its culture.

One of the most famous aphorisms in management is Peter Drucker’s observation that “if you can’t measure it, you can’t improve it.” This phrase succinctly captures an important idea: when deciding the way forward, data is your friend. Rather than discussing directions in the abstract, this concept encourages us to break down problems into impartial facets we can trace over time.

However, as useful as it is, there’s a flip side to this concept: with a compelling enough measure, we can lose sight of the ultimate “it” we’re trying to improve. The point of losing weight isn’t to read a lower number on a scale; it’s to get healthier. The number is a proxy for health — and an imperfect one at that. “Health” is a complex subject with lots of nuances. Articulating it as a single number can make it easier to understand, but oversimplifies a complex whole.

We compound the problem when we base incentives on these numbers. Let’s say you’re promised a $500 bonus if you lose a certain amount of weight by a particular date. At that point “health” is twice abstracted: your goal is now neither health nor weight but the money. The numbers start to become more important than the ultimate thing we want to achieve. The map is not the territory, but we’re being incentivized to navigate the map.

We hope to get to the goal on the real ground the map represents. But sometimes we don’t. Sometimes the map is so compelling that it becomes the territory. This has happened with measures in social media such as follower counts on Twitter.

Back to The Verge article. High-level summary: After a recent kerfuffle between two “creators,” YouTube is changing how its system displays subscriber counts. Creators compete for subscribers, and their fortunes wax and wane accordingly. In this system, follower counts are a proxy for popularity. It’s an imperfect measure, but it’s clear and compelling, and so emerges as the locus of attention for an economy of influence. I didn’t realize it until reading about this issue, but there’s a secondary market on these stats: websites like Social Blade exist solely to track how these people are doing relative to each other. It’s a big deal.

But what’s the ultimate goal here? What social function is this system enabling? (What’s the equivalent of “health”?) Is it entertainment? Commerce? Both?

Proudshamed

| Jorge Arango

Paul Ford, writing for WIRED:

NERDS, WE DID it. We have graduated, along with oil, real estate, insurance, and finance, to the big T. Trillions of dollars. Trillions! Get to that number any way you like: Sum up the market cap of the major tech companies, or just take Apple’s valuation on a good day. Measure the number of dollars pumped into the economy by digital productivity, whatever that is. Imagine the possible future earnings of Amazon.

THE THINGS WE loved — the Commodore Amigas and AOL chat rooms, the Pac-Man machines and Tamagotchis, the Lisp machines and RFCs, the Ace paperback copies of Neuromancer in the pockets of our dusty jeans—these very specific things have come together into a postindustrial Voltron that keeps eating the world. We accelerated progress itself, at least the capitalist and dystopian parts. Sometimes I’m proud, although just as often I’m ashamed. I am proudshamed.

This piece captures a mood I’ve perceived among my cohort of techie designers: A radical swing from the unbridled optimism many of us felt in the 1990s — the sense that the internet was a transformational force comparable only to Gutenberg — to moroseness and guilt at the effects of these changes on society.

The transition from the Middle Ages to the modern era was anything but smooth. Gutenberg’s innovation wrought tremendous upheaval: Long-standing mental models collapsed; social and political systems were replaced. The technological changes of the last five decades — the wiring up of the planet into a real-time nervous system that democratizes access to the world’s information — are in some ways more radical than those of the 15th-16th Centuries. We’ve not just changed the ways we interact with each other and the world, we’ve changed change itself — scaling and speeding it up in ways that lead to unpredictable outcomes.

The article frames (digital) technology as an industry alongside others such as energy and finance. That’s a common underestimation spurred by the pervasive mental model of our time: that of the market economy. Yes, tech is an industry in that sense. But tech is also a meta-industry: it changes the character of the other industries thoroughly. The call to more responsible design is urgent not because tech requires it, but because we are re-building society atop tech.

Why should we expect such radical changes to be easy or comfortable? People of my vintage (I’m squarely Gen X) and younger in the developed world have thus far led lives of relative peace and stability. Cold War notwithstanding, we came of age inside a certainty bubble. When dealing with (deep) disruption, we fail to account both for the fragility of social institutions and the resilience of individuals under such conditions.

Mr. Ford concludes:

I was exceptionally lucky to be born into this moment. I got to see what happened, to live as a child of acceleration. The mysteries of software caught my eye when I was a boy, and I still see it with the same wonder, even though I’m now an adult. Proudshamed, yes, but I still love it, the mess of it, the code and toolkits, down to the pixels and the processors, and up to the buses and bridges. I love the whole made world. But I can’t deny that the miracle is over, and that there is an unbelievable amount of work left for us to do.

I, too, feel lucky. Yes, there is lots of work to do. But the miracle is far from over; it’s ongoing. Responding skillfully to the changes it bring requires being present; that we see clearly so we can use our (real!) abilities towards increasing agency and compassion.

WHY I (STILL) LOVE TECH: IN DEFENSE OF A DIFFICULT INDUSTRY

“Build More Roads” is Often the Wrong Answer

| Jorge Arango

The Power Broker, Robert Caro’s masterful biography of Robert Moses, tells the story of how modern New York City came to be. Moses and his team were responsible for some of the most important public infrastructure interventions in and around New York City during a span of four decades starting in the mid-1920s. Under Moses’s leadership, the city gained new parks, playgrounds, bridges, and especially roads — sometimes at the expense of beloved neighborhoods and the disruption of countless lives. An unelected official, Moses became incredibly powerful; you can see his influence in New York even today.

The core of Moses’s power lay in the Triborough Bridge Commission, which controlled the flow of traffic between Manhattan, The Bronx, and Queens. Tolls from vehicles driving across the Commission’s bridges generated immense revenues that paid for other projects, which in turn generated more revenues. For Moses, vehicular flow in New York City’s streets translated directly to more power. Thus, when considering means for transporting the city’s inhabitants, Moses’s team would gravitate towards more roads — even when it’d become clear that building more roads only increased congestion. The result was the defacement of major parts of the city with highways and roads that were constantly jammed with traffic.

You gotta watch incentives.

This comes to mind because of something that Jessica Ivins said on her interview for The Informed Life podcast:

So not everybody has the detail-oriented mind that I do, or not everybody works the same way I do. And because we’re small and I’ve been working here for almost 5 years, I have a sense of what works well for my co-workers. So for example, my coworker Thomas, I will assign him a to-do in Basecamp and he’ll get an email notification, but I’ll also send him a notice on Slack. And I’ll say, “Hey Thomas, I signed you this to-do on Basecamp, please review it by this date and see the to-do for details.” And that Slack message really helps him because he… I guess some people are really good at wrangling their inbox, other people really struggle with it. And I think that’s really typical.

It is typical. I’ve faced similar situations in other projects: Someone new joins the team, and you don’t yet know their communications preferences. Are they an email person? Do they prefer Slack? Or maybe iMessage? It’s not unusual to get a message in Slack followed by an email asking, “Did you see the message I put in Slack?” And of course, people’s preferences vary depending on other factors. For example, perhaps they only use Slack on their computers and find it easier to respond via SMS when they’re on the road.

Proprietary communications systems like Slack and Basecamp set out to solve the problems with older systems like email (and there are many!), but like Moses’s roads they can end up creating other problems. Now instead of one or two places to check your communications, you have five. You get reminders on one system (e.g., email) to check messages sent to you on another (e.g., Basecamp.) I’ve gotten messages on Slack to check something that was posted in Basecamp; later that evening I’ll also get an email reminder from Basecamp of all the things that were posted there during the day. The signal-to-noise ratio goes through the floor. And of course, now you have to keep track of where you saw the message or file if you ever need to get back to it. (“Did you send that document over email, or was it in Slack?”) As far as I know, it isn’t possible to search across all of these systems; it’s up to you to keep track of where things are. This is difficult, given the challenge I mentioned above. (I.e., different people have different communications preferences.)

The added complexity would be worthwhile if these systems enabled amazing new ways of working, but these proprietary communications systems bring little new to the table; most of the functionality they enable has been around for a while. Check out Jon Udell’s book Practical Internet Groupware for the state of the art twenty years ago; you’ll find lots that is familiar. It’s worth noting that that book advocated for building these communications systems atop existing open standard technologies. In this sense at least, the current state represents a regression from where things were in the late 90s.

I’m cranky about this stuff because in the past month I’ve joined three new Slacks and one Basecamp project. This is part of the job when you’re an independent consultant who works on several projects with different teams throughout the year. But I don’t enter these spaces casually. As someone who thinks about the longevity and resilience of information ecosystems, I’m concerned about my ability to find and keep track of stuff — especially in the long term. I dislike the proliferation of disconnected, single-purpose silos of information. While the discussions that happen in these places can be rich, vibrant, and productive (that is, when they’re not generating more noise in other channels), they’re locked away, inaccessible for future (even near-future) reference.

In the long term, building more roads isn’t the solution to the challenge of moving lots of people from one point to another. Past a certain volume, individual vehicles are inherently inefficient; one needs to re-think the structure of the transportation system from the ground up. This is more challenging if the people charged with the stewardship of these systems are incentivized to get more cars on the road. I sense similar problems with the communications challenges we face in organizations. When what you’re selling is access to Slack, your incentive​ is to get more people using Slack. That’s a different goal than getting people to communicate effectively. Yes, there is some overlap there — up to a point. How does the system scale?​