Modeling for Automated Organization

Zach Winn reporting in MIT News:

MIT alumnus-founded Netra is using artificial intelligence to improve video analysis at scale. The company’s system can identify activities, objects, emotions, locations, and more to organize and provide context to videos in new ways.

Netra’s solution analyzes video content to identify meaningful constructs in service of more accurate organization. This improves searchability and the pairing of video content with relevant ads. How does this work?

Netra can quickly analyze videos and organize the content based on what’s going on in different clips, including scenes where people are doing similar things, expressing similar emotions, using similar products, and more. Netra’s analysis generates metadata for different scenes, but [Netra CTO Shashi Kant] says Netra’s system provides much more than keyword tagging.

“What we work with are embeddings,” Kant explains, referring to how his system classifies content. “If there’s a scene of someone hitting a home run, there’s a certain signature to that, and we generate an embedding for that. An embedding is a sequence of numbers, or a ‘vector,’ that captures the essence of a piece of content. Tags are just human readable representations of that. So, we’ll train a model that detects all the home runs, but underneath the cover there’s a neural network, and it’s creating an embedding of that video, and that differentiates the scene in other ways from an out or a walk.”

This notion of ‘vectors’ is intriguing — and it sounds like an approach that might be applicable beyond videos. I imagine analyzing the evolution of such vectors over time is essential to deriving relevant contextual information from timeline-based media like video and audio. But I expect such meaningful relationships could also be derived from text.

Systems that do this type of analysis could supplement (or eventually replace) the more granular aspects of IA work. Given the pace of progress in ML modeling, “big” IA (especially high-level conceptual modeling) represents the future of the discipline.

Improving the way videos are organized | MIT News | Massachusetts Institute of Technology

TAOI is Dead; Long Live TAOI

TL;DR: I’ve pulled the plug on The Architecture of Information as a separate site. The posts I published there are now hosted on this site. I’ve set up a redirect, so links don’t break.

Longer version:

For the last few years, I’ve published posts here about information architecture under the rubric of The Architecture of Information, or TAOI. Last year, I undertook an experiment: I migrated those posts to a separate blog in a different domain.

The goal was twofold:

  1. Create a more neutral (i.e., not associated with me personally) repository of IA examples from around the web, and
  2. Create a separate site where I could experiment with IA structures.

Readers received the site with a burst of excitement, which encouraged me. But traffic soon tapered off, and now it’s a fragment of what serves. At this point, I expect it’d take years to build towards goal #1.

I planned to write one new TAOI post per month. I sustained that pace for a while, but then fell behind. Whereas I treat as a public scratchpad, TAOI felt more formal, which led me to spend more time on each post there. Eventually, writing became a chore.

Furthermore, I’ve been too busy with other projects to work towards goal #2. At one point, I considered moving the site to Jekyll. I learned the system and created a new template, but eventually decided not to devote more time on that project.

Recently, I’ve been spotting items I want to write about under the TAOI rubric but aren’t fit for the format of the other site. I missed writing here and didn’t see an upside to maintaining a separate site and social media accounts. The experiment had run its course.

So, I’ve migrated the nine posts I published at to this site, and set up a redirect, so the links don’t break. I will continue publishing in the TAOI line, but from now on, posts will appear on this site.

Thanks to everyone who checked out or shared my writing on the other site. I hope you stick around as I continue to write about IA and related subjects. (If you want to stay in the loop, please subscribe to my newsletter.)

Stripping in Information

Charlotte Shane, reporting for The New York Times on the adoption of content subscription website OnlyFans by sex workers during the pandemic:

Gia [the Smutty Mystic] describes the environment as a virtual strip club, and as is true in an actual strip club, a majority of visitors aren’t forking over much. The cost of subscribing to an account — often less than $20 — is like the handful of dollars slipped into a dancer’s garter while she’s on the main stage: appreciated, but not why she shows up to work. But some customers spend thousands, or even tens of thousands, on their favorite accounts. Personalized product sales and interactions through messages and cam shows — the equivalent of lap dances and time in the champagne room — are how the real money is made. “Eighty percent of your income comes from 20 percent of your customers,” Gia, who goes by a stage name, told me. “I’ve learned that’s a rule of business across industries.”

OnlyFans has provided a venue for many sex workers to continue making money during this time of social distancing. But the site’s information architecture doesn’t help:

Several performers I spoke with attributed their success on OnlyFans to the site’s traffic, but that’s not exactly true. OnlyFans’ search function is so unhelpful that several third-party websites exist solely to help users thoroughly explore the platform’s offerings. Explicit accounts aren’t showcased among the suggested creators on OnlyFans’ home page or tweeted.

What’s more, this appears to be by design as the company looks to avoid legal complications:

The fragility of payment processing may explain why OnlyFans is so averse to discussing the sexual dimension of its site. (Representatives for the company declined to speak on the record for this article after learning of its focus.) The company must rely on the same deflection, euphemisms and implausible plausible deniability that many sex workers use to minimize the damage of pervasive persecution.

Of course, this won’t go over well with the people who depend on the system’s findability:

Sex workers deeply resent OnlyFans’ absence of a sitewide search function and menu of categories and tags to browse, not only because it makes their jobs harder but also because it seems like proof that the site is eager to jettison them entirely — as so many have done before. But Ashley, the organizer, surmised that this choice is a canny tactic for minimizing legal liability, thereby keeping the site up and running. In other words, adult creators are right that the site tries to hide them.

Information architectures aren’t designed in a vacuum; they’re always constrained by the realities of the context in which they exist. OnlyFans sounds like an example of a marketplace whose architecture is driven more by its regulatory environment than the needs or wants of its consumers or producers.

OnlyFans Isn’t Just Porn 😉 – The New York Times

Changes to Sketch’s Conceptual Model

A few days ago, I received an email from Sketch informing me (a customer) of upcoming changes to the app’s licensing model. They come as part of broader changes, which you can read about here.

I was thrilled when I discovered Sketch. For a long time, one of my primary tools was Macromedia Fireworks, a pioneering drawing and prototyping application that blended bitmaps and vectors and featured ‘symbols,’ reusable components that could take variables.

At the time, nothing matched Fireworks for UI work. The component-based paradigm allowed you to design a button and drop it into several screens using different labels. If you tweaked the original button, all instances would change. Amazing.

Continue reading

TAOI: Spotify’s ‘Your Library’ Redesign

The architecture of information:

[Spotify] recently released a major update to a core feature of its mobile apps, ‘Your Library.’ The new version promises better findability and discoverability — i.e., it focuses on the system’s information architecture.

‘Your Library’ allows user to access the content they’ve saved on Spotify. This includes playlists, songs, streaming radio stations, artists and albums, etc. As such, it’s a central ‘place’ within the app. Over time, users can build large libraries.

Streaming media apps offer access to at least two sets of content: the ‘global’ collection, which includes as much stuff as possible, and the user’s ‘personal’ collection, which is (at a minimum) a one-off subset of the former.

The two sets have different objectives. The global collection aims towards discovery and findability, whereas the personal collection aims towards curation and familiarity.

Both parts of the environment must be distinct from each other to accommodate particular affordances and make clear to the user what s/he is looking at, yet maintain enough similarities to seem part of the same system.

This is a tricky line to walk. It’s interesting to see how major players like Spotify are doing it.

Spotify’s ‘Your Library’ Redesign – The Architecture of Information

How I Take Notes

Earlier this year, I asked what you’d like to know about how I get things done. I received many interesting requests, more than fit in a single post. So, I’m covering aspects of my setup in separate entries. In this post, I’ll explain how I take notes.

First, a caveat: my personal information ecosystem is always evolving. If you’re reading this over a year since I published it, and you don’t see any timestamped updates, this information is likely outdated. That said, I’ll share where things stand now.

Continue reading

The Informed Life with Jeff Sussna

Episode 61 of The Informed Life podcast features another conversation with consultant and author Jeff Sussna. I say ‘another conversation’ because Jeff was previously on the show talking about cybernetics. This time, he shares with us Customer Value Charting, a tool to balance strategy and agility.

The ultimate purpose for creating such balance is to drive customer benefits — which are related to, but not the same as business benefits:

the benefit of the dry cleaner is that I can get my tuxedo cleaned in time to go to the formal event. It’s not fundamentally about a cash register or a counter or even cleaning chemicals. And I mention that because a lot of the conversation I see around outcomes over outputs tends to actually talk about business outcomes. You know, revenue growth and customer retention, and time on site and business outcomes are great. I don’t have any problem with them, but people tend to skip this step. We have a hypothesis that this feature will cause this change in customer behavior, which will lead to this business outcome or business impact. But it leaves open the question of, well, why is the customer changing their behavior? What is the benefit to them?

Jeff sees the process of creating customer benefits through the lens of promises: “an intention that may or may not actually come to pass.” This forces you

to think about the possibility of failure, which on the one hand helps you do a better job of not failing, but it also gives you an opportunity to think about improvement and repair.

Customer Value Charts allow teams to map such promises visually, so they can discuss them. As such, they’re a prime example of organizing information to get things done. Listen to our conversation for details on how to structure CVCs and how they can help you deliver greater value to customers.

The Informed Life episode 61: Jeff Sussna on Customer Value Charting

Book Notes: ‘All Things Shining’

All Things Shining: Reading the Western Classics to Find Meaning in a Secular Age
By Hubert L. Dreyfus and Sean Dorrance Kelly
Free Press, 2011

We’re beset with wicked problems. Ecological degradation. Political extremism. Social injustice. Wealth inequality. On top of all those, a pandemic. It’s easy to despair given so many complex challenges. Our response depends on how we frame our understanding of reality.

Dreyfus and Dorrance Kelly are philosophers, and All Things Shining is a book of philosophy in the practical sense: not a dry, academic tome about esoteric distinctions but a guide on how to lead a better life. At its core is one of the key questions of modern living: how do we keep nihilism at bay?

Continue reading

The Key to Understanding Why Things Happen

When a systems thinker encounters a problem, the first thing he or she does is look for data, time graphs, the history of the system. That’s because long term behavior provides clues to the underlying system structure. And structure is the key to understanding not just what is happening, but why.

— Donella H. Meadows, Thinking in Systems

Every year, I introduce systems students to the iceberg model. The model is a helpful way of understanding situations by looking ‘beneath the surface’ of the things we experience, to the structures and mental models they manifest.

In case you’re unfamiliar with the iceberg model, it’s a framework that encourages you to think about situations at four levels:

  1. Events, or the tangible manifestations of the situation; the things we can see, hear, and record — “just the facts.”
  2. Patterns we perceive in events; outcomes that happen not just once but manifest time and again.
  3. Structures that may be causing the patterns we perceive; these could include rules, regulations, incentives, etc.
  4. Mental models that bring these structures into being.

Notice the fourth level is more abstract than the first: we can ascertain events, but we must hypothesize mental models. There’s also a causal relationship between levels: mental models elicit structures that elicit patterns of events.

As a result, events are easier to grok than mental models. But as with pace layers, the deep levels are where the true power lies. A change at the level you can see has less impact than tweaking the mental models that bring it forth. The ability to change minds is an incredibly powerful lever.

The iceberg model is helpful when doing research. Research produces lots of data points: Google Analytics and search logs tell you about usage, landscape analyses tell you about competitors and analogs, user interviews tell you about intent, etc.

But research doesn’t stop with data. Insights only emerge once you spot patterns in data. If lots of people enter the same term into the search box and do not get good results, that tells you something important about your system.

But you can go deeper still. Patterns only tell you what is happening, not why. You should at least have a hypothesis about why things are happening. This calls for understanding the underlying structures and the mental models that enable them.

Collaborating on these levels can be uncomfortable since the work is speculative. Acknowledge the awkwardness upfront. Allow the team to speculate. You’re not making anything normative yet, just understanding why things might be happening.

Knowing causes helps produce better outcomes. You might not know causes precisely, but you can test hypotheses. Ultimately, a better understanding of the system’s structures and underlying mental models will lead to more skillful interventions.

Cover image: NOAA’s National Ocean Survey (CC BY 2.0)

Subscribe to my newsletter

If you find this post useful, you may also like my newsletter. Every other Sunday, I share ideas and resources about strategic design, systems thinking, and information architecture. Join us!

Success! You're on the list.