Planning for Information Growth

I'm sure there is academic literature somewhere that has the exact formula, but my guess is that organizational information grows at approximately the square of the membership of the organization.

This isn't just because one more person adds that much more information, but the reason you are adding people is because the product is growing in complexity and needs more discussion, planning, and documentation. Methods of communication that work perfectly fine with 5 people become unwieldy with 8, and downright untenable with 12 people.

Story part 1

Pretty much every company these days manages work with some combination of group meetings and async communication, whether that's email or slack or google groups or whatever. In small groups where everyone knows pretty much what's going on, people take their own meeting notes, create their own task lists, do their work, asking questions and bring up problems in chat as needed.

As the group grows, teams will centralize the meeting agenda and notes into a Google doc or something for reference by everyone that no one reads or references. By 8 employees (if not before) the company decides it is time to get in bed with formal project management software. There are the old school enterprise products like Jira, Pivotal Tracker, and Basecamp; mid-life products like Trello, Asana, and Wrike; and literally hundreds of new competitors with slightly different takes on how to bolt project management ideas onto general purpose data stores.

The company will evaluate a few tools that seem promising based on their price, features, and ease of setup and then move forward with one of them. 8 people is also approximately the time when a team starts to formalize the pull request process, require reviews, and documentation for each merge. Now you have around 8 people and about 5 different systems for containing information.

  1. email: every company does at least some work via email
  2. chat: every company does at least some work in chat
  3. docs: cloud docs are the only thing that make sense for slides or long-form writing
  4. project management tool: we need it to define, assign, and track work items
  5. pull requests in some form of git manager: we need to be able to see what a commit does and why it does it

Even with a pretty small team so far, the need for purpose-built tools has distributed organizational information across at least 5 different tools, and it's very unlikely that any member of the org owns any particular tool. Just some rough guidelines around how to use them.

Fast forward to 12 people on a team. You've added some handy integrations to the chat app to let you create tickets in the project management tool. You've set up some alerts in a channel so that people see when a PR is submitted for review. You've taken steps to streamline interactions between the knowledge systems. But the information is still fundamentally disconnected, and it's hard to remember where to look for any given piece of information. All of these systems individually have decent search capabilities, but because they are distributed, there's no guidance about where to search.

Was that conversation in email or slack? Or was it in the comments on a Jira ticket? Oh crap, maybe it was in the comments on the PR. You're spending a lot of time trying to figure out where the information is that you need to know. You might have even hired a project manager to help keep track of the things. The timing will be different for every company, but once you get to this point, one thing is absolutely certain: someone will declare that what we need is a Wiki. We'll centralize process and information in the Wiki, and that will solve the problem.

Spoiler alert: it will not solve the problem.

Every team/company I've worked on since 2006 has, at some point, decided to create a Wiki. And every Wiki project has been a total failure.

Story Part 2

When everyone agrees that yeah a Wiki is a great idea, one person champions the project and recruits maybe 1 or 2 other people. They choose a platform. They build out the categories, subcategories, and topics. They stub some articles. They write the ones that they can. They bring this new beautiful knowledge base to the team and ask people to spend time writing articles. Some people do; some people don't. But ultimately on paper it looks like a pretty good start.

People keep asking in chat, "Where do I find X?" And at first the Wiki-Champions are excited to show them that it's right here.

"Oh, I couldn't find that."

A few months later, that's still happening, and the Champions are annoyed. So are the users because they still can't find things, and the Champions are getting testy about no one using the fruit of their labor.

By 6 months after launch, no one talks about the Wiki or uses it or thinks about it. Everyone has moved on, and some people are disgruntled about it. The company has decided that we need more meetings rather than better documentation. Because this just isn't working.

Why does this story happen over and over every time someone wants to build a Wiki for a team or company?

  1. It's because the basic model for every Wiki system is a hierarchy of information.

The basic hierarchical model imposes a huge amount of friction to add information to it, so people don't want to. Right up front when you decide to add new information, you have to decide where to put it in the hierarchy, or if it fits in at all or needs a new category or topic or subtopic. For many people, they can't write if they don't know the context of how the information fits into what's around it. So you start off with a great piece of information and are immediately stalled because you have to have a conversation either with yourself or with others about where this thing actually belongs. And where you put it alters in some ways how you approach writing it.

2. It's because the hierarchy design is a reflection of the designer's mental model of the company.

This is why no one can ever find anything. A software engineer is going to organize information in a different way than a marketing person or a sales person or a customer support person. That doesn't mean any of them are right or wrong or stupid or bad at their jobs. It means that the model itself is so subjective that it's useless.

3. It's because information is naturally a graph, not a hierarchy.

Most pieces of information—even very specific ones—naturally fall into multiple categories. This article itself could fall into multiple categories: information design, startup advice, Wiki usage, and project management. At least. If you were going to link to this in your internal company Wiki, where would you put it? It's pretty hard to choose.

4. It's because deciding to relocate an item is a lot of work.

Most Wiki systems automatically generate permalinks based on the context of an article's position in an information hierarchy. If a team decides that one article belongs somewhere else, then the permalink is completely broken, and that is a very bad practice. You don't want to break people's bookmarks in that way, so you have to either never have permalinks or set up tools to redirect to the location of new links. This is bad, and we should feel bad.

5. There's an obvious objection here, and it's that Wikipedia is one of the most successful websites in the world. So why can't we be like that in our company?

First, Wikipedia is organized by hundreds of thousands of people who spend all of their free time arguing about where and how things should be organized, and they do it for free.

Second, Wikipedia is successful because Google indexes all of it, and Google is actually good at search, whereas the base Wikimedia package absolutely sucks at it. If people had to depend only on the base install to run Wikipedia and there were no external search engines helping it out, it would be a total and complete failure.

Do you want to spend your time solving both of those problems internally? No. You do not. You want to focus on your business.

The Wiki is never the right answer.

What's a better story?

  1. Intentionally choose to centralize your organizational information.

If you do not do this, vital information will naturally distribute itself over multiple platforms and the value will attenuate over time.

2. Choose a single platform to centralize your information and build human or automated processes around ensuring that all information reaches that platform.

Cross-platform integrations can be useful for this, depending on how they are implemented. Creating a new ticket from a slack command? That's cool.

A slack command that only notifies a channel that something has moved to a different stage? Not so cool. The important part of a cross-platform integration isn't that there's a notification. We are all overwhelmed by notifications. The important thing is that the event is captured.

3. Choose your platform for centralization based on the efficiency of creating tags and how fast it is to lookup those tags via APIs.

The concept of a tag is absolutely key to building a graph of your organizational information. It reduces the friction of creating new content. It allows for faster searches for relevant content. It allows people to update content with new tags without having to step through actually moving articles and changing links.

4. In my experience, the best place to do all of this is in your project management tool.

You can link to pull requests, you can link to slack conversations, you can link to emails, you can link to other tickets in the system and you can link to any cloud docs. And also in my experience, project management tools have better APIs and tagging functions than comm systems. For example, don't ever rely on Slack APIs for anything ever. At all. Jira, on the other hand . . . it's not nearly as bad as the UI. It's faster, at least. And totally consistent.

Closing Remarks

People should be free to use the comms of their choice to get the work done that needs to get done. Some people prefer slack or email. Some people prefer discussion inside of Github PRs. And that's all fine. As long as these things are linked into an item in the PM software there's one and only one central location to search for organizational information. It becomes the "root node" of the graph search space. The process that backs that centralization is really honestly pretty lightweight. And on top of that, if you adopt such a model, you've given yourself a gift when your organization grows large enough that you want to pull data out of the API and start doing internal analytics.

Centralizing organizational information saves you money. Companies hire project managers in the early stages mostly to be central repositories of distributed information. If you design the information up front and the process around that, you can spend that money hiring more engineers or product people or sales people or whatever you need at the moment.

There's an enormous amount of value locked up in organizational information that we don't normally think about until we're missing it. Generally speaking, that value is locked up in the minds of founders and early stage employees. And we are mostly not very good at recovering it when those employees are too busy to interact directly. Early stage companies need to have these resources in place at about the time they need to disconnect from day-to-day operations by inserting a middle-management layer between themselves and the newer employees.

It's possible to design very lightweight processes around simple services that expose that information to machine-readable formats that can make searching across the brain-space of early employees not only possible but easy. This can be a huge productivity boost when companies begin to onboard new people quickly. And it can also be a great resource for older employees to understand architecture and infrastructure decisions.

You've successfully subscribed to Actually Sprained
Great! Next, complete checkout to get full access to all premium content.
Welcome back! You've successfully signed in.
Success! Your account is fully activated, you now have access to all content.
Error! Stripe checkout failed.
Success! Your billing info is updated.
Error! Billing info update failed.