Product Thesis

There are three categories of tools you can use to make product decisions. You can use data, frameworks, and intuition. Experienced product managers know when and how to use all three tools.

A pattern I have seen repeatedly is that new product managers overly depend on their intuition, rather than on data and frameworks. Such intuition is often wrong and leads to decisions that are based on the whims of an unseasoned PM, rather than on more rigorous analysis.

As PMs gain more experience, they become much more data-driven. This is generally A Good Thing. Some PMs become so data-driven that they become paralyzed in the absence of data. They are also at-risk of being misled by data. If all product decisions could be made purely with data, then we could automate the job of product managers. Data is a tool and an experienced PM knows when and how to leverage this tool in intellectually honest and rigorous ways. The PM needs to determine what metrics to track, understand Type 1 and Type 2 statistical errors, understand the limits and biases of the datasets that underlie their analyses, etc.

The next step up from data are frameworks. Frameworks are mental models that help you to understand an ambiguous problem space. Frameworks can be used to structure a data-driven analysis. For example, a framework can be constructed to determine the metrics to track, the goals to set, the variables to measure, etc. Frameworks are also useful as decision making tools in the absence of data. Product design decisions on zero-to-one products without precedent cannot be made in a data-driven way because the data does not exist. In such scenarios, frameworks are very useful to distill ambiguity into clarity.

The last step up is a full-circle back to intuition. Very experienced product managers are able to make “correct” decisions rapidly and without conscious consideration of an option space, statistical analysis, or explicit use of frameworks. Daniel Kahneman’s “Thinking Fast and Slow” makes his groundbreaking research on human cognitive decision systems accessible to the layperson and should be required reading for any product manager. The premise is that human decision making relies on two distinct systems. System 1 makes fast “automatic” decisions using subconscious pattern recognition mechanisms. System 2 is slower and uses conscious, effortful, logical, and analytical mechanisms. PMs with highly developed product intuition are in effect able to make some/most product design decisions using System 1.

Experienced PMs will instinctively know when to rely on their intuition and when to rely on frameworks and data. They are able to make high velocity correct decisions by relying on their intuition and can shift to lower velocity systems (frameworks, data) when making decisions in areas outside of their developed intuition.

I would assert that for 99% of the population, sound product intuition is earned via years of applying rigorous thinking tools (frameworks, data). Seasoned PMs learned by shipping many product successes and product failures. A very small set of exceptional product leaders possess highly developed “built-in” product intuition. The canonical examples are obvious: Steve Jobs and Elon Musk (I cannot think of a third example). However, most of us do not operate in these rarified circles, and must develop our product sense the old fashioned way.

An individual contributor (IC) product manager’s job is to produce artifacts: strategy memos, vision decks, product requirements documents, roadmaps, and so on. A mid-level IC PM should be able to hammer out a high-quality PRD on autopilot. A senior IC PM should be able to write cogent strategy memos and set long-term roadmaps with meaningful intermediate milestones without too much trouble. I’ve found my English writing classes to be much more useful in my job than my undergraduate and graduate degrees in computer science and my MBA. Business writing done succinctly, persuasively, and quickly is one of the most important skills for a rising product manager.

There are a number of resources on how to write better: Strunk and White’s “The Elements of Style” is a perennial. The Minto Pyramid Principle is one of the best books on logical thinking and writing that I have read. Various business and technology blogs, such as those of Steven Sinofsky, Ben Thompson, and others demonstrate great examples of solid business writing.

However, as one progresses in one’s career, critical reading becomes the overriding skill. A PM Lead is responsible for ensuring that their team’s artifacts are high quality, strategy memos are well-reasoned with no obvious holes in the argument, PRDs are thorough and comprehensive, etc. A PM Director will produce few artifacts themselves but will review many documents and decks every week. A PM Lead, Director, or VP who is incapable of providing useful and insightful feedback to their subordinates is not upleveling the skills of their team.

Unfortunately, good resources on critical reading are much harder to find. If you’ve studied for the SATs or GMATs, you will recall reading comprehension questions in which you had to quickly digest and analyze a passage of text. Exercises like these are a useful starting point, but not sufficient.

The challenge for leaders is that they need to be able to provide useful feedback to the team/IC even though the team/IC is the subject matter expert in the area being discussed. Even within a functional organization in which the leaders are experts in their discipline, the leader cannot know every detail of every corner of their scope. Leaders who are bad at providing feedback will often say things like “Have you thought about X?”, where “X” is some very obvious idea that the team/IC had already considered and dismissed. Product reviews that go like this can be very frustrating. It quickly becomes obvious that the leader does not know what they are talking about. Left unchecked, such leaders can lose their credibility with the team.

How do you provide good feedback in that case? Here are a few tips that have worked for me.

Tools and environment

Focus time: Carve out uninterrupted blocks time for reviewing your team’s work. Blocks of focus time are even more important for reading than they are for writing.

Pre-reads: I like organizations that require presenters to send a document or deck 24 or 48 hours before the review. This gives all attendees time to read, digest, and provide feedback on the content. Some organizations require the first 20 minutes of a 1 hour review to be spent reading the materials (Amazon reportedly does this). I prefer the former because 20 minutes is not enough time to digest a complex document and provide high quality feedback. Organizations that review the materials (whether deck or document) live are the most ineffective. Personally, I find such live reviews to be a total waste of everyone’s time.

Tools: Try out a variety of tools to see what works for you. I don’t like reviewing decks and documents electronically, because I get distracted with notifications, tab switching, etc. Printing a document and reviewing with pen and paper work better for me, but I hate wasting so much paper and waiting for the printout to complete. I’ve tried using a Surface Pro with a stylus and iPad Pro with a Pencil but neither were satisfying. Eventually, I landed on the Remarkable 2 table and I am liking it so far. It feels and looks like paper, does not permit distractions and notifications, but has the digital conveniences of an iPad. I “print” PDFs and decks to the Remarkable and then mark up the document with my feedback using the stylus.

Read twice: Go through the artifact with a pen in hand and mark up all your feedback. Then go back over it again (ideally after an hour or two break) and see if your feedback still makes sense.

OK, so now you have gotten your environment set up to provide high quality feedback. What next? Good business writing will be persuasive by definition. How do you provide valuable feedback?

The actual reading

Challenge the assumptions: Every argument rests on a set of axioms or assumptions. Good business writing will make these assumptions explicit early in the document. If they are explicit, discussing and challenging the assumptions is a useful exercise. What if X were not true? What if resources were not a constraint?

Align on goals: Make sure you and the team align on goals upfront. What is in-scope? What is out-of-scope? If you are misaligned on goals, you will argue about the goals by proxy later in the discussion.

Scrutinize the logic: A set of logical arguments will flow from the assumptions. Analyze the logic. Does it make sense? Under what conditions does it break down? What is the probability of those breaking conditions occurring?

Identify the crux: Many (but not all) arguments will rest on a pivot point or crux. If you can identify the crux, then you might be able to convert the ambiguity of the problem space into clarity or reduce complexity to simplicity. The team may be so close to the work that they are unable to see the forest from the trees and this is where your role as a leader can be valuable.

Partition the space: Sometimes the space can be partitioned into mutually exclusive, comprehensively exhaustive areas, such as X and NOT(X). Other times, the problem can be reduced down to bi- or trilemmas such as “One of A, B, or C must be true”. See if the work can be distilled down to such essences.

Connect the dots: You will have context and breadth of information that the team will not have. Help them connect the dots. Is there another team working in another area of the organization that has already solved a similar problem? Should this team join forces with that team? Is there an over-the-horizon change that you are aware off that impacts this project?

Ask about rejected options: A good piece of business writing will show the options that the team considered and rejected (with solid rationales). Analyze the rationale on the rejected options. If the writing does not show alternative options, then provide feedback to the team to always come to you with the considered option space and a recommendation, rather than just “walking you down the garden path”.

Lastly, sometimes you may really have to ask the team if they have thought about X (they might not have!). There is a more sophisticated and less patronizing way to do this. Simply ask them, “You guys have probably considered and rejected X. Can you help me understand your reasoning?”.

Hopefully, you find these tips useful as you progress in your product management career!