🌱 5-Bit Fridays: Overchoice and how to avoid it, The Difficulty Ratio, big and bad A/B testing pitfalls, insights on product management, and more.

#48

👋 Welcome to this week’s edition of 5-Bit Fridays. Your weekly roundup of 5 snackable—and actionable—insights from the best operators, bringing you concrete advice on how to build and grow a product.

---

To get better at product, growth, strategy, and startups… join 12,183 PMs and founders.

---

Happy Friday, friends 🍻

In case you missed it this week:

• Guilty is charged. Justice is served. SBF, the weasel behind FTX, was found guilty on all seven counts. His last act before going to prison for a very long time apparently was just a scathing closing argument where he just repeatedly lied under oath.

• OpenAI, and their engineers who just btw make a median $900K/year, are also working on their own phone—the “iPhone for AI'“, as they call it. That’s big news, and definitely confirms their ambitions to create their own platform. It might seem odd to see them leaning into both consumer and enterprise, but it’s not that uncommon of a strategy at a certain scale when looking under the hood of companies like Microsoft, Google, Facebook, and Apple. I’m very curious to see this rollout.

• Leaked messages show Apple paused dedicated internal Slack channels for Muslim and Jewish employees. The pause comes after Apple deleted messages from employees related to the Israel-Hamas war.

• Google just dropped the ability to purchase a new type of domain extension. You can now grab your new .ing URL extension. Pretty cool. If you have a cool $130K laying around, you can snatch up X.ing before Papa Elon does.

  • Should you? 🤷 Who’s to say, and don’t take this as investment advice please, but with Elon’s obsessions over the letter X and his fever dream to make it a Super App…stealing one of the richest men in the worlds lunch (one who has a big ego), may not be the worst idea ever. If you do, just don’t forget me later 🤭

    

    It’s available here

  

  👀

• Microsoft 365 AI Copilot has launched for enterprise. It’s coming in at $30/m with a 300 users minimum. Forrester thinks there will be 6.9M US workers using Copilot by 2024. We’re seeing their OpenAI investment come to fruition.

• Oh, and a tad bit of housekeeping, if you missed this week’s deep dive, be sure to check out our analysis into Cash App…perhaps the most undervalued FinTech product around.

Ok cool, onto this weeks roundup.

Here’s what we’ve got this week:

• Overchoice and how to avoid it

• The Difficulty Ratio

• What your focus should be during your first 90 days as Head of Product (including generalized advice)

• Non-obvious, but important, insights on product management

• The agony and ecstasy of building with data

p.s If you’re reading this in your email, as usual, the post gets cut short. Click here to read the full thing.

---

(#1) Overchoice and how to avoid it

American journalist, Hunter Stockton Thompson, says it well:

A man who procrastinates in his choosing will inevitably have his choice made for him by circumstance.

And when it comes to the minute-by-minute plays of of our lives—whether at work or at home—there’s just so much to always choose from.

A few seconds ago, you had the choice of which newsletter to read. Thankfully, you chose this one. And when writing this weeks 5-bit, I had so many great articles to pick from…I chose a great one by Gurwinder about five heuristics to help us make decisions.

As he points out, research shows we spend about 3 hours per day making trivial and mundane decisions. Yup, all those “What should we do for dinner tonight?” and “Wait, what should we watch after dinner?” decisions add up. If you do the math, which is sobering, that amounts to a full 40 days each year of just dithering.

And yes, that’s just the petty and easy decision. It doesn’t count actual consequential ones, like your job, choice of spouse, etc.

This brings us to the world of decision theory, where physicist Edward Fredkin proposes this:

The more equally attractive two alternatives seem, the harder it can be to choose between them—no matter that, to the same degree, the choice can only matter less.

That’s Fredkin’s paradox, and it means we waste the most time on the least important decisions.

It’s best illustrated by what’s know as Buridan’s ass:

Basically, a hungry donkey is placed precisely in the middle between two identical stacks of hay. Since the paradox assumes the donkey will always go to whichever is closer, it dies of hunger since it cannot make any rational decision between which one to eat. It wavers until starvation.

Put you hand up if you’ve been an ass…scrolling Uber Eats while starving, trying to find just what you’re looking for…🙋‍♂️

Simply, the paradox drives home the important point we often forget: the choices we make don’t need to be the best—they just have to be worth more than the time spent making them.

In Gurwinder’s post, he shares what he calls 5 “philosophical razors” that can help us made decisions in a world of endless choice.

Here they are:

Uphill Decisions:

“If you can't decide between two equally difficult choices, take the path that's more difficult/painful in the short term.”

Why?

Because we like to overvalue short-term pain/reward and undervalue long-term pain/reward. So if a decision is painful in the short term, you’re likely overestimating that pain, and should choose it over the longer term pain that only seems small because it’s far away.

The Two-Minute Rule:

“If a task will take less than two minutes, it should be done at the moment it’s defined.”

Why?

Short tasks can pile up if you just keeping kicking them down the can an occupying space on your physical and mental to-do-list. Just get it done.

Solomon's Paradox:

“If you can’t decide, pretend you’re deciding for a friend.”

Why?

Being detached from something brings objectivity and clarity. It’s the same reason we’re better and giving advice than taking our own. We’re often just better at solving other peoples problems, and Solomon’s paradox is a forcing function to change our POV.

The Hesitation Heuristic:

“If you can’t decide, the answer is no.”

We live in an age of abundance, where new options are constantly becoming available. But every option has an opportunity cost, so if you keep taking opportunities you’re not eager for, you’ll miss out on ones to which you’d unequivocally answer “Yes!”.

Regret Minimization:

“The opinion you should care about most is your future self’s.”

We all have a future self—someone who will look back and have memories around the decision we made today. This relates to the first razor (Uphill Decisions), where we prioritize the gain/reward in the long-run.

Tactically, think of the 10:10:10 strategy, where you consider how a decision will affect you in 10 minutes, 10 months, and 10 years.

…

And with these 5 razors…go into the world my friend, armed with all the philosophical tools you need to simplify the decisions in your life.

For example, I present you with the choice of whether to upgrade your HTG subscription and become a founding member. 😅 It’s less than $7 a month, you get exclusive posts (like the upcoming essay on velocity killers), access to community Chat or email where you can ask me for advice anytime, and over 80 hours of my research/writing each month. If you apply even just one lesson per year successfully, I am certain this will pay for itself many times over.

So, which razor will you use you make a decision…30 seconds. 👀

---

(#2) The Difficulty Ratio

There are a lot of factors that determine which SaaS products will win.

Let’s just talk about 1 today—the ratio of how much a deal is worth vs how long it takes to close the deal. AKA: Reward / Effort.

David Sacks has a more memorable term for it—The Difficulty Ratio.

Logically it makes sense, and almost not worth spelling out: Larger deals can take longer to close; smaller deals must close quickly.

Yet, it is worth putting to paper because as we just covered in with “philosophical razors”, we’re often not rational.

Chart illustrating the concept, by David Sacks

Simply, if your product isn’t expensive and you’re closing deals with a sales motion, you must close them fast. And if you’re selling a big-tag software, it’s okay to for things to take time.

You might be wondering what a low ACV is. Luckily David has some example ratios here for us to consider as rules of thumb.

Those ACV’s are annualized.

In a great post, David and Brian Murray answer the important question: How do we get out of the losing quadrant—the low ACV and slow to close time?

Here’s what they suggest:

1. Push ACVs higher

• Reevaluate your pricing model: Ensure your pricing model aligns and scales with the value you’re providing. Consider monetizing usage instead of seats (or vice versa), and implement tiered pricing to capture more value from different customer segments.

• Improve your product: Enhance your product with additional features, integrations, or customizable solutions that address more valuable pain points or that help you engage new budget holders. Make sure that your features appeal not just to end users but to decision-makers who have the budget and authority to buy your product.

• Target larger customers or verticals: Shift your focus towards larger customers with bigger budgets or specific industries that can derive more value from your solution. If you’re a horizontal app selling small deals to departments, consider verticalizing and targeting a particular industry to drive more value for a more defined set of buyers (leading to larger deals).

• Expand within accounts: Identify opportunities to land and expand or upsell existing customers to higher value plans or cross-sell complimentary products or services.

2. Increase velocity

• Conduct an audit of your sales engine: Assess your sales team setup, pipeline availability for your reps, lead qualification processes, and the definition of when a lead becomes an opportunity. Evaluate whether reps can create the necessary urgency to drive deals across the finish line. Find the compelling event that makes your product an immediate need to some buyer.

• Track the time a prospect spends in each stage of your sales process: While most SaaS startups have accurate data around their ACVs, they often lack good data on deal closing times because they don’t properly classify when an opportunity enters the top of the sales funnel. Tracking cycle time will help you identify bottlenecks or inefficiencies that slow down deals.

• Implement lead scoring to prioritize high-quality leads: Focus your sales efforts on leads that are more likely to convert quickly to shorten the sale cycle.

• Consider how you might introduce PLG into your distribution model: Freemium offers can compress cycle time and drive more top-of-funnel activity.

Soundbite: You just can’t have a GTM where it takes you a long time to sell low-cost goods. Either you get better at sales so you can sell faster, or you hike your prices. Ideally, you do both. If you can’t do either, you need to pivot somewhere, such as distribution model/product/market/customer.

---

(#3) What your focus should be during your first 90 days as Head of Product

I’m a huge Gibson Biddle fan. Gibs was the VP of Product at Netflix and has a long track history of Head of Product roles at different startups.

He has a ton of experience, and I’m very glad he writes about it over at his “Ask Gibs” newsletter.

A few years ago he wrote about what senior product leadership should do in their first 3 months.

While his advice is focused towards heads of product (CPO/VPs/Directors), here’s what anyone can abstract for how yo approach those key first 90 days:

• Ensure you’re heading in the right direction and that the strategy, metrics, and projects against these strategies are clear.

• Spend lots of time with the organization's people, helping you learn the process, culture, and where decisions are made

• Focus on developing high-cadence output, and where you can, eliminate any weak links like lack of tools, systems, or resources.

Okay, let’s zoom into his more specific advice for product leaders:

Gibs has a great analogy, and he uses this line of thinking to shape a 3-point plan when he joins a team: “Think of the product organization as a cannon. Is it pointed in the right direction? Does it have the right people to load and fire it? Are there enough cannonballs and gunpowder for rapid-fire blasts?”

With that, here’s his playbook for the first 3 months:

1. Outline the product strategy

During my first two weeks, I create a SWAG (Stupid Wild-Ass Guess) of the product strategy. Then I share the strategy in one-on-one meetings with peers, direct reports, and the CEO. The deadline forces me to learn the product, the company, and the industry very quickly, so I spend my first two weeks asking lots of questions.

The SWAG product strategy includes:

• Product Vision

• Prioritization of Growth, Engagement, and Monetization

• Product strategies, along with metrics and tactics, and a

• High-level four-quarter rolling roadmap

This is the perfect place to start, because remember, for any product manager strategy is the source of our leverage power.

2. Build the product team

To learn how everything works, Gibs makes sure to spend a lot of time speaking to people:

• With the CEO, he develops an understanding of the big picture: vision, thesis, strategy, goals, big moves, etc.

• With peers on the product team, he figures out how the product team fits into the overall company, and looks for where there’s misalignment (i.e marketing <> product, product <> sales, etc)

• With direct reports, he ramps up on each person’s skills and motivations. He starts building trust, and learns where his key players and weaker links are.

• In all his conversations, the goal is to understand the culture.

And after all this, he’s in a position where he knows who should be doubled down on, who needs investment to get to the right level, and who on the team just isn’t cutting it. Once he knows the latter, he drives home a hard lesson for leaders: don’t waste time firing people and building the team you need:

Most heads of product procrastinate on these conversations, try to “make do” with what they’ve got, or are overwhelmed by the recruiting requirements. I have learned to address these thorny issues within the first three months to ensure I have the right team by the end of my first year. By the end of month three, I spend one to two days a week recruiting to ensure I have the right team in place by the end of my first year.

3. Enable high-cadence product development

In the early days, Gibs work to identify the gaiting resources, and who is on the hook to address them. Teams are only as strong as their weakest link. One tactic in particular he uses stands out to me—a great case of how as a leader to set other people up to lead by example:

I spend more than half of my time with the swimlane (product area/pod/squad) that I feel has the highest potential to deliver results. Here’s what I do to help them accelerate their progress.

• Work with the product leader to clarify the swimlane’s strategy along with their proxy metrics, and projects.

• Encourage the team to think of their projects as hypotheses and to quicken the pace of experimentation in order to learn faster.

• Establish a monthly strategy meeting that includes the team’s product, design, engineering, and data leaders. The intent is to encourage the team to think more strategically and to handle execution details in other forums.

• Celebrate results and learnings from this swimlane across the product organization to broadcast their skills, behaviors, and outcomes.

My intent? I establish the team as a role model that the other swimlanes can emulate. People don’t like to be told what to do and how to do it. Working shoulder to shoulder with the role model team helps them get to know me and, over time, provides an example other swimlanes can follow.

That kind of method is the mark of true, hard-won, leadership experience. Love it.

---

(#4) Non-obvious, but important, insights on product management

Dan Hill, former Head of Growth at Brex and Director of Product at Airbnb, has spent many years building products. In his essay, “Observations on Product Management”, he took the time to consolidate some of his biggest lessons. He dubs them “mostly things that are either obvious but no-one ever says them, or non-obvious but important.”

Had me hooked with that line.

So, here are Dan’s 12 observations:

1. The job of Product Management is to deliver good products to end users. The sheer number of possible definitions of good, product and user mean there’s no standard look to a Product Manager. But if you don’t deliver, the product is not good, or no-one uses it, you’ve done it wrong.

2. Your product, especially if you manage people directly, is not the product. It’s the process that builds the product. Designers output interfaces/interactions, engineers output code, product managers output process.

3. ‘Process’ includes the frameworks for reasoning about a problem, how the team interacts and communicates, expectations of the product, expectations of the team, timelines, what success and quality look like, how decisions are made.

4. Process is not inherently bad. There’s good process and bad process. Good process is any that allows the team to produce better work faster, with joy and elegance. Bad process is anything else.

5. Be wary of people who hate process. Sometimes they have been burned by bad process, often though they fear process is a) the same as bureaucracy or b) will limit their contributions.

6. If you are a new Product Manager to a team that hasn’t worked with one before, the first order of business is proving to the team that your process increases their contributions and joy, not decreases it. It almost doesn’t matter what the product is.

7. The greatest attribute of a Product Manager is a strongly developed ability for metacognition. First, to be able to introspect oneself and see how your words, actions, choices are affecting an outcome. Second, to introspect a process and understand why it is or is not working.

8. Most Product Managers (and everyone probably) spend too little time thinking about how to solve a problem. They jump straight into solving it. Problems come in different shapes, and not all need the same process. The process that shipped the last product is unlikely to be the one that you need for the next.

9. It is easier to macrobullshit than to microbullshit. It is impossible to bullshit good engineers, designers, data scientists, researchers. If the team isn’t calling you out on something fairly regularly, you’re doing it wrong.

10. Where there is skill to move mountains, there is no need for the faith that moves mountains. Faith cannot last — eventually someone has to build something.

11. Incremental development and vision are not orthogonal; they both require the other. All product must start with a vision — a point of view — but then be built critically step by step. It’s ok to learn something new as you go.

12. Being data-driven is not vision. People who cling to being data-driven rarely create anything new or interesting. I also personally find it hard to explain why to them.

Excellent observations from a long career in the trenches. And that last one on data is a nice and convenient segue to the final bit this week. 👇

---

(#5) The agony and ecstasy of building with data

I saw this headline by Julie Zhuo and it literally screamed out at me. Just recently my team and I have been going through some issues around shipping features with A/B tests and making data-driven decisions.

I could not second this point Julie makes more: Data and A/B testing are amazing tools, and that they have “the power to wrap you in the warm comfort of understanding, soothe philosophical disagreements, and make the hard decisions easier.” But…but…they are not cheat codes. And it’s easy to over use them, where we call enthusiastically and proudly for “Just test it!” or “Well, look at this metric!”. While we might think that means we’re being analytical and data-driven (yes, essential PM traits), it often is just a crutch for lack of conviction and understanding.

Can’t decide how you truly feel about the future? Just have data and experimentation take the decision out of your hands. 🤷‍♂️

As Julie says:

At their worst, Data and A/B Test lead to the kind of paralysis that comes from sprinting full-steam ahead only to find that you’ve been running on the treadmill known as Local Maximum.

So, as good and useful as they are are tools, there are a few common pitfalls to watch out for. Julie wrote about all of them in this post, but given some of the pain I’ve been working through on the testing front, here’s a PSA on A/B testing.

Pitfall #1: Spending too long perfecting tests 🚩

In general, the goal of an A/B test is to narrow down a long list of ideas into a few key levers to get signal on which directions are exciting to pursue.

Tests should be directional, and show help you validate hypotheses and learn quickly. If you’re spending forever figuring out your test and getting it ready, it’s not worth it. Don’t kill your velocity by over polishing tests.

Pitfall #2: Shipping successful tests right away 🚩

If you’re doing the above correctly and testing quickly, be mindful that you shouldn’t just ship something fully that has an immediately positive result.

A good test outcome doesn’t mean it’s ready to go.

Since you rushed getting the A/B tests out in order to save time on narrowing down the options, you now need to invest the time and energy into building out an idea the right way.

Pitfall #3: Running too many tests on details that don’t matter 🚩

Yes, it actually is. It’s bang-your-head-against-the-wall annoying.

If you’re testing stuff where literal fractional % point increases are a win, you’re probably doing soul-sucking work. Unless you’re in the FAANG world where those bumps do matter, you could easily do that for years and just be treading water around local maximums.

Instead, spend time figuring out which ideas are going to give you the step function improvements you’re hoping for. Build some level of conviction around them (assumption testing), and then go A/B test it.

You might be thinking, but what’s the harm? At minimum, we learn something. True, but is it worth it?

There are costs to too many A/B tests piling up, including but not limited to: time spent designing, building and running A/B test; time spent on data analysis and post-analysis decision-making; code complexity due to tons of branching pathways; your users all having slightly different (probably subpar) experiences; bugs due to difficulty in observing/testing multiple variations of a product; etc.

Amen, amen, amen. That’s all just spot on.

Pitfall #4: Relying on A/B tests to do anything innovative or large or multi-faceted 🚩

You can’t A/B test your way into big, bold new strategies. Something like the iPhone is impossible to A/B test. If you had asked people or invited them to come into the lab to try some stuff out, they would have preferred a physical keyboard to a virtual one. If you had them use an early prototype of the touch screen where not every gesture registered perfectly, it would have felt bad and tested poorly.

…

Soundbite: You need data and A/B testing in your arsenal, and you should be testing things regularly and bringing data into your decisions, but they are still not replacements for clear-headed and strong decision making that brings intuition and good-ol’ product sense to the table. Don’t be lazy and rely on pure numbers, and make sure you’re able to explain your ideas without needing stats as a crutch.

---

🌱 And now, byte on this 🧠

AI is weird. No one actually knows the full range of capabilities of the most advanced Large Language Models, like GPT-4. No one really knows the best ways to use them, or the conditions under which they fail. There is no instruction manual. On some tasks AI is immensely powerful, and on others it fails completely or subtly. And, unless you use AI a lot, you won’t know which is which.

The result is what we call the “Jagged Frontier” of AI. Imagine a fortress wall, with some towers and battlements jutting out into the countryside, while others fold back towards the center of the castle. That wall is the capability of AI, and the further from the center, the harder the task. Everything inside the wall can be done by the AI, everything outside is hard for the AI to do. The problem is ….

Read: Centaurs and Cyborgs on the Jagged Frontier

by One Useful Thing

---

And that’s everything for this week, folks.

If you learned anything new, the best way to support me and this newsletter is to give this post a like or share. Or, if you really want to go the extra mile, I’d be incredibly grateful if you considered upgrading. The first 500 folks who become founding subscribers get 30% off this newsletter ($7/m, or $70/year), as well as any other products I sell, for life.

Thanks so much for reading. I hope you have an awesome weekend.

Until next time.
— Jaryd✌️

---

Want to sponsor this newsletter?

Learn more here.