In data we (don’t) trust

There’s a growing legion of companies and product teams that aspire to call themselves “data driven.” When they make decisions, they tell tales of how Google tested 40 shades of blue and eliminated the need for intuition and gut-based decision making.

But, as data might suggest, extreme beliefs in any approach are problematic and a belief in data driven decisions is no exception.

For the data to point the way, we need suitable problems, the right inputs and tracking based on good questions and thoughtful hypotheses, reliable data pipelines, good analytical judgment in overlooking outliers and picking a robust methodology, and versatility in the tools to analyze and interpret the outputs. Every once a while, all of these align and it all just works.

But, for the most part, we’re better off marrying a desire for data with a healthy skepticism for what it is telling us. It is that skepticism that will ensure we keep pushing for the right questions and iterate our way into insights that get us closer to the truth.

Better to be data informed than data driven.

Optimal Stopping

We were trying to sell an old car recently and had placed it in a used car parking lot. These lots charge a monthly rent. So, you are incentivized to sell your vehicle within the first month. The question for us, then, – when is the optimal time to sell? For example, do we hold out till the end of the month and wait for the best offer?

Luckily, mathematics has a solution for us. The optimal time to stop is at 37%. If you have a 100 candidates for your next role, the most optimal way to make a decision on the best candidate is to reject the first 37 and then pick the first of the next few that is better than the first 37. Essentially, the algorithm suggests we use the first 37 to calibrate.

Optimal stopping can be extended to time as well. In this case, we had 30 days to sell and 37% of 30 days is 11.1 days. By that logic, we would hold out for the first 11 days and then sell – assuming a half decent offer comes along. Our first offer came after 14 days and we sold for a price that was eventually slightly lesser than we’d planned for. But, we had no regrets because math told us that we’d made the optimal decision.

Algorithms like optimal stopping are likely the future of psychology and behavioral economics. Optimal stopping can be applied to choosing a restaurant, a spouse, and while buying a house. As we learn about our fallibility in making decisions, we can use algorithms like this one to get better at making decisions.

(H/T: Algorithms to Live By – Brian Christian and Tom Griffiths)

Lessons from buying jeans

A decade or so ago, I bought two pairs of jeans and loved them. I wore those pairs until a year ago when they couldn’t handle my bulging abs (okay, okay, I just put on some weight).

Over the past couple of years, I’ve been wearing jeans more frequently thanks to graduate school and a workplace where wearing jeans is the norm. So, I reflected on my jeans purchase record recently. Since my 100% hit rate a decade ago, I’ve purchased 6 pairs of jeans in the last 3 years and my hit rate is around 50%. I gave away 2 pairs and still use 4 pairs but, really, use only 3.

So, why the low hit rate? Each time I went to buy jeans, I went in attempting to solve a particular frustration. It was either trying to get a more comfortable pair or trying to find a certain color or, more recently, trying to find a more comfortable waist size. So, I always over indexed on that and ended up forgetting other my key priorities.

I abhor waste. My wife and I moved continents two years ago with just 6 suitcases that contained all of our belongings. And, we try hard to keep things simple and make the most of what we buy. So, thinking about this jeans hit rate does annoy me.

However, as writing here as taught me, mis-steps are simply learning opportunities. And, in that spirit, here are 3 simple steps to better buying decisions. One note before we get to the list – this is from a satisficer’s point of view. So, I view over-analyzing small decisions as waste as well. :-)

3 steps to better buying decisions –

  1. Envision success. What does success look like?
  2. Stack rank priorities. Make sure you have a list of 3-4 things that really matter to you.
  3. Keep this list of priorities with you when shopping.

The best part about this process is that it takes all of five minutes. However, the five minutes are well spent as they’ll save money and eliminate any unhappiness from unnecessary or uncomfortable purchases. And, given we spend a significant portion of what we earn buying things, it is worth the investment to do this right.

It is likely you do some version of this for your “big” purchases. However, I think it is worth doing for the small things as well – they add up.

Minimalism and efficiency are a beautiful thing. And, besides, excellence is not an act, but a habit.

Default setting

There’s plenty of great research that shows that our actions are heavily biased by the default option. Given how important defaults are to our decision making, every once a while, it is helpful to ask ourselves – what is our default setting?

For instance, we can choose to:

– Trust or to doubt.

– Take responsibility or make excuses.

– Read a non-fiction book or scroll further down our Facebook feed.

– Ask the hard question or stay silent.

– Acknowledge mistakes and learn from them or pretend they didn’t happen.

– Observe or judge.

– Save or spend.

– Respond with fantastic attitude or be defensive and prickly.

– Love or hate.

– Exercise or watch TV.

– Care or be ambivalent.

Whatever the decision, our actions are likely to follow our default setting.

It is on us to choose wisely.

Five career priorities

There are five career priorities –

1. Location
2. Industry
3. Company
4. Role
5. Team/people

Every career choice we make comes back to how we solve for these. We can make career decisions easier for ourselves by keeping four things in mind.

First, we ought to know that it gets harder to change multiple priorities. If you are trying to change just a role or team within your company, that is likely among the easier things to do. It is harder to change companies, industries, locations. And, of course, it is harder to change two or three things at a time. That doesn’t mean it can’t be done. It is just very hard. Location can be particularly hard for those who weren’t born with American or European passports. And, for most people, graduate school tends to a way to enable such change.

Second, every time you make a move, it helps to create a stack rank of these priorities. What are you trying to achieve? And, at what cost? It is rare you are going to end up with the perfect combination. You have to know what you are willing to trade off. Also, I’ve noticed that when most folks make career decisions, they focus on points 1-4. That is natural. There is just one problem – the people we surround ourselves with have a massive impact on our daily well-being. So, beware ignoring the team/people priority.

Third, the first two priorities are the hardest to solve for. So, if we can find a way to resolve this or simply eliminate them, it’ll ease any transition. For example, if you are focused on one industry, you can now focus on four priorities instead of five.

Finally, the best way to think about career moves is to layer a longer term / directional perspective. Instead of attempting to change multiple priorities today, look to work on changing one or two at a time. For example, you can make a move across industries within the same role as a starting point. Then, attempt to change role and so on.

As a bonus point, it is easy to second guess your past career decisions when you try to make changes. It is easy to look around and feel “behind.” But, it is worth reminding ourselves that we’ve gotten here by doing the best we could with what we knew.

Now that we know better, we will do better.

First thing

What is the first thing that gets thrown out of the window when things get busy or difficult?

In my life, it used to be either sleep or exercise a few years ago. I know folk who would point to a good diet or reading good books. And, then there are others who would probably point to time with family.

The first thing thrown out of the window is very instructive because it generally points to the thing we take for granted. If we take our health for granted, we’d probably throw sleep, exercise or food. If we take our growth for granted, we’d probably throw books or learning. And, if we take our relationships for granted, we’d ignore them while we are busy.

All of these are the easy choices. That’s why we let go of them so easily when push comes to shove. But, more often than not, easy is a good proxy for wrong.

Every one of these falls under the “important and not urgent” bucket. And, we’ll never get to the important investments if all we do is fight fires every day. Furthermore, the challenge with many of these investments is that, unlike the urgent stuff, it doesn’t feel like our effort is paying off for the longest time.

Until it does.

Roads not taken

Our ability to reflect and see ourselves from an outside point of view is a big part of what makes us human. A side-effect of that ability is to dwell on roads not taken. We could spend days wondering about “what if I had..”

I’ve noticed two patterns whenever I think of roads not taken. First, I only consider the best outcomes from that road I didn’t take. If I’m thinking about an opportunity, I focus on the best case situation if I’d taken the opportunity and ignore any negative scenarios. Second, I neglect all the thought I’d put into making the decision.

Hindsight is always 20:20, of course. There is definitely a lot to be learnt from observing our past behavior. But, we must only move to correct something if we see a distinct pattern. For example, if we find ourselves repeatedly over-weighting risk in our decisions, there’s good reason to be mindful of that the next time we find ourselves making an important decision.

Beyond that, fantasizing about roads not taken is a pointless exercise. We must learn to build a solid decision making process and then trust that process. To do otherwise is just to invite unhappiness. Besides, it is worth remembering that our current state is a result of the best decisions we could have taken given what we knew. So, in theory, we did our best with the cards we were dealt. That’s all we can do.

We just have to keep the faith that we’ll push ourselves to learn from what is happening to us and, over time, to know better. And, when we know better, we will do better.