The Art of Questions

The answers you have are only as good as the questions you ask.

Mar 6

We often get caught up in revolving statements when trying to problem-solve, and despite having what seem like the “right arguments”, there are no effective outcomes or breakthroughs. Why is this?

Sometimes, to take steps forward, we need to take steps backward and reexamine the questions we’ve been asking.

Just because our current answers are plausible — and our approach coherent — it doesn’t necessarily mean that we’re on the right track. We need to make sure we have identified the right problems, and that we’re looking at them through the right lens.

The questions we ask shape the lens and funnel we use to perceive and solve problems.

The Frustrated Team Leader

Think of a frustrated team leader in a strategy meeting. Their team is smart, their arguments are logical and coherent, but the problems are not getting solved.

What might be needed is a reexamination of the entire approach — is the path they’re on one that can lead to the desired result?

Your leader might ask:

What can we do to achieve more sales?
OR, they might ask
What problem does our product solve?

Both are intended to lead to an outcome: achieving more sales.

Yet the first focuses on what you’re doing now, and if it isn’t working, you’re stuck in a loop trying to fix something that is in itself part of the problem.

The second allows you to address the issue from a broader perspective. It forces you to take a step back and broaden the lens through which you look at the situation.

What are the problems that your product solves?
Does the market you are addressing actually have this problem?
Should you be looking at other markets?

The question is what sets up the dialogue and answers that follow.

Easier said than done. We all get caught up in the hustle and bustle of the day, and it’s easy to get distracted by the trees and lose sight of the forest.

And it’s exactly because of this that we need to remind ourselves whether we’re asking the right questions as we go.

Data: Trap-temptations

When pressure takes over, we can be tempted to switch to an easier question.

You might think: what question or problem statement can I come up with that validates the data I have already; what gives it a story?

How does this subtle trap creep up on us?

We haven’t led our strategy with guiding questions
The questions we are trying to answer are too difficult
There is an “easier” question that allows us to use data we already have

I’m a believer in saying “I don’t know yet” or “I’ll need more time to find out” — I consider these data points. They suggest that there is more to find out, rather than forcing me to draw an early or premature conclusion and be led astray.

One of my most important learnings has been to not rush into the false safety of “hard data”, especially if it doesn't yet explain the real-life behavior or qualities of a situation.

What you see at a given time might only resemble a snapshot or part of a much broader picture.

"What we observe is not nature itself, but nature exposed to our method of questioning." - Werner Heisenberg

Think as a human, not as a robot

This calls to why it’s so important to step back and visualize the problem from a behavioral and human perspective — you’d be surprised (or not) how easy it is to forget this when we’re staring at a spreadsheet all day.

This is something I’ve become very passionate about over the years. It was sparked by Rory Sutherland’s book, Alchemy.

There’s usually much more to a problem than simply fitting technical and “logical” pieces together. Ask yourself: how would someone in this situation actually behave?

Here’s an example from a previous experience I had working with a quick service restaurant partner: they were curious and eager to get their customers to spend more when they visited.

You might look at this technically and ask: how can I get someone to spend more at lunch?

Thinking of a customer as a human with a finite desire and ability to consume so much food in one sitting prompted me to ask a different question: how can I get them to visit more frequently?

The idea was that: while we may have been able to incentivize them to buy a side item or snack with lunch, it was unlikely that they were going to order two main courses.

And so, in trying to achieve our goal of “maximizing potential spend”, we were better off solving this by trying to get a customer to visit more often.

Here you acknowledge the real-life behavior element: the notion of “more” hits a limit at a certain point because a person will simply not behave that way [consume copious amounts of food regularly in one sitting].

And despite what my spreadsheet and extrapolation model told me, the customers were very unlikely to order two main courses at lunch.

I was better off looking for ways to get customers to buy breakfast, an afternoon coffee, or to visit more days per week. The increased spend came that way.

Startups: the impact of questions on what you measure

We are also susceptible to changing the way we measure results.

For example, a pressured startup needing to fundraise might focus on boosting a metric that helps them imply “progress” or “success”.

But those chosen metrics might also inherently have limited practical and economic value in the long-term.

The startup might also loosen the way the metric is counted.

I can arbitrarily lower the hurdle that allows me to count one unit of a certain metric — by doing this, I can create “better” results.

A story I love on this subject is one from Sarah Tavel, who was the first ever product manager at Pinterest.

On Shane Parrish’s Knowledge Project, she talked about how her method of using thought experiments, proofs, and premises came from her studying of Philosophy.

And in relation to asking questions and measuring results, she gives a really powerful example:

Scaling Pinterest: what you measure matters

Initially, Pinterest’s growth team had decided to focus on MAUs [monthly active users] as the main metric and objective — if a user came to Pinterest once a month, that would count as an MAU.

But Sarah noticed that if you chose the wrong metric [MAUs in this case], you ended up trying to optimize for the wrong thing, and you ended up deciding on the wrong projects and features as a consequence.

Focusing on MAUs optimized for a top-of-the-funnel metric, what is known as a vanity metric.

She questioned the long-term efficacy of this and asked whether Pinterest would be better off optimizing for “real engagement”.

She then changed the objective metric to “Weekly Active Pinners”. This would require a user to actually pin something at least once a week vs. just coming to the site once a month.

This was a much more meaningful interaction from the user, and it allowed Pinterest to optimize for something that would have a longer-term impact.

Sarah says that, as a result: we were able to make the users that were signing up happier, successful, and more engaged. And she credits this exercise to an intellectual rigor of really being honest with yourself and what you are measuring, is it really a long-term thing?

Ask yourself: are the metrics you are prioritizing fostering the behaviors and outcomes you need for your product to succeed in the long-term?

If not set correctly, the questions we end up answering can lead us astray from what matters. And while there might be short-term reasons at play, it’s crucial to maintain honesty about what our true markers of progress are.

Questions are prompted by the senses

Questions and decision making aren’t only about logic and thinking; they’re also about emotions, and about feeling and experience.

A fascinating story I came across in Tom Morgan’s work is of George Soros’ backpain. The story goes like this:

When George Soros’ portfolio was positioned poorly, or something was “off”, he would experience an acute backpain. The reality that something was wrong was signaled to him by his senses.

He experienced a feeling before he was able to articulate consciously what the issue was. And connecting this feeling to a question and problem overtime provided him an intuition that allowed him to improve his read on a situation.

Here’s how his son describes it:

My father will sit down and give you theories to explain why he does this or that. But I remember seeing it as a kid and thinking, at least half of this is bull.

I mean, you know the reason he changes his position on the market or whatever is because his back starts killing him. It has nothing to do with reason. He literally goes into a spasm, and it’s this early warning sign.

We see that the call came to George Soros as a sense first — a physical pain. The back pain was a niggling call that something wasn’t right.

What compels and intrigues me about the story is the notion that sometimes the “data” can’t yet be expressed or articulated through language, it is felt.

The way that our deeper thoughts and engrained experience communicate with us is via the senses.

And once we’ve probed enough to be able to take notice, we can bridge our thoughts and senses to create a better understanding of the situation — senses are data.

How I try to apply this:

Be curious. Follow the feeling. Probe it.
- What actions, contexts, and/or environments cause it to surface?
- Which ones cause it to go away?
What does the feeling remind you of?
What are the assumptions at the roof it?

Leaving you with a quote that resonated strongly while I was doing my research.

“It’s not that they can’t see the solution. They can’t see the problem.” 

- G. K. Chesterton

Decipher, by Firas Marafie