Hello Everyone,
Data and gut instincts! 
Detectives and data analysts have a lot in common. Both depend on facts and clues to make decisions. Both collect and look at the evidence. Both talk to people who know part of the story. And both might even follow some footprints to see where they lead. Because whether you’re a detective or a data analyst, your job is all about following steps to collect and understand facts.
Analysts use data-driven decision-making and follow a step-by-step process. You have learned (if not, read my Six step process to Data Analysis blog) that there are six steps to this process:
- Ask questions and define the problem.
- Prepare data by collecting and storing the information.
- Process data by cleaning and checking the information.
- Analyze data to find patterns, relationships, and trends.
- Share data with your audience.
- Act on the data and use the analysis results.
Analyzing facts is a key part of data-driven decision making because facts lead to patterns that help guide the decisions we make — big and small. Data-driven decision-making is rooted in using facts to guide business strategy. As an analyst, you will be tasked with creating a verified story about the data and sharing it with stakeholders. These stakeholders use your story to make choices based on facts, and make sure that the company is focused on the right goals.
Gut instinct can be a problem
There are other factors influencing the decision making process, too, though. You may have read mysteries where the detective used their gut instinct, and followed a hunch that helped them solve the case. Gut instinct is an intuitive understanding of something with little or no explanation. This isn’t always something conscious; we often pick up on signals without even realizing. You just have a “feeling” it’s right.
But for data analysts, just trusting our gut instinct can be a problem. At the heart of data-driven decision making is data, so we always want to focus on the data to ensure that we’re making informed decisions. When we make decisions based on our gut instinct without any data to back it up, it can lead to mistakes. Or worse, when we ignore the data based on our own personal experiences, we can create bias in our analysis. Businesses that rely on gut instinct to make decisions often make bad choices because they aren’t considering the story their data is actually telling.
Instead of relying on gut instinct, you can build your business knowledge and experience over time. The more you know about how a business works, the easier it will be to figure out what that business needs. And that business knowledge and experience can also help you identify errors and gaps in your data and communicate your findings. For example, a detective might be able to crack open a case because they remember an old case just like the one they’re solving today. Their past experience could help them make a connection that no one else would notice. Maybe their unique background knowledge helps them discover someone is lying, or it could help them uncover new clues. Your business knowledge and experience may help you understand problems intuitively. But, unlike gut instinct, it will give you more than just a feeling to go on.
Data + business knowledge = mystery solved
Blending facts and data with your business knowledge will be a common part of your process. The key is figuring out the exact mix of data and business knowledge for each particular project. A lot of times it will depend on the goals of your analysis. That is why analysts often ask, “How do I define success for this project?”
Successful analysis needs to be accurate, and fast enough to help decision-makers. So try asking yourself these questions about a project:
- What kind of results are needed?
- Who will be informed?
- Am I answering the question being asked?
- How quickly does a decision need to be made?
For example, if you are working on a rush project, you might need to rely on your own knowledge and experience more than usual. There just isn’t enough time to thoroughly analyze all of the available data. But if you get a project that involves plenty of time and resources, then the best strategy would be to be more data-driven. It’s up to you, the data analyst, to think about the situation and make the best possible choice. You will probably blend facts and knowledge a million different ways over the course of your data analytics career. And the more you practice, the better you will get at finding that perfect blend.
Do share your thoughts on this!
Keep Learning.