We’re excited to share that our technology partner, Mixpanel, has released their report on the State of Product Analytics, detailing how global product teams drive growth with data. To create this report, the product analytics leader, in collaboration with Product School, surveyed almost 500 product professionals from companies across industries to discover what kind of data informs product decisions, what are the most important metrics to product teams, and many more valuable insights.
As a digital product agency, WillowTree is particularly enthusiastic about the findings of this report because our role is not just to build digital products with our clients, but also to help them answer the strategic questions of what to build and why to build it. Sophisticated product analytics, quantitative and qualitative data, and user-centered research help our teams answer these questions and satisfy the needs of both the users and the business.
In reviewing the report, our WillowTree analytics architects and researchers found key insights that will help us continue to support our clients in making data-driven product decisions.
Notable Insights on Data-Driven Product Teams
1. “Product managers are everywhere.”
Mixpanel interviewed product managers, UX designers, and other product leaders from around the world. They found that the highest concentration of product managers are in San Francisco, New York, and London — but that sophisticated product innovation is happening all over the world, even beyond these traditional tech hubs.
WillowTree has intentionally chosen non-traditional tech hubs like Charlottesville, VA, Durham, NC, and Columbus, OH as home to our main offices, with the goal of increasing the longevity of our team members’ tenures in these emerging cities with a low cost of living and high quality of life. The reality is, however, that 85% of our team members are not from these small cities originally — rather, we attract and relocate top digital talent from major cities and tech hubs like San Francisco, Chicago, D.C., New York, and more.
Many of our growth marketing experts work out of our smaller New York City office to maintain proximity to technology partners like Mixpanel and to various clients, plus enabling us to continue to recruit top talent from this smaller community.
While it certainly doesn’t surprise us that major tech cities are populated with product talent, we are proud to play a role in continuing to disperse that talent throughout the world into emerging, highly livable cities.
2. “Sophisticated product teams rely most heavily on user data — mostly on quantifiable data from analytics solutions rather than qualitative data.”
Mixpanel found that product teams are leveraging many different data sources, both quantitative and qualitative, to inform product decisions. The most sophisticated teams rely more heavily on quantifiable product analytics.
WillowTree’s PhD-led Research + Insights team uses a unique blend and qualitative and quantitative data to inform product decisions, with frameworks that enable us to operationalize and scale qualitative data.
According to Principal Product Researcher, Claire Maiers, one of the first questions to ask in any research engagement is whether or not we already know the key factors or phenomena that will influence our findings. Qualitative approaches are usually best when you don’t already know what matters within a given space, context, or market. Although quantitative methods can discover new relationships between phenomena, we need qualitative methods in order to discover new things.
For this reason, we almost always include qualitative approaches in our innovation work - such as envisioning a new type of digital product or reimagining our client’s strategy for customer acquisition and growth. It is through qualitative research that we are best able to provide a vision for consumer experience, enhance real-world experiences with digital ones, and generate a rich understanding of consumers and their needs and values.
On the flip side, quantitative methods are often an efficient choice when you’re tasked with identifying a relationship between known concepts, such how age influences technology adoption or product preferences.
3. The top question that product teams are trying to answer is “Which of my feature launches made an impact?"
The survey results indicate that there is no one question that leads to building a great product, but rather a range of questions and different angles that must inform decisions. The top question that product teams are trying to answer, however, is particularly interesting in helping us understand what is most important to our clients and their users.
The #1 thing that product teams want to know is, “Which of my feature launches made an impact?” — essentially asking what matters most to users. The WillowTree strategy team often considers this question in terms of what “job” is most important to users, using the Jobs to Be Done Framework and journey mapping to identify what features are the most impactful. By quantifying how important a particular “job” or feature is to users and how satisfied they are with the current state of that feature, we can determine the most significant areas of opportunity for a digital product to make an impact on users. We often start with this strategy when exploring a new product, new features, or product updates to ensure that we prioritize the most pressing user needs.
4. The top three questions that are important to product teams, in order, are: “Which of my feature launches made an impact?”, “How are users navigating the product?” and “What keeps users engaged and coming back for more?”
Beyond the #1 question that product teams are trying to answer, the second and third most important questions to product teams highlight the importance of utilizing analytics and insights at every stage of the product lifecycle.
The first question, “Which of my feature launches made an impact?” asks what is most important to the user, which can be answered with product strategy capabilities that help remove the guesswork of feature prioritization and new product updates and launches.
The second question, “How are users navigating the product?” seeks to understand how the product actually works, pointing to the importance of expert UX design and development to create a frictionless user experience and identify pain points quickly.
The third-most important question to product teams, “What keeps users engaged and coming back for more?” demonstrates that building a successful product doesn’t end with launch — teams need to be able to lean on growth marketing expertise to understand how to continuously engage and retain users, promote loyalty, and reach customers where they are.
The order and importance of these questions aligns directly with each step of the product lifecycle:
- Learning and understanding through Strategy + Innovation
- Building and creating through Design + Development
- Growing and engaging through Growth Marketing + Insights
Companies often need to engage different strategic partners to cover each of these steps, which can create a disjointed experience and is part of the reason that 95% of new product introductions fail. That’s why WillowTree covers every stage of the product lifecycle, with proven expertise in helping to answer all three of the most important questions to product leaders.
5. “Having data is not enough — you need access to the right kind of data.”
Mixpanel’s research shows that while there is an abundance of data available to product teams, only 10% of those surveyed reported being able to validate their key product decisions with data. Product teams reported that not having access to the data that they need, not trusting the data, and not having the technical skills to answer questions with data, were the top three barriers that prevented them from using data to inform product decisions.
Having data, and even accessing the right kind of data, is not enough to effectively inform product decisions — you need the tools and the expertise to effectively use the data.
Long-standing, legacy organizations have continued to be disrupted by digital natives that have extensive data analytics capabilities. Legacy companies actually have the advantage of many years more of customer data than new competitors, but often don’t have the tools and digital integrations to effectively use the data.
Product analytics platforms like Mixpanel and digital product agencies like WillowTree help to unlock this data for our clients, building secure and effective solutions that make data actionable.