Product Management in Data Science: Why DS Needs Product Managers
In a session at the Data Science Innovators Conference 2023, Medidata product manager Srishti Gupta argued that data science teams need product managers with data science experience. She covered how data science products differ, how to prioritize by complexity and value, and when to stop a project.
By The AI Enterprise Magazine Team4 min read
Key takeaways
- Data science products are different. Gupta said they are trial-and-error, depend on data quality and improve with user feedback.
- Data science experience helps product managers. She argued it closes the gap between user needs and the solution.
- Check the data early. Discovery must test whether the right data exists, not only whether users need a solution.
- Prioritize by value and complexity. High value, low complexity projects are the easy wins.
- Know when to stop. Ending a project after one or two years costs less than after five.
In a session at the Data Science Innovators Conference 2023, Srishti Gupta, Product Manager for data science products at Medidata, made the case that data science teams need more product managers with data science backgrounds. Gupta, who works on products for clinical trial management and is based in New York, said she moved from data scientist to product manager the year before the talk. The change showed her how development work is actually used in the business.
"As a data scientist you sit in silo," she said, describing the view from the development side.
What does a product manager do?
Gupta described product management as understanding the thinking of stakeholders and users and turning it into a plan and a solution that is easy to follow. She outlined three phases. In discovery, the manager studies customer needs and competitors and judges whether an opportunity is worth pursuing. In planning, the manager sets the product mission, prioritizes projects and builds a roadmap with deadlines. In development, the manager communicates requirements to developers, makes sure user testing is in place and collects user feedback so the algorithm improves over time.
How are data science products different?
Gupta listed several differences from other products. Many people think data science is magic, she said, when it spans statistics, computer science and domain knowledge. Projects are long-term investments with no guaranteed return, and they run on trial and error rather than a defined endpoint. Outcomes depend on data quality and model performance, which decide whether a model goes into production.
She added that data science products do not follow the usual product life cycle from early adopters to mass adoption to decline. Their popularity usually grows over time, she said, as models improve with more user feedback.
Why do data science teams need data science product managers?
Gupta said many of these pain points are best handled by someone who has worked as a developer. She cited her view that 80 percent of data science projects or models never reach production, in large part because people do not understand the work involved in deciding whether a solution is useful. A product manager with a data science background, she argued, can close the gap between user needs and the solution.
She also pointed to diminishing returns. Someone has to recognize when the time put into a model exceeds the return and decide whether to stop training or end the project.
How does product discovery work for data science?
Traditional discovery relies on user interviews and competitor and market research. Gupta said data science adds a second question: whether the right data exists to solve the problem. She described a top-down path of identifying customer problems, brainstorming solutions with the development team, then checking which datasets could be used and whether they are accessible and clean enough.
A bottom-up path starts with the datasets a team can reach, brainstorms what they could support, and then finds the user problems they could solve.
How should data science projects be planned and built?
For planning, Gupta described a chart that plots technology complexity against business value. High value and low complexity is an easy win. High value and high complexity matters for strategy but is difficult, so it should be broken into parts. Low value and low complexity goes to the backlog, and low value with high complexity is not worth the effort.
Development, she said, should be agile and involve data scientists, data engineers, software, testing and release engineers. Phases run from design and basic analysis to prototyping with user feedback to deployment and model monitoring.
What did Gupta say when asked about dropping a project?
Asked for an example of ending a project, Gupta described a loan default prediction model from her time as a data scientist. After a year and a half and many approaches, she said, the model's lift over the human baseline was only 5 to 10 percent, and the model was hard to interpret in a high-stakes setting. A prior analysis of complexity and value, she said, would have shown it was not worth pursuing.
On explaining sunk effort to stakeholders, she said big companies can more easily absorb a cancelled project than small ones. Still, ending a project after one or two years costs less than finding out after five. "Not all the problems are solvable," she said.
Where to hear more from data practitioners
AI Connect continues to host sessions with practitioners across data and AI. The 2027 schedule includes 24 free live webinars in four series: the AI × Enterprise Panel Series, AI Demo Days, Agentic AI Showcases and the Plug & Play Series. Details are at /webinars/, and registration is coming soon.
Frequently asked questions
How is product management different for data science?
Gupta said data science work is trial and error, depends on data quality and model performance, and must check that the right data exists, not only user needs.
Why do data science projects fail to reach production?
Gupta said about 80 percent never do, partly because stakeholders do not understand the work involved in judging usefulness.
How should data science projects be prioritized?
She described a chart of technology complexity versus business value: pursue high value, low complexity first, and avoid low value, high complexity.
What math foundation does a data scientist need?
Asked in Q&A, Gupta said statistics, so practitioners can explain why a model behaves as it does.
This article is based on remarks by Srishti Gupta, Product Manager, Data Science, Medidata at Data Science Innovators Conference 2023 on April 12, 2023. More coverage: More AI Connect Events.
Filed under: People, Culture & Leadership, AI Strategy & ROI
