Practical Time Series Machine Learning: Comcast's Subscriber Forecast

Data Science Innovators Conference 2023Practical Time Series Machine LearningWatch the full session · 21 min

In a session at the Data Science Innovators Conference 2023, Doug Wynne, Director of Strategic Analytics at Comcast, described how his econometrics team forecasts net subscriber additions with time series machine learning, and offered six lessons for making such projects succeed. Wynne leads the team that focuses on time series forecasting and analysis, and he called the subscriber forecast probably one of the most successful projects of his career.

What is time series machine learning?

Wynne separated analytics problems into two broad groups. Classification covers yes-or-no outcomes such as customer churn. Regression covers continuous outcomes such as the average bill of a prospective customer. Add the dimension of time and you get time series regression: continuous outcomes that vary over time for one or more entities.

He grouped the toolkit into two buckets. The traditional tools are mostly econometric: linear regression, ARIMA, Markov chains and plain indexing against the prior year or month. The newer methods include tree-based models such as random forest, XGBoost and LightGBM, Bayesian approaches such as Facebook Prophet, and neural networks. Wynne used a James Bond comparison, calling the first group Sean Connery and the second Daniel Craig. He added that "there's a lot to be said for just simple" methods.

How does Comcast forecast net subscriber additions?

Wynne said the question is whether Comcast is adding or losing customers on balance, and executive leadership needs a daily answer. He noted it is one of the main determinants of the stock price that people use to value the company.

The forecast covers 15 geographic regions and three order types: connects, voluntary disconnects and non-pay disconnects. It looks up to one quarter ahead, or 94 days, with each day forecast individually. Three order types times 15 regions times 94 days means 4,230 micro models run daily.

Wynne said his team compared several algorithms, including Prophet and linear methods, and XGBoost "by far and away" gave the best results. The feature set is deliberately small, because the models run daily and must fit delivery timelines and existing data assets. He said all of the models fit in about 10 minutes.

How is the forecast judged?

Wynne could not show visuals because the data is sensitive. He said the model beat both baselines by a significant margin: the prior-year index the business had used, and the business forecast built by analyst teams with Excel-style trend extrapolation. The ML forecast beat the business number "more often than we don't," he said.

The metric he considered most important was use. Executive leadership reads the report daily to see whether trends are up or down and whether they need to intervene.

What makes a time series project succeed?

Wynne shared a "six step system" with these elements:

• Apply the right solution to the right problem. When he joined, he said, there was a quiet ban on regression forecasting because forecasts are always wrong. He worked regression into his analysis anyway and kept making the case.
• Find a champion. A leader above him had part of an economics PhD and understood the tools, which helped when a team was formed.
• Look for highly manual legacy processes. High-dimensional, data-heavy problems suit machine learning.
• Beat a clear baseline. Without a basis of comparison, he said, it is hard to say how much better a new method is.
• Over-communicate. What feels like over-communicating is often "barely enough." His team sends stakeholders a biweekly email and tries to make them feel like owners of the project.
• Stay humble. A model that outperforms will eventually go wrong, and those conversations are easier with people you have not talked down to.

What did Wynne say about Markov chains and tools?

In Q&A, Wynne said Markov chains suit non-pay disconnects because the system is stable and predictable. A disconnect is scheduled about 30 days ahead, and the customer either pays and cures the order or voluntarily disconnects first. He said his team had tried Markov chains elsewhere with less success.

The micro-model segments were set with business stakeholders, he said, and the XGBoost models were mostly off the shelf with some hyperparameter tuning. The team uses Python in Databricks.

Where to hear more from data and AI practitioners

AI Connect continues to host practitioner sessions. The 2027 schedule includes 24 free live webinars in four series: the AI × Enterprise Panel Series, AI Demo Days, Agentic AI Showcases and the Plug & Play Series. Details are at /webinars/, and registration is coming soon.

Frequently asked questions

What is time series machine learning?

Wynne described it as regression with a time dimension: continuous outcomes that vary over time for one or more entities, modeled with methods such as XGBoost.

How does Comcast forecast net subscriber additions?

A set of 4,230 daily micro models, one per order type, region and forecast day, mostly using XGBoost with a limited feature set.

When are Markov chains appropriate?

Wynne said they suit stable, predictable systems such as non-pay disconnects, where a scheduled disconnect is either cured or voluntarily cancelled.

What tools does the team use?

Python in Databricks, Wynne said.

This article is based on remarks by Doug Wynne, Director of Strategic Analytics, Comcast at Data Science Innovators Conference 2023 on April 12, 2023. More coverage: More AI Connect Events.

Filed under: AI Use Cases & Applications, AI Strategy & ROI