Generative AI and Feature Engineering: Lessons for Predictive Models
In a 2024 AI Connect webinar made possible by DataRobot, data science practitioners discussed feature engineering for predictive models. They said the biggest obstacles are organizational, and that generative AI helps most with scarce or unstructured data when paired with expertise and risk controls.
By The AI Enterprise Magazine Team4 min read
Key takeaways
- Define the target first. Panelists said business purpose and subject matter expertise should guide which features to build.
- It is a people problem. Shaikh said shared data literacy is the biggest challenge and adds time to every model.
- Standardize and own features. Different definitions of churn or default, and lost owners after reorganizations, undermine reuse.
- Synthetic data has promise. Sengar said generative AI can fill data gaps but enterprise use needs accuracy checks and model risk management.
- Teams need data, ML and risk skills. Panelists said a central feature repository takes data engineering, machine learning engineering and risk management, plus collaboration across technical and business staff.
In a 2024 AI Connect webinar made possible by DataRobot, four practitioners discussed how feature engineering works, why it so often stalls, and where generative AI might help. Dr. Frances Boykin, Director of Advanced Analytics at AT&T, moderated. The panelists were Dushyant Singh Sengar, a data science advisory director at BDO; Jessica Larson, a software engineer and author formerly at Pinterest and Instacart; and Jhonathan David Shaikh, formerly executive manager of data models at Grant Thornton. Boykin noted that their comments did not reflect the views of their employers.
The panel's main point was that the hardest feature engineering problems were organizational, and that generative AI needed subject matter expertise and risk management around it.
What is feature engineering and why does it matter?
Larson described feature engineering as narrowing data to the smaller set of features that matter. "It's really expensive to look at everything all the time," she said, and teams cannot act on everything at once. A defined set also makes it easier to apply policy to models.
Shaikh compared it to taking a shower before getting dressed up: getting the data set ready for the business goal. He said it has a technical side and a business side, since people must understand why it is being done. Sengar said the work used to be called model data preparation and now needs more agility because models change quickly.
Boykin said engineered features can be reused across an organization, so teams know they are working from solid ones.
What does feature engineering look like in practice?
Shaikh described a bank's probability-of-default models. A team could add a feature such as tax payments, but only for real estate lending portfolios, and the model would carry limits on where it applies. Shaikh also said teams should define the target and business purpose before engineering anything.
What are the biggest feature engineering challenges?
Shaikh said "the biggest challenge is not a technical challenge, it's actually a people challenge." Teams need shared data literacy so that everyone understands why a feature is added, he said, and getting there can add a month or two to a model that takes a month or two to build.
Boykin pointed to standardization: one team may calculate churn one way and another team differently. Shaikh said teams can also define the same target variable differently.
Sengar named data availability, completeness and accuracy. In capital markets, he said, data is often scarce or labeled for limited use, and alternative data sources need legal and compliance approval before they enter a model.
Larson raised operations and governance: who maintains features, and who decided how they are standardized. She said "organizational thrash can completely destroy any organizational work that you have done." If the owner of a feature leaves in a reorganization, she said, nobody maintains it.
Where can generative AI help with feature engineering?
Sengar said generative AI can create synthetic data when data is scarce, including images, text such as consumer complaints, and tabular records. For enterprise use, he said, accuracy and completeness conditions apply, and "there has to be risk management around the model."
Larson said the first question is where a human stays in the loop, especially when AI generates code that builds models. She said unstructured data such as wikis, Slack messages and emails could surface features that matter to a company.
Shaikh called generative AI "the great librarian" and said the user still needs subject matter expertise to check results. Boykin said many teams start with earlier traditional modeling work and apply generative AI as a second version of it.
How should teams integrate generative AI into feature engineering?
Boykin recommended a shared feature repository, as AT&T built, but said her team keeps generative AI separate from it for now.
Shaikh said banking is split on the topic. Risk managers are "trained to say no" to features suggested by generative AI, he said, so he mainly uses it to generate code and test databases. Sengar added a qualifier: language models are not ready for enterprise use and suit research, whereas generative adversarial networks can produce structured data whose statistics can be checked against the real data set.
Larson said that with regulations such as GDPR and California's CCPA, "there's basically not a such thing as not a regulated industry anymore." Paying for a private model keeps data from flowing back, she said, but security teams should be involved from the start. She also said teams should define up front when to scrap an experiment whose output will not cover its cost.
Which skills do teams need?
Larson urged learners to study theory through full online courses so they know what they don't know. Shaikh said that from his consulting work, success needs both technical and managerial knowledge plus a collaborative tone, since technical staff focus on models and business staff on clients.
Sengar said a central repository demands data engineering to keep it current, machine learning engineering to monitor models that generate data, and risk management, so "it's not one person's job."
In the closing round, one panelist said "research" is the key word for generative AI today. Sengar chose model operations. Larson said "this is not magic," so teams must understand what is happening. Boykin urged teams to leverage what they already know, because the new tools work alongside older methods.
Hear more from data and AI leaders in 2027
AI Connect has 24 free live webinars on its 2027 schedule, including AI Demo Days and the AI × Enterprise Panel Series. See /webinars/. Practitioners also gather at COLLIDE 2027, the largest AI conference in the Southern U.S., on Wednesday, October 13, 2027, at the Sandy Springs Performing Arts Center in Atlanta (/events/collide/).
Frequently asked questions
What is feature engineering?
Panelists described it as narrowing and preparing data into the set of features that matter for a business goal, which is also known as model data preparation.
What is the biggest challenge in feature engineering?
Jhonathan David Shaikh said it is a people and data literacy challenge, since everyone involved must understand why each feature is used.
How can generative AI help with feature engineering?
Dushyant Singh Sengar said it can generate synthetic images, text and tabular data when data is scarce, while Jessica Larson saw potential in mining unstructured company data.
Does generative AI replace subject matter expertise?
No. Panelists said users still need domain expertise to define the target and to check what the AI produces, and need risk management around generated data.
This article is based on remarks by Dr. Frances Boykin, Director of Advanced Analytics, AT&T (moderator); Dushyant Singh Sengar, Director, Data Science Advisory, BDO; Jessica Larson, software engineer and author, formerly Pinterest and Instacart; Jhonathan David Shaikh, formerly Executive Manager, Data Models, Grant Thornton at AI Connect webinar, 2024 on March 21, 2024. More coverage: Webinars.
Filed under: AI Use Cases & Applications, Data & AI Infrastructure, People, Culture & Leadership
