Symbolic AI and Neural Networks: How They Can Work Together
At COLLIDE 2023 in Atlanta, Georgia State University's Peter Molnar compared symbolic AI with neural networks and large language models. He said the approaches have complementary strengths, and that LLMs could help automate knowledge graphs and rule-based expert systems.
By The AI Enterprise Magazine Team4 min read
Key takeaways
- Two traditions, different math. Symbolic AI rests on logic and relations, neural AI on linear algebra and gradient descent.
- Symbolic AI is explainable. Rule-based inference is deterministic, traceable and can encode domain expertise.
- Neural AI scales. It handles large data and avoids manual feature extraction, which symbolic systems struggled with.
- LLMs can feed rule systems. Molnar suggested models could build knowledge graphs and rules and convert questions into formal queries.
- Control the inputs. Combined systems still need safeguards on what outside information they pull in.
At COLLIDE 2023 in Atlanta, Peter Molnar, an associate professor at the Institute for Insight in the Robinson College of Business at Georgia State University, argued that today's neural network and large language model boom could benefit from the older tradition of symbolic AI. In a 28-minute session, he compared the strengths and weaknesses of both approaches and pointed to places where they could work together.
Molnar, a former computer science professor who spent about ten years in industry before returning to academia, said everything in AI had been neural-network-based "for the last couple of years." His aim was to look back at roughly 50 years of AI advances and ask how earlier concepts could be brought back in.
What is the difference between symbolic and neural AI?
Molnar said the two approaches rest on different mathematics. Symbolic AI draws on propositional and first-order logic, while neural networks rely on linear algebra, gradient descent and numerical methods.
He traced the split to two programming languages from the late 1960s. Fortran introduced arrays, which led to vectors, matrices and numerical computing. Lisp's designers took an algebraic view, using symbols to represent entities and the relations between them. The closest modern analogue to symbolic representation, he said, is a graph database with nodes and relationships, while neural AI works in vector space, which is where vector databases come in.
Neural methods have advanced quickly in the past five to seven years, Molnar said, thanks to new libraries, GPUs, CUDA and an explosion in data. Symbolic AI, he said, has not seen comparable technical breakthroughs yet, though he would not count them out.
How does symbolic AI reason?
Molnar walked through several examples:
- Bayesian networks, which add probabilities to logical dependencies and have been used in diagnostic applications.
- Prolog, a 1970s logic language. He showed a family example in which facts such as "Pam is a parent of Bob" combine with a rule defining a predecessor, so the system can answer questions such as who is a descendant of Bob.
- Grammar-based natural language processing, where formal grammars parse a sentence like "the dog saw a man in the park" into subject and object, and resources such as WordNet catalog word relationships.
- Terry Winograd's blocks world, a 1970-71 dissertation project in which a system took English commands and questions about blocks on a table. Molnar said it could reason, combine actions and plan.
"The beauty of that is it's fully deterministic," he said, meaning every answer can be explained through logical inference.
What are the limits of symbolic AI?
Scale was the main one. Molnar said many symbolic problems have been limited in scope, with small problem spaces and mostly hand-coded rules. "There's definitely a challenge of scalability that needs to be overcome," he said. Algorithms often rely on heuristics to search large state spaces.
He also described inductive logic programming, the symbolic counterpart to learning from data. Given background facts and positive and negative examples, an algorithm proposes candidate rules, such as a definition of grandfather as the father of a parent. Molnar said this can work with far fewer examples than neural training, and that the output is a rule that can be interpreted and reproduced.
Where can symbolic and neural AI work together?
Molnar contrasted the strengths. Symbolic AI offers interpretability, the ability to encode domain expertise as business rules, and traceable reasoning, including asking what would change an outcome. Neural networks offer scalability, large-scale training and freedom from manual feature extraction.
He used rule-based expert systems as an example. Systems such as MYCIN, for infectious diseases, relied on experts to define rules by hand. Large language models, he said, could automate parts of that work by extracting knowledge from text to build knowledge graphs and rules, and could turn a plain-language question into a formal query. He also suggested an agent that uses a language model with web search, as in LangChain-style examples, resembles the iterative matching of a rules engine.
Older AI waves created hype and then a letdown, Molnar noted, and he suggested generative AI could follow that cycle. That "doesn't mean that any of these Technologies are completely obsolete," he said.
What risks did the audience raise?
In the question period, an attendee raised the risk of what gets pulled into a system from outside sources. Molnar agreed that builders need to control what comes in, since searching the internet could open a system to whatever it returns, and said safeguards must be built in.
Where to hear more on AI strategy and architecture
COLLIDE 2027, billed as the largest AI conference in the Southern U.S. with more than 1,000 enterprise data and AI leaders, takes place Wednesday, October 13, 2027, at the Sandy Springs Performing Arts Center in Atlanta. Details are on the COLLIDE event page, and registration is coming soon.
Frequently asked questions
What is the difference between symbolic AI and neural AI?
Symbolic AI represents knowledge as symbols, relations and logical rules, while neural AI learns numerical patterns from data in vector space.
What is inductive logic programming?
Molnar described it as learning rules from background facts and positive and negative examples, producing interpretable rules from relatively few examples.
Can LLMs and expert systems work together?
He said language models could automate building the knowledge base and translate user questions into formal queries, reducing the manual work that limited classic expert systems.
Why did classic symbolic AI struggle to scale?
According to Molnar, problem spaces were small and most rules were hand-coded, so scalability remained a challenge.
This article is based on remarks by Peter Molnar, Associate Professor, Institute for Insight, Robinson College of Business, Georgia State University at COLLIDE Data + AI Conference 2023 on October 3, 2023. More coverage: COLLIDE AI Conference.
Filed under: Context, Semantics & RAG, AI Use Cases & Applications
