2402 00854 SymbolicAI: A framework for logic-based approaches combining generative models and solvers

We refer to the empirical score as the “Vector Embedding for Relational Trajectory Evaluation through Cross-similarity”, or VERTEX score for short. For organizations looking forward to the day they can interact with AI just like a person, symbolic AI is how it will happen, says tech journalist Surya Maddula. After all, we humans developed reason by first learning the rules of how things interrelate, then applying those rules to other situations – pretty much the way symbolic AI is trained.

Although operating with 256,000 noisy nanoscale phase-change memristive devices, there was just a 2.7 percent accuracy drop compared to the conventional software realizations in high precision. There are several flavors of question answering (QA) tasks – text-based QA, context-based QA (in the context of interaction or dialog) or knowledge-based QA (KBQA). We chose to focus on KBQA because such tasks truly demand advanced reasoning such as multi-hop, quantitative, geographic, and temporal reasoning.

Resources for Deep Learning and Symbolic Reasoning

Researchers are uncovering the connections between deep nets and principles in physics and mathematics. This video shows a more sophisticated challenge, called CLEVRER, in which artificial intelligences had to answer questions about video sequences showing objects in motion. The video previews the sorts of questions that could be asked, and later parts of the video show how one AI converted the questions into machine-understandable form. If you ask it questions for which the knowledge is either missing or erroneous, it fails. In the emulated duckling example, the AI doesn’t know whether a pyramid and cube are similar, because a pyramid doesn’t exist in the knowledge base. To reason effectively, therefore, symbolic AI needs large knowledge bases that have been painstakingly built using human expertise.

Symbolic AI algorithms have played an important role in AI’s history, but they face challenges in learning on their own. After IBM Watson used symbolic reasoning to beat Brad Rutter and Ken Jennings at Jeopardy in 2011, the technology has been eclipsed by neural networks trained by deep learning. Building on the foundations of deep learning and symbolic AI, we have developed technology that can answer complex questions with minimal domain-specific training. Initial results are very encouraging – the system outperforms current state-of-the-art techniques on two prominent datasets with no need for specialized end-to-end training.

Logic Programming and Symbolic AI:

We also briefly touch upon the Turing Test for Ethical AI, and the pluralistic nature of Turing-type Tests for Integrated AI. Overall, we believe that the integrated approach to cognition gives the key to the next generation paradigm for AI and Cognitive Science in general, and that Categorical Integrated AI or Categorical Integrative AI Robotics would be arguably the most promising approach to it. However, the black-box nature of classic neural models, with most confirmations on their learning abilities being done empirically rather than analytically, renders some direct integration with the symbolic systems, possibly providing the missing capabilities, rather complicated. This is easy to think of as a boolean circuit (neural network) sitting on top of a propositional interpretation (feature vector).

They involve every individual memory entry instead of a single discrete entry. We use symbols all the time to define things (cat, car, airplane, etc.) and people (teacher, police, salesperson). Symbols can represent abstract concepts (bank transaction) or things that don’t physically exist (web page, blog post, etc.).

In this article, we will look into some of the original symbolic AI principles and how they can be combined with deep learning to leverage the benefits of both of these, seemingly unrelated (or even contradictory), approaches to learning and AI. Our researchers are working to usher in a new era of AI where machines can learn more like the way humans do, by connecting words with images and mastering abstract concepts. The team solved the first problem by using a number of convolutional neural networks, a type of deep net that’s optimized for image recognition.

• YAGO incorporates WordNet as part of its ontology, to align facts extracted from Wikipedia with WordNet synsets.
• The similarity search on these wide vectors can be efficiently computed by exploiting physical laws such as Ohm’s law and Kirchhoff’s current summation law.
• Symbolic AI programs are based on creating explicit structures and behavior rules.
• The universe is written in the language of mathematics and its characters are triangles, circles, and other geometric objects.

Class instances can also perform actions, also known as functions, methods, or procedures. Each method executes a series of rule-based instructions that might read and change the properties of the current and other objects. The difficulties encountered by symbolic AI have, however, been deep, possibly unresolvable ones. One difficult problem encountered by symbolic AI pioneers came to be known as the common sense knowledge problem.

Its history was also influenced by Carl Hewitt’s PLANNER, an assertional database with pattern-directed invocation of methods. For more detail see the section on the origins of Prolog in the PLANNER article. Interestingly, we note that the simple logical XOR function is actually still challenging to learn properly even in modern-day deep learning, which we will discuss in the follow-up article. AI researchers like Gary Marcus have argued that these systems struggle with answering questions like, “Which direction is a nail going into the floor pointing?” This is not the kind of question that is likely to be written down, since it is common sense. This approach was experimentally verified for a few-shot image classification task involving a dataset of 100 classes of images with just five training examples per class.

Symbolic AI-driven chatbots exemplify the application of AI algorithms in customer service, showcasing the integration of AI Research findings into real-world AI Applications. Rule-Based AI, a cornerstone of Symbolic AI, involves creating AI systems that apply predefined rules. This concept is fundamental in AI Research Labs and universities, contributing to significant Development Milestones in AI. At the heart of Symbolic AI lie key concepts such as Logic Programming, Knowledge Representation, and Rule-Based AI. These elements work together to form the building blocks of Symbolic AI systems. McCarthy’s approach to fix the frame problem was circumscription, a kind of non-monotonic logic where deductions could be made from actions that need only specify what would change while not having to explicitly specify everything that would not change.

Moreover, they lack the ability to reason on an abstract level, which makes it difficult to implement high-level cognitive functions such as transfer learning, analogical reasoning, and hypothesis-based reasoning. Finally, their operation is largely opaque to humans, rendering them unsuitable for domains in which verifiability is important. In this paper, we propose an end-to-end reinforcement learning architecture comprising a neural back end and a symbolic front end with the potential to overcome each of these shortcomings. As proof-of-concept, we present a preliminary implementation what is symbolic ai of the architecture and apply it to several variants of a simple video game. We show that the resulting system – though just a prototype – learns effectively, and, by acquiring a set of symbolic rules that are easily comprehensible to humans, dramatically outperforms a conventional, fully neural DRL system on a stochastic variant of the game. Symbolic AI, also known as Good Old-Fashioned Artificial Intelligence (GOFAI), is a paradigm in artificial intelligence research that relies on high-level symbolic representations of problems, logic, and search to solve complex tasks.

• Roughly speaking, the hybrid uses deep nets to replace humans in building the knowledge base and propositions that symbolic AI relies on.
• During the first AI summer, many people thought that machine intelligence could be achieved in just a few years.
• Similarly, LISP machines were built to run LISP, but as the second AI boom turned to bust these companies could not compete with new workstations that could now run LISP or Prolog natively at comparable speeds.
• Symbolic AI-driven chatbots exemplify the application of AI algorithms in customer service, showcasing the integration of AI Research findings into real-world AI Applications.

The team’s solution was about 88 percent accurate in answering descriptive questions, about 83 percent for predictive questions and about 74 percent for counterfactual queries, by one measure of accuracy. Such causal and counterfactual reasoning about things that are changing with time is extremely difficult for today’s deep neural networks, which mainly excel at discovering static patterns in data, Kohli says. The unlikely marriage of two major artificial intelligence approaches has given rise to a new hybrid called neurosymbolic AI. It’s taking baby steps toward reasoning like humans and might one day take the wheel in self-driving cars. The thing symbolic processing can do is provide formal guarantees that a hypothesis is correct. This could prove important when the revenue of the business is on the line and companies need a way of proving the model will behave in a way that can be predicted by humans.

Hadayat Seddiqi, director of machine learning at InCloudCounsel, a legal technology company, said the time is right for developing a neuro-symbolic learning approach. “Deep learning in its present state cannot learn logical rules, since its strength comes from analyzing correlations in the data,” he said. This directed mapping helps the system to use high-dimensional algebraic operations for richer object manipulations, such as variable binding — an open problem in neural networks. When these “structured” mappings are stored in the AI’s memory (referred to as explicit memory), they help the system learn—and learn not only fast but also all the time. The ability to rapidly learn new objects from a few training examples of never-before-seen data is known as few-shot learning. Symbolic AI is a sub-field of artificial intelligence that focuses on the high-level symbolic (human-readable) representation of problems, logic, and search.

More advanced knowledge-based systems, such as Soar can also perform meta-level reasoning, that is reasoning about their own reasoning in terms of deciding how to solve problems and monitoring the success of problem-solving strategies. For instance, one prominent idea was to encode the (possibly infinite) interpretation structures of a logic program by (vectors of) real numbers and represent the relational inference as a (black-box) mapping between these, based on the universal approximation theorem. However, this assumes the unbound relational information to be hidden in the unbound decimal fractions of the underlying real numbers, which is naturally completely impractical for any gradient-based learning. This idea has also been later extended by providing corresponding algorithms for symbolic knowledge extraction back from the learned network, completing what is known in the NSI community as the “neural-symbolic learning cycle”.

In symbolic AI (upper left), humans must supply a “knowledge base” that the AI uses to answer questions. During training, they adjust the strength of the connections between layers of nodes. The hybrid uses deep nets, instead of humans, to generate only those portions of the knowledge base that it needs to answer a given question.

Neuro-symbolic artificial intelligence can be defined as the subfield of artificial intelligence (AI) that combines neural and symbolic approaches. By symbolic we mean approaches that rely on the explicit representation of knowledge using formal languages—including formal logic—and the manipulation of language items (‘symbols’) by algorithms to achieve a goal. Neuro-symbolic AI has a long history; however, it remained a rather niche topic until recently, when landmark advances in machine learning—prompted by deep learning—caused a significant rise in interest and research activity in combining neural and symbolic methods. In this overview, we provide a rough guide to key research directions, and literature pointers for anybody interested in learning more about the field.

We have laid out some of the most important currently investigated research directions, and provided literature pointers suitable as entry points to an in-depth study of the current state of the art. First of all, every deep neural net trained by supervised learning combines deep learning and symbolic manipulation, at least in a rudimentary sense. Because symbolic reasoning encodes knowledge in symbols and strings of characters. In supervised learning, those strings of characters are called labels, the categories by which we classify input data using a statistical model. The output of a classifier (let’s say we’re dealing with an image recognition algorithm that tells us whether we’re looking at a pedestrian, a stop sign, a traffic lane line or a moving semi-truck), can trigger business logic that reacts to each classification. The greatest promise here is analogous to experimental particle physics, where large particle accelerators are built to crash atoms together and monitor their behaviors.

In particular, the level of reasoning required by these questions is relatively simple. In this work, we approach KBQA with the basic premise that if we can correctly translate the natural language questions into an abstract form that captures the question’s conceptual meaning, we can reason over existing knowledge to answer complex questions. Table 1 illustrates the kinds of questions NSQA can handle and the form of reasoning required to answer different questions. This approach provides interpretability, generalizability, and robustness— all critical requirements in enterprise NLP settings . OOP languages allow you to define classes, specify their properties, and organize them in hierarchies. You can create instances of these classes (called objects) and manipulate their properties.

Mimicking the brain: Deep learning meets vector-symbolic AI

New deep learning approaches based on Transformer models have now eclipsed these earlier symbolic AI approaches and attained state-of-the-art performance in natural language processing. However, Transformer models are opaque and do not yet produce human-interpretable semantic representations for sentences and documents. Instead, they produce task-specific vectors where the meaning of the vector components is opaque. Neurosymbolic AI is also demonstrating the ability to ask questions, an important aspect of human learning.

First, a neural network learns to break up the video clip into a frame-by-frame representation of the objects. This is fed to another neural network, which learns to analyze the movements of these objects and how they interact with each other and can predict the motion of objects and collisions, if any. The other two modules process the question and apply it to the generated knowledge base.

Cell meets robot in hybrid microbots

Symbolic AI has been used in a wide range of applications, including expert systems, natural language processing, and game playing. It can be difficult to represent complex, ambiguous, or uncertain knowledge with symbolic AI. Furthermore, symbolic AI systems are typically hand-coded and do not learn from data, which can make them brittle and inflexible. A key component of the system architecture for all expert systems is the knowledge base, which stores facts and rules for problem-solving.[52]
The simplest approach for an expert system knowledge base is simply a collection or network of production rules. Production rules connect symbols in a relationship similar to an If-Then statement.

Symbolic AI’s strength lies in its knowledge representation and reasoning through logic, making it more akin to Kahneman’s “System 2” mode of thinking, which is slow, takes work and demands attention. That is because it is based on relatively simple underlying logic that relies on things being true, and on rules providing a means of inferring new things from things already known to be true. Question-answering is the first major use case for the LNN technology we’ve developed. While achieving state-of-the-art performance on the two KBQA datasets is an advance over other AI approaches, these datasets do not display the full range of complexities that our neuro-symbolic approach can address.

Real-World Applications of Symbolic AI:

Many other approaches only support simpler forms of logic like propositional logic, or Horn clauses, or only approximate the behavior of first-order logic. Symbolic artificial intelligence showed early progress at the dawn of AI and computing. You can easily visualize the logic of rule-based programs, communicate them, and troubleshoot them. This article was written to answer the question, “what is symbolic artificial intelligence.” Looking to enhance your understanding of the world of AI? Symbolic AI has numerous applications, from Cognitive Computing in healthcare to AI Research in academia. Its ability to process complex rules and logic makes it ideal for fields requiring precision and explainability, such as legal and financial domains.

The expert system processes the rules to make deductions and to determine what additional information it needs, i.e. what questions to ask, using human-readable symbols. For example, OPS5, CLIPS and their successors Jess and Drools operate in this fashion. We believe that our results are what is symbolic ai the first step to direct learning representations in the neural networks towards symbol-like entities that can be manipulated by high-dimensional computing. Such an approach facilitates fast and lifelong learning and paves the way for high-level reasoning and manipulation of objects.

The benefits and limits of symbolic AI

The world is presented to applications that use symbolic AI as images, video and natural language, which is not the same as symbols. This is important because all AI systems in the real world deal with messy data. For example, in an application that uses AI to answer questions about legal contracts, simple business logic can filter out data from documents that are not contracts or that are contracts in a different domain such as financial services versus real estate.

It represents problems using relations, rules, and facts, providing a foundation for AI reasoning and decision-making, a core aspect of Cognitive Computing. Marvin Minsky first proposed frames as a way of interpreting common visual situations, such as an office, and Roger Schank extended this idea to scripts for common routines, such as dining out. Cyc has attempted to capture useful common-sense knowledge and has “micro-theories” to handle particular kinds of domain-specific reasoning. Perhaps surprisingly, the correspondence between the neural and logical calculus has been well established throughout history, due to the discussed dominance of symbolic AI in the early days.

The idea was based on the, now commonly exemplified, fact that logical connectives of conjunction and disjunction can be easily encoded by binary threshold units with weights — i.e., the perceptron, an elegant learning algorithm for which was introduced shortly. But adding a small amount of white noise to the image (indiscernible to humans) causes the deep net to confidently misidentify it as a gibbon. Research in neuro-symbolic AI has a very long tradition, and we refer the interested reader to overview works such as Refs [1,3] that were written before the most recent developments. Indeed, neuro-symbolic AI has seen a significant increase in activity and research output in recent years, together with an apparent shift in emphasis, as discussed in Ref. [2]. Below, we identify what we believe are the main general research directions the field is currently pursuing.

But the benefits of deep learning and neural networks are not without tradeoffs. Deep learning has several deep challenges and disadvantages in comparison to symbolic AI. Notably, deep learning algorithms are opaque, and figuring out how they work perplexes even their creators. Deep learning and neural networks excel at exactly the tasks that symbolic AI struggles with. They have created a revolution in computer vision applications such as facial recognition and cancer detection.

A gentle introduction to model-free and model-based reinforcement learning

Cognitive architectures such as ACT-R may have additional capabilities, such as the ability to compile frequently used knowledge into higher-level chunks. Japan championed Prolog for its Fifth Generation Project, intending to build special hardware for high performance. Similarly, LISP machines were built to run LISP, but as the second AI boom turned to bust these companies could not compete with new workstations that could now run LISP or Prolog natively at comparable speeds. Our chemist was Carl Djerassi, inventor of the chemical behind the birth control pill, and also one of the world’s most respected mass spectrometrists. We began to add to their knowledge, inventing knowledge of engineering as we went along.