Constrained Neural Networks for Interpretable Heuristic Creation to Optimise Computer Algebra Systems
2404.17508
0
0
🧠
Abstract
We present a new methodology for utilising machine learning technology in symbolic computation research. We explain how a well known human-designed heuristic to make the choice of variable ordering in cylindrical algebraic decomposition may be represented as a constrained neural network. This allows us to then use machine learning methods to further optimise the heuristic, leading to new networks of similar size, representing new heuristics of similar complexity as the original human-designed one. We present this as a form of ante-hoc explainability for use in computer algebra development.
Get summaries of the top AI research delivered straight to your inbox:
Overview
- Presents a new methodology for using machine learning in symbolic computation research
- Demonstrates how a human-designed heuristic for choosing variable ordering in cylindrical algebraic decomposition can be represented as a constrained neural network
- Explores using machine learning to further optimize the heuristic, leading to new networks of similar size and complexity as the original
Plain English Explanation
This paper introduces a novel approach to leveraging machine learning technology in the field of symbolic computation research. The researchers explain how a well-known human-created heuristic, used to make the choice of variable ordering in a mathematical technique called cylindrical algebraic decomposition, can be represented as a constrained neural network. This allows the researchers to then use machine learning methods to further optimize the heuristic, resulting in new neural networks that are similar in size and complexity to the original human-designed one. The researchers present this as a form of ante-hoc explainability for use in the development of computer algebra systems.
Technical Explanation
The paper demonstrates how a human-designed heuristic for choosing variable ordering in cylindrical algebraic decomposition can be represented as a constrained neural network. By doing so, the researchers are able to leverage machine learning techniques to further optimize the heuristic, leading to the development of new neural networks that are of similar size and complexity as the original. This process is presented as a form of ante-hoc explainability, which can be useful for enhancing the development of computer algebra systems.
Critical Analysis
The paper presents a novel approach to using machine learning to improve symbolic computation research, which could have significant implications for the field. However, the researchers do not discuss any potential limitations or caveats of their methodology. Additionally, it would be helpful to see more details on the specific machine learning techniques used to optimize the heuristic, as well as the performance of the new networks compared to the original. Further research could explore the broader applicability of this constrained neural network approach to other areas of symbolic computation.
Conclusion
This paper introduces a new methodology for integrating machine learning into symbolic computation research, specifically by using neural networks to represent and optimize a human-designed heuristic for variable ordering in cylindrical algebraic decomposition. The researchers present this as a form of ante-hoc explainability, which could aid in the development of more robust and transparent computer algebra systems. While the approach shows promise, further research is needed to fully understand its limitations and broader applications.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
🛠️
New!Distributed Constrained Combinatorial Optimization leveraging Hypergraph Neural Networks
Nasimeh Heydaribeni, Xinrui Zhan, Ruisi Zhang, Tina Eliassi-Rad, Farinaz Koushanfar
0
0
Scalable addressing of high dimensional constrained combinatorial optimization problems is a challenge that arises in several science and engineering disciplines. Recent work introduced novel application of graph neural networks for solving quadratic-cost combinatorial optimization problems. However, effective utilization of models such as graph neural networks to address general problems with higher order constraints is an unresolved challenge. This paper presents a framework, HypOp, which advances the state of the art for solving combinatorial optimization problems in several aspects: (i) it generalizes the prior results to higher order constrained problems with arbitrary cost functions by leveraging hypergraph neural networks; (ii) enables scalability to larger problems by introducing a new distributed and parallel training architecture; (iii) demonstrates generalizability across different problem formulations by transferring knowledge within the same hypergraph; (iv) substantially boosts the solution accuracy compared with the prior art by suggesting a fine-tuning step using simulated annealing; (v) shows a remarkable progress on numerous benchmark examples, including hypergraph MaxCut, satisfiability, and resource allocation problems, with notable run time improvements using a combination of fine-tuning and distributed training techniques. We showcase the application of HypOp in scientific discovery by solving a hypergraph MaxCut problem on NDC drug-substance hypergraph. Through extensive experimentation on various optimization problems, HypOp demonstrates superiority over existing unsupervised learning-based solvers and generic optimization methods.
5/20/2024
🛠️
An algorithmic framework for the optimization of deep neural networks architectures and hyperparameters
Julie Keisler (EDF R&D OSIRIS, EDF R&D, CRIStAL, CRIStAL), El-Ghazali Talbi (CRIStAL, CRIStAL), Sandra Claudel (EDF R&D OSIRIS, EDF R&D), Gilles Cabriel (EDF R&D OSIRIS, EDF R&D)
0
0
In this paper, we propose an algorithmic framework to automatically generate efficient deep neural networks and optimize their associated hyperparameters. The framework is based on evolving directed acyclic graphs (DAGs), defining a more flexible search space than the existing ones in the literature. It allows mixtures of different classical operations: convolutions, recurrences and dense layers, but also more newfangled operations such as self-attention. Based on this search space we propose neighbourhood and evolution search operators to optimize both the architecture and hyper-parameters of our networks. These search operators can be used with any metaheuristic capable of handling mixed search spaces. We tested our algorithmic framework with an evolutionary algorithm on a time series prediction benchmark. The results demonstrate that our framework was able to find models outperforming the established baseline on numerous datasets.
5/15/2024
Neural Networks with Causal Graph Constraints: A New Approach for Treatment Effects Estimation
Roger Pros, Jordi Vitri`a
0
0
In recent years, there has been a growing interest in using machine learning techniques for the estimation of treatment effects. Most of the best-performing methods rely on representation learning strategies that encourage shared behavior among potential outcomes to increase the precision of treatment effect estimates. In this paper we discuss and classify these models in terms of their algorithmic inductive biases and present a new model, NN-CGC, that considers additional information from the causal graph. NN-CGC tackles bias resulting from spurious variable interactions by implementing novel constraints on models, and it can be integrated with other representation learning methods. We test the effectiveness of our method using three different base models on common benchmarks. Our results indicate that our model constraints lead to significant improvements, achieving new state-of-the-art results in treatment effects estimation. We also show that our method is robust to imperfect causal graphs and that using partial causal information is preferable to ignoring it.
4/19/2024
🧠
Design Requirements for Human-Centered Graph Neural Network Explanations
Pantea Habibi, Peyman Baghershahi, Sourav Medya, Debaleena Chattopadhyay
0
0
Graph neural networks (GNNs) are powerful graph-based machine-learning models that are popular in various domains, e.g., social media, transportation, and drug discovery. However, owing to complex data representations, GNNs do not easily allow for human-intelligible explanations of their predictions, which can decrease trust in them as well as deter any collaboration opportunities between the AI expert and non-technical, domain expert. Here, we first discuss the two papers that aim to provide GNN explanations to domain experts in an accessible manner and then establish a set of design requirements for human-centered GNN explanations. Finally, we offer two example prototypes to demonstrate some of those proposed requirements.
5/14/2024