MaxSAT Evaluations SURVEY & RECENT RESULTS Outline Max-SAT overview Sub problems of Max-SAT Applications of Max-SAT Annual Evaluations, 2015 evaluation Example of a solver evaluations Summary V Reminder What is MaxSAT? The problem of determining the maximum number of clauses, of a given Boolean formula in conjunctive normal form (CNF), that can be made true by an assignment of truth values to the variables of the formula (Wikipedia). Reminder NP-HARD Optimization of SAT Difficult to find an approximate solution of the problem Large-size MAX-SAT instances cannot be solved exactly, and one must resort to approximation algorithms and heuristics. Reminder The weighted maximum satisfiability problem (Weighted MAX-SAT) asks for the maximum weight which can be satisfied by any assignment, given a set of weighted clauses. The partial maximum satisfiability problem (PMAX-SAT) asks for the maximum number of clauses which can be satisfied by any assignment of a given subset of clauses (Hard clauses). The rest of the clauses is like in Max-SAT. Applications of MaxSAT Cook Levin theorem Many real-world applications can be encoded to MaxSAT: Eclipse platform uses MaxSAT for managing the plugins dependencies Error localization in C code Debugging of hardware designs Reasoning over Biological Networks (Genes) Course timetabling... Applications of Max-SAT in Automotive (Re)Configuration Applications of MaxSAT in Automotive Configuration paper Written by: Rouven Walter and Christoph Zengler and Wolfgang Kuchlin The after-sales business asks for extensions, replacements, or removal of components of a valid configuration with minimal effort. Automotive configuration can be represented as a constraint satisfaction problem and also as a CNF formula in propositional logic, where each satisfying assignment is called a valid configuration of a car. Applications of Max-SAT in Automotive (Re)Configuration SAT example With the SAT based configuration, two main problems arise. First, if the configuration is not valid, it is not possible to know which components cause the conflict. Second, even if we know the components causing the conflict, we do not know, which components to omit to get a valid configuration with a maximal number of components we wanted originally. Outline Max-SAT overview Sub problems of Max-SAT Applications of Max-SAT Annual Evaluations, 2015 evaluation Example of a solver evaluations Summary Introduction The International Conferences on Theory and Applications of Satisfiability Testing are the primary annual meetings for researchers studying the propositional satisfiability problem (SAT), a prominent problem in both theoretical and applied computer science. SAT lies at the heart of the most important open problem in complexity theory (P vs NP) and underlies many applications in, among other examples, artificial intelligence, operations research, and electronic design engineering. Introduction The primary objective of the conferences is to bring together researchers from various areas and communities, including theoretical and experimental computer science as well as the many relevant application areas, to promote collaboration and the communication of new theoretical and practical results in SAT-related research and its industrial applications. Location of conferences SAT 2015 conference website Journal Introduction There have been 10 evaluations so far. The objective of the evaluation is assessing the state of the art in the field of Max-SAT solvers, as well as creating a collection of publicly available Max-SAT benchmark instances. The evaluation allows the submission of incomplete solvers in a Special Track, with the same three categories than for complete solvers, but with a reduced number of instances. Dates Opening April 25 th, 2015 Submission deadline June 30 th, 2015 Results of the evaluation -September 24 th -27 th, 2015 After a few days of the submission, a representative sample of benchmark instances were provided to the authors of solvers with the aim of allowing them to correct possible bugs and tune their solvers. We will focus on last evaluation in 2015 Rules The most attention is on complete solvers, but there is also a special track for incomplete solvers. We will discuss the difference and definitions of complete and incomplete solvers soon, but first lets see how we register to this event. There are three categories: Max-SAT (Unweighted) Partial Max-SAT Weighted Partial Max-SAT We will focus on the Max-SAT category What is a solver? Solver Benchmarks and solver requirements Input The input file must be read from the file given in parameter, for example:./mysolver The input file format for Max-SAT will be in DIMACS format, for example: c comments p cnf Benchmarks and Solver Requirements DIMACS The file can start with comments, that is lines beginning with the character 'c'. Right after the comments, there is the line "p cnf nbvar nbclauses" indicating that the instance is in CNF format; nbvar is the number of a variables appearing in the file; nbclauses is the exact number of clauses contained in the file. Then the clauses follow. Each clause is a sequence of distinct non-null numbers between nbvar and nbvar ending with 0 on the same line. Positive numbers denote the corresponding variables. Negative numbers denote the negations of the corresponding variables. Benchmarks and Solver Requirements Output The solvers must output messages on the standard output that will be used to check the results. The output format is inspired by the DIMACS output specification of the SAT competition and may be used to manually check some results. The solver cannot write to any files except standard output and standard error (only standard output will be parsed for results, but both output and error will be memorized during the whole evaluation process, for all executions). Benchmarks and Solver Requirements Output messages Lets go over it in the website: MaxSAT 2015 Benchmarks and Solver Requirements Benchmarks categories industrial - come from a real application random - are randomly generated crafted - all others - they are usually designed to give a hard time to solvers Machine Specifications Only two versions of the same solver will be accepted for each category. In earlier competitions one version was allowed per solver. For each instance and solver there will be a time limit of 30 minutes. Assessment of solvers will be based on the number of successfully solved instances and the time needed to solve them. Solvers will run on machines with the following specification: Operating System: CentOS release x86_64 GNU/Linux Processor: Intel(R) Xeon(R) CPU E GHz Memory: 3.5 GB Cache: KB Compilers: GCC 4.4.6, javac J2RE 2.3 Complete vs incomplete Complete - identify provenly optimal solutions in finite time. Meaning if the solver is finished then an optimal solution is given, if exists. Incomplete solvers cannot determine wither the solution given is an optimal one. In general, incomplete solvers are faster. Complete vs incomplete Per instance a solver gets - Complete : 1800 seconds (30 minutes) - Incomplete : 300 seconds (5 minutes) Incomplete track The input format is the same than for complete solvers. The output format will be the following: The solver must print the optimal solution lines ("o" line) as soon as possible. For each "o" the environment will record the user time at which the line has been printed. In order to verify the optimum, a values line ("v" line) will be required. Only the last "v" line printed will be considered, in which all the solution values must be. The best solver, for a given instance, will be the one giving the best solution. Ties will be broken by user time of the best optimum The benchmarks in each category of this Special Track will be a selection of instances of each sub-category Results - Incomplete Results random Presentation Max-SAT 2015 Presentation given in 2015 after evaluations Outline Max-SAT overview Sub problems of Max-SAT Applications of Max-SAT Annual Evaluations, 2015 evaluation Example of a solver evaluations Summary Solvers Branch and Bound (B&B) based: Clone, MaxSatz, IncMaxSatz, IUT_MaxSatz, GIDSHSat. Satisfiability based: SAT4J, QMaxSat. Unsatisfiability based: WBO, msuncore, WPM1, PM2. Example of a solver Open-WBO Authors: Ruben martins, Vasco Manquinho, Ines Lynce Contributors: Saurabh Joshi An open source version of the MaxSAT solver WBO Won several awards in MaxSAT evaluation 2014 Implements unsatisfiability-based algorithm and linear search algorithm Can solve all MaxSAT variants, namely, unweighted MaxSAT, partial MaxSAT, and weighted partial MaxSAT Result in 2015 Open-WBO Linear search algorithm Optimum solution (OPT): Assignment with minimum cost Upper Bound (UB) value: Cost greater than or equal to OPT Linear search algorithms: Refine UB value until OPT is found UB OPT Open-WBO Linear search algorithm Optimum solution (OPT): Assignment with minimum cost Upper Bound (UB) value: Cost greater than or equal to OPT Linear search algorithms: Refine UB value until OPT is found UB OPT Open-WBO Linear search algorithm Optimum solution (OPT): Assignment with minimum cost Upper Bound (UB) value: Cost greater than or equal to OPT Linear search algorithms: Refine UB value until OPT is found UB OPT Open-WBO Unsatisfiability-based algorithms Optimum solution (OPT): Assignment with minimum cost Lower Bound (LB) value: Cost smaller than or equal to OPT Unsatisfiability-based algorithms: Use unsatisfiable subformulas to refine LB value until OPT is found LB OPT Open-WBO Unsatisfiability-based algorithms Optimum solution (OPT): Assignment with minimum cost Lower Bound (LB) value: Cost smaller than or equal to OPT Unsatisfiability-based algorithms: Use unsatisfiable subformulas to refine LB value until OPT is found LB OPT Open-WBO Unsatisfiability-based algorithms Optimum solution (OPT): Assignment with minimum cost Lower Bound (LB) value: Cost smaller than or equal to OPT Unsatisfiability-based algorithms: Use unsatisfiable subformulas to refine LB value until OPT is found LB OPT Open-WBO Unsatisfiability-based algorithms Consists of identifying unsatisfiable subformulas, relaxing each clause in each unsatisfiable subformula, and adding a new constraint requiring exactly one relaxation variable to be relaxed in each unsatisfiable subformula. x xn 1 Ladder encoding Open-WBO Outline Max-SAT overview Sub problems of Max-SAT Applications of Max-SAT Annual Evaluations, 2015 evaluation Example of a solver evaluations Summary The First and Second Max-SAT Evaluations Written by: Josep Argelich Chu-Min Li Felip Many`a Jordi Planes The First and Second Max-SAT Evaluations Reasons for establish and organize the evaluations: Growing interest of the SAT community in developing fast exact Max-SAT solvers Lack of a good collection of benchmarks Good experiences gathered from the SAT competitions Competitions Competitions The First and Second Max-SAT Evaluations Main objectives of the evaluations: Assessing the advancements in the field of Max-SAT solvers through a comparison of their performances Identifying successful solving techniques Encouraging researchers to develop new ones Creating a publicly available collection of challenging Max- SAT benchmarks 2006 Max-SAT 6 Solvers Unweighted Max-SAT and Weighted Max-SAT 2007 Max-SAT 12 Solvers Two additional categories: Partial Max-SAT and Weighted Partial Max-SAT Focus on complete solvers Most of the benchmarks used in the first two evaluations were contributed The organizers added random Max-2-SAT and Max-3- SAT instances Changing the clausetovariable ratio The First and Second Max-SAT Evaluations zChaff A solver participated in 2006 Max-SAT. Based on Chaff algorithm, which is an improvement of DPLL algorithm. The First and Second Max-SAT Evaluations DPLL algorithm DavisPutnamLogemannLoveland 50 Years ago Basic algorithm for SAT Many of the algorithms and heuristics are expansions and improvement of this algorithm The First and Second Max-SAT Evaluations DPLL algorithm DavisPutnamLogemannLoveland Unit Propagation Pure Variable Assignment Backtracking The First and Second Max-SAT Evaluations Results unweighted Max-SAT The First and Second Max-SAT Evaluations What the first evaluations have contributed? Provided a quite accurate snapshot of the current state-of-the-art of complete Max-SAT solvers Contributed to increase the interest and activity of the research community on Max-SAT Allowed to identify a number of good performing solving technique The First and Second Max-SAT Evaluations What the first evaluations have contributed? Promoted the creation of a publicly available collection of challenging Max-SAT benchmarks Most authors of solvers had the opportunity of correcting a significant number of bugs in their code From benchmarks, in each category, are divided into 3 types: random, crafted and industrial The First and Second Max-SAT Evaluations Outline Max-SAT overview Sub problems of Max-SAT Applications of Max-SAT Annual Evaluations, 2015 evaluation Example of a solver evaluations Summary Thank You!