From 6e1288eaecb1d1253579be87d9d0f0ffe67360f2 Mon Sep 17 00:00:00 2001 From: Guilherme Bernardo Date: Sat, 22 Aug 2026 18:44:49 -0400 Subject: [PATCH 01/17] feat(testcomp): Cover-Branches suites from KLEE's own path vectors Cover-Branches is the second scoreable Test-Comp category and Map2Check scored nothing in it, because it had no source of per-path input vectors. KLEE already had them. It writes klee-last/testNNNNNN.ktest per explored path, each recording the symbolic objects in the order the program consumed them. Nothing needed to be instrumented; the vectors were being computed and thrown away. The alternative was tried first and abandoned on measurement. Having the instrumented program write one log per terminating state turned a 1-second run that answered FALSE into a 100-second run that exhausted its budget and answered TRUE: every write is an external call KLEE executes concretely, and enough of them corrupt the result rather than merely slowing it. Reading .ktest files after the search has finished costs the search nothing. modules/frontend/test_suite/ktest_reader.{hpp,cpp} Parses the format directly rather than shelling out to ktest-tool -- nine lines of big-endian counts, against putting a Python interpreter between a verification run and its suite. Decoding is driven by the object NAME, which is where the type lives: a .ktest records name, size and bytes but no type, so signedness has nowhere else to come from. Read as unsigned, -1 becomes 4294967295. --cover-branches Emits one test case per path, coversError false throughout: these are paths, not violations. Capped at 500 -- the competition scores coverage, not volume, and every test case costs the validator a compile-and-run. The wrapper and the BenchExec tool-info stop refusing the property. Both used to raise rather than accept it, deliberately, so that scoring zero could not be mistaken for participating; that reason is gone. Evaluation harness, for runs this repository could not do before: tests/testcomp/fetch-benchmarks.sh shallow clone of sv-benchmarks (~13 GB, gitignored, never committed) tests/testcomp/build_corpus.py stratified manifest, quota per subcategory and a deterministic stride inside each -- a uniform sample of 33k tasks would just report the shape of the benchmark, where Floats and ECA alone are 2.6k tests/testcomp/run_testcomp_evaluation.sh runs the corpus, hands every suite to TestCov, resumes from its own CSV Verified end to end: 7 test cases from KLEE paths on a six-branch program, TestCov reporting 100.0% coverage over 6 goals. Unit tests 9/9 suites, including 16 new ones pinning the .ktest parse against bytes built by hand -- a misparse yields plausible numbers, not a crash, and is silent all the way down to a suite that covers nothing. Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/ci.yml | 6 + .gitignore | 6 + modules/frontend/map2check.cpp | 48 +- modules/frontend/test_suite/CMakeLists.txt | 5 +- modules/frontend/test_suite/ktest_reader.cpp | 204 ++++++++ modules/frontend/test_suite/ktest_reader.hpp | 73 +++ modules/frontend/utils/tools.hpp | 5 + tests/integration/test_benchexec_toolinfo.py | 15 +- tests/integration/test_cover_branches.sh | 126 +++++ tests/testcomp/build_corpus.py | 167 +++++++ tests/testcomp/corpus/cover-branches.tsv | 481 +++++++++++++++++++ tests/testcomp/corpus/cover-error.tsv | 373 ++++++++++++++ tests/testcomp/fetch-benchmarks.sh | 27 ++ tests/testcomp/run_testcomp_evaluation.sh | 173 +++++++ tests/unit/frontend/CMakeLists.txt | 6 + tests/unit/frontend/KtestReaderTest.cpp | 257 ++++++++++ utils/map2check-testcomp-wrapper.py | 18 +- utils/moduleBenchExec/map2check_testcomp.py | 19 +- 18 files changed, 1985 insertions(+), 24 deletions(-) create mode 100644 modules/frontend/test_suite/ktest_reader.cpp create mode 100644 modules/frontend/test_suite/ktest_reader.hpp create mode 100755 tests/integration/test_cover_branches.sh create mode 100755 tests/testcomp/build_corpus.py create mode 100644 tests/testcomp/corpus/cover-branches.tsv create mode 100644 tests/testcomp/corpus/cover-error.tsv create mode 100755 tests/testcomp/fetch-benchmarks.sh create mode 100755 tests/testcomp/run_testcomp_evaluation.sh create mode 100644 tests/unit/frontend/KtestReaderTest.cpp diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 28cfbf0ce..71629701a 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -504,6 +504,12 @@ jobs: # infrastructure. Nothing else here exercises it, and a wrong one # only shows up days after submission as every task erroring out. python3 tests/integration/test_benchexec_toolinfo.py + # Cover-Branches: the other competition category, and the one + # whose suite has more than one test case. Validated by TestCov, + # not by inspecting the XML -- only the validator can say whether + # the branches are actually exercised. + MAP2CHECK_PATH=/workspace/install_testcov_ci \ + bash tests/integration/test_cover_branches.sh bash tests/testcomp/run_testcov_suite.sh ' diff --git a/.gitignore b/.gitignore index 3ffe901be..d93e6c8dc 100644 --- a/.gitignore +++ b/.gitignore @@ -109,3 +109,9 @@ install_testcov_ci/ .claude/ .opencode/ CLAUDE.md.bak-* + +# Test-Comp benchmark corpus (sv-benchmarks, ~13 GB). Cloned on demand for the +# stratified runs; never committed. The pre-existing "sv-benchmarks/*" rule +# above ignores the CONTENTS of such a directory but not the directory entry +# itself, which is why this line names the path we actually clone into. +tests/testcomp/bench/ diff --git a/modules/frontend/map2check.cpp b/modules/frontend/map2check.cpp index 314309176..3f3a96b33 100644 --- a/modules/frontend/map2check.cpp +++ b/modules/frontend/map2check.cpp @@ -29,6 +29,7 @@ #include "caller.hpp" #include "counter_example/counter_example.hpp" +#include "test_suite/ktest_reader.hpp" #include "test_suite/test_suite.hpp" #include "utils/gen_crypto_hash.hpp" #include "utils/log.hpp" @@ -105,10 +106,20 @@ std::string resolveSpecification(const std::string &propertyFile, * * Must run before Caller::cleanGarbage(): klee_log.csv lives in the scratch * directory that cleanGarbage() deletes, and the suite must not. */ +/** Upper bound on a Cover-Branches suite. + * + * KLEE can terminate tens of thousands of paths, and every test case costs the + * validator a compile-and-run. The competition scores coverage, not volume, so + * past a few hundred vectors the marginal branch is rare and the validation + * cost is not. The bound is here rather than in the runtime because this is + * the side that knows what the suite is for. */ +constexpr size_t kMaxBranchTestCases = 500; + void emitTestSuite(const std::string &outputDir, const std::string &programFile, const std::string &entryFunction, const std::string &architecture, - const std::string &specification, bool foundViolation) { + const std::string &specification, bool foundViolation, + bool coverBranches) { Map2Check::TestSuiteMetadata metadata; metadata.producer = std::string("Map2Check ") + Map2CheckVersion; metadata.specification = specification; @@ -124,6 +135,32 @@ void emitTestSuite(const std::string &outputDir, const std::string &programFile, outputDir); return; } + // Cover-Branches: one test case per path KLEE explored, taken from its own + // .ktest output. Nothing is asked of the instrumented program -- measured, + // when it was: making it write a log per state turned a 1-second run that + // answered FALSE into a 100-second run that exhausted its budget and + // answered TRUE, because each write is an external call KLEE executes + // concretely. Reading afterwards costs the finished search nothing. + // + // coversError is false throughout: these vectors are paths, not violations. + // The violating one, when there is one, is still in klee_log.csv and still + // goes out under Cover-Error. + if (coverBranches) { + std::vector> vectors = + Map2Check::readKtestVectors(Map2Check::kleeOutputDir, + kMaxBranchTestCases); + for (const std::vector &inputs : vectors) { + if (!writer.writeTestCase(inputs, false)) { + Map2Check::Log::Warning("could not write test case to " + outputDir); + return; + } + } + Map2Check::Log::Info("Test suite written to " + outputDir + " (" + + std::to_string(vectors.size()) + + " test cases from KLEE paths)"); + return; + } + // No violation means no test case, but the suite still has to exist. A // missing test-suite/ directory reads to the competition harness as a tool // that crashed; a suite carrying metadata and zero test cases says the tool @@ -245,6 +282,7 @@ struct map2check_args { bool btree = false; bool invCrabLlvm = false; bool generateTestSuite = false; + bool coverBranches = false; std::string testSuiteDir = "test-suite"; std::string propertyFile; std::string architecture = "64bit"; @@ -448,7 +486,7 @@ int map2check_execution(map2check_args args) { emitTestSuite(outputDir, caller->c_program_fullpath, args.entryFunction, args.architecture, resolveSpecification(args.propertyFile, args.mode), - foundViolation); + foundViolation, args.coverBranches); } // (6) Clean map2check execution (folders and temp files) @@ -508,6 +546,9 @@ z3 (Z3 is default), btor (Boolector), and yices2 (Yices))") "\temits a Test-Comp test suite reproducing the violation found") ("test-suite-dir", po::value()->default_value("test-suite"), "\tdirectory to write the test suite into") + ("cover-branches", + "\temit one test case per path KLEE explored, from its .ktest output, " + "instead of the single violating vector (Test-Comp Cover-Branches)") ("property-file", po::value(), "\tproperty file whose contents go verbatim into ") ("architecture", po::value()->default_value("64bit"), @@ -614,6 +655,9 @@ z3 (Z3 is default), btor (Boolector), and yices2 (Yices))") if (vm.count("test-suite-dir")) { args.testSuiteDir = vm["test-suite-dir"].as(); } + if (vm.count("cover-branches")) { + args.coverBranches = true; + } if (vm.count("property-file")) { args.propertyFile = vm["property-file"].as(); } diff --git a/modules/frontend/test_suite/CMakeLists.txt b/modules/frontend/test_suite/CMakeLists.txt index e6b421324..8e71ed715 100644 --- a/modules/frontend/test_suite/CMakeLists.txt +++ b/modules/frontend/test_suite/CMakeLists.txt @@ -1,2 +1,5 @@ -add_library(TestSuite OBJECT test_suite.cpp) +# One object library for both sources: they are two halves of the same job -- +# serialising a Test-Comp suite -- and the frontend links the pair together in +# every configuration. +add_library(TestSuite OBJECT test_suite.cpp ktest_reader.cpp) set_target_properties(TestSuite PROPERTIES COMPILE_FLAGS ${CPP_FLAGS}) diff --git a/modules/frontend/test_suite/ktest_reader.cpp b/modules/frontend/test_suite/ktest_reader.cpp new file mode 100644 index 000000000..85b066b0d --- /dev/null +++ b/modules/frontend/test_suite/ktest_reader.cpp @@ -0,0 +1,204 @@ +/** + * Copyright (C) 2014 - 2026 Map2Check tool + * This file is part of the Map2Check tool, and is made available under + * the terms of the GNU General Public License version 2. + * + * SPDX-License-Identifier: (GPL-2.0) + **/ + +#include "ktest_reader.hpp" + +#include +#include +#include +#include +#include +#include +#include + +namespace Map2Check { + +namespace { + +// Layout, from KLEE's own ktest-tool: a five-byte magic, then big-endian +// 32-bit counts throughout. Version 2 and later carry two extra symbolic-argv +// fields between the arguments and the objects. +constexpr char kMagicKtest[] = "KTEST"; +constexpr char kMagicBout[] = "BOUT\n"; +constexpr size_t kMagicSize = 5; + +// A sanity bound on any length read from the file. A truncated or foreign file +// yields huge counts, and resizing a vector to them is how a parser turns a +// bad input into an out-of-memory kill rather than a skipped file. +constexpr uint32_t kMaxReasonableLength = 1u << 24; // 16 MiB + +bool readBigEndian32(std::ifstream& in, uint32_t* out) { + unsigned char raw[4]; + in.read(reinterpret_cast(raw), 4); + if (in.gcount() != 4) return false; + *out = (static_cast(raw[0]) << 24) | + (static_cast(raw[1]) << 16) | + (static_cast(raw[2]) << 8) | static_cast(raw[3]); + return true; +} + +bool skipBlock(std::ifstream& in) { + uint32_t size = 0; + if (!readBigEndian32(in, &size)) return false; + if (size > kMaxReasonableLength) return false; + in.seekg(size, std::ios::cur); + return in.good(); +} + +/** Reads `size` bytes of little-endian integer into a 64-bit value. + * + * x86-64 is the only architecture this tool targets, and KLEE writes object + * bytes in the target's own order. */ +uint64_t leBytesToUnsigned(const std::vector& bytes) { + uint64_t value = 0; + const size_t width = std::min(bytes.size(), 8); + for (size_t i = 0; i < width; ++i) { + value |= static_cast(bytes[i]) << (8 * i); + } + return value; +} + +int64_t signExtend(uint64_t value, size_t byteWidth) { + if (byteWidth == 0 || byteWidth >= 8) return static_cast(value); + const uint64_t signBit = 1ull << (byteWidth * 8 - 1); + if (value & signBit) { + value |= ~((1ull << (byteWidth * 8)) - 1); + } + return static_cast(value); +} + +bool isUnsignedName(const std::string& name) { + static const char* kUnsigned[] = {"non_det_unsigned", "non_det_uint", + "non_det_ulong", "non_det_ushort", + "non_det_uchar", "non_det_size_t", + "non_det_bool", "non_det_pointer"}; + for (const char* candidate : kUnsigned) { + if (name == candidate) return true; + } + return false; +} + +} // namespace + +std::vector readKtestFile(const std::string& path) { + std::vector objects; + std::ifstream in(path, std::ios::binary); + if (!in.is_open()) return objects; + + char magic[kMagicSize + 1] = {0}; + in.read(magic, kMagicSize); + if (in.gcount() != static_cast(kMagicSize)) return objects; + if (std::memcmp(magic, kMagicKtest, kMagicSize) != 0 && + std::memcmp(magic, kMagicBout, kMagicSize) != 0) { + return objects; + } + + uint32_t version = 0; + if (!readBigEndian32(in, &version)) return objects; + + uint32_t numArgs = 0; + if (!readBigEndian32(in, &numArgs)) return objects; + if (numArgs > kMaxReasonableLength) return objects; + for (uint32_t i = 0; i < numArgs; ++i) { + if (!skipBlock(in)) return objects; + } + + if (version >= 2) { + uint32_t symArgvs = 0; + uint32_t symArgvLen = 0; + if (!readBigEndian32(in, &symArgvs)) return objects; + if (!readBigEndian32(in, &symArgvLen)) return objects; + } + + uint32_t numObjects = 0; + if (!readBigEndian32(in, &numObjects)) return objects; + if (numObjects > kMaxReasonableLength) return objects; + + for (uint32_t i = 0; i < numObjects; ++i) { + uint32_t nameSize = 0; + if (!readBigEndian32(in, &nameSize)) break; + if (nameSize > kMaxReasonableLength) break; + std::string name(nameSize, '\0'); + in.read(name.data(), nameSize); + if (in.gcount() != static_cast(nameSize)) break; + + uint32_t dataSize = 0; + if (!readBigEndian32(in, &dataSize)) break; + if (dataSize > kMaxReasonableLength) break; + std::vector bytes(dataSize); + if (dataSize > 0) { + in.read(reinterpret_cast(bytes.data()), dataSize); + if (in.gcount() != static_cast(dataSize)) break; + } + + // A partial object is dropped, but the ones already read are kept: a + // .ktest truncated by a kill still describes a usable prefix of the path, + // and the same forgiving rule governs the CSV log. + objects.push_back(KtestObject{name, std::move(bytes)}); + } + + return objects; +} + +std::string decodeKtestObject(const KtestObject& object) { + if (object.bytes.empty()) return "0"; + + if (object.name == "non_det_double") { + double value = 0.0; + const size_t width = std::min(object.bytes.size(), sizeof(double)); + std::memcpy(&value, object.bytes.data(), width); + // %f-style, matching what the CSV log emits for a double, so a suite reads + // the same whichever source produced it. + std::string text = std::to_string(value); + return text; + } + + const uint64_t raw = leBytesToUnsigned(object.bytes); + if (isUnsignedName(object.name)) { + return std::to_string(raw); + } + // Signed by default, including for an unrecognised name. Getting the + // signedness wrong turns -1 into 4294967295, which a validator rejects as + // out of range far more visibly than it would a wrong-but-plausible value. + return std::to_string(signExtend(raw, object.bytes.size())); +} + +std::vector> readKtestVectors( + const std::string& kleeOutDir, size_t limit) { + std::vector> vectors; + std::error_code error; + if (!std::filesystem::is_directory(kleeOutDir, error)) return vectors; + + // Collected and sorted rather than taken in directory order: readdir order + // is filesystem-dependent, and KLEE's testNNNNNN numbering is the only + // stable ordering available. A suite numbered differently on each machine + // cannot be diffed or reproduced. + std::vector paths; + for (const auto& entry : + std::filesystem::directory_iterator(kleeOutDir, error)) { + if (entry.path().extension() != ".ktest") continue; + paths.push_back(entry.path().string()); + } + std::sort(paths.begin(), paths.end()); + + for (const std::string& path : paths) { + if (vectors.size() >= limit) break; + std::vector objects = readKtestFile(path); + if (objects.empty()) continue; + + std::vector inputs; + inputs.reserve(objects.size()); + for (const KtestObject& object : objects) { + inputs.push_back(decodeKtestObject(object)); + } + vectors.push_back(std::move(inputs)); + } + return vectors; +} + +} // namespace Map2Check diff --git a/modules/frontend/test_suite/ktest_reader.hpp b/modules/frontend/test_suite/ktest_reader.hpp new file mode 100644 index 000000000..9bbe74be9 --- /dev/null +++ b/modules/frontend/test_suite/ktest_reader.hpp @@ -0,0 +1,73 @@ +/** + * Copyright (C) 2014 - 2026 Map2Check tool + * This file is part of the Map2Check tool, and is made available under + * the terms of the GNU General Public License version 2. + * + * SPDX-License-Identifier: (GPL-2.0) + **/ + +/********************************************************************** + * Reading KLEE's own per-path test vectors. + * + * Cover-Error needs one test case, the violating one, and klee_log.csv holds + * it. Cover-Branches needs many, one per explored path -- and KLEE already + * computes exactly that: it writes klee-last/testNNNNNN.ktest per terminating + * path, recording each symbolic object in the order the program consumed it. + * + * Taking them from KLEE rather than having the runtime write its own logs is + * not a stylistic choice, it was measured. Making the instrumented program + * write one file per state turned a 1-second run that answered FAILED into a + * 100-second run that exhausted its budget and answered SUCCEEDED: every write + * is an external call KLEE executes concretely, and enough of them corrupt the + * result rather than merely slowing it. Reading .ktest files afterwards costs + * the search nothing, because the search is already over. + * + * The format is read directly rather than by shelling out to ktest-tool: the + * layout is nine lines of parsing, and a subprocess would put a Python + * interpreter on the path between a verification run and its test suite. + ***********************************************************************/ + +#ifndef MODULES_FRONTEND_TEST_SUITE_KTEST_READER_HPP_ +#define MODULES_FRONTEND_TEST_SUITE_KTEST_READER_HPP_ + +#include +#include +#include + +namespace Map2Check { + +/** One symbolic object: the name klee_make_symbolic was given, and its bytes. + * + * The name is where the type lives. A .ktest records name, size and bytes but + * no type, so a four-byte object could be an int, a float or half of a long; + * NonDetGeneratorKlee.c names them non_det_ precisely so this side can + * tell. That naming was broken until recently -- every object was called + * "non_det_#type", the macro's stringify operator having been written inside + * the string literal -- which is why decoding by name is worth asserting. */ +struct KtestObject { + std::string name; + std::vector bytes; +}; + +/** Parses one .ktest file. Empty on a file that is absent or not a ktest. */ +std::vector readKtestFile(const std::string& path); + +/** Renders one object as the decimal string a element carries. + * + * Decoding is driven by the object's NAME, falling back to its size when the + * name is unknown -- an unrecognised name is a new nondet type, not a reason + * to drop a test case. */ +std::string decodeKtestObject(const KtestObject& object); + +/** Every input vector KLEE recorded under `kleeOutDir`, one per .ktest. + * + * Ordered by file name, which is KLEE's own path numbering: stable across + * runs, so two runs over the same output number their test cases alike. + * `limit` bounds the suite. Vectors with no objects are dropped -- a path that + * read no input is not a test case. */ +std::vector> readKtestVectors( + const std::string& kleeOutDir, size_t limit); + +} // namespace Map2Check + +#endif // MODULES_FRONTEND_TEST_SUITE_KTEST_READER_HPP_ diff --git a/modules/frontend/utils/tools.hpp b/modules/frontend/utils/tools.hpp index 0096400c9..9cb683efa 100755 --- a/modules/frontend/utils/tools.hpp +++ b/modules/frontend/utils/tools.hpp @@ -69,6 +69,11 @@ constexpr char const* kleeBinary = "${MAP2CHECK_PATH}/bin/klee"; constexpr int killGracePeriod = 10; /** Path to generated klee log file (check MemoryUtils implementation) */ constexpr char const* kleeLogCSV = "klee_log.csv"; +/** KLEE's own output directory, relative to the scratch directory the run + * executes in. It is a symlink KLEE maintains to the latest klee-out-N, and it + * holds one testNNNNNN.ktest per explored path -- the input vectors a + * Cover-Branches suite is built from. */ +constexpr char const* kleeOutputDir = "klee-last"; /** Path to generated Correctness log file (check MemoryUtils implementation) */ constexpr char const* stateTrueLogCSV = "automata_list_log.st"; /** Path to generated Correctness log file (check MemoryUtils implementation) */ diff --git a/tests/integration/test_benchexec_toolinfo.py b/tests/integration/test_benchexec_toolinfo.py index bbb3b768b..bfea6f0ce 100644 --- a/tests/integration/test_benchexec_toolinfo.py +++ b/tests/integration/test_benchexec_toolinfo.py @@ -119,9 +119,18 @@ def main(): # --- unsupported inputs must be refused, not quietly mishandled ---------- # A tool-info that accepts cover-branches would produce runs that score # zero while looking like participation. Refusing makes BenchExec skip. - raises("cover-branches is refused rather than faked", - lambda: tool.cmdline("x", [], template.BaseTool2.Task.with_files( - [program], property_file=cover_branches), limits)) + # cover-branches used to be refused, because there was no source of + # per-path vectors. KLEE's .ktest output is now that source, so the module + # must accept it -- and a regression to the refusal would silently drop a + # whole competition category. + cmd_branches = tool.cmdline("/opt/m2c/map2check-testcomp-wrapper.py", [], + template.BaseTool2.Task.with_files( + [program], property_file=cover_branches), + limits) + check("cover-branches is accepted", cmd_branches[-1] == program) + check("cover-branches passes its own property file", + cmd_branches[cmd_branches.index("-p") + 1] == cover_branches) + raises("an SV-COMP CHECK property is refused by the Test-Comp module", lambda: tool.cmdline("x", [], template.BaseTool2.Task.with_files( [program], property_file=unreachability), limits)) diff --git a/tests/integration/test_cover_branches.sh b/tests/integration/test_cover_branches.sh new file mode 100755 index 000000000..6c20e6bf9 --- /dev/null +++ b/tests/integration/test_cover_branches.sh @@ -0,0 +1,126 @@ +#!/bin/bash +# test_cover_branches.sh -- the Cover-Branches suite, validated by TestCov. +# +# Cover-Error asks for one test case, the violating one. Cover-Branches asks +# for a suite that exercises as many branches as possible, and the difference +# is not a parameter -- it is a different source of input vectors. +# +# The vectors come from KLEE's own .ktest output, one per explored path. The +# alternative, having the instrumented program write a log per state, was tried +# and measured: it turned a 1-second run answering FALSE into a 100-second run +# that exhausted its budget and answered TRUE, because every write is an +# external call KLEE executes concretely. Reading .ktest files after the search +# has finished costs the search nothing. +# +# The assertion that matters is the last one. Well-formed XML proves nothing +# about coverage; only an independent validator can say whether the suite +# actually exercises the branches, and TestCov is that validator. + +set -u + +MAP2CHECK_DIR="${MAP2CHECK_PATH:-/workspace/install_e2e}" +MAP2CHECK="$MAP2CHECK_DIR/map2check" +TESTCOV="${TESTCOV:-testcov}" +BUDGET="${BUDGET:-120}" + +PASSED=0 +FAILED=0 +ok() { echo " PASS $1"; PASSED=$((PASSED+1)); } +fail() { echo " FAIL $1: $2"; FAILED=$((FAILED+1)); } + +WORK=$(mktemp -d) +trap 'rm -rf "$WORK"' EXIT + +cat > "$WORK/branches.prp" <<'EOF' +COVER( init(main()), FQL(COVER EDGES(@DECISIONEDGE)) ) +EOF + +# Six decision edges over two independent inputs. A single-test-case suite can +# reach at most half of them, so a tool that quietly fell back to the +# Cover-Error path would show up as ~50% coverage rather than as an error. +cat > "$WORK/branches.c" <<'EOF' +extern int __VERIFIER_nondet_int(void); +int main(void) { + int x = __VERIFIER_nondet_int(); + int y = __VERIFIER_nondet_int(); + int r = 0; + if (x > 10) { r += 1; } else { r += 2; } + if (y < -5) { r += 4; } else { r += 8; } + if (x == y) { r += 16; } + return r; +} +EOF + +echo "=== Cover-Branches suite ===" + +if ! command -v "$TESTCOV" >/dev/null 2>&1; then + echo " SKIP: $TESTCOV not installed (pip3 install testcov)" + exit 0 +fi + +( cd "$WORK" && MAP2CHECK_PATH="$MAP2CHECK_DIR" timeout -k 10 $((BUDGET + 60)) \ + "$MAP2CHECK" --nondet-generator symex --generate-test-suite --cover-branches \ + --property-file branches.prp --architecture 64bit \ + --timeout "$BUDGET" branches.c ) > "$WORK/m2c.log" 2>&1 + +n_tests=$(ls "$WORK/test-suite"/testcase-*.xml 2>/dev/null | wc -l) + +# More than one, because that is the whole point: Cover-Error emits exactly one. +if [ "$n_tests" -gt 1 ]; then + ok "the suite holds more than one test case ($n_tests)" +else + fail "test case count" "got $n_tests, expected several (one per KLEE path)" + tail -5 "$WORK/m2c.log" | sed 's/^/ /' + echo " Results: $PASSED passed, $FAILED failed" + exit 1 +fi + +# Distinct vectors. Identical ones would mean the same path was recorded many +# times, which inflates the suite while covering nothing extra. +vectors=$(for f in "$WORK/test-suite"/testcase-*.xml; do + sed -n 's:.*\(.*\).*:\1:p' "$f" | tr '\n' ',' ; echo + done) +n_distinct=$(printf '%s\n' "$vectors" | sort -u | grep -c .) +if [ "$n_distinct" -eq "$n_tests" ]; then + ok "every test case is a distinct input vector ($n_distinct)" +else + fail "vector distinctness" "$n_distinct distinct out of $n_tests" +fi + +# Cover-Branches test cases are paths, not violations. +if grep -q 'coversError="true"' "$WORK/test-suite"/testcase-*.xml 2>/dev/null; then + fail "coversError" "a branch test case claims to cover the error" +else + ok "no branch test case claims coversError" +fi + +if grep -q "@DECISIONEDGE" "$WORK/test-suite/metadata.xml"; then + ok "metadata records the branch-coverage specification" +else + fail "specification" "metadata does not carry the cover-branches property" +fi + +# The one that cannot be faked. +( cd "$WORK/test-suite" && zip -q -r ../suite.zip . ) +( cd "$WORK" && timeout -k 10 300 "$TESTCOV" --test-suite suite.zip \ + --no-isolation -64 --goal branches.prp branches.c ) > "$WORK/tc.log" 2>&1 + +coverage=$(grep -oE '^Coverage: [0-9.]+' "$WORK/tc.log" | tail -1 | awk '{print $2}') +if [ -z "$coverage" ]; then + fail "TestCov ran" "no Coverage line -- see below" + tail -15 "$WORK/tc.log" | sed 's/^/ /' +else + # 100% is what this program should reach; the threshold is set below it so a + # small change in KLEE's path ordering does not turn into a red build, while + # a fallback to a one-case suite (~50%) still fails. + reached=$(python3 -c "print(1 if float('$coverage') >= 80.0 else 0)") + if [ "$reached" = "1" ]; then + ok "TestCov measures ${coverage}% branch coverage" + else + fail "branch coverage" "only ${coverage}%, expected >= 80%" + fi +fi + +echo " ---" +echo " Results: $PASSED passed, $FAILED failed" +[ "$FAILED" -eq 0 ] || exit 1 diff --git a/tests/testcomp/build_corpus.py b/tests/testcomp/build_corpus.py new file mode 100755 index 000000000..ab219940d --- /dev/null +++ b/tests/testcomp/build_corpus.py @@ -0,0 +1,167 @@ +#!/usr/bin/env python3 +"""Builds a stratified Test-Comp corpus manifest from sv-benchmarks. + +Stratified rather than random, and by SUBCATEGORY rather than over the whole +pool, for the same reason the Juliet runs are sharded by CWE: a uniform sample +of 33k tasks is dominated by whichever families happen to be largest (Floats +and ECA alone are 2.6k), and a result computed over it says more about the +benchmark's shape than about the tool. Fixing a quota per subcategory makes +"Map2Check does badly on Recursive" a statement the numbers can support. + +Selection inside a subcategory is deterministic: tasks are sorted by path and +taken at an even stride. No RNG, no seed to record, and re-running the script +on the same checkout produces the same manifest -- which is what makes two runs +comparable at all. + +Usage: + build_corpus.py --property cover-error --per-category 40 --out manifest.tsv + build_corpus.py --property cover-branches --per-category 40 --out manifest.tsv +""" + +import argparse +import os +import sys + +HERE = os.path.dirname(os.path.abspath(__file__)) +BENCH = os.path.join(HERE, "bench", "sv-benchmarks", "c") + +# The subcategories, and why these. They span the shapes a test generator is +# actually challenged by -- array reasoning, bit-level operations, control-flow +# depth, heap shape, loop invariants, recursion, floating point, event-condition +# -action tables, feature interaction -- rather than the largest directories. +# Concurrency and the Termination sets are excluded: Map2Check declares no +# support for either, and running them would report zeros that mean nothing. +CATEGORIES = [ + "Arrays", + "BitVectors", + "ControlFlow", + "Heap", + "Loops", + "Recursive", + "Sequentialized", + "Floats", + "ECA", + "ProductLines", + "XCSP", + "LinkedLists", +] + +PROPERTY_FILE = { + "cover-error": "coverage-error-call.prp", + "cover-branches": "coverage-branches.prp", +} + + +def expand_set(name): + """Every task definition a .set file names, in sorted order.""" + import glob + + paths = [] + set_path = os.path.join(BENCH, name + ".set") + with open(set_path) as handle: + for line in handle: + line = line.strip() + if not line or line.startswith("#"): + continue + paths.extend(glob.glob(os.path.join(BENCH, line))) + return sorted(set(paths)) + + +def task_info(yml_path, wanted_property): + """Reads the fields the runner needs, or None if the task does not apply. + + Parsed by hand rather than with PyYAML: the fields wanted are three flat + scalars and one list of maps, the format is stable and version-stamped, and + a dependency here would have to exist inside the runner container too. + """ + input_file = None + data_model = "ILP32" # the sv-benchmarks default when unstated + has_property = False + expected = "" + in_properties = False + current_property = None + + with open(yml_path, errors="replace") as handle: + for raw in handle: + line = raw.rstrip("\n") + stripped = line.strip() + if stripped.startswith("input_files:"): + input_file = stripped.split(":", 1)[1].strip().strip("'\"") + elif stripped.startswith("data_model:"): + data_model = stripped.split(":", 1)[1].strip().strip("'\"") + elif stripped.startswith("properties:"): + in_properties = True + elif in_properties and stripped.startswith("- property_file:"): + current_property = stripped.split(":", 1)[1].strip() + elif in_properties and stripped.startswith("expected_verdict:"): + if current_property and current_property.endswith( + "unreach-call.prp"): + expected = stripped.split(":", 1)[1].strip() + if current_property and current_property.endswith( + PROPERTY_FILE[wanted_property]): + has_property = True + + if input_file is None or not has_property: + return None + # A task definition may name several input files; Map2Check takes one. + if " " in input_file or "," in input_file: + return None + program = os.path.join(os.path.dirname(yml_path), input_file) + if not os.path.isfile(program): + return None + return program, data_model, expected + + +def stride_sample(items, quota): + """Even stride, not the first N. + + Task names inside a family are ordered, and the first N of a sorted list is + usually N variations of one program. A stride spreads the sample across the + family's whole range at no cost in reproducibility. + """ + if quota >= len(items): + return items + if quota <= 0: + return [] + step = len(items) / float(quota) + return [items[int(i * step)] for i in range(quota)] + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("--property", choices=sorted(PROPERTY_FILE), + required=True) + parser.add_argument("--per-category", type=int, default=40) + parser.add_argument("--out", required=True) + args = parser.parse_args() + + if not os.path.isdir(BENCH): + sys.exit("sv-benchmarks not found at %s -- run fetch-benchmarks.sh" + % BENCH) + + rows = [] + summary = [] + for category in CATEGORIES: + applicable = [] + for yml in expand_set(category): + info = task_info(yml, args.property) + if info is not None: + applicable.append((yml,) + info) + chosen = stride_sample(applicable, args.per_category) + summary.append((category, len(applicable), len(chosen))) + for yml, program, data_model, expected in chosen: + rows.append((category, os.path.relpath(program, BENCH), data_model, + expected or "unknown")) + + with open(args.out, "w") as handle: + handle.write("# category\tprogram\tdata_model\texpected_unreach\n") + for row in rows: + handle.write("\t".join(row) + "\n") + + for category, available, taken in summary: + print(" %-16s %5d applicable, %3d taken" % (category, available, taken)) + print(" %-16s %5d tasks written to %s" % ("TOTAL", len(rows), args.out)) + + +if __name__ == "__main__": + main() diff --git a/tests/testcomp/corpus/cover-branches.tsv b/tests/testcomp/corpus/cover-branches.tsv new file mode 100644 index 000000000..9bef7c250 --- /dev/null +++ b/tests/testcomp/corpus/cover-branches.tsv @@ -0,0 +1,481 @@ +# category program data_model expected_unreach +Arrays array-cav19/array_init_nondet_vars.c ILP32 true +Arrays array-crafted/bAnd4.i ILP32 true +Arrays array-crafted/mapsum1.i ILP32 true +Arrays array-crafted/zero_sum3.c ILP32 true +Arrays array-crafted/zero_sum_m2.c ILP32 true +Arrays array-examples/sanfoundry_24-2.i ILP32 unknown +Arrays array-examples/standard_copy1_ground-1.i ILP32 true +Arrays array-examples/standard_copy7_ground-1.i ILP32 false +Arrays array-examples/standard_palindrome_ground.i ILP32 true +Arrays array-examples/standard_strcmp_ground.i ILP32 true +Arrays array-examples/standard_two_index_09.i ILP32 true +Arrays array-fpi/brs5_overflow.c ILP32 unknown +Arrays array-fpi/eqn1f.c ILP32 false +Arrays array-fpi/ifeqn1f.c ILP32 false +Arrays array-fpi/indp1f.c ILP32 false +Arrays array-fpi/mods.c ILP32 true +Arrays array-fpi/ncomp.c ILP32 true +Arrays array-fpi/res2.c ILP32 true +Arrays array-fpi/s2if.c ILP32 true +Arrays array-fpi/s42iff.c ILP32 false +Arrays array-fpi/s5if.c ILP32 true +Arrays array-fpi/sina3.c ILP32 true +Arrays array-fpi/ss2.c ILP32 true +Arrays array-industry-pattern/array_of_struct_ptr_monotonic.i ILP32 true +Arrays array-lopstr16/single_elem_safe.i ILP32 true +Arrays array-memsafety/count_down_unsafe.i ILP32 unknown +Arrays array-memsafety/cstrncpy-alloca-2.i ILP32 unknown +Arrays array-memsafety/mult_array-alloca-1.i ILP32 unknown +Arrays array-memsafety/openbsd_cstrlcpy-alloca-1.i ILP32 unknown +Arrays array-memsafety/selectionsort-alloca-1.i ILP32 unknown +Arrays array-multidimensional/copy-partial-3-u.c ILP32 true +Arrays array-multidimensional/rev-2-n-u.c ILP32 true +Arrays array-patterns/array19_pattern.c ILP32 true +Arrays array-patterns/array29_pattern.c ILP32 true +Arrays array-patterns/array7_pattern.c ILP32 true +Arrays array-programs/partial_mod_count_limited_base.c ILP32 true +Arrays array-tiling/pnr2.c ILP32 true +Arrays array-tiling/rewnif.c ILP32 true +Arrays reducercommutativity/avg60-1.i ILP32 true +Arrays reducercommutativity/rangesum60.i ILP32 false +BitVectors bitvector-loops/diamond_2-1.c ILP32 false +BitVectors bitvector-loops/verisec_sendmail_tTflag_arr_one_loop.c ILP32 false +BitVectors bitvector/byte_add-1.i ILP32 false +BitVectors bitvector/byte_add-2.i ILP32 unknown +BitVectors bitvector/byte_add_1-2.i ILP32 unknown +BitVectors bitvector/byte_add_2-1.i ILP32 unknown +BitVectors bitvector/gcd_1.c ILP32 true +BitVectors bitvector/gcd_2.c ILP32 true +BitVectors bitvector/interleave_bits.i ILP32 true +BitVectors bitvector/jain_1-2.c ILP32 unknown +BitVectors bitvector/jain_4-1.c ILP32 unknown +BitVectors bitvector/jain_7-1.c ILP32 unknown +BitVectors bitvector/modulus-1.i ILP32 unknown +BitVectors bitvector/num_conversion_2.c ILP32 true +BitVectors bitvector/parity.i ILP32 true +BitVectors bitvector/s3_clnt_1.BV.c.cil-1a.c ILP32 unknown +BitVectors bitvector/s3_clnt_1.BV.c.cil-2.c ILP32 unknown +BitVectors bitvector/s3_clnt_2.BV.c.cil-1.c ILP32 unknown +BitVectors bitvector/s3_clnt_2.BV.c.cil-1a.c ILP32 true +BitVectors bitvector/s3_clnt_2.BV.c.cil-2a.c ILP32 false +BitVectors bitvector/s3_clnt_3.BV.c.cil-1a.c ILP32 true +BitVectors bitvector/s3_clnt_3.BV.c.cil-2.c ILP32 unknown +BitVectors bitvector/s3_srvr_1.BV.c.cil.c ILP32 unknown +BitVectors bitvector/s3_srvr_1_alt.BV.c.cil.c ILP32 unknown +BitVectors bitvector/s3_srvr_1a_alt.BV.c.cil.c ILP32 unknown +BitVectors bitvector/s3_srvr_2.BV.c.cil.c ILP32 unknown +BitVectors bitvector/s3_srvr_3.BV.c.cil.c ILP32 unknown +BitVectors bitvector/s3_srvr_3_alt.BV.c.cil.c ILP32 unknown +BitVectors bitvector/s3_srvr_3a_alt.BV.c.cil.c ILP32 true +BitVectors bitvector/soft_float_1-1.c.cil.c ILP32 unknown +BitVectors bitvector/soft_float_1-2a.c.cil.c ILP32 true +BitVectors bitvector/soft_float_1-3a.c.cil.c ILP32 false +BitVectors bitvector/soft_float_2.c.cil.c ILP32 unknown +BitVectors bitvector/soft_float_3.c.cil.c ILP32 unknown +BitVectors bitvector/soft_float_3a.c.cil.c ILP32 true +BitVectors bitvector/soft_float_4-2.c.cil.c ILP32 unknown +BitVectors bitvector/soft_float_4-2a.c.cil.c ILP32 true +BitVectors bitvector/soft_float_4-3a.c.cil.c ILP32 false +BitVectors bitvector/soft_float_5.c.cil.c ILP32 unknown +BitVectors bitvector/sum02-1.c ILP32 false +ControlFlow infeasible-control-flow/conflict_branch1.c ILP32 true +ControlFlow infeasible-control-flow/conflict_branch3.c ILP32 true +ControlFlow infeasible-control-flow/unreach_branch2.c ILP32 true +ControlFlow infeasible-control-flow/unreach_branch5.c ILP32 true +ControlFlow memory-model/4SB.i ILP32 false +ControlFlow ntdrivers-simplified/diskperf_simpl1.cil.c ILP32 unknown +ControlFlow ntdrivers-simplified/floppy_simpl3.cil-2.c ILP32 unknown +ControlFlow ntdrivers-simplified/kbfiltr_simpl1.cil.c ILP32 unknown +ControlFlow ntdrivers-simplified/kbfiltr_simpl2.cil-2.c ILP32 unknown +ControlFlow ntdrivers/diskperf.i.cil-1.c ILP32 true +ControlFlow ntdrivers/floppy.i.cil-1.c ILP32 false +ControlFlow ntdrivers/floppy.i.cil-3.c ILP32 true +ControlFlow ntdrivers/kbfiltr.i.cil-2.c ILP32 false +ControlFlow ntdrivers/parport.i.cil-2.c ILP32 true +ControlFlow openssl-simplified/s3_clnt_2.cil-1.c ILP32 unknown +ControlFlow openssl-simplified/s3_clnt_3.cil-1.c ILP32 unknown +ControlFlow openssl-simplified/s3_clnt_4.cil-1.c ILP32 unknown +ControlFlow openssl-simplified/s3_srvr_1.cil-1.c ILP32 unknown +ControlFlow openssl-simplified/s3_srvr_11.cil.c ILP32 unknown +ControlFlow openssl-simplified/s3_srvr_13.cil.c ILP32 unknown +ControlFlow openssl-simplified/s3_srvr_2.cil-1.c ILP32 unknown +ControlFlow openssl-simplified/s3_srvr_3.cil.c ILP32 unknown +ControlFlow openssl-simplified/s3_srvr_6.cil-1.c ILP32 unknown +ControlFlow openssl-simplified/s3_srvr_8.cil.c ILP32 unknown +ControlFlow openssl/s3_clnt.blast.01.i.cil-2.c ILP32 unknown +ControlFlow openssl/s3_clnt.blast.03.i.cil-1.c ILP32 unknown +ControlFlow openssl/s3_clnt.blast.04.i.cil-1.c ILP32 unknown +ControlFlow openssl/s3_srvr.blast.01.i.cil-2.c ILP32 unknown +ControlFlow openssl/s3_srvr.blast.02.i.cil-2.c ILP32 unknown +ControlFlow openssl/s3_srvr.blast.06.i.cil-1.c ILP32 unknown +ControlFlow openssl/s3_srvr.blast.07.i.cil-1.c ILP32 unknown +ControlFlow openssl/s3_srvr.blast.08.i.cil-1.c ILP32 unknown +ControlFlow openssl/s3_srvr.blast.09.i.cil-2.c ILP32 unknown +ControlFlow openssl/s3_srvr.blast.10.i.cil-2.c ILP32 unknown +ControlFlow openssl/s3_srvr.blast.12.i.cil-1.c ILP32 unknown +ControlFlow openssl/s3_srvr.blast.13.i.cil-1.c ILP32 unknown +ControlFlow openssl/s3_srvr.blast.14.i.cil-2.c ILP32 unknown +ControlFlow openssl/s3_srvr.blast.15.i.cil-2.c ILP32 unknown +ControlFlow unsignedintegeroverflow-sas23/bilinear_interpolation.i ILP32 true +ControlFlow unsignedintegeroverflow-sas23/distance_through_overflow.i ILP32 true +Heap heap-data/calendar.i ILP32 true +Heap heap-data/shared_mem1.i ILP32 true +Heap ldv-memsafety/memleaks_test11.i ILP32 unknown +Heap ldv-memsafety/memleaks_test13_2.i ILP32 unknown +Heap ldv-memsafety/memleaks_test16.i ILP32 unknown +Heap ldv-memsafety/memleaks_test17_3.i ILP32 unknown +Heap ldv-memsafety/memleaks_test20-1.i ILP32 unknown +Heap ldv-memsafety/memleaks_test22_2-2.i ILP32 unknown +Heap ldv-memsafety/memleaks_test23_3.i ILP32 unknown +Heap ldv-memsafety/memleaks_test5_1.i ILP32 unknown +Heap ldv-memsafety/memleaks_test7-2.i ILP32 unknown +Heap ldv-regression/alias_of_return_2.c_1.i ILP32 true +Heap ldv-regression/rule60_list2.i ILP32 true +Heap ldv-regression/test21-1.c ILP32 true +Heap ldv-regression/test24-1.c ILP32 true +Heap ldv-regression/test28-2.c ILP32 true +Heap ldv-sets/test_mutex_double_unlock.i ILP32 false +Heap list-ext-properties/list-ext.i ILP32 false +Heap list-ext-properties/test-0158_1-1.i ILP32 unknown +Heap list-ext-properties/test-0504_1.i ILP32 unknown +Heap memsafety-cve/MojoJson.i ILP32 unknown +Heap memsafety-cve/dlt-daemon.i ILP32 unknown +Heap memsafety-cve/gpac_10.i ILP32 unknown +Heap memsafety-cve/gpac_18.i ILP32 unknown +Heap memsafety-cve/gpac_8.i ILP32 unknown +Heap memsafety-cve/json-parserFixed.i ILP32 unknown +Heap memsafety-cve/libpe.i ILP32 unknown +Heap memsafety-cve/minizip-ngFixed.i ILP32 unknown +Heap memsafety-cve/nanopb.i ILP32 unknown +Heap memsafety-cve/openSCFixed.i ILP32 unknown +Heap memsafety-cve/picotcpFixed.i ILP32 unknown +Heap memsafety-cve/radare2_2Fixed.i ILP32 unknown +Heap memsafety-cve/sofia-sip_1.i ILP32 unknown +Heap memsafety-cve/zchunkFixed.i ILP32 unknown +Heap memsafety-ext/tree_of_cslls.i ILP32 unknown +Heap memsafety-ext2/optional_data_creation_test04-1.i ILP32 unknown +Heap memsafety/global-atexit-1.i ILP32 unknown +Heap memsafety/lockfree-3.2.i ILP32 unknown +Heap memsafety/test-0220.i ILP32 unknown +Heap memsafety/test-0235-2.i ILP32 unknown +Loops loop-acceleration/array3.i ILP32 unknown +Loops loop-crafted/simple_array_index_value_4.i.v+nlh-reducer.c ILP32 true +Loops loop-invgen/apache-get-tag.i.p+lhb-reducer.c ILP32 true +Loops loop-invgen/sendmail-close-angle.i ILP32 true +Loops loop-lit/gsv2008.i ILP32 true +Loops loop-simple/nested_2.c ILP32 true +Loops loop-zilu/benchmark13_conjunctive.i ILP32 unknown +Loops loop-zilu/benchmark31_disjunctive.i ILP32 unknown +Loops loop-zilu/benchmark49_linear.i ILP32 unknown +Loops loops-crafted-1/nested3-1_abstracted.c ILP32 true +Loops loops/array-2.c ILP32 false +Loops loops/n.c24.i ILP32 false +Loops loops/terminator_02-2_abstracted.c ILP32 true +Loops nla-digbench-scaling/bresenham-ll_unwindbound100.c ILP32 false +Loops nla-digbench-scaling/cohencu-ll_valuebound10.c ILP32 true +Loops nla-digbench-scaling/cohendiv-ll_valuebound50.c ILP32 true +Loops nla-digbench-scaling/divbin2_unwindbound20.i ILP32 true +Loops nla-digbench-scaling/divbin_valuebound100.i ILP32 true +Loops nla-digbench-scaling/egcd2-ll_unwindbound1.c ILP32 false +Loops nla-digbench-scaling/egcd3-ll_unwindbound5.c ILP32 false +Loops nla-digbench-scaling/fermat1-ll_valuebound20.c ILP32 true +Loops nla-digbench-scaling/freire1_valuebound100.c ILP32 true +Loops nla-digbench-scaling/geo1-ll2_valuebound100.c ILP32 unknown +Loops nla-digbench-scaling/geo1-u2_unwindbound100.c ILP32 unknown +Loops nla-digbench-scaling/geo1-u_valuebound20.c ILP32 false +Loops nla-digbench-scaling/geo2-ll_unwindbound50.c ILP32 true +Loops nla-digbench-scaling/geo3-ll_unwindbound10.c ILP32 true +Loops nla-digbench-scaling/hard-ll_valuebound1.c ILP32 true +Loops nla-digbench-scaling/hard-u_valuebound5.c ILP32 true +Loops nla-digbench-scaling/knuth_unwindbound2.i ILP32 true +Loops nla-digbench-scaling/lcm1_valuebound2.c ILP32 true +Loops nla-digbench-scaling/mannadiv_unwindbound10.c ILP32 false +Loops nla-digbench-scaling/prod4br-ll_valuebound50.c ILP32 true +Loops nla-digbench-scaling/ps2-ll_unwindbound20.c ILP32 true +Loops nla-digbench-scaling/ps3-ll_valuebound2.c ILP32 true +Loops nla-digbench-scaling/ps5-ll_unwindbound10.c ILP32 false +Loops nla-digbench-scaling/ps6-ll_unwindbound50.c ILP32 false +Loops nla-digbench-scaling/sqrt1-ll_valuebound20.c ILP32 true +Loops nla-digbench/fermat1-ll.c ILP32 true +Loops nla-digbench/knuth.i ILP32 true +Recursive fuzzle-programs/01_fuzzle_30x30.c LP64 false +Recursive fuzzle-programs/04_fuzzle_60x60.c LP64 false +Recursive fuzzle-programs/07_fuzzle_50x50_25-cycle.c LP64 false +Recursive fuzzle-programs/10_fuzzle_50x50_CVE-2016-4487.c LP64 false +Recursive fuzzle-programs/14_fuzzle_50x50_CVE-2016-6131.c LP64 false +Recursive recursified_loop-crafted/recursified_simple_array_index_value_4.i.v+lhb-reducer.c ILP32 unknown +Recursive recursified_loop-invariants/recursified_linear-inequality-inv-a.c ILP32 unknown +Recursive recursified_loop-simple/recursified_nested_1b.c ILP32 unknown +Recursive recursified_loop-simple/recursified_nested_5.c ILP32 unknown +Recursive recursified_nla-digbench/recursified_bresenham.c ILP32 unknown +Recursive recursified_nla-digbench/recursified_cohendiv-ll.c ILP32 unknown +Recursive recursified_nla-digbench/recursified_divbin.i ILP32 unknown +Recursive recursified_nla-digbench/recursified_egcd.c ILP32 unknown +Recursive recursified_nla-digbench/recursified_egcd3-ll.c ILP32 unknown +Recursive recursified_nla-digbench/recursified_fermat1.c ILP32 unknown +Recursive recursified_nla-digbench/recursified_geo1-ll2.c ILP32 unknown +Recursive recursified_nla-digbench/recursified_geo1.c ILP32 unknown +Recursive recursified_nla-digbench/recursified_geo2.c ILP32 unknown +Recursive recursified_nla-digbench/recursified_geo3.c ILP32 unknown +Recursive recursified_nla-digbench/recursified_hard2.c ILP32 unknown +Recursive recursified_nla-digbench/recursified_lcm2.c ILP32 unknown +Recursive recursified_nla-digbench/recursified_prod4br.c ILP32 unknown +Recursive recursified_nla-digbench/recursified_ps2.c ILP32 unknown +Recursive recursified_nla-digbench/recursified_ps4-ll.c ILP32 unknown +Recursive recursified_nla-digbench/recursified_ps5.c ILP32 unknown +Recursive recursified_nla-digbench/recursified_sqrt1-ll.c ILP32 unknown +Recursive recursive-simple/id2_b3_o5.c ILP32 true +Recursive recursive-simple/id_b3_o2-1.c ILP32 unknown +Recursive recursive-simple/id_b3_o5-2.c ILP32 true +Recursive recursive-simple/id_o10.c ILP32 false +Recursive recursive-simple/id_o200.c ILP32 false +Recursive recursive-simple/sum_non_eq-1.c ILP32 unknown +Recursive recursive/Ackermann01-2.c ILP32 true +Recursive recursive/Addition01-2.c ILP32 true +Recursive recursive/Addition03-1.c ILP32 true +Recursive recursive/EvenOdd01-1.c ILP32 true +Recursive recursive/Fibonacci01-1.c ILP32 true +Recursive recursive/Fibonacci05-overflow.c ILP32 unknown +Recursive recursive/McCarthy91-2.c ILP32 true +Recursive recursive/gcd02.c ILP32 true +Sequentialized seq-mthreaded/pals_STARTPALS_ActiveStandby.1.ufo.BOUNDED-10.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_STARTPALS_ActiveStandby.4_2.ufo.BOUNDED-10.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_STARTPALS_Triplicated.1.ufo.BOUNDED-10.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.3.1.ufo.BOUNDED-6.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.3.2.ufo.UNBOUNDED.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.3.4.ufo.BOUNDED-6.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.4.1.ufo.BOUNDED-8.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.4.2.ufo.UNBOUNDED.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.4.4.ufo.BOUNDED-8.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.5.1.ufo.BOUNDED-10.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.5.3.ufo.BOUNDED-10.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.5.ufo.BOUNDED-10.pals.c ILP32 true +Sequentialized seq-mthreaded/pals_lcr-var-start-time.3.2.ufo.BOUNDED-6.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr-var-start-time.4.2.ufo.BOUNDED-8.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr-var-start-time.5.1.ufo.BOUNDED-10.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr-var-start-time.6.1.ufo.BOUNDED-12.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr.3.1.ufo.BOUNDED-6.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr.3_overflow.ufo.UNBOUNDED.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr.4_overflow.ufo.UNBOUNDED.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr.5_overflow.ufo.UNBOUNDED.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr.6_overflow.ufo.UNBOUNDED.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr.7.ufo.BOUNDED-14.pals.c ILP32 true +Sequentialized seq-mthreaded/pals_lcr.8.ufo.BOUNDED-16.pals.c ILP32 true +Sequentialized seq-mthreaded/pals_opt-floodmax.3.2.ufo.BOUNDED-6.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.3.3.ufo.BOUNDED-6.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.3.ufo.BOUNDED-6.pals.c ILP32 true +Sequentialized seq-mthreaded/pals_opt-floodmax.4.2.ufo.BOUNDED-8.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.4.3.ufo.BOUNDED-8.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.4.ufo.BOUNDED-8.pals.c ILP32 true +Sequentialized seq-mthreaded/pals_opt-floodmax.5.2.ufo.BOUNDED-10.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.5.4.ufo.BOUNDED-10.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.5_overflow.ufo.UNBOUNDED.pals.c ILP32 false +Sequentialized seq-pthread/cs_fib-1.i ILP32 true +Sequentialized seq-pthread/cs_fib_longer-2.i ILP32 false +Sequentialized seq-pthread/cs_lazy.i ILP32 unknown +Sequentialized seq-pthread/cs_queue-2.i ILP32 false +Sequentialized seq-pthread/cs_stack-1.i ILP32 true +Sequentialized seq-pthread/cs_stateful-1.i ILP32 false +Sequentialized seq-pthread/cs_szymanski.i ILP32 true +Sequentialized systemc/token_ring.01.cil-2.c ILP32 unknown +Floats float-benchs/arctan_Pade.c ILP32 true +Floats float-benchs/image_filter.c ILP32 true +Floats float-benchs/nan_float_range.c ILP32 true +Floats float-benchs/zonotope_3.c ILP32 true +Floats float-newlib/double_req_bl_0832.c ILP32 true +Floats float-newlib/double_req_bl_1251b.c ILP32 true +Floats float-newlib/float_req_bl_0870b.c ILP32 true +Floats float-newlib/float_req_bl_1231.c ILP32 true +Floats floats-cbmc-regression/float6.c ILP32 true +Floats floats-cdfpl/newton_3_4.i ILP32 true +Floats floats-esbmc-regression/floor_nondet.i ILP32 true +Floats loop-floats-scientific-comp/loop1-1.c ILP32 true +Floats neural-networks/cartpole_22_safe.c-amalgamation.i ILP32 true +Floats neural-networks/cartpole_41_safe.c-amalgamation.i ILP32 true +Floats neural-networks/cartpole_61_safe.c-amalgamation.i ILP32 true +Floats neural-networks/cartpole_80_safe.c-amalgamation.i ILP32 true +Floats neural-networks/cartpole_9_safe.c-amalgamation.i ILP32 true +Floats neural-networks/dubinsrejoin_21_safe.c-amalgamation.i ILP32 true +Floats neural-networks/dubinsrejoin_40_safe.c-amalgamation.i ILP32 true +Floats neural-networks/dubinsrejoin_5_safe.c-amalgamation.i ILP32 true +Floats neural-networks/dubinsrejoin_7_safe.c-amalgamation.i ILP32 true +Floats neural-networks/elu_2_safe.c-amalgamation.i ILP32 true +Floats neural-networks/gcas_3_safe.c-amalgamation.i ILP32 true +Floats neural-networks/logistic_1_safe.c-amalgamation.i ILP32 true +Floats neural-networks/lunarlander_22_unsafe.c-amalgamation.i ILP32 false +Floats neural-networks/lunarlander_40_unsafe.c-amalgamation.i ILP32 false +Floats neural-networks/lunarlander_5_unsafe.c-amalgamation.i ILP32 false +Floats neural-networks/lunarlander_7_unsafe.c-amalgamation.i ILP32 false +Floats neural-networks/lunarlander_98_unsafe.c-amalgamation.i ILP32 false +Floats neural-networks/poly_16_16_16_16_thresh_0_safe.c-amalgamation.i ILP32 true +Floats neural-networks/poly_256_thresh_4_unsafe.c-amalgamation.i ILP32 false +Floats neural-networks/poly_4_4_4_4_thresh_1_safe.c-amalgamation.i ILP32 true +Floats neural-networks/poly_64_64_thresh_4_unsafe.c-amalgamation.i ILP32 false +Floats neural-networks/robot_3_safe.c-amalgamation.i ILP32 true +Floats neural-networks/softmax_3_unsafe.c-amalgamation.i ILP32 false +Floats neural-networks/softsign_w16_r2_case_0_safe.c-amalgamation.i ILP32 true +Floats neural-networks/softsign_w4_r4_case_1_unsafe.c-amalgamation.i ILP32 false +Floats neural-networks/sqrt_5_safe.c-amalgamation.i ILP32 true +Floats neural-networks/tanh_w32_r1_case_1_safe.c-amalgamation.i ILP32 true +Floats neural-networks/tanh_w64_r4_case_0_safe.c-amalgamation.i ILP32 true +ECA eca-programs/Problem101_label00.c ILP32 false +ECA eca-programs/Problem102_label02.c ILP32 false +ECA eca-programs/Problem102_label28.c ILP32 false +ECA eca-programs/Problem102_label55.c ILP32 false +ECA eca-programs/Problem103_label59.c ILP32 false +ECA eca-rers2012/Problem03_label26.c ILP32 false +ECA eca-rers2012/Problem03_label52.c ILP32 false +ECA eca-rers2012/Problem04_label26.c ILP32 false +ECA eca-rers2012/Problem05_label04.c ILP32 true +ECA eca-rers2012/Problem05_label31.c ILP32 true +ECA eca-rers2012/Problem05_label57.c ILP32 false +ECA eca-rers2012/Problem06_label24.c ILP32 false +ECA eca-rers2012/Problem06_label50.c ILP32 true +ECA eca-rers2012/Problem07_label17.c ILP32 true +ECA eca-rers2012/Problem07_label43.c ILP32 true +ECA eca-rers2012/Problem08_label10.c ILP32 false +ECA eca-rers2012/Problem08_label36.c ILP32 true +ECA eca-rers2012/Problem09_label03.c ILP32 false +ECA eca-rers2012/Problem09_label29.c ILP32 true +ECA eca-rers2012/Problem09_label56.c ILP32 false +ECA eca-rers2012/Problem10_label22.c ILP32 true +ECA eca-rers2012/Problem10_label48.c ILP32 false +ECA eca-rers2012/Problem11_label15.c ILP32 false +ECA eca-rers2012/Problem11_label41.c ILP32 true +ECA eca-rers2012/Problem12_label08.c ILP32 false +ECA eca-rers2012/Problem12_label34.c ILP32 false +ECA eca-rers2012/Problem13_label01.c ILP32 true +ECA eca-rers2012/Problem13_label27.c ILP32 true +ECA eca-rers2012/Problem13_label54.c ILP32 false +ECA eca-rers2012/Problem15_label21.c ILP32 true +ECA eca-rers2012/Problem15_label48.c ILP32 false +ECA eca-rers2012/Problem16_label14.c ILP32 false +ECA eca-rers2012/Problem16_label41.c ILP32 false +ECA eca-rers2012/Problem17_label07.c ILP32 false +ECA eca-rers2012/Problem17_label34.c ILP32 false +ECA eca-rers2012/Problem18_label00.c ILP32 false +ECA eca-rers2012/Problem18_label27.c ILP32 false +ECA eca-rers2012/Problem18_label53.c ILP32 true +ECA eca-rers2012/Problem19_label20.c ILP32 true +ECA eca-rers2012/Problem19_label46.c ILP32 true +ProductLines product-lines/elevator_spec14_product03.cil.c ILP32 true +ProductLines product-lines/elevator_spec14_product27.cil.c ILP32 true +ProductLines product-lines/elevator_spec1_product09.cil.c ILP32 true +ProductLines product-lines/elevator_spec1_product21.cil.c ILP32 true +ProductLines product-lines/elevator_spec1_product28.cil.c ILP32 false +ProductLines product-lines/elevator_spec2_product01.cil.c ILP32 true +ProductLines product-lines/elevator_spec2_product20.cil.c ILP32 false +ProductLines product-lines/elevator_spec2_product27.cil.c ILP32 true +ProductLines product-lines/elevator_spec2_productSimulator.cil.c ILP32 false +ProductLines product-lines/elevator_spec3_product19.cil.c ILP32 false +ProductLines product-lines/elevator_spec3_product25.cil.c ILP32 true +ProductLines product-lines/elevator_spec3_product32.cil.c ILP32 false +ProductLines product-lines/elevator_spec9_product27.cil.c ILP32 true +ProductLines product-lines/email_spec0_product05.cil.c ILP32 true +ProductLines product-lines/email_spec0_product22.cil.c ILP32 false +ProductLines product-lines/email_spec0_product33.cil.c ILP32 false +ProductLines product-lines/email_spec0_productSimulator.cil.c ILP32 false +ProductLines product-lines/email_spec11_product18.cil.c ILP32 true +ProductLines product-lines/email_spec11_product30.cil.c ILP32 false +ProductLines product-lines/email_spec11_product39.cil.c ILP32 true +ProductLines product-lines/email_spec1_product20.cil.c ILP32 false +ProductLines product-lines/email_spec1_product31.cil.c ILP32 false +ProductLines product-lines/email_spec27_product13.cil.c ILP32 true +ProductLines product-lines/email_spec27_product27.cil.c ILP32 false +ProductLines product-lines/email_spec27_product33.cil.c ILP32 false +ProductLines product-lines/email_spec3_product19.cil.c ILP32 false +ProductLines product-lines/email_spec3_product29.cil.c ILP32 false +ProductLines product-lines/email_spec3_productSimulator.cil.c ILP32 false +ProductLines product-lines/email_spec4_product25.cil.c ILP32 false +ProductLines product-lines/email_spec4_product32.cil.c ILP32 false +ProductLines product-lines/email_spec6_product15.cil.c ILP32 false +ProductLines product-lines/email_spec6_product28.cil.c ILP32 false +ProductLines product-lines/email_spec6_product35.cil.c ILP32 false +ProductLines product-lines/email_spec7_product23.cil.c ILP32 true +ProductLines product-lines/email_spec7_product31.cil.c ILP32 false +ProductLines product-lines/email_spec8_product14.cil.c ILP32 true +ProductLines product-lines/email_spec8_product26.cil.c ILP32 false +ProductLines product-lines/email_spec8_product34.cil.c ILP32 false +ProductLines product-lines/email_spec9_product16.cil.c ILP32 false +ProductLines product-lines/email_spec9_product30.cil.c ILP32 false +XCSP xcsp/AllInterval-005.c ILP32 false +XCSP xcsp/AllInterval-007.c ILP32 false +XCSP xcsp/AllInterval-010.c ILP32 false +XCSP xcsp/AllInterval-013.c ILP32 false +XCSP xcsp/AllInterval-016.c ILP32 false +XCSP xcsp/AllInterval-019.c ILP32 false +XCSP xcsp/AllInterval-030.c ILP32 false +XCSP xcsp/CostasArray-11.c ILP32 false +XCSP xcsp/CostasArray-14.c ILP32 false +XCSP xcsp/CostasArray-17.c ILP32 false +XCSP xcsp/Dubois-017.c ILP32 true +XCSP xcsp/Dubois-020.c ILP32 true +XCSP xcsp/Dubois-023.c ILP32 true +XCSP xcsp/Dubois-026.c ILP32 true +XCSP xcsp/Dubois-029.c ILP32 true +XCSP xcsp/Dubois-040.c ILP32 true +XCSP xcsp/Dubois-055.c ILP32 true +XCSP xcsp/Dubois-070.c ILP32 true +XCSP xcsp/Dubois-085.c ILP32 true +XCSP xcsp/Dubois-100.c ILP32 true +XCSP xcsp/Haystacks-08.c ILP32 true +XCSP xcsp/Haystacks-11.c ILP32 true +XCSP xcsp/Haystacks-14.c ILP32 true +XCSP xcsp/Haystacks-17.c ILP32 true +XCSP xcsp/aim-100-1-6-sat-1.c ILP32 false +XCSP xcsp/aim-100-1-6-sat-4.c ILP32 false +XCSP xcsp/aim-100-1-6-unsat-3.c ILP32 true +XCSP xcsp/aim-100-2-0-sat-2.c ILP32 false +XCSP xcsp/aim-100-2-0-unsat-1.c ILP32 true +XCSP xcsp/aim-100-2-0-unsat-4.c ILP32 true +XCSP xcsp/aim-100-3-4-sat-3.c ILP32 false +XCSP xcsp/aim-100-6-0-sat-2.c ILP32 false +XCSP xcsp/aim-200-1-6-sat-1.c ILP32 false +XCSP xcsp/aim-200-1-6-sat-4.c ILP32 false +XCSP xcsp/aim-200-1-6-unsat-3.c ILP32 true +XCSP xcsp/aim-200-2-0-sat-2.c ILP32 false +XCSP xcsp/aim-200-2-0-unsat-1.c ILP32 true +XCSP xcsp/aim-200-2-0-unsat-4.c ILP32 true +XCSP xcsp/aim-200-3-4-sat-3.c ILP32 false +XCSP xcsp/aim-200-6-0-sat-2.c ILP32 false +LinkedLists forester-heap/dll-01-1.i ILP32 false +LinkedLists forester-heap/dll-circular-1.i ILP32 false +LinkedLists forester-heap/dll-optional-2.i ILP32 false +LinkedLists forester-heap/dll-queue-2.i ILP32 false +LinkedLists forester-heap/dll-rb-sentinel-1.i ILP32 false +LinkedLists forester-heap/dll-simple-white-blue-1.i ILP32 false +LinkedLists forester-heap/dll-sorted-1.i ILP32 false +LinkedLists forester-heap/dll-token-2.i ILP32 false +LinkedLists forester-heap/sll-01-2.i ILP32 false +LinkedLists forester-heap/sll-circular-1.i ILP32 true +LinkedLists forester-heap/sll-optional-2.i ILP32 false +LinkedLists forester-heap/sll-queue-2.i ILP32 false +LinkedLists forester-heap/sll-rb-sentinel-1.i ILP32 true +LinkedLists forester-heap/sll-reverse_simple.i ILP32 true +LinkedLists forester-heap/sll-sorted-1.i ILP32 false +LinkedLists forester-heap/sll-token-2.i ILP32 true +LinkedLists heap-manipulation/bubble_sort_linux-2.i ILP32 false +LinkedLists heap-manipulation/dll_of_dll-2.i ILP32 false +LinkedLists heap-manipulation/merge_sort-2.i ILP32 true +LinkedLists heap-manipulation/tree-1.i ILP32 unknown +LinkedLists heap-manipulation/tree-4.i ILP32 false +LinkedLists list-ext3-properties/dll_nondet_free_order-2.i ILP32 unknown +LinkedLists list-ext3-properties/sll_length_check-2.i ILP32 true +LinkedLists list-ext3-properties/sll_nondet_insert-2.i ILP32 true +LinkedLists list-properties/alternating_list-1.i ILP32 true +LinkedLists list-properties/list-2.i ILP32 false +LinkedLists list-properties/list_flag-2.i ILP32 true +LinkedLists list-properties/simple_built_from_end.i ILP32 true +LinkedLists list-properties/splice-2.i ILP32 true +LinkedLists memsafety-broom/dll-fst-shared.i LP64 unknown +LinkedLists memsafety-broom/dll-middle-data.i LP64 unknown +LinkedLists memsafety-broom/dll.i LP64 unknown +LinkedLists memsafety-broom/linux-hlist-middle-data.i LP64 unknown +LinkedLists memsafety-broom/linux-list-lst-data.i LP64 unknown +LinkedLists memsafety-broom/sll-fst-shared.i LP64 unknown +LinkedLists memsafety-broom/sll-middle-data.i LP64 unknown +LinkedLists memsafety-broom/sll-nested-linux-list.i LP64 unknown +LinkedLists memsafety-broom/sll-nested-sll-twice.i LP64 unknown +LinkedLists memsafety-broom/sll-shared-after.i LP64 unknown +LinkedLists memsafety-broom/sll-shared-sll-before.i LP64 unknown diff --git a/tests/testcomp/corpus/cover-error.tsv b/tests/testcomp/corpus/cover-error.tsv new file mode 100644 index 000000000..1e0fbfd60 --- /dev/null +++ b/tests/testcomp/corpus/cover-error.tsv @@ -0,0 +1,373 @@ +# category program data_model expected_unreach +Arrays array-examples/data_structures_set_multi_proc_ground-1.i ILP32 false +Arrays array-examples/sorting_bubblesort_ground-2.i ILP32 false +Arrays array-examples/standard_allDiff2_ground.i ILP32 false +Arrays array-examples/standard_copy2_ground-1.i ILP32 false +Arrays array-examples/standard_copy5_ground-2.i ILP32 false +Arrays array-examples/standard_copy7_ground-1.i ILP32 false +Arrays array-examples/standard_minInArray_ground-1.i ILP32 false +Arrays array-examples/standard_running-1.i ILP32 false +Arrays array-fpi/brs3f.c ILP32 false +Arrays array-fpi/brs5f.c ILP32 false +Arrays array-fpi/condmf.c ILP32 false +Arrays array-fpi/eqn1f.c ILP32 false +Arrays array-fpi/eqn4f.c ILP32 false +Arrays array-fpi/ifcompf.c ILP32 false +Arrays array-fpi/ifeqn3f.c ILP32 false +Arrays array-fpi/ifeqn5f.c ILP32 false +Arrays array-fpi/indp2f.c ILP32 false +Arrays array-fpi/indp4f.c ILP32 false +Arrays array-fpi/modpf.c ILP32 false +Arrays array-fpi/ms1f.c ILP32 false +Arrays array-fpi/ms4f.c ILP32 false +Arrays array-fpi/ncompf.c ILP32 false +Arrays array-fpi/pcompf.c ILP32 false +Arrays array-fpi/res1of.c ILP32 false +Arrays array-fpi/s12iff.c ILP32 false +Arrays array-fpi/s1liff.c ILP32 false +Arrays array-fpi/s2liff.c ILP32 false +Arrays array-fpi/s3iff.c ILP32 false +Arrays array-fpi/s4iff.c ILP32 false +Arrays array-fpi/s52iff.c ILP32 false +Arrays array-fpi/sina1f.c ILP32 false +Arrays array-fpi/sina3f.c ILP32 false +Arrays array-fpi/sqm-iff.c ILP32 false +Arrays array-fpi/ss1f.c ILP32 false +Arrays array-fpi/ss4f.c ILP32 false +Arrays array-industry-pattern/array_ptr_single_elem_init-2.i ILP32 false +Arrays array-programs/copysome1-2.i ILP32 false +Arrays array-tiling/mlceu.c ILP32 false +Arrays reducercommutativity/rangesum05.i ILP32 false +Arrays reducercommutativity/rangesum20.i ILP32 false +BitVectors bitvector-loops/diamond_2-1.c ILP32 false +BitVectors bitvector-loops/verisec_sendmail_tTflag_arr_one_loop.c ILP32 false +BitVectors bitvector-regression/recHanoi03-1.c ILP32 false +BitVectors bitvector/byte_add-1.i ILP32 false +BitVectors bitvector/s3_clnt_2.BV.c.cil-2a.c ILP32 false +BitVectors bitvector/s3_clnt_3.BV.c.cil-2a.c ILP32 false +BitVectors bitvector/soft_float_1-3a.c.cil.c ILP32 false +BitVectors bitvector/soft_float_4-3a.c.cil.c ILP32 false +BitVectors bitvector/sum02-1.c ILP32 false +ControlFlow ntdrivers/cdaudio.i.cil-1.c ILP32 false +ControlFlow ntdrivers/diskperf.i.cil-2.c ILP32 false +ControlFlow ntdrivers/floppy.i.cil-1.c ILP32 false +ControlFlow ntdrivers/kbfiltr.i.cil-2.c ILP32 false +ControlFlow ntdrivers/parport.i.cil-1.c ILP32 false +Heap ldv-regression/fo_test.i ILP32 false +Heap ldv-regression/rule57_ebda_blast_2.i ILP32 false +Heap ldv-regression/rule60_list2.c_1.i ILP32 false +Heap ldv-regression/test21-2.c ILP32 false +Heap ldv-regression/test22-3.c ILP32 false +Heap ldv-regression/test23-1.c ILP32 false +Heap ldv-regression/test24-2.c ILP32 false +Heap ldv-regression/test25-1.c ILP32 false +Heap ldv-regression/test28-1.c ILP32 false +Heap ldv-regression/test29-1.c ILP32 false +Heap ldv-sets/test_add-1.i ILP32 false +Heap ldv-sets/test_mutex_double_lock.i ILP32 false +Heap ldv-sets/test_mutex_double_unlock.i ILP32 false +Heap ldv-sets/test_mutex_unbounded-2.i ILP32 false +Heap ldv-sets/test_mutex_unlock_at_exit.i ILP32 false +Heap list-ext-properties/list-ext.i ILP32 false +Heap list-ext-properties/simple-ext.i ILP32 false +Heap list-ext2-properties/simple_and_skiplist_2lvl-1.i ILP32 false +Heap list-ext2-properties/simple_search_value-2.i ILP32 false +Loops loop-acceleration/array_2-1.i ILP32 false +Loops loop-acceleration/multivar_1-2.c ILP32 false +Loops loop-invariants/linear-inequality-inv-b.c ILP32 false +Loops loops/array-2.c ILP32 false +Loops loops/count_up_down-2.c ILP32 false +Loops loops/invert_string-1.c ILP32 false +Loops loops/n.c24.i ILP32 false +Loops loops/sum01_bug02.i ILP32 false +Loops loops/trex01-1.c ILP32 false +Loops loops/vogal-2.i ILP32 false +Loops nla-digbench-scaling/bresenham-ll_unwindbound100.c ILP32 false +Loops nla-digbench-scaling/bresenham-ll_unwindbound5.c ILP32 false +Loops nla-digbench-scaling/egcd-ll_unwindbound10.c ILP32 false +Loops nla-digbench-scaling/egcd-ll_unwindbound20.c ILP32 false +Loops nla-digbench-scaling/egcd2-ll_unwindbound1.c ILP32 false +Loops nla-digbench-scaling/egcd2-ll_unwindbound2.c ILP32 false +Loops nla-digbench-scaling/egcd2-ll_unwindbound50.c ILP32 false +Loops nla-digbench-scaling/egcd3-ll_unwindbound100.c ILP32 false +Loops nla-digbench-scaling/egcd3-ll_unwindbound5.c ILP32 false +Loops nla-digbench-scaling/fermat1-ll_unwindbound10.c ILP32 false +Loops nla-digbench-scaling/fermat1-ll_unwindbound20.c ILP32 false +Loops nla-digbench-scaling/fermat2-ll_unwindbound1.c ILP32 false +Loops nla-digbench-scaling/fermat2-ll_unwindbound2.c ILP32 false +Loops nla-digbench-scaling/fermat2-ll_unwindbound50.c ILP32 false +Loops nla-digbench-scaling/freire2_valuebound100.c ILP32 false +Loops nla-digbench-scaling/freire2_valuebound5.c ILP32 false +Loops nla-digbench-scaling/hard2_unwindbound10.c ILP32 false +Loops nla-digbench-scaling/mannadiv_unwindbound1.c ILP32 false +Loops nla-digbench-scaling/mannadiv_unwindbound2.c ILP32 false +Loops nla-digbench-scaling/mannadiv_unwindbound50.c ILP32 false +Loops nla-digbench-scaling/prod4br-ll_unwindbound100.c ILP32 false +Loops nla-digbench-scaling/prod4br-ll_unwindbound5.c ILP32 false +Loops nla-digbench-scaling/prodbin-ll_unwindbound10.c ILP32 false +Loops nla-digbench-scaling/prodbin-ll_unwindbound5.c ILP32 false +Loops nla-digbench-scaling/ps4-ll_unwindbound100.c ILP32 false +Loops nla-digbench-scaling/ps4-ll_unwindbound5.c ILP32 false +Loops nla-digbench-scaling/ps5-ll_unwindbound10.c ILP32 false +Loops nla-digbench-scaling/ps5-ll_unwindbound20.c ILP32 false +Loops nla-digbench-scaling/ps6-ll_unwindbound1.c ILP32 false +Loops nla-digbench-scaling/ps6-ll_unwindbound2.c ILP32 false +Recursive fuzzle-programs/01_fuzzle_30x30.c LP64 false +Recursive fuzzle-programs/02_fuzzle_40x40.c LP64 false +Recursive fuzzle-programs/03_fuzzle_50x50.c LP64 false +Recursive fuzzle-programs/04_fuzzle_60x60.c LP64 false +Recursive fuzzle-programs/05_fuzzle_70x70.c LP64 false +Recursive fuzzle-programs/06_fuzzle_50x50_0-cycle.c LP64 false +Recursive fuzzle-programs/07_fuzzle_50x50_25-cycle.c LP64 false +Recursive fuzzle-programs/08_fuzzle_50x50_50-cycle.c LP64 false +Recursive fuzzle-programs/09_fuzzle_50x50_75-cycle.c LP64 false +Recursive fuzzle-programs/10_fuzzle_50x50_CVE-2016-4487.c LP64 false +Recursive fuzzle-programs/11_fuzzle_50x50_CVE-2016-4489.c LP64 false +Recursive fuzzle-programs/12_fuzzle_50x50_CVE-2016-4492.c LP64 false +Recursive fuzzle-programs/13_fuzzle_50x50_CVE-2016-4493.c LP64 false +Recursive fuzzle-programs/14_fuzzle_50x50_CVE-2016-6131.c LP64 false +Recursive fuzzle-programs/15_fuzzle_50x50_equality-checks.c LP64 false +Recursive recursified_loop-invariants/recursified_linear-inequality-inv-b.c ILP32 unknown +Recursive recursive-simple/id2_b3_o2.c ILP32 false +Recursive recursive-simple/id_b3_o2-2.c ILP32 false +Recursive recursive-simple/id_o10.c ILP32 false +Recursive recursive-simple/id_o100.c ILP32 false +Recursive recursive-simple/id_o1000.c ILP32 false +Recursive recursive-simple/id_o20.c ILP32 false +Recursive recursive-simple/id_o200.c ILP32 false +Recursive recursive-simple/id_o3.c ILP32 false +Recursive recursive-simple/sum_non_eq-3.c ILP32 false +Recursive recursive/Ackermann02.c ILP32 false +Recursive recursive/Addition02.c ILP32 false +Recursive recursive/BallRajamani-SPIN2000-Fig1.c ILP32 false +Recursive recursive/EvenOdd03.c ILP32 false +Recursive recursive/Fibonacci04.c ILP32 false +Recursive recursive/Fibonacci05.c ILP32 false +Recursive recursive/McCarthy91-1.c ILP32 false +Sequentialized seq-mthreaded/pals_STARTPALS_ActiveStandby.1.ufo.BOUNDED-10.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_STARTPALS_ActiveStandby.4_1.ufo.BOUNDED-10.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_STARTPALS_ActiveStandby.4_2.ufo.UNBOUNDED.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_STARTPALS_ActiveStandby.5.ufo.UNBOUNDED.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_STARTPALS_Triplicated.2.ufo.BOUNDED-10.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.3.1.ufo.UNBOUNDED.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.3.2.ufo.UNBOUNDED.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.3.4.ufo.BOUNDED-6.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.4.1.ufo.BOUNDED-8.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.4.2.ufo.BOUNDED-8.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.4.3.ufo.UNBOUNDED.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.4_overflow.ufo.UNBOUNDED.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.5.1.ufo.UNBOUNDED.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.5.3.ufo.BOUNDED-10.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.5.4.ufo.UNBOUNDED.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr-var-start-time.3.1.ufo.BOUNDED-6.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr-var-start-time.3.2.ufo.UNBOUNDED.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr-var-start-time.4.2.ufo.BOUNDED-8.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr-var-start-time.5.1.ufo.BOUNDED-10.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr-var-start-time.5.2.ufo.UNBOUNDED.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr-var-start-time.6.2.ufo.BOUNDED-12.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr.3.1.ufo.BOUNDED-6.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr.4.1.ufo.BOUNDED-8.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr.4_overflow.ufo.UNBOUNDED.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr.5_overflow.ufo.UNBOUNDED.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr.6_overflow.ufo.UNBOUNDED.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr.7.1.ufo.UNBOUNDED.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr.8.1.ufo.UNBOUNDED.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.3.1.ufo.UNBOUNDED.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.3.2.ufo.UNBOUNDED.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.3.4.ufo.BOUNDED-6.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.4.1.ufo.BOUNDED-8.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.4.2.ufo.BOUNDED-8.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.4.3.ufo.UNBOUNDED.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.4_overflow.ufo.UNBOUNDED.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.5.1.ufo.UNBOUNDED.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.5.3.ufo.BOUNDED-10.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.5.4.ufo.UNBOUNDED.pals.c ILP32 false +Sequentialized seq-pthread/cs_fib-2.i ILP32 false +Sequentialized seq-pthread/cs_read_write_lock-2.i ILP32 false +Floats float-benchs/cast_float_ptr.c ILP32 false +Floats float-benchs/cast_union_loose.c ILP32 false +Floats float-benchs/cast_union_tight.c ILP32 false +Floats float-benchs/float_int_inv_square.c ILP32 false +Floats float-benchs/inv_Newton-2.c ILP32 false +Floats float-benchs/inv_Newton.c.p+cfa-reducer.c ILP32 false +Floats float-benchs/inv_square-1.c ILP32 false +Floats float-benchs/nan_double.c ILP32 false +Floats float-benchs/nan_float.c ILP32 false +Floats float-benchs/sin_interpolated_index-1.c ILP32 false +Floats float-benchs/sqrt_poly2.c ILP32 false +Floats float-newlib/double_req_bl_0870a.c ILP32 false +Floats float-newlib/float_req_bl_0870a.c ILP32 false +Floats floats-cdfpl/newton_1_4.i ILP32 false +Floats floats-cdfpl/newton_1_5.i ILP32 false +Floats floats-cdfpl/newton_1_6.i ILP32 false +Floats floats-cdfpl/newton_1_7.i ILP32 false +Floats floats-cdfpl/newton_1_8.i ILP32 false +Floats floats-cdfpl/newton_2_6.i ILP32 false +Floats floats-cdfpl/newton_2_7.i ILP32 false +Floats floats-cdfpl/newton_2_8.i ILP32 false +Floats floats-cdfpl/newton_3_6.i ILP32 false +Floats floats-cdfpl/newton_3_7.i ILP32 false +Floats floats-cdfpl/newton_3_8.i ILP32 false +Floats floats-cdfpl/sine_1.i ILP32 false +Floats floats-cdfpl/sine_2.i ILP32 false +Floats floats-cdfpl/sine_3.i ILP32 false +Floats floats-cdfpl/square_1.i ILP32 false +Floats floats-cdfpl/square_2.i ILP32 false +Floats floats-cdfpl/square_3.i ILP32 false +Floats loop-floats-scientific-comp/loop1-2.c ILP32 false +Floats loop-floats-scientific-comp/loop2-1.c ILP32 false +Floats loop-floats-scientific-comp/loop4.i ILP32 false +ECA eca-programs/Problem101_label00.c ILP32 false +ECA eca-programs/Problem101_label10.c ILP32 false +ECA eca-programs/Problem101_label21.c ILP32 false +ECA eca-programs/Problem102_label13.c ILP32 false +ECA eca-programs/Problem102_label48.c ILP32 false +ECA eca-programs/Problem103_label46.c ILP32 false +ECA eca-rers2012/Problem03_label27.c ILP32 false +ECA eca-rers2012/Problem04_label04.c ILP32 false +ECA eca-rers2012/Problem04_label38.c ILP32 false +ECA eca-rers2012/Problem05_label18.c ILP32 false +ECA eca-rers2012/Problem05_label41.c ILP32 false +ECA eca-rers2012/Problem06_label02.c ILP32 false +ECA eca-rers2012/Problem06_label27.c ILP32 false +ECA eca-rers2012/Problem06_label58.c ILP32 false +ECA eca-rers2012/Problem07_label20.c ILP32 false +ECA eca-rers2012/Problem07_label44.c ILP32 false +ECA eca-rers2012/Problem08_label10.c ILP32 false +ECA eca-rers2012/Problem08_label43.c ILP32 false +ECA eca-rers2012/Problem09_label08.c ILP32 false +ECA eca-rers2012/Problem09_label38.c ILP32 false +ECA eca-rers2012/Problem10_label12.c ILP32 false +ECA eca-rers2012/Problem10_label50.c ILP32 false +ECA eca-rers2012/Problem11_label31.c ILP32 false +ECA eca-rers2012/Problem12_label06.c ILP32 false +ECA eca-rers2012/Problem12_label30.c ILP32 false +ECA eca-rers2012/Problem12_label52.c ILP32 false +ECA eca-rers2012/Problem13_label23.c ILP32 false +ECA eca-rers2012/Problem13_label44.c ILP32 false +ECA eca-rers2012/Problem15_label09.c ILP32 false +ECA eca-rers2012/Problem15_label38.c ILP32 false +ECA eca-rers2012/Problem16_label01.c ILP32 false +ECA eca-rers2012/Problem16_label27.c ILP32 false +ECA eca-rers2012/Problem16_label51.c ILP32 false +ECA eca-rers2012/Problem17_label26.c ILP32 false +ECA eca-rers2012/Problem17_label50.c ILP32 false +ECA eca-rers2012/Problem18_label09.c ILP32 false +ECA eca-rers2012/Problem18_label34.c ILP32 false +ECA eca-rers2012/Problem19_label11.c ILP32 false +ECA eca-rers2012/Problem19_label29.c ILP32 false +ECA eca-rers2012/Problem19_label58.c ILP32 false +ProductLines product-lines/elevator_spec14_product20.cil.c ILP32 false +ProductLines product-lines/elevator_spec14_productSimulator.cil.c ILP32 false +ProductLines product-lines/elevator_spec1_product24.cil.c ILP32 false +ProductLines product-lines/elevator_spec1_product32.cil.c ILP32 false +ProductLines product-lines/elevator_spec2_product22.cil.c ILP32 false +ProductLines product-lines/elevator_spec2_product32.cil.c ILP32 false +ProductLines product-lines/elevator_spec3_product19.cil.c ILP32 false +ProductLines product-lines/elevator_spec3_product27.cil.c ILP32 false +ProductLines product-lines/elevator_spec3_productSimulator.cil.c ILP32 false +ProductLines product-lines/elevator_spec9_productSimulator.cil.c ILP32 false +ProductLines product-lines/email_spec0_product26.cil.c ILP32 false +ProductLines product-lines/email_spec0_product35.cil.c ILP32 false +ProductLines product-lines/email_spec11_product22.cil.c ILP32 false +ProductLines product-lines/email_spec11_product33.cil.c ILP32 false +ProductLines product-lines/email_spec1_product16.cil.c ILP32 false +ProductLines product-lines/email_spec1_product26.cil.c ILP32 false +ProductLines product-lines/email_spec1_product32.cil.c ILP32 false +ProductLines product-lines/email_spec1_productSimulator.cil.c ILP32 false +ProductLines product-lines/email_spec27_product24.cil.c ILP32 false +ProductLines product-lines/email_spec27_product30.cil.c ILP32 false +ProductLines product-lines/email_spec27_product34.cil.c ILP32 false +ProductLines product-lines/email_spec3_product17.cil.c ILP32 false +ProductLines product-lines/email_spec3_product24.cil.c ILP32 false +ProductLines product-lines/email_spec3_product30.cil.c ILP32 false +ProductLines product-lines/email_spec3_product34.cil.c ILP32 false +ProductLines product-lines/email_spec4_product19.cil.c ILP32 false +ProductLines product-lines/email_spec4_product27.cil.c ILP32 false +ProductLines product-lines/email_spec4_product34.cil.c ILP32 false +ProductLines product-lines/email_spec6_product14.cil.c ILP32 false +ProductLines product-lines/email_spec6_product21.cil.c ILP32 false +ProductLines product-lines/email_spec6_product29.cil.c ILP32 false +ProductLines product-lines/email_spec6_product33.cil.c ILP32 false +ProductLines product-lines/email_spec7_product29.cil.c ILP32 false +ProductLines product-lines/email_spec7_product33.cil.c ILP32 false +ProductLines product-lines/email_spec8_product15.cil.c ILP32 false +ProductLines product-lines/email_spec8_product22.cil.c ILP32 false +ProductLines product-lines/email_spec8_product33.cil.c ILP32 false +ProductLines product-lines/email_spec9_product15.cil.c ILP32 false +ProductLines product-lines/email_spec9_product22.cil.c ILP32 false +ProductLines product-lines/email_spec9_product32.cil.c ILP32 false +XCSP xcsp/AllInterval-005.c ILP32 false +XCSP xcsp/AllInterval-006.c ILP32 false +XCSP xcsp/AllInterval-007.c ILP32 false +XCSP xcsp/AllInterval-009.c ILP32 false +XCSP xcsp/AllInterval-010.c ILP32 false +XCSP xcsp/AllInterval-012.c ILP32 false +XCSP xcsp/AllInterval-013.c ILP32 false +XCSP xcsp/AllInterval-015.c ILP32 false +XCSP xcsp/AllInterval-016.c ILP32 false +XCSP xcsp/AllInterval-018.c ILP32 false +XCSP xcsp/AllInterval-019.c ILP32 false +XCSP xcsp/AllInterval-025.c ILP32 false +XCSP xcsp/AllInterval-030.c ILP32 false +XCSP xcsp/CostasArray-10.c ILP32 false +XCSP xcsp/CostasArray-11.c ILP32 false +XCSP xcsp/CostasArray-13.c ILP32 false +XCSP xcsp/CostasArray-14.c ILP32 false +XCSP xcsp/CostasArray-16.c ILP32 false +XCSP xcsp/CostasArray-17.c ILP32 false +XCSP xcsp/aim-100-1-6-sat-2.c ILP32 false +XCSP xcsp/aim-100-1-6-sat-3.c ILP32 false +XCSP xcsp/aim-100-1-6-sat-4.c ILP32 false +XCSP xcsp/aim-100-2-0-sat-2.c ILP32 false +XCSP xcsp/aim-100-2-0-sat-3.c ILP32 false +XCSP xcsp/aim-100-3-4-sat-1.c ILP32 false +XCSP xcsp/aim-100-3-4-sat-2.c ILP32 false +XCSP xcsp/aim-100-3-4-sat-4.c ILP32 false +XCSP xcsp/aim-100-6-0-sat-1.c ILP32 false +XCSP xcsp/aim-100-6-0-sat-3.c ILP32 false +XCSP xcsp/aim-100-6-0-sat-4.c ILP32 false +XCSP xcsp/aim-200-1-6-sat-2.c ILP32 false +XCSP xcsp/aim-200-1-6-sat-3.c ILP32 false +XCSP xcsp/aim-200-2-0-sat-1.c ILP32 false +XCSP xcsp/aim-200-2-0-sat-2.c ILP32 false +XCSP xcsp/aim-200-2-0-sat-4.c ILP32 false +XCSP xcsp/aim-200-3-4-sat-1.c ILP32 false +XCSP xcsp/aim-200-3-4-sat-3.c ILP32 false +XCSP xcsp/aim-200-3-4-sat-4.c ILP32 false +XCSP xcsp/aim-200-6-0-sat-2.c ILP32 false +XCSP xcsp/aim-200-6-0-sat-3.c ILP32 false +LinkedLists forester-heap/dll-01-1.i ILP32 false +LinkedLists forester-heap/dll-circular-1.i ILP32 false +LinkedLists forester-heap/dll-optional-2.i ILP32 false +LinkedLists forester-heap/dll-queue-2.i ILP32 false +LinkedLists forester-heap/dll-rb-cnstr_1-2.i ILP32 false +LinkedLists forester-heap/dll-rb-sentinel-1.i ILP32 false +LinkedLists forester-heap/dll-simple-white-blue-1.i ILP32 false +LinkedLists forester-heap/dll-sorted-1.i ILP32 false +LinkedLists forester-heap/dll-token-2.i ILP32 false +LinkedLists forester-heap/sll-01-2.i ILP32 false +LinkedLists forester-heap/sll-buckets-2.i ILP32 false +LinkedLists forester-heap/sll-circular-2.i ILP32 false +LinkedLists forester-heap/sll-optional-2.i ILP32 false +LinkedLists forester-heap/sll-queue-2.i ILP32 false +LinkedLists forester-heap/sll-rb-cnstr_1-2.i ILP32 false +LinkedLists forester-heap/sll-rb-sentinel-2.i ILP32 false +LinkedLists forester-heap/sll-simple-white-blue-2.i ILP32 false +LinkedLists forester-heap/sll-sorted-1.i ILP32 false +LinkedLists forester-heap/sll-token-1.i ILP32 false +LinkedLists heap-manipulation/bubble_sort_linux-2.i ILP32 false +LinkedLists heap-manipulation/dll_of_dll-2.i ILP32 false +LinkedLists heap-manipulation/merge_sort-1.i ILP32 false +LinkedLists heap-manipulation/sll_to_dll_rev-1.i ILP32 false +LinkedLists heap-manipulation/tree-2.i ILP32 false +LinkedLists heap-manipulation/tree-4.i ILP32 false +LinkedLists list-ext3-properties/dll_nullified-1.i ILP32 false +LinkedLists list-ext3-properties/sll_length_check-1.i ILP32 false +LinkedLists list-ext3-properties/sll_nondet_insert-1.i ILP32 false +LinkedLists list-ext3-properties/sll_of_sll_nondet_append-2.i ILP32 false +LinkedLists list-properties/alternating_list-2.i ILP32 false +LinkedLists list-properties/list-2.i ILP32 false +LinkedLists list-properties/list_flag-1.i ILP32 false +LinkedLists list-properties/simple-1.i ILP32 false +LinkedLists list-properties/splice-1.i ILP32 false diff --git a/tests/testcomp/fetch-benchmarks.sh b/tests/testcomp/fetch-benchmarks.sh new file mode 100755 index 000000000..4f79671e8 --- /dev/null +++ b/tests/testcomp/fetch-benchmarks.sh @@ -0,0 +1,27 @@ +#!/bin/bash +# fetch-benchmarks.sh -- clones sv-benchmarks for the stratified Test-Comp runs. +# +# Not a submodule and not committed: the checkout is ~13 GB, it is needed only +# for evaluation runs, and pinning it as a submodule would put that weight on +# every clone of Map2Check including CI, which does not use it. +# +# --depth 1 on purpose. The corpus is used as a snapshot of programs, never for +# its history, and the full history is several times the size again. + +set -eu + +HERE="$(cd "$(dirname "$0")" && pwd)" +DEST="$HERE/bench/sv-benchmarks" +REPO="${SV_BENCHMARKS_REPO:-https://gitlab.com/sosy-lab/benchmarking/sv-benchmarks.git}" + +if [ -d "$DEST/c" ]; then + echo "sv-benchmarks already present at $DEST" + exit 0 +fi + +mkdir -p "$HERE/bench" +echo "Cloning sv-benchmarks (~13 GB, shallow) into $DEST ..." +git clone --depth 1 "$REPO" "$DEST" +echo "Done. Build a corpus with:" +echo " python3 $HERE/build_corpus.py --property cover-error --per-category 40 \\" +echo " --out $HERE/corpus/cover-error.tsv" diff --git a/tests/testcomp/run_testcomp_evaluation.sh b/tests/testcomp/run_testcomp_evaluation.sh new file mode 100755 index 000000000..3b818048d --- /dev/null +++ b/tests/testcomp/run_testcomp_evaluation.sh @@ -0,0 +1,173 @@ +#!/bin/bash +# run_testcomp_evaluation.sh -- Map2Check over a stratified Test-Comp corpus, +# with every suite handed to TestCov for validation. +# +# What this measures, and what it does not. The per-task budget here is a small +# fraction of Test-Comp's 900s, so the numbers are NOT competition scores and +# must never be quoted as such. What they are is a broad functional sweep: does +# the tool produce a well-formed suite across the whole shape of the benchmark, +# and does an independent validator agree the suite covers what it claims. +# +# Resumable by design. The CSV is the state: a task already in it is skipped, so +# a container that dies, a WSL restart, or a deadline hit can all be continued by +# re-running the same command. That is also why the deadline is checked between +# tasks rather than enforced with an outer timeout -- a killed run mid-write +# leaves a truncated row that the resume logic would then trust. +# +# Environment: +# MANIFEST tab-separated corpus from build_corpus.py (required) +# PROPERTY cover-error | cover-branches (required) +# RESULTS_DIR where the CSV and raw logs go (required) +# SHARD/SHARDS process every SHARDS-th task starting at SHARD (default 0/1) +# BUDGET seconds given to map2check per task (default 60) +# TESTCOV_S seconds given to TestCov per task (default 90) +# DEADLINE_S stop starting new tasks after this many seconds (default 18000) +# +# The container needs gcc-multilib. Most sv-benchmarks tasks declare ILP32, and +# without it TestCov cannot compile the program for -32 and reports no Result +# line at all -- which looks like a tool error and is really a missing package. + +set -u + +SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" +BENCH="$SCRIPT_DIR/bench/sv-benchmarks/c" + +MANIFEST="${MANIFEST:?set MANIFEST}" +PROPERTY="${PROPERTY:?set PROPERTY to cover-error or cover-branches}" +RESULTS_DIR="${RESULTS_DIR:?set RESULTS_DIR}" +SHARD="${SHARD:-0}" +SHARDS="${SHARDS:-1}" +BUDGET="${BUDGET:-60}" +TESTCOV_S="${TESTCOV_S:-90}" +DEADLINE_S="${DEADLINE_S:-18000}" + +MAP2CHECK_DIR="${MAP2CHECK_PATH:?set MAP2CHECK_PATH}" +MAP2CHECK="$MAP2CHECK_DIR/map2check" + +case "$PROPERTY" in + cover-error) PRP="$BENCH/properties/coverage-error-call.prp" + GOAL_FLAGS="--target-function --target-function-name reach_error" ;; + cover-branches) PRP="$BENCH/properties/coverage-branches.prp" + GOAL_FLAGS="--cover-branches" ;; + *) echo "unknown PROPERTY: $PROPERTY" >&2; exit 2 ;; +esac + +mkdir -p "$RESULTS_DIR/raw" +CSV="$RESULTS_DIR/results.csv" +if [ ! -f "$CSV" ]; then + echo "category,program,data_model,expected_unreach,verdict,testcov,coverage,goals,n_tests,elapsed_s" > "$CSV" +fi + +# Already-done set, read once. Re-reading per task turns a resume into O(n^2) +# over a corpus of hundreds. +declare -A DONE +while IFS=, read -r _cat prog _rest; do + [ -n "$prog" ] && DONE["$prog"]=1 +done < <(tail -n +2 "$CSV") + +started=$(date +%s) +index=-1 +processed=0 +skipped=0 + +# fd 3, not stdin: map2check, zip and testcov all inherit stdin, and one of them +# consuming it eats the rest of the manifest. That is not hypothetical -- it is +# how an earlier harness processed exactly one case per chunk. +while IFS=$'\t' read -r category program data_model expected <&3; do + case "$category" in ''|\#*) continue ;; esac + index=$((index + 1)) + [ $((index % SHARDS)) -eq "$SHARD" ] || continue + + if [ -n "${DONE[$program]:-}" ]; then + skipped=$((skipped + 1)) + continue + fi + + now=$(date +%s) + if [ $((now - started)) -ge "$DEADLINE_S" ]; then + echo "[deadline] stopping after ${processed} tasks ($(( (now-started)/60 )) min)" + break + fi + + src="$BENCH/$program" + if [ ! -f "$src" ]; then + echo "$category,$program,$data_model,$expected,MISSING,NA,,,0,0" >> "$CSV" + continue + fi + + # A private directory per task. map2check names its scratch by the hash of the + # input, so two tasks with identical content would otherwise share -- and a + # directory left behind by an aborted run would be reused with stale contents. + work=$(mktemp -d) + cp "$src" "$work/" 2>/dev/null || { rm -rf "$work"; continue; } + cp "$PRP" "$work/prop.prp" + name=$(basename "$src") + arch=$([ "$data_model" = "LP64" ] && echo 64bit || echo 32bit) + + t0=$(date +%s) + ( + cd "$work" || exit 1 + MAP2CHECK_PATH="$MAP2CHECK_DIR" timeout -k 10 $((BUDGET + 30)) "$MAP2CHECK" \ + --nondet-generator symex --generate-test-suite $GOAL_FLAGS \ + --property-file prop.prp --architecture "$arch" \ + --timeout "$BUDGET" "$name" + ) > "$work/map2check.log" 2>&1 /dev/null | wc -l) + + testcov="NO_SUITE" + coverage="" + goals="" + if [ -d "$work/test-suite" ]; then + ( cd "$work/test-suite" && zip -q -r ../suite.zip . ) "$work/testcov.log" 2>&1 > "$CSV" + printf '[%s] %-14s %-52s %-9s %-13s %s%%\n' \ + "$(date +%H:%M:%S)" "$category" "$(basename "$program")" "$verdict" "$testcov" "${coverage:-—}" + + # Raw logs only for the interesting cases: keeping every log over hundreds of + # tasks fills the disk with successes nobody reads. + if [ "$testcov" = "TESTCOV_ERROR" ] || [ "$verdict" = "ERROR" ]; then + cp "$work/map2check.log" "$RESULTS_DIR/raw/$(echo "$program" | tr '/' '_').m2c.log" 2>/dev/null + cp "$work/testcov.log" "$RESULTS_DIR/raw/$(echo "$program" | tr '/' '_').tc.log" 2>/dev/null + fi + + rm -rf "$work" + processed=$((processed + 1)) +done 3< "$MANIFEST" + +echo "==========================================================" +echo "shard $SHARD/$SHARDS: $processed processed, $skipped already done" +echo "CSV: $CSV" +echo "==========================================================" diff --git a/tests/unit/frontend/CMakeLists.txt b/tests/unit/frontend/CMakeLists.txt index 39bd9db2c..c4ecc97f0 100644 --- a/tests/unit/frontend/CMakeLists.txt +++ b/tests/unit/frontend/CMakeLists.txt @@ -9,3 +9,9 @@ add_executable(TestSuiteTest $ ) map2check_test(TestSuiteTest) + +add_executable(KtestReaderTest + KtestReaderTest.cpp + $ +) +map2check_test(KtestReaderTest) diff --git a/tests/unit/frontend/KtestReaderTest.cpp b/tests/unit/frontend/KtestReaderTest.cpp new file mode 100644 index 000000000..f58feed9e --- /dev/null +++ b/tests/unit/frontend/KtestReaderTest.cpp @@ -0,0 +1,257 @@ +/** + * Copyright (C) 2014 - 2026 Map2Check tool + * This file is part of the Map2Check tool, and is made available under + * the terms of the GNU General Public License version 2. + * + * SPDX-License-Identifier: (GPL-2.0) + **/ + +// A .ktest is a binary format written by another project. Parsing it wrong +// does not crash: it yields plausible-looking numbers, which become plausible- +// looking elements, which a validator scores as covering nothing. The +// failure is silent all the way down, so the parsing is pinned here against +// bytes this file builds by hand rather than against whatever KLEE happened to +// emit on the day. + +#include + +#include +#include +#include +#include +#include +#include + +#include "../../../modules/frontend/test_suite/ktest_reader.hpp" + +namespace fs = std::filesystem; + +namespace { + +// The format, from KLEE's ktest-tool: five-byte magic, then big-endian 32-bit +// counts. Version >= 2 carries two symbolic-argv fields before the objects. +class KtestBuilder { + public: + explicit KtestBuilder(uint32_t version = 3) : version_(version) {} + + KtestBuilder& object(const std::string& name, + const std::vector& bytes) { + objects_.push_back({name, bytes}); + return *this; + } + + void writeTo(const fs::path& path, const char* magic = "KTEST") const { + std::ofstream out(path, std::ios::binary); + out.write(magic, 5); + putBE(out, version_); + putBE(out, 1); // one argument, the bitcode path + const std::string arg = "./program.bc"; + putBE(out, static_cast(arg.size())); + out.write(arg.data(), arg.size()); + if (version_ >= 2) { + putBE(out, 0); // symArgvs + putBE(out, 0); // symArgvLen + } + putBE(out, static_cast(objects_.size())); + for (const auto& object : objects_) { + putBE(out, static_cast(object.first.size())); + out.write(object.first.data(), object.first.size()); + putBE(out, static_cast(object.second.size())); + if (!object.second.empty()) { + out.write(reinterpret_cast(object.second.data()), + object.second.size()); + } + } + } + + private: + static void putBE(std::ofstream& out, uint32_t value) { + const unsigned char raw[4] = { + static_cast((value >> 24) & 0xff), + static_cast((value >> 16) & 0xff), + static_cast((value >> 8) & 0xff), + static_cast(value & 0xff)}; + out.write(reinterpret_cast(raw), 4); + } + + uint32_t version_; + std::vector>> objects_; +}; + +std::vector le32(int32_t value) { + std::vector bytes(4); + std::memcpy(bytes.data(), &value, 4); + return bytes; +} + +std::vector leDouble(double value) { + std::vector bytes(8); + std::memcpy(bytes.data(), &value, 8); + return bytes; +} + +fs::path freshDir(const std::string& name) { + fs::path d = fs::temp_directory_path() / ("m2c_ktest_" + name); + fs::remove_all(d); + fs::create_directories(d); + return d; +} + +Map2Check::KtestObject obj(const std::string& name, + const std::vector& bytes) { + return Map2Check::KtestObject{name, bytes}; +} + +} // namespace + +// --- parsing ---------------------------------------------------------------- + +TEST(ReadKtestFile, ReadsObjectsInOrder) { + fs::path d = freshDir("order"); + KtestBuilder() + .object("non_det_int", le32(42)) + .object("non_det_char", {'z'}) + .writeTo(d / "test000001.ktest"); + + auto objects = Map2Check::readKtestFile((d / "test000001.ktest").string()); + ASSERT_EQ(objects.size(), 2u); + EXPECT_EQ(objects[0].name, "non_det_int"); + EXPECT_EQ(objects[0].bytes.size(), 4u); + EXPECT_EQ(objects[1].name, "non_det_char"); + EXPECT_EQ(objects[1].bytes.size(), 1u); + fs::remove_all(d); +} + +// KLEE has written both magics over its lifetime; refusing the older one would +// make the suite silently empty rather than loudly wrong. +TEST(ReadKtestFile, AcceptsTheLegacyBoutMagic) { + fs::path d = freshDir("bout"); + KtestBuilder(1).object("non_det_int", le32(7)).writeTo(d / "a.ktest", "BOUT\n"); + auto objects = Map2Check::readKtestFile((d / "a.ktest").string()); + ASSERT_EQ(objects.size(), 1u); + EXPECT_EQ(objects[0].name, "non_det_int"); + fs::remove_all(d); +} + +// Version 1 has no symbolic-argv fields. Reading them anyway would consume +// eight bytes of the object table and produce garbage names. +TEST(ReadKtestFile, HandlesVersionOneWithoutSymArgvFields) { + fs::path d = freshDir("v1"); + KtestBuilder(1).object("non_det_int", le32(5)).writeTo(d / "a.ktest"); + auto objects = Map2Check::readKtestFile((d / "a.ktest").string()); + ASSERT_EQ(objects.size(), 1u); + EXPECT_EQ(objects[0].name, "non_det_int"); + fs::remove_all(d); +} + +TEST(ReadKtestFile, RejectsAFileThatIsNotAKtest) { + fs::path d = freshDir("garbage"); + { std::ofstream(d / "a.ktest") << "this is not a ktest file at all"; } + EXPECT_TRUE(Map2Check::readKtestFile((d / "a.ktest").string()).empty()); + fs::remove_all(d); +} + +TEST(ReadKtestFile, MissingFileYieldsNothing) { + EXPECT_TRUE(Map2Check::readKtestFile("/nonexistent/a.ktest").empty()); +} + +// --- decoding --------------------------------------------------------------- + +TEST(DecodeKtestObject, DecodesSignedIntegers) { + EXPECT_EQ(Map2Check::decodeKtestObject(obj("non_det_int", le32(42))), "42"); + EXPECT_EQ(Map2Check::decodeKtestObject(obj("non_det_int", le32(-7))), "-7"); +} + +// The whole reason the object name had to be fixed. Read as unsigned, -1 +// becomes 4294967295; read as signed, an unsigned 4294967295 becomes -1. Only +// the name says which. +TEST(DecodeKtestObject, SignednessComesFromTheName) { + EXPECT_EQ(Map2Check::decodeKtestObject(obj("non_det_int", le32(-1))), "-1"); + EXPECT_EQ(Map2Check::decodeKtestObject(obj("non_det_unsigned", le32(-1))), + "4294967295"); + EXPECT_EQ(Map2Check::decodeKtestObject(obj("non_det_uint", le32(-1))), + "4294967295"); +} + +// Narrow types must sign-extend from their own width, not from 32 bits. +TEST(DecodeKtestObject, SignExtendsFromTheObjectWidth) { + EXPECT_EQ(Map2Check::decodeKtestObject(obj("non_det_char", {0xff})), "-1"); + EXPECT_EQ(Map2Check::decodeKtestObject(obj("non_det_uchar", {0xff})), "255"); + EXPECT_EQ(Map2Check::decodeKtestObject(obj("non_det_short", {0xff, 0xff})), + "-1"); +} + +TEST(DecodeKtestObject, DecodesDoubles) { + const std::string text = + Map2Check::decodeKtestObject(obj("non_det_double", leDouble(1.5))); + EXPECT_EQ(text.rfind("1.5", 0), 0u) << "got " << text; +} + +// An unknown name is a nondet type nobody has taught this decoder about yet -- +// a reason to guess, not to drop the test case. +TEST(DecodeKtestObject, FallsBackToSignedForAnUnknownName) { + EXPECT_EQ(Map2Check::decodeKtestObject(obj("something_new", le32(-3))), "-3"); +} + +TEST(DecodeKtestObject, EmptyObjectDecodesToZero) { + EXPECT_EQ(Map2Check::decodeKtestObject(obj("non_det_int", {})), "0"); +} + +// --- directory sweep -------------------------------------------------------- + +TEST(ReadKtestVectors, OneVectorPerKtestOrderedByName) { + fs::path d = freshDir("sweep"); + KtestBuilder().object("non_det_int", le32(1)).writeTo(d / "test000002.ktest"); + KtestBuilder().object("non_det_int", le32(0)).writeTo(d / "test000001.ktest"); + KtestBuilder().object("non_det_int", le32(2)).writeTo(d / "test000003.ktest"); + + auto vectors = Map2Check::readKtestVectors(d.string(), 100); + ASSERT_EQ(vectors.size(), 3u); + EXPECT_EQ(vectors[0][0], "0"); + EXPECT_EQ(vectors[1][0], "1"); + EXPECT_EQ(vectors[2][0], "2"); + fs::remove_all(d); +} + +// klee-last holds .err, .kquery and statistics files alongside the vectors. +TEST(ReadKtestVectors, IgnoresEverythingThatIsNotAKtest) { + fs::path d = freshDir("filter"); + KtestBuilder().object("non_det_int", le32(9)).writeTo(d / "test000001.ktest"); + { std::ofstream(d / "test000001.abort.err") << "error"; } + { std::ofstream(d / "run.stats") << "stats"; } + { std::ofstream(d / "assembly.ll") << "; ir"; } + + auto vectors = Map2Check::readKtestVectors(d.string(), 100); + ASSERT_EQ(vectors.size(), 1u); + EXPECT_EQ(vectors[0][0], "9"); + fs::remove_all(d); +} + +TEST(ReadKtestVectors, StopsAtTheLimit) { + fs::path d = freshDir("limit"); + for (int i = 0; i < 6; ++i) { + char name[64]; + snprintf(name, sizeof(name), "test%06d.ktest", i); + KtestBuilder().object("non_det_int", le32(i)).writeTo(d / name); + } + auto vectors = Map2Check::readKtestVectors(d.string(), 3); + ASSERT_EQ(vectors.size(), 3u); + EXPECT_EQ(vectors[2][0], "2"); + fs::remove_all(d); +} + +// A path that read no input is not a test case; emitting one would put an +// empty into the suite for every such path. +TEST(ReadKtestVectors, DropsVectorsWithNoObjects) { + fs::path d = freshDir("empty"); + KtestBuilder().writeTo(d / "test000001.ktest"); + KtestBuilder().object("non_det_int", le32(4)).writeTo(d / "test000002.ktest"); + auto vectors = Map2Check::readKtestVectors(d.string(), 100); + ASSERT_EQ(vectors.size(), 1u); + EXPECT_EQ(vectors[0][0], "4"); + fs::remove_all(d); +} + +TEST(ReadKtestVectors, MissingDirectoryYieldsNothing) { + EXPECT_TRUE(Map2Check::readKtestVectors("/nonexistent/klee-last", 10).empty()); +} diff --git a/utils/map2check-testcomp-wrapper.py b/utils/map2check-testcomp-wrapper.py index 54a61723a..24be0e998 100755 --- a/utils/map2check-testcomp-wrapper.py +++ b/utils/map2check-testcomp-wrapper.py @@ -14,6 +14,11 @@ * the suite is written whether or not a violation was found -- an absent directory reads as a crashed tool, an empty suite as an honest zero * exit 0 means "I ran"; it does not claim the suite covers anything + +Both Test-Comp goals are supported, from two different sources: + + cover-error klee_log.csv, the violating input vector + cover-branches klee-last/*.ktest, one input vector per path KLEE explored """ import argparse @@ -78,15 +83,14 @@ def main(argv): prop = read_property(args.propertyfile) if COVER_ERROR in prop: + # One test case: the violating vector, from klee_log.csv. goal_flags = ["--target-function", "--target-function-name", "reach_error"] elif COVER_BRANCHES in prop: - # Not a silent zero. Branch coverage needs one test case per input - # vector, and the runtime currently writes a single nondet log per run - # (H1.3 in docs/TESTCOMP-CHECKLIST.md). Emitting a one-case suite here - # would look like participation while scoring nothing and hiding why. - print("Unsupported Property: cover-branches needs per-input test cases " - "(see H1.3 in docs/TESTCOMP-CHECKLIST.md)") - return 1 + # One test case per path KLEE explored, from its own .ktest output. + # No target function: the goal is to cover branches, and asking the + # tool to hunt for reach_error would make it stop at the first error + # instead of exploring. + goal_flags = ["--cover-branches"] else: print("Unsupported Property") return 1 diff --git a/utils/moduleBenchExec/map2check_testcomp.py b/utils/moduleBenchExec/map2check_testcomp.py index 63501015c..c2a5b0e30 100644 --- a/utils/moduleBenchExec/map2check_testcomp.py +++ b/utils/moduleBenchExec/map2check_testcomp.py @@ -21,7 +21,13 @@ class Tool(benchexec.tools.template.BaseTool2): - """Tool adaptor for Map2Check (https://github.com/hbgit/Map2Check).""" + """Tool adaptor for Map2Check (https://github.com/hbgit/Map2Check). + + Both Test-Comp goals are supported. The wrapper picks the source of test + vectors from the property: cover-error takes the single violating vector + from the runtime's nondet log, cover-branches takes one vector per path + from KLEE's own .ktest output. + """ REQUIRED_PATHS = [ "map2check", @@ -55,16 +61,7 @@ def cmdline(self, executable, options, task, rlimits): with open(task.property_file, "r") as handle: spec = handle.read() - if COVER_BRANCHES in spec: - # Declared, not faked. Branch coverage needs one test case per input - # vector and the runtime writes one nondet log per run (H1.3 in - # docs/TESTCOMP-CHECKLIST.md). Raising here makes BenchExec skip the - # task instead of recording a run that could only ever score zero. - raise benchexec.tools.template.UnsupportedFeatureException( - "cover-branches is not supported yet: it needs per-input test " - "cases (H1.3)" - ) - if COVER_ERROR not in spec: + if COVER_ERROR not in spec and COVER_BRANCHES not in spec: raise benchexec.tools.template.UnsupportedFeatureException( "unsupported property: " + spec.strip() ) From 376e31b221136e31fba7d0ab7f52ad51095d4a21 Mon Sep 17 00:00:00 2001 From: Guilherme Bernardo Date: Sat, 22 Aug 2026 18:46:17 -0400 Subject: [PATCH 02/17] fix(testcomp): stop labelling TestCov's Result line as a coverage claim The first version mapped Result: DONE to FULL and Result: UNKNOWN to PARTIAL for cover-branches, on the inference that DONE meant 100% coverage. The running corpus disproved it within minutes: DONE appears at 77.78% and at 11.11% as readily as at 100%. What the Result line actually distinguishes is whether TestCov validated cleanly (DONE) or had to abort a test's execution (UNKNOWN). Neither says anything about how much was covered -- the coverage column beside it does, in both cases. FULL and PARTIAL read as a claim the line is not making, so they are now VALIDATED and VALIDATED_ABORTS. Replaced by rename rather than edited in place: nine containers have this script open, and truncating the inode under a running bash makes it execute whatever lands at its current offset. Co-Authored-By: Claude Opus 5 (1M context) --- tests/testcomp/run_testcomp_evaluation.sh | 27 ++++++++++++++--------- 1 file changed, 17 insertions(+), 10 deletions(-) diff --git a/tests/testcomp/run_testcomp_evaluation.sh b/tests/testcomp/run_testcomp_evaluation.sh index 3b818048d..042d1c582 100755 --- a/tests/testcomp/run_testcomp_evaluation.sh +++ b/tests/testcomp/run_testcomp_evaluation.sh @@ -131,20 +131,27 @@ while IFS=$'\t' read -r category program data_model expected <&3; do # TestCov exits 0 whether or not the suite covers, so the verdict is the # Result line, never $?. # - # The label is property-aware, because the same Result line means different - # things. For cover-error, TRUE means the suite reproduced the bug and - # UNKNOWN means it did not -- pass and fail. For cover-branches there is no - # pass/fail: the competition scores the coverage FRACTION, and TestCov - # prints DONE only at 100% and UNKNOWN for anything less. Calling a 90% - # branch suite NOT_COVERED would read as a failure when it is most of a - # good result, so partial coverage is labelled PARTIAL and the percentage - # in the next column is the number that matters. + # The label is property-aware, because the same Result line does not mean + # the same thing in both categories. + # + # cover-error is pass/fail: TRUE means the suite reproduced the bug, + # UNKNOWN means it did not. + # + # cover-branches has no pass/fail. The competition scores the coverage + # FRACTION, and the Result line says nothing about it: measured, DONE + # appears at 100% and at 77.78%, while UNKNOWN appears when TestCov had to + # abort a test's execution. So DONE means "validated cleanly" and UNKNOWN + # means "validated, with a test that would not run" -- and in BOTH cases + # the coverage column beside it is the number that matters. Labelling them + # FULL and PARTIAL, as a first version did, reads as a coverage claim the + # Result line is not making. if grep -qE '^Result: TRUE' "$work/testcov.log"; then testcov="COVERED" elif grep -qE '^Result: DONE' "$work/testcov.log"; then - testcov=$([ "$PROPERTY" = "cover-branches" ] && echo FULL || echo DONE) + testcov=$([ "$PROPERTY" = "cover-branches" ] && echo VALIDATED || echo DONE) elif grep -qE '^Result: UNKNOWN' "$work/testcov.log"; then - testcov=$([ "$PROPERTY" = "cover-branches" ] && echo PARTIAL || echo NOT_COVERED) + testcov=$([ "$PROPERTY" = "cover-branches" ] && echo VALIDATED_ABORTS \ + || echo NOT_COVERED) else testcov="TESTCOV_ERROR" fi From d6da0e97adef9346e031c62953107e54b2057957 Mon Sep 17 00:00:00 2001 From: Guilherme Bernardo Date: Sat, 22 Aug 2026 18:50:31 -0400 Subject: [PATCH 03/17] docs(testcomp): record how to read the 2026-08-22 corpus results The running containers hold the pre-fix inode of the evaluation script, so this run writes FULL/PARTIAL where the script now writes VALIDATED/VALIDATED_ABORTS. Recording the equivalence beside the data, because in five hours the CSV will be read by someone -- me -- who no longer has that in mind, and FULL reads as a coverage claim it never was. Co-Authored-By: Claude Opus 5 (1M context) --- tests/testcomp/RUN-2026-08-22-NOTES.md | 45 ++++++++++++++++++++++++++ 1 file changed, 45 insertions(+) create mode 100644 tests/testcomp/RUN-2026-08-22-NOTES.md diff --git a/tests/testcomp/RUN-2026-08-22-NOTES.md b/tests/testcomp/RUN-2026-08-22-NOTES.md new file mode 100644 index 000000000..8d8440678 --- /dev/null +++ b/tests/testcomp/RUN-2026-08-22-NOTES.md @@ -0,0 +1,45 @@ +# Corrida Test-Comp de 2026-08-22 — como ler os CSVs + +Resultados em `results_cover-error_s{0,1}/` e `results_cover-branches_s{0,1}/`, +produzidos por `run_testcomp_evaluation.sh` a partir dos manifestos em +`corpus/`. + +## Os rótulos `testcov` desta corrida são os antigos + +O script foi corrigido **depois** que os containers já estavam rodando, e um +`bash` em execução mantém o inode que abriu — a correção vale para as próximas +corridas, não para esta. Equivalência: + +| nesta corrida | significa | rótulo atual do script | +|---|---|---| +| `FULL` | TestCov validou sem abortar nenhum teste (`Result: DONE`) | `VALIDATED` | +| `PARTIAL` | TestCov abortou a execução de algum teste (`Result: UNKNOWN`) | `VALIDATED_ABORTS` | +| `COVERED` / `NOT_COVERED` | (cover-error) o erro foi ou não reproduzido | inalterados | + +**`FULL` não significa cobertura total.** Foi exatamente essa a inferência que +a corrida desmentiu nos primeiros minutos: `Result: DONE` aparece a 100%, a +77,78% e a 11,11%. A linha `Result` diz se a validação correu limpa, não quanto +foi coberto. + +## Qual coluna vale + +- **cover-branches**: `coverage` (percentual sobre `goals`). A competição + pontua a fração de ramos, não um passa/não-passa. +- **cover-error**: `testcov = COVERED`. Aí sim é binário — ou a suíte reproduz + o erro, ou não. + +## Estes números não são pontuação de competição + +`BUDGET=60s` por tarefa contra os **900s** do Test-Comp, e um subconjunto +estratificado de 372 + 480 tarefas contra o benchmark inteiro. O que a corrida +mede é cobertura funcional — a ferramenta produz suíte bem-formada em toda a +extensão do benchmark, e um validador independente concorda com o que a suíte +afirma. Citar como score seria enganoso. + +## Amostragem + +Cota fixa por subcategoria com passo determinístico (`build_corpus.py`), não +amostra uniforme: Floats e ECA sozinhos somam 2,6 mil das 33 mil tarefas, e uma +amostra uniforme reportaria o formato do benchmark em vez do comportamento da +ferramenta. Sem RNG — reexecutar o script no mesmo checkout dá o mesmo +manifesto. From 381aeaecffb9cbf6e2e85360061e5e292456bb7d Mon Sep 17 00:00:00 2001 From: Guilherme Bernardo Date: Sat, 22 Aug 2026 18:51:02 -0400 Subject: [PATCH 04/17] =?UTF-8?q?docs:=20H1.3=20and=20H2.5=20done=20?= =?UTF-8?q?=E2=80=94=20Cover-Branches=20lands=20via=20KLEE's=20ktest=20out?= =?UTF-8?q?put?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit H1.3 is closed by a different route than the plan assumed. It asked for a per-input log from the runtime; that was built, measured, and thrown away (1s/FALSE became 100s/wrong-SUCCEEDED under KLEE). KLEE's own .ktest files are the same vectors at no cost, which is what the entry now records. H2.5 follows immediately: --cover-branches, accepted by the wrapper and the tool-info, validated by TestCov at 100% on a six-branch program. Co-Authored-By: Claude Opus 5 (1M context) --- docs/TESTCOMP-CHECKLIST.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/docs/TESTCOMP-CHECKLIST.md b/docs/TESTCOMP-CHECKLIST.md index a29fa5d54..9150bde1f 100644 --- a/docs/TESTCOMP-CHECKLIST.md +++ b/docs/TESTCOMP-CHECKLIST.md @@ -4,7 +4,7 @@ Estado das frentes do [plano de desenvolvimento](../.opencode/Plano%20de%20desen (local, fora do repositório). Este arquivo é o índice vivo: cada item aponta para a evidência que sustenta o estado declarado. -**Última atualização:** 2026-08-22 +**Última atualização:** 2026-08-22 (noite) ## Legenda @@ -23,7 +23,7 @@ evidência que sustenta o estado declarado. |---|---|---| | **H1.1** Emissor de test suite XML | ✅ | `modules/frontend/test_suite/`. `--generate-test-suite` emite `metadata.xml` + `testcase-N.xml` no formato de intercâmbio. 13 testes unitários + 12 asserções de integração. **Custo real muito abaixo das 2 pw estimadas**: o runtime já registrava os valores em ordem de consumo, faltava só serializar | | **H1.2** Conversor ktest→XML | ✅ | **Colapsou dentro do H1.1.** O log é escrito pelo binário instrumentado em tempo de execução, então já carrega os valores concretos do KLEE. Não precisou de conversor separado | -| **H1.3** Conversor corpus-LibFuzzer→XML | 🟡 | O emissor é agnóstico de engine por construção (`NonDetGeneratorLibFuzzy.c:47` descarrega o mesmo log). **Falta:** log por input, para suítes com múltiplos test cases. **É o gargalo declarado**: bloqueia H2.5 e é o motivo pelo qual o tool-info recusa `cover-branches` | +| **H1.3** Vetores de entrada por caminho | ✅ | **Resolvido por outro caminho que o planejado.** O KLEE já grava um `.ktest` por caminho explorado, com os objetos simbólicos em ordem de consumo; `ktest_reader.cpp` os lê. A alternativa (o runtime gravar um log por estado) foi **medida e descartada**: levou um run de 1s/`FALSE` para 100s/`SUCCEEDED` errado, porque cada escrita é chamada externa que o KLEE executa concretamente. 16 testes unitários fixam o parse contra bytes montados à mão | | **H1.4** Empacotamento BenchExec / fm-tools | ✅ | `utils/moduleBenchExec/map2check_testcomp.py` (`BaseTool2`) + `utils/map2check-testcomp-wrapper.py`, ambos no zip de release. 15 asserções contra o benchexec real em `tests/integration/test_benchexec_toolinfo.py`. **Falta:** registrar em `fm-tools` (fora deste repositório). O `map2check.py` SV-COMP segue em `BaseTool` 1.x — obsoleto mas ainda presente no benchexec 3.35, então não está morto | | **H1.5** E2E no CI com TestCov | ✅ | Job `Test-Comp Validation (TestCov)`, 6/6 contra manifesto medido. Ver `tests/testcomp/` | @@ -35,7 +35,7 @@ evidência que sustenta o estado declarado. | **H2.2** Orquestração com time-slicing | ⬜ | Default híbrido atual (LibFuzzer 0.2× → KLEE 0.8×) preservado | | **H2.3** Seed exchange fuzzer↔KLEE | ⬜ | — | | **H2.4** Slicing pré-simbólico | ⬜ | Ver nota de prioridade abaixo | -| **H2.5** Modo Cover-Branches | ⬜ | Bloqueado por H1.3 (log por input). O módulo tool-info **recusa** a propriedade em vez de aceitá-la e pontuar zero | +| **H2.5** Modo Cover-Branches | ✅ | `--cover-branches` emite um test case por caminho do KLEE, `coversError` falso (são caminhos, não violações), teto de 500. Wrapper e tool-info **aceitam** a propriedade. Validado ponta a ponta: 7 test cases num programa de 6 ramos, **TestCov 100,0%**. Gate no CI (`test_cover_branches.sh`) mede cobertura pelo validador, não inspecionando XML | ### Nota de prioridade sobre H2 @@ -72,6 +72,7 @@ medir o efeito de K nesse corpus. | Invariantes: gate diferencial p/ promover a default | ⬜ | Task 7 do [plano de revival](superpowers/plans/2026-08-16-add-invariants-revival.md). Depende de rodar sobre corpus real | | **Baseline CASTLE + Juliet (v5)** | ✅ | PR #55. CASTLE 98,2% precisão, Juliet 90,8% (99,2% descontando artefatos de mapeamento) | | Corrida v6 (depois/antes) | 🟡 | **CASTLE fechado** (217/250): precisão 98,2%, recall 74,0%; excluindo o modo degenerado do achado B, recall 79,4%. Juliet: 3 de 4 fatias fechadas, `c` em andamento. **Regra de atribuição: invariantes DESLIGADOS na corrida principal**, senão overflow + timeout + amostragem + invariantes ficam conjuntamente não creditáveis | +| **Corpus Test-Comp estratificado** | 🟡 | `fetch-benchmarks.sh` + `build_corpus.py` + `run_testcomp_evaluation.sh`. 372 tarefas cover-error e 480 cover-branches, cota por subcategoria e passo determinístico. Corrida de 2026-08-22 em andamento; ver [as notas de leitura](../tests/testcomp/RUN-2026-08-22-NOTES.md) — **o orçamento é 60s contra os 900s da competição, não são scores** | | **Alcançabilidade (CWE-843/628/770/835)** | ⬜ | Adiado por decisão. Sob a lente SV-COMP as quatro colapsam em uma: só 835 mapeia para categoria pontuável (Termination), e os wrappers declaram não suportar | --- @@ -110,7 +111,7 @@ Todos com evidência em [findings](reports/2026-08-12-castle-juliet-findings.md) | O quê | Contagem | |---|---| | `ctest` (unitários) | 8 | -| Integração | 97 asserções em 9 scripts | +| Integração | ~110 asserções em 11 scripts | | Conformidade Test-Comp | 6 programas | | Jobs de CI | 10, todos verdes na PR #59 (o 10º builda a imagem no próprio PR) | From 67b8378347780b767661b4ab5641da6215d6242a Mon Sep 17 00:00:00 2001 From: Guilherme Bernardo Date: Sat, 22 Aug 2026 19:04:39 -0400 Subject: [PATCH 05/17] fix(testcomp): nested sampling and a category-interleaved manifest; quota 150 Two defects in the corpus builder, both of which only bite when the corpus grows or a run is cut short -- which is exactly what happened. Sampling was not nested. An even stride recomputes every index when the quota changes, so raising it substitutes tasks rather than adding them. Measured on the real pool sizes: going from 40 to 150 per subcategory kept 12 of 40 tasks in Arrays and 10 of 40 in Loops, Floats and ECA -- 70% overall, but the large families almost entirely resampled, which would have thrown away measurements already taken. Replaced by a breadth-first recursive bisection order, whose prefixes both spread evenly over the family AND nest: the first 40 of the sequence are the first 40 of any longer one. Verified 40/40 preserved at 150. The manifest was grouped by subcategory. Every run here is deadline-bounded, and a truncated grouped manifest means the run finished Arrays and never reached XCSP -- discarding the stratification the sampling exists to provide. Rows are now written round-robin across subcategories, so any prefix is balanced. Quota raised to 150: 818 cover-error tasks (limited by availability in eight of the twelve subcategories) and 1522 cover-branches. sv-benchmarks checked against upstream while doing this: cd68d4ed3, current. Co-Authored-By: Claude Opus 5 (1M context) --- tests/testcomp/build_corpus.py | 87 +- tests/testcomp/corpus/cover-branches.tsv | 1908 +++++++++++++++++----- tests/testcomp/corpus/cover-error.tsv | 1080 ++++++++---- 3 files changed, 2307 insertions(+), 768 deletions(-) diff --git a/tests/testcomp/build_corpus.py b/tests/testcomp/build_corpus.py index ab219940d..d7e19cada 100755 --- a/tests/testcomp/build_corpus.py +++ b/tests/testcomp/build_corpus.py @@ -8,10 +8,23 @@ benchmark's shape than about the tool. Fixing a quota per subcategory makes "Map2Check does badly on Recursive" a statement the numbers can support. -Selection inside a subcategory is deterministic: tasks are sorted by path and -taken at an even stride. No RNG, no seed to record, and re-running the script -on the same checkout produces the same manifest -- which is what makes two runs -comparable at all. +Selection inside a subcategory is deterministic and NESTED: the candidates are +sorted by path, then ordered by recursive bisection, and the first N are taken. +No RNG, no seed to record, and two properties fall out that a plain stride does +not give: + + * every prefix is evenly spread over the family, so a run cut short by its + deadline still sampled the whole range rather than one alphabetical corner; + * a larger quota is a SUPERSET of a smaller one. That matters as soon as the + corpus grows: with an even stride, raising the quota from 40 to 150 + preserved only 12 of 40 tasks in Arrays and 10 of 40 in Loops, Floats and + ECA -- 70% overall, but the big families almost entirely resampled, so the + measurements already taken could not be reused. + +The manifest itself is written round-robin across subcategories, not grouped by +them. A run that stops at its deadline stops after a balanced prefix; grouped +by category it would have finished Arrays and never reached XCSP, which throws +away the stratification the sampling exists to provide. Usage: build_corpus.py --property cover-error --per-category 40 --out manifest.tsv @@ -112,19 +125,44 @@ def task_info(yml_path, wanted_property): return program, data_model, expected -def stride_sample(items, quota): - """Even stride, not the first N. +def spread_order(items): + """Deterministic permutation whose every prefix is evenly spread. - Task names inside a family are ordered, and the first N of a sorted list is - usually N variations of one program. A stride spreads the sample across the - family's whole range at no cost in reproducibility. + Breadth-first recursive bisection: the middle element first, then the + middles of the two halves, then of the four quarters, and so on. Two + properties follow, and both are why this replaced an even stride. + + Any prefix covers the whole range. Taking the first N of a sorted list + would give N variations of one program, since task names inside a family + are ordered; taking a bisection prefix gives N spread across it. + + Prefixes nest. Because the order does not depend on N, the first 40 of this + sequence are the first 40 of any longer one -- so raising the quota ADDS + tasks instead of substituting them, and measurements already taken at the + smaller quota stay valid. An even stride does not have that property: at + the real pool sizes here, going from 40 to 150 kept 12 of 40 tasks in + Arrays and 10 of 40 in Loops, Floats and ECA. """ - if quota >= len(items): - return items + from collections import deque + + order = [] + queue = deque([(0, len(items) - 1)]) + while queue: + low, high = queue.popleft() + if low > high: + continue + middle = (low + high) // 2 + order.append(items[middle]) + queue.append((low, middle - 1)) + queue.append((middle + 1, high)) + return order + + +def nested_sample(items, quota): + """The first `quota` of the spread order.""" if quota <= 0: return [] - step = len(items) / float(quota) - return [items[int(i * step)] for i in range(quota)] + return spread_order(items)[:quota] def main(): @@ -139,7 +177,7 @@ def main(): sys.exit("sv-benchmarks not found at %s -- run fetch-benchmarks.sh" % BENCH) - rows = [] + per_category = {} summary = [] for category in CATEGORIES: applicable = [] @@ -147,11 +185,24 @@ def main(): info = task_info(yml, args.property) if info is not None: applicable.append((yml,) + info) - chosen = stride_sample(applicable, args.per_category) + chosen = nested_sample(applicable, args.per_category) summary.append((category, len(applicable), len(chosen))) - for yml, program, data_model, expected in chosen: - rows.append((category, os.path.relpath(program, BENCH), data_model, - expected or "unknown")) + per_category[category] = [ + (category, os.path.relpath(program, BENCH), data_model, + expected or "unknown") + for yml, program, data_model, expected in chosen + ] + + # Round-robin, not grouped. A run that stops at its deadline should stop + # after a balanced prefix of every subcategory; written grouped, it would + # finish Arrays and never reach XCSP. + rows = [] + depth = max((len(v) for v in per_category.values()), default=0) + for index in range(depth): + for category in CATEGORIES: + bucket = per_category.get(category, []) + if index < len(bucket): + rows.append(bucket[index]) with open(args.out, "w") as handle: handle.write("# category\tprogram\tdata_model\texpected_unreach\n") diff --git a/tests/testcomp/corpus/cover-branches.tsv b/tests/testcomp/corpus/cover-branches.tsv index 9bef7c250..6d2ab1e63 100644 --- a/tests/testcomp/corpus/cover-branches.tsv +++ b/tests/testcomp/corpus/cover-branches.tsv @@ -1,481 +1,1523 @@ # category program data_model expected_unreach -Arrays array-cav19/array_init_nondet_vars.c ILP32 true -Arrays array-crafted/bAnd4.i ILP32 true -Arrays array-crafted/mapsum1.i ILP32 true -Arrays array-crafted/zero_sum3.c ILP32 true -Arrays array-crafted/zero_sum_m2.c ILP32 true -Arrays array-examples/sanfoundry_24-2.i ILP32 unknown -Arrays array-examples/standard_copy1_ground-1.i ILP32 true -Arrays array-examples/standard_copy7_ground-1.i ILP32 false -Arrays array-examples/standard_palindrome_ground.i ILP32 true -Arrays array-examples/standard_strcmp_ground.i ILP32 true -Arrays array-examples/standard_two_index_09.i ILP32 true -Arrays array-fpi/brs5_overflow.c ILP32 unknown -Arrays array-fpi/eqn1f.c ILP32 false -Arrays array-fpi/ifeqn1f.c ILP32 false -Arrays array-fpi/indp1f.c ILP32 false -Arrays array-fpi/mods.c ILP32 true -Arrays array-fpi/ncomp.c ILP32 true -Arrays array-fpi/res2.c ILP32 true -Arrays array-fpi/s2if.c ILP32 true -Arrays array-fpi/s42iff.c ILP32 false -Arrays array-fpi/s5if.c ILP32 true -Arrays array-fpi/sina3.c ILP32 true -Arrays array-fpi/ss2.c ILP32 true -Arrays array-industry-pattern/array_of_struct_ptr_monotonic.i ILP32 true -Arrays array-lopstr16/single_elem_safe.i ILP32 true -Arrays array-memsafety/count_down_unsafe.i ILP32 unknown -Arrays array-memsafety/cstrncpy-alloca-2.i ILP32 unknown -Arrays array-memsafety/mult_array-alloca-1.i ILP32 unknown -Arrays array-memsafety/openbsd_cstrlcpy-alloca-1.i ILP32 unknown -Arrays array-memsafety/selectionsort-alloca-1.i ILP32 unknown -Arrays array-multidimensional/copy-partial-3-u.c ILP32 true -Arrays array-multidimensional/rev-2-n-u.c ILP32 true -Arrays array-patterns/array19_pattern.c ILP32 true -Arrays array-patterns/array29_pattern.c ILP32 true -Arrays array-patterns/array7_pattern.c ILP32 true -Arrays array-programs/partial_mod_count_limited_base.c ILP32 true -Arrays array-tiling/pnr2.c ILP32 true -Arrays array-tiling/rewnif.c ILP32 true -Arrays reducercommutativity/avg60-1.i ILP32 true -Arrays reducercommutativity/rangesum60.i ILP32 false +Arrays array-fpi/s52iff_overflow.c ILP32 unknown +BitVectors bitvector/s3_clnt_3.BV.c.cil-1.c ILP32 unknown +ControlFlow openssl-simplified/s3_srvr_1a.cil.c ILP32 unknown +Heap memory-alloca/c.03-alloca-2.i ILP32 unknown +Loops nla-digbench-scaling/fermat1-ll_valuebound2.c ILP32 true +Recursive recursified_nla-digbench/recursified_lcm2.c ILP32 unknown +Sequentialized seq-mthreaded/pals_lcr.6.ufo.BOUNDED-12.pals.c ILP32 true +Floats neural-networks/dubinsrejoin_79_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem10_label22.c ILP32 true +ProductLines product-lines/email_spec1_product20.cil.c ILP32 false +XCSP xcsp/Haystacks-08.c ILP32 true +LinkedLists heap-manipulation/tree-3.i ILP32 true +Arrays array-examples/standard_two_index_08.i ILP32 true +BitVectors bitvector/jain_2-2.c ILP32 unknown +ControlFlow ntdrivers/diskperf.i.cil-2.c ILP32 false +Heap ldv-memsafety/memleaks_test7-1.i ILP32 unknown +Loops loops/array-1.c ILP32 true +Recursive recursified_nla-digbench/recursified_cohendiv-ll.c ILP32 unknown +Sequentialized seq-mthreaded/pals_floodmax.5.2.ufo.UNBOUNDED.pals.c ILP32 false +Floats floats-esbmc-regression/ceil_nondet.i ILP32 true +ECA eca-rers2012/Problem05_label57.c ILP32 false +ProductLines product-lines/elevator_spec3_product25.cil.c ILP32 true +XCSP xcsp/Dubois-017.c ILP32 true +LinkedLists forester-heap/sll-optional-1.i ILP32 true +Arrays array-multidimensional/copy-partial-2-n-u.c ILP32 true +BitVectors bitvector/soft_float_1-2a.c.cil.c ILP32 true +ControlFlow openssl/s3_srvr.blast.07.i.cil-1.c ILP32 unknown +Heap memsafety-cve/picotcp.i ILP32 unknown +Loops nla-digbench-scaling/lcm1_valuebound2.c ILP32 true +Recursive recursive-simple/id_o200.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.5.3.ufo.BOUNDED-10.pals.c ILP32 false +Floats neural-networks/poly_256_thresh_3_unsafe.c-amalgamation.i ILP32 false +ECA eca-rers2012/Problem15_label48.c ILP32 false +ProductLines product-lines/email_spec6_product15.cil.c ILP32 false +XCSP xcsp/aim-100-3-4-sat-3.c ILP32 false +LinkedLists memsafety-broom/dll-lst-shared.i LP64 unknown +Arrays array-examples/sanfoundry_24-1.i ILP32 true +BitVectors bitvector/byte_add_1-2.i ILP32 unknown +ControlFlow ntdrivers-simplified/cdaudio_simpl1.cil-2.c ILP32 unknown +Heap ldv-memsafety/memleaks_test17_2-2.i ILP32 unknown +Loops loop-simple/nested_1b.c ILP32 false +Recursive recursified_loop-crafted/recursified_simple_array_index_value_2.i ILP32 unknown +Sequentialized seq-mthreaded/pals_floodmax.3.3.ufo.UNBOUNDED.pals.c ILP32 false +Floats float-newlib/double_req_bl_1231b.c ILP32 true +ECA eca-rers2012/Problem03_label25.c ILP32 true +ProductLines product-lines/elevator_spec2_product01.cil.c ILP32 true +XCSP xcsp/AllInterval-019.c ILP32 false +LinkedLists forester-heap/dll-reverse.i ILP32 true +Arrays array-fpi/modpf.c ILP32 false +BitVectors bitvector/s3_clnt_1.BV.c.cil-1.c ILP32 unknown +ControlFlow openssl-simplified/s3_clnt_2.cil-2.c ILP32 unknown +Heap ldv-regression/test28-1.c ILP32 false +Loops nla-digbench-scaling/cohendiv-ll_valuebound5.c ILP32 true +Recursive recursified_nla-digbench/recursified_geo1-ll.c ILP32 unknown +Sequentialized seq-mthreaded/pals_lcr-var-start-time.5.ufo.BOUNDED-10.pals.c ILP32 true +Floats neural-networks/cartpole_7_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem08_label09.c ILP32 true +ProductLines product-lines/email_spec0_product33.cil.c ILP32 false +XCSP xcsp/Dubois-040.c ILP32 true +LinkedLists forester-heap/sll-token-1.i ILP32 false +Arrays array-memsafety/count_down-alloca-1.i ILP32 unknown +BitVectors bitvector/s3_srvr_2.BV.c.cil.c ILP32 unknown +ControlFlow openssl/s3_clnt.blast.02.i.cil-2.c ILP32 unknown +Heap memsafety-cve/json-parser.i ILP32 unknown +Loops nla-digbench-scaling/geo2-ll_unwindbound50.c ILP32 true +Recursive recursified_nla-digbench/recursified_sqrt1-ll.c ILP32 unknown +Sequentialized seq-mthreaded/pals_opt-floodmax.3.4.ufo.BOUNDED-6.pals.c ILP32 false +Floats neural-networks/lunarlander_3_unsafe.c-amalgamation.i ILP32 false +ECA eca-rers2012/Problem12_label34.c ILP32 false +ProductLines product-lines/email_spec3_product19.cil.c ILP32 false +XCSP xcsp/aim-100-1-6-sat-4.c ILP32 false +LinkedLists list-properties/list-1.i ILP32 true +Arrays array-programs/partial_mod_count_limited_5.c ILP32 true +BitVectors bitvector/soft_float_4-2.c.cil.c ILP32 unknown +ControlFlow openssl/s3_srvr.blast.13.i.cil-1.c ILP32 unknown +Heap memsafety-ext2/length_test03-2.i ILP32 unknown +Loops nla-digbench-scaling/ps5-ll_unwindbound10.c ILP32 false +Recursive recursive/EvenOdd01-1.c ILP32 true +Sequentialized seq-pthread/cs_queue-2.i ILP32 false +Floats neural-networks/softsign_w16_r2_case_0_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem18_label00.c ILP32 false +ProductLines product-lines/email_spec8_product14.cil.c ILP32 true +XCSP xcsp/aim-200-2-0-sat-2.c ILP32 false +LinkedLists memsafety-broom/sll-lst-shared.i LP64 unknown +Arrays array-crafted/xor1.i ILP32 true +BitVectors bitvector-regression/recHanoi03-1.c ILP32 false +ControlFlow infeasible-control-flow/unreach_branch3.c ILP32 true +Heap ldv-memsafety/memleaks_test11_2.i ILP32 unknown +Loops loop-invgen/half_2.i ILP32 true +Recursive fuzzle-programs/08_fuzzle_50x50_50-cycle.c LP64 false +Sequentialized seq-mthreaded/pals_STARTPALS_Triplicated.1.ufo.BOUNDED-10.pals.c ILP32 false +Floats float-benchs/sin_interpolated_negation.c ILP32 true +ECA eca-programs/Problem102_label41.c ILP32 true +ProductLines product-lines/elevator_spec1_product17.cil.c ILP32 true +XCSP xcsp/AllInterval-011.c ILP32 false +LinkedLists forester-heap/dll-optional-2.i ILP32 false +Arrays array-examples/standard_copy9_ground-2.i ILP32 true +BitVectors bitvector/gcd_2.c ILP32 true +ControlFlow ntdrivers-simplified/kbfiltr_simpl1.cil.c ILP32 unknown +Heap ldv-memsafety/memleaks_test22_3-2.i ILP32 unknown +Loops loop-zilu/benchmark39_conjunctive.i ILP32 unknown +Recursive recursified_loop-simple/recursified_nested_2.c ILP32 unknown +Sequentialized seq-mthreaded/pals_floodmax.4.3.ufo.BOUNDED-8.pals.c ILP32 false +Floats floats-cbmc-regression/float-flags-simp1.i ILP32 true +ECA eca-rers2012/Problem04_label47.c ILP32 true +ProductLines product-lines/elevator_spec2_product29.cil.c ILP32 true +XCSP xcsp/CostasArray-12.c ILP32 false +LinkedLists forester-heap/dll-token-2.i ILP32 false +Arrays array-fpi/eqn4.c ILP32 true +BitVectors bitvector/modulus-1.i ILP32 unknown +ControlFlow ntdrivers/kbfiltr.i.cil-2.c ILP32 false +Heap ldv-regression/test10.c ILP32 true +Loops loops/trex03-2.c ILP32 true +Recursive recursified_nla-digbench/recursified_egcd2-ll.c ILP32 unknown +Sequentialized seq-mthreaded/pals_lcr-var-start-time.3.2.ufo.BOUNDED-6.pals.c ILP32 false +Floats neural-networks/cartpole_31_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem07_label03.c ILP32 false +ProductLines product-lines/elevator_spec9_product30.cil.c ILP32 false +XCSP xcsp/Dubois-024.c ILP32 true +LinkedLists forester-heap/sll-rb-sentinel-1.i ILP32 true +Arrays array-fpi/s12iff.c ILP32 false +BitVectors bitvector/s3_clnt_2.BV.c.cil-1.c ILP32 unknown +ControlFlow openssl-simplified/s3_srvr_1.cil-1.c ILP32 unknown +Heap list-ext-properties/list-ext_2.i ILP32 unknown +Loops nla-digbench-scaling/egcd-ll_unwindbound20.c ILP32 false +Recursive recursified_nla-digbench/recursified_geo3-ll.c ILP32 unknown +Sequentialized seq-mthreaded/pals_lcr.4.1.ufo.BOUNDED-8.pals.c ILP32 false +Floats neural-networks/dubinsrejoin_2_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem09_label15.c ILP32 false +ProductLines product-lines/email_spec11_product23.cil.c ILP32 true +XCSP xcsp/Dubois-075.c ILP32 true +LinkedLists heap-manipulation/dll_of_dll-2.i ILP32 false +Arrays array-fpi/ss4f.c ILP32 false +BitVectors bitvector/s3_srvr_1.BV.c.cil.c ILP32 unknown +ControlFlow openssl-simplified/s3_srvr_6.cil-2.c ILP32 unknown +Heap memsafety-cve/gpac_15.i ILP32 unknown +Loops nla-digbench-scaling/geo1-ll_unwindbound50.c ILP32 true +Recursive recursified_nla-digbench/recursified_ps3-ll.c ILP32 unknown +Sequentialized seq-mthreaded/pals_lcr.8.ufo.BOUNDED-16.pals.c ILP32 true +Floats neural-networks/gelu_3_unsafe.c-amalgamation.i ILP32 false +ECA eca-rers2012/Problem11_label28.c ILP32 true +ProductLines product-lines/email_spec27_product19.cil.c ILP32 false +XCSP xcsp/Haystacks-15.c ILP32 true +LinkedLists list-ext3-properties/sll_length_check-2.i ILP32 true +Arrays array-memsafety/openbsd_cmemset-alloca-1.i ILP32 unknown +BitVectors bitvector/s3_srvr_3a.BV.c.cil.c ILP32 true +ControlFlow openssl/s3_srvr.blast.01.i.cil-2.c ILP32 unknown +Heap memsafety-cve/minizip-ng_1Fixed.i ILP32 unknown +Loops nla-digbench-scaling/hard-u_unwindbound100.c ILP32 false +Recursive recursive-simple/id_b3_o2-2.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.4.3.ufo.UNBOUNDED.pals.c ILP32 false +Floats neural-networks/lunarlander_88_unsafe.c-amalgamation.i ILP32 false +ECA eca-rers2012/Problem13_label40.c ILP32 false +ProductLines product-lines/email_spec4_product18.cil.c ILP32 false +XCSP xcsp/aim-100-2-0-sat-3.c ILP32 false +LinkedLists list-properties/simple_built_from_end.i ILP32 true +Arrays array-patterns/array22_pattern.c ILP32 true +BitVectors bitvector/soft_float_2a.c.cil.c ILP32 true +ControlFlow openssl/s3_srvr.blast.10.i.cil-1.c ILP32 unknown +Heap memsafety-cve/stb.i ILP32 unknown +Loops nla-digbench-scaling/prodbin-ll_valuebound100.c ILP32 true +Recursive recursive/Ackermann03.c ILP32 true +Sequentialized seq-pthread/cs_fib-2.i ILP32 false +Floats neural-networks/poly_8_8_8_thresh_2_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem16_label54.c ILP32 false +ProductLines product-lines/email_spec7_product17.cil.c ILP32 true +XCSP xcsp/aim-200-1-6-sat-2.c ILP32 false +LinkedLists memsafety-broom/linux-hlist-middle-data.i LP64 unknown +Arrays array-tiling/tcpy.c ILP32 true +BitVectors bitvector/soft_float_5.c.cil.c ILP32 unknown +ControlFlow openssl/s3_srvr.blast.16.i.cil-1.c ILP32 unknown +Heap memsafety/test-0102-2.i ILP32 unknown +Loops nla-digbench/dijkstra.c ILP32 unknown +Recursive recursive/Fibonacci05.c ILP32 false +Sequentialized seq-pthread/cs_sync.i ILP32 true +Floats neural-networks/tanh_6_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem19_label06.c ILP32 true +ProductLines product-lines/email_spec9_product12.cil.c ILP32 true +XCSP xcsp/aim-200-3-4-sat-1.c ILP32 false +LinkedLists memsafety-broom/sll-nested-sll.i LP64 unknown +Arrays array-crafted/bor1.i ILP32 true BitVectors bitvector-loops/diamond_2-1.c ILP32 false +ControlFlow infeasible-control-flow/conflict_branch3.c ILP32 true +Heap heap-data/shared_mem1.i ILP32 true +Loops loop-industry-pattern/aiob_2.c ILP32 true +Recursive fuzzle-programs/04_fuzzle_60x60.c LP64 false +Sequentialized seq-mthreaded/pals_STARTPALS_ActiveStandby.4_2.ufo.BOUNDED-10.pals.c ILP32 false +Floats float-benchs/interpolation.c ILP32 true +ECA eca-programs/Problem102_label08.c ILP32 true +ProductLines product-lines/elevator_spec14_product28.cil.c ILP32 false +XCSP xcsp/AllInterval-007.c ILP32 false +LinkedLists forester-heap/dll-circular-1.i ILP32 false +Arrays array-crafted/zero_sum_const_m2.c ILP32 true +BitVectors bitvector/byte_add-2.i ILP32 unknown +ControlFlow memory-model/2SB.i ILP32 false +Heap ldv-memsafety/memleaks_test14_3.i ILP32 unknown +Loops loop-lit/gj2007.i ILP32 true +Recursive fuzzle-programs/12_fuzzle_50x50_CVE-2016-4492.c LP64 false +Sequentialized seq-mthreaded/pals_floodmax.3.1.ufo.BOUNDED-6.pals.c ILP32 false +Floats float-newlib/double_req_bl_0670.c ILP32 true +ECA eca-programs/Problem103_label52.c ILP32 true +ProductLines product-lines/elevator_spec1_product25.cil.c ILP32 true +XCSP xcsp/AllInterval-015.c ILP32 false +LinkedLists forester-heap/dll-rb-cnstr_1-1.i ILP32 true +Arrays array-examples/standard_copy2_ground-1.i ILP32 false +BitVectors bitvector/byte_add_2-2.i ILP32 true +ControlFlow ntdrivers-simplified/floppy_simpl3.cil-2.c ILP32 unknown +Heap ldv-memsafety/memleaks_test20-1.i ILP32 unknown +Loops loop-zilu/benchmark17_conjunctive.i ILP32 true +Recursive recursified_loop-invariants/recursified_linear-inequality-inv-a.c ILP32 unknown +Sequentialized seq-mthreaded/pals_floodmax.3_overflow.ufo.UNBOUNDED.pals.c ILP32 false +Floats float-newlib/float_req_bl_0875.c ILP32 true +ECA eca-rers2012/Problem03_label58.c ILP32 true +ProductLines product-lines/elevator_spec2_product21.cil.c ILP32 true +XCSP xcsp/AllInterval-030.c ILP32 false +LinkedLists forester-heap/dll-sorted-1.i ILP32 false +Arrays array-examples/standard_running-2.i ILP32 true +BitVectors bitvector/interleave_bits.i ILP32 true +ControlFlow ntdrivers/cdaudio.i.cil-1.c ILP32 false +Heap ldv-memsafety/memleaks_test4-1.i ILP32 unknown +Loops loops-crafted-1/in-de52.c ILP32 true +Recursive recursified_loop-simple/recursified_nested_6.c ILP32 unknown +Sequentialized seq-mthreaded/pals_floodmax.4.ufo.BOUNDED-8.pals.c ILP32 true +Floats floats-cdfpl/newton_2_6.i ILP32 false +ECA eca-rers2012/Problem05_label23.c ILP32 true +ProductLines product-lines/elevator_spec3_product11.cil.c ILP32 false +XCSP xcsp/CostasArray-16.c ILP32 false +LinkedLists forester-heap/sll-buckets-1.i ILP32 true +Arrays array-fpi/brs5f_overflow.c ILP32 unknown +BitVectors bitvector/jain_6-2.c ILP32 unknown +ControlFlow ntdrivers/floppy.i.cil-3.c ILP32 true +Heap ldv-regression/alias_of_return_2.c_1.i ILP32 true +Loops loops/s3.i ILP32 unknown +Recursive recursified_nla-digbench/recursified_divbin.i ILP32 unknown +Sequentialized seq-mthreaded/pals_floodmax.5.ufo.BOUNDED-10.pals.c ILP32 true +Floats loop-floats-scientific-comp/loop3.c ILP32 true +ECA eca-rers2012/Problem06_label30.c ILP32 true +ProductLines product-lines/elevator_spec3_productSimulator.cil.c ILP32 false +XCSP xcsp/Dubois-020.c ILP32 true +LinkedLists forester-heap/sll-queue-2.i ILP32 false +Arrays array-fpi/ifeqn5f.c ILP32 false +BitVectors bitvector/num_conversion_2.c ILP32 true +ControlFlow openssl-simplified/s3_clnt_1.cil-1.c ILP32 unknown +Heap ldv-regression/test22-3.c ILP32 false +Loops nla-digbench-scaling/cohencu-ll_unwindbound100.c ILP32 false +Recursive recursified_nla-digbench/recursified_fermat1-ll.c ILP32 unknown +Sequentialized seq-mthreaded/pals_lcr-var-start-time.4.2.ufo.BOUNDED-8.pals.c ILP32 false +Floats neural-networks/cartpole_55_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem07_label36.c ILP32 false +ProductLines product-lines/email_spec0_product16.cil.c ILP32 false +XCSP xcsp/Dubois-028.c ILP32 true +LinkedLists forester-heap/sll-simple-white-blue-1.i ILP32 true +Arrays array-fpi/nsqm-if.c ILP32 true +BitVectors bitvector/s3_clnt_1.BV.c.cil-2.c ILP32 unknown +ControlFlow openssl-simplified/s3_clnt_3.cil-3.c ILP32 unknown +Heap ldv-sets/test_mutex_double_unlock.i ILP32 false +Loops nla-digbench-scaling/divbin2_valuebound10.i ILP32 true +Recursive recursified_nla-digbench/recursified_geo1.c ILP32 unknown +Sequentialized seq-mthreaded/pals_lcr-var-start-time.6.ufo.BOUNDED-12.pals.c ILP32 true +Floats neural-networks/cos_3_unsafe.c-amalgamation.i ILP32 false +ECA eca-rers2012/Problem08_label42.c ILP32 true +ProductLines product-lines/email_spec11_product03.cil.c ILP32 true +XCSP xcsp/Dubois-055.c ILP32 true +LinkedLists heap-manipulation/bubble_sort_linux-2.i ILP32 false +Arrays array-fpi/s3lif.c ILP32 true +BitVectors bitvector/s3_clnt_2.BV.c.cil-2.c ILP32 unknown +ControlFlow openssl-simplified/s3_srvr_11.cil.c ILP32 unknown +Heap list-ext-properties/test-0217_1.i ILP32 unknown +Loops nla-digbench-scaling/egcd3-ll_unwindbound1.c ILP32 false +Recursive recursified_nla-digbench/recursified_hard-u.c ILP32 unknown +Sequentialized seq-mthreaded/pals_lcr.5.1.ufo.BOUNDED-10.pals.c ILP32 false +Floats neural-networks/dubinsrejoin_54_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem09_label48.c ILP32 true +ProductLines product-lines/email_spec11_product36.cil.c ILP32 true +XCSP xcsp/Dubois-095.c ILP32 true +LinkedLists heap-manipulation/sll_to_dll_rev-1.i ILP32 false +Arrays array-fpi/sina4.c ILP32 true +BitVectors bitvector/s3_clnt_3.BV.c.cil-2.c ILP32 unknown +ControlFlow openssl-simplified/s3_srvr_3.cil.c ILP32 unknown +Heap memsafety-cve/dlt-daemon.i ILP32 unknown +Loops nla-digbench-scaling/freire1_valuebound50.c ILP32 true +Recursive recursified_nla-digbench/recursified_prodbin-ll.c ILP32 unknown +Sequentialized seq-mthreaded/pals_lcr.7.ufo.BOUNDED-14.pals.c ILP32 true +Floats neural-networks/erf_1_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem10_label55.c ILP32 false +ProductLines product-lines/email_spec1_product32.cil.c ILP32 false +XCSP xcsp/Haystacks-11.c ILP32 true +LinkedLists list-ext3-properties/dll_nondet_free_order-2.i ILP32 unknown +Arrays array-lopstr16/base_case.i ILP32 true +BitVectors bitvector/s3_srvr_1a.BV.c.cil.c ILP32 true +ControlFlow openssl/s3_clnt.blast.01.i.cil-1.c ILP32 unknown +Heap memsafety-cve/gpac_5.i ILP32 unknown +Loops nla-digbench-scaling/geo1-u_unwindbound50.c ILP32 false +Recursive recursified_nla-digbench/recursified_ps5-ll.c ILP32 unknown +Sequentialized seq-mthreaded/pals_opt-floodmax.3.2.ufo.BOUNDED-6.pals.c ILP32 false +Floats neural-networks/lunarlander_16_unsafe.c-amalgamation.i ILP32 false +ECA eca-rers2012/Problem12_label01.c ILP32 true +ProductLines product-lines/email_spec27_product31.cil.c ILP32 false +XCSP xcsp/Haystacks-19.c ILP32 true +LinkedLists list-ext3-properties/sll_of_sll_nondet_append-1.i ILP32 true +Arrays array-memsafety/cstrpbrk-alloca-2.i ILP32 unknown +BitVectors bitvector/s3_srvr_3.BV.c.cil.c ILP32 unknown +ControlFlow openssl/s3_clnt.blast.04.i.cil-1.c ILP32 unknown +Heap memsafety-cve/libpe.i ILP32 unknown +Loops nla-digbench-scaling/geo3-ll_unwindbound50.c ILP32 true +Recursive recursive-simple/id2_b3_o5.c ILP32 true +Sequentialized seq-mthreaded/pals_opt-floodmax.4.1.ufo.BOUNDED-8.pals.c ILP32 false +Floats neural-networks/lunarlander_63_unsafe.c-amalgamation.i ILP32 false +ECA eca-rers2012/Problem13_label07.c ILP32 false +ProductLines product-lines/email_spec3_product31.cil.c ILP32 false +XCSP xcsp/aim-100-1-6-unsat-3.c ILP32 true +LinkedLists list-properties/list_flag-2.i ILP32 true +Arrays array-memsafety/openbsd_cstrstr-alloca-2.i ILP32 unknown +BitVectors bitvector/soft_float_1-1.c.cil.c ILP32 unknown +ControlFlow openssl/s3_srvr.blast.03.i.cil.c ILP32 unknown +Heap memsafety-cve/openNDS_1.i ILP32 unknown +Loops nla-digbench-scaling/hard2_valuebound5.c ILP32 true +Recursive recursive-simple/id_b5_o10-2.c ILP32 true +Sequentialized seq-mthreaded/pals_opt-floodmax.4_overflow.ufo.UNBOUNDED.pals.c ILP32 false +Floats neural-networks/poly_128_thresh_0_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem15_label13.c ILP32 true +ProductLines product-lines/email_spec4_product30.cil.c ILP32 false +XCSP xcsp/aim-100-2-0-unsat-3.c ILP32 true +LinkedLists memsafety-broom/dll-backwards.i LP64 unknown +Arrays array-multidimensional/transpose-u.c ILP32 true +BitVectors bitvector/soft_float_1-3a.c.cil.c ILP32 false +ControlFlow openssl/s3_srvr.blast.08.i.cil-2.c ILP32 unknown +Heap memsafety-cve/radare2_2Fixed.i ILP32 unknown +Loops nla-digbench-scaling/mannadiv_unwindbound50.c ILP32 false +Recursive recursive-simple/sum_non_eq-2.c ILP32 true +Sequentialized seq-mthreaded/pals_opt-floodmax.5_overflow.ufo.UNBOUNDED.pals.c ILP32 false +Floats neural-networks/poly_4_4_4_4_thresh_5_unsafe.c-amalgamation.i ILP32 false +ECA eca-rers2012/Problem16_label21.c ILP32 true +ProductLines product-lines/email_spec6_product30.cil.c ILP32 false +XCSP xcsp/aim-100-6-0-sat-2.c ILP32 false +LinkedLists memsafety-broom/dll.i LP64 unknown +Arrays array-patterns/array5_pattern.c ILP32 true +BitVectors bitvector/soft_float_3a.c.cil.c ILP32 true +ControlFlow openssl/s3_srvr.blast.11.i.cil-2.c ILP32 unknown +Heap memsafety-ext/dll_extends_pointer.i ILP32 unknown +Loops nla-digbench-scaling/ps3-ll_unwindbound5.c ILP32 true +Recursive recursive/Addition02WithOverflowBug.c ILP32 unknown +Sequentialized seq-pthread/cs_lamport.i ILP32 true +Floats neural-networks/softmax_10_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem17_label27.c ILP32 true +ProductLines product-lines/email_spec7_product29.cil.c ILP32 false +XCSP xcsp/aim-200-1-6-unsat-2.c ILP32 true +LinkedLists memsafety-broom/linux-list-middle-data.i LP64 unknown +Arrays array-tiling/pnr4.c ILP32 true +BitVectors bitvector/soft_float_4-3.c.cil.c ILP32 unknown +ControlFlow openssl/s3_srvr.blast.14.i.cil-2.c ILP32 unknown +Heap memsafety/global-atexit-2.i ILP32 unknown +Loops nla-digbench-scaling/ps6-ll_valuebound20.c ILP32 true +Recursive recursive/Fibonacci03.c ILP32 true +Sequentialized seq-pthread/cs_stack-1.i ILP32 true +Floats neural-networks/softsign_w64_r3_case_0_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem18_label33.c ILP32 false +ProductLines product-lines/email_spec8_product29.cil.c ILP32 true +XCSP xcsp/aim-200-2-0-unsat-1.c ILP32 true +LinkedLists memsafety-broom/sll-nested-linux-list.i LP64 unknown +Arrays reducercommutativity/rangesum10.i ILP32 false +BitVectors bitvector/sum02-1.c ILP32 false +ControlFlow unsignedintegeroverflow-sas23/cancel_var_through_overflow.i ILP32 true +Heap memsafety/test-0234-2.i ILP32 unknown +Loops nla-digbench/hard-ll.c ILP32 true +Recursive recursive/gcd01-1.c ILP32 true +Sequentialized systemc/token_ring.01.cil-1.c ILP32 unknown +Floats neural-networks/tanh_w64_r1_case_1_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem19_label39.c ILP32 true +ProductLines product-lines/email_spec9_product28.cil.c ILP32 true +XCSP xcsp/aim-200-6-0-sat-1.c ILP32 false +LinkedLists memsafety-broom/sll-shared-sll-after.i LP64 unknown +Arrays array-cav19/array_min_and_copy_shift_sum_add.c ILP32 true BitVectors bitvector-loops/verisec_sendmail_tTflag_arr_one_loop.c ILP32 false +ControlFlow infeasible-control-flow/conflict_branch1.c ILP32 true +Heap heap-data/min_max.i ILP32 true +Loops loop-acceleration/multivar_1-2.c ILP32 false +Recursive fuzzle-programs/02_fuzzle_40x40.c LP64 false +Sequentialized seq-mthreaded/pals_STARTPALS_ActiveStandby.1.ufo.BOUNDED-10.pals.c ILP32 false +Floats float-benchs/filter1.c.v+lhb-reducer.c ILP32 true +ECA eca-programs/Problem101_label15.c ILP32 false +ProductLines product-lines/elevator_spec14_product20.cil.c ILP32 false +XCSP xcsp/AllInterval-005.c ILP32 false +LinkedLists forester-heap/dll-01-1.i ILP32 false +Arrays array-crafted/mapavg3.i ILP32 true BitVectors bitvector/byte_add-1.i ILP32 false -BitVectors bitvector/byte_add-2.i ILP32 unknown -BitVectors bitvector/byte_add_1-2.i ILP32 unknown +ControlFlow infeasible-control-flow/unreach_branch1.c ILP32 true +Heap ldv-memsafety/memleaks_test1-3.i ILP32 unknown +Loops loop-invgen/NetBSD_loop.i ILP32 true +Recursive fuzzle-programs/06_fuzzle_50x50_0-cycle.c LP64 false +Sequentialized seq-mthreaded/pals_STARTPALS_ActiveStandby.5.ufo.BOUNDED-10.pals.c ILP32 false +Floats float-benchs/mea8000.c ILP32 true +ECA eca-programs/Problem102_label24.c ILP32 false +ProductLines product-lines/elevator_spec1_product01.cil.c ILP32 true +XCSP xcsp/AllInterval-009.c ILP32 false +LinkedLists forester-heap/dll-circular-2.i ILP32 true +Arrays array-crafted/zero_sum3.c ILP32 true +BitVectors bitvector/byte_add_1-1.i ILP32 true +ControlFlow infeasible-control-flow/unreach_branch4.c ILP32 true +Heap ldv-memsafety/memleaks_test13_1.i ILP32 unknown +Loops loop-invgen/seq-3.i ILP32 true +Recursive fuzzle-programs/10_fuzzle_50x50_CVE-2016-4487.c LP64 false +Sequentialized seq-mthreaded/pals_STARTPALS_Triplicated.2.ufo.BOUNDED-10.pals.c ILP32 false +Floats float-benchs/zonotope_loose.c ILP32 true +ECA eca-programs/Problem102_label57.c ILP32 true +ProductLines product-lines/elevator_spec1_product21.cil.c ILP32 true +XCSP xcsp/AllInterval-013.c ILP32 false +LinkedLists forester-heap/dll-queue-1.i ILP32 true +Arrays array-examples/data_structures_set_multi_proc_ground-1.i ILP32 false BitVectors bitvector/byte_add_2-1.i ILP32 unknown +ControlFlow memory-model/4SB.i ILP32 false +Heap ldv-memsafety/memleaks_test16_1.i ILP32 unknown +Loops loop-new/count_by_1.i ILP32 true +Recursive fuzzle-programs/14_fuzzle_50x50_CVE-2016-6131.c LP64 false +Sequentialized seq-mthreaded/pals_floodmax.3.2.ufo.UNBOUNDED.pals.c ILP32 false +Floats float-newlib/double_req_bl_0876.c ILP32 true +ECA eca-rers2012/Problem03_label08.c ILP32 true +ProductLines product-lines/elevator_spec1_product29.cil.c ILP32 true +XCSP xcsp/AllInterval-017.c ILP32 false +LinkedLists forester-heap/dll-rb-sentinel-1.i ILP32 false +Arrays array-examples/sorting_selectionsort_2_ground.i ILP32 false BitVectors bitvector/gcd_1.c ILP32 true -BitVectors bitvector/gcd_2.c ILP32 true -BitVectors bitvector/interleave_bits.i ILP32 true +ControlFlow ntdrivers-simplified/diskperf_simpl1.cil.c ILP32 unknown +Heap ldv-memsafety/memleaks_test18_2.i ILP32 unknown +Loops loop-zilu/benchmark06_conjunctive.i ILP32 unknown +Recursive recursified_loop-crafted/recursified_simple_array_index_value_4.i.v+nlh-reducer.c ILP32 unknown +Sequentialized seq-mthreaded/pals_floodmax.3.4.ufo.BOUNDED-6.pals.c ILP32 false +Floats float-newlib/float_req_bl_0670.c ILP32 true +ECA eca-rers2012/Problem03_label41.c ILP32 true +ProductLines product-lines/elevator_spec2_product17.cil.c ILP32 true +XCSP xcsp/AllInterval-020.c ILP32 false +LinkedLists forester-heap/dll-simple-white-blue-1.i ILP32 false +Arrays array-examples/standard_copy5_ground-2.i ILP32 false +BitVectors bitvector/gcd_3.c ILP32 true +ControlFlow ntdrivers-simplified/floppy_simpl4.cil-1.c ILP32 unknown +Heap ldv-memsafety/memleaks_test22_1-1.i ILP32 unknown +Loops loop-zilu/benchmark27_linear.i ILP32 unknown +Recursive recursified_loop-simple/recursified_nested_1.c ILP32 unknown +Sequentialized seq-mthreaded/pals_floodmax.4.2.ufo.BOUNDED-8.pals.c ILP32 false +Floats float-newlib/float_req_bl_1130b.c ILP32 true +ECA eca-rers2012/Problem04_label18.c ILP32 false +ProductLines product-lines/elevator_spec2_product25.cil.c ILP32 true +XCSP xcsp/CostasArray-10.c ILP32 false +LinkedLists forester-heap/dll-sorted-2.i ILP32 true +Arrays array-examples/standard_palindrome_ground.i ILP32 true BitVectors bitvector/jain_1-2.c ILP32 unknown +ControlFlow ntdrivers-simplified/kbfiltr_simpl2.cil-1.c ILP32 unknown +Heap ldv-memsafety/memleaks_test23_2.i ILP32 unknown +Loops loop-zilu/benchmark50_linear.i ILP32 unknown +Recursive recursified_loop-simple/recursified_nested_4.c ILP32 unknown +Sequentialized seq-mthreaded/pals_floodmax.4.3.ufo.UNBOUNDED.pals.c ILP32 false +Floats floats-cbmc-regression/float8.i ILP32 true +ECA eca-rers2012/Problem05_label06.c ILP32 true +ProductLines product-lines/elevator_spec2_productSimulator.cil.c ILP32 false +XCSP xcsp/CostasArray-14.c ILP32 false +LinkedLists forester-heap/sll-01-1.i ILP32 true +Arrays array-examples/standard_strcpy_original-2.i ILP32 true BitVectors bitvector/jain_4-1.c ILP32 unknown +ControlFlow ntdrivers/cdaudio.i.cil-2.c ILP32 unknown +Heap ldv-memsafety/memleaks_test5_2.i ILP32 unknown +Loops loops-crafted-1/sumt2.c ILP32 true +Recursive recursified_nla-digbench/recursified_bresenham.c ILP32 unknown +Sequentialized seq-mthreaded/pals_floodmax.5.1.ufo.BOUNDED-10.pals.c ILP32 false +Floats floats-cdfpl/sine_3.i ILP32 false +ECA eca-rers2012/Problem05_label40.c ILP32 false +ProductLines product-lines/elevator_spec3_product20.cil.c ILP32 false +XCSP xcsp/Dubois-015.c ILP32 true +LinkedLists forester-heap/sll-circular-1.i ILP32 true +Arrays array-fpi/brs2f.c ILP32 false BitVectors bitvector/jain_7-1.c ILP32 unknown -BitVectors bitvector/modulus-1.i ILP32 unknown -BitVectors bitvector/num_conversion_2.c ILP32 true +ControlFlow ntdrivers/floppy.i.cil-1.c ILP32 false +Heap ldv-memsafety/memleaks_test8_2.i ILP32 unknown +Loops loops/invert_string-1.c ILP32 false +Recursive recursified_nla-digbench/recursified_dijkstra-u.c ILP32 unknown +Sequentialized seq-mthreaded/pals_floodmax.5.3.ufo.BOUNDED-10.pals.c ILP32 false +Floats floats-nonlinear/coral-benchmark01.c ILP32 false +ECA eca-rers2012/Problem06_label13.c ILP32 true +ProductLines product-lines/elevator_spec3_product29.cil.c ILP32 true +XCSP xcsp/Dubois-018.c ILP32 true +LinkedLists forester-heap/sll-optional-2.i ILP32 false +Arrays array-fpi/condn.c ILP32 true +BitVectors bitvector/modulus-2.i ILP32 true +ControlFlow ntdrivers/floppy2.i.cil.c ILP32 unknown +Heap ldv-regression/rule57_ebda_blast.i ILP32 true +Loops loops/terminator_01.c ILP32 false +Recursive recursified_nla-digbench/recursified_egcd-ll.c ILP32 unknown +Sequentialized seq-mthreaded/pals_floodmax.5_overflow.ufo.UNBOUNDED.pals.c ILP32 false +Floats neural-networks/cartpole_19_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem06_label46.c ILP32 true +ProductLines product-lines/elevator_spec9_product26.cil.c ILP32 false +XCSP xcsp/Dubois-022.c ILP32 true +LinkedLists forester-heap/sll-rb-cnstr_1-1.i ILP32 true +Arrays array-fpi/ifeqn1f.c ILP32 false BitVectors bitvector/parity.i ILP32 true +ControlFlow ntdrivers/parport.i.cil-1.c ILP32 false +Heap ldv-regression/test19.c ILP32 true +Loops nla-digbench-scaling/bresenham-ll_unwindbound20.c ILP32 false +Recursive recursified_nla-digbench/recursified_egcd3-ll.c ILP32 unknown +Sequentialized seq-mthreaded/pals_lcr-var-start-time.3.ufo.BOUNDED-6.pals.c ILP32 true +Floats neural-networks/cartpole_43_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem07_label19.c ILP32 false +ProductLines product-lines/email_spec0_product05.cil.c ILP32 true +XCSP xcsp/Dubois-026.c ILP32 true +LinkedLists forester-heap/sll-rb-sentinel-2.i ILP32 false +Arrays array-fpi/indp3f.c ILP32 false BitVectors bitvector/s3_clnt_1.BV.c.cil-1a.c ILP32 unknown -BitVectors bitvector/s3_clnt_1.BV.c.cil-2.c ILP32 unknown -BitVectors bitvector/s3_clnt_2.BV.c.cil-1.c ILP32 unknown +ControlFlow openssl-simplified/s3_clnt_1.cil-2.c ILP32 unknown +Heap ldv-regression/test24-2.c ILP32 false +Loops nla-digbench-scaling/cohendiv-ll_unwindbound1.c ILP32 true +Recursive recursified_nla-digbench/recursified_fermat2-ll.c ILP32 unknown +Sequentialized seq-mthreaded/pals_lcr-var-start-time.5.1.ufo.BOUNDED-10.pals.c ILP32 false +Floats neural-networks/cartpole_67_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem07_label52.c ILP32 true +ProductLines product-lines/email_spec0_product24.cil.c ILP32 true +XCSP xcsp/Dubois-030.c ILP32 true +LinkedLists forester-heap/sll-sorted-1.i ILP32 false +Arrays array-fpi/ms3.c ILP32 true +BitVectors bitvector/s3_clnt_1.BV.c.cil-2a.c ILP32 unknown +ControlFlow openssl-simplified/s3_clnt_3.cil-1.c ILP32 unknown +Heap ldv-sets/test_add-1.i ILP32 false +Loops nla-digbench-scaling/dijkstra-u_valuebound2.c ILP32 true +Recursive recursified_nla-digbench/recursified_geo1-u.c ILP32 unknown +Sequentialized seq-mthreaded/pals_lcr-var-start-time.6.1.ufo.BOUNDED-12.pals.c ILP32 false +Floats neural-networks/cartpole_91_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem08_label25.c ILP32 false +ProductLines product-lines/email_spec0_product37.cil.c ILP32 true +XCSP xcsp/Dubois-045.c ILP32 true +LinkedLists forester-heap/sll-token-2.i ILP32 true +Arrays array-fpi/res1f.c ILP32 false BitVectors bitvector/s3_clnt_2.BV.c.cil-1a.c ILP32 true +ControlFlow openssl-simplified/s3_clnt_4.cil-1.c ILP32 unknown +Heap list-ext-properties/960521-1_1-1.i ILP32 unknown +Loops nla-digbench-scaling/divbin_unwindbound50.i ILP32 false +Recursive recursified_nla-digbench/recursified_geo2-ll2.c ILP32 unknown +Sequentialized seq-mthreaded/pals_lcr.3.ufo.BOUNDED-6.pals.c ILP32 true +Floats neural-networks/dubinsrejoin_17_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem08_label58.c ILP32 true +ProductLines product-lines/email_spec11_product15.cil.c ILP32 false +XCSP xcsp/Dubois-065.c ILP32 true +LinkedLists heap-manipulation/dancing.i ILP32 true +Arrays array-fpi/s2if.c ILP32 true BitVectors bitvector/s3_clnt_2.BV.c.cil-2a.c ILP32 false +ControlFlow openssl-simplified/s3_srvr_1.cil-2.c ILP32 unknown +Heap list-ext-properties/simple-ext_1.i ILP32 unknown +Loops nla-digbench-scaling/egcd2-ll_unwindbound100.c ILP32 false +Recursive recursified_nla-digbench/recursified_geo3.c ILP32 unknown +Sequentialized seq-mthreaded/pals_lcr.4.ufo.BOUNDED-8.pals.c ILP32 true +Floats neural-networks/dubinsrejoin_41_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem09_label31.c ILP32 true +ProductLines product-lines/email_spec11_product30.cil.c ILP32 false +XCSP xcsp/Dubois-085.c ILP32 true +LinkedLists heap-manipulation/merge_sort-1.i ILP32 false +Arrays array-fpi/s4iff_overflow.c ILP32 unknown BitVectors bitvector/s3_clnt_3.BV.c.cil-1a.c ILP32 true -BitVectors bitvector/s3_clnt_3.BV.c.cil-2.c ILP32 unknown -BitVectors bitvector/s3_srvr_1.BV.c.cil.c ILP32 unknown +ControlFlow openssl-simplified/s3_srvr_13.cil.c ILP32 unknown +Heap list-ext-properties/test-0513_1.i ILP32 unknown +Loops nla-digbench-scaling/egcd3-ll_valuebound5.c ILP32 true +Recursive recursified_nla-digbench/recursified_hard2.c ILP32 unknown +Sequentialized seq-mthreaded/pals_lcr.5_overflow.ufo.UNBOUNDED.pals.c ILP32 false +Floats neural-networks/dubinsrejoin_66_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem10_label05.c ILP32 true +ProductLines product-lines/email_spec11_productSimulator.cil.c ILP32 false +XCSP xcsp/Haystacks-06.c ILP32 true +LinkedLists heap-manipulation/tree-1.i ILP32 unknown +Arrays array-fpi/s5liff.c ILP32 false +BitVectors bitvector/s3_clnt_3.BV.c.cil-2a.c ILP32 false +ControlFlow openssl-simplified/s3_srvr_2.cil-1.c ILP32 unknown +Heap memsafety-cve/asterisk.i ILP32 unknown +Loops nla-digbench-scaling/fermat2-ll_valuebound10.c ILP32 true +Recursive recursified_nla-digbench/recursified_prod4br-ll.c ILP32 unknown +Sequentialized seq-mthreaded/pals_lcr.6_overflow.ufo.UNBOUNDED.pals.c ILP32 false +Floats neural-networks/dubinsrejoin_90_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem10_label38.c ILP32 true +ProductLines product-lines/email_spec1_product28.cil.c ILP32 true +XCSP xcsp/Haystacks-09.c ILP32 true +LinkedLists heap-manipulation/tree-4.i ILP32 false +Arrays array-fpi/sqmf.c ILP32 false BitVectors bitvector/s3_srvr_1_alt.BV.c.cil.c ILP32 unknown +ControlFlow openssl-simplified/s3_srvr_4.cil.c ILP32 unknown +Heap memsafety-cve/frrFixed.i ILP32 unknown +Loops nla-digbench-scaling/geo1-ll2_unwindbound20.c ILP32 unknown +Recursive recursified_nla-digbench/recursified_ps2-ll.c ILP32 unknown +Sequentialized seq-mthreaded/pals_lcr.7_overflow.ufo.UNBOUNDED.pals.c ILP32 false +Floats neural-networks/gcas_10_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem11_label11.c ILP32 true +ProductLines product-lines/email_spec1_productSimulator.cil.c ILP32 false +XCSP xcsp/Haystacks-13.c ILP32 true +LinkedLists list-ext3-properties/dll_nullified-1.i ILP32 false +Arrays array-industry-pattern/array_of_struct_ptr_monotonic.i ILP32 true BitVectors bitvector/s3_srvr_1a_alt.BV.c.cil.c ILP32 unknown -BitVectors bitvector/s3_srvr_2.BV.c.cil.c ILP32 unknown -BitVectors bitvector/s3_srvr_3.BV.c.cil.c ILP32 unknown +ControlFlow openssl-simplified/s3_srvr_7.cil.c ILP32 unknown +Heap memsafety-cve/gpac_17.i ILP32 unknown +Loops nla-digbench-scaling/geo1-u2_unwindbound20.c ILP32 unknown +Recursive recursified_nla-digbench/recursified_ps4-ll.c ILP32 unknown +Sequentialized seq-mthreaded/pals_lcr.8_overflow.ufo.UNBOUNDED.pals.c ILP32 false +Floats neural-networks/logistic_2_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem11_label44.c ILP32 true +ProductLines product-lines/email_spec27_product27.cil.c ILP32 false +XCSP xcsp/Haystacks-17.c ILP32 true +LinkedLists list-ext3-properties/sll_nondet_insert-1.i ILP32 false +Arrays array-memsafety-realloc/array-realloc-3.i ILP32 unknown +BitVectors bitvector/s3_srvr_2_alt.BV.c.cil.c ILP32 unknown +ControlFlow openssl/s3_clnt.blast.01.i.cil-2.c ILP32 unknown +Heap memsafety-cve/hyperkit_1.i ILP32 unknown +Loops nla-digbench-scaling/geo2-ll2_unwindbound20.c ILP32 unknown +Recursive recursified_nla-digbench/recursified_ps6-ll.c ILP32 unknown +Sequentialized seq-mthreaded/pals_opt-floodmax.3.3.ufo.BOUNDED-6.pals.c ILP32 false +Floats neural-networks/lunarlander_27_unsafe.c-amalgamation.i ILP32 false +ECA eca-rers2012/Problem12_label17.c ILP32 true +ProductLines product-lines/email_spec27_product35.cil.c ILP32 false +XCSP xcsp/aim-100-1-6-sat-2.c ILP32 false +LinkedLists list-properties/alternating_list-1.i ILP32 true +Arrays array-memsafety/cstrcspn-alloca-1.i ILP32 unknown BitVectors bitvector/s3_srvr_3_alt.BV.c.cil.c ILP32 unknown +ControlFlow openssl/s3_clnt.blast.03.i.cil-1.c ILP32 unknown +Heap memsafety-cve/krb5Fixed.i ILP32 unknown +Loops nla-digbench-scaling/geo3-ll2_unwindbound20.c ILP32 unknown +Recursive recursive-simple/id2_b2_o3.c ILP32 true +Sequentialized seq-mthreaded/pals_opt-floodmax.3.ufo.BOUNDED-6.pals.c ILP32 true +Floats neural-networks/lunarlander_51_unsafe.c-amalgamation.i ILP32 false +ECA eca-rers2012/Problem12_label50.c ILP32 false +ProductLines product-lines/email_spec3_product27.cil.c ILP32 false +XCSP xcsp/aim-100-1-6-unsat-1.c ILP32 true +LinkedLists list-properties/list-2.i ILP32 false +Arrays array-memsafety/knapsack_alloca_unsafe.i ILP32 unknown BitVectors bitvector/s3_srvr_3a_alt.BV.c.cil.c ILP32 true -BitVectors bitvector/soft_float_1-1.c.cil.c ILP32 unknown -BitVectors bitvector/soft_float_1-2a.c.cil.c ILP32 true -BitVectors bitvector/soft_float_1-3a.c.cil.c ILP32 false +ControlFlow openssl/s3_clnt.blast.04.i.cil-2.c ILP32 unknown +Heap memsafety-cve/libsndfile.i ILP32 unknown +Loops nla-digbench-scaling/hard-ll_unwindbound20.c ILP32 false +Recursive recursive-simple/id_b2_o3.c ILP32 true +Sequentialized seq-mthreaded/pals_opt-floodmax.4.2.ufo.UNBOUNDED.pals.c ILP32 false +Floats neural-networks/lunarlander_75_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem13_label23.c ILP32 false +ProductLines product-lines/email_spec3_product35.cil.c ILP32 false +XCSP xcsp/aim-100-2-0-sat-1.c ILP32 false +LinkedLists list-properties/simple-1.i ILP32 false +Arrays array-memsafety/openbsd_cstrlcpy-alloca-1.i ILP32 unknown +BitVectors bitvector/soft_float_1-2.c.cil.c ILP32 unknown +ControlFlow openssl/s3_srvr.blast.02.i.cil-1.c ILP32 unknown +Heap memsafety-cve/mujs.i ILP32 unknown +Loops nla-digbench-scaling/hard2_unwindbound1.c ILP32 false +Recursive recursive-simple/id_b3_o5-2.c ILP32 true +Sequentialized seq-mthreaded/pals_opt-floodmax.4.4.ufo.BOUNDED-8.pals.c ILP32 false +Floats neural-networks/lunarlander_99_unsafe.c-amalgamation.i ILP32 false +ECA eca-rers2012/Problem13_label56.c ILP32 true +ProductLines product-lines/email_spec4_product25.cil.c ILP32 false +XCSP xcsp/aim-100-2-0-unsat-1.c ILP32 true +LinkedLists list-properties/splice-1.i ILP32 false +Arrays array-memsafety/stroeder2-alloca-1.i ILP32 unknown +BitVectors bitvector/soft_float_1-3.c.cil.c ILP32 unknown +ControlFlow openssl/s3_srvr.blast.06.i.cil-1.c ILP32 unknown +Heap memsafety-cve/openrazer.i ILP32 unknown +Loops nla-digbench-scaling/knuth_valuebound5.i ILP32 true +Recursive recursive-simple/id_o100.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.5.2.ufo.BOUNDED-10.pals.c ILP32 false +Floats neural-networks/poly_16_16_16_thresh_1_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem15_label31.c ILP32 true +ProductLines product-lines/email_spec4_product34.cil.c ILP32 false +XCSP xcsp/aim-100-3-4-sat-1.c ILP32 false +LinkedLists memsafety-broom/dll-fst-shared.i LP64 unknown +Arrays array-multidimensional/init-non-constant-2-n-u.c ILP32 true BitVectors bitvector/soft_float_2.c.cil.c ILP32 unknown +ControlFlow openssl/s3_srvr.blast.07.i.cil-2.c ILP32 unknown +Heap memsafety-cve/radare2Fixed.i ILP32 unknown +Loops nla-digbench-scaling/lcm2_valuebound10.c ILP32 true +Recursive recursive-simple/sum_non.c ILP32 unknown +Sequentialized seq-mthreaded/pals_opt-floodmax.5.4.ufo.BOUNDED-10.pals.c ILP32 false +Floats neural-networks/poly_32_32_32_thresh_4_unsafe.c-amalgamation.i ILP32 false +ECA eca-rers2012/Problem16_label04.c ILP32 false +ProductLines product-lines/email_spec6_product22.cil.c ILP32 false +XCSP xcsp/aim-100-3-4-sat-4.c ILP32 false +LinkedLists memsafety-broom/dll-middle-data.i LP64 unknown +Arrays array-patterns/array16_pattern.c ILP32 true BitVectors bitvector/soft_float_3.c.cil.c ILP32 unknown -BitVectors bitvector/soft_float_3a.c.cil.c ILP32 true -BitVectors bitvector/soft_float_4-2.c.cil.c ILP32 unknown +ControlFlow openssl/s3_srvr.blast.09.i.cil-1.c ILP32 unknown +Heap memsafety-cve/sniproxyFixed.i ILP32 unknown +Loops nla-digbench-scaling/prod4br-ll_valuebound20.c ILP32 true +Recursive recursive/Ackermann01-2.c ILP32 true +Sequentialized seq-pthread/cs_dekker_unfair.i ILP32 true +Floats neural-networks/poly_64_64_thresh_0_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem16_label37.c ILP32 false +ProductLines product-lines/email_spec6_product34.cil.c ILP32 false +XCSP xcsp/aim-100-6-0-sat-4.c ILP32 false +LinkedLists memsafety-broom/linux-hlist-fst-data.i LP64 unknown +Arrays array-patterns/array29_pattern.c ILP32 true +BitVectors bitvector/soft_float_4-1.c.cil.c ILP32 unknown +ControlFlow openssl/s3_srvr.blast.10.i.cil-2.c ILP32 unknown +Heap memsafety-cve/zchunk.i ILP32 unknown +Loops nla-digbench-scaling/ps2-ll_valuebound1.c ILP32 true +Recursive recursive/Addition01-2.c ILP32 true +Sequentialized seq-pthread/cs_fib_longer-1.i ILP32 true +Floats neural-networks/robot_5_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem17_label10.c ILP32 true +ProductLines product-lines/email_spec7_product24.cil.c ILP32 true +XCSP xcsp/aim-200-1-6-sat-4.c ILP32 false +LinkedLists memsafety-broom/linux-list-fst-data.i LP64 unknown +Arrays array-programs/copysome1-1.i ILP32 true BitVectors bitvector/soft_float_4-2a.c.cil.c ILP32 true +ControlFlow openssl/s3_srvr.blast.12.i.cil-1.c ILP32 unknown +Heap memsafety-ext2/complex_data_creation_test01-1.i ILP32 unknown +Loops nla-digbench-scaling/ps4-ll_unwindbound2.c ILP32 false +Recursive recursive/Addition03-2.c ILP32 unknown +Sequentialized seq-pthread/cs_peterson.i ILP32 true +Floats neural-networks/softplus_0_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem17_label43.c ILP32 true +ProductLines product-lines/email_spec7_product33.cil.c ILP32 false +XCSP xcsp/aim-200-1-6-unsat-4.c ILP32 true +LinkedLists memsafety-broom/sll-fst-shared.i LP64 unknown +Arrays array-tiling/mlceu.c ILP32 false BitVectors bitvector/soft_float_4-3a.c.cil.c ILP32 false -BitVectors bitvector/soft_float_5.c.cil.c ILP32 unknown -BitVectors bitvector/sum02-1.c ILP32 false -ControlFlow infeasible-control-flow/conflict_branch1.c ILP32 true -ControlFlow infeasible-control-flow/conflict_branch3.c ILP32 true +ControlFlow openssl/s3_srvr.blast.13.i.cil-2.c ILP32 unknown +Heap memsafety-ext2/split_list_test05-2.i ILP32 unknown +Loops nla-digbench-scaling/ps5-ll_valuebound50.c ILP32 true +Recursive recursive/EvenOdd03WithOverflowBug.c ILP32 unknown +Sequentialized seq-pthread/cs_read_write_lock-1.i ILP32 true +Floats neural-networks/softsign_w32_r4_case_1_unsafe.c-amalgamation.i ILP32 false +ECA eca-rers2012/Problem18_label16.c ILP32 true +ProductLines product-lines/email_spec8_product21.cil.c ILP32 false +XCSP xcsp/aim-200-2-0-sat-3.c ILP32 false +LinkedLists memsafety-broom/sll-middle-data.i LP64 unknown +Arrays array-tiling/revcpyswp2.c ILP32 true +BitVectors bitvector/soft_float_5a.c.cil.c ILP32 true +ControlFlow openssl/s3_srvr.blast.15.i.cil-1.c ILP32 unknown +Heap memsafety/lockfree-3.0.i ILP32 unknown +Loops nla-digbench-scaling/sqrt1-ll_valuebound100.c ILP32 true +Recursive recursive/Fibonacci04.c ILP32 false +Sequentialized seq-pthread/cs_stateful-1.i ILP32 false +Floats neural-networks/sqrt_1_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem18_label49.c ILP32 false +ProductLines product-lines/email_spec8_product33.cil.c ILP32 false +XCSP xcsp/aim-200-2-0-unsat-3.c ILP32 true +LinkedLists memsafety-broom/sll-nested-sll-lst-data.i LP64 unknown +Arrays reducercommutativity/max.i ILP32 true +BitVectors bitvector/sum02-2.c ILP32 true +ControlFlow openssl/s3_srvr.blast.16.i.cil-2.c ILP32 unknown +Heap memsafety/test-0220.i ILP32 unknown +Loops nla-digbench/fermat2.c ILP32 unknown +Recursive recursive/McCarthy91-2.c ILP32 true +Sequentialized seq-pthread/cs_szymanski.i ILP32 true +Floats neural-networks/tanh_w32_r2_case_1_unsafe.c-amalgamation.i ILP32 false +ECA eca-rers2012/Problem19_label22.c ILP32 false +ProductLines product-lines/email_spec9_product20.cil.c ILP32 false +XCSP xcsp/aim-200-3-4-sat-3.c ILP32 false +LinkedLists memsafety-broom/sll-shared-after.i LP64 unknown +Arrays reducercommutativity/sep40-1.i ILP32 true +ControlFlow unsignedintegeroverflow-sas23/linear_interpolation.i ILP32 true +Heap memsafety/test-0236.i ILP32 unknown +Loops nla-digbench/ps2-ll.c ILP32 true +Recursive recursive/recHanoi01.c ILP32 true +Sequentialized systemc/token_ring.02.cil-1.c ILP32 unknown +Floats neural-networks/tanh_w8_r4_case_0_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem19_label56.c ILP32 true +ProductLines product-lines/email_spec9_product32.cil.c ILP32 false +XCSP xcsp/aim-200-6-0-sat-3.c ILP32 false +LinkedLists memsafety-broom/sll-shared-sll.i LP64 unknown +Arrays array-cav19/array_init_pair_symmetr2.c ILP32 true +ControlFlow infeasible-control-flow/conflict_branch2.c ILP32 true +Heap heap-data/cart.i ILP32 true +Loops loop-acceleration/array_3-2.i ILP32 false +Recursive fuzzle-programs/01_fuzzle_30x30.c LP64 false +Sequentialized seq-mthreaded/pals_STARTPALS_ActiveStandby.4_1.ufo.BOUNDED-10.pals.c ILP32 false +Floats float-benchs/cast_union_tight.c ILP32 false +ECA eca-programs/Problem101_label07.c ILP32 false +ProductLines product-lines/elevator_spec14_product11.cil.c ILP32 true +XCSP xcsp/AllInterval-006.c ILP32 false +LinkedLists forester-heap/dll-01-2.i ILP32 true +Arrays array-crafted/bAnd2.i ILP32 true ControlFlow infeasible-control-flow/unreach_branch2.c ILP32 true +Heap heap-data/quick_sort_split.i ILP32 true +Loops loop-acceleration/simple_4-2_abstracted.c ILP32 true +Recursive fuzzle-programs/03_fuzzle_50x50.c LP64 false +Sequentialized seq-mthreaded/pals_STARTPALS_ActiveStandby.ufo.BOUNDED-10.pals.c ILP32 true +Floats float-benchs/filter2_set.c ILP32 true +ECA eca-programs/Problem101_label23.c ILP32 false +ProductLines product-lines/elevator_spec14_product24.cil.c ILP32 false +XCSP xcsp/AllInterval-008.c ILP32 false +LinkedLists forester-heap/dll-optional-1.i ILP32 true +Arrays array-crafted/bor4.i ILP32 true ControlFlow infeasible-control-flow/unreach_branch5.c ILP32 true -ControlFlow memory-model/4SB.i ILP32 false -ControlFlow ntdrivers-simplified/diskperf_simpl1.cil.c ILP32 unknown -ControlFlow ntdrivers-simplified/floppy_simpl3.cil-2.c ILP32 unknown -ControlFlow ntdrivers-simplified/kbfiltr_simpl1.cil.c ILP32 unknown +Heap ldv-memsafety/memleaks_test1-1.i ILP32 unknown +Loops loop-industry-pattern/nested-3.c ILP32 unknown +Recursive fuzzle-programs/05_fuzzle_70x70.c LP64 false +Sequentialized seq-mthreaded/pals_STARTPALS_Triplicated.ufo.BOUNDED-10.pals.c ILP32 true +Floats float-benchs/inv_Newton-2.c ILP32 false +ECA eca-programs/Problem102_label16.c ILP32 true +ProductLines product-lines/elevator_spec14_product32.cil.c ILP32 false +XCSP xcsp/AllInterval-010.c ILP32 false +LinkedLists forester-heap/dll-queue-2.i ILP32 false +Arrays array-crafted/mapsum2.i ILP32 true +ControlFlow ntdrivers-simplified/cdaudio_simpl1.cil-1.c ILP32 unknown +Heap ldv-memsafety/memleaks_test10-2.i ILP32 unknown +Loops loop-invgen/apache-get-tag.i.p+sep-reducer.c ILP32 true +Recursive fuzzle-programs/07_fuzzle_50x50_25-cycle.c LP64 false +Sequentialized seq-mthreaded/pals_floodmax.3.2.ufo.BOUNDED-6.pals.c ILP32 false +Floats float-benchs/rlim_invariant.c.v+lhb-reducer.c ILP32 true +ECA eca-programs/Problem102_label32.c ILP32 true +ProductLines product-lines/elevator_spec1_product09.cil.c ILP32 true +XCSP xcsp/AllInterval-012.c ILP32 false +LinkedLists forester-heap/dll-rb-cnstr_1-2.i ILP32 false +Arrays array-crafted/xor4.i ILP32 true +ControlFlow ntdrivers-simplified/floppy_simpl3.cil-1.c ILP32 unknown +Heap ldv-memsafety/memleaks_test12-2.i ILP32 unknown +Loops loop-invgen/id_trans.i ILP32 false +Recursive fuzzle-programs/09_fuzzle_50x50_75-cycle.c LP64 false +Sequentialized seq-mthreaded/pals_floodmax.3.3.ufo.BOUNDED-6.pals.c ILP32 false +Floats float-benchs/sqrt_biNewton_pseudoconstant.c ILP32 true +ECA eca-programs/Problem102_label49.c ILP32 false +ProductLines product-lines/elevator_spec1_product19.cil.c ILP32 true +XCSP xcsp/AllInterval-014.c ILP32 false +LinkedLists forester-heap/dll-rb-sentinel-2.i ILP32 true +Arrays array-crafted/zero_sum_const2.c ILP32 true +ControlFlow ntdrivers-simplified/floppy_simpl4.cil-2.c ILP32 unknown +Heap ldv-memsafety/memleaks_test14.i ILP32 unknown +Loops loop-lit/cggmp2005b.i ILP32 true +Recursive fuzzle-programs/11_fuzzle_50x50_CVE-2016-4489.c LP64 false +Sequentialized seq-mthreaded/pals_floodmax.3.ufo.BOUNDED-6.pals.c ILP32 true +Floats float-newlib/double_req_bl_0660b.c ILP32 true +ECA eca-programs/Problem103_label43.c ILP32 true +ProductLines product-lines/elevator_spec1_product23.cil.c ILP32 true +XCSP xcsp/AllInterval-016.c ILP32 false +LinkedLists forester-heap/dll-simple-white-blue-2.i ILP32 true +Arrays array-crafted/zero_sum_m2.c ILP32 true ControlFlow ntdrivers-simplified/kbfiltr_simpl2.cil-2.c ILP32 unknown +Heap ldv-memsafety/memleaks_test15-2.i ILP32 unknown +Loops loop-lit/gsv2008.i ILP32 true +Recursive fuzzle-programs/13_fuzzle_50x50_CVE-2016-4493.c LP64 false +Sequentialized seq-mthreaded/pals_floodmax.4.1.ufo.BOUNDED-8.pals.c ILP32 false +Floats float-newlib/double_req_bl_0684b.c ILP32 true +ECA eca-rers2012/Problem03_label00.c ILP32 true +ProductLines product-lines/elevator_spec1_product27.cil.c ILP32 true +XCSP xcsp/AllInterval-018.c ILP32 false +LinkedLists forester-heap/dll-token-1.i ILP32 true +Arrays array-examples/relax-2-2.i ILP32 true ControlFlow ntdrivers/diskperf.i.cil-1.c ILP32 true -ControlFlow ntdrivers/floppy.i.cil-1.c ILP32 false -ControlFlow ntdrivers/floppy.i.cil-3.c ILP32 true -ControlFlow ntdrivers/kbfiltr.i.cil-2.c ILP32 false +Heap ldv-memsafety/memleaks_test17_1-1.i ILP32 unknown +Loops loop-new/gauss_sum.i.p+lhb-reducer.c ILP32 true +Recursive fuzzle-programs/15_fuzzle_50x50_equality-checks.c LP64 false +Sequentialized seq-mthreaded/pals_floodmax.4.2.ufo.UNBOUNDED.pals.c ILP32 false +Floats float-newlib/double_req_bl_1122a.c ILP32 true +ECA eca-rers2012/Problem03_label16.c ILP32 true +ProductLines product-lines/elevator_spec1_product31.cil.c ILP32 true +XCSP xcsp/AllInterval-025.c ILP32 false +LinkedLists forester-heap/sll-01-2.i ILP32 false +Arrays array-examples/sanfoundry_43_ground.i ILP32 true +ControlFlow ntdrivers/floppy.i.cil-2.c ILP32 unknown +Heap ldv-memsafety/memleaks_test18.i ILP32 unknown +Loops loop-zilu/benchmark01_conjunctive.i ILP32 unknown +Recursive recursified_loop-crafted/recursified_simple_array_index_value_4.i.v+lhb-reducer.c ILP32 unknown +Sequentialized seq-mthreaded/pals_floodmax.4.4.ufo.BOUNDED-8.pals.c ILP32 false +Floats float-newlib/float_req_bl_0660b.c ILP32 true +ECA eca-rers2012/Problem03_label33.c ILP32 true +ProductLines product-lines/elevator_spec2_product09.cil.c ILP32 true +XCSP xcsp/AllInterval-035.c ILP32 false +LinkedLists forester-heap/sll-buckets-2.i ILP32 false +Arrays array-examples/standard_compareModified_ground.i ILP32 true +ControlFlow ntdrivers/kbfiltr.i.cil-1.c ILP32 unknown +Heap ldv-memsafety/memleaks_test19-1.i ILP32 unknown +Loops loop-zilu/benchmark11_linear_abstracted.i ILP32 true +Recursive recursified_loop-crafted/recursified_simple_array_index_value_4.i ILP32 unknown +Sequentialized seq-mthreaded/pals_floodmax.4_overflow.ufo.UNBOUNDED.pals.c ILP32 false +Floats float-newlib/float_req_bl_0684b.c ILP32 true +ECA eca-rers2012/Problem03_label49.c ILP32 true +ProductLines product-lines/elevator_spec2_product19.cil.c ILP32 true +XCSP xcsp/CostasArray-11.c ILP32 false +LinkedLists forester-heap/sll-circular-2.i ILP32 false +Arrays array-examples/standard_copy3_ground-2.i ILP32 false ControlFlow ntdrivers/parport.i.cil-2.c ILP32 true +Heap ldv-memsafety/memleaks_test21-1.i ILP32 unknown +Loops loop-zilu/benchmark23_conjunctive.i ILP32 true +Recursive recursified_loop-invariants/recursified_linear-inequality-inv-b.c ILP32 unknown +Sequentialized seq-mthreaded/pals_floodmax.5.2.ufo.BOUNDED-10.pals.c ILP32 false +Floats float-newlib/float_req_bl_0920a.c ILP32 true +ECA eca-rers2012/Problem04_label06.c ILP32 false +ProductLines product-lines/elevator_spec2_product23.cil.c ILP32 true +XCSP xcsp/CostasArray-13.c ILP32 false +LinkedLists forester-heap/sll-queue-1.i ILP32 true +Arrays array-examples/standard_copy7_ground-2.i ILP32 true ControlFlow openssl-simplified/s3_clnt_2.cil-1.c ILP32 unknown -ControlFlow openssl-simplified/s3_clnt_3.cil-1.c ILP32 unknown -ControlFlow openssl-simplified/s3_clnt_4.cil-1.c ILP32 unknown -ControlFlow openssl-simplified/s3_srvr_1.cil-1.c ILP32 unknown -ControlFlow openssl-simplified/s3_srvr_11.cil.c ILP32 unknown -ControlFlow openssl-simplified/s3_srvr_13.cil.c ILP32 unknown -ControlFlow openssl-simplified/s3_srvr_2.cil-1.c ILP32 unknown -ControlFlow openssl-simplified/s3_srvr_3.cil.c ILP32 unknown -ControlFlow openssl-simplified/s3_srvr_6.cil-1.c ILP32 unknown -ControlFlow openssl-simplified/s3_srvr_8.cil.c ILP32 unknown -ControlFlow openssl/s3_clnt.blast.01.i.cil-2.c ILP32 unknown -ControlFlow openssl/s3_clnt.blast.03.i.cil-1.c ILP32 unknown -ControlFlow openssl/s3_clnt.blast.04.i.cil-1.c ILP32 unknown -ControlFlow openssl/s3_srvr.blast.01.i.cil-2.c ILP32 unknown -ControlFlow openssl/s3_srvr.blast.02.i.cil-2.c ILP32 unknown -ControlFlow openssl/s3_srvr.blast.06.i.cil-1.c ILP32 unknown -ControlFlow openssl/s3_srvr.blast.07.i.cil-1.c ILP32 unknown -ControlFlow openssl/s3_srvr.blast.08.i.cil-1.c ILP32 unknown -ControlFlow openssl/s3_srvr.blast.09.i.cil-2.c ILP32 unknown -ControlFlow openssl/s3_srvr.blast.10.i.cil-2.c ILP32 unknown -ControlFlow openssl/s3_srvr.blast.12.i.cil-1.c ILP32 unknown -ControlFlow openssl/s3_srvr.blast.13.i.cil-1.c ILP32 unknown -ControlFlow openssl/s3_srvr.blast.14.i.cil-2.c ILP32 unknown -ControlFlow openssl/s3_srvr.blast.15.i.cil-2.c ILP32 unknown -ControlFlow unsignedintegeroverflow-sas23/bilinear_interpolation.i ILP32 true -ControlFlow unsignedintegeroverflow-sas23/distance_through_overflow.i ILP32 true -Heap heap-data/calendar.i ILP32 true -Heap heap-data/shared_mem1.i ILP32 true -Heap ldv-memsafety/memleaks_test11.i ILP32 unknown -Heap ldv-memsafety/memleaks_test13_2.i ILP32 unknown -Heap ldv-memsafety/memleaks_test16.i ILP32 unknown -Heap ldv-memsafety/memleaks_test17_3.i ILP32 unknown -Heap ldv-memsafety/memleaks_test20-1.i ILP32 unknown -Heap ldv-memsafety/memleaks_test22_2-2.i ILP32 unknown -Heap ldv-memsafety/memleaks_test23_3.i ILP32 unknown -Heap ldv-memsafety/memleaks_test5_1.i ILP32 unknown -Heap ldv-memsafety/memleaks_test7-2.i ILP32 unknown -Heap ldv-regression/alias_of_return_2.c_1.i ILP32 true -Heap ldv-regression/rule60_list2.i ILP32 true -Heap ldv-regression/test21-1.c ILP32 true -Heap ldv-regression/test24-1.c ILP32 true -Heap ldv-regression/test28-2.c ILP32 true -Heap ldv-sets/test_mutex_double_unlock.i ILP32 false -Heap list-ext-properties/list-ext.i ILP32 false -Heap list-ext-properties/test-0158_1-1.i ILP32 unknown -Heap list-ext-properties/test-0504_1.i ILP32 unknown -Heap memsafety-cve/MojoJson.i ILP32 unknown -Heap memsafety-cve/dlt-daemon.i ILP32 unknown -Heap memsafety-cve/gpac_10.i ILP32 unknown -Heap memsafety-cve/gpac_18.i ILP32 unknown -Heap memsafety-cve/gpac_8.i ILP32 unknown -Heap memsafety-cve/json-parserFixed.i ILP32 unknown -Heap memsafety-cve/libpe.i ILP32 unknown -Heap memsafety-cve/minizip-ngFixed.i ILP32 unknown -Heap memsafety-cve/nanopb.i ILP32 unknown -Heap memsafety-cve/openSCFixed.i ILP32 unknown -Heap memsafety-cve/picotcpFixed.i ILP32 unknown -Heap memsafety-cve/radare2_2Fixed.i ILP32 unknown -Heap memsafety-cve/sofia-sip_1.i ILP32 unknown -Heap memsafety-cve/zchunkFixed.i ILP32 unknown -Heap memsafety-ext/tree_of_cslls.i ILP32 unknown -Heap memsafety-ext2/optional_data_creation_test04-1.i ILP32 unknown -Heap memsafety/global-atexit-1.i ILP32 unknown -Heap memsafety/lockfree-3.2.i ILP32 unknown -Heap memsafety/test-0220.i ILP32 unknown -Heap memsafety/test-0235-2.i ILP32 unknown -Loops loop-acceleration/array3.i ILP32 unknown -Loops loop-crafted/simple_array_index_value_4.i.v+nlh-reducer.c ILP32 true -Loops loop-invgen/apache-get-tag.i.p+lhb-reducer.c ILP32 true -Loops loop-invgen/sendmail-close-angle.i ILP32 true -Loops loop-lit/gsv2008.i ILP32 true -Loops loop-simple/nested_2.c ILP32 true -Loops loop-zilu/benchmark13_conjunctive.i ILP32 unknown -Loops loop-zilu/benchmark31_disjunctive.i ILP32 unknown -Loops loop-zilu/benchmark49_linear.i ILP32 unknown -Loops loops-crafted-1/nested3-1_abstracted.c ILP32 true -Loops loops/array-2.c ILP32 false -Loops loops/n.c24.i ILP32 false -Loops loops/terminator_02-2_abstracted.c ILP32 true -Loops nla-digbench-scaling/bresenham-ll_unwindbound100.c ILP32 false -Loops nla-digbench-scaling/cohencu-ll_valuebound10.c ILP32 true -Loops nla-digbench-scaling/cohendiv-ll_valuebound50.c ILP32 true -Loops nla-digbench-scaling/divbin2_unwindbound20.i ILP32 true -Loops nla-digbench-scaling/divbin_valuebound100.i ILP32 true -Loops nla-digbench-scaling/egcd2-ll_unwindbound1.c ILP32 false -Loops nla-digbench-scaling/egcd3-ll_unwindbound5.c ILP32 false -Loops nla-digbench-scaling/fermat1-ll_valuebound20.c ILP32 true -Loops nla-digbench-scaling/freire1_valuebound100.c ILP32 true -Loops nla-digbench-scaling/geo1-ll2_valuebound100.c ILP32 unknown -Loops nla-digbench-scaling/geo1-u2_unwindbound100.c ILP32 unknown -Loops nla-digbench-scaling/geo1-u_valuebound20.c ILP32 false -Loops nla-digbench-scaling/geo2-ll_unwindbound50.c ILP32 true -Loops nla-digbench-scaling/geo3-ll_unwindbound10.c ILP32 true -Loops nla-digbench-scaling/hard-ll_valuebound1.c ILP32 true -Loops nla-digbench-scaling/hard-u_valuebound5.c ILP32 true -Loops nla-digbench-scaling/knuth_unwindbound2.i ILP32 true -Loops nla-digbench-scaling/lcm1_valuebound2.c ILP32 true -Loops nla-digbench-scaling/mannadiv_unwindbound10.c ILP32 false -Loops nla-digbench-scaling/prod4br-ll_valuebound50.c ILP32 true -Loops nla-digbench-scaling/ps2-ll_unwindbound20.c ILP32 true -Loops nla-digbench-scaling/ps3-ll_valuebound2.c ILP32 true -Loops nla-digbench-scaling/ps5-ll_unwindbound10.c ILP32 false -Loops nla-digbench-scaling/ps6-ll_unwindbound50.c ILP32 false -Loops nla-digbench-scaling/sqrt1-ll_valuebound20.c ILP32 true -Loops nla-digbench/fermat1-ll.c ILP32 true -Loops nla-digbench/knuth.i ILP32 true -Recursive fuzzle-programs/01_fuzzle_30x30.c LP64 false -Recursive fuzzle-programs/04_fuzzle_60x60.c LP64 false -Recursive fuzzle-programs/07_fuzzle_50x50_25-cycle.c LP64 false -Recursive fuzzle-programs/10_fuzzle_50x50_CVE-2016-4487.c LP64 false -Recursive fuzzle-programs/14_fuzzle_50x50_CVE-2016-6131.c LP64 false -Recursive recursified_loop-crafted/recursified_simple_array_index_value_4.i.v+lhb-reducer.c ILP32 unknown -Recursive recursified_loop-invariants/recursified_linear-inequality-inv-a.c ILP32 unknown +Heap ldv-memsafety/memleaks_test22_2-1.i ILP32 unknown +Loops loop-zilu/benchmark33_linear.i ILP32 true Recursive recursified_loop-simple/recursified_nested_1b.c ILP32 unknown +Sequentialized seq-mthreaded/pals_floodmax.5.4.ufo.BOUNDED-10.pals.c ILP32 false +Floats float-newlib/float_req_bl_1270a.c ILP32 true +ECA eca-rers2012/Problem04_label31.c ILP32 false +ProductLines product-lines/elevator_spec2_product27.cil.c ILP32 true +XCSP xcsp/CostasArray-15.c ILP32 false +LinkedLists forester-heap/sll-rb-cnstr_1-2.i ILP32 false +Arrays array-examples/standard_find_ground-2.i ILP32 unknown +ControlFlow openssl-simplified/s3_clnt_3.cil-2.c ILP32 unknown +Heap ldv-memsafety/memleaks_test22_5.i ILP32 unknown +Loops loop-zilu/benchmark44_disjunctive.i ILP32 unknown +Recursive recursified_loop-simple/recursified_nested_3.c ILP32 unknown +Sequentialized seq-mthreaded/pals_lcr-var-start-time.3.1.ufo.BOUNDED-6.pals.c ILP32 false +Floats floats-cbmc-regression/float19.i ILP32 true +ECA eca-rers2012/Problem04_label56.c ILP32 true +ProductLines product-lines/elevator_spec2_product31.cil.c ILP32 true +XCSP xcsp/CostasArray-17.c ILP32 false +LinkedLists forester-heap/sll-reverse_simple.i ILP32 true +Arrays array-examples/standard_partition_original_ground.i ILP32 true +ControlFlow openssl-simplified/s3_clnt_4.cil-2.c ILP32 unknown +Heap ldv-memsafety/memleaks_test23_4.i ILP32 unknown +Loops loops-crafted-1/in-de20.c ILP32 true Recursive recursified_loop-simple/recursified_nested_5.c ILP32 unknown -Recursive recursified_nla-digbench/recursified_bresenham.c ILP32 unknown -Recursive recursified_nla-digbench/recursified_cohendiv-ll.c ILP32 unknown -Recursive recursified_nla-digbench/recursified_divbin.i ILP32 unknown +Sequentialized seq-mthreaded/pals_lcr-var-start-time.4.1.ufo.BOUNDED-8.pals.c ILP32 false +Floats floats-cdfpl/newton_1_7.i ILP32 false +ECA eca-rers2012/Problem05_label14.c ILP32 true +ProductLines product-lines/elevator_spec3_product03.cil.c ILP32 false +XCSP xcsp/Dubois-016.c ILP32 true +LinkedLists forester-heap/sll-simple-white-blue-2.i ILP32 false +Arrays array-examples/standard_strcmp_ground.i ILP32 true +ControlFlow openssl-simplified/s3_srvr_10.cil.c ILP32 unknown +Heap ldv-memsafety/memleaks_test5.i ILP32 unknown +Loops loops-crafted-1/nested3-1_abstracted.c ILP32 true +Recursive recursified_nla-digbench/recursified_bresenham-ll.c ILP32 unknown +Sequentialized seq-mthreaded/pals_lcr-var-start-time.4.ufo.BOUNDED-8.pals.c ILP32 true +Floats floats-cdfpl/newton_3_4.i ILP32 true +ECA eca-rers2012/Problem05_label31.c ILP32 true +ProductLines product-lines/elevator_spec3_product18.cil.c ILP32 true +XCSP xcsp/Dubois-019.c ILP32 true +LinkedLists forester-heap/sll-sorted-2.i ILP32 true +Arrays array-examples/standard_two_index_04.i ILP32 true +ControlFlow openssl-simplified/s3_srvr_12.cil.c ILP32 unknown +Heap ldv-memsafety/memleaks_test6_1.i ILP32 unknown +Loops loops-crafted-1/sumt8.c ILP32 true +Recursive recursified_nla-digbench/recursified_cohencu-ll.c ILP32 unknown +Sequentialized seq-mthreaded/pals_lcr-var-start-time.5.2.ufo.BOUNDED-10.pals.c ILP32 false +Floats floats-cdfpl/square_2.i ILP32 false +ECA eca-rers2012/Problem05_label48.c ILP32 false +ProductLines product-lines/elevator_spec3_product22.cil.c ILP32 true +XCSP xcsp/Dubois-021.c ILP32 true +LinkedLists heap-manipulation/bubble_sort_linux-1.i ILP32 true +Arrays array-examples/standard_vector_difference_ground.i ILP32 true +ControlFlow openssl-simplified/s3_srvr_14.cil.c ILP32 unknown +Heap ldv-memsafety/memleaks_test8.i ILP32 unknown +Loops loops/count_up_down-2.c ILP32 false +Recursive recursified_nla-digbench/recursified_cohendiv.c ILP32 unknown +Sequentialized seq-mthreaded/pals_lcr-var-start-time.6.2.ufo.BOUNDED-12.pals.c ILP32 false +Floats floats-esbmc-regression/trunc_nondet_2.i ILP32 true +ECA eca-rers2012/Problem06_label05.c ILP32 false +ProductLines product-lines/elevator_spec3_product27.cil.c ILP32 false +XCSP xcsp/Dubois-023.c ILP32 true +LinkedLists heap-manipulation/dll_of_dll-1.i ILP32 true +Arrays array-fpi/brs4f.c ILP32 false +ControlFlow openssl-simplified/s3_srvr_2.cil-2.c ILP32 unknown +Heap ldv-memsafety/memleaks_test9_1.i ILP32 unknown +Loops loops/matrix-2-2.c ILP32 false +Recursive recursified_nla-digbench/recursified_dijkstra.c ILP32 unknown +Sequentialized seq-mthreaded/pals_lcr.3.1.ufo.BOUNDED-6.pals.c ILP32 false +Floats floats-nonlinear/coral-benchmark21.c ILP32 false +ECA eca-rers2012/Problem06_label21.c ILP32 false +ProductLines product-lines/elevator_spec3_product31.cil.c ILP32 false +XCSP xcsp/Dubois-025.c ILP32 true +LinkedLists heap-manipulation/merge_sort-2.i ILP32 true +Arrays array-fpi/condg.c ILP32 true +ControlFlow openssl-simplified/s3_srvr_6.cil-1.c ILP32 unknown +Heap ldv-regression/fo_test.i ILP32 false +Loops loops/sum01_bug02_sum01_bug02_base.case.i ILP32 unknown +Recursive recursified_nla-digbench/recursified_divbin2.i ILP32 unknown +Sequentialized seq-mthreaded/pals_lcr.3_overflow.ufo.UNBOUNDED.pals.c ILP32 false +Floats neural-networks/cartpole_12_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem06_label38.c ILP32 false +ProductLines product-lines/elevator_spec9_product11.cil.c ILP32 true +XCSP xcsp/Dubois-027.c ILP32 true +LinkedLists heap-manipulation/sll_to_dll_rev-2.i ILP32 true +Arrays array-fpi/eqn2.c ILP32 true +ControlFlow openssl-simplified/s3_srvr_8.cil.c ILP32 unknown +Heap ldv-regression/rule60_list2.c_1.i ILP32 false +Loops loops/terminator_03-2_abstracted.i ILP32 true Recursive recursified_nla-digbench/recursified_egcd.c ILP32 unknown -Recursive recursified_nla-digbench/recursified_egcd3-ll.c ILP32 unknown +Sequentialized seq-mthreaded/pals_lcr.4_overflow.ufo.UNBOUNDED.pals.c ILP32 false +Floats neural-networks/cartpole_25_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem06_label54.c ILP32 true +ProductLines product-lines/elevator_spec9_product28.cil.c ILP32 false +XCSP xcsp/Dubois-029.c ILP32 true +LinkedLists heap-manipulation/tree-2.i ILP32 false +Arrays array-fpi/eqn5f.c ILP32 false +ControlFlow openssl/s3_clnt.blast.02.i.cil-1.c ILP32 unknown +Heap ldv-regression/test14.c ILP32 true +Loops loops/vogal-1.i ILP32 true +Recursive recursified_nla-digbench/recursified_egcd2.c ILP32 unknown +Sequentialized seq-mthreaded/pals_lcr.5.ufo.BOUNDED-10.pals.c ILP32 true +Floats neural-networks/cartpole_37_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem07_label11.c ILP32 false +ProductLines product-lines/elevator_spec9_product32.cil.c ILP32 false +XCSP xcsp/Dubois-035.c ILP32 true +LinkedLists list-ext3-properties/dll_nondet_free_order-1.i ILP32 unknown +Arrays array-fpi/ifeqn3f.c ILP32 false +ControlFlow openssl/s3_clnt.blast.03.i.cil-2.c ILP32 unknown +Heap ldv-regression/test21-2.c ILP32 false +Loops nla-digbench-scaling/bresenham-ll_valuebound2.c ILP32 true +Recursive recursified_nla-digbench/recursified_egcd3.c ILP32 unknown +Sequentialized seq-mthreaded/pals_lcr.6.1.ufo.BOUNDED-12.pals.c ILP32 false +Floats neural-networks/cartpole_49_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem07_label27.c ILP32 true +ProductLines product-lines/email_spec0_product10.cil.c ILP32 true +XCSP xcsp/Dubois-050.c ILP32 true +LinkedLists list-ext3-properties/sll_length_check-1.i ILP32 false +Arrays array-fpi/indp1f.c ILP32 false +ControlFlow openssl/s3_srvr.blast.01.i.cil-1.c ILP32 unknown +Heap ldv-regression/test23-2.c ILP32 true +Loops nla-digbench-scaling/cohencu-ll_valuebound10.c ILP32 true Recursive recursified_nla-digbench/recursified_fermat1.c ILP32 unknown +Sequentialized seq-mthreaded/pals_lcr.7.1.ufo.BOUNDED-14.pals.c ILP32 false +Floats neural-networks/cartpole_60_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem07_label44.c ILP32 false +ProductLines product-lines/email_spec0_product21.cil.c ILP32 false +XCSP xcsp/Dubois-060.c ILP32 true +LinkedLists list-ext3-properties/sll_nondet_insert-2.i ILP32 true +Arrays array-fpi/indp5f.c ILP32 false +ControlFlow openssl/s3_srvr.blast.02.i.cil-2.c ILP32 unknown +Heap ldv-regression/test25-2.c ILP32 true +Loops nla-digbench-scaling/cohendiv-ll_unwindbound50.c ILP32 true +Recursive recursified_nla-digbench/recursified_fermat2.c ILP32 unknown +Sequentialized seq-mthreaded/pals_lcr.8.1.ufo.BOUNDED-16.pals.c ILP32 false +Floats neural-networks/cartpole_73_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem08_label00.c ILP32 true +ProductLines product-lines/email_spec0_product26.cil.c ILP32 false +XCSP xcsp/Dubois-070.c ILP32 true +LinkedLists list-ext3-properties/sll_of_sll_nondet_append-2.i ILP32 false +Arrays array-fpi/ms1.c ILP32 true +ControlFlow openssl/s3_srvr.blast.04.i.cil.c ILP32 unknown +Heap ldv-regression/test29-1.c ILP32 false +Loops nla-digbench-scaling/dijkstra-u_unwindbound20.c ILP32 false Recursive recursified_nla-digbench/recursified_geo1-ll2.c ILP32 unknown -Recursive recursified_nla-digbench/recursified_geo1.c ILP32 unknown +Sequentialized seq-mthreaded/pals_opt-floodmax.3.1.ufo.BOUNDED-6.pals.c ILP32 false +Floats neural-networks/cartpole_85_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem08_label17.c ILP32 true +ProductLines product-lines/email_spec0_product35.cil.c ILP32 false +XCSP xcsp/Dubois-080.c ILP32 true +LinkedLists list-properties/alternating_list-2.i ILP32 false +Arrays array-fpi/ms5.c ILP32 true +ControlFlow openssl/s3_srvr.blast.06.i.cil-2.c ILP32 unknown +Heap ldv-sets/test_mutex.i ILP32 true +Loops nla-digbench-scaling/divbin2_unwindbound100.i ILP32 true +Recursive recursified_nla-digbench/recursified_geo1-u2.c ILP32 unknown +Sequentialized seq-mthreaded/pals_opt-floodmax.3.2.ufo.UNBOUNDED.pals.c ILP32 false +Floats neural-networks/cartpole_97_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem08_label33.c ILP32 true +ProductLines product-lines/email_spec0_product40.cil.c ILP32 true +XCSP xcsp/Dubois-090.c ILP32 true +LinkedLists list-properties/list_flag-1.i ILP32 false +Arrays array-fpi/nsqmf.c ILP32 false +ControlFlow openssl/s3_srvr.blast.08.i.cil-1.c ILP32 unknown +Heap ldv-sets/test_mutex_unbounded-2.i ILP32 false +Loops nla-digbench-scaling/divbin_unwindbound1.i ILP32 true +Recursive recursified_nla-digbench/recursified_geo2-ll.c ILP32 unknown +Sequentialized seq-mthreaded/pals_opt-floodmax.3.3.ufo.UNBOUNDED.pals.c ILP32 false +Floats neural-networks/dubinsrejoin_10_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem08_label50.c ILP32 false +ProductLines product-lines/email_spec11_product08.cil.c ILP32 true +XCSP xcsp/Dubois-100.c ILP32 true +LinkedLists list-properties/simple-2.i ILP32 true +Arrays array-fpi/res2f.c ILP32 false +ControlFlow openssl/s3_srvr.blast.09.i.cil-2.c ILP32 unknown +Heap list-ext-properties/960521-1_1-3.i ILP32 unknown +Loops nla-digbench-scaling/divbin_valuebound5.i ILP32 true Recursive recursified_nla-digbench/recursified_geo2.c ILP32 unknown -Recursive recursified_nla-digbench/recursified_geo3.c ILP32 unknown -Recursive recursified_nla-digbench/recursified_hard2.c ILP32 unknown -Recursive recursified_nla-digbench/recursified_lcm2.c ILP32 unknown -Recursive recursified_nla-digbench/recursified_prod4br.c ILP32 unknown -Recursive recursified_nla-digbench/recursified_ps2.c ILP32 unknown -Recursive recursified_nla-digbench/recursified_ps4-ll.c ILP32 unknown -Recursive recursified_nla-digbench/recursified_ps5.c ILP32 unknown -Recursive recursified_nla-digbench/recursified_sqrt1-ll.c ILP32 unknown -Recursive recursive-simple/id2_b3_o5.c ILP32 true -Recursive recursive-simple/id_b3_o2-1.c ILP32 unknown -Recursive recursive-simple/id_b3_o5-2.c ILP32 true -Recursive recursive-simple/id_o10.c ILP32 false -Recursive recursive-simple/id_o200.c ILP32 false -Recursive recursive-simple/sum_non_eq-1.c ILP32 unknown -Recursive recursive/Ackermann01-2.c ILP32 true -Recursive recursive/Addition01-2.c ILP32 true -Recursive recursive/Addition03-1.c ILP32 true -Recursive recursive/EvenOdd01-1.c ILP32 true -Recursive recursive/Fibonacci01-1.c ILP32 true -Recursive recursive/Fibonacci05-overflow.c ILP32 unknown -Recursive recursive/McCarthy91-2.c ILP32 true -Recursive recursive/gcd02.c ILP32 true -Sequentialized seq-mthreaded/pals_STARTPALS_ActiveStandby.1.ufo.BOUNDED-10.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_STARTPALS_ActiveStandby.4_2.ufo.BOUNDED-10.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_STARTPALS_Triplicated.1.ufo.BOUNDED-10.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_floodmax.3.1.ufo.BOUNDED-6.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_floodmax.3.2.ufo.UNBOUNDED.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_floodmax.3.4.ufo.BOUNDED-6.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_floodmax.4.1.ufo.BOUNDED-8.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_floodmax.4.2.ufo.UNBOUNDED.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_floodmax.4.4.ufo.BOUNDED-8.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_floodmax.5.1.ufo.BOUNDED-10.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_floodmax.5.3.ufo.BOUNDED-10.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_floodmax.5.ufo.BOUNDED-10.pals.c ILP32 true -Sequentialized seq-mthreaded/pals_lcr-var-start-time.3.2.ufo.BOUNDED-6.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_lcr-var-start-time.4.2.ufo.BOUNDED-8.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_lcr-var-start-time.5.1.ufo.BOUNDED-10.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_lcr-var-start-time.6.1.ufo.BOUNDED-12.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_lcr.3.1.ufo.BOUNDED-6.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_lcr.3_overflow.ufo.UNBOUNDED.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_lcr.4_overflow.ufo.UNBOUNDED.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_lcr.5_overflow.ufo.UNBOUNDED.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_lcr.6_overflow.ufo.UNBOUNDED.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_lcr.7.ufo.BOUNDED-14.pals.c ILP32 true -Sequentialized seq-mthreaded/pals_lcr.8.ufo.BOUNDED-16.pals.c ILP32 true -Sequentialized seq-mthreaded/pals_opt-floodmax.3.2.ufo.BOUNDED-6.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_opt-floodmax.3.3.ufo.BOUNDED-6.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_opt-floodmax.3.ufo.BOUNDED-6.pals.c ILP32 true +Sequentialized seq-mthreaded/pals_opt-floodmax.3_overflow.ufo.UNBOUNDED.pals.c ILP32 false +Floats neural-networks/dubinsrejoin_23_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem09_label06.c ILP32 false +ProductLines product-lines/email_spec11_product20.cil.c ILP32 false +XCSP xcsp/Haystacks-07.c ILP32 true +LinkedLists list-properties/splice-2.i ILP32 true +Arrays array-fpi/s1lif.c ILP32 true +ControlFlow openssl/s3_srvr.blast.11.i.cil-1.c ILP32 unknown +Heap list-ext-properties/list-ext_flag_1.i ILP32 unknown +Loops nla-digbench-scaling/egcd-ll_valuebound2.c ILP32 true +Recursive recursified_nla-digbench/recursified_geo3-ll2.c ILP32 unknown Sequentialized seq-mthreaded/pals_opt-floodmax.4.2.ufo.BOUNDED-8.pals.c ILP32 false +Floats neural-networks/dubinsrejoin_35_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem09_label23.c ILP32 true +ProductLines product-lines/email_spec11_product26.cil.c ILP32 false +XCSP xcsp/Haystacks-10.c ILP32 true +LinkedLists memsafety-broom/dll-fst-data.i LP64 unknown +Arrays array-fpi/s32if.c ILP32 true +ControlFlow openssl/s3_srvr.blast.12.i.cil-2.c ILP32 unknown +Heap list-ext-properties/test-0158_1-2.i ILP32 unknown +Loops nla-digbench-scaling/egcd2-ll_valuebound10.c ILP32 true +Recursive recursified_nla-digbench/recursified_hard-ll.c ILP32 unknown Sequentialized seq-mthreaded/pals_opt-floodmax.4.3.ufo.BOUNDED-8.pals.c ILP32 false +Floats neural-networks/dubinsrejoin_48_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem09_label39.c ILP32 true +ProductLines product-lines/email_spec11_product33.cil.c ILP32 false +XCSP xcsp/Haystacks-12.c ILP32 true +LinkedLists memsafety-broom/dll-lst-data.i LP64 unknown +Arrays array-fpi/s42iff.c ILP32 false +ControlFlow openssl/s3_srvr.blast.14.i.cil-1.c ILP32 unknown +Heap list-ext-properties/test-0232_1-2.i ILP32 unknown +Loops nla-digbench-scaling/egcd3-ll_unwindbound50.c ILP32 false +Recursive recursified_nla-digbench/recursified_hard.c ILP32 unknown Sequentialized seq-mthreaded/pals_opt-floodmax.4.ufo.BOUNDED-8.pals.c ILP32 true -Sequentialized seq-mthreaded/pals_opt-floodmax.5.2.ufo.BOUNDED-10.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_opt-floodmax.5.4.ufo.BOUNDED-10.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_opt-floodmax.5_overflow.ufo.UNBOUNDED.pals.c ILP32 false -Sequentialized seq-pthread/cs_fib-1.i ILP32 true -Sequentialized seq-pthread/cs_fib_longer-2.i ILP32 false -Sequentialized seq-pthread/cs_lazy.i ILP32 unknown -Sequentialized seq-pthread/cs_queue-2.i ILP32 false -Sequentialized seq-pthread/cs_stack-1.i ILP32 true -Sequentialized seq-pthread/cs_stateful-1.i ILP32 false -Sequentialized seq-pthread/cs_szymanski.i ILP32 true -Sequentialized systemc/token_ring.01.cil-2.c ILP32 unknown -Floats float-benchs/arctan_Pade.c ILP32 true -Floats float-benchs/image_filter.c ILP32 true -Floats float-benchs/nan_float_range.c ILP32 true -Floats float-benchs/zonotope_3.c ILP32 true -Floats float-newlib/double_req_bl_0832.c ILP32 true -Floats float-newlib/double_req_bl_1251b.c ILP32 true -Floats float-newlib/float_req_bl_0870b.c ILP32 true -Floats float-newlib/float_req_bl_1231.c ILP32 true -Floats floats-cbmc-regression/float6.c ILP32 true -Floats floats-cdfpl/newton_3_4.i ILP32 true -Floats floats-esbmc-regression/floor_nondet.i ILP32 true -Floats loop-floats-scientific-comp/loop1-1.c ILP32 true -Floats neural-networks/cartpole_22_safe.c-amalgamation.i ILP32 true -Floats neural-networks/cartpole_41_safe.c-amalgamation.i ILP32 true -Floats neural-networks/cartpole_61_safe.c-amalgamation.i ILP32 true -Floats neural-networks/cartpole_80_safe.c-amalgamation.i ILP32 true -Floats neural-networks/cartpole_9_safe.c-amalgamation.i ILP32 true -Floats neural-networks/dubinsrejoin_21_safe.c-amalgamation.i ILP32 true -Floats neural-networks/dubinsrejoin_40_safe.c-amalgamation.i ILP32 true Floats neural-networks/dubinsrejoin_5_safe.c-amalgamation.i ILP32 true -Floats neural-networks/dubinsrejoin_7_safe.c-amalgamation.i ILP32 true -Floats neural-networks/elu_2_safe.c-amalgamation.i ILP32 true -Floats neural-networks/gcas_3_safe.c-amalgamation.i ILP32 true -Floats neural-networks/logistic_1_safe.c-amalgamation.i ILP32 true -Floats neural-networks/lunarlander_22_unsafe.c-amalgamation.i ILP32 false -Floats neural-networks/lunarlander_40_unsafe.c-amalgamation.i ILP32 false -Floats neural-networks/lunarlander_5_unsafe.c-amalgamation.i ILP32 false -Floats neural-networks/lunarlander_7_unsafe.c-amalgamation.i ILP32 false -Floats neural-networks/lunarlander_98_unsafe.c-amalgamation.i ILP32 false -Floats neural-networks/poly_16_16_16_16_thresh_0_safe.c-amalgamation.i ILP32 true -Floats neural-networks/poly_256_thresh_4_unsafe.c-amalgamation.i ILP32 false -Floats neural-networks/poly_4_4_4_4_thresh_1_safe.c-amalgamation.i ILP32 true -Floats neural-networks/poly_64_64_thresh_4_unsafe.c-amalgamation.i ILP32 false -Floats neural-networks/robot_3_safe.c-amalgamation.i ILP32 true -Floats neural-networks/softmax_3_unsafe.c-amalgamation.i ILP32 false -Floats neural-networks/softsign_w16_r2_case_0_safe.c-amalgamation.i ILP32 true -Floats neural-networks/softsign_w4_r4_case_1_unsafe.c-amalgamation.i ILP32 false -Floats neural-networks/sqrt_5_safe.c-amalgamation.i ILP32 true -Floats neural-networks/tanh_w32_r1_case_1_safe.c-amalgamation.i ILP32 true -Floats neural-networks/tanh_w64_r4_case_0_safe.c-amalgamation.i ILP32 true -ECA eca-programs/Problem101_label00.c ILP32 false -ECA eca-programs/Problem102_label02.c ILP32 false -ECA eca-programs/Problem102_label28.c ILP32 false -ECA eca-programs/Problem102_label55.c ILP32 false -ECA eca-programs/Problem103_label59.c ILP32 false -ECA eca-rers2012/Problem03_label26.c ILP32 false -ECA eca-rers2012/Problem03_label52.c ILP32 false -ECA eca-rers2012/Problem04_label26.c ILP32 false -ECA eca-rers2012/Problem05_label04.c ILP32 true -ECA eca-rers2012/Problem05_label31.c ILP32 true -ECA eca-rers2012/Problem05_label57.c ILP32 false -ECA eca-rers2012/Problem06_label24.c ILP32 false -ECA eca-rers2012/Problem06_label50.c ILP32 true -ECA eca-rers2012/Problem07_label17.c ILP32 true -ECA eca-rers2012/Problem07_label43.c ILP32 true -ECA eca-rers2012/Problem08_label10.c ILP32 false -ECA eca-rers2012/Problem08_label36.c ILP32 true -ECA eca-rers2012/Problem09_label03.c ILP32 false -ECA eca-rers2012/Problem09_label29.c ILP32 true ECA eca-rers2012/Problem09_label56.c ILP32 false -ECA eca-rers2012/Problem10_label22.c ILP32 true -ECA eca-rers2012/Problem10_label48.c ILP32 false -ECA eca-rers2012/Problem11_label15.c ILP32 false -ECA eca-rers2012/Problem11_label41.c ILP32 true -ECA eca-rers2012/Problem12_label08.c ILP32 false -ECA eca-rers2012/Problem12_label34.c ILP32 false -ECA eca-rers2012/Problem13_label01.c ILP32 true -ECA eca-rers2012/Problem13_label27.c ILP32 true -ECA eca-rers2012/Problem13_label54.c ILP32 false -ECA eca-rers2012/Problem15_label21.c ILP32 true -ECA eca-rers2012/Problem15_label48.c ILP32 false -ECA eca-rers2012/Problem16_label14.c ILP32 false -ECA eca-rers2012/Problem16_label41.c ILP32 false -ECA eca-rers2012/Problem17_label07.c ILP32 false -ECA eca-rers2012/Problem17_label34.c ILP32 false -ECA eca-rers2012/Problem18_label00.c ILP32 false -ECA eca-rers2012/Problem18_label27.c ILP32 false -ECA eca-rers2012/Problem18_label53.c ILP32 true -ECA eca-rers2012/Problem19_label20.c ILP32 true -ECA eca-rers2012/Problem19_label46.c ILP32 true -ProductLines product-lines/elevator_spec14_product03.cil.c ILP32 true -ProductLines product-lines/elevator_spec14_product27.cil.c ILP32 true -ProductLines product-lines/elevator_spec1_product09.cil.c ILP32 true -ProductLines product-lines/elevator_spec1_product21.cil.c ILP32 true -ProductLines product-lines/elevator_spec1_product28.cil.c ILP32 false -ProductLines product-lines/elevator_spec2_product01.cil.c ILP32 true -ProductLines product-lines/elevator_spec2_product20.cil.c ILP32 false -ProductLines product-lines/elevator_spec2_product27.cil.c ILP32 true -ProductLines product-lines/elevator_spec2_productSimulator.cil.c ILP32 false -ProductLines product-lines/elevator_spec3_product19.cil.c ILP32 false -ProductLines product-lines/elevator_spec3_product25.cil.c ILP32 true -ProductLines product-lines/elevator_spec3_product32.cil.c ILP32 false -ProductLines product-lines/elevator_spec9_product27.cil.c ILP32 true -ProductLines product-lines/email_spec0_product05.cil.c ILP32 true -ProductLines product-lines/email_spec0_product22.cil.c ILP32 false -ProductLines product-lines/email_spec0_product33.cil.c ILP32 false -ProductLines product-lines/email_spec0_productSimulator.cil.c ILP32 false -ProductLines product-lines/email_spec11_product18.cil.c ILP32 true -ProductLines product-lines/email_spec11_product30.cil.c ILP32 false ProductLines product-lines/email_spec11_product39.cil.c ILP32 true -ProductLines product-lines/email_spec1_product20.cil.c ILP32 false -ProductLines product-lines/email_spec1_product31.cil.c ILP32 false -ProductLines product-lines/email_spec27_product13.cil.c ILP32 true -ProductLines product-lines/email_spec27_product27.cil.c ILP32 false -ProductLines product-lines/email_spec27_product33.cil.c ILP32 false -ProductLines product-lines/email_spec3_product19.cil.c ILP32 false -ProductLines product-lines/email_spec3_product29.cil.c ILP32 false -ProductLines product-lines/email_spec3_productSimulator.cil.c ILP32 false -ProductLines product-lines/email_spec4_product25.cil.c ILP32 false -ProductLines product-lines/email_spec4_product32.cil.c ILP32 false -ProductLines product-lines/email_spec6_product15.cil.c ILP32 false -ProductLines product-lines/email_spec6_product28.cil.c ILP32 false -ProductLines product-lines/email_spec6_product35.cil.c ILP32 false -ProductLines product-lines/email_spec7_product23.cil.c ILP32 true -ProductLines product-lines/email_spec7_product31.cil.c ILP32 false -ProductLines product-lines/email_spec8_product14.cil.c ILP32 true -ProductLines product-lines/email_spec8_product26.cil.c ILP32 false -ProductLines product-lines/email_spec8_product34.cil.c ILP32 false -ProductLines product-lines/email_spec9_product16.cil.c ILP32 false -ProductLines product-lines/email_spec9_product30.cil.c ILP32 false -XCSP xcsp/AllInterval-005.c ILP32 false -XCSP xcsp/AllInterval-007.c ILP32 false -XCSP xcsp/AllInterval-010.c ILP32 false -XCSP xcsp/AllInterval-013.c ILP32 false -XCSP xcsp/AllInterval-016.c ILP32 false -XCSP xcsp/AllInterval-019.c ILP32 false -XCSP xcsp/AllInterval-030.c ILP32 false -XCSP xcsp/CostasArray-11.c ILP32 false -XCSP xcsp/CostasArray-14.c ILP32 false -XCSP xcsp/CostasArray-17.c ILP32 false -XCSP xcsp/Dubois-017.c ILP32 true -XCSP xcsp/Dubois-020.c ILP32 true -XCSP xcsp/Dubois-023.c ILP32 true -XCSP xcsp/Dubois-026.c ILP32 true -XCSP xcsp/Dubois-029.c ILP32 true -XCSP xcsp/Dubois-040.c ILP32 true -XCSP xcsp/Dubois-055.c ILP32 true -XCSP xcsp/Dubois-070.c ILP32 true -XCSP xcsp/Dubois-085.c ILP32 true -XCSP xcsp/Dubois-100.c ILP32 true -XCSP xcsp/Haystacks-08.c ILP32 true -XCSP xcsp/Haystacks-11.c ILP32 true XCSP xcsp/Haystacks-14.c ILP32 true -XCSP xcsp/Haystacks-17.c ILP32 true +LinkedLists memsafety-broom/dll-middle-shared.i LP64 unknown +Arrays array-fpi/s4liff_overflow.c ILP32 unknown +ControlFlow openssl/s3_srvr.blast.15.i.cil-2.c ILP32 unknown +Heap list-ext2-properties/simple_and_skiplist_2lvl-2.i ILP32 true +Loops nla-digbench-scaling/fermat1-ll_unwindbound20.c ILP32 false +Recursive recursified_nla-digbench/recursified_knuth.i ILP32 unknown +Sequentialized seq-mthreaded/pals_opt-floodmax.5.1.ufo.BOUNDED-10.pals.c ILP32 false +Floats neural-networks/dubinsrejoin_72_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem10_label13.c ILP32 true +ProductLines product-lines/email_spec1_product14.cil.c ILP32 false +XCSP xcsp/Haystacks-16.c ILP32 true +LinkedLists memsafety-broom/linux-hlist-lst-data.i LP64 unknown +Arrays array-fpi/s5iff.c ILP32 false +ControlFlow unsignedintegeroverflow-sas23/bilinear_interpolation.i ILP32 true +Heap memsafety-cve/NanoNNG.i ILP32 unknown +Loops nla-digbench-scaling/fermat2-ll_unwindbound100.c ILP32 false +Recursive recursified_nla-digbench/recursified_mannadiv.c ILP32 unknown +Sequentialized seq-mthreaded/pals_opt-floodmax.5.2.ufo.UNBOUNDED.pals.c ILP32 false +Floats neural-networks/dubinsrejoin_84_unsafe.c-amalgamation.i ILP32 false +ECA eca-rers2012/Problem10_label30.c ILP32 true +ProductLines product-lines/email_spec1_product22.cil.c ILP32 false +XCSP xcsp/Haystacks-18.c ILP32 true +LinkedLists memsafety-broom/linux-list-lst-data.i LP64 unknown +Arrays array-fpi/sina2.c ILP32 true +ControlFlow unsignedintegeroverflow-sas23/distance_through_overflow.i ILP32 true +Heap memsafety-cve/bzip3Fixed.i ILP32 unknown +Loops nla-digbench-scaling/freire1_valuebound1.c ILP32 true +Recursive recursified_nla-digbench/recursified_prod4br.c ILP32 unknown +Sequentialized seq-mthreaded/pals_opt-floodmax.5.ufo.BOUNDED-10.pals.c ILP32 true +Floats neural-networks/dubinsrejoin_9_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem10_label46.c ILP32 false +ProductLines product-lines/email_spec1_product30.cil.c ILP32 false XCSP xcsp/aim-100-1-6-sat-1.c ILP32 false -XCSP xcsp/aim-100-1-6-sat-4.c ILP32 false -XCSP xcsp/aim-100-1-6-unsat-3.c ILP32 true +LinkedLists memsafety-broom/sll-fst-data.i LP64 unknown +Arrays array-fpi/sina5f.c ILP32 false +ControlFlow unsignedintegeroverflow-sas23/linear_interpolation_2.i ILP32 true +Heap memsafety-cve/editorconfig-core-cFixed.i ILP32 unknown +Loops nla-digbench-scaling/freire2_valuebound5.c ILP32 false +Recursive recursified_nla-digbench/recursified_prodbin.c ILP32 unknown +Sequentialized seq-pthread/cs_dekker.i ILP32 true +Floats neural-networks/exp_1_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem11_label03.c ILP32 true +ProductLines product-lines/email_spec1_product34.cil.c ILP32 false +XCSP xcsp/aim-100-1-6-sat-3.c ILP32 false +LinkedLists memsafety-broom/sll-lst-data.i LP64 unknown +Arrays array-fpi/ss2f.c ILP32 false +Heap memsafety-cve/gpac_1.i ILP32 unknown +Loops nla-digbench-scaling/geo1-ll_unwindbound1.c ILP32 true +Recursive recursified_nla-digbench/recursified_ps2.c ILP32 unknown +Sequentialized seq-pthread/cs_fib-1.i ILP32 true +Floats neural-networks/gcas_6_unsafe.c-amalgamation.i ILP32 false +ECA eca-rers2012/Problem11_label19.c ILP32 true +ProductLines product-lines/email_spec27_product17.cil.c ILP32 false +XCSP xcsp/aim-100-1-6-unsat-2.c ILP32 true +LinkedLists memsafety-broom/sll-middle-shared.i LP64 unknown +Arrays array-industry-pattern/array_monotonic.i ILP32 true +Heap memsafety-cve/gpac_16Fixed.i ILP32 unknown +Loops nla-digbench-scaling/geo1-ll_valuebound5.c ILP32 true +Recursive recursified_nla-digbench/recursified_ps3.c ILP32 unknown +Sequentialized seq-pthread/cs_fib_longer-2.i ILP32 false +Floats neural-networks/log_3_unsafe.c-amalgamation.i ILP32 false +ECA eca-rers2012/Problem11_label36.c ILP32 false +ProductLines product-lines/email_spec27_product24.cil.c ILP32 false +XCSP xcsp/aim-100-1-6-unsat-4.c ILP32 true +LinkedLists memsafety-broom/sll-nested-sll-inline.i LP64 unknown +Arrays array-industry-pattern/array_ptr_single_elem_init-2.i ILP32 false +Heap memsafety-cve/gpac_19.i ILP32 unknown +Loops nla-digbench-scaling/geo1-u_unwindbound1.c ILP32 false +Recursive recursified_nla-digbench/recursified_ps4.c ILP32 unknown +Sequentialized seq-pthread/cs_lazy.i ILP32 unknown +Floats neural-networks/lunarlander_0_unsafe.c-amalgamation.i ILP32 false +ECA eca-rers2012/Problem11_label52.c ILP32 true +ProductLines product-lines/email_spec27_product29.cil.c ILP32 false XCSP xcsp/aim-100-2-0-sat-2.c ILP32 false -XCSP xcsp/aim-100-2-0-unsat-1.c ILP32 true +LinkedLists memsafety-broom/sll-nested-sll-twice.i LP64 unknown +Arrays array-lopstr16/single_elem_safe.i ILP32 true +Heap memsafety-cve/gpac_6.i ILP32 unknown +Loops nla-digbench-scaling/geo1-u_valuebound5.c ILP32 false +Recursive recursified_nla-digbench/recursified_ps5.c ILP32 unknown +Sequentialized seq-pthread/cs_queue-1.i ILP32 true +Floats neural-networks/lunarlander_21_unsafe.c-amalgamation.i ILP32 false +ECA eca-rers2012/Problem12_label09.c ILP32 true +ProductLines product-lines/email_spec27_product33.cil.c ILP32 false +XCSP xcsp/aim-100-2-0-sat-4.c ILP32 false +LinkedLists memsafety-broom/sll-shared-before.i LP64 unknown +Arrays array-memsafety/array02-alloca-2.i ILP32 unknown +Heap memsafety-cve/jhead.i ILP32 unknown +Loops nla-digbench-scaling/geo2-ll_unwindbound1.c ILP32 true +Recursive recursified_nla-digbench/recursified_ps6.c ILP32 unknown +Sequentialized seq-pthread/cs_read_write_lock-2.i ILP32 false +Floats neural-networks/lunarlander_33_unsafe.c-amalgamation.i ILP32 false +ECA eca-rers2012/Problem12_label25.c ILP32 false +ProductLines product-lines/email_spec3_product13.cil.c ILP32 false +XCSP xcsp/aim-100-2-0-unsat-2.c ILP32 true +LinkedLists memsafety-broom/sll-shared-sll-before.i LP64 unknown +Arrays array-memsafety/cstrchr-alloca-2.i ILP32 unknown +Heap memsafety-cve/kilo.i ILP32 unknown +Loops nla-digbench-scaling/geo2-ll_valuebound5.c ILP32 true +Recursive recursified_nla-digbench/recursified_sqrt1.c ILP32 unknown +Sequentialized seq-pthread/cs_stack-2.i ILP32 false +Floats neural-networks/lunarlander_45_unsafe.c-amalgamation.i ILP32 false +ECA eca-rers2012/Problem12_label42.c ILP32 false +ProductLines product-lines/email_spec3_product24.cil.c ILP32 false XCSP xcsp/aim-100-2-0-unsat-4.c ILP32 true -XCSP xcsp/aim-100-3-4-sat-3.c ILP32 false -XCSP xcsp/aim-100-6-0-sat-2.c ILP32 false +LinkedLists memsafety-broom/sll.i LP64 unknown +Arrays array-memsafety/cstrncat_unsafe.c ILP32 unknown +Heap memsafety-cve/liblouis.i ILP32 unknown +Loops nla-digbench-scaling/geo3-ll_unwindbound1.c ILP32 true +Recursive recursive-simple/id2_b3_o2.c ILP32 false +Sequentialized seq-pthread/cs_stateful-2.i ILP32 true +Floats neural-networks/lunarlander_57_unsafe.c-amalgamation.i ILP32 false +ECA eca-rers2012/Problem12_label58.c ILP32 true +ProductLines product-lines/email_spec3_product29.cil.c ILP32 false +XCSP xcsp/aim-100-3-4-sat-2.c ILP32 false +Arrays array-memsafety/diff-alloca-2.i ILP32 unknown +Heap memsafety-cve/libredwg_1.i ILP32 unknown +Loops nla-digbench-scaling/geo3-ll_valuebound5.c ILP32 true +Recursive recursive-simple/id2_b5_o10.c ILP32 true +Sequentialized seq-pthread/cs_time_var_mutex.i ILP32 true +Floats neural-networks/lunarlander_69_unsafe.c-amalgamation.i ILP32 false +ECA eca-rers2012/Problem13_label15.c ILP32 true +ProductLines product-lines/email_spec3_product33.cil.c ILP32 false +XCSP xcsp/aim-100-6-0-sat-1.c ILP32 false +Arrays array-memsafety/mult_array_unsafe.i ILP32 unknown +Heap memsafety-cve/minizip-ng.i ILP32 unknown +Loops nla-digbench-scaling/hard-ll_valuebound2.c ILP32 true +Recursive recursive-simple/id_b3_o2-1.c ILP32 unknown +Sequentialized systemc/token_ring.01.cil-2.c ILP32 unknown +Floats neural-networks/lunarlander_81_unsafe.c-amalgamation.i ILP32 false +ECA eca-rers2012/Problem13_label31.c ILP32 true +ProductLines product-lines/email_spec4_product13.cil.c ILP32 true +XCSP xcsp/aim-100-6-0-sat-3.c ILP32 false +Arrays array-memsafety/openbsd_cstrcat-alloca-2.i ILP32 unknown +Heap memsafety-cve/mongooseFixed.i ILP32 unknown +Loops nla-digbench-scaling/hard-u_valuebound10.c ILP32 true +Recursive recursive-simple/id_b3_o5-1.c ILP32 unknown +Sequentialized systemc/token_ring.02.cil-2.c ILP32 unknown +Floats neural-networks/lunarlander_92_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem13_label48.c ILP32 false +ProductLines product-lines/email_spec4_product23.cil.c ILP32 false XCSP xcsp/aim-200-1-6-sat-1.c ILP32 false -XCSP xcsp/aim-200-1-6-sat-4.c ILP32 false +Arrays array-memsafety/openbsd_cstrncpy-alloca-2.i ILP32 unknown +Heap memsafety-cve/netatalk.i ILP32 unknown +Loops nla-digbench-scaling/hard2_unwindbound50.c ILP32 unknown +Recursive recursive-simple/id_b5_o10-1.c ILP32 unknown +Floats neural-networks/poly_1024_thresh_5_unsafe.c-amalgamation.i ILP32 false +ECA eca-rers2012/Problem15_label04.c ILP32 true +ProductLines product-lines/email_spec4_product28.cil.c ILP32 true +XCSP xcsp/aim-200-1-6-sat-3.c ILP32 false +Arrays array-memsafety/selectionsort-alloca-1.i ILP32 unknown +Heap memsafety-cve/openSC.i ILP32 unknown +Loops nla-digbench-scaling/knuth_unwindbound20.i ILP32 true +Recursive recursive-simple/id_o10.c ILP32 false +Floats neural-networks/poly_16_16_16_16_thresh_0_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem15_label22.c ILP32 false +ProductLines product-lines/email_spec4_product32.cil.c ILP32 false +XCSP xcsp/aim-200-1-6-unsat-1.c ILP32 true +Arrays array-multidimensional/add-2-n-u.c ILP32 true +Heap memsafety-cve/openrazerFound.i ILP32 unknown +Loops nla-digbench-scaling/lcm1_unwindbound20.c ILP32 false +Recursive recursive-simple/id_o1000.c ILP32 false +Floats neural-networks/poly_16_16_thresh_2_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem15_label39.c ILP32 false +ProductLines product-lines/email_spec4_productSimulator.cil.c ILP32 false XCSP xcsp/aim-200-1-6-unsat-3.c ILP32 true -XCSP xcsp/aim-200-2-0-sat-2.c ILP32 false -XCSP xcsp/aim-200-2-0-unsat-1.c ILP32 true +Arrays array-multidimensional/diff-3-n-u.c ILP32 true +Heap memsafety-cve/plutovg.i ILP32 unknown +Loops nla-digbench-scaling/lcm2_unwindbound100.c ILP32 true +Recursive recursive-simple/id_o3.c ILP32 false +Floats neural-networks/poly_32_32_32_32_thresh_3_unsafe.c-amalgamation.i ILP32 false +ECA eca-rers2012/Problem15_label56.c ILP32 true +ProductLines product-lines/email_spec6_product20.cil.c ILP32 false +XCSP xcsp/aim-200-2-0-sat-1.c ILP32 false +Arrays array-multidimensional/min-2-u.c ILP32 true +Heap memsafety-cve/radare2_1Fixed.i ILP32 unknown +Loops nla-digbench-scaling/mannadiv_unwindbound1.c ILP32 false +Recursive recursive-simple/sum_non_eq-1.c ILP32 unknown +Floats neural-networks/poly_32_32_thresh_4_unsafe.c-amalgamation.i ILP32 false +ECA eca-rers2012/Problem16_label12.c ILP32 true +ProductLines product-lines/email_spec6_product28.cil.c ILP32 false +XCSP xcsp/aim-200-2-0-sat-4.c ILP32 false +Arrays array-patterns/array12_pattern.c ILP32 true +Heap memsafety-cve/smartdns.i ILP32 unknown +Loops nla-digbench-scaling/prod4br-ll_unwindbound5.c ILP32 false +Recursive recursive-simple/sum_non_eq-3.c ILP32 false +Floats neural-networks/poly_512_thresh_5_unsafe.c-amalgamation.i ILP32 false +ECA eca-rers2012/Problem16_label29.c ILP32 true +ProductLines product-lines/email_spec6_product32.cil.c ILP32 false +XCSP xcsp/aim-200-2-0-unsat-2.c ILP32 true +Arrays array-patterns/array1_pattern.c ILP32 true +Heap memsafety-cve/sofia-sipFixed.i ILP32 unknown +Loops nla-digbench-scaling/prodbin-ll_unwindbound2.c ILP32 false +Recursive recursive/Ackermann02.c ILP32 false +Floats neural-networks/poly_8_8_8_8_thresh_1_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem16_label45.c ILP32 true +ProductLines product-lines/email_spec6_productSimulator.cil.c ILP32 false XCSP xcsp/aim-200-2-0-unsat-4.c ILP32 true -XCSP xcsp/aim-200-3-4-sat-3.c ILP32 false +Arrays array-patterns/array25_pattern.c ILP32 true +Heap memsafety-cve/wasm-micro-runtime.i ILP32 unknown +Loops nla-digbench-scaling/ps2-ll_unwindbound10.c ILP32 true +Recursive recursive/Ackermann04.c ILP32 true +Floats neural-networks/relu_2_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem17_label02.c ILP32 true +ProductLines product-lines/email_spec7_product19.cil.c ILP32 true +XCSP xcsp/aim-200-3-4-sat-2.c ILP32 false +Arrays array-patterns/array3_pattern.c ILP32 true +Heap memsafety-cve/zephyr.i ILP32 unknown +Loops nla-digbench-scaling/ps2-ll_valuebound50.c ILP32 true +Recursive recursive/Addition02.c ILP32 false +Floats neural-networks/sin_2_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem17_label18.c ILP32 true +ProductLines product-lines/email_spec7_product27.cil.c ILP32 true +XCSP xcsp/aim-200-3-4-sat-4.c ILP32 false +Arrays array-patterns/array7_pattern.c ILP32 true +Heap memsafety-ext/skiplist_3lvl.i ILP32 unknown +Loops nla-digbench-scaling/ps3-ll_valuebound20.c ILP32 true +Recursive recursive/Addition03-1.c ILP32 true +Floats neural-networks/softmax_3_unsafe.c-amalgamation.i ILP32 false +ECA eca-rers2012/Problem17_label35.c ILP32 false +ProductLines product-lines/email_spec7_product31.cil.c ILP32 false XCSP xcsp/aim-200-6-0-sat-2.c ILP32 false -LinkedLists forester-heap/dll-01-1.i ILP32 false -LinkedLists forester-heap/dll-circular-1.i ILP32 false -LinkedLists forester-heap/dll-optional-2.i ILP32 false -LinkedLists forester-heap/dll-queue-2.i ILP32 false -LinkedLists forester-heap/dll-rb-sentinel-1.i ILP32 false -LinkedLists forester-heap/dll-simple-white-blue-1.i ILP32 false -LinkedLists forester-heap/dll-sorted-1.i ILP32 false -LinkedLists forester-heap/dll-token-2.i ILP32 false -LinkedLists forester-heap/sll-01-2.i ILP32 false -LinkedLists forester-heap/sll-circular-1.i ILP32 true -LinkedLists forester-heap/sll-optional-2.i ILP32 false -LinkedLists forester-heap/sll-queue-2.i ILP32 false -LinkedLists forester-heap/sll-rb-sentinel-1.i ILP32 true -LinkedLists forester-heap/sll-reverse_simple.i ILP32 true -LinkedLists forester-heap/sll-sorted-1.i ILP32 false -LinkedLists forester-heap/sll-token-2.i ILP32 true -LinkedLists heap-manipulation/bubble_sort_linux-2.i ILP32 false -LinkedLists heap-manipulation/dll_of_dll-2.i ILP32 false -LinkedLists heap-manipulation/merge_sort-2.i ILP32 true -LinkedLists heap-manipulation/tree-1.i ILP32 unknown -LinkedLists heap-manipulation/tree-4.i ILP32 false -LinkedLists list-ext3-properties/dll_nondet_free_order-2.i ILP32 unknown -LinkedLists list-ext3-properties/sll_length_check-2.i ILP32 true -LinkedLists list-ext3-properties/sll_nondet_insert-2.i ILP32 true -LinkedLists list-properties/alternating_list-1.i ILP32 true -LinkedLists list-properties/list-2.i ILP32 false -LinkedLists list-properties/list_flag-2.i ILP32 true -LinkedLists list-properties/simple_built_from_end.i ILP32 true -LinkedLists list-properties/splice-2.i ILP32 true -LinkedLists memsafety-broom/dll-fst-shared.i LP64 unknown -LinkedLists memsafety-broom/dll-middle-data.i LP64 unknown -LinkedLists memsafety-broom/dll.i LP64 unknown -LinkedLists memsafety-broom/linux-hlist-middle-data.i LP64 unknown -LinkedLists memsafety-broom/linux-list-lst-data.i LP64 unknown -LinkedLists memsafety-broom/sll-fst-shared.i LP64 unknown -LinkedLists memsafety-broom/sll-middle-data.i LP64 unknown -LinkedLists memsafety-broom/sll-nested-linux-list.i LP64 unknown -LinkedLists memsafety-broom/sll-nested-sll-twice.i LP64 unknown -LinkedLists memsafety-broom/sll-shared-after.i LP64 unknown -LinkedLists memsafety-broom/sll-shared-sll-before.i LP64 unknown +Arrays array-programs/partial_mod_count_limited_1.c ILP32 true +Heap memsafety-ext2/complex_data_creation_test02-1.i ILP32 unknown +Loops nla-digbench-scaling/ps4-ll_valuebound100.c ILP32 true +Recursive recursive/BallRajamani-SPIN2000-Fig1.c ILP32 false +Floats neural-networks/softsign_1_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem17_label51.c ILP32 true +ProductLines product-lines/email_spec7_product35.cil.c ILP32 false +XCSP xcsp/aim-200-6-0-sat-4.c ILP32 false +Arrays array-tiling/mbpr2.c ILP32 true +Heap memsafety-ext2/optional_data_creation_test04-2.i ILP32 unknown +Loops nla-digbench-scaling/ps5-ll_valuebound1.c ILP32 true +Recursive recursive/EvenOdd03.c ILP32 false +Floats neural-networks/softsign_w32_r1_case_0_unsafe.c-amalgamation.i ILP32 false +ECA eca-rers2012/Problem18_label08.c ILP32 false +ProductLines product-lines/email_spec8_product16.cil.c ILP32 false +Arrays array-tiling/nr4.c ILP32 true +Heap memsafety-ext3/realloc1.c ILP32 unknown +Loops nla-digbench-scaling/ps6-ll_unwindbound5.c ILP32 false +Recursive recursive/Fibonacci01-1.c ILP32 true +Floats neural-networks/softsign_w4_r3_case_1_unsafe.c-amalgamation.i ILP32 false +ECA eca-rers2012/Problem18_label24.c ILP32 true +ProductLines product-lines/email_spec8_product26.cil.c ILP32 false +Arrays array-tiling/pr2.c ILP32 true +Heap memsafety/global-atexit-5.i ILP32 unknown +Loops nla-digbench-scaling/sqrt1-ll_unwindbound2.c ILP32 true +Recursive recursive/Fibonacci04-overflow.c ILP32 unknown +Floats neural-networks/softsign_w8_r2_case_0_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem18_label41.c ILP32 true +ProductLines product-lines/email_spec8_product31.cil.c ILP32 false +Arrays array-tiling/rewnifrev2.c ILP32 true +Heap memsafety/lockfree-3.2.i ILP32 unknown +Loops nla-digbench/bresenham.c ILP32 unknown +Recursive recursive/Fibonacci05-overflow.c ILP32 unknown +Floats neural-networks/tanh_0_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem18_label57.c ILP32 false +ProductLines product-lines/email_spec8_product35.cil.c ILP32 false +Arrays reducercommutativity/avg10-2.i ILP32 true +Heap memsafety/test-0137.i ILP32 unknown +Loops nla-digbench/egcd2.c ILP32 unknown +Recursive recursive/McCarthy91-1.c ILP32 false +Floats neural-networks/tanh_w16_r3_case_0_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem19_label14.c ILP32 false +ProductLines product-lines/email_spec9_product15.cil.c ILP32 false +Arrays reducercommutativity/max40-1.i ILP32 true +Heap memsafety/test-0232-2.i ILP32 unknown +Loops nla-digbench/geo2-ll.c ILP32 true +Recursive recursive/MultCommutative-2.c ILP32 true +Floats neural-networks/tanh_w4_r2_case_0_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem19_label30.c ILP32 true +ProductLines product-lines/email_spec9_product22.cil.c ILP32 false +Arrays reducercommutativity/sep.i ILP32 true +Heap memsafety/test-0235-2.i ILP32 unknown +Loops nla-digbench/lcm2.c ILP32 true +Recursive recursive/gcd02.c ILP32 true +Floats neural-networks/tanh_w64_r4_case_1_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem19_label47.c ILP32 false +ProductLines product-lines/email_spec9_product30.cil.c ILP32 false +Arrays reducercommutativity/sum10-1.i ILP32 true +Heap memsafety/test-0504.i ILP32 unknown +Loops nla-digbench/ps5-ll.c ILP32 true +Recursive recursive/recHanoi02-2.c ILP32 true +Floats neural-networks/vdp_3_unsafe.c-amalgamation.i ILP32 false +ECA eca-rers2018/Problem14.c ILP32 false +ProductLines product-lines/email_spec9_product34.cil.c ILP32 false +Arrays array-cav19/array_init_nondet_vars.c ILP32 true +Heap heap-data/calendar.i ILP32 true +Loops loop-acceleration/array_2-1.i ILP32 false +Recursive recursified_nla-digbench/recursified_cohencu.c ILP32 unknown +Floats float-benchs/cast_float_ptr.c ILP32 false +ECA eca-programs/Problem101_label03.c ILP32 false +ProductLines product-lines/elevator_spec14_product03.cil.c ILP32 true +Arrays array-cav19/array_init_var_plus_ind2.c ILP32 true +Heap heap-data/hash_fun.i ILP32 true +Loops loop-acceleration/diamond_1-2.c ILP32 false +Recursive recursified_nla-digbench/recursified_lcm1.c ILP32 unknown +Floats float-benchs/divmul_diverge.c ILP32 true +ECA eca-programs/Problem101_label11.c ILP32 false +ProductLines product-lines/elevator_spec14_product19.cil.c ILP32 true +Arrays array-cav19/array_tiling_tcpy.c ILP32 true +Heap heap-data/process_queue.i ILP32 true +Loops loop-acceleration/simple_2-2.c ILP32 false +Recursive recursive-simple/id_o20.c ILP32 false +Floats float-benchs/filter2.c ILP32 true +ECA eca-programs/Problem101_label19.c ILP32 false +ProductLines product-lines/elevator_spec14_product23.cil.c ILP32 true +Arrays array-crafted/bAnd4.i ILP32 true +Heap heap-data/running_example.i ILP32 true +Loops loop-crafted/simple_array_index_value_4.i.v+nlh-reducer.c ILP32 true +Recursive recursive/recHanoi03-2.c ILP32 true +Floats float-benchs/image_filter.c ILP32 true +ECA eca-programs/Problem102_label03.c ILP32 true +ProductLines product-lines/elevator_spec14_product27.cil.c ILP32 true +Arrays array-crafted/bor2.i ILP32 true +Heap heap-data/shared_mem2.i ILP32 true +Loops loop-industry-pattern/mod3.c.v+lhb-reducer.c ILP32 true +Floats float-benchs/interpolation2.c.v+nlh-reducer.c ILP32 true +ECA eca-programs/Problem102_label12.c ILP32 true +ProductLines product-lines/elevator_spec14_product31.cil.c ILP32 true +Arrays array-crafted/mapavg1.i ILP32 true +Heap ldv-memsafety/memleaks_test1-2.i ILP32 unknown +Loops loop-invariants/linear-inequality-inv-c.c ILP32 true +Floats float-benchs/inv_sqrt_Quake.c ILP32 true +ECA eca-programs/Problem102_label20.c ILP32 true +ProductLines product-lines/elevator_spec14_productSimulator.cil.c ILP32 false +Arrays array-crafted/mapavg5.i ILP32 true +Heap ldv-memsafety/memleaks_test10-1.i ILP32 unknown +Loops loop-invgen/apache-escape-absolute.i ILP32 true +Floats float-benchs/nan_float.c ILP32 false +ECA eca-programs/Problem102_label28.c ILP32 false +ProductLines product-lines/elevator_spec1_product03.cil.c ILP32 true +Arrays array-crafted/mapsum4.i ILP32 true +Heap ldv-memsafety/memleaks_test11.i ILP32 unknown +Loops loop-invgen/apache-get-tag.i ILP32 true +Floats float-benchs/sin_interpolated_bigrange_loose.c ILP32 true +ECA eca-programs/Problem102_label36.c ILP32 true +ProductLines product-lines/elevator_spec1_product11.cil.c ILP32 true +Arrays array-crafted/xor2.i ILP32 true +Heap ldv-memsafety/memleaks_test12-1.i ILP32 unknown +Loops loop-invgen/id_build.i.p+sep-reducer.c ILP32 true +Floats float-benchs/sqrt_Householder_interval.c ILP32 true +ECA eca-programs/Problem102_label45.c ILP32 true +ProductLines product-lines/elevator_spec1_product18.cil.c ILP32 false +Arrays array-crafted/zero_sum1.c ILP32 true +Heap ldv-memsafety/memleaks_test13.i ILP32 unknown +Loops loop-invgen/nested6.i ILP32 true +Floats float-benchs/zonotope_3.c.p+cfa-reducer.c ILP32 true +ECA eca-programs/Problem102_label53.c ILP32 true +ProductLines product-lines/elevator_spec1_product20.cil.c ILP32 false +Arrays array-crafted/zero_sum5.c ILP32 true +Heap ldv-memsafety/memleaks_test13_2.i ILP32 unknown +Loops loop-lit/bhmr2007.i ILP32 true +Floats float-newlib/double_req_bl_0320.c ILP32 true +ECA eca-programs/Problem103_label35.c ILP32 false +ProductLines product-lines/elevator_spec1_product22.cil.c ILP32 false +Arrays array-crafted/zero_sum_const4.c ILP32 true +Heap ldv-memsafety/memleaks_test14_1.i ILP32 unknown +Loops loop-lit/gcnr2008.i ILP32 unknown +Floats float-newlib/double_req_bl_0662a.c ILP32 true +ECA eca-programs/Problem103_label47.c ILP32 false +ProductLines product-lines/elevator_spec1_product24.cil.c ILP32 false +Arrays array-crafted/zero_sum_const_m4.c ILP32 true +Heap ldv-memsafety/memleaks_test15-1.i ILP32 unknown +Loops loop-lit/gsv2008.c.i.p+cfa-reducer.c ILP32 true +Floats float-newlib/double_req_bl_0683a.c ILP32 true +ECA eca-programs/Problem103_label56.c ILP32 false +ProductLines product-lines/elevator_spec1_product26.cil.c ILP32 false +Arrays array-crafted/zero_sum_m4.c ILP32 true +Heap ldv-memsafety/memleaks_test16.i ILP32 unknown +Loops loop-lit/jm2006.i ILP32 true +Floats float-newlib/double_req_bl_0870b.c ILP32 true +ECA eca-rers2012/Problem03_label04.c ILP32 true +ProductLines product-lines/elevator_spec1_product28.cil.c ILP32 false +Arrays array-examples/data_structures_set_multi_proc_trivial_ground.i ILP32 true +Heap ldv-memsafety/memleaks_test16_2.i ILP32 unknown +Loops loop-new/count_by_k.i ILP32 true +Floats float-newlib/double_req_bl_0920a.c ILP32 true +ECA eca-rers2012/Problem03_label12.c ILP32 true +ProductLines product-lines/elevator_spec1_product30.cil.c ILP32 false +Arrays array-examples/sanfoundry_02_ground.i ILP32 true +Heap ldv-memsafety/memleaks_test17_1-2.i ILP32 unknown +Loops loop-new/half.i ILP32 true +Floats float-newlib/double_req_bl_1131a.c ILP32 true +ECA eca-rers2012/Problem03_label20.c ILP32 true +ProductLines product-lines/elevator_spec1_product32.cil.c ILP32 false +Arrays array-examples/sanfoundry_24-2.i ILP32 unknown +Heap ldv-memsafety/memleaks_test17_3.i ILP32 unknown +Loops loop-simple/nested_4.c ILP32 true +Floats float-newlib/float_req_bl_0281.c ILP32 true +ECA eca-rers2012/Problem03_label29.c ILP32 true +ProductLines product-lines/elevator_spec2_product03.cil.c ILP32 true +Arrays array-examples/sorting_bubblesort_ground-1.i ILP32 true +Heap ldv-memsafety/memleaks_test18_1.i ILP32 unknown +Loops loop-zilu/benchmark03_linear.i ILP32 unknown +Floats float-newlib/float_req_bl_0662a.c ILP32 true +ECA eca-rers2012/Problem03_label37.c ILP32 false +ProductLines product-lines/elevator_spec2_product11.cil.c ILP32 true +Arrays array-examples/sorting_selectionsort_ground-2.i ILP32 true +Heap ldv-memsafety/memleaks_test18_3.i ILP32 unknown +Loops loop-zilu/benchmark09_conjunctive.i ILP32 true +Floats float-newlib/float_req_bl_0683a.c ILP32 true +ECA eca-rers2012/Problem03_label45.c ILP32 false +ProductLines product-lines/elevator_spec2_product18.cil.c ILP32 false +Arrays array-examples/standard_copy1_ground-1.i ILP32 true +Heap ldv-memsafety/memleaks_test19-2.i ILP32 unknown +Loops loop-zilu/benchmark14_linear.i ILP32 true +Floats float-newlib/float_req_bl_0870b.c ILP32 true +ECA eca-rers2012/Problem03_label53.c ILP32 true +ProductLines product-lines/elevator_spec2_product20.cil.c ILP32 false +Arrays array-examples/standard_copy2_ground-2.i ILP32 true +Heap ldv-memsafety/memleaks_test20-2.i ILP32 unknown +Loops loop-zilu/benchmark20_conjunctive.i ILP32 true +Floats float-newlib/float_req_bl_0880.c ILP32 true +ECA eca-rers2012/Problem04_label02.c ILP32 true +ProductLines product-lines/elevator_spec2_product22.cil.c ILP32 false +Arrays array-examples/standard_copy4_ground-2.i ILP32 false +Heap ldv-memsafety/memleaks_test21-2.i ILP32 unknown +Loops loop-zilu/benchmark25_linear_abstracted.i ILP32 true +Floats float-newlib/float_req_bl_1121b.c ILP32 true +ECA eca-rers2012/Problem04_label10.c ILP32 true +ProductLines product-lines/elevator_spec2_product24.cil.c ILP32 false +Arrays array-examples/standard_copy6_ground-2.i ILP32 true +Heap ldv-memsafety/memleaks_test22_1-2.i ILP32 unknown +Loops loop-zilu/benchmark30_conjunctive.i ILP32 unknown +Floats float-newlib/float_req_bl_1211a.c ILP32 true +ECA eca-rers2012/Problem04_label25.c ILP32 true +ProductLines product-lines/elevator_spec2_product26.cil.c ILP32 false diff --git a/tests/testcomp/corpus/cover-error.tsv b/tests/testcomp/corpus/cover-error.tsv index 1e0fbfd60..c6666ecb9 100644 --- a/tests/testcomp/corpus/cover-error.tsv +++ b/tests/testcomp/corpus/cover-error.tsv @@ -1,373 +1,819 @@ # category program data_model expected_unreach -Arrays array-examples/data_structures_set_multi_proc_ground-1.i ILP32 false -Arrays array-examples/sorting_bubblesort_ground-2.i ILP32 false -Arrays array-examples/standard_allDiff2_ground.i ILP32 false -Arrays array-examples/standard_copy2_ground-1.i ILP32 false -Arrays array-examples/standard_copy5_ground-2.i ILP32 false -Arrays array-examples/standard_copy7_ground-1.i ILP32 false -Arrays array-examples/standard_minInArray_ground-1.i ILP32 false -Arrays array-examples/standard_running-1.i ILP32 false -Arrays array-fpi/brs3f.c ILP32 false -Arrays array-fpi/brs5f.c ILP32 false -Arrays array-fpi/condmf.c ILP32 false -Arrays array-fpi/eqn1f.c ILP32 false -Arrays array-fpi/eqn4f.c ILP32 false -Arrays array-fpi/ifcompf.c ILP32 false -Arrays array-fpi/ifeqn3f.c ILP32 false -Arrays array-fpi/ifeqn5f.c ILP32 false -Arrays array-fpi/indp2f.c ILP32 false -Arrays array-fpi/indp4f.c ILP32 false -Arrays array-fpi/modpf.c ILP32 false -Arrays array-fpi/ms1f.c ILP32 false -Arrays array-fpi/ms4f.c ILP32 false -Arrays array-fpi/ncompf.c ILP32 false -Arrays array-fpi/pcompf.c ILP32 false -Arrays array-fpi/res1of.c ILP32 false -Arrays array-fpi/s12iff.c ILP32 false -Arrays array-fpi/s1liff.c ILP32 false -Arrays array-fpi/s2liff.c ILP32 false -Arrays array-fpi/s3iff.c ILP32 false -Arrays array-fpi/s4iff.c ILP32 false -Arrays array-fpi/s52iff.c ILP32 false -Arrays array-fpi/sina1f.c ILP32 false -Arrays array-fpi/sina3f.c ILP32 false -Arrays array-fpi/sqm-iff.c ILP32 false -Arrays array-fpi/ss1f.c ILP32 false -Arrays array-fpi/ss4f.c ILP32 false -Arrays array-industry-pattern/array_ptr_single_elem_init-2.i ILP32 false -Arrays array-programs/copysome1-2.i ILP32 false -Arrays array-tiling/mlceu.c ILP32 false -Arrays reducercommutativity/rangesum05.i ILP32 false -Arrays reducercommutativity/rangesum20.i ILP32 false -BitVectors bitvector-loops/diamond_2-1.c ILP32 false -BitVectors bitvector-loops/verisec_sendmail_tTflag_arr_one_loop.c ILP32 false -BitVectors bitvector-regression/recHanoi03-1.c ILP32 false -BitVectors bitvector/byte_add-1.i ILP32 false +Arrays array-fpi/ms3f.c ILP32 false BitVectors bitvector/s3_clnt_2.BV.c.cil-2a.c ILP32 false -BitVectors bitvector/s3_clnt_3.BV.c.cil-2a.c ILP32 false -BitVectors bitvector/soft_float_1-3a.c.cil.c ILP32 false -BitVectors bitvector/soft_float_4-3a.c.cil.c ILP32 false -BitVectors bitvector/sum02-1.c ILP32 false -ControlFlow ntdrivers/cdaudio.i.cil-1.c ILP32 false -ControlFlow ntdrivers/diskperf.i.cil-2.c ILP32 false ControlFlow ntdrivers/floppy.i.cil-1.c ILP32 false +Heap ldv-regression/test29-1.c ILP32 false +Loops nla-digbench-scaling/fermat1-ll_unwindbound20.c ILP32 false +Recursive recursified_loop-invariants/recursified_linear-inequality-inv-b.c ILP32 unknown +Sequentialized seq-mthreaded/pals_lcr-var-start-time.6.1.ufo.UNBOUNDED.pals.c ILP32 false +Floats floats-cdfpl/newton_1_7.i ILP32 false +ECA eca-rers2012/Problem10_label12.c ILP32 false +ProductLines product-lines/email_spec27_product34.cil.c ILP32 false +XCSP xcsp/aim-100-1-6-sat-3.c ILP32 false +LinkedLists forester-heap/sll-simple-white-blue-2.i ILP32 false +Arrays array-fpi/condgf.c ILP32 false +BitVectors bitvector-loops/verisec_sendmail_tTflag_arr_one_loop.c ILP32 false +ControlFlow ntdrivers/cdaudio.i.cil-1.c ILP32 false +Heap ldv-regression/test22-3.c ILP32 false +Loops nla-digbench-scaling/bresenham-ll_unwindbound10.c ILP32 false +Recursive fuzzle-programs/08_fuzzle_50x50_50-cycle.c LP64 false +Sequentialized seq-mthreaded/pals_floodmax.4.3.ufo.BOUNDED-8.pals.c ILP32 false +Floats float-benchs/nan_double.c ILP32 false +ECA eca-rers2012/Problem05_label40.c ILP32 false +ProductLines product-lines/email_spec0_product22.cil.c ILP32 false +XCSP xcsp/AllInterval-019.c ILP32 false +LinkedLists forester-heap/dll-sorted-1.i ILP32 false +Arrays array-fpi/s5liff.c ILP32 false +BitVectors bitvector/soft_float_1-3a.c.cil.c ILP32 false ControlFlow ntdrivers/kbfiltr.i.cil-2.c ILP32 false -ControlFlow ntdrivers/parport.i.cil-1.c ILP32 false -Heap ldv-regression/fo_test.i ILP32 false +Heap ldv-sets/test_mutex_unlock_at_exit.i ILP32 false +Loops nla-digbench-scaling/prod4br-ll_unwindbound100.c ILP32 false +Recursive recursive-simple/id_o3.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.3.4.ufo.BOUNDED-6.pals.c ILP32 false +Floats floats-cdfpl/sine_1.i ILP32 false +ECA eca-rers2012/Problem16_label01.c ILP32 false +ProductLines product-lines/email_spec6_product29.cil.c ILP32 false +XCSP xcsp/aim-200-1-6-sat-2.c ILP32 false +LinkedLists list-ext3-properties/dll_nullified-1.i ILP32 false +Arrays array-examples/standard_copy6_ground-1.i ILP32 false +BitVectors bitvector-loops/diamond_2-1.c ILP32 false +ControlFlow ntdrivers/diskperf.i.cil-2.c ILP32 false Heap ldv-regression/rule57_ebda_blast_2.i ILP32 false -Heap ldv-regression/rule60_list2.c_1.i ILP32 false -Heap ldv-regression/test21-2.c ILP32 false -Heap ldv-regression/test22-3.c ILP32 false -Heap ldv-regression/test23-1.c ILP32 false +Loops loops/insertion_sort-1-2.c ILP32 false +Recursive fuzzle-programs/04_fuzzle_60x60.c LP64 false +Sequentialized seq-mthreaded/pals_floodmax.3.1.ufo.BOUNDED-6.pals.c ILP32 false +Floats float-benchs/float_int_inv_square.c ILP32 false +ECA eca-programs/Problem103_label44.c ILP32 false +ProductLines product-lines/elevator_spec2_product30.cil.c ILP32 false +XCSP xcsp/AllInterval-011.c ILP32 false +LinkedLists forester-heap/dll-queue-2.i ILP32 false +Arrays array-fpi/ifeqn4f.c ILP32 false +BitVectors bitvector-regression/recHanoi03-1.c ILP32 false +ControlFlow ntdrivers/parport.i.cil-1.c ILP32 false Heap ldv-regression/test24-2.c ILP32 false -Heap ldv-regression/test25-1.c ILP32 false -Heap ldv-regression/test28-1.c ILP32 false -Heap ldv-regression/test29-1.c ILP32 false -Heap ldv-sets/test_add-1.i ILP32 false +Loops nla-digbench-scaling/egcd2-ll_unwindbound100.c ILP32 false +Recursive fuzzle-programs/12_fuzzle_50x50_CVE-2016-4492.c LP64 false +Sequentialized seq-mthreaded/pals_floodmax.5_overflow.ufo.UNBOUNDED.pals.c ILP32 false +Floats float-newlib/double_req_bl_0870a.c ILP32 false +ECA eca-rers2012/Problem07_label44.c ILP32 false +ProductLines product-lines/email_spec1_product22.cil.c ILP32 false +XCSP xcsp/CostasArray-12.c ILP32 false +LinkedLists forester-heap/sll-circular-2.i ILP32 false +Arrays array-fpi/s1iff.c ILP32 false +BitVectors bitvector/s3_clnt_3.BV.c.cil-2a.c ILP32 false Heap ldv-sets/test_mutex_double_lock.i ILP32 false -Heap ldv-sets/test_mutex_double_unlock.i ILP32 false -Heap ldv-sets/test_mutex_unbounded-2.i ILP32 false -Heap ldv-sets/test_mutex_unlock_at_exit.i ILP32 false -Heap list-ext-properties/list-ext.i ILP32 false -Heap list-ext-properties/simple-ext.i ILP32 false -Heap list-ext2-properties/simple_and_skiplist_2lvl-1.i ILP32 false -Heap list-ext2-properties/simple_search_value-2.i ILP32 false -Loops loop-acceleration/array_2-1.i ILP32 false -Loops loop-acceleration/multivar_1-2.c ILP32 false -Loops loop-invariants/linear-inequality-inv-b.c ILP32 false -Loops loops/array-2.c ILP32 false -Loops loops/count_up_down-2.c ILP32 false -Loops loops/invert_string-1.c ILP32 false -Loops loops/n.c24.i ILP32 false -Loops loops/sum01_bug02.i ILP32 false -Loops loops/trex01-1.c ILP32 false -Loops loops/vogal-2.i ILP32 false -Loops nla-digbench-scaling/bresenham-ll_unwindbound100.c ILP32 false -Loops nla-digbench-scaling/bresenham-ll_unwindbound5.c ILP32 false -Loops nla-digbench-scaling/egcd-ll_unwindbound10.c ILP32 false -Loops nla-digbench-scaling/egcd-ll_unwindbound20.c ILP32 false -Loops nla-digbench-scaling/egcd2-ll_unwindbound1.c ILP32 false -Loops nla-digbench-scaling/egcd2-ll_unwindbound2.c ILP32 false -Loops nla-digbench-scaling/egcd2-ll_unwindbound50.c ILP32 false -Loops nla-digbench-scaling/egcd3-ll_unwindbound100.c ILP32 false -Loops nla-digbench-scaling/egcd3-ll_unwindbound5.c ILP32 false -Loops nla-digbench-scaling/fermat1-ll_unwindbound10.c ILP32 false -Loops nla-digbench-scaling/fermat1-ll_unwindbound20.c ILP32 false -Loops nla-digbench-scaling/fermat2-ll_unwindbound1.c ILP32 false -Loops nla-digbench-scaling/fermat2-ll_unwindbound2.c ILP32 false -Loops nla-digbench-scaling/fermat2-ll_unwindbound50.c ILP32 false -Loops nla-digbench-scaling/freire2_valuebound100.c ILP32 false Loops nla-digbench-scaling/freire2_valuebound5.c ILP32 false -Loops nla-digbench-scaling/hard2_unwindbound10.c ILP32 false -Loops nla-digbench-scaling/mannadiv_unwindbound1.c ILP32 false -Loops nla-digbench-scaling/mannadiv_unwindbound2.c ILP32 false -Loops nla-digbench-scaling/mannadiv_unwindbound50.c ILP32 false -Loops nla-digbench-scaling/prod4br-ll_unwindbound100.c ILP32 false -Loops nla-digbench-scaling/prod4br-ll_unwindbound5.c ILP32 false -Loops nla-digbench-scaling/prodbin-ll_unwindbound10.c ILP32 false -Loops nla-digbench-scaling/prodbin-ll_unwindbound5.c ILP32 false -Loops nla-digbench-scaling/ps4-ll_unwindbound100.c ILP32 false +Recursive recursive-simple/id_o100.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr.6.1.ufo.UNBOUNDED.pals.c ILP32 false +Floats floats-cdfpl/newton_2_8.i ILP32 false +ECA eca-rers2012/Problem12_label51.c ILP32 false +ProductLines product-lines/email_spec4_product19.cil.c ILP32 false +XCSP xcsp/aim-100-3-4-sat-2.c ILP32 false +LinkedLists heap-manipulation/dll_of_dll-2.i ILP32 false +Arrays array-industry-pattern/array_ptr_single_elem_init-2.i ILP32 false +BitVectors bitvector/soft_float_4-3a.c.cil.c ILP32 false +Heap list-ext-properties/simple-ext.i ILP32 false Loops nla-digbench-scaling/ps4-ll_unwindbound5.c ILP32 false -Loops nla-digbench-scaling/ps5-ll_unwindbound10.c ILP32 false -Loops nla-digbench-scaling/ps5-ll_unwindbound20.c ILP32 false -Loops nla-digbench-scaling/ps6-ll_unwindbound1.c ILP32 false -Loops nla-digbench-scaling/ps6-ll_unwindbound2.c ILP32 false -Recursive fuzzle-programs/01_fuzzle_30x30.c LP64 false +Recursive recursive/BallRajamani-SPIN2000-Fig1.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.5.1.ufo.UNBOUNDED.pals.c ILP32 false +Floats floats-cdfpl/square_2.i ILP32 false +ECA eca-rers2012/Problem18_label09.c ILP32 false +ProductLines product-lines/email_spec8_product22.cil.c ILP32 false +XCSP xcsp/aim-200-3-4-sat-1.c ILP32 false +LinkedLists list-properties/alternating_list-2.i ILP32 false +Arrays array-examples/standard_allDiff2_ground.i ILP32 false +BitVectors bitvector/byte_add-1.i ILP32 false +Heap ldv-regression/fo_test.i ILP32 false +Loops loop-invariants/linear-inequality-inv-b.c ILP32 false Recursive fuzzle-programs/02_fuzzle_40x40.c LP64 false -Recursive fuzzle-programs/03_fuzzle_50x50.c LP64 false -Recursive fuzzle-programs/04_fuzzle_60x60.c LP64 false -Recursive fuzzle-programs/05_fuzzle_70x70.c LP64 false +Sequentialized seq-mthreaded/pals_STARTPALS_ActiveStandby.4_2.ufo.UNBOUNDED.pals.c ILP32 false +Floats float-benchs/cast_union_loose.c ILP32 false +ECA eca-programs/Problem102_label02.c ILP32 false +ProductLines product-lines/elevator_spec1_product26.cil.c ILP32 false +XCSP xcsp/AllInterval-007.c ILP32 false +LinkedLists forester-heap/dll-circular-1.i ILP32 false +Arrays array-examples/standard_running-1.i ILP32 false +BitVectors bitvector/sum02-1.c ILP32 false +Heap ldv-regression/rule60_list2.c_1.i ILP32 false +Loops loops/sum01_bug02.i ILP32 false Recursive fuzzle-programs/06_fuzzle_50x50_0-cycle.c LP64 false -Recursive fuzzle-programs/07_fuzzle_50x50_25-cycle.c LP64 false -Recursive fuzzle-programs/08_fuzzle_50x50_50-cycle.c LP64 false -Recursive fuzzle-programs/09_fuzzle_50x50_75-cycle.c LP64 false +Sequentialized seq-mthreaded/pals_floodmax.3.4.ufo.BOUNDED-6.pals.c ILP32 false +Floats float-benchs/inv_Newton.c.p+cfa-reducer.c ILP32 false +ECA eca-rers2012/Problem04_label15.c ILP32 false +ProductLines product-lines/elevator_spec3_product28.cil.c ILP32 false +XCSP xcsp/AllInterval-015.c ILP32 false +LinkedLists forester-heap/dll-rb-sentinel-1.i ILP32 false +Arrays array-fpi/eqn4f.c ILP32 false +Heap ldv-regression/test23-1.c ILP32 false +Loops nla-digbench-scaling/egcd-ll_unwindbound10.c ILP32 false Recursive fuzzle-programs/10_fuzzle_50x50_CVE-2016-4487.c LP64 false -Recursive fuzzle-programs/11_fuzzle_50x50_CVE-2016-4489.c LP64 false -Recursive fuzzle-programs/12_fuzzle_50x50_CVE-2016-4492.c LP64 false -Recursive fuzzle-programs/13_fuzzle_50x50_CVE-2016-4493.c LP64 false +Sequentialized seq-mthreaded/pals_floodmax.5.1.ufo.UNBOUNDED.pals.c ILP32 false +Floats float-benchs/sin_interpolated_index-1.c ILP32 false +ECA eca-rers2012/Problem06_label37.c ILP32 false +ProductLines product-lines/email_spec11_product26.cil.c ILP32 false +XCSP xcsp/AllInterval-030.c ILP32 false +LinkedLists forester-heap/sll-01-2.i ILP32 false +Arrays array-fpi/indp4f.c ILP32 false +Heap ldv-regression/test25-1.c ILP32 false +Loops nla-digbench-scaling/egcd3-ll_unwindbound2.c ILP32 false Recursive fuzzle-programs/14_fuzzle_50x50_CVE-2016-6131.c LP64 false -Recursive fuzzle-programs/15_fuzzle_50x50_equality-checks.c LP64 false -Recursive recursified_loop-invariants/recursified_linear-inequality-inv-b.c ILP32 unknown -Recursive recursive-simple/id2_b3_o2.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr-var-start-time.4.2.ufo.BOUNDED-8.pals.c ILP32 false +Floats floats-cdfpl/newton_1_4.i ILP32 false +ECA eca-rers2012/Problem08_label51.c ILP32 false +ProductLines product-lines/email_spec27_product18.cil.c ILP32 false +XCSP xcsp/CostasArray-16.c ILP32 false +LinkedLists forester-heap/sll-queue-2.i ILP32 false +Arrays array-fpi/pcompf.c ILP32 false +Heap ldv-sets/test_add-1.i ILP32 false +Loops nla-digbench-scaling/fermat2-ll_unwindbound20.c ILP32 false Recursive recursive-simple/id_b3_o2-2.c ILP32 false -Recursive recursive-simple/id_o10.c ILP32 false -Recursive recursive-simple/id_o100.c ILP32 false -Recursive recursive-simple/id_o1000.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr.4.1.ufo.BOUNDED-8.pals.c ILP32 false +Floats floats-cdfpl/newton_2_6.i ILP32 false +ECA eca-rers2012/Problem11_label43.c ILP32 false +ProductLines product-lines/email_spec3_product27.cil.c ILP32 false +XCSP xcsp/aim-100-2-0-sat-2.c ILP32 false +LinkedLists forester-heap/sll-token-1.i ILP32 false +Arrays array-fpi/s3iff.c ILP32 false +Heap ldv-sets/test_mutex_double_unlock.i ILP32 false +Loops nla-digbench-scaling/mannadiv_unwindbound10.c ILP32 false Recursive recursive-simple/id_o20.c ILP32 false -Recursive recursive-simple/id_o200.c ILP32 false -Recursive recursive-simple/id_o3.c ILP32 false -Recursive recursive-simple/sum_non_eq-3.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr.8_overflow.ufo.UNBOUNDED.pals.c ILP32 false +Floats floats-cdfpl/newton_3_7.i ILP32 false +ECA eca-rers2012/Problem13_label54.c ILP32 false +ProductLines product-lines/email_spec4_product35.cil.c ILP32 false +XCSP xcsp/aim-100-6-0-sat-2.c ILP32 false +LinkedLists heap-manipulation/sll_to_dll_rev-1.i ILP32 false +Arrays array-fpi/sqm-iff.c ILP32 false +Heap list-ext-properties/list-ext.i ILP32 false +Loops nla-digbench-scaling/prodbin-ll_unwindbound2.c ILP32 false Recursive recursive/Ackermann02.c ILP32 false -Recursive recursive/Addition02.c ILP32 false -Recursive recursive/BallRajamani-SPIN2000-Fig1.c ILP32 false -Recursive recursive/EvenOdd03.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.4.2.ufo.UNBOUNDED.pals.c ILP32 false +Floats floats-cdfpl/sine_3.i ILP32 false +ECA eca-rers2012/Problem17_label09.c ILP32 false +ProductLines product-lines/email_spec7_product30.cil.c ILP32 false +XCSP xcsp/aim-200-2-0-sat-1.c ILP32 false +LinkedLists list-ext3-properties/sll_nondet_insert-1.i ILP32 false +Arrays array-tiling/skippedu.c ILP32 false +Heap list-ext2-properties/simple_and_skiplist_2lvl-1.i ILP32 false +Loops nla-digbench-scaling/ps5-ll_unwindbound50.c ILP32 false Recursive recursive/Fibonacci04.c ILP32 false -Recursive recursive/Fibonacci05.c ILP32 false -Recursive recursive/McCarthy91-1.c ILP32 false -Sequentialized seq-mthreaded/pals_STARTPALS_ActiveStandby.1.ufo.BOUNDED-10.pals.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.5_overflow.ufo.UNBOUNDED.pals.c ILP32 false +Floats loop-floats-scientific-comp/loop1-2.c ILP32 false +ECA eca-rers2012/Problem19_label21.c ILP32 false +ProductLines product-lines/email_spec9_product20.cil.c ILP32 false +XCSP xcsp/aim-200-6-0-sat-1.c ILP32 false +LinkedLists list-properties/list_flag-1.i ILP32 false +Arrays array-examples/sorting_bubblesort_ground-2.i ILP32 false +Heap ldv-regression/test21-2.c ILP32 false +Loops loop-acceleration/diamond_1-2.c ILP32 false +Recursive fuzzle-programs/01_fuzzle_30x30.c LP64 false Sequentialized seq-mthreaded/pals_STARTPALS_ActiveStandby.4_1.ufo.BOUNDED-10.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_STARTPALS_ActiveStandby.4_2.ufo.UNBOUNDED.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_STARTPALS_ActiveStandby.5.ufo.UNBOUNDED.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_STARTPALS_Triplicated.2.ufo.BOUNDED-10.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_floodmax.3.1.ufo.UNBOUNDED.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_floodmax.3.2.ufo.UNBOUNDED.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_floodmax.3.4.ufo.BOUNDED-6.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_floodmax.4.1.ufo.BOUNDED-8.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_floodmax.4.2.ufo.BOUNDED-8.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_floodmax.4.3.ufo.UNBOUNDED.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_floodmax.4_overflow.ufo.UNBOUNDED.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_floodmax.5.1.ufo.UNBOUNDED.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_floodmax.5.3.ufo.BOUNDED-10.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_floodmax.5.4.ufo.UNBOUNDED.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_lcr-var-start-time.3.1.ufo.BOUNDED-6.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_lcr-var-start-time.3.2.ufo.UNBOUNDED.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_lcr-var-start-time.4.2.ufo.BOUNDED-8.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_lcr-var-start-time.5.1.ufo.BOUNDED-10.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_lcr-var-start-time.5.2.ufo.UNBOUNDED.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_lcr-var-start-time.6.2.ufo.BOUNDED-12.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_lcr.3.1.ufo.BOUNDED-6.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_lcr.4.1.ufo.BOUNDED-8.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_lcr.4_overflow.ufo.UNBOUNDED.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_lcr.5_overflow.ufo.UNBOUNDED.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_lcr.6_overflow.ufo.UNBOUNDED.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_lcr.7.1.ufo.UNBOUNDED.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_lcr.8.1.ufo.UNBOUNDED.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_opt-floodmax.3.1.ufo.UNBOUNDED.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_opt-floodmax.3.2.ufo.UNBOUNDED.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_opt-floodmax.3.4.ufo.BOUNDED-6.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_opt-floodmax.4.1.ufo.BOUNDED-8.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_opt-floodmax.4.2.ufo.BOUNDED-8.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_opt-floodmax.4.3.ufo.UNBOUNDED.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_opt-floodmax.4_overflow.ufo.UNBOUNDED.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_opt-floodmax.5.1.ufo.UNBOUNDED.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_opt-floodmax.5.3.ufo.BOUNDED-10.pals.c ILP32 false -Sequentialized seq-mthreaded/pals_opt-floodmax.5.4.ufo.UNBOUNDED.pals.c ILP32 false -Sequentialized seq-pthread/cs_fib-2.i ILP32 false -Sequentialized seq-pthread/cs_read_write_lock-2.i ILP32 false Floats float-benchs/cast_float_ptr.c ILP32 false -Floats float-benchs/cast_union_loose.c ILP32 false +ECA eca-programs/Problem101_label12.c ILP32 false +ProductLines product-lines/elevator_spec14_productSimulator.cil.c ILP32 false +XCSP xcsp/AllInterval-005.c ILP32 false +LinkedLists forester-heap/dll-01-1.i ILP32 false +Arrays array-examples/standard_copy3_ground-2.i ILP32 false +Heap ldv-regression/test28-1.c ILP32 false +Loops loops/bubble_sort-2.i ILP32 false +Recursive fuzzle-programs/03_fuzzle_50x50.c LP64 false +Sequentialized seq-mthreaded/pals_STARTPALS_Triplicated.1.ufo.BOUNDED-10.pals.c ILP32 false Floats float-benchs/cast_union_tight.c ILP32 false -Floats float-benchs/float_int_inv_square.c ILP32 false +ECA eca-programs/Problem102_label34.c ILP32 false +ProductLines product-lines/elevator_spec2_product18.cil.c ILP32 false +XCSP xcsp/AllInterval-009.c ILP32 false +LinkedLists forester-heap/dll-optional-2.i ILP32 false +Arrays array-examples/standard_copy9_ground-1.i ILP32 false +Heap ldv-sets/test_mutex_unbounded-2.i ILP32 false +Loops loops/matrix-2-2.c ILP32 false +Recursive fuzzle-programs/05_fuzzle_70x70.c LP64 false +Sequentialized seq-mthreaded/pals_floodmax.3.2.ufo.UNBOUNDED.pals.c ILP32 false Floats float-benchs/inv_Newton-2.c ILP32 false -Floats float-benchs/inv_Newton.c.p+cfa-reducer.c ILP32 false +ECA eca-rers2012/Problem03_label28.c ILP32 false +ProductLines product-lines/elevator_spec3_product19.cil.c ILP32 false +XCSP xcsp/AllInterval-013.c ILP32 false +LinkedLists forester-heap/dll-rb-cnstr_1-2.i ILP32 false +Arrays array-fpi/brs3f.c ILP32 false +Heap list-ext2-properties/simple_search_value-2.i ILP32 false +Loops loops/trex02-2.c ILP32 false +Recursive fuzzle-programs/07_fuzzle_50x50_25-cycle.c LP64 false +Sequentialized seq-mthreaded/pals_floodmax.4.1.ufo.BOUNDED-8.pals.c ILP32 false Floats float-benchs/inv_square-1.c ILP32 false -Floats float-benchs/nan_double.c ILP32 false +ECA eca-rers2012/Problem05_label01.c ILP32 false +ProductLines product-lines/elevator_spec9_product28.cil.c ILP32 false +XCSP xcsp/AllInterval-017.c ILP32 false +LinkedLists forester-heap/dll-simple-white-blue-1.i ILP32 false +Arrays array-fpi/eqn1f.c ILP32 false +Loops nla-digbench-scaling/bresenham-ll_unwindbound20.c ILP32 false +Recursive fuzzle-programs/09_fuzzle_50x50_75-cycle.c LP64 false +Sequentialized seq-mthreaded/pals_floodmax.4.4.ufo.UNBOUNDED.pals.c ILP32 false Floats float-benchs/nan_float.c ILP32 false -Floats float-benchs/sin_interpolated_index-1.c ILP32 false +ECA eca-rers2012/Problem06_label05.c ILP32 false +ProductLines product-lines/email_spec0_product35.cil.c ILP32 false +XCSP xcsp/AllInterval-020.c ILP32 false +LinkedLists forester-heap/dll-token-2.i ILP32 false +Arrays array-fpi/ifeqn1f.c ILP32 false +Loops nla-digbench-scaling/egcd-ll_unwindbound5.c ILP32 false +Recursive fuzzle-programs/11_fuzzle_50x50_CVE-2016-4489.c LP64 false +Sequentialized seq-mthreaded/pals_floodmax.5.3.ufo.BOUNDED-10.pals.c ILP32 false Floats float-benchs/sqrt_poly2.c ILP32 false -Floats float-newlib/double_req_bl_0870a.c ILP32 false +ECA eca-rers2012/Problem07_label11.c ILP32 false +ProductLines product-lines/email_spec11_productSimulator.cil.c ILP32 false +XCSP xcsp/CostasArray-10.c ILP32 false +LinkedLists forester-heap/sll-buckets-2.i ILP32 false +Arrays array-fpi/indp1f.c ILP32 false +Loops nla-digbench-scaling/egcd2-ll_unwindbound50.c ILP32 false +Recursive fuzzle-programs/13_fuzzle_50x50_CVE-2016-4493.c LP64 false +Sequentialized seq-mthreaded/pals_lcr-var-start-time.3.2.ufo.BOUNDED-6.pals.c ILP32 false Floats float-newlib/float_req_bl_0870a.c ILP32 false -Floats floats-cdfpl/newton_1_4.i ILP32 false +ECA eca-rers2012/Problem08_label15.c ILP32 false +ProductLines product-lines/email_spec1_product32.cil.c ILP32 false +XCSP xcsp/CostasArray-14.c ILP32 false +LinkedLists forester-heap/sll-optional-2.i ILP32 false +Arrays array-fpi/modpf.c ILP32 false +Loops nla-digbench-scaling/fermat1-ll_unwindbound1.c ILP32 false +Recursive fuzzle-programs/15_fuzzle_50x50_equality-checks.c LP64 false +Sequentialized seq-mthreaded/pals_lcr-var-start-time.5.1.ufo.UNBOUNDED.pals.c ILP32 false Floats floats-cdfpl/newton_1_5.i ILP32 false -Floats floats-cdfpl/newton_1_6.i ILP32 false -Floats floats-cdfpl/newton_1_7.i ILP32 false +ECA eca-rers2012/Problem09_label34.c ILP32 false +ProductLines product-lines/email_spec27_product27.cil.c ILP32 false +XCSP xcsp/aim-100-1-6-sat-1.c ILP32 false +LinkedLists forester-heap/sll-rb-cnstr_1-2.i ILP32 false +Arrays array-fpi/ncompf.c ILP32 false +Loops nla-digbench-scaling/fermat2-ll_unwindbound1.c ILP32 false +Recursive recursive-simple/id2_b3_o2.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr.3.1.ufo.BOUNDED-6.pals.c ILP32 false Floats floats-cdfpl/newton_1_8.i ILP32 false -Floats floats-cdfpl/newton_2_6.i ILP32 false +ECA eca-rers2012/Problem10_label57.c ILP32 false +ProductLines product-lines/email_spec3_product18.cil.c ILP32 false +XCSP xcsp/aim-100-1-6-sat-4.c ILP32 false +LinkedLists forester-heap/sll-sorted-1.i ILP32 false +Arrays array-fpi/res2f.c ILP32 false +Loops nla-digbench-scaling/freire2_valuebound10.c ILP32 false +Recursive recursive-simple/id_o10.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr.5.1.ufo.BOUNDED-10.pals.c ILP32 false Floats floats-cdfpl/newton_2_7.i ILP32 false -Floats floats-cdfpl/newton_2_8.i ILP32 false +ECA eca-rers2012/Problem12_label21.c ILP32 false +ProductLines product-lines/email_spec3_product32.cil.c ILP32 false +XCSP xcsp/aim-100-2-0-sat-4.c ILP32 false +LinkedLists heap-manipulation/bubble_sort_linux-2.i ILP32 false +Arrays array-fpi/s2iff.c ILP32 false +Loops nla-digbench-scaling/hard2_unwindbound10.c ILP32 false +Recursive recursive-simple/id_o1000.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr.7.1.ufo.UNBOUNDED.pals.c ILP32 false Floats floats-cdfpl/newton_3_6.i ILP32 false -Floats floats-cdfpl/newton_3_7.i ILP32 false +ECA eca-rers2012/Problem13_label25.c ILP32 false +ProductLines product-lines/email_spec4_product30.cil.c ILP32 false +XCSP xcsp/aim-100-3-4-sat-4.c ILP32 false +LinkedLists heap-manipulation/merge_sort-1.i ILP32 false +Arrays array-fpi/s4iff.c ILP32 false +Loops nla-digbench-scaling/mannadiv_unwindbound5.c ILP32 false +Recursive recursive-simple/id_o200.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.3.2.ufo.BOUNDED-6.pals.c ILP32 false Floats floats-cdfpl/newton_3_8.i ILP32 false -Floats floats-cdfpl/sine_1.i ILP32 false +ECA eca-rers2012/Problem15_label30.c ILP32 false +ProductLines product-lines/email_spec6_product16.cil.c ILP32 false +XCSP xcsp/aim-100-6-0-sat-4.c ILP32 false +LinkedLists heap-manipulation/tree-2.i ILP32 false +Arrays array-fpi/sina3f.c ILP32 false +Loops nla-digbench-scaling/prod4br-ll_unwindbound5.c ILP32 false +Recursive recursive-simple/sum_non_eq-3.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.4.1.ufo.BOUNDED-8.pals.c ILP32 false Floats floats-cdfpl/sine_2.i ILP32 false -Floats floats-cdfpl/sine_3.i ILP32 false +ECA eca-rers2012/Problem16_label31.c ILP32 false +ProductLines product-lines/email_spec6_product34.cil.c ILP32 false +XCSP xcsp/aim-200-1-6-sat-3.c ILP32 false +LinkedLists list-ext3-properties/sll_length_check-1.i ILP32 false +Arrays array-fpi/ss2f.c ILP32 false +Loops nla-digbench-scaling/ps4-ll_unwindbound10.c ILP32 false +Recursive recursive/Addition02.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.4.4.ufo.BOUNDED-8.pals.c ILP32 false Floats floats-cdfpl/square_1.i ILP32 false -Floats floats-cdfpl/square_2.i ILP32 false +ECA eca-rers2012/Problem17_label40.c ILP32 false +ProductLines product-lines/email_spec7_product35.cil.c ILP32 false +XCSP xcsp/aim-200-2-0-sat-3.c ILP32 false +LinkedLists list-ext3-properties/sll_of_sll_nondet_append-2.i ILP32 false +Arrays array-programs/copysome1-2.i ILP32 false +Loops nla-digbench-scaling/ps5-ll_unwindbound100.c ILP32 false +Recursive recursive/EvenOdd03.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.5.3.ufo.BOUNDED-10.pals.c ILP32 false Floats floats-cdfpl/square_3.i ILP32 false -Floats loop-floats-scientific-comp/loop1-2.c ILP32 false +ECA eca-rers2012/Problem18_label38.c ILP32 false +ProductLines product-lines/email_spec8_product33.cil.c ILP32 false +XCSP xcsp/aim-200-3-4-sat-3.c ILP32 false +LinkedLists list-properties/list-2.i ILP32 false +Arrays reducercommutativity/rangesum10.i ILP32 false +Loops nla-digbench-scaling/ps6-ll_unwindbound2.c ILP32 false +Recursive recursive/Fibonacci05.c ILP32 false +Sequentialized seq-pthread/cs_queue-2.i ILP32 false Floats loop-floats-scientific-comp/loop2-1.c ILP32 false +ECA eca-rers2012/Problem19_label51.c ILP32 false +ProductLines product-lines/email_spec9_product31.cil.c ILP32 false +XCSP xcsp/aim-200-6-0-sat-3.c ILP32 false +LinkedLists list-properties/simple-1.i ILP32 false +Arrays array-examples/data_structures_set_multi_proc_ground-1.i ILP32 false +Loops loop-acceleration/array_2-1.i ILP32 false +Recursive recursive/McCarthy91-1.c ILP32 false +Sequentialized seq-mthreaded/pals_STARTPALS_ActiveStandby.1.ufo.BOUNDED-10.pals.c ILP32 false +Floats floats-cdfpl/newton_1_6.i ILP32 false +ECA eca-programs/Problem101_label05.c ILP32 false +ProductLines product-lines/elevator_spec14_product24.cil.c ILP32 false +XCSP xcsp/AllInterval-006.c ILP32 false +LinkedLists forester-heap/sll-rb-sentinel-2.i ILP32 false +Arrays array-examples/sorting_selectionsort_2_ground.i ILP32 false +Loops loop-acceleration/simple_2-2.c ILP32 false +Sequentialized seq-mthreaded/pals_STARTPALS_ActiveStandby.4_1.ufo.UNBOUNDED.pals.c ILP32 false Floats loop-floats-scientific-comp/loop4.i ILP32 false -ECA eca-programs/Problem101_label00.c ILP32 false -ECA eca-programs/Problem101_label10.c ILP32 false -ECA eca-programs/Problem101_label21.c ILP32 false +ECA eca-programs/Problem101_label18.c ILP32 false +ProductLines product-lines/elevator_spec1_product20.cil.c ILP32 false +XCSP xcsp/AllInterval-008.c ILP32 false +LinkedLists heap-manipulation/tree-4.i ILP32 false +Arrays array-examples/standard_copy1_ground-2.i ILP32 false +Loops loops-crafted-1/nested_delay_notd2.c ILP32 false +Sequentialized seq-mthreaded/pals_STARTPALS_ActiveStandby.5.ufo.BOUNDED-10.pals.c ILP32 false ECA eca-programs/Problem102_label13.c ILP32 false -ECA eca-programs/Problem102_label48.c ILP32 false -ECA eca-programs/Problem103_label46.c ILP32 false -ECA eca-rers2012/Problem03_label27.c ILP32 false -ECA eca-rers2012/Problem04_label04.c ILP32 false -ECA eca-rers2012/Problem04_label38.c ILP32 false -ECA eca-rers2012/Problem05_label18.c ILP32 false -ECA eca-rers2012/Problem05_label41.c ILP32 false -ECA eca-rers2012/Problem06_label02.c ILP32 false -ECA eca-rers2012/Problem06_label27.c ILP32 false +ProductLines product-lines/elevator_spec1_product30.cil.c ILP32 false +XCSP xcsp/AllInterval-010.c ILP32 false +LinkedLists list-properties/splice-1.i ILP32 false +Arrays array-examples/standard_copy4_ground-2.i ILP32 false +Loops loops/count_up_down-2.c ILP32 false +Sequentialized seq-mthreaded/pals_STARTPALS_Triplicated.2.ufo.BOUNDED-10.pals.c ILP32 false +ECA eca-programs/Problem102_label50.c ILP32 false +ProductLines product-lines/elevator_spec2_product24.cil.c ILP32 false +XCSP xcsp/AllInterval-012.c ILP32 false +Arrays array-examples/standard_copy7_ground-1.i ILP32 false +Loops loops/invert_string-1.c ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.3.1.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-programs/Problem103_label56.c ILP32 false +ProductLines product-lines/elevator_spec2_productSimulator.cil.c ILP32 false +XCSP xcsp/AllInterval-014.c ILP32 false +Arrays array-examples/standard_minInArray_ground-1.i ILP32 false +Loops loops/nec20.c ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.3.3.ufo.BOUNDED-6.pals.c ILP32 false +ECA eca-rers2012/Problem03_label45.c ILP32 false +ProductLines product-lines/elevator_spec3_product23.cil.c ILP32 false +XCSP xcsp/AllInterval-016.c ILP32 false +Arrays array-fpi/brs1f.c ILP32 false +Loops loops/terminator_01.c ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.3.4.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-rers2012/Problem04_label35.c ILP32 false +ProductLines product-lines/elevator_spec3_product32.cil.c ILP32 false +XCSP xcsp/AllInterval-018.c ILP32 false +Arrays array-fpi/brs5f.c ILP32 false +Loops loops/vogal-2.i ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.4.2.ufo.BOUNDED-8.pals.c ILP32 false +ECA eca-rers2012/Problem05_label30.c ILP32 false +ProductLines product-lines/elevator_spec9_productSimulator.cil.c ILP32 false +XCSP xcsp/AllInterval-025.c ILP32 false +Arrays array-fpi/condmf.c ILP32 false +Loops nla-digbench-scaling/bresenham-ll_unwindbound100.c ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.4.3.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-rers2012/Problem05_label55.c ILP32 false +ProductLines product-lines/email_spec0_product31.cil.c ILP32 false +XCSP xcsp/AllInterval-035.c ILP32 false +Arrays array-fpi/eqn2f.c ILP32 false +Loops nla-digbench-scaling/bresenham-ll_unwindbound50.c ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.4_overflow.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-rers2012/Problem06_label20.c ILP32 false +ProductLines product-lines/email_spec11_product15.cil.c ILP32 false +XCSP xcsp/CostasArray-11.c ILP32 false +Arrays array-fpi/eqn5f.c ILP32 false +Loops nla-digbench-scaling/egcd-ll_unwindbound2.c ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.5.2.ufo.BOUNDED-10.pals.c ILP32 false ECA eca-rers2012/Problem06_label58.c ILP32 false -ECA eca-rers2012/Problem07_label20.c ILP32 false -ECA eca-rers2012/Problem07_label44.c ILP32 false -ECA eca-rers2012/Problem08_label10.c ILP32 false -ECA eca-rers2012/Problem08_label43.c ILP32 false +ProductLines product-lines/email_spec11_product32.cil.c ILP32 false +XCSP xcsp/CostasArray-13.c ILP32 false +Arrays array-fpi/ifeqn2f.c ILP32 false +Loops nla-digbench-scaling/egcd2-ll_unwindbound1.c ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.5.4.ufo.BOUNDED-10.pals.c ILP32 false +ECA eca-rers2012/Problem07_label31.c ILP32 false +ProductLines product-lines/email_spec1_product16.cil.c ILP32 false +XCSP xcsp/CostasArray-15.c ILP32 false +Arrays array-fpi/ifeqn5f.c ILP32 false +Loops nla-digbench-scaling/egcd2-ll_unwindbound20.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr-var-start-time.3.1.ufo.BOUNDED-6.pals.c ILP32 false +ECA eca-rers2012/Problem08_label02.c ILP32 false +ProductLines product-lines/email_spec1_product29.cil.c ILP32 false +XCSP xcsp/CostasArray-17.c ILP32 false +Arrays array-fpi/indp2f.c ILP32 false +Loops nla-digbench-scaling/egcd3-ll_unwindbound10.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr-var-start-time.4.1.ufo.BOUNDED-8.pals.c ILP32 false +ECA eca-rers2012/Problem08_label34.c ILP32 false +ProductLines product-lines/email_spec1_product35.cil.c ILP32 false +XCSP xcsp/aim-100-1-6-sat-2.c ILP32 false +Arrays array-fpi/indp5f.c ILP32 false +Loops nla-digbench-scaling/egcd3-ll_unwindbound5.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr-var-start-time.4.2.ufo.UNBOUNDED.pals.c ILP32 false ECA eca-rers2012/Problem09_label08.c ILP32 false -ECA eca-rers2012/Problem09_label38.c ILP32 false -ECA eca-rers2012/Problem10_label12.c ILP32 false -ECA eca-rers2012/Problem10_label50.c ILP32 false -ECA eca-rers2012/Problem11_label31.c ILP32 false +ProductLines product-lines/email_spec27_product23.cil.c ILP32 false +XCSP xcsp/aim-100-2-0-sat-1.c ILP32 false +Arrays array-fpi/ms1f.c ILP32 false +Loops nla-digbench-scaling/fermat1-ll_unwindbound100.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr-var-start-time.5.2.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-rers2012/Problem09_label47.c ILP32 false +ProductLines product-lines/email_spec27_product31.cil.c ILP32 false +XCSP xcsp/aim-100-2-0-sat-3.c ILP32 false +Arrays array-fpi/ms4f.c ILP32 false +Loops nla-digbench-scaling/fermat1-ll_unwindbound5.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr-var-start-time.6.2.ufo.BOUNDED-12.pals.c ILP32 false +ECA eca-rers2012/Problem10_label41.c ILP32 false +ProductLines product-lines/email_spec27_productSimulator.cil.c ILP32 false +XCSP xcsp/aim-100-3-4-sat-1.c ILP32 false +Arrays array-fpi/nsqm-iff.c ILP32 false +Loops nla-digbench-scaling/fermat2-ll_unwindbound100.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr.3.1.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-rers2012/Problem11_label20.c ILP32 false +ProductLines product-lines/email_spec3_product23.cil.c ILP32 false +XCSP xcsp/aim-100-3-4-sat-3.c ILP32 false +Arrays array-fpi/res1f.c ILP32 false +Loops nla-digbench-scaling/fermat2-ll_unwindbound50.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr.4.1.ufo.UNBOUNDED.pals.c ILP32 false ECA eca-rers2012/Problem12_label06.c ILP32 false -ECA eca-rers2012/Problem12_label30.c ILP32 false -ECA eca-rers2012/Problem12_label52.c ILP32 false -ECA eca-rers2012/Problem13_label23.c ILP32 false -ECA eca-rers2012/Problem13_label44.c ILP32 false +ProductLines product-lines/email_spec3_product29.cil.c ILP32 false +XCSP xcsp/aim-100-6-0-sat-1.c ILP32 false +Arrays array-fpi/res2of.c ILP32 false +Loops nla-digbench-scaling/freire2_valuebound2.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr.5_overflow.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-rers2012/Problem12_label37.c ILP32 false +ProductLines product-lines/email_spec3_product35.cil.c ILP32 false +XCSP xcsp/aim-100-6-0-sat-3.c ILP32 false +Arrays array-fpi/s1liff.c ILP32 false +Loops nla-digbench-scaling/freire2_valuebound50.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr.6_overflow.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-rers2012/Problem13_label11.c ILP32 false +ProductLines product-lines/email_spec4_product24.cil.c ILP32 false +XCSP xcsp/aim-200-1-6-sat-1.c ILP32 false +Arrays array-fpi/s2liff.c ILP32 false +Loops nla-digbench-scaling/hard2_unwindbound5.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr.8.1.ufo.BOUNDED-16.pals.c ILP32 false +ECA eca-rers2012/Problem13_label36.c ILP32 false +ProductLines product-lines/email_spec4_product32.cil.c ILP32 false +XCSP xcsp/aim-200-1-6-sat-4.c ILP32 false +Arrays array-fpi/s3liff.c ILP32 false +Loops nla-digbench-scaling/mannadiv_unwindbound2.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.3.1.ufo.BOUNDED-6.pals.c ILP32 false ECA eca-rers2012/Problem15_label09.c ILP32 false -ECA eca-rers2012/Problem15_label38.c ILP32 false -ECA eca-rers2012/Problem16_label01.c ILP32 false -ECA eca-rers2012/Problem16_label27.c ILP32 false -ECA eca-rers2012/Problem16_label51.c ILP32 false +ProductLines product-lines/email_spec6_product12.cil.c ILP32 false +XCSP xcsp/aim-200-2-0-sat-2.c ILP32 false +Arrays array-fpi/s52iff.c ILP32 false +Loops nla-digbench-scaling/prod4br-ll_unwindbound1.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.3.3.ufo.BOUNDED-6.pals.c ILP32 false +ECA eca-rers2012/Problem15_label41.c ILP32 false +ProductLines product-lines/email_spec6_product22.cil.c ILP32 false +XCSP xcsp/aim-200-2-0-sat-4.c ILP32 false +Arrays array-fpi/sina1f.c ILP32 false +Loops nla-digbench-scaling/prod4br-ll_unwindbound2.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.3.4.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-rers2012/Problem16_label14.c ILP32 false +ProductLines product-lines/email_spec6_product31.cil.c ILP32 false +XCSP xcsp/aim-200-3-4-sat-2.c ILP32 false +Arrays array-fpi/sina4f.c ILP32 false +Loops nla-digbench-scaling/prodbin-ll_unwindbound1.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.4.1.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-rers2012/Problem16_label44.c ILP32 false +ProductLines product-lines/email_spec6_productSimulator.cil.c ILP32 false +XCSP xcsp/aim-200-3-4-sat-4.c ILP32 false +Arrays array-fpi/sqmf.c ILP32 false +Loops nla-digbench-scaling/prodbin-ll_unwindbound5.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.4.3.ufo.BOUNDED-8.pals.c ILP32 false ECA eca-rers2012/Problem17_label26.c ILP32 false -ECA eca-rers2012/Problem17_label50.c ILP32 false -ECA eca-rers2012/Problem18_label09.c ILP32 false -ECA eca-rers2012/Problem18_label34.c ILP32 false -ECA eca-rers2012/Problem19_label11.c ILP32 false -ECA eca-rers2012/Problem19_label29.c ILP32 false -ECA eca-rers2012/Problem19_label58.c ILP32 false +ProductLines product-lines/email_spec7_product32.cil.c ILP32 false +XCSP xcsp/aim-200-6-0-sat-2.c ILP32 false +Arrays array-fpi/ss4f.c ILP32 false +Loops nla-digbench-scaling/ps4-ll_unwindbound2.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.4_overflow.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-rers2012/Problem17_label55.c ILP32 false +ProductLines product-lines/email_spec8_product16.cil.c ILP32 false +XCSP xcsp/aim-200-6-0-sat-4.c ILP32 false +Arrays array-industry-pattern/array_single_elem_init.i ILP32 false +Loops nla-digbench-scaling/ps5-ll_unwindbound1.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.5.2.ufo.BOUNDED-10.pals.c ILP32 false +ECA eca-rers2012/Problem18_label27.c ILP32 false +ProductLines product-lines/email_spec8_product30.cil.c ILP32 false +Arrays array-programs/copysome2-2.i ILP32 false +Loops nla-digbench-scaling/ps5-ll_unwindbound20.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.5.4.ufo.BOUNDED-10.pals.c ILP32 false +ECA eca-rers2012/Problem18_label57.c ILP32 false +ProductLines product-lines/email_spec8_productSimulator.cil.c ILP32 false +Arrays reducercommutativity/rangesum.i ILP32 false +Loops nla-digbench-scaling/ps6-ll_unwindbound10.c ILP32 false +Sequentialized seq-pthread/cs_fib-2.i ILP32 false +ECA eca-rers2012/Problem19_label31.c ILP32 false +ProductLines product-lines/email_spec9_product22.cil.c ILP32 false +Arrays reducercommutativity/rangesum40.i ILP32 false +Loops nla-digbench-scaling/ps6-ll_unwindbound5.c ILP32 false +Sequentialized seq-pthread/cs_stack-2.i ILP32 false +ECA eca-rers2018/Problem12.c ILP32 false +ProductLines product-lines/email_spec9_product34.cil.c ILP32 false +Arrays array-examples/sorting_bubblesort_2_ground.i ILP32 false +Loops loop-acceleration/array_3-2.i ILP32 false +Sequentialized seq-mthreaded/pals_STARTPALS_ActiveStandby.1.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-programs/Problem101_label02.c ILP32 false ProductLines product-lines/elevator_spec14_product20.cil.c ILP32 false -ProductLines product-lines/elevator_spec14_productSimulator.cil.c ILP32 false -ProductLines product-lines/elevator_spec1_product24.cil.c ILP32 false +Arrays array-examples/sorting_selectionsort_ground-1.i ILP32 false +Loops loop-acceleration/multivar_1-2.c ILP32 false +Sequentialized seq-mthreaded/pals_STARTPALS_ActiveStandby.4_2.ufo.BOUNDED-10.pals.c ILP32 false +ECA eca-programs/Problem101_label08.c ILP32 false +ProductLines product-lines/elevator_spec14_product28.cil.c ILP32 false +Arrays array-examples/standard_copy2_ground-1.i ILP32 false +Loops loop-acceleration/simple_3-1.c ILP32 false +Sequentialized seq-mthreaded/pals_STARTPALS_ActiveStandby.5.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-programs/Problem101_label15.c ILP32 false +ProductLines product-lines/elevator_spec1_product18.cil.c ILP32 false +Arrays array-examples/standard_copy5_ground-2.i ILP32 false +Loops loop-invgen/id_trans.i ILP32 false +Sequentialized seq-mthreaded/pals_STARTPALS_Triplicated.1.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-programs/Problem101_label21.c ILP32 false +ProductLines product-lines/elevator_spec1_product22.cil.c ILP32 false +Arrays array-examples/standard_copy8_ground-2.i ILP32 false +Loops loops/array-2.c ILP32 false +Sequentialized seq-mthreaded/pals_STARTPALS_Triplicated.2.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-programs/Problem102_label06.c ILP32 false +ProductLines product-lines/elevator_spec1_product28.cil.c ILP32 false +Arrays array-examples/standard_partition_ground-1.i ILP32 false +Loops loops/compact.c ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.3.2.ufo.BOUNDED-6.pals.c ILP32 false +ECA eca-programs/Problem102_label25.c ILP32 false ProductLines product-lines/elevator_spec1_product32.cil.c ILP32 false -ProductLines product-lines/elevator_spec2_product22.cil.c ILP32 false +Arrays array-fpi/brs2f.c ILP32 false +Loops loops/for_bounded_loop1.c ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.3.3.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-programs/Problem102_label46.c ILP32 false +ProductLines product-lines/elevator_spec2_product20.cil.c ILP32 false +Arrays array-fpi/brs4f.c ILP32 false +Loops loops/linear_search.c ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.3_overflow.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-programs/Problem102_label59.c ILP32 false +ProductLines product-lines/elevator_spec2_product26.cil.c ILP32 false +Arrays array-fpi/condaf.c ILP32 false +Loops loops/n.c24.i ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.4.1.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-programs/Problem103_label51.c ILP32 false ProductLines product-lines/elevator_spec2_product32.cil.c ILP32 false -ProductLines product-lines/elevator_spec3_product19.cil.c ILP32 false -ProductLines product-lines/elevator_spec3_product27.cil.c ILP32 false +Arrays array-fpi/condnf.c ILP32 false +Loops loops/string-2.i ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.4.2.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-rers2012/Problem03_label09.c ILP32 false +ProductLines product-lines/elevator_spec3_product03.cil.c ILP32 false +Arrays array-fpi/eqn3f.c ILP32 false +Loops loops/sum03-1.i ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.4.4.ufo.BOUNDED-8.pals.c ILP32 false +ECA eca-rers2012/Problem03_label37.c ILP32 false +ProductLines product-lines/elevator_spec3_product20.cil.c ILP32 false +Arrays array-fpi/ifcompf.c ILP32 false +Loops loops/trex01-1.c ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.5.1.ufo.BOUNDED-10.pals.c ILP32 false +ECA eca-rers2012/Problem04_label04.c ILP32 false +ProductLines product-lines/elevator_spec3_product24.cil.c ILP32 false +Arrays array-fpi/ifeqn3f.c ILP32 false +Loops loops/trex03-1.c ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.5.2.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-rers2012/Problem04_label26.c ILP32 false +ProductLines product-lines/elevator_spec3_product31.cil.c ILP32 false +Arrays array-fpi/ifncompf.c ILP32 false +Loops nla-digbench-scaling/bresenham-ll_unwindbound1.c ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.5.3.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-rers2012/Problem04_label45.c ILP32 false ProductLines product-lines/elevator_spec3_productSimulator.cil.c ILP32 false -ProductLines product-lines/elevator_spec9_productSimulator.cil.c ILP32 false +Arrays array-fpi/indp3f.c ILP32 false +Loops nla-digbench-scaling/bresenham-ll_unwindbound2.c ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.5.4.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-rers2012/Problem05_label15.c ILP32 false +ProductLines product-lines/elevator_spec9_product30.cil.c ILP32 false +Arrays array-fpi/modnf.c ILP32 false +Loops nla-digbench-scaling/bresenham-ll_unwindbound5.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr-var-start-time.3.1.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-rers2012/Problem05_label36.c ILP32 false +ProductLines product-lines/email_spec0_product16.cil.c ILP32 false +Arrays array-fpi/modsf.c ILP32 false +Loops nla-digbench-scaling/egcd-ll_unwindbound1.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr-var-start-time.3.2.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-rers2012/Problem05_label47.c ILP32 false ProductLines product-lines/email_spec0_product26.cil.c ILP32 false -ProductLines product-lines/email_spec0_product35.cil.c ILP32 false -ProductLines product-lines/email_spec11_product22.cil.c ILP32 false +Arrays array-fpi/ms2f.c ILP32 false +Loops nla-digbench-scaling/egcd-ll_unwindbound100.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr-var-start-time.4.1.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-rers2012/Problem06_label00.c ILP32 false +ProductLines product-lines/email_spec0_product33.cil.c ILP32 false +Arrays array-fpi/ms5f.c ILP32 false +Loops nla-digbench-scaling/egcd-ll_unwindbound20.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr-var-start-time.5.1.ufo.BOUNDED-10.pals.c ILP32 false +ECA eca-rers2012/Problem06_label11.c ILP32 false +ProductLines product-lines/email_spec0_productSimulator.cil.c ILP32 false +Arrays array-fpi/nsqmf.c ILP32 false +Loops nla-digbench-scaling/egcd-ll_unwindbound50.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr-var-start-time.5.2.ufo.BOUNDED-10.pals.c ILP32 false +ECA eca-rers2012/Problem06_label27.c ILP32 false +ProductLines product-lines/email_spec11_product20.cil.c ILP32 false +Arrays array-fpi/res1of.c ILP32 false +Loops nla-digbench-scaling/egcd2-ll_unwindbound10.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr-var-start-time.6.1.ufo.BOUNDED-12.pals.c ILP32 false +ECA eca-rers2012/Problem06_label47.c ILP32 false +ProductLines product-lines/email_spec11_product30.cil.c ILP32 false +Arrays array-fpi/s12iff.c ILP32 false +Loops nla-digbench-scaling/egcd2-ll_unwindbound2.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr-var-start-time.6.2.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-rers2012/Problem07_label05.c ILP32 false ProductLines product-lines/email_spec11_product33.cil.c ILP32 false -ProductLines product-lines/email_spec1_product16.cil.c ILP32 false +Arrays array-fpi/s22iff.c ILP32 false +Loops nla-digbench-scaling/egcd2-ll_unwindbound5.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr.3_overflow.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-rers2012/Problem07_label19.c ILP32 false +ProductLines product-lines/email_spec1_product14.cil.c ILP32 false +Arrays array-fpi/s32iff.c ILP32 false +Loops nla-digbench-scaling/egcd3-ll_unwindbound1.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr.4_overflow.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-rers2012/Problem07_label37.c ILP32 false +ProductLines product-lines/email_spec1_product20.cil.c ILP32 false +Arrays array-fpi/s42iff.c ILP32 false +Loops nla-digbench-scaling/egcd3-ll_unwindbound100.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr.5.1.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-rers2012/Problem07_label48.c ILP32 false ProductLines product-lines/email_spec1_product26.cil.c ILP32 false -ProductLines product-lines/email_spec1_product32.cil.c ILP32 false +Arrays array-fpi/s4liff.c ILP32 false +Loops nla-digbench-scaling/egcd3-ll_unwindbound20.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr.6.1.ufo.BOUNDED-12.pals.c ILP32 false +ECA eca-rers2012/Problem08_label06.c ILP32 false +ProductLines product-lines/email_spec1_product30.cil.c ILP32 false +Arrays array-fpi/s5iff.c ILP32 false +Loops nla-digbench-scaling/egcd3-ll_unwindbound50.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr.7.1.ufo.BOUNDED-14.pals.c ILP32 false +ECA eca-rers2012/Problem08_label26.c ILP32 false +ProductLines product-lines/email_spec1_product33.cil.c ILP32 false +Arrays array-fpi/sina2f.c ILP32 false +Loops nla-digbench-scaling/fermat1-ll_unwindbound10.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr.7_overflow.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-rers2012/Problem08_label46.c ILP32 false ProductLines product-lines/email_spec1_productSimulator.cil.c ILP32 false +Arrays array-fpi/sina5f.c ILP32 false +Loops nla-digbench-scaling/fermat1-ll_unwindbound2.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr.8.1.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-rers2012/Problem09_label02.c ILP32 false +ProductLines product-lines/email_spec27_product19.cil.c ILP32 false +Arrays array-fpi/ss1f.c ILP32 false +Loops nla-digbench-scaling/fermat1-ll_unwindbound50.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.3.1.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-rers2012/Problem09_label15.c ILP32 false ProductLines product-lines/email_spec27_product24.cil.c ILP32 false -ProductLines product-lines/email_spec27_product30.cil.c ILP32 false -ProductLines product-lines/email_spec27_product34.cil.c ILP32 false -ProductLines product-lines/email_spec3_product17.cil.c ILP32 false +Arrays array-fpi/ss3f.c ILP32 false +Loops nla-digbench-scaling/fermat2-ll_unwindbound10.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.3.2.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-rers2012/Problem09_label38.c ILP32 false +ProductLines product-lines/email_spec27_product29.cil.c ILP32 false +Arrays array-fpi/ssinaf.c ILP32 false +Loops nla-digbench-scaling/fermat2-ll_unwindbound2.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.3.3.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-rers2012/Problem09_label54.c ILP32 false +ProductLines product-lines/email_spec27_product32.cil.c ILP32 false +Arrays array-industry-pattern/check_removal_from_set_after_insertion.c ILP32 false +Loops nla-digbench-scaling/fermat2-ll_unwindbound5.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.3_overflow.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-rers2012/Problem10_label26.c ILP32 false +ProductLines product-lines/email_spec27_product35.cil.c ILP32 false +Arrays array-tiling/mlceu.c ILP32 false +Loops nla-digbench-scaling/freire2_valuebound1.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.4.2.ufo.BOUNDED-8.pals.c ILP32 false +ECA eca-rers2012/Problem10_label47.c ILP32 false +ProductLines product-lines/email_spec3_product13.cil.c ILP32 false +Arrays reducercommutativity/rangesum05.i ILP32 false +Loops nla-digbench-scaling/freire2_valuebound100.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.4.3.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-rers2012/Problem11_label08.c ILP32 false +ProductLines product-lines/email_spec3_product19.cil.c ILP32 false +Arrays reducercommutativity/rangesum20.i ILP32 false +Loops nla-digbench-scaling/freire2_valuebound20.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.4.4.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-rers2012/Problem11_label34.c ILP32 false ProductLines product-lines/email_spec3_product24.cil.c ILP32 false +Arrays reducercommutativity/rangesum60.i ILP32 false +Loops nla-digbench-scaling/hard2_unwindbound1.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.5.1.ufo.BOUNDED-10.pals.c ILP32 false +ECA eca-rers2012/Problem11_label58.c ILP32 false +ProductLines product-lines/email_spec3_product28.cil.c ILP32 false +Loops nla-digbench-scaling/hard2_unwindbound2.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.5.2.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-rers2012/Problem12_label10.c ILP32 false ProductLines product-lines/email_spec3_product30.cil.c ILP32 false -ProductLines product-lines/email_spec3_product34.cil.c ILP32 false -ProductLines product-lines/email_spec4_product19.cil.c ILP32 false -ProductLines product-lines/email_spec4_product27.cil.c ILP32 false -ProductLines product-lines/email_spec4_product34.cil.c ILP32 false +Loops nla-digbench-scaling/mannadiv_unwindbound1.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.5.3.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-rers2012/Problem12_label30.c ILP32 false +ProductLines product-lines/email_spec3_product33.cil.c ILP32 false +Loops nla-digbench-scaling/mannadiv_unwindbound100.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.5.4.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-rers2012/Problem12_label40.c ILP32 false +ProductLines product-lines/email_spec3_productSimulator.cil.c ILP32 false +Loops nla-digbench-scaling/mannadiv_unwindbound20.c ILP32 false +Sequentialized seq-pthread/cs_fib_longer-2.i ILP32 false +ECA eca-rers2012/Problem13_label04.c ILP32 false +ProductLines product-lines/email_spec4_product23.cil.c ILP32 false +Loops nla-digbench-scaling/mannadiv_unwindbound50.c ILP32 false +Sequentialized seq-pthread/cs_read_write_lock-2.i ILP32 false +ECA eca-rers2012/Problem13_label19.c ILP32 false +ProductLines product-lines/email_spec4_product25.cil.c ILP32 false +Loops nla-digbench-scaling/prod4br-ll_unwindbound10.c ILP32 false +Sequentialized seq-pthread/cs_stateful-1.i ILP32 false +ECA eca-rers2012/Problem13_label30.c ILP32 false +ProductLines product-lines/email_spec4_product31.cil.c ILP32 false +Loops nla-digbench-scaling/prod4br-ll_unwindbound20.c ILP32 false +ECA eca-rers2012/Problem13_label44.c ILP32 false +ProductLines product-lines/email_spec4_product33.cil.c ILP32 false +Loops nla-digbench-scaling/prod4br-ll_unwindbound50.c ILP32 false +ECA eca-rers2012/Problem15_label02.c ILP32 false +ProductLines product-lines/email_spec4_productSimulator.cil.c ILP32 false +Loops nla-digbench-scaling/prodbin-ll_unwindbound10.c ILP32 false +ECA eca-rers2012/Problem15_label22.c ILP32 false ProductLines product-lines/email_spec6_product14.cil.c ILP32 false -ProductLines product-lines/email_spec6_product21.cil.c ILP32 false -ProductLines product-lines/email_spec6_product29.cil.c ILP32 false -ProductLines product-lines/email_spec6_product33.cil.c ILP32 false -ProductLines product-lines/email_spec7_product29.cil.c ILP32 false +Loops nla-digbench-scaling/prodbin-ll_unwindbound20.c ILP32 false +ECA eca-rers2012/Problem15_label37.c ILP32 false +ProductLines product-lines/email_spec6_product20.cil.c ILP32 false +Loops nla-digbench-scaling/ps4-ll_unwindbound1.c ILP32 false +ECA eca-rers2012/Problem15_label48.c ILP32 false +ProductLines product-lines/email_spec6_product26.cil.c ILP32 false +Loops nla-digbench-scaling/ps4-ll_unwindbound100.c ILP32 false +ECA eca-rers2012/Problem16_label05.c ILP32 false +ProductLines product-lines/email_spec6_product30.cil.c ILP32 false +Loops nla-digbench-scaling/ps4-ll_unwindbound20.c ILP32 false +ECA eca-rers2012/Problem16_label20.c ILP32 false +ProductLines product-lines/email_spec6_product32.cil.c ILP32 false +Loops nla-digbench-scaling/ps4-ll_unwindbound50.c ILP32 false +ECA eca-rers2012/Problem16_label38.c ILP32 false +ProductLines product-lines/email_spec6_product35.cil.c ILP32 false +Loops nla-digbench-scaling/ps5-ll_unwindbound10.c ILP32 false +ECA eca-rers2012/Problem16_label52.c ILP32 false +ProductLines product-lines/email_spec7_product28.cil.c ILP32 false +Loops nla-digbench-scaling/ps5-ll_unwindbound2.c ILP32 false +ECA eca-rers2012/Problem17_label20.c ILP32 false +ProductLines product-lines/email_spec7_product31.cil.c ILP32 false +Loops nla-digbench-scaling/ps5-ll_unwindbound5.c ILP32 false +ECA eca-rers2012/Problem17_label33.c ILP32 false ProductLines product-lines/email_spec7_product33.cil.c ILP32 false -ProductLines product-lines/email_spec8_product15.cil.c ILP32 false -ProductLines product-lines/email_spec8_product22.cil.c ILP32 false -ProductLines product-lines/email_spec8_product33.cil.c ILP32 false +Loops nla-digbench-scaling/ps6-ll_unwindbound1.c ILP32 false +ECA eca-rers2012/Problem17_label50.c ILP32 false +ProductLines product-lines/email_spec7_productSimulator.cil.c ILP32 false +Loops nla-digbench-scaling/ps6-ll_unwindbound100.c ILP32 false +ECA eca-rers2012/Problem18_label01.c ILP32 false +ProductLines product-lines/email_spec8_product20.cil.c ILP32 false +Loops nla-digbench-scaling/ps6-ll_unwindbound20.c ILP32 false +ECA eca-rers2012/Problem18_label19.c ILP32 false +ProductLines product-lines/email_spec8_product26.cil.c ILP32 false +Loops nla-digbench-scaling/ps6-ll_unwindbound50.c ILP32 false +ECA eca-rers2012/Problem18_label33.c ILP32 false +ProductLines product-lines/email_spec8_product31.cil.c ILP32 false +ECA eca-rers2012/Problem18_label49.c ILP32 false +ProductLines product-lines/email_spec8_product34.cil.c ILP32 false +ECA eca-rers2012/Problem19_label14.c ILP32 false ProductLines product-lines/email_spec9_product15.cil.c ILP32 false -ProductLines product-lines/email_spec9_product22.cil.c ILP32 false +ECA eca-rers2012/Problem19_label27.c ILP32 false +ProductLines product-lines/email_spec9_product21.cil.c ILP32 false +ECA eca-rers2012/Problem19_label42.c ILP32 false +ProductLines product-lines/email_spec9_product26.cil.c ILP32 false +ECA eca-rers2012/Problem19_label58.c ILP32 false ProductLines product-lines/email_spec9_product32.cil.c ILP32 false -XCSP xcsp/AllInterval-005.c ILP32 false -XCSP xcsp/AllInterval-006.c ILP32 false -XCSP xcsp/AllInterval-007.c ILP32 false -XCSP xcsp/AllInterval-009.c ILP32 false -XCSP xcsp/AllInterval-010.c ILP32 false -XCSP xcsp/AllInterval-012.c ILP32 false -XCSP xcsp/AllInterval-013.c ILP32 false -XCSP xcsp/AllInterval-015.c ILP32 false -XCSP xcsp/AllInterval-016.c ILP32 false -XCSP xcsp/AllInterval-018.c ILP32 false -XCSP xcsp/AllInterval-019.c ILP32 false -XCSP xcsp/AllInterval-025.c ILP32 false -XCSP xcsp/AllInterval-030.c ILP32 false -XCSP xcsp/CostasArray-10.c ILP32 false -XCSP xcsp/CostasArray-11.c ILP32 false -XCSP xcsp/CostasArray-13.c ILP32 false -XCSP xcsp/CostasArray-14.c ILP32 false -XCSP xcsp/CostasArray-16.c ILP32 false -XCSP xcsp/CostasArray-17.c ILP32 false -XCSP xcsp/aim-100-1-6-sat-2.c ILP32 false -XCSP xcsp/aim-100-1-6-sat-3.c ILP32 false -XCSP xcsp/aim-100-1-6-sat-4.c ILP32 false -XCSP xcsp/aim-100-2-0-sat-2.c ILP32 false -XCSP xcsp/aim-100-2-0-sat-3.c ILP32 false -XCSP xcsp/aim-100-3-4-sat-1.c ILP32 false -XCSP xcsp/aim-100-3-4-sat-2.c ILP32 false -XCSP xcsp/aim-100-3-4-sat-4.c ILP32 false -XCSP xcsp/aim-100-6-0-sat-1.c ILP32 false -XCSP xcsp/aim-100-6-0-sat-3.c ILP32 false -XCSP xcsp/aim-100-6-0-sat-4.c ILP32 false -XCSP xcsp/aim-200-1-6-sat-2.c ILP32 false -XCSP xcsp/aim-200-1-6-sat-3.c ILP32 false -XCSP xcsp/aim-200-2-0-sat-1.c ILP32 false -XCSP xcsp/aim-200-2-0-sat-2.c ILP32 false -XCSP xcsp/aim-200-2-0-sat-4.c ILP32 false -XCSP xcsp/aim-200-3-4-sat-1.c ILP32 false -XCSP xcsp/aim-200-3-4-sat-3.c ILP32 false -XCSP xcsp/aim-200-3-4-sat-4.c ILP32 false -XCSP xcsp/aim-200-6-0-sat-2.c ILP32 false -XCSP xcsp/aim-200-6-0-sat-3.c ILP32 false -LinkedLists forester-heap/dll-01-1.i ILP32 false -LinkedLists forester-heap/dll-circular-1.i ILP32 false -LinkedLists forester-heap/dll-optional-2.i ILP32 false -LinkedLists forester-heap/dll-queue-2.i ILP32 false -LinkedLists forester-heap/dll-rb-cnstr_1-2.i ILP32 false -LinkedLists forester-heap/dll-rb-sentinel-1.i ILP32 false -LinkedLists forester-heap/dll-simple-white-blue-1.i ILP32 false -LinkedLists forester-heap/dll-sorted-1.i ILP32 false -LinkedLists forester-heap/dll-token-2.i ILP32 false -LinkedLists forester-heap/sll-01-2.i ILP32 false -LinkedLists forester-heap/sll-buckets-2.i ILP32 false -LinkedLists forester-heap/sll-circular-2.i ILP32 false -LinkedLists forester-heap/sll-optional-2.i ILP32 false -LinkedLists forester-heap/sll-queue-2.i ILP32 false -LinkedLists forester-heap/sll-rb-cnstr_1-2.i ILP32 false -LinkedLists forester-heap/sll-rb-sentinel-2.i ILP32 false -LinkedLists forester-heap/sll-simple-white-blue-2.i ILP32 false -LinkedLists forester-heap/sll-sorted-1.i ILP32 false -LinkedLists forester-heap/sll-token-1.i ILP32 false -LinkedLists heap-manipulation/bubble_sort_linux-2.i ILP32 false -LinkedLists heap-manipulation/dll_of_dll-2.i ILP32 false -LinkedLists heap-manipulation/merge_sort-1.i ILP32 false -LinkedLists heap-manipulation/sll_to_dll_rev-1.i ILP32 false -LinkedLists heap-manipulation/tree-2.i ILP32 false -LinkedLists heap-manipulation/tree-4.i ILP32 false -LinkedLists list-ext3-properties/dll_nullified-1.i ILP32 false -LinkedLists list-ext3-properties/sll_length_check-1.i ILP32 false -LinkedLists list-ext3-properties/sll_nondet_insert-1.i ILP32 false -LinkedLists list-ext3-properties/sll_of_sll_nondet_append-2.i ILP32 false -LinkedLists list-properties/alternating_list-2.i ILP32 false -LinkedLists list-properties/list-2.i ILP32 false -LinkedLists list-properties/list_flag-1.i ILP32 false -LinkedLists list-properties/simple-1.i ILP32 false -LinkedLists list-properties/splice-1.i ILP32 false +ECA eca-rers2018/Problem15.c ILP32 false +ProductLines product-lines/email_spec9_product35.cil.c ILP32 false +ECA eca-programs/Problem101_label00.c ILP32 false +ProductLines product-lines/elevator_spec14_product32.cil.c ILP32 false +ECA eca-programs/Problem101_label03.c ILP32 false +ProductLines product-lines/elevator_spec1_product24.cil.c ILP32 false +ECA eca-programs/Problem101_label06.c ILP32 false +ProductLines product-lines/elevator_spec1_productSimulator.cil.c ILP32 false +ECA eca-programs/Problem101_label10.c ILP32 false +ProductLines product-lines/elevator_spec2_product22.cil.c ILP32 false +ECA eca-programs/Problem101_label13.c ILP32 false +ProductLines product-lines/elevator_spec2_product28.cil.c ILP32 false +ECA eca-programs/Problem101_label16.c ILP32 false +ProductLines product-lines/elevator_spec3_product11.cil.c ILP32 false +ECA eca-programs/Problem101_label19.c ILP32 false +ProductLines product-lines/elevator_spec3_product27.cil.c ILP32 false +ECA eca-programs/Problem101_label23.c ILP32 false +ProductLines product-lines/elevator_spec9_product26.cil.c ILP32 false +ECA eca-programs/Problem102_label04.c ILP32 false +ProductLines product-lines/elevator_spec9_product32.cil.c ILP32 false +ECA eca-programs/Problem102_label07.c ILP32 false +ProductLines product-lines/email_spec0_product21.cil.c ILP32 false +ECA eca-programs/Problem102_label15.c ILP32 false +ProductLines product-lines/email_spec0_product34.cil.c ILP32 false +ECA eca-programs/Problem102_label28.c ILP32 false +ProductLines product-lines/email_spec11_product22.cil.c ILP32 false +ECA eca-programs/Problem102_label37.c ILP32 false +ProductLines product-lines/email_spec11_product35.cil.c ILP32 false +ECA eca-programs/Problem102_label48.c ILP32 false +ProductLines product-lines/email_spec1_product15.cil.c ILP32 false +ECA eca-programs/Problem102_label51.c ILP32 false +ProductLines product-lines/email_spec1_product21.cil.c ILP32 false +ECA eca-programs/Problem103_label35.c ILP32 false +ProductLines product-lines/email_spec1_product31.cil.c ILP32 false +ECA eca-programs/Problem103_label46.c ILP32 false +ProductLines product-lines/email_spec1_product34.cil.c ILP32 false +ECA eca-programs/Problem103_label53.c ILP32 false +ProductLines product-lines/email_spec27_product17.cil.c ILP32 false +ECA eca-programs/Problem103_label57.c ILP32 false +ProductLines product-lines/email_spec27_product25.cil.c ILP32 false +ECA eca-rers2012/Problem03_label26.c ILP32 false +ProductLines product-lines/email_spec27_product30.cil.c ILP32 false +ECA eca-rers2012/Problem03_label31.c ILP32 false +ProductLines product-lines/email_spec27_product33.cil.c ILP32 false +ECA eca-rers2012/Problem03_label39.c ILP32 false +ProductLines product-lines/email_spec3_product17.cil.c ILP32 false +ECA eca-rers2012/Problem03_label50.c ILP32 false +ProductLines product-lines/email_spec3_product25.cil.c ILP32 false From 2ed743b9f8fbf23bdb0ed1b783fab01b8f9b3ca4 Mon Sep 17 00:00:00 2001 From: Guilherme Bernardo Date: Sat, 22 Aug 2026 23:05:36 -0400 Subject: [PATCH 06/17] fix(testcomp): six defects the corpus caught, and the tests that catch them free Running 2340 real Test-Comp tasks surfaced six defects that every existing test was blind to. Each produced well-formed output, a plausible verdict and a green run; what none produced was a suite a validator would accept. L. A Cover-Error suite came out with no at all. The state that reaches the target aborts, an aborted KLEE state runs no exit handler, and the flush that writes klee_log.csv never happens. 290 of the 376 runs that reported FAILED emitted an empty test case: the tool had found the bug and could not prove it. KLEE marks the failing path with a .err beside its .ktest, so the vector is recoverable without the runtime's cooperation. M. --property-file was never read when relative, which is how BenchExec and every harness here pass it: resolveSpecification runs after the pipeline has chdir'd into the scratch directory. Invisible because the fallback guess happens to match the real property text for both categories -- no output differed, so no test could see it. The regression test uses a specification the fallback could not invent. N. --cover-branches fell through to the MEMTRACK default and instrumented memory tracking for a task that checks no property. There is now a COVER_BRANCHES_MODE whose pipeline is nondet-pass and nothing else. O. MemoryTrackPass registered map2check_malloc with an i64 size and passed the program's own operand through uncoerced. A program allocating with i32 -- what the CIL-processed sources in ProductLines and ECA do -- produced a call whose type did not match its signature, and LLVM rejected the module: 110 of 110 ProductLines tasks died two seconds into a sixty-second budget. It survived because Juliet and CASTLE allocate with i64. P. The 500-test-case cap made suites unvalidatable. Of 116 tasks whose validation failed outright, 115 had at least 100 test cases and the median was exactly 500. Lowered to 50. Q. KLEE lost its entire exploration when the budget ran out, which in a competition run is the normal case rather than an edge. It was bounded only by an external `timeout`, and the states still live at the kill were never written: "unable to write output test case, losing it", 3982 times, for a total of zero test cases from 3982 explored paths. Two changes: KLEE now gets its own --max-time below the external one so it halts rather than being killed, and Cover-Branches searches depth-first so states terminate in sequence and each writes its test as it goes, instead of thousands sitting live until a halt that cannot write them. tests/integration/test_testcomp_regressions.sh covers all six, wired into CI. Writing it caught one more thing worth keeping: the cap assertion first read "at most 50" and passed against a run that produced ZERO test cases. It asserts equality now -- an upper bound is satisfied by emptiness, which is the failure it was meant to detect. Also committed: the measurement data behind the findings. cover-error is the complete 818-task campaign; cover-branches is 1344 of 1522, stopped to implement these fixes. Partial is still balanced -- the manifest interleaves subcategories, so a prefix samples all twelve. Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/ci.yml | 1 + docs/TESTCOMP-CHECKLIST.md | 10 +- modules/backend/pass/MemoryTrackPass.cpp | 38 ++- modules/frontend/caller.cpp | 68 ++++- modules/frontend/caller.hpp | 12 +- modules/frontend/map2check.cpp | 68 ++++- modules/frontend/test_suite/ktest_reader.cpp | 39 +++ modules/frontend/test_suite/ktest_reader.hpp | 14 + .../integration/test_testcomp_regressions.sh | 225 ++++++++++++++ tests/testcomp/RUN-2026-08-22-NOTES.md | 24 ++ .../results_cover-branches_s0/results.csv | 282 +++++++++++++++++ .../results.q40-stride.csv | 32 ++ .../results_cover-branches_s1/results.csv | 270 +++++++++++++++++ .../results.q40-stride.csv | 28 ++ .../results_cover-branches_s2/results.csv | 283 ++++++++++++++++++ .../results_cover-branches_s3/results.csv | 270 +++++++++++++++++ .../results_cover-branches_s4/results.csv | 244 +++++++++++++++ .../results_cover-error_s0/results.csv | 274 +++++++++++++++++ .../results.q40-stride.csv | 56 ++++ .../results_cover-error_s1/results.csv | 274 +++++++++++++++++ .../results.q40-stride.csv | 47 +++ .../results_cover-error_s2/results.csv | 273 +++++++++++++++++ 22 files changed, 2816 insertions(+), 16 deletions(-) mode change 100755 => 100644 modules/frontend/caller.hpp create mode 100755 tests/integration/test_testcomp_regressions.sh create mode 100644 tests/testcomp/results_cover-branches_s0/results.csv create mode 100644 tests/testcomp/results_cover-branches_s0/results.q40-stride.csv create mode 100644 tests/testcomp/results_cover-branches_s1/results.csv create mode 100644 tests/testcomp/results_cover-branches_s1/results.q40-stride.csv create mode 100644 tests/testcomp/results_cover-branches_s2/results.csv create mode 100644 tests/testcomp/results_cover-branches_s3/results.csv create mode 100644 tests/testcomp/results_cover-branches_s4/results.csv create mode 100644 tests/testcomp/results_cover-error_s0/results.csv create mode 100644 tests/testcomp/results_cover-error_s0/results.q40-stride.csv create mode 100644 tests/testcomp/results_cover-error_s1/results.csv create mode 100644 tests/testcomp/results_cover-error_s1/results.q40-stride.csv create mode 100644 tests/testcomp/results_cover-error_s2/results.csv diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 71629701a..08dfa4708 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -366,6 +366,7 @@ jobs: bash tests/integration/test_test_suite_emission.sh bash tests/integration/test_target_coverage.sh bash tests/integration/test_ktest_object_names.sh + bash tests/integration/test_testcomp_regressions.sh ' # =========================================================== diff --git a/docs/TESTCOMP-CHECKLIST.md b/docs/TESTCOMP-CHECKLIST.md index 9150bde1f..51cb3db2e 100644 --- a/docs/TESTCOMP-CHECKLIST.md +++ b/docs/TESTCOMP-CHECKLIST.md @@ -4,7 +4,7 @@ Estado das frentes do [plano de desenvolvimento](../.opencode/Plano%20de%20desen (local, fora do repositório). Este arquivo é o índice vivo: cada item aponta para a evidência que sustenta o estado declarado. -**Última atualização:** 2026-08-22 (noite) +**Última atualização:** 2026-08-23 (v8) ## Legenda @@ -93,6 +93,12 @@ Todos com evidência em [findings](reports/2026-08-12-castle-juliet-findings.md) | — | Imagem: Clam compilava com GCC 11 em vez de clang-16 (o `ENV CC` está 88 linhas abaixo do `RUN`), e GCC rejeita o que o clang só avisa. **A receita nunca compilou** | ✅ compiladores nomeados no `CLAM_CFG` (#58) | | — | `Dockerfile.dev` só era compilado **depois** do merge, então um Dockerfile inválido chegava à `develop` sem nada poder barrá-lo | ✅ o workflow de publish roda em PRs que tocam o arquivo, com `push: false` (#58) | | — | Clam e suas três dependências vinham de branches móveis; `crab@dev` mudou em 21/08 | ✅ pinados por SHA como `ARG` (#59) | +| **L** | **Suíte de Cover-Error saía sem nenhum ``.** O estado que alcança o alvo aborta, e estado abortado não roda handler de saída, então o `klee_log.csv` nunca era escrito. **290 de 376 execuções que reportaram FAILED** emitiram um test case vazio: a ferramenta achava o bug e não conseguia prová-lo | ✅ fallback para o `.ktest` do caminho que o KLEE marcou com `.err` | +| **M** | **`--property-file` relativo nunca era lido.** `resolveSpecification` roda depois do `chdir` para o scratch. Invisível porque o palpite do fallback coincide com a propriedade real das duas categorias — nenhuma saída diferia | ✅ resolvido contra `getOriginalPath()` | +| **N** | **`--cover-branches` caía no default `MEMTRACK_MODE`** e instrumentava rastreamento de memória para uma tarefa sem propriedade | ✅ `COVER_BRANCHES_MODE`, pipeline só com `nondet-pass` | +| **O** | **`MemoryTrackPass` emitia chamada com tipo incompatível.** Declara `map2check_malloc(ptr, i64)` e repassava o operando sem converter; programa que aloca com `i32` gerava módulo quebrado. **110 de 110 tarefas ProductLines** morriam em 2s de um orçamento de 60s. Sobreviveu porque Juliet e CASTLE alocam com `i64` | ✅ coerção nos 3 sítios | +| **P** | **Teto de 500 test cases inviabilizava a validação.** Dos 116 casos cuja validação falhou, 115 tinham ≥100 casos e a mediana era exatamente 500 | ✅ teto em 50 | +| **Q** | **O KLEE perdia toda a exploração ao esgotar o orçamento.** Só um `timeout` externo o limitava; um programa com 12 leituras nondet explorou 3982 caminhos e gerou **zero** `.ktest` — `unable to write output test case, losing it`, 3982 vezes. Atingir o orçamento é o caso NORMAL numa corrida de competição | ✅ `--max-time` próprio + busca `dfs` no cover-branches, para os estados terminarem em sequência e gravarem à medida que terminam | ## Defeitos abertos @@ -111,7 +117,7 @@ Todos com evidência em [findings](reports/2026-08-12-castle-juliet-findings.md) | O quê | Contagem | |---|---| | `ctest` (unitários) | 8 | -| Integração | ~110 asserções em 11 scripts | +| Integração | ~117 asserções em 12 scripts | | Conformidade Test-Comp | 6 programas | | Jobs de CI | 10, todos verdes na PR #59 (o 10º builda a imagem no próprio PR) | diff --git a/modules/backend/pass/MemoryTrackPass.cpp b/modules/backend/pass/MemoryTrackPass.cpp index cde1dfc1b..a35477a80 100644 --- a/modules/backend/pass/MemoryTrackPass.cpp +++ b/modules/backend/pass/MemoryTrackPass.cpp @@ -98,6 +98,35 @@ void MemoryTrackPass::instrumentPointer() { builder.CreateCall(this->map2check_pointer, args); } +/** Widens or narrows a size operand to the type the runtime call declares. + * + * The allocation size is taken straight from the program's own call, and its + * type is whatever that program declared. The runtime helpers are registered + * with an i64 size, so any program allocating with a narrower integer produced + * a call whose argument type did not match its signature -- and LLVM rejects + * the module outright: + * + * Call parameter type does not match function signature! + * LLVM ERROR: Broken module found, compilation aborted! + * + * Not a corner case. On the Test-Comp corpus this killed all 110 ProductLines + * tasks, two seconds into a sixty-second budget, plus a share of ECA -- the + * CIL-processed sources there allocate with i32. It went unseen because the + * Juliet and CASTLE programs allocate with i64, so every benchmark this tool + * is regularly run against happened to match. */ +static Value *coerceToParamType(IRBuilder<> &builder, Value *value, + FunctionCallee callee, unsigned argIndex) { + Type *wanted = callee.getFunctionType()->getParamType(argIndex); + if (value->getType() == wanted) return value; + if (value->getType()->isIntegerTy() && wanted->isIntegerTy()) { + // Signed: an allocation size is non-negative in practice, but a program is + // free to compute one in a signed type, and truncating a sign-extended + // value back is what preserves it either way. + return builder.CreateSExtOrTrunc(value, wanted); + } + return value; +} + void MemoryTrackPass::instrumentPosixMemAllign() { CallInst *callInst = dyn_cast(&*this->currentInstruction); @@ -116,7 +145,8 @@ void MemoryTrackPass::instrumentPosixMemAllign() { Value *varPointerCast = CastInst::CreatePointerCast( pointer, PointerType::get(*this->Ctx, 0), bitcast, BBIteratorToInst(j)); - Value *args[] = {varPointerCast, size}; + Value *args[] = {varPointerCast, + coerceToParamType(builder, size, map2check_posix, 1)}; builder.CreateCall(map2check_posix, args); } @@ -132,7 +162,8 @@ void MemoryTrackPass::instrumentMalloc() { Twine bitcast("bitcast"); // if (size == NULL) { // } - Value *args[] = {callInst, size}; + Value *args[] = {callInst, + coerceToParamType(builder, size, map2check_malloc, 1)}; builder.CreateCall(map2check_malloc, args); } @@ -155,7 +186,8 @@ void MemoryTrackPass::instrumentRealloc() { // Adds map2check_malloc with allocated address and size IRBuilder<> builder(BBIteratorToInst(j)); auto size = callInst->getArgOperand(1); - Value *args[] = {callInst, size}; + Value *args[] = {callInst, + coerceToParamType(builder, size, map2check_malloc, 1)}; builder.CreateCall(map2check_malloc, args); } diff --git a/modules/frontend/caller.cpp b/modules/frontend/caller.cpp index 66ac4b365..8c1f8679d 100644 --- a/modules/frontend/caller.cpp +++ b/modules/frontend/caller.cpp @@ -186,6 +186,15 @@ int Caller::callPass(std::string target_function, bool sv_comp) { passesArg << ",overflow-pass"; break; } + case Map2CheckMode::COVER_BRANCHES_MODE: { + // Nothing beyond nondet-pass. The inputs have to be recorded, so that + // pass stays; there is no property to instrument for, so nothing else + // does. This is also the leanest pipeline the tool has, which is the + // point: fewer instrumented calls means KLEE explores more paths in the + // same budget, and paths are what a branch suite is made of. + Map2Check::Log::Info("Running cover-branches mode (no property check)"); + break; + } case Map2CheckMode::REACHABILITY_MODE: { Map2Check::Log::Info("Running reachability mode"); Map2Check::Log::Debug("Target function: " + target_function); @@ -292,6 +301,11 @@ void Caller::linkLLVM() { linkCommand << " ${MAP2CHECK_PATH}/lib/AnalysisModeNone.bc"; break; } + case Map2CheckMode::COVER_BRANCHES_MODE: { + // No property, so no analysis -- the run exists to explore and record. + linkCommand << " ${MAP2CHECK_PATH}/lib/AnalysisModeNone.bc"; + break; + } } switch (nonDetGenerator) { @@ -403,6 +417,56 @@ void Caller::executeAnalysis(std::string solvername) { << (0.8 * this->timeout) << " "; kleeCommand << Map2Check::kleeBinary; + // KLEE's own deadline, set BELOW the external one so it is KLEE that + // stops, not the kill. + // + // Without it every path explored up to the budget is thrown away. + // Measured: a program with twelve nondeterministic reads explored 3982 + // paths and produced ZERO .ktest files, because `timeout` killed KLEE + // before it wrote any -- and those files are where a Cover-Branches + // suite comes from, and where a Cover-Error suite now recovers its + // input vector. Reaching the budget is the NORMAL case in a competition + // run, so this was not an edge: it was the common path discarding all + // of its work. + // + // The external timeout above stays as the backstop for the case its + // comment describes, a solver wedged so deep that KLEE's own deadline + // never gets a turn. + kleeCommand << " --max-time=" << (0.7 * this->timeout) << "s"; + + + // Halting on the first error is right when there is a property to + // decide -- the answer is known, and more exploration is waste. It is + // wrong for Cover-Branches, where there is no property and the paths ARE + // the product: stopping at the first error throws away every path not + // yet explored, and with it the test cases they would have produced. + // + // Measured before this: the same twelve-branch program produced 1020 + // .ktest files on one run and zero on the next, the difference being + // whether an error happened to be hit early. A suite that depends on + // that is not a suite. + // Depth-first for Cover-Branches, and the reason is about what survives + // the deadline rather than about search quality. + // + // KLEE's default search keeps thousands of states alive at once. Each + // writes its .ktest only when it terminates, and the ones still live + // when the budget expires are dumped at halt -- where writing them + // fails wholesale: "unable to write output test case, losing it", 3982 + // times in one measured run, for a total of zero test cases from 3982 + // explored paths. Depth-first finishes states one after another, so + // each one's test is on disk long before the deadline matters. + // + // For the property modes the default search stays: there the goal is to + // find one violating path quickly, not to harvest many. + std::string searchPolicy = + (map2checkMode == Map2CheckMode::COVER_BRANCHES_MODE) + ? " --search=dfs" + : ""; + + std::string stopPolicy = + (map2checkMode == Map2CheckMode::COVER_BRANCHES_MODE) + ? "" + : " --exit-on-error-type=Abort"; std::vector kleebackendsolver = {"z3", "stp"}; std::vector kleemetasolver = {"btor", "yices2"}; @@ -416,7 +480,7 @@ void Caller::executeAnalysis(std::string solvername) { // --allow-external-sym-calls // -use-cache kleeCommand << " --external-calls=all" - << " --exit-on-error-type=Abort" + << stopPolicy << searchPolicy << " --optimize" << " --use-cex-cache" << " --solver-backend=" + solvername + " " @@ -429,7 +493,7 @@ void Caller::executeAnalysis(std::string solvername) { Map2Check::Log::Info("Solver metaSMT caller: " + solvername); kleeCommand << " --external-calls=all" - << " --exit-on-error-type=Abort" + << stopPolicy << searchPolicy << " --optimize" << " --use-cex-cache" << " --solver-backend=metasmt " diff --git a/modules/frontend/caller.hpp b/modules/frontend/caller.hpp old mode 100755 new mode 100644 index d27dfd289..1c3432d02 --- a/modules/frontend/caller.hpp +++ b/modules/frontend/caller.hpp @@ -23,7 +23,17 @@ enum class Map2CheckMode { REACHABILITY_MODE, /**< Check if a target function can be executed */ OVERFLOW_MODE, /**< Check for signed integer overflows */ ASSERT_MODE, /**< Check for asserts (__VERIFIER_assert) */ - MEMCLEANUP_MODE /**< Check for memcleanup errors */ + MEMCLEANUP_MODE, /**< Check for memcleanup errors */ + /** Explore paths and record their inputs; check no property. + * + * Test-Comp Cover-Branches asks for a suite that exercises branches, not for + * a verdict, so every property check is overhead here -- and worse than + * overhead. With no mode flag this used to fall through to MEMTRACK_MODE and + * run the memory-tracking pass: on the Test-Comp corpus that meant 110 of + * 110 ProductLines tasks producing an empty suite in two seconds, because + * that pass emits a broken module on them. The same tasks work under + * Cover-Error, which never loads it. */ + COVER_BRANCHES_MODE }; /** NonDet generators */ diff --git a/modules/frontend/map2check.cpp b/modules/frontend/map2check.cpp index 3f3a96b33..ae736f365 100644 --- a/modules/frontend/map2check.cpp +++ b/modules/frontend/map2check.cpp @@ -79,11 +79,26 @@ bool invariantGeneratorAvailable() { * BenchExec always hands the tool a property file, and copying it verbatim is * what keeps the metadata correct when the competition revises a property * string -- guessing it here would silently drift. The fallbacks exist only so - * a manual run without --property-file still produces a valid suite. */ + * a manual run without --property-file still produces a valid suite. + * + * `invocationDir` is not optional decoration. This runs AFTER the pipeline has + * chdir'd into the scratch directory, so a relative --property-file -- which is + * what BenchExec and every harness here pass -- resolved against the wrong + * directory and silently fell through to the guess below. Measured on the + * Test-Comp corpus: "could not read property file: prop.prp" on every task. + * + * It went unnoticed because the guess HAPPENS to match the real property text + * for both categories today, so no output differed and no test could see it. + * The drift the comment above warns about was therefore already in place. */ std::string resolveSpecification(const std::string &propertyFile, + const std::string &invocationDir, Map2Check::Map2CheckMode mode) { if (!propertyFile.empty()) { - std::ifstream in(propertyFile); + std::string path = propertyFile; + if (!fs::path(path).is_absolute() && !invocationDir.empty()) { + path = invocationDir + "/" + path; + } + std::ifstream in(path); if (in.is_open()) { std::ostringstream buffer; buffer << in.rdbuf(); @@ -94,7 +109,7 @@ std::string resolveSpecification(const std::string &propertyFile, } if (!text.empty()) return text; } - Map2Check::Log::Warning("could not read property file: " + propertyFile); + Map2Check::Log::Warning("could not read property file: " + path); } if (mode == Map2Check::Map2CheckMode::REACHABILITY_MODE) { return "COVER( init(main()), FQL(COVER EDGES(@CALL(reach_error))) )"; @@ -110,10 +125,17 @@ std::string resolveSpecification(const std::string &propertyFile, * * KLEE can terminate tens of thousands of paths, and every test case costs the * validator a compile-and-run. The competition scores coverage, not volume, so - * past a few hundred vectors the marginal branch is rare and the validation - * cost is not. The bound is here rather than in the runtime because this is - * the side that knows what the suite is for. */ -constexpr size_t kMaxBranchTestCases = 500; + * past a few dozen vectors the marginal branch is rare and the validation cost + * is not. The bound is here rather than in the runtime because this is the + * side that knows what the suite is for. + * + * Lowered from 500 after measuring what 500 does: of 116 tasks whose + * validation failed outright on the Test-Comp corpus, 115 had at least 100 + * test cases and the median was exactly 500 -- suites so large that TestCov + * could not finish validating them, so the extra vectors scored nothing and + * cost everything. Generating them was work spent to make the result + * unmeasurable. */ +constexpr size_t kMaxBranchTestCases = 50; void emitTestSuite(const std::string &outputDir, const std::string &programFile, const std::string &entryFunction, @@ -173,6 +195,27 @@ void emitTestSuite(const std::string &outputDir, const std::string &programFile, std::vector inputs = Map2Check::readNonDetLog(Map2Check::kleeLogCSV); + + // Fall back to KLEE's record of the failing path when the runtime's log is + // empty, which is the common case rather than the exception. The state that + // reaches the target aborts; an aborted state runs no exit handler; the + // flush that writes klee_log.csv never happens. Measured on the Test-Comp + // corpus: 290 of 376 runs that reported FAILED emitted a test case with zero + // elements, and TestCov scored every one of them as covering + // nothing -- the tool had found the bug and could not prove it. + // + // The log stays the first choice where it exists: it records values in the + // order the program consumed them, straight from the instrumented run, while + // the .ktest is KLEE's view of the same path. + if (inputs.empty()) { + inputs = Map2Check::readViolatingKtest(Map2Check::kleeOutputDir); + if (!inputs.empty()) { + Map2Check::Log::Info( + "nondet log was empty; recovered the violating input vector from " + "KLEE's test output (" + std::to_string(inputs.size()) + " inputs)"); + } + } + if (!writer.writeTestCase(inputs, true)) { Map2Check::Log::Warning("could not write test case to " + outputDir); return; @@ -485,7 +528,8 @@ int map2check_execution(map2check_args args) { } emitTestSuite(outputDir, caller->c_program_fullpath, args.entryFunction, args.architecture, - resolveSpecification(args.propertyFile, args.mode), + resolveSpecification(args.propertyFile, + caller->getOriginalPath(), args.mode), foundViolation, args.coverBranches); } @@ -657,6 +701,14 @@ z3 (Z3 is default), btor (Boolector), and yices2 (Yices))") } if (vm.count("cover-branches")) { args.coverBranches = true; + // The mode has to change too, not just the emitter. Without this the run + // falls through to the MEMTRACK default and instruments memory tracking + // for a task that checks no property -- which on the Test-Comp corpus + // produced a broken module and an empty suite on all 110 ProductLines + // tasks. Set here rather than beside the other mode flags because + // --cover-branches is a goal, and the other flags are properties; this + // is the one goal that implies the absence of a property. + args.mode = Map2Check::Map2CheckMode::COVER_BRANCHES_MODE; } if (vm.count("property-file")) { args.propertyFile = vm["property-file"].as(); diff --git a/modules/frontend/test_suite/ktest_reader.cpp b/modules/frontend/test_suite/ktest_reader.cpp index 85b066b0d..4a91a563b 100644 --- a/modules/frontend/test_suite/ktest_reader.cpp +++ b/modules/frontend/test_suite/ktest_reader.cpp @@ -168,6 +168,45 @@ std::string decodeKtestObject(const KtestObject& object) { return std::to_string(signExtend(raw, object.bytes.size())); } +std::vector readViolatingKtest(const std::string& kleeOutDir) { + std::error_code error; + if (!std::filesystem::is_directory(kleeOutDir, error)) return {}; + + // KLEE names the report for a failing path testNNNNNN..err beside + // testNNNNNN.ktest -- .abort.err for an abort, .ptr.err for a bad + // dereference, and so on. The stem up to the first dot is the path number, + // which is the whole link between the two files. + std::vector candidates; + for (const auto& entry : + std::filesystem::directory_iterator(kleeOutDir, error)) { + const std::string name = entry.path().filename().string(); + if (name.size() < 5 || name.compare(name.size() - 4, 4, ".err") != 0) { + continue; + } + const size_t dot = name.find('.'); + if (dot == std::string::npos) continue; + candidates.push_back(name.substr(0, dot)); + } + // Sorted so that a run with several failing paths always yields the same + // one. Arbitrary, but arbitrary-and-stable beats arbitrary-and-not: a suite + // that changes between identical runs cannot be diffed. + std::sort(candidates.begin(), candidates.end()); + + for (const std::string& stem : candidates) { + std::vector objects = + readKtestFile((std::filesystem::path(kleeOutDir) / (stem + ".ktest")) + .string()); + if (objects.empty()) continue; + std::vector inputs; + inputs.reserve(objects.size()); + for (const KtestObject& object : objects) { + inputs.push_back(decodeKtestObject(object)); + } + return inputs; + } + return {}; +} + std::vector> readKtestVectors( const std::string& kleeOutDir, size_t limit) { std::vector> vectors; diff --git a/modules/frontend/test_suite/ktest_reader.hpp b/modules/frontend/test_suite/ktest_reader.hpp index 9bbe74be9..e943f26f1 100644 --- a/modules/frontend/test_suite/ktest_reader.hpp +++ b/modules/frontend/test_suite/ktest_reader.hpp @@ -59,6 +59,20 @@ std::vector readKtestFile(const std::string& path); * to drop a test case. */ std::string decodeKtestObject(const KtestObject& object); +/** The input vector of the path KLEE flagged as an error, if there is one. + * + * For Cover-Error, where the suite needs the ONE vector that reaches the bug. + * The runtime's own klee_log.csv is supposed to hold it, and usually does not: + * the state that reaches the target aborts, an aborted state runs no exit + * handler, and the flush never happens. Measured on the Test-Comp corpus -- + * 290 of 376 runs that reported FAILED emitted a test case with zero + * elements, and TestCov scored every one of them as covering nothing. + * + * KLEE marks the offending path by writing testNNNNNN..err beside its + * .ktest, so the vector is identifiable without any cooperation from the + * runtime. Returns an empty vector when no path errored. */ +std::vector readViolatingKtest(const std::string& kleeOutDir); + /** Every input vector KLEE recorded under `kleeOutDir`, one per .ktest. * * Ordered by file name, which is KLEE's own path numbering: stable across diff --git a/tests/integration/test_testcomp_regressions.sh b/tests/integration/test_testcomp_regressions.sh new file mode 100755 index 000000000..1d11d8a14 --- /dev/null +++ b/tests/integration/test_testcomp_regressions.sh @@ -0,0 +1,225 @@ +#!/bin/bash +# test_testcomp_regressions.sh -- the five defects the Test-Comp corpus caught. +# +# Every one of these was invisible to the tests that existed. They produced +# well-formed output, plausible verdicts and green runs; what they did not +# produce was a suite a validator would accept. They were found by running 2340 +# real tasks and reading the numbers, which is expensive, so each one gets a +# cheap test here to make sure it is found for free next time. +# +# 1 a Cover-Error suite with no at all (290 of 376 FAILED runs) +# 2 --property-file ignored when relative (every task in the corpus) +# 3 --cover-branches loading the memtrack pipeline (110 of 110 ProductLines) +# 4 MemoryTrackPass emitting a type-mismatched call (same 110, plus ECA) +# 5 a branch suite too large for TestCov to check (116 validation failures) +# 6 KLEE killed by the budget, discarding everything (3982 paths -> 0 tests) + +set -u + +MAP2CHECK_DIR="${MAP2CHECK_PATH:-/workspace/install_e2e}" +MAP2CHECK="$MAP2CHECK_DIR/map2check" + +PASSED=0 +FAILED=0 +ok() { echo " PASS $1"; PASSED=$((PASSED+1)); } +fail() { echo " FAIL $1: $2"; FAILED=$((FAILED+1)); } + +WORK=$(mktemp -d) +trap 'rm -rf "$WORK"' EXIT + +echo "=== Test-Comp regressions ===" + +# --- 1. a Cover-Error suite must carry the input vector ---------------------- +# The state that reaches the target aborts, and an aborted KLEE state runs no +# exit handler, so the runtime's klee_log.csv is never written. The suite used +# to come out with a testcase element and nothing inside it: the tool had found +# the bug and could not prove it. KLEE's own .ktest for the failing path holds +# the vector, and that is now the fallback. +mkdir -p "$WORK/one" +cat > "$WORK/one/reach.prp" <<'EOF' +COVER( init(main()), FQL(COVER EDGES(@CALL(reach_error))) ) +EOF +cat > "$WORK/one/reach.c" <<'EOF' +extern int __VERIFIER_nondet_int(void); +extern void reach_error(void); +int main(void) { + int a = __VERIFIER_nondet_int(); + if (a == 4242) { reach_error(); } + return 0; +} +EOF +( cd "$WORK/one" && MAP2CHECK_PATH="$MAP2CHECK_DIR" timeout -k 10 200 "$MAP2CHECK" \ + --target-function --target-function-name reach_error --nondet-generator symex \ + --generate-test-suite --property-file reach.prp --timeout 120 reach.c ) \ + > "$WORK/one/run.log" 2>&1 + +inputs=$(sed -n 's:.*\(.*\).*:\1:p' "$WORK/one/test-suite/testcase-1.xml" 2>/dev/null) +if [ -n "$inputs" ]; then + ok "a Cover-Error test case carries its input vector [$(echo $inputs | tr '\n' ' ')]" +else + fail "Cover-Error inputs" "test case has no -- the suite proves nothing" + tail -4 "$WORK/one/run.log" | sed 's/^/ /' +fi + +# --- 2. a relative --property-file must be read ------------------------------ +# resolveSpecification runs after the pipeline has chdir'd into the scratch +# directory, so a relative path -- what BenchExec and every harness here pass -- +# resolved against the wrong place and fell through to a guessed specification. +# +# The property text below is DELIBERATELY not one of the two the fallback +# guesses. That is the whole difficulty: with a real property file the guess +# happens to match, so the bug produced identical output and no test could see +# it. Only a specification the tool could not have invented proves it was read. +mkdir -p "$WORK/spec" +cat > "$WORK/spec/custom.prp" <<'EOF' +COVER( init(main()), FQL(COVER EDGES(@CALL(a_marker_no_fallback_would_guess))) ) +EOF +cp "$WORK/one/reach.c" "$WORK/spec/" +( cd "$WORK/spec" && MAP2CHECK_PATH="$MAP2CHECK_DIR" timeout -k 10 200 "$MAP2CHECK" \ + --target-function --target-function-name reach_error --nondet-generator symex \ + --generate-test-suite --property-file custom.prp --timeout 60 reach.c ) \ + > "$WORK/spec/run.log" 2>&1 + +if grep -q "a_marker_no_fallback_would_guess" "$WORK/spec/test-suite/metadata.xml" 2>/dev/null; then + ok "a relative --property-file reaches verbatim" +else + fail "property file" "metadata does not carry the supplied specification" + grep -o ".*" "$WORK/spec/test-suite/metadata.xml" 2>/dev/null | sed 's/^/ got: /' +fi +if grep -q "could not read property file" "$WORK/spec/run.log"; then + fail "property file resolution" "still reported unreadable" +else + ok "no 'could not read property file' warning" +fi + +# --- 3. --cover-branches must not run the memory-tracking pipeline ----------- +# With no mode flag the run fell through to the MEMTRACK default and +# instrumented memory tracking for a task that checks no property: pure +# overhead, and on some programs a broken module. +mkdir -p "$WORK/lean" +cat > "$WORK/lean/branches.prp" <<'EOF' +COVER( init(main()), FQL(COVER EDGES(@DECISIONEDGE)) ) +EOF +cat > "$WORK/lean/lean.c" <<'EOF' +extern int __VERIFIER_nondet_int(void); +int main(void) { + int x = __VERIFIER_nondet_int(); + if (x > 0) { return 1; } + return 2; +} +EOF +( cd "$WORK/lean" && MAP2CHECK_PATH="$MAP2CHECK_DIR" timeout -k 10 200 "$MAP2CHECK" \ + --nondet-generator symex --generate-test-suite --cover-branches \ + --property-file branches.prp --debug --timeout 60 lean.c ) \ + > "$WORK/lean/run.log" 2>&1 + +if grep -q "memory-track" "$WORK/lean/run.log"; then + fail "lean pipeline" "--cover-branches still loads the memory-track pass" +else + ok "--cover-branches runs without the memory-track pass" +fi + +# --- 4. MemoryTrackPass must coerce the size to the declared parameter type -- +# The helper is registered with an i64 size and the operand comes straight from +# the program's own call. A program allocating with a narrower integer -- which +# is what the CIL-processed sources in ProductLines and ECA do -- produced a +# call whose argument type did not match its signature, and LLVM rejected the +# module outright, two seconds into a sixty-second budget. +mkdir -p "$WORK/i32" +cat > "$WORK/i32/narrow.c" <<'EOF' +/* Declares malloc with a 32-bit size, as CIL-processed sources do. */ +extern void *malloc(unsigned int size); +extern void free(void *p); +extern int __VERIFIER_nondet_int(void); +int main(void) { + unsigned int n = 16; + char *p = (char *)malloc(n); + if (p) { + p[0] = (char)__VERIFIER_nondet_int(); + free(p); + } + return 0; +} +EOF +( cd "$WORK/i32" && MAP2CHECK_PATH="$MAP2CHECK_DIR" timeout -k 10 200 "$MAP2CHECK" \ + --memtrack --nondet-generator symex --timeout 60 narrow.c ) \ + > "$WORK/i32/run.log" 2>&1 + +if grep -qE "Broken module|does not match function signature" "$WORK/i32/run.log"; then + fail "size coercion" "memtrack still emits a type-mismatched call" + grep -m2 -E "Broken module|does not match" "$WORK/i32/run.log" | sed 's/^/ /' +else + ok "memtrack instruments a 32-bit allocation without breaking the module" +fi + +# --- 5. the branch suite must stay small enough to validate ------------------ +# Of 116 tasks whose validation failed outright, 115 had at least 100 test +# cases and the median was exactly 500 -- the old cap. Suites that large cannot +# be checked in the time available, so the extra vectors scored nothing and +# cost everything. +mkdir -p "$WORK/cap" +cp "$WORK/lean/branches.prp" "$WORK/cap/" +# Six independent branches: 64 terminating paths, comfortably more than the cap +# and few enough that KLEE finishes them well inside the budget. The assertion +# is EQUALITY, not "at most 50" -- a suite that came out empty would satisfy an +# upper bound while proving nothing, which is exactly how the first version of +# this test passed against a run that produced zero test cases. +cat > "$WORK/cap/many.c" <<'EOF' +extern int __VERIFIER_nondet_int(void); +int main(void) { + int total = 0; + for (int i = 0; i < 6; i++) { + int x = __VERIFIER_nondet_int(); + if (x > i) { total += 1; } else { total -= 1; } + } + return total; +} +EOF +( cd "$WORK/cap" && MAP2CHECK_PATH="$MAP2CHECK_DIR" timeout -k 10 250 "$MAP2CHECK" \ + --nondet-generator symex --generate-test-suite --cover-branches \ + --property-file branches.prp --timeout 120 many.c ) \ + > "$WORK/cap/run.log" 2>&1 + +n=$(ls "$WORK/cap/test-suite"/testcase-*.xml 2>/dev/null | wc -l) +if [ "$n" -eq 50 ]; then + ok "a 64-path program is capped to exactly 50 test cases" +else + fail "suite cap" "$n test cases, expected exactly 50 (64 paths, cap 50)" + tail -3 "$WORK/cap/run.log" | sed 's/^/ /' +fi + +# --- 6. KLEE must keep the paths it explored when its budget runs out -------- +# The tests are written by KLEE as states terminate, and an external `timeout` +# that kills it discards every one. Measured before the fix: a program with +# twelve nondeterministic reads explored 3982 paths and produced ZERO .ktest +# files. Reaching the budget is the normal case in a competition run, so this +# was the common path throwing away all of its work. +mkdir -p "$WORK/deep" +cp "$WORK/lean/branches.prp" "$WORK/deep/" +cat > "$WORK/deep/deep.c" <<'EOF' +extern int __VERIFIER_nondet_int(void); +int main(void) { + int total = 0; + for (int i = 0; i < 12; i++) { + int x = __VERIFIER_nondet_int(); + if (x > i) { total += 1; } else { total -= 1; } + } + return total; +} +EOF +( cd "$WORK/deep" && MAP2CHECK_PATH="$MAP2CHECK_DIR" timeout -k 10 200 "$MAP2CHECK" \ + --nondet-generator symex --generate-test-suite --cover-branches \ + --property-file branches.prp --timeout 90 deep.c ) \ + > "$WORK/deep/run.log" 2>&1 + +n_deep=$(ls "$WORK/deep/test-suite"/testcase-*.xml 2>/dev/null | wc -l) +if [ "$n_deep" -gt 0 ]; then + ok "a path-heavy program still yields a suite ($n_deep test cases)" +else + fail "budget exhaustion" "no test cases -- KLEE's exploration was discarded" + tail -3 "$WORK/deep/run.log" | sed 's/^/ /' +fi + +echo " ---" +echo " Results: $PASSED passed, $FAILED failed" +[ "$FAILED" -eq 0 ] || exit 1 diff --git a/tests/testcomp/RUN-2026-08-22-NOTES.md b/tests/testcomp/RUN-2026-08-22-NOTES.md index 8d8440678..3af265a75 100644 --- a/tests/testcomp/RUN-2026-08-22-NOTES.md +++ b/tests/testcomp/RUN-2026-08-22-NOTES.md @@ -43,3 +43,27 @@ amostra uniforme: Floats e ECA sozinhos somam 2,6 mil das 33 mil tarefas, e uma amostra uniforme reportaria o formato do benchmark em vez do comportamento da ferramenta. Sem RNG — reexecutar o script no mesmo checkout dá o mesmo manifesto. + + +## O que estes arquivos contêm + +| arquivo | tarefas | completude | +|---|---|---| +| `results_cover-error_s{0,1,2}/results.csv` | **818 de 818** | ✅ campanha completa | +| `results_cover-branches_s{0..4}/results.csv` | **1344 de 1522** | 🟡 parcial — as corridas foram encerradas para implementar as correções que estes dados motivaram | +| `results.q40-stride.csv` | 159 | amostragem ANTIGA (cota 40, passo uniforme), preservada para não se perder; **não misturar** com as acima, vieram de amostra diferente | + +Os parciais continuam válidos: o manifesto é intercalado por subcategoria, então +um prefixo é uma amostra balanceada das 12, não "as primeiras categorias +inteiras". + +## Os seis defeitos que estes dados revelaram + +Estão em [docs/TESTCOMP-CHECKLIST.md](../../docs/TESTCOMP-CHECKLIST.md) como +achados **L** a **Q**, todos corrigidos e cobertos por +`tests/integration/test_testcomp_regressions.sh`. Em resumo: suíte de +cover-error sem `` (290 casos), `--property-file` relativo nunca lido, +`--cover-branches` caindo no pipeline de memtrack, coerção de tipo no +`MemoryTrackPass` (110 tarefas mortas em 2s), teto de 500 casos inviabilizando +a validação (116 falhas), e o KLEE descartando 3982 caminhos explorados ao +esgotar o orçamento. diff --git a/tests/testcomp/results_cover-branches_s0/results.csv b/tests/testcomp/results_cover-branches_s0/results.csv new file mode 100644 index 000000000..2ce1931a7 --- /dev/null +++ b/tests/testcomp/results_cover-branches_s0/results.csv @@ -0,0 +1,282 @@ +category,program,data_model,expected_unreach,verdict,testcov,coverage,goals,n_tests,elapsed_s +Arrays,array-fpi/s52iff_overflow.c,ILP32,unknown,UNKNOWN,VALIDATED,,,0,1 +Recursive,recursified_nla-digbench/recursified_lcm2.c,ILP32,unknown,SUCCEEDED,VALIDATED,88.89,9,19,10 +XCSP,xcsp/Haystacks-08.c,ILP32,true,FAILED,VALIDATED,100.0,2,23,9 +Heap,ldv-memsafety/memleaks_test7-1.i,ILP32,unknown,FAILED,VALIDATED,20.0,40,36,48 +ECA,eca-rers2012/Problem05_label57.c,ILP32,false,UNKNOWN,VALIDATED,,,0,54 +BitVectors,bitvector/soft_float_1-2a.c.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,47 +Sequentialized,seq-mthreaded/pals_opt-floodmax.5.3.ufo.BOUNDED-10.pals.c,ILP32,false,UNKNOWN,TESTCOV_ERROR,,,153,48 +LinkedLists,memsafety-broom/dll-lst-shared.i,LP64,unknown,UNKNOWN,VALIDATED,87.5,8,13,48 +Loops,loop-simple/nested_1b.c,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +ProductLines,product-lines/elevator_spec2_product01.cil.c,ILP32,true,ERROR,NO_SUITE,,,0,89 +ControlFlow,openssl-simplified/s3_clnt_2.cil-2.c,ILP32,unknown,UNKNOWN,VALIDATED,45.14,175,142,47 +Floats,neural-networks/cartpole_7_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,3 +Arrays,array-memsafety/count_down-alloca-1.i,ILP32,unknown,UNKNOWN,VALIDATED,91.67,12,149,49 +Recursive,recursified_nla-digbench/recursified_sqrt1-ll.c,ILP32,unknown,UNKNOWN,VALIDATED,57.14,7,16,48 +XCSP,xcsp/aim-100-1-6-sat-4.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,100.0,2,23,15 +Heap,memsafety-ext2/length_test03-2.i,ILP32,unknown,FAILED,TESTCOV_ERROR,,,270,48 +ECA,eca-rers2012/Problem18_label00.c,ILP32,false,UNKNOWN,VALIDATED,,,0,50 +BitVectors,bitvector-regression/recHanoi03-1.c,ILP32,false,UNKNOWN,VALIDATED_ABORTS,100.0,6,54,50 +Sequentialized,seq-mthreaded/pals_STARTPALS_Triplicated.1.ufo.BOUNDED-10.pals.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,58.04,143,26,43 +LinkedLists,forester-heap/dll-optional-2.i,ILP32,false,UNKNOWN,TESTCOV_ERROR,,,246,49 +Loops,loop-zilu/benchmark39_conjunctive.i,ILP32,unknown,UNKNOWN,VALIDATED_ABORTS,100.0,7,49,48 +ProductLines,product-lines/elevator_spec2_product29.cil.c,ILP32,true,ERROR,NO_SUITE,,,0,90 +ControlFlow,ntdrivers/kbfiltr.i.cil-2.c,ILP32,false,UNKNOWN,VALIDATED,,,0,49 +Floats,neural-networks/cartpole_31_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-fpi/s12iff.c,ILP32,false,UNKNOWN,VALIDATED,,,0,5 +Recursive,recursified_nla-digbench/recursified_geo3-ll.c,ILP32,unknown,UNKNOWN,VALIDATED_ABORTS,57.14,7,14,49 +XCSP,xcsp/Dubois-075.c,ILP32,true,SUCCEEDED,VALIDATED,100.0,2,25,24 +Heap,memsafety-cve/gpac_15.i,ILP32,unknown,SUCCEEDED,VALIDATED,9.09,22,1,5 +ECA,eca-rers2012/Problem11_label28.c,ILP32,true,UNKNOWN,VALIDATED,,,0,49 +BitVectors,bitvector/s3_srvr_3a.BV.c.cil.c,ILP32,true,UNKNOWN,VALIDATED,55.43,184,193,49 +Sequentialized,seq-mthreaded/pals_opt-floodmax.4.3.ufo.UNBOUNDED.pals.c,ILP32,false,UNKNOWN,VALIDATED,,,0,48 +LinkedLists,list-properties/simple_built_from_end.i,ILP32,true,FAILED,VALIDATED,66.67,9,31,15 +Loops,nla-digbench-scaling/prodbin-ll_valuebound100.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,90.0,10,43,15 +ProductLines,product-lines/email_spec7_product17.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,2 +ControlFlow,openssl/s3_srvr.blast.16.i.cil-1.c,ILP32,unknown,FAILED,VALIDATED,,,0,48 +Floats,neural-networks/tanh_6_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-crafted/bor1.i,ILP32,true,SUCCEEDED,VALIDATED,77.78,9,1,3 +Recursive,fuzzle-programs/04_fuzzle_60x60.c,LP64,false,UNKNOWN,VALIDATED,,,0,61 +XCSP,xcsp/AllInterval-007.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,14.29,14,19,4 +Heap,ldv-memsafety/memleaks_test14_3.i,ILP32,unknown,FAILED,VALIDATED_ABORTS,8.33,36,21,28 +ECA,eca-programs/Problem103_label52.c,ILP32,true,ERROR,NO_SUITE,,,0,90 +BitVectors,bitvector/byte_add_2-2.i,ILP32,true,UNKNOWN,VALIDATED,,,0,48 +Sequentialized,seq-mthreaded/pals_floodmax.3_overflow.ufo.UNBOUNDED.pals.c,ILP32,false,FAILED,TESTCOV_ERROR,,,243,49 +LinkedLists,forester-heap/dll-sorted-1.i,ILP32,false,UNKNOWN,TESTCOV_ERROR,,,476,49 +Loops,loops-crafted-1/in-de52.c,ILP32,true,UNKNOWN,VALIDATED,91.67,12,146,48 +ProductLines,product-lines/elevator_spec3_product11.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,2 +ControlFlow,ntdrivers/floppy.i.cil-3.c,ILP32,true,UNKNOWN,VALIDATED,1.8,1164,2,50 +Floats,loop-floats-scientific-comp/loop3.c,ILP32,true,UNKNOWN,VALIDATED,,,0,2 +Arrays,array-fpi/ifeqn5f.c,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Recursive,recursified_nla-digbench/recursified_fermat1-ll.c,ILP32,unknown,UNKNOWN,VALIDATED,,,0,48 +XCSP,xcsp/Dubois-028.c,ILP32,true,SUCCEEDED,VALIDATED,100.0,2,23,9 +Heap,ldv-sets/test_mutex_double_unlock.i,ILP32,false,UNKNOWN,VALIDATED,,,0,4 +ECA,eca-rers2012/Problem08_label42.c,ILP32,true,ERROR,NO_SUITE,,,0,90 +BitVectors,bitvector/s3_clnt_2.BV.c.cil-2.c,ILP32,unknown,UNKNOWN,VALIDATED,46.29,175,201,48 +Sequentialized,seq-mthreaded/pals_lcr.5.1.ufo.BOUNDED-10.pals.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,44.63,121,27,28 +LinkedLists,heap-manipulation/sll_to_dll_rev-1.i,ILP32,false,UNKNOWN,VALIDATED_ABORTS,83.33,48,84,49 +Loops,nla-digbench-scaling/freire1_valuebound50.c,ILP32,true,SUCCEEDED,VALIDATED,62.5,8,1,2 +ProductLines,product-lines/email_spec1_product32.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,22 +ControlFlow,openssl/s3_clnt.blast.01.i.cil-1.c,ILP32,unknown,UNKNOWN,TESTCOV_ERROR,,,500,50 +Floats,neural-networks/lunarlander_16_unsafe.c-amalgamation.i,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-memsafety/cstrpbrk-alloca-2.i,ILP32,unknown,SUCCEEDED,VALIDATED,57.14,14,6,8 +Recursive,recursive-simple/id2_b3_o5.c,ILP32,true,UNKNOWN,VALIDATED,90.0,10,54,49 +XCSP,xcsp/aim-100-1-6-unsat-3.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,100.0,2,23,12 +Heap,memsafety-cve/openNDS_1.i,ILP32,unknown,UNKNOWN,TESTCOV_ERROR,,,500,50 +ECA,eca-rers2012/Problem15_label13.c,ILP32,true,UNKNOWN,VALIDATED,,,0,49 +BitVectors,bitvector/soft_float_1-3a.c.cil.c,ILP32,false,UNKNOWN,TESTCOV_ERROR,,,500,49 +Sequentialized,seq-mthreaded/pals_opt-floodmax.5_overflow.ufo.UNBOUNDED.pals.c,ILP32,false,UNKNOWN,TESTCOV_ERROR,,,178,49 +LinkedLists,memsafety-broom/dll.i,LP64,unknown,UNKNOWN,VALIDATED,100.0,6,4,48 +Loops,nla-digbench-scaling/ps3-ll_unwindbound5.c,ILP32,true,SUCCEEDED,VALIDATED,62.5,8,6,5 +ProductLines,product-lines/email_spec7_product29.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,30 +ControlFlow,openssl/s3_srvr.blast.14.i.cil-2.c,ILP32,unknown,FAILED,VALIDATED,8.6,221,135,49 +Floats,neural-networks/softsign_w64_r3_case_0_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,2 +Arrays,reducercommutativity/rangesum10.i,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,92.31,13,2,3 +Recursive,recursive/gcd01-1.c,ILP32,true,FAILED,VALIDATED,,,0,49 +XCSP,xcsp/aim-200-6-0-sat-1.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,100.0,2,25,22 +Heap,heap-data/min_max.i,ILP32,true,FAILED,VALIDATED,76.92,13,186,31 +ECA,eca-programs/Problem101_label15.c,ILP32,false,ERROR,NO_SUITE,,,0,90 +BitVectors,bitvector/byte_add-1.i,ILP32,false,UNKNOWN,VALIDATED,98.25,57,129,49 +Sequentialized,seq-mthreaded/pals_STARTPALS_ActiveStandby.5.ufo.BOUNDED-10.pals.c,ILP32,false,SUCCEEDED,VALIDATED,18.24,159,13,35 +LinkedLists,forester-heap/dll-circular-2.i,ILP32,true,UNKNOWN,VALIDATED,71.43,21,2,48 +Loops,loop-invgen/seq-3.i,ILP32,true,UNKNOWN,VALIDATED,73.33,15,14,48 +ProductLines,product-lines/elevator_spec1_product21.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +ControlFlow,memory-model/4SB.i,ILP32,false,SUCCEEDED,VALIDATED,15.73,178,9,6 +Floats,float-newlib/double_req_bl_0876.c,ILP32,true,SUCCEEDED,VALIDATED,8.74,183,2,30 +Arrays,array-examples/sorting_selectionsort_2_ground.i,ILP32,false,UNKNOWN,VALIDATED,,,0,48 +Recursive,recursified_loop-crafted/recursified_simple_array_index_value_4.i.v+nlh-reducer.c,ILP32,unknown,UNKNOWN,VALIDATED,,,0,48 +XCSP,xcsp/AllInterval-020.c,ILP32,false,SUCCEEDED,VALIDATED,5.0,40,22,8 +Heap,ldv-memsafety/memleaks_test22_1-1.i,ILP32,unknown,FAILED,VALIDATED,34.38,32,21,21 +ECA,eca-rers2012/Problem04_label18.c,ILP32,false,UNKNOWN,VALIDATED,,,0,51 +BitVectors,bitvector/jain_1-2.c,ILP32,unknown,UNKNOWN,VALIDATED,66.67,3,1,48 +Sequentialized,seq-mthreaded/pals_floodmax.4.3.ufo.UNBOUNDED.pals.c,ILP32,false,UNKNOWN,VALIDATED,,,0,49 +LinkedLists,forester-heap/sll-01-1.i,ILP32,true,UNKNOWN,VALIDATED,73.21,56,223,48 +Loops,loops-crafted-1/sumt2.c,ILP32,true,FAILED,VALIDATED,70.0,10,144,48 +ProductLines,product-lines/elevator_spec3_product20.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +ControlFlow,ntdrivers/floppy.i.cil-1.c,ILP32,false,UNKNOWN,VALIDATED,1.72,1160,2,49 +Floats,floats-nonlinear/coral-benchmark01.c,ILP32,false,SUCCEEDED,VALIDATED,50.0,4,1,2 +Arrays,array-fpi/condn.c,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Recursive,recursified_nla-digbench/recursified_egcd-ll.c,ILP32,unknown,SUCCEEDED,VALIDATED,88.89,9,17,11 +XCSP,xcsp/Dubois-022.c,ILP32,true,SUCCEEDED,VALIDATED,100.0,2,23,8 +Heap,ldv-regression/test19.c,ILP32,true,SUCCEEDED,VALIDATED,50.0,2,1,2 +ECA,eca-rers2012/Problem07_label19.c,ILP32,false,ERROR,NO_SUITE,,,0,90 +BitVectors,bitvector/s3_clnt_1.BV.c.cil-1a.c,ILP32,unknown,UNKNOWN,VALIDATED,46.29,175,178,49 +Sequentialized,seq-mthreaded/pals_lcr-var-start-time.5.1.ufo.BOUNDED-10.pals.c,ILP32,false,SUCCEEDED,VALIDATED,34.84,155,27,33 +LinkedLists,forester-heap/sll-sorted-1.i,ILP32,false,UNKNOWN,TESTCOV_ERROR,,,449,49 +Loops,nla-digbench-scaling/dijkstra-u_valuebound2.c,ILP32,true,SUCCEEDED,VALIDATED,85.71,14,3,3 +ProductLines,product-lines/email_spec0_product37.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,2 +ControlFlow,openssl-simplified/s3_clnt_4.cil-1.c,ILP32,unknown,FAILED,VALIDATED,45.71,175,177,49 +Floats,neural-networks/dubinsrejoin_17_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,2 +Arrays,array-fpi/s2if.c,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Recursive,recursified_nla-digbench/recursified_geo3.c,ILP32,unknown,UNKNOWN,VALIDATED,57.14,7,15,49 +XCSP,xcsp/Dubois-085.c,ILP32,true,SUCCEEDED,VALIDATED,100.0,2,25,23 +Heap,list-ext-properties/test-0513_1.i,ILP32,unknown,SUCCEEDED,VALIDATED,82.14,28,6,32 +ECA,eca-rers2012/Problem10_label05.c,ILP32,true,UNKNOWN,VALIDATED,34.33,201,6,49 +BitVectors,bitvector/s3_clnt_3.BV.c.cil-2a.c,ILP32,false,UNKNOWN,TESTCOV_ERROR,,,274,49 +Sequentialized,seq-mthreaded/pals_lcr.6_overflow.ufo.UNBOUNDED.pals.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,40.14,147,28,30 +LinkedLists,heap-manipulation/tree-4.i,ILP32,false,UNKNOWN,VALIDATED_ABORTS,84.21,38,5,48 +Loops,nla-digbench-scaling/geo1-ll2_unwindbound20.c,ILP32,unknown,UNKNOWN,VALIDATED,,,0,48 +ProductLines,product-lines/email_spec1_productSimulator.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,37 +ControlFlow,openssl-simplified/s3_srvr_7.cil.c,ILP32,unknown,FAILED,VALIDATED,,,0,48 +Floats,neural-networks/logistic_2_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-memsafety-realloc/array-realloc-3.i,ILP32,unknown,FAILED,VALIDATED,83.33,6,13,7 +Recursive,recursified_nla-digbench/recursified_ps6-ll.c,ILP32,unknown,SUCCEEDED,VALIDATED_ABORTS,85.71,7,9,7 +XCSP,xcsp/aim-100-1-6-sat-2.c,ILP32,false,SUCCEEDED,VALIDATED,100.0,2,23,10 +Heap,memsafety-cve/krb5Fixed.i,ILP32,unknown,UNKNOWN,TESTCOV_ERROR,,,500,50 +ECA,eca-rers2012/Problem12_label50.c,ILP32,false,UNKNOWN,VALIDATED,,,0,58 +BitVectors,bitvector/s3_srvr_3a_alt.BV.c.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,49 +Sequentialized,seq-mthreaded/pals_opt-floodmax.4.2.ufo.UNBOUNDED.pals.c,ILP32,false,UNKNOWN,TESTCOV_ERROR,,,491,49 +LinkedLists,list-properties/simple-1.i,ILP32,false,FAILED,VALIDATED_ABORTS,81.82,11,4,3 +Loops,nla-digbench-scaling/hard2_unwindbound1.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,50.0,14,2,2 +ProductLines,product-lines/email_spec4_product25.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,15 +ControlFlow,openssl/s3_srvr.blast.06.i.cil-1.c,ILP32,unknown,FAILED,VALIDATED,,,0,49 +Floats,neural-networks/poly_16_16_16_thresh_1_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,2 +Arrays,array-multidimensional/init-non-constant-2-n-u.c,ILP32,true,ERROR,VALIDATED_ABORTS,91.67,12,38,6 +Recursive,recursive-simple/sum_non.c,ILP32,unknown,FAILED,VALIDATED,75.0,4,57,48 +XCSP,xcsp/aim-100-3-4-sat-4.c,ILP32,false,SUCCEEDED,VALIDATED,100.0,2,23,11 +Heap,memsafety-cve/sniproxyFixed.i,ILP32,unknown,UNKNOWN,TESTCOV_ERROR,,,500,50 +ECA,eca-rers2012/Problem16_label37.c,ILP32,false,UNKNOWN,VALIDATED,17.9,391,7,49 +BitVectors,bitvector/soft_float_4-1.c.cil.c,ILP32,unknown,UNKNOWN,TESTCOV_ERROR,,,500,50 +Sequentialized,seq-pthread/cs_fib_longer-1.i,ILP32,true,UNKNOWN,TESTCOV_ERROR,,,500,49 +LinkedLists,memsafety-broom/linux-list-fst-data.i,LP64,unknown,UNKNOWN,VALIDATED,100.0,7,13,50 +Loops,nla-digbench-scaling/ps4-ll_unwindbound2.c,ILP32,false,SUCCEEDED,VALIDATED,75.0,8,4,3 +ProductLines,product-lines/email_spec7_product33.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,2 +ControlFlow,openssl/s3_srvr.blast.13.i.cil-2.c,ILP32,unknown,FAILED,VALIDATED,15.17,211,150,49 +Floats,neural-networks/softsign_w32_r4_case_1_unsafe.c-amalgamation.i,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-tiling/revcpyswp2.c,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Recursive,recursive/Fibonacci04.c,ILP32,false,UNKNOWN,VALIDATED_ABORTS,100.0,7,48,49 +XCSP,xcsp/aim-200-2-0-unsat-3.c,ILP32,true,SUCCEEDED,VALIDATED,100.0,2,25,17 +Heap,memsafety/test-0220.i,ILP32,unknown,FAILED,VALIDATED,63.64,11,130,48 +ECA,eca-rers2012/Problem19_label22.c,ILP32,false,UNKNOWN,VALIDATED,,,0,53 +ControlFlow,unsignedintegeroverflow-sas23/linear_interpolation.i,ILP32,true,SUCCEEDED,VALIDATED,,,0,59 +Floats,neural-networks/tanh_w8_r4_case_0_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,2 +Arrays,array-cav19/array_init_pair_symmetr2.c,ILP32,true,ERROR,VALIDATED_ABORTS,30.0,10,19,4 +Sequentialized,seq-mthreaded/pals_STARTPALS_ActiveStandby.4_1.ufo.BOUNDED-10.pals.c,ILP32,false,SUCCEEDED,VALIDATED,18.47,157,13,36 +LinkedLists,forester-heap/dll-01-2.i,ILP32,true,UNKNOWN,VALIDATED,73.21,56,174,49 +Recursive,fuzzle-programs/03_fuzzle_50x50.c,LP64,false,UNKNOWN,VALIDATED,,,0,54 +XCSP,xcsp/AllInterval-008.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,12.5,16,20,3 +Loops,loop-industry-pattern/nested-3.c,ILP32,unknown,UNKNOWN,TESTCOV_ERROR,,,145,50 +ProductLines,product-lines/elevator_spec14_product32.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,5 +Heap,ldv-memsafety/memleaks_test10-2.i,ILP32,unknown,SUCCEEDED,VALIDATED,5.26,38,1,24 +ECA,eca-programs/Problem102_label32.c,ILP32,true,ERROR,NO_SUITE,,,0,90 +ControlFlow,ntdrivers-simplified/floppy_simpl3.cil-1.c,ILP32,unknown,UNKNOWN,VALIDATED,51.93,181,58,48 +Floats,float-benchs/sqrt_biNewton_pseudoconstant.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,20.0,10,1,2 +Arrays,array-crafted/zero_sum_const2.c,ILP32,true,UNKNOWN,VALIDATED,,,0,14 +Sequentialized,seq-mthreaded/pals_floodmax.3.ufo.BOUNDED-6.pals.c,ILP32,true,FAILED,VALIDATED,,,0,49 +LinkedLists,forester-heap/dll-simple-white-blue-2.i,ILP32,true,FAILED,VALIDATED,85.0,20,43,11 +Recursive,fuzzle-programs/13_fuzzle_50x50_CVE-2016-4493.c,LP64,false,UNKNOWN,VALIDATED,,,0,54 +XCSP,xcsp/AllInterval-018.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,5.56,36,21,4 +Loops,loop-new/gauss_sum.i.p+lhb-reducer.c,ILP32,true,UNKNOWN,VALIDATED,80.0,20,152,49 +ProductLines,product-lines/elevator_spec1_product31.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,56 +Heap,ldv-memsafety/memleaks_test18.i,ILP32,unknown,UNKNOWN,VALIDATED,25.0,40,9,49 +ECA,eca-rers2012/Problem03_label33.c,ILP32,true,UNKNOWN,VALIDATED,3.04,296,7,49 +ControlFlow,ntdrivers/kbfiltr.i.cil-1.c,ILP32,unknown,UNKNOWN,VALIDATED,,,0,39 +Floats,float-newlib/float_req_bl_0684b.c,ILP32,true,UNKNOWN,VALIDATED,,,0,2 +Arrays,array-examples/standard_copy3_ground-2.i,ILP32,false,UNKNOWN,VALIDATED,7.69,13,1,48 +Sequentialized,seq-mthreaded/pals_floodmax.5.2.ufo.BOUNDED-10.pals.c,ILP32,false,UNKNOWN,TESTCOV_ERROR,,,216,48 +LinkedLists,forester-heap/sll-queue-1.i,ILP32,true,UNKNOWN,VALIDATED,72.88,59,87,47 +Recursive,recursified_loop-simple/recursified_nested_1b.c,ILP32,unknown,UNKNOWN,VALIDATED,,,0,1 +XCSP,xcsp/CostasArray-15.c,ILP32,false,FAILED,VALIDATED_ABORTS,100.0,2,24,10 +Loops,loop-zilu/benchmark44_disjunctive.i,ILP32,unknown,FAILED,VALIDATED,100.0,11,183,48 +ProductLines,product-lines/elevator_spec2_product31.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Heap,ldv-memsafety/memleaks_test23_4.i,ILP32,unknown,FAILED,VALIDATED,38.46,52,6,48 +ECA,eca-rers2012/Problem05_label14.c,ILP32,true,UNKNOWN,VALIDATED,,,0,53 +ControlFlow,openssl-simplified/s3_srvr_10.cil.c,ILP32,unknown,FAILED,VALIDATED,10.24,205,1,48 +Floats,floats-cdfpl/newton_3_4.i,ILP32,true,UNKNOWN,VALIDATED,,,0,2 +Arrays,array-examples/standard_two_index_04.i,ILP32,true,UNKNOWN,VALIDATED,11.11,9,1,48 +Sequentialized,seq-mthreaded/pals_lcr-var-start-time.5.2.ufo.BOUNDED-10.pals.c,ILP32,false,SUCCEEDED,VALIDATED,35.53,152,27,28 +LinkedLists,heap-manipulation/bubble_sort_linux-1.i,ILP32,true,UNKNOWN,VALIDATED,73.44,64,55,49 +Recursive,recursified_nla-digbench/recursified_cohendiv.c,ILP32,unknown,SUCCEEDED,VALIDATED,90.0,10,15,14 +XCSP,xcsp/Dubois-023.c,ILP32,true,SUCCEEDED,VALIDATED,100.0,2,23,8 +Loops,loops/matrix-2-2.c,ILP32,false,FAILED,VALIDATED_ABORTS,71.43,14,56,48 +ProductLines,product-lines/elevator_spec3_product31.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Heap,ldv-regression/fo_test.i,ILP32,false,UNKNOWN,VALIDATED,100.0,4,4,3 +ECA,eca-rers2012/Problem06_label38.c,ILP32,false,UNKNOWN,VALIDATED,,,0,53 +ControlFlow,openssl-simplified/s3_srvr_8.cil.c,ILP32,unknown,UNKNOWN,VALIDATED,49.28,209,176,48 +Floats,neural-networks/cartpole_25_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-fpi/eqn5f.c,ILP32,false,UNKNOWN,VALIDATED,,,0,36 +Sequentialized,seq-mthreaded/pals_lcr.5.ufo.BOUNDED-10.pals.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,44.63,121,27,22 +LinkedLists,list-ext3-properties/dll_nondet_free_order-1.i,ILP32,unknown,SUCCEEDED,VALIDATED,86.67,15,6,26 +Recursive,recursified_nla-digbench/recursified_egcd3.c,ILP32,unknown,SUCCEEDED,VALIDATED_ABORTS,92.31,13,21,22 +XCSP,xcsp/Dubois-050.c,ILP32,true,SUCCEEDED,VALIDATED,100.0,2,24,12 +Loops,nla-digbench-scaling/cohencu-ll_valuebound10.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,87.5,8,6,2 +ProductLines,product-lines/email_spec0_product21.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,48 +Heap,ldv-regression/test25-2.c,ILP32,true,SUCCEEDED,VALIDATED,90.0,10,6,10 +ECA,eca-rers2012/Problem08_label00.c,ILP32,true,ERROR,NO_SUITE,,,0,89 +ControlFlow,openssl/s3_srvr.blast.04.i.cil.c,ILP32,unknown,FAILED,VALIDATED,15.92,201,162,49 +Floats,neural-networks/cartpole_85_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-fpi/ms5.c,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Sequentialized,seq-mthreaded/pals_opt-floodmax.3.2.ufo.UNBOUNDED.pals.c,ILP32,false,UNKNOWN,TESTCOV_ERROR,,,298,48 +LinkedLists,list-properties/list_flag-1.i,ILP32,false,FAILED,VALIDATED_ABORTS,76.47,17,26,9 +Recursive,recursified_nla-digbench/recursified_geo2-ll.c,ILP32,unknown,UNKNOWN,VALIDATED_ABORTS,57.14,7,21,48 +XCSP,xcsp/Dubois-100.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,100.0,2,25,16 +Loops,nla-digbench-scaling/divbin_valuebound5.i,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,86.67,15,9,2 +ProductLines,product-lines/email_spec11_product20.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,4 +Heap,list-ext-properties/list-ext_flag_1.i,ILP32,unknown,UNKNOWN,VALIDATED,71.43,21,255,47 +ECA,eca-rers2012/Problem09_label23.c,ILP32,true,ERROR,NO_SUITE,,,0,89 +ControlFlow,openssl/s3_srvr.blast.12.i.cil-2.c,ILP32,unknown,FAILED,VALIDATED,14.75,217,189,47 +Floats,neural-networks/dubinsrejoin_48_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,2 +Arrays,array-fpi/s42iff.c,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Sequentialized,seq-mthreaded/pals_opt-floodmax.4.ufo.BOUNDED-8.pals.c,ILP32,true,UNKNOWN,TESTCOV_ERROR,,,500,47 +LinkedLists,memsafety-broom/dll-middle-shared.i,LP64,unknown,UNKNOWN,VALIDATED,87.5,8,5,47 +Recursive,recursified_nla-digbench/recursified_knuth.i,ILP32,unknown,SUCCEEDED,VALIDATED,35.71,14,3,12 +XCSP,xcsp/Haystacks-16.c,ILP32,true,SUCCEEDED,VALIDATED,100.0,2,25,21 +Loops,nla-digbench-scaling/fermat2-ll_unwindbound100.c,ILP32,false,SUCCEEDED,VALIDATED,,,0,57 +ProductLines,product-lines/email_spec1_product22.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Heap,memsafety-cve/bzip3Fixed.i,ILP32,unknown,UNKNOWN,TESTCOV_ERROR,,,500,49 +ECA,eca-rers2012/Problem10_label46.c,ILP32,false,UNKNOWN,VALIDATED,39.3,201,16,48 +ControlFlow,unsignedintegeroverflow-sas23/linear_interpolation_2.i,ILP32,true,UNKNOWN,VALIDATED_ABORTS,77.78,9,4,48 +Floats,neural-networks/exp_1_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,2 +Arrays,array-fpi/ss2f.c,ILP32,false,UNKNOWN,VALIDATED,,,0,24 +Floats,neural-networks/gcas_6_unsafe.c-amalgamation.i,ILP32,false,UNKNOWN,VALIDATED,,,0,3 +Arrays,array-industry-pattern/array_monotonic.i,ILP32,true,UNKNOWN,TESTCOV_ERROR,,,500,52 +Floats,neural-networks/log_3_unsafe.c-amalgamation.i,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-industry-pattern/array_ptr_single_elem_init-2.i,ILP32,false,UNKNOWN,TESTCOV_ERROR,,,500,49 +Floats,neural-networks/lunarlander_0_unsafe.c-amalgamation.i,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-lopstr16/single_elem_safe.i,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Floats,neural-networks/lunarlander_21_unsafe.c-amalgamation.i,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-memsafety/array02-alloca-2.i,ILP32,unknown,ERROR,VALIDATED,100.0,10,200,48 +Floats,neural-networks/lunarlander_33_unsafe.c-amalgamation.i,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-memsafety/cstrchr-alloca-2.i,ILP32,unknown,SUCCEEDED,VALIDATED,87.5,8,5,4 +Floats,neural-networks/lunarlander_45_unsafe.c-amalgamation.i,ILP32,false,UNKNOWN,VALIDATED,,,0,2 +Arrays,array-memsafety/cstrncat_unsafe.c,ILP32,unknown,FAILED,VALIDATED,66.67,6,21,12 +Floats,neural-networks/lunarlander_57_unsafe.c-amalgamation.i,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Heap,memsafety-cve/libredwg_1.i,ILP32,unknown,FAILED,TESTCOV_ERROR,,,500,49 +ECA,eca-rers2012/Problem13_label15.c,ILP32,true,UNKNOWN,VALIDATED,,,0,50 +Loops,nla-digbench-scaling/hard-ll_valuebound2.c,ILP32,true,SUCCEEDED,VALIDATED,92.86,14,6,2 +ProductLines,product-lines/email_spec4_product13.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Recursive,recursive-simple/id_b3_o5-1.c,ILP32,unknown,FAILED,VALIDATED,83.33,6,75,49 +XCSP,xcsp/aim-200-1-6-sat-1.c,ILP32,false,SUCCEEDED,VALIDATED,100.0,2,25,14 +Floats,neural-networks/poly_1024_thresh_5_unsafe.c-amalgamation.i,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Heap,memsafety-cve/openSC.i,ILP32,unknown,UNKNOWN,TESTCOV_ERROR,,,500,48 +ProductLines,product-lines/email_spec4_product32.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,26 +Recursive,recursive-simple/id_o1000.c,ILP32,false,FAILED,VALIDATED,75.0,4,77,48 +Arrays,array-multidimensional/diff-3-n-u.c,ILP32,true,UNKNOWN,VALIDATED,,,0,26 +ECA,eca-rers2012/Problem15_label56.c,ILP32,true,UNKNOWN,VALIDATED,16.71,419,7,48 +Loops,nla-digbench-scaling/mannadiv_unwindbound1.c,ILP32,false,SUCCEEDED,VALIDATED,100.0,10,6,3 +XCSP,xcsp/aim-200-2-0-sat-4.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,100.0,2,25,22 +Floats,neural-networks/poly_512_thresh_5_unsafe.c-amalgamation.i,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Heap,memsafety-cve/sofia-sipFixed.i,ILP32,unknown,UNKNOWN,TESTCOV_ERROR,,,500,48 +ProductLines,product-lines/email_spec6_productSimulator.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,40 +Recursive,recursive/Ackermann04.c,ILP32,true,UNKNOWN,VALIDATED,90.0,10,15,47 +Arrays,array-patterns/array3_pattern.c,ILP32,true,SUCCEEDED,VALIDATED,91.67,12,14,4 +ECA,eca-rers2012/Problem17_label18.c,ILP32,true,UNKNOWN,VALIDATED,,,0,48 +Loops,nla-digbench-scaling/ps3-ll_valuebound20.c,ILP32,true,SUCCEEDED,VALIDATED,87.5,8,11,2 +XCSP,xcsp/aim-200-6-0-sat-2.c,ILP32,false,SUCCEEDED,VALIDATED,100.0,2,25,15 +Floats,neural-networks/softsign_1_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Heap,memsafety-ext2/optional_data_creation_test04-2.i,ILP32,unknown,UNKNOWN,VALIDATED,92.86,14,204,47 +ProductLines,product-lines/email_spec8_product16.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,10 +Floats,neural-networks/softsign_w4_r3_case_1_unsafe.c-amalgamation.i,ILP32,false,UNKNOWN,VALIDATED,,,0,2 +Loops,nla-digbench-scaling/sqrt1-ll_unwindbound2.c,ILP32,true,SUCCEEDED,VALIDATED,62.5,8,3,2 +Arrays,array-tiling/rewnifrev2.c,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +ECA,eca-rers2012/Problem18_label57.c,ILP32,false,UNKNOWN,VALIDATED,,,0,49 +Recursive,recursive/McCarthy91-1.c,ILP32,false,UNKNOWN,VALIDATED_ABORTS,100.0,4,132,49 +Heap,memsafety/test-0232-2.i,ILP32,unknown,UNKNOWN,VALIDATED,75.0,8,24,48 +ProductLines,product-lines/email_spec9_product22.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,45 +Floats,neural-networks/tanh_w64_r4_case_1_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,2 +Loops,nla-digbench/ps5-ll.c,ILP32,true,SUCCEEDED,VALIDATED,87.5,8,9,2 +Arrays,array-cav19/array_init_nondet_vars.c,ILP32,true,SUCCEEDED,VALIDATED,50.0,8,8,3 +ECA,eca-programs/Problem101_label03.c,ILP32,false,UNKNOWN,VALIDATED,,,0,84 +Recursive,recursified_nla-digbench/recursified_lcm1.c,ILP32,unknown,SUCCEEDED,VALIDATED,92.31,13,15,44 +Heap,heap-data/process_queue.i,ILP32,true,FAILED,VALIDATED,87.5,16,265,47 +ProductLines,product-lines/elevator_spec14_product23.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,0 diff --git a/tests/testcomp/results_cover-branches_s0/results.q40-stride.csv b/tests/testcomp/results_cover-branches_s0/results.q40-stride.csv new file mode 100644 index 000000000..bb7231b30 --- /dev/null +++ b/tests/testcomp/results_cover-branches_s0/results.q40-stride.csv @@ -0,0 +1,32 @@ +category,program,data_model,expected_unreach,verdict,testcov,coverage,goals,n_tests,elapsed_s +Arrays,array-cav19/array_init_nondet_vars.c,ILP32,true,SUCCEEDED,FULL,50.0,8,8,5 +Arrays,array-crafted/mapsum1.i,ILP32,true,SUCCEEDED,FULL,77.78,9,1,4 +Arrays,array-crafted/zero_sum_m2.c,ILP32,true,SUCCEEDED,FULL,81.25,16,3,7 +Arrays,array-examples/standard_copy1_ground-1.i,ILP32,true,UNKNOWN,FULL,,,0,44 +Arrays,array-examples/standard_palindrome_ground.i,ILP32,true,UNKNOWN,FULL,11.11,9,1,45 +Arrays,array-examples/standard_two_index_09.i,ILP32,true,UNKNOWN,FULL,11.11,9,1,47 +Arrays,array-fpi/eqn1f.c,ILP32,false,UNKNOWN,FULL,,,0,73 +Arrays,array-fpi/indp1f.c,ILP32,false,UNKNOWN,FULL,,,0,33 +Arrays,array-fpi/ncomp.c,ILP32,true,UNKNOWN,FULL,,,0,29 +Arrays,array-fpi/s2if.c,ILP32,true,UNKNOWN,FULL,,,0,43 +Arrays,array-fpi/s5if.c,ILP32,true,UNKNOWN,FULL,,,0,24 +Arrays,array-fpi/ss2.c,ILP32,true,UNKNOWN,FULL,,,0,44 +Arrays,array-lopstr16/single_elem_safe.i,ILP32,true,UNKNOWN,FULL,,,0,25 +Arrays,array-memsafety/cstrncpy-alloca-2.i,ILP32,unknown,SUCCEEDED,PARTIAL,83.33,12,6,7 +Arrays,array-memsafety/openbsd_cstrlcpy-alloca-1.i,ILP32,unknown,SUCCEEDED,PARTIAL,60.0,20,6,10 +Arrays,array-multidimensional/copy-partial-3-u.c,ILP32,true,UNKNOWN,FULL,,,0,25 +Arrays,array-patterns/array19_pattern.c,ILP32,true,FAILED,PARTIAL,35.0,20,4,12 +Arrays,array-patterns/array7_pattern.c,ILP32,true,SUCCEEDED,PARTIAL,91.67,12,11,8 +Arrays,array-tiling/pnr2.c,ILP32,true,UNKNOWN,FULL,,,0,45 +Arrays,reducercommutativity/avg60-1.i,ILP32,true,SUCCEEDED,FULL,77.78,9,1,2 +BitVectors,bitvector-loops/diamond_2-1.c,ILP32,false,SUCCEEDED,PARTIAL,100.0,24,2,1 +BitVectors,bitvector/byte_add-1.i,ILP32,false,UNKNOWN,PARTIAL,61.4,57,1,46 +BitVectors,bitvector/byte_add_1-2.i,ILP32,unknown,UNKNOWN,FULL,,,0,45 +BitVectors,bitvector/gcd_1.c,ILP32,true,SUCCEEDED,FULL,80.0,10,4,5 +BitVectors,bitvector/interleave_bits.i,ILP32,true,SUCCEEDED,FULL,60.0,5,1,2 +BitVectors,bitvector/jain_4-1.c,ILP32,unknown,UNKNOWN,FULL,66.67,3,1,44 +BitVectors,bitvector/modulus-1.i,ILP32,unknown,FAILED,FULL,,,0,47 +BitVectors,bitvector/parity.i,ILP32,true,SUCCEEDED,FULL,77.78,9,33,34 +BitVectors,bitvector/s3_clnt_1.BV.c.cil-2.c,ILP32,unknown,UNKNOWN,FULL,46.29,175,185,46 +BitVectors,bitvector/s3_clnt_2.BV.c.cil-1a.c,ILP32,true,UNKNOWN,FULL,46.29,175,203,44 +BitVectors,bitvector/s3_clnt_3.BV.c.cil-1a.c,ILP32,true,FAILED,TESTCOV_ERROR,,,274,47 diff --git a/tests/testcomp/results_cover-branches_s1/results.csv b/tests/testcomp/results_cover-branches_s1/results.csv new file mode 100644 index 000000000..b5ee15478 --- /dev/null +++ b/tests/testcomp/results_cover-branches_s1/results.csv @@ -0,0 +1,270 @@ +category,program,data_model,expected_unreach,verdict,testcov,coverage,goals,n_tests,elapsed_s +BitVectors,bitvector/s3_clnt_3.BV.c.cil-1.c,ILP32,unknown,FAILED,TESTCOV_ERROR,,,274,48 +Sequentialized,seq-mthreaded/pals_lcr.6.ufo.BOUNDED-12.pals.c,ILP32,true,SUCCEEDED,VALIDATED,40.54,148,29,36 +LinkedLists,heap-manipulation/tree-3.i,ILP32,true,UNKNOWN,VALIDATED,80.0,40,95,48 +Loops,loops/array-1.c,ILP32,true,SUCCEEDED,VALIDATED,83.33,6,2,2 +ProductLines,product-lines/elevator_spec3_product25.cil.c,ILP32,true,ERROR,NO_SUITE,,,0,89 +ControlFlow,openssl/s3_srvr.blast.07.i.cil-1.c,ILP32,unknown,FAILED,VALIDATED,9.0,211,134,48 +Floats,neural-networks/poly_256_thresh_3_unsafe.c-amalgamation.i,ILP32,false,UNKNOWN,VALIDATED,,,0,3 +Arrays,array-examples/sanfoundry_24-1.i,ILP32,true,UNKNOWN,TESTCOV_ERROR,,,500,48 +Recursive,recursified_loop-crafted/recursified_simple_array_index_value_2.i,ILP32,unknown,ERROR,VALIDATED_ABORTS,33.33,9,12,8 +XCSP,xcsp/AllInterval-019.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,5.26,38,21,6 +Heap,ldv-regression/test28-1.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,100.0,6,3,3 +ECA,eca-rers2012/Problem08_label09.c,ILP32,true,ERROR,NO_SUITE,,,0,88 +BitVectors,bitvector/s3_srvr_2.BV.c.cil.c,ILP32,unknown,UNKNOWN,TESTCOV_ERROR,,,283,48 +Sequentialized,seq-mthreaded/pals_opt-floodmax.3.4.ufo.BOUNDED-6.pals.c,ILP32,false,FAILED,TESTCOV_ERROR,,,192,50 +LinkedLists,list-properties/list-1.i,ILP32,true,FAILED,VALIDATED,70.59,17,37,7 +Loops,nla-digbench-scaling/ps5-ll_unwindbound10.c,ILP32,false,SUCCEEDED,VALIDATED,100.0,8,9,3 +ProductLines,product-lines/email_spec8_product14.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +ControlFlow,infeasible-control-flow/unreach_branch3.c,ILP32,true,UNKNOWN,VALIDATED,16.67,6,1,48 +Floats,float-benchs/sin_interpolated_negation.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,16.67,12,1,3 +Arrays,array-examples/standard_copy9_ground-2.i,ILP32,true,UNKNOWN,VALIDATED,4.0,25,1,48 +Recursive,recursified_loop-simple/recursified_nested_2.c,ILP32,unknown,SUCCEEDED,VALIDATED,,,0,6 +XCSP,xcsp/CostasArray-12.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,100.0,2,23,11 +Heap,ldv-regression/test10.c,ILP32,true,SUCCEEDED,VALIDATED,50.0,2,1,3 +ECA,eca-rers2012/Problem07_label03.c,ILP32,false,ERROR,NO_SUITE,,,0,90 +BitVectors,bitvector/s3_clnt_2.BV.c.cil-1.c,ILP32,unknown,UNKNOWN,VALIDATED,,,0,49 +Sequentialized,seq-mthreaded/pals_lcr.4.1.ufo.BOUNDED-8.pals.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,81.25,96,24,21 +LinkedLists,heap-manipulation/dll_of_dll-2.i,ILP32,false,UNKNOWN,VALIDATED,76.06,71,65,49 +Loops,nla-digbench-scaling/geo1-ll_unwindbound50.c,ILP32,true,SUCCEEDED,VALIDATED,,,0,59 +ProductLines,product-lines/email_spec27_product19.cil.c,ILP32,false,ERROR,NO_SUITE,,,0,90 +ControlFlow,openssl/s3_srvr.blast.01.i.cil-2.c,ILP32,unknown,FAILED,TESTCOV_ERROR,,,258,49 +Floats,neural-networks/lunarlander_88_unsafe.c-amalgamation.i,ILP32,false,UNKNOWN,VALIDATED,,,0,2 +Arrays,array-patterns/array22_pattern.c,ILP32,true,FAILED,VALIDATED,31.82,22,4,18 +Recursive,recursive/Ackermann03.c,ILP32,true,UNKNOWN,VALIDATED_ABORTS,90.0,10,77,49 +XCSP,xcsp/aim-200-1-6-sat-2.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,100.0,2,25,19 +Heap,memsafety/test-0102-2.i,ILP32,unknown,FAILED,VALIDATED,72.73,11,85,49 +ECA,eca-rers2012/Problem19_label06.c,ILP32,true,UNKNOWN,VALIDATED,,,0,54 +BitVectors,bitvector-loops/diamond_2-1.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,100.0,24,2,2 +Sequentialized,seq-mthreaded/pals_STARTPALS_ActiveStandby.4_2.ufo.BOUNDED-10.pals.c,ILP32,false,UNKNOWN,VALIDATED,,,0,48 +LinkedLists,forester-heap/dll-circular-1.i,ILP32,false,UNKNOWN,VALIDATED,28.57,21,1,48 +Loops,loop-lit/gj2007.i,ILP32,true,SUCCEEDED,VALIDATED,,,0,2 +ProductLines,product-lines/elevator_spec1_product25.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,2 +ControlFlow,ntdrivers-simplified/floppy_simpl3.cil-2.c,ILP32,unknown,UNKNOWN,VALIDATED,,,0,49 +Floats,float-newlib/float_req_bl_0875.c,ILP32,true,UNKNOWN,VALIDATED,,,0,4 +Arrays,array-examples/standard_running-2.i,ILP32,true,UNKNOWN,VALIDATED,6.67,15,1,49 +Recursive,recursified_loop-simple/recursified_nested_6.c,ILP32,unknown,UNKNOWN,VALIDATED,,,0,49 +XCSP,xcsp/CostasArray-16.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,100.0,2,24,15 +Heap,ldv-regression/alias_of_return_2.c_1.i,ILP32,true,SUCCEEDED,VALIDATED,,,0,1 +ECA,eca-rers2012/Problem06_label30.c,ILP32,true,UNKNOWN,VALIDATED,,,0,53 +BitVectors,bitvector/num_conversion_2.c,ILP32,true,UNKNOWN,VALIDATED,,,0,48 +Sequentialized,seq-mthreaded/pals_lcr-var-start-time.4.2.ufo.BOUNDED-8.pals.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,72.5,120,25,23 +LinkedLists,forester-heap/sll-simple-white-blue-1.i,ILP32,true,FAILED,VALIDATED,83.33,18,48,13 +Loops,nla-digbench-scaling/divbin2_valuebound10.i,ILP32,true,SUCCEEDED,VALIDATED,86.67,15,12,3 +ProductLines,product-lines/email_spec11_product03.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +ControlFlow,openssl-simplified/s3_srvr_11.cil.c,ILP32,unknown,FAILED,VALIDATED,45.54,213,150,48 +Floats,neural-networks/dubinsrejoin_54_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,3 +Arrays,array-fpi/sina4.c,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Recursive,recursified_nla-digbench/recursified_prodbin-ll.c,ILP32,unknown,SUCCEEDED,VALIDATED,,,0,58 +XCSP,xcsp/Haystacks-11.c,ILP32,true,SUCCEEDED,VALIDATED,100.0,2,24,15 +Heap,memsafety-cve/gpac_5.i,ILP32,unknown,FAILED,TESTCOV_ERROR,,,500,50 +ECA,eca-rers2012/Problem12_label01.c,ILP32,true,UNKNOWN,VALIDATED,,,0,51 +BitVectors,bitvector/s3_srvr_3.BV.c.cil.c,ILP32,unknown,FAILED,VALIDATED,55.43,184,183,49 +Sequentialized,seq-mthreaded/pals_opt-floodmax.4.1.ufo.BOUNDED-8.pals.c,ILP32,false,UNKNOWN,TESTCOV_ERROR,,,384,49 +LinkedLists,list-properties/list_flag-2.i,ILP32,true,FAILED,VALIDATED,76.47,17,37,16 +Loops,nla-digbench-scaling/hard2_valuebound5.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,92.86,14,7,3 +ProductLines,product-lines/email_spec4_product30.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,43 +ControlFlow,openssl/s3_srvr.blast.08.i.cil-2.c,ILP32,unknown,FAILED,VALIDATED,9.0,211,134,49 +Floats,neural-networks/poly_4_4_4_4_thresh_5_unsafe.c-amalgamation.i,ILP32,false,UNKNOWN,VALIDATED,,,0,2 +Arrays,array-patterns/array5_pattern.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,87.5,16,30,9 +Recursive,recursive/Addition02WithOverflowBug.c,ILP32,unknown,UNKNOWN,VALIDATED_ABORTS,87.5,8,17,49 +XCSP,xcsp/aim-200-1-6-unsat-2.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,100.0,2,25,19 +Heap,memsafety/global-atexit-2.i,ILP32,unknown,FAILED,VALIDATED,75.0,4,2,2 +ECA,eca-rers2012/Problem18_label33.c,ILP32,false,UNKNOWN,VALIDATED,,,0,50 +BitVectors,bitvector/sum02-1.c,ILP32,false,FAILED,VALIDATED_ABORTS,75.0,4,152,49 +Sequentialized,systemc/token_ring.01.cil-1.c,ILP32,unknown,UNKNOWN,VALIDATED,72.97,74,135,48 +LinkedLists,memsafety-broom/sll-shared-sll-after.i,LP64,unknown,UNKNOWN,VALIDATED,80.0,10,1,48 +Loops,loop-acceleration/multivar_1-2.c,ILP32,false,UNKNOWN,VALIDATED,75.0,4,16,49 +ProductLines,product-lines/elevator_spec14_product20.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,2 +ControlFlow,infeasible-control-flow/unreach_branch1.c,ILP32,true,FAILED,VALIDATED,75.0,8,155,49 +Floats,float-benchs/mea8000.c,ILP32,true,UNKNOWN,VALIDATED,,,0,48 +Arrays,array-crafted/zero_sum3.c,ILP32,true,SUCCEEDED,VALIDATED,85.0,20,3,6 +Recursive,fuzzle-programs/10_fuzzle_50x50_CVE-2016-4487.c,LP64,false,UNKNOWN,VALIDATED,,,0,57 +XCSP,xcsp/AllInterval-013.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,7.69,26,21,5 +Heap,ldv-memsafety/memleaks_test16_1.i,ILP32,unknown,FAILED,VALIDATED,11.76,34,22,40 +ECA,eca-rers2012/Problem03_label08.c,ILP32,true,UNKNOWN,VALIDATED,,,0,49 +BitVectors,bitvector/gcd_1.c,ILP32,true,SUCCEEDED,VALIDATED,80.0,10,4,4 +Sequentialized,seq-mthreaded/pals_floodmax.3.4.ufo.BOUNDED-6.pals.c,ILP32,false,UNKNOWN,VALIDATED,21.98,182,257,48 +LinkedLists,forester-heap/dll-simple-white-blue-1.i,ILP32,false,UNKNOWN,VALIDATED,95.0,20,39,48 +Loops,loop-zilu/benchmark27_linear.i,ILP32,unknown,UNKNOWN,VALIDATED,100.0,7,89,48 +ProductLines,product-lines/elevator_spec2_product25.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,2 +ControlFlow,ntdrivers-simplified/kbfiltr_simpl2.cil-1.c,ILP32,unknown,UNKNOWN,VALIDATED,78.0,200,172,49 +Floats,floats-cbmc-regression/float8.i,ILP32,true,SUCCEEDED,VALIDATED,57.14,7,1,1 +Arrays,array-examples/standard_strcpy_original-2.i,ILP32,true,UNKNOWN,VALIDATED,,,0,48 +Recursive,recursified_nla-digbench/recursified_bresenham.c,ILP32,unknown,ERROR,VALIDATED,66.67,9,3,49 +XCSP,xcsp/Dubois-015.c,ILP32,true,FAILED,VALIDATED,100.0,2,22,6 +Heap,ldv-memsafety/memleaks_test8_2.i,ILP32,unknown,FAILED,VALIDATED,5.88,34,21,23 +ECA,eca-rers2012/Problem06_label13.c,ILP32,true,UNKNOWN,VALIDATED,,,0,52 +BitVectors,bitvector/modulus-2.i,ILP32,true,FAILED,VALIDATED,84.62,13,45,49 +Sequentialized,seq-mthreaded/pals_floodmax.5_overflow.ufo.UNBOUNDED.pals.c,ILP32,false,UNKNOWN,TESTCOV_ERROR,,,266,48 +LinkedLists,forester-heap/sll-rb-cnstr_1-1.i,ILP32,true,UNKNOWN,VALIDATED,80.0,25,3,49 +Loops,nla-digbench-scaling/bresenham-ll_unwindbound20.c,ILP32,false,UNKNOWN,VALIDATED,80.0,10,74,50 +ProductLines,product-lines/email_spec0_product05.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +ControlFlow,openssl-simplified/s3_clnt_1.cil-2.c,ILP32,unknown,FAILED,VALIDATED,45.71,175,177,48 +Floats,neural-networks/cartpole_67_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-fpi/ms3.c,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Recursive,recursified_nla-digbench/recursified_geo1-u.c,ILP32,unknown,UNKNOWN,VALIDATED,71.43,7,5,48 +XCSP,xcsp/Dubois-045.c,ILP32,true,SUCCEEDED,VALIDATED,100.0,2,24,15 +Heap,list-ext-properties/960521-1_1-1.i,ILP32,unknown,FAILED,VALIDATED,77.78,9,19,6 +ECA,eca-rers2012/Problem08_label58.c,ILP32,true,ERROR,NO_SUITE,,,0,90 +BitVectors,bitvector/s3_clnt_2.BV.c.cil-2a.c,ILP32,false,UNKNOWN,VALIDATED,,,0,49 +Sequentialized,seq-mthreaded/pals_lcr.4.ufo.BOUNDED-8.pals.c,ILP32,true,SUCCEEDED,VALIDATED,81.25,96,24,25 +LinkedLists,heap-manipulation/merge_sort-1.i,ILP32,false,UNKNOWN,VALIDATED,39.06,64,4,51 +Loops,nla-digbench-scaling/egcd3-ll_valuebound5.c,ILP32,true,SUCCEEDED,VALIDATED,93.75,16,14,4 +ProductLines,product-lines/email_spec11_productSimulator.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,2 +ControlFlow,openssl-simplified/s3_srvr_2.cil-1.c,ILP32,unknown,UNKNOWN,VALIDATED,52.28,197,170,49 +Floats,neural-networks/dubinsrejoin_90_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,3 +Arrays,array-fpi/sqmf.c,ILP32,false,UNKNOWN,VALIDATED,,,0,4 +Recursive,recursified_nla-digbench/recursified_ps2-ll.c,ILP32,unknown,UNKNOWN,VALIDATED,57.14,7,19,48 +XCSP,xcsp/Haystacks-13.c,ILP32,true,FAILED,VALIDATED,100.0,2,25,17 +Heap,memsafety-cve/gpac_17.i,ILP32,unknown,UNKNOWN,TESTCOV_ERROR,,,500,49 +ECA,eca-rers2012/Problem11_label44.c,ILP32,true,UNKNOWN,VALIDATED,,,0,49 +BitVectors,bitvector/s3_srvr_2_alt.BV.c.cil.c,ILP32,unknown,FAILED,TESTCOV_ERROR,,,368,49 +Sequentialized,seq-mthreaded/pals_opt-floodmax.3.3.ufo.BOUNDED-6.pals.c,ILP32,false,FAILED,TESTCOV_ERROR,,,240,49 +LinkedLists,list-properties/alternating_list-1.i,ILP32,true,UNKNOWN,VALIDATED,73.68,19,7,48 +Loops,nla-digbench-scaling/geo3-ll2_unwindbound20.c,ILP32,unknown,SUCCEEDED,VALIDATED,62.5,8,21,17 +ProductLines,product-lines/email_spec3_product27.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,10 +ControlFlow,openssl/s3_clnt.blast.04.i.cil-2.c,ILP32,unknown,UNKNOWN,VALIDATED,,,0,49 +Floats,neural-networks/lunarlander_75_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,2 +Arrays,array-memsafety/openbsd_cstrlcpy-alloca-1.i,ILP32,unknown,SUCCEEDED,VALIDATED_ABORTS,60.0,20,6,8 +Recursive,recursive-simple/id_b3_o5-2.c,ILP32,true,UNKNOWN,VALIDATED,83.33,6,48,49 +XCSP,xcsp/aim-100-2-0-unsat-1.c,ILP32,true,SUCCEEDED,VALIDATED,100.0,2,23,9 +Heap,memsafety-cve/openrazer.i,ILP32,unknown,FAILED,VALIDATED,10.71,28,1,49 +ECA,eca-rers2012/Problem15_label31.c,ILP32,true,UNKNOWN,VALIDATED,,,0,49 +BitVectors,bitvector/soft_float_2.c.cil.c,ILP32,unknown,FAILED,VALIDATED,,,0,48 +Sequentialized,seq-mthreaded/pals_opt-floodmax.5.4.ufo.BOUNDED-10.pals.c,ILP32,false,UNKNOWN,VALIDATED,,,0,48 +LinkedLists,memsafety-broom/dll-middle-data.i,LP64,unknown,UNKNOWN,VALIDATED,100.0,6,4,48 +Loops,nla-digbench-scaling/prod4br-ll_valuebound20.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,91.67,12,39,11 +ProductLines,product-lines/email_spec6_product34.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,9 +ControlFlow,openssl/s3_srvr.blast.10.i.cil-2.c,ILP32,unknown,FAILED,VALIDATED,9.27,205,142,48 +Floats,neural-networks/robot_5_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,2 +Arrays,array-programs/copysome1-1.i,ILP32,true,UNKNOWN,VALIDATED,,,0,48 +Recursive,recursive/Addition03-2.c,ILP32,unknown,UNKNOWN,VALIDATED,75.0,8,26,48 +XCSP,xcsp/aim-200-1-6-unsat-4.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,100.0,2,25,18 +Heap,memsafety-ext2/split_list_test05-2.i,ILP32,unknown,UNKNOWN,TESTCOV_ERROR,,,500,50 +ECA,eca-rers2012/Problem18_label16.c,ILP32,true,UNKNOWN,VALIDATED,,,0,50 +BitVectors,bitvector/soft_float_5a.c.cil.c,ILP32,true,UNKNOWN,TESTCOV_ERROR,,,500,50 +Sequentialized,seq-pthread/cs_stateful-1.i,ILP32,false,UNKNOWN,VALIDATED,3.94,127,194,49 +LinkedLists,memsafety-broom/sll-nested-sll-lst-data.i,LP64,unknown,UNKNOWN,VALIDATED,,,0,48 +Loops,nla-digbench/fermat2.c,ILP32,unknown,SUCCEEDED,VALIDATED,10.0,10,1,2 +ProductLines,product-lines/email_spec9_product20.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Heap,memsafety/test-0236.i,ILP32,unknown,UNKNOWN,VALIDATED,84.62,13,146,49 +ECA,eca-rers2012/Problem19_label56.c,ILP32,true,UNKNOWN,VALIDATED,,,0,53 +ControlFlow,infeasible-control-flow/conflict_branch2.c,ILP32,true,UNKNOWN,VALIDATED,,,0,48 +Floats,float-benchs/cast_union_tight.c,ILP32,false,SUCCEEDED,VALIDATED,50.0,2,1,1 +Arrays,array-crafted/bAnd2.i,ILP32,true,SUCCEEDED,VALIDATED,77.78,9,1,19 +Sequentialized,seq-mthreaded/pals_STARTPALS_ActiveStandby.ufo.BOUNDED-10.pals.c,ILP32,true,SUCCEEDED,VALIDATED,18.01,161,13,37 +LinkedLists,forester-heap/dll-optional-1.i,ILP32,true,UNKNOWN,VALIDATED,88.89,18,280,49 +Recursive,fuzzle-programs/05_fuzzle_70x70.c,LP64,false,UNKNOWN,VALIDATED,,,0,62 +XCSP,xcsp/AllInterval-010.c,ILP32,false,FAILED,VALIDATED_ABORTS,10.0,20,20,3 +Loops,loop-invgen/apache-get-tag.i.p+sep-reducer.c,ILP32,true,UNKNOWN,VALIDATED,,,0,0 +ProductLines,product-lines/elevator_spec1_product09.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Heap,ldv-memsafety/memleaks_test12-2.i,ILP32,unknown,UNKNOWN,VALIDATED,58.73,63,13,48 +ECA,eca-programs/Problem102_label49.c,ILP32,false,ERROR,NO_SUITE,,,0,90 +ControlFlow,ntdrivers-simplified/floppy_simpl4.cil-2.c,ILP32,unknown,UNKNOWN,TESTCOV_ERROR,,,158,49 +Floats,float-newlib/double_req_bl_0660b.c,ILP32,true,SUCCEEDED,VALIDATED,1.41,71,1,4 +Arrays,array-crafted/zero_sum_m2.c,ILP32,true,SUCCEEDED,VALIDATED,81.25,16,3,6 +Sequentialized,seq-mthreaded/pals_floodmax.4.1.ufo.BOUNDED-8.pals.c,ILP32,false,UNKNOWN,TESTCOV_ERROR,,,500,49 +LinkedLists,forester-heap/dll-token-1.i,ILP32,true,UNKNOWN,VALIDATED,72.0,25,7,48 +Recursive,fuzzle-programs/15_fuzzle_50x50_equality-checks.c,LP64,false,UNKNOWN,VALIDATED,,,0,55 +XCSP,xcsp/AllInterval-025.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,4.0,50,22,5 +Loops,loop-zilu/benchmark01_conjunctive.i,ILP32,unknown,UNKNOWN,VALIDATED_ABORTS,100.0,7,36,48 +ProductLines,product-lines/elevator_spec2_product09.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,14 +Heap,ldv-memsafety/memleaks_test19-1.i,ILP32,unknown,SUCCEEDED,VALIDATED,7.14,42,1,27 +ECA,eca-rers2012/Problem03_label49.c,ILP32,true,UNKNOWN,VALIDATED,3.04,296,7,49 +ControlFlow,ntdrivers/parport.i.cil-2.c,ILP32,true,UNKNOWN,VALIDATED,,,0,49 +Floats,float-newlib/float_req_bl_0920a.c,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-examples/standard_copy7_ground-2.i,ILP32,true,UNKNOWN,VALIDATED,4.76,21,1,49 +Sequentialized,seq-mthreaded/pals_floodmax.5.4.ufo.BOUNDED-10.pals.c,ILP32,false,UNKNOWN,TESTCOV_ERROR,,,299,49 +LinkedLists,forester-heap/sll-rb-cnstr_1-2.i,ILP32,false,FAILED,TESTCOV_ERROR,,,172,28 +Recursive,recursified_loop-simple/recursified_nested_3.c,ILP32,unknown,SUCCEEDED,VALIDATED,,,0,20 +XCSP,xcsp/CostasArray-17.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,100.0,2,24,13 +Loops,loops-crafted-1/in-de20.c,ILP32,true,FAILED,VALIDATED,83.33,6,85,47 +ProductLines,product-lines/elevator_spec3_product03.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,2 +Heap,ldv-memsafety/memleaks_test5.i,ILP32,unknown,SUCCEEDED,VALIDATED,5.88,34,1,13 +ECA,eca-rers2012/Problem05_label31.c,ILP32,true,UNKNOWN,VALIDATED,,,0,53 +ControlFlow,openssl-simplified/s3_srvr_12.cil.c,ILP32,unknown,FAILED,VALIDATED,,,0,48 +Floats,floats-cdfpl/square_2.i,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-examples/standard_vector_difference_ground.i,ILP32,true,UNKNOWN,VALIDATED,,,0,48 +Sequentialized,seq-mthreaded/pals_lcr-var-start-time.6.2.ufo.BOUNDED-12.pals.c,ILP32,false,SUCCEEDED,VALIDATED,32.8,186,29,33 +LinkedLists,heap-manipulation/dll_of_dll-1.i,ILP32,true,UNKNOWN,VALIDATED,76.06,71,71,49 +Recursive,recursified_nla-digbench/recursified_dijkstra.c,ILP32,unknown,UNKNOWN,VALIDATED,66.67,12,2,48 +XCSP,xcsp/Dubois-025.c,ILP32,true,SUCCEEDED,VALIDATED,100.0,2,23,8 +Loops,loops/sum01_bug02_sum01_bug02_base.case.i,ILP32,unknown,UNKNOWN,VALIDATED,100.0,7,17,48 +ProductLines,product-lines/elevator_spec9_product11.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Heap,ldv-regression/rule60_list2.c_1.i,ILP32,false,SUCCEEDED,VALIDATED,91.67,12,16,26 +ECA,eca-rers2012/Problem06_label54.c,ILP32,true,UNKNOWN,VALIDATED,,,0,54 +ControlFlow,openssl/s3_clnt.blast.02.i.cil-1.c,ILP32,unknown,UNKNOWN,VALIDATED,,,0,48 +Floats,neural-networks/cartpole_37_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-fpi/ifeqn3f.c,ILP32,false,UNKNOWN,VALIDATED,,,0,6 +Sequentialized,seq-mthreaded/pals_lcr.6.1.ufo.BOUNDED-12.pals.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,39.86,148,28,30 +LinkedLists,list-ext3-properties/sll_length_check-1.i,ILP32,false,UNKNOWN,VALIDATED_ABORTS,61.54,13,1,26 +Recursive,recursified_nla-digbench/recursified_fermat1.c,ILP32,unknown,SUCCEEDED,VALIDATED_ABORTS,7.69,13,1,4 +XCSP,xcsp/Dubois-060.c,ILP32,true,FAILED,VALIDATED_ABORTS,100.0,2,25,14 +Loops,nla-digbench-scaling/cohendiv-ll_unwindbound50.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,91.67,12,10,2 +ProductLines,product-lines/email_spec0_product26.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,4 +Heap,ldv-regression/test29-1.c,ILP32,false,SUCCEEDED,VALIDATED,100.0,6,3,2 +ECA,eca-rers2012/Problem08_label17.c,ILP32,true,ERROR,NO_SUITE,,,0,89 +ControlFlow,openssl/s3_srvr.blast.06.i.cil-2.c,ILP32,unknown,FAILED,VALIDATED,13.62,235,149,48 +Floats,neural-networks/cartpole_97_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-fpi/nsqmf.c,ILP32,false,UNKNOWN,VALIDATED,,,0,22 +Sequentialized,seq-mthreaded/pals_opt-floodmax.3.3.ufo.UNBOUNDED.pals.c,ILP32,false,UNKNOWN,VALIDATED,,,0,48 +LinkedLists,list-properties/simple-2.i,ILP32,true,FAILED,VALIDATED,63.64,11,10,2 +Recursive,recursified_nla-digbench/recursified_geo2.c,ILP32,unknown,UNKNOWN,VALIDATED,57.14,7,19,48 +XCSP,xcsp/Haystacks-07.c,ILP32,true,FAILED,VALIDATED,100.0,2,22,3 +Loops,nla-digbench-scaling/egcd-ll_valuebound2.c,ILP32,true,SUCCEEDED,VALIDATED,90.0,10,9,2 +ProductLines,product-lines/email_spec11_product26.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Heap,list-ext-properties/test-0158_1-2.i,ILP32,unknown,FAILED,VALIDATED,80.0,5,2,1 +ECA,eca-rers2012/Problem09_label39.c,ILP32,true,ERROR,NO_SUITE,,,0,89 +ControlFlow,openssl/s3_srvr.blast.14.i.cil-1.c,ILP32,unknown,FAILED,VALIDATED,14.35,223,216,48 +Floats,neural-networks/dubinsrejoin_5_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,2 +Arrays,array-fpi/s4liff_overflow.c,ILP32,unknown,UNKNOWN,VALIDATED,,,0,1 +Sequentialized,seq-mthreaded/pals_opt-floodmax.5.1.ufo.BOUNDED-10.pals.c,ILP32,false,UNKNOWN,TESTCOV_ERROR,,,394,48 +LinkedLists,memsafety-broom/linux-hlist-lst-data.i,LP64,unknown,UNKNOWN,VALIDATED,100.0,18,13,48 +Recursive,recursified_nla-digbench/recursified_mannadiv.c,ILP32,unknown,SUCCEEDED,VALIDATED_ABORTS,88.89,9,18,5 +XCSP,xcsp/Haystacks-18.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,100.0,2,25,20 +Loops,nla-digbench-scaling/freire1_valuebound1.c,ILP32,true,SUCCEEDED,VALIDATED,62.5,8,1,1 +ProductLines,product-lines/email_spec1_product30.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,49 +Heap,memsafety-cve/editorconfig-core-cFixed.i,ILP32,unknown,FAILED,TESTCOV_ERROR,,,500,49 +ECA,eca-rers2012/Problem11_label03.c,ILP32,true,UNKNOWN,VALIDATED,25.54,278,7,47 +Heap,memsafety-cve/gpac_1.i,ILP32,unknown,UNKNOWN,TESTCOV_ERROR,,,500,49 +ECA,eca-rers2012/Problem11_label19.c,ILP32,true,UNKNOWN,VALIDATED,25.54,278,7,48 +Heap,memsafety-cve/gpac_16Fixed.i,ILP32,unknown,UNKNOWN,TESTCOV_ERROR,,,500,49 +ECA,eca-rers2012/Problem11_label36.c,ILP32,false,UNKNOWN,VALIDATED,,,0,48 +Heap,memsafety-cve/gpac_19.i,ILP32,unknown,UNKNOWN,TESTCOV_ERROR,,,500,52 +ECA,eca-rers2012/Problem11_label52.c,ILP32,true,UNKNOWN,VALIDATED,25.54,278,7,47 +Heap,memsafety-cve/gpac_6.i,ILP32,unknown,FAILED,VALIDATED_ABORTS,43.33,30,92,48 +ECA,eca-rers2012/Problem12_label09.c,ILP32,true,UNKNOWN,VALIDATED,,,0,49 +Heap,memsafety-cve/jhead.i,ILP32,unknown,UNKNOWN,TESTCOV_ERROR,,,500,50 +ECA,eca-rers2012/Problem12_label25.c,ILP32,false,UNKNOWN,VALIDATED,,,0,49 +Heap,memsafety-cve/kilo.i,ILP32,unknown,UNKNOWN,VALIDATED,29.17,24,111,31 +ECA,eca-rers2012/Problem12_label42.c,ILP32,false,UNKNOWN,VALIDATED,,,0,49 +Heap,memsafety-cve/liblouis.i,ILP32,unknown,UNKNOWN,TESTCOV_ERROR,,,500,48 +ECA,eca-rers2012/Problem12_label58.c,ILP32,true,UNKNOWN,VALIDATED,,,0,49 +Loops,nla-digbench-scaling/geo3-ll_valuebound5.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,87.5,8,11,3 +ProductLines,product-lines/email_spec3_product33.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Recursive,recursive-simple/id_b3_o2-1.c,ILP32,unknown,UNKNOWN,VALIDATED_ABORTS,100.0,6,65,48 +XCSP,xcsp/aim-100-6-0-sat-3.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,100.0,2,23,12 +Sequentialized,systemc/token_ring.02.cil-2.c,ILP32,unknown,UNKNOWN,VALIDATED,64.35,115,136,48 +Arrays,array-memsafety/openbsd_cstrncpy-alloca-2.i,ILP32,unknown,SUCCEEDED,VALIDATED_ABORTS,76.92,13,6,9 +ECA,eca-rers2012/Problem15_label04.c,ILP32,true,UNKNOWN,VALIDATED,,,0,47 +Loops,nla-digbench-scaling/knuth_unwindbound20.i,ILP32,true,SUCCEEDED,VALIDATED,46.15,13,3,2 +XCSP,xcsp/aim-200-1-6-unsat-1.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,100.0,2,25,15 +Floats,neural-networks/poly_16_16_thresh_2_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Heap,memsafety-cve/plutovg.i,ILP32,unknown,UNKNOWN,VALIDATED,19.23,26,41,48 +ProductLines,product-lines/email_spec6_product20.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,41 +Recursive,recursive-simple/sum_non_eq-1.c,ILP32,unknown,FAILED,VALIDATED,75.0,4,66,48 +Arrays,array-patterns/array12_pattern.c,ILP32,true,FAILED,VALIDATED,43.75,16,4,9 +ECA,eca-rers2012/Problem16_label29.c,ILP32,true,UNKNOWN,VALIDATED,17.9,391,7,47 +Loops,nla-digbench-scaling/prodbin-ll_unwindbound2.c,ILP32,false,SUCCEEDED,VALIDATED,,,0,58 +XCSP,xcsp/aim-200-2-0-unsat-4.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,100.0,2,25,14 +Floats,neural-networks/relu_2_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Heap,memsafety-cve/zephyr.i,ILP32,unknown,UNKNOWN,TESTCOV_ERROR,,,500,48 +ProductLines,product-lines/email_spec7_product27.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,34 +Recursive,recursive/Addition03-1.c,ILP32,true,UNKNOWN,VALIDATED_ABORTS,75.0,8,108,47 +Arrays,array-programs/partial_mod_count_limited_1.c,ILP32,true,SUCCEEDED,VALIDATED,92.86,14,15,4 +ECA,eca-rers2012/Problem17_label51.c,ILP32,true,UNKNOWN,VALIDATED,12.14,585,7,49 +Loops,nla-digbench-scaling/ps5-ll_valuebound1.c,ILP32,true,SUCCEEDED,VALIDATED,87.5,8,4,2 +Arrays,array-tiling/nr4.c,ILP32,true,FAILED,VALIDATED_ABORTS,87.5,16,34,18 +ECA,eca-rers2012/Problem18_label24.c,ILP32,true,UNKNOWN,VALIDATED,,,0,48 +Recursive,recursive/Fibonacci04-overflow.c,ILP32,unknown,UNKNOWN,VALIDATED_ABORTS,100.0,5,76,49 diff --git a/tests/testcomp/results_cover-branches_s1/results.q40-stride.csv b/tests/testcomp/results_cover-branches_s1/results.q40-stride.csv new file mode 100644 index 000000000..57d606b47 --- /dev/null +++ b/tests/testcomp/results_cover-branches_s1/results.q40-stride.csv @@ -0,0 +1,28 @@ +category,program,data_model,expected_unreach,verdict,testcov,coverage,goals,n_tests,elapsed_s +Arrays,array-crafted/bAnd4.i,ILP32,true,UNKNOWN,FULL,,,0,44 +Arrays,array-crafted/zero_sum3.c,ILP32,true,SUCCEEDED,FULL,85.0,20,3,7 +Arrays,array-examples/sanfoundry_24-2.i,ILP32,unknown,UNKNOWN,TESTCOV_ERROR,,,500,48 +Arrays,array-examples/standard_copy7_ground-1.i,ILP32,false,UNKNOWN,FULL,4.76,21,1,55 +Arrays,array-examples/standard_strcmp_ground.i,ILP32,true,UNKNOWN,FULL,7.69,13,1,48 +Arrays,array-fpi/brs5_overflow.c,ILP32,unknown,UNKNOWN,FULL,,,0,12 +Arrays,array-fpi/ifeqn1f.c,ILP32,false,UNKNOWN,FULL,,,0,43 +Arrays,array-fpi/mods.c,ILP32,true,UNKNOWN,FULL,,,0,24 +Arrays,array-fpi/res2.c,ILP32,true,UNKNOWN,FULL,,,0,44 +Arrays,array-fpi/s42iff.c,ILP32,false,UNKNOWN,FULL,,,0,25 +Arrays,array-fpi/sina3.c,ILP32,true,UNKNOWN,FULL,,,0,41 +Arrays,array-industry-pattern/array_of_struct_ptr_monotonic.i,ILP32,true,UNKNOWN,TESTCOV_ERROR,,,500,47 +Arrays,array-memsafety/count_down_unsafe.i,ILP32,unknown,FAILED,PARTIAL,91.67,12,44,17 +Arrays,array-memsafety/mult_array-alloca-1.i,ILP32,unknown,SUCCEEDED,FULL,90.0,10,11,10 +Arrays,array-memsafety/selectionsort-alloca-1.i,ILP32,unknown,SUCCEEDED,FULL,50.0,10,4,5 +Arrays,array-multidimensional/rev-2-n-u.c,ILP32,true,UNKNOWN,FULL,,,0,47 +Arrays,array-patterns/array29_pattern.c,ILP32,true,FAILED,PARTIAL,92.86,14,4,11 +Arrays,array-programs/partial_mod_count_limited_base.c,ILP32,true,SUCCEEDED,PARTIAL,92.86,14,2,3 +Arrays,array-tiling/rewnif.c,ILP32,true,UNKNOWN,FULL,,,0,34 +Arrays,reducercommutativity/rangesum60.i,ILP32,false,SUCCEEDED,PARTIAL,92.31,13,2,6 +BitVectors,bitvector-loops/verisec_sendmail_tTflag_arr_one_loop.c,ILP32,false,SUCCEEDED,PARTIAL,100.0,6,22,12 +BitVectors,bitvector/byte_add-2.i,ILP32,unknown,UNKNOWN,PARTIAL,61.4,57,1,44 +BitVectors,bitvector/byte_add_2-1.i,ILP32,unknown,UNKNOWN,FULL,71.93,57,28,46 +BitVectors,bitvector/gcd_2.c,ILP32,true,UNKNOWN,FULL,90.0,10,42,47 +BitVectors,bitvector/jain_1-2.c,ILP32,unknown,UNKNOWN,FULL,66.67,3,1,45 +BitVectors,bitvector/jain_7-1.c,ILP32,unknown,UNKNOWN,FULL,66.67,3,1,45 +BitVectors,bitvector/num_conversion_2.c,ILP32,true,UNKNOWN,FULL,83.33,6,256,48 diff --git a/tests/testcomp/results_cover-branches_s2/results.csv b/tests/testcomp/results_cover-branches_s2/results.csv new file mode 100644 index 000000000..293f2b8e8 --- /dev/null +++ b/tests/testcomp/results_cover-branches_s2/results.csv @@ -0,0 +1,283 @@ +category,program,data_model,expected_unreach,verdict,testcov,coverage,goals,n_tests,elapsed_s +ControlFlow,openssl-simplified/s3_srvr_1a.cil.c,ILP32,unknown,FAILED,VALIDATED,91.23,57,95,47 +Floats,neural-networks/dubinsrejoin_79_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,4 +Arrays,array-examples/standard_two_index_08.i,ILP32,true,UNKNOWN,VALIDATED,11.11,9,1,48 +Recursive,recursified_nla-digbench/recursified_cohendiv-ll.c,ILP32,unknown,SUCCEEDED,VALIDATED,90.0,10,15,16 +XCSP,xcsp/Dubois-017.c,ILP32,true,SUCCEEDED,VALIDATED,100.0,2,22,7 +Heap,memsafety-cve/picotcp.i,ILP32,unknown,FAILED,TESTCOV_ERROR,,,500,50 +ECA,eca-rers2012/Problem15_label48.c,ILP32,false,UNKNOWN,VALIDATED,16.71,419,7,49 +BitVectors,bitvector/byte_add_1-2.i,ILP32,unknown,UNKNOWN,VALIDATED,,,0,48 +Sequentialized,seq-mthreaded/pals_floodmax.3.3.ufo.UNBOUNDED.pals.c,ILP32,false,UNKNOWN,TESTCOV_ERROR,,,202,48 +LinkedLists,forester-heap/dll-reverse.i,ILP32,true,UNKNOWN,VALIDATED,65.38,26,2,48 +Loops,nla-digbench-scaling/cohendiv-ll_valuebound5.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,91.67,12,10,5 +ProductLines,product-lines/email_spec0_product33.cil.c,ILP32,false,ERROR,NO_SUITE,,,0,88 +ControlFlow,openssl/s3_clnt.blast.02.i.cil-2.c,ILP32,unknown,UNKNOWN,TESTCOV_ERROR,,,500,49 +Floats,neural-networks/lunarlander_3_unsafe.c-amalgamation.i,ILP32,false,UNKNOWN,VALIDATED,,,0,2 +Arrays,array-programs/partial_mod_count_limited_5.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,92.86,14,3,3 +Recursive,recursive/EvenOdd01-1.c,ILP32,true,UNKNOWN,VALIDATED,80.0,10,53,48 +XCSP,xcsp/aim-200-2-0-sat-2.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,100.0,2,25,32 +Heap,ldv-memsafety/memleaks_test11_2.i,ILP32,unknown,FAILED,VALIDATED,23.81,42,4,48 +ECA,eca-programs/Problem102_label41.c,ILP32,true,ERROR,NO_SUITE,,,0,90 +BitVectors,bitvector/gcd_2.c,ILP32,true,FAILED,VALIDATED,90.0,10,39,49 +Sequentialized,seq-mthreaded/pals_floodmax.4.3.ufo.BOUNDED-8.pals.c,ILP32,false,UNKNOWN,VALIDATED,,,0,48 +LinkedLists,forester-heap/dll-token-2.i,ILP32,false,UNKNOWN,VALIDATED_ABORTS,56.0,25,7,48 +Loops,loops/trex03-2.c,ILP32,true,FAILED,VALIDATED,85.71,7,11,48 +ProductLines,product-lines/elevator_spec9_product30.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,85 +ControlFlow,openssl-simplified/s3_srvr_1.cil-1.c,ILP32,unknown,UNKNOWN,VALIDATED,51.27,197,163,48 +Floats,neural-networks/dubinsrejoin_2_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,3 +Arrays,array-fpi/ss4f.c,ILP32,false,UNKNOWN,VALIDATED,,,0,41 +Recursive,recursified_nla-digbench/recursified_ps3-ll.c,ILP32,unknown,UNKNOWN,VALIDATED,57.14,7,9,49 +XCSP,xcsp/Haystacks-15.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,100.0,2,25,21 +Heap,memsafety-cve/minizip-ng_1Fixed.i,ILP32,unknown,UNKNOWN,VALIDATED,9.38,32,1,48 +ECA,eca-rers2012/Problem13_label40.c,ILP32,false,UNKNOWN,VALIDATED,,,0,51 +BitVectors,bitvector/soft_float_2a.c.cil.c,ILP32,true,UNKNOWN,TESTCOV_ERROR,,,500,49 +Sequentialized,seq-pthread/cs_fib-2.i,ILP32,false,UNKNOWN,TESTCOV_ERROR,,,500,49 +LinkedLists,memsafety-broom/linux-hlist-middle-data.i,LP64,unknown,UNKNOWN,VALIDATED,77.78,18,2,49 +Loops,nla-digbench/dijkstra.c,ILP32,unknown,UNKNOWN,VALIDATED_ABORTS,78.57,14,57,49 +ProductLines,product-lines/email_spec9_product12.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,5 +ControlFlow,infeasible-control-flow/conflict_branch3.c,ILP32,true,UNKNOWN,VALIDATED,11.11,9,1,49 +Floats,float-benchs/interpolation.c,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-crafted/zero_sum_const_m2.c,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Recursive,fuzzle-programs/12_fuzzle_50x50_CVE-2016-4492.c,LP64,false,UNKNOWN,VALIDATED,,,0,55 +XCSP,xcsp/AllInterval-015.c,ILP32,false,SUCCEEDED,VALIDATED,6.67,30,21,5 +Heap,ldv-memsafety/memleaks_test20-1.i,ILP32,unknown,FAILED,VALIDATED,11.76,34,21,26 +ECA,eca-rers2012/Problem03_label58.c,ILP32,true,UNKNOWN,VALIDATED,,,0,49 +BitVectors,bitvector/interleave_bits.i,ILP32,true,SUCCEEDED,VALIDATED,60.0,5,1,2 +Sequentialized,seq-mthreaded/pals_floodmax.4.ufo.BOUNDED-8.pals.c,ILP32,true,UNKNOWN,TESTCOV_ERROR,,,421,49 +LinkedLists,forester-heap/sll-buckets-1.i,ILP32,true,UNKNOWN,VALIDATED,61.29,31,2,48 +Loops,loops/s3.i,ILP32,unknown,UNKNOWN,TESTCOV_ERROR,,,500,49 +ProductLines,product-lines/elevator_spec3_productSimulator.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,2 +ControlFlow,openssl-simplified/s3_clnt_1.cil-1.c,ILP32,unknown,UNKNOWN,VALIDATED,,,0,49 +Floats,neural-networks/cartpole_55_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-fpi/nsqm-if.c,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Recursive,recursified_nla-digbench/recursified_geo1.c,ILP32,unknown,SUCCEEDED,VALIDATED,57.14,7,2,58 +XCSP,xcsp/Dubois-055.c,ILP32,true,FAILED,VALIDATED,100.0,2,25,18 +Heap,list-ext-properties/test-0217_1.i,ILP32,unknown,UNKNOWN,VALIDATED,85.0,40,167,48 +ECA,eca-rers2012/Problem09_label48.c,ILP32,true,ERROR,NO_SUITE,,,0,90 +BitVectors,bitvector/s3_clnt_3.BV.c.cil-2.c,ILP32,unknown,UNKNOWN,TESTCOV_ERROR,,,274,49 +Sequentialized,seq-mthreaded/pals_lcr.7.ufo.BOUNDED-14.pals.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,37.85,177,31,38 +LinkedLists,list-ext3-properties/dll_nondet_free_order-2.i,ILP32,unknown,FAILED,VALIDATED,86.67,15,6,33 +Loops,nla-digbench-scaling/geo1-u_unwindbound50.c,ILP32,false,UNKNOWN,VALIDATED_ABORTS,75.0,8,18,54 +ProductLines,product-lines/email_spec27_product31.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,31 +ControlFlow,openssl/s3_clnt.blast.04.i.cil-1.c,ILP32,unknown,UNKNOWN,TESTCOV_ERROR,,,500,49 +Floats,neural-networks/lunarlander_63_unsafe.c-amalgamation.i,ILP32,false,UNKNOWN,VALIDATED,,,0,2 +Arrays,array-memsafety/openbsd_cstrstr-alloca-2.i,ILP32,unknown,SUCCEEDED,VALIDATED,39.13,23,6,13 +Recursive,recursive-simple/id_b5_o10-2.c,ILP32,true,FAILED,VALIDATED,83.33,6,59,49 +XCSP,xcsp/aim-100-2-0-unsat-3.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,100.0,2,23,11 +Heap,memsafety-cve/radare2_2Fixed.i,ILP32,unknown,FAILED,TESTCOV_ERROR,,,500,50 +ECA,eca-rers2012/Problem16_label21.c,ILP32,true,UNKNOWN,VALIDATED,,,0,49 +BitVectors,bitvector/soft_float_3a.c.cil.c,ILP32,true,FAILED,TESTCOV_ERROR,,,500,50 +Sequentialized,seq-pthread/cs_lamport.i,ILP32,true,UNKNOWN,TESTCOV_ERROR,,,323,48 +LinkedLists,memsafety-broom/linux-list-middle-data.i,LP64,unknown,UNKNOWN,VALIDATED,100.0,7,3,48 +Loops,nla-digbench-scaling/ps6-ll_valuebound20.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,87.5,8,9,3 +ProductLines,product-lines/email_spec8_product29.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,19 +ControlFlow,unsignedintegeroverflow-sas23/cancel_var_through_overflow.i,ILP32,true,SUCCEEDED,VALIDATED,60.0,5,2,1 +Floats,neural-networks/tanh_w64_r1_case_1_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,2 +Arrays,array-cav19/array_min_and_copy_shift_sum_add.c,ILP32,true,UNKNOWN,VALIDATED,33.33,12,1,49 +Recursive,fuzzle-programs/02_fuzzle_40x40.c,LP64,false,UNKNOWN,VALIDATED,,,0,52 +XCSP,xcsp/AllInterval-005.c,ILP32,false,SUCCEEDED,VALIDATED,20.0,10,19,3 +Heap,ldv-memsafety/memleaks_test1-3.i,ILP32,unknown,SUCCEEDED,VALIDATED,3.12,32,1,13 +ECA,eca-programs/Problem102_label24.c,ILP32,false,ERROR,NO_SUITE,,,0,90 +BitVectors,bitvector/byte_add_1-1.i,ILP32,true,UNKNOWN,VALIDATED,71.93,57,28,49 +Sequentialized,seq-mthreaded/pals_STARTPALS_Triplicated.2.ufo.BOUNDED-10.pals.c,ILP32,false,SUCCEEDED,VALIDATED,40.14,142,25,29 +LinkedLists,forester-heap/dll-queue-1.i,ILP32,true,UNKNOWN,VALIDATED,72.88,59,8,48 +Loops,loop-new/count_by_1.i,ILP32,true,UNKNOWN,VALIDATED,,,0,49 +ProductLines,product-lines/elevator_spec1_product29.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +ControlFlow,ntdrivers-simplified/diskperf_simpl1.cil.c,ILP32,unknown,UNKNOWN,VALIDATED,38.6,171,112,48 +Floats,float-newlib/float_req_bl_0670.c,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-examples/standard_copy5_ground-2.i,ILP32,false,UNKNOWN,VALIDATED,,,0,48 +Recursive,recursified_loop-simple/recursified_nested_1.c,ILP32,unknown,SUCCEEDED,VALIDATED,,,0,2 +XCSP,xcsp/CostasArray-10.c,ILP32,false,SUCCEEDED,VALIDATED,100.0,2,22,6 +Heap,ldv-memsafety/memleaks_test23_2.i,ILP32,unknown,FAILED,VALIDATED,5.77,52,1,48 +ECA,eca-rers2012/Problem05_label06.c,ILP32,true,UNKNOWN,VALIDATED,,,0,53 +BitVectors,bitvector/jain_4-1.c,ILP32,unknown,UNKNOWN,VALIDATED,,,0,48 +Sequentialized,seq-mthreaded/pals_floodmax.5.1.ufo.BOUNDED-10.pals.c,ILP32,false,UNKNOWN,TESTCOV_ERROR,,,240,48 +LinkedLists,forester-heap/sll-circular-1.i,ILP32,true,FAILED,VALIDATED_ABORTS,80.95,21,156,48 +Loops,loops/invert_string-1.c,ILP32,false,SUCCEEDED,VALIDATED,90.0,10,3,2 +ProductLines,product-lines/elevator_spec3_product29.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +ControlFlow,ntdrivers/floppy2.i.cil.c,ILP32,unknown,UNKNOWN,VALIDATED,,,0,3 +Floats,neural-networks/cartpole_19_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-fpi/ifeqn1f.c,ILP32,false,UNKNOWN,VALIDATED,,,0,2 +Recursive,recursified_nla-digbench/recursified_egcd3-ll.c,ILP32,unknown,SUCCEEDED,VALIDATED,92.31,13,21,30 +XCSP,xcsp/Dubois-026.c,ILP32,true,SUCCEEDED,VALIDATED,100.0,2,23,10 +Heap,ldv-regression/test24-2.c,ILP32,false,SUCCEEDED,VALIDATED,87.5,8,5,11 +ECA,eca-rers2012/Problem07_label52.c,ILP32,true,ERROR,NO_SUITE,,,0,90 +BitVectors,bitvector/s3_clnt_1.BV.c.cil-2a.c,ILP32,unknown,FAILED,VALIDATED,46.29,175,182,49 +Sequentialized,seq-mthreaded/pals_lcr-var-start-time.6.1.ufo.BOUNDED-12.pals.c,ILP32,false,UNKNOWN,VALIDATED_ABORTS,31.75,189,28,50 +LinkedLists,forester-heap/sll-token-2.i,ILP32,true,UNKNOWN,VALIDATED,72.0,25,34,49 +Loops,nla-digbench-scaling/divbin_unwindbound50.i,ILP32,false,FAILED,VALIDATED,100.0,15,34,5 +ProductLines,product-lines/email_spec11_product15.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +ControlFlow,openssl-simplified/s3_srvr_1.cil-2.c,ILP32,unknown,UNKNOWN,VALIDATED,51.27,197,163,49 +Floats,neural-networks/dubinsrejoin_41_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,2 +Arrays,array-fpi/s4iff_overflow.c,ILP32,unknown,UNKNOWN,VALIDATED,,,0,1 +Recursive,recursified_nla-digbench/recursified_hard2.c,ILP32,unknown,UNKNOWN,VALIDATED,66.67,12,2,49 +XCSP,xcsp/Haystacks-06.c,ILP32,true,SUCCEEDED,VALIDATED,100.0,2,21,5 +Heap,memsafety-cve/asterisk.i,ILP32,unknown,UNKNOWN,TESTCOV_ERROR,,,500,49 +ECA,eca-rers2012/Problem10_label38.c,ILP32,true,UNKNOWN,VALIDATED,34.33,201,6,48 +BitVectors,bitvector/s3_srvr_1_alt.BV.c.cil.c,ILP32,unknown,UNKNOWN,VALIDATED,,,0,48 +Sequentialized,seq-mthreaded/pals_lcr.7_overflow.ufo.UNBOUNDED.pals.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,38.07,176,31,38 +LinkedLists,list-ext3-properties/dll_nullified-1.i,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,88.24,17,2,8 +Loops,nla-digbench-scaling/geo1-u2_unwindbound20.c,ILP32,unknown,UNKNOWN,VALIDATED,75.0,8,13,53 +ProductLines,product-lines/email_spec27_product27.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +ControlFlow,openssl/s3_clnt.blast.01.i.cil-2.c,ILP32,unknown,UNKNOWN,TESTCOV_ERROR,,,500,49 +Floats,neural-networks/lunarlander_27_unsafe.c-amalgamation.i,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-memsafety/cstrcspn-alloca-1.i,ILP32,unknown,SUCCEEDED,VALIDATED_ABORTS,57.14,14,6,7 +Recursive,recursive-simple/id2_b2_o3.c,ILP32,true,UNKNOWN,VALIDATED,90.0,10,57,49 +XCSP,xcsp/aim-100-1-6-unsat-1.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,100.0,2,23,8 +Heap,memsafety-cve/libsndfile.i,ILP32,unknown,SUCCEEDED,VALIDATED,15.0,20,1,6 +ECA,eca-rers2012/Problem13_label23.c,ILP32,false,UNKNOWN,VALIDATED,,,0,50 +BitVectors,bitvector/soft_float_1-2.c.cil.c,ILP32,unknown,FAILED,TESTCOV_ERROR,,,500,49 +Sequentialized,seq-mthreaded/pals_opt-floodmax.4.4.ufo.BOUNDED-8.pals.c,ILP32,false,UNKNOWN,VALIDATED,,,0,48 +LinkedLists,list-properties/splice-1.i,ILP32,false,FAILED,VALIDATED,65.22,23,8,2 +Loops,nla-digbench-scaling/knuth_valuebound5.i,ILP32,true,SUCCEEDED,VALIDATED,46.15,13,4,3 +ProductLines,product-lines/email_spec4_product34.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +ControlFlow,openssl/s3_srvr.blast.07.i.cil-2.c,ILP32,unknown,FAILED,VALIDATED,9.0,211,142,48 +Floats,neural-networks/poly_32_32_32_thresh_4_unsafe.c-amalgamation.i,ILP32,false,UNKNOWN,VALIDATED,,,0,2 +Arrays,array-patterns/array16_pattern.c,ILP32,true,FAILED,VALIDATED_ABORTS,38.89,18,4,12 +Recursive,recursive/Ackermann01-2.c,ILP32,true,UNKNOWN,VALIDATED_ABORTS,90.0,10,77,49 +XCSP,xcsp/aim-100-6-0-sat-4.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,100.0,2,23,10 +Heap,memsafety-cve/zchunk.i,ILP32,unknown,FAILED,TESTCOV_ERROR,,,500,49 +ECA,eca-rers2012/Problem17_label10.c,ILP32,true,UNKNOWN,VALIDATED,,,0,50 +BitVectors,bitvector/soft_float_4-2a.c.cil.c,ILP32,true,FAILED,TESTCOV_ERROR,,,500,49 +Sequentialized,seq-pthread/cs_peterson.i,ILP32,true,UNKNOWN,TESTCOV_ERROR,,,500,51 +LinkedLists,memsafety-broom/sll-fst-shared.i,LP64,unknown,UNKNOWN,VALIDATED,87.5,8,18,48 +Loops,nla-digbench-scaling/ps5-ll_valuebound50.c,ILP32,true,SUCCEEDED,VALIDATED,87.5,8,9,2 +ProductLines,product-lines/email_spec8_product21.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +ControlFlow,openssl/s3_srvr.blast.15.i.cil-1.c,ILP32,unknown,FAILED,VALIDATED,8.76,217,141,48 +Floats,neural-networks/sqrt_1_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Arrays,reducercommutativity/max.i,ILP32,true,FAILED,VALIDATED,81.82,11,15,11 +Recursive,recursive/McCarthy91-2.c,ILP32,true,UNKNOWN,VALIDATED,75.0,4,125,48 +XCSP,xcsp/aim-200-3-4-sat-3.c,ILP32,false,SUCCEEDED,VALIDATED,100.0,2,25,18 +Loops,nla-digbench/ps2-ll.c,ILP32,true,UNKNOWN,VALIDATED,62.5,8,55,49 +ProductLines,product-lines/email_spec9_product32.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Heap,heap-data/cart.i,ILP32,true,FAILED,TESTCOV_ERROR,,,500,49 +ECA,eca-programs/Problem101_label07.c,ILP32,false,ERROR,NO_SUITE,,,0,90 +ControlFlow,infeasible-control-flow/unreach_branch2.c,ILP32,true,FAILED,VALIDATED,75.0,8,46,49 +Floats,float-benchs/filter2_set.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,16.67,6,1,1 +Arrays,array-crafted/bor4.i,ILP32,true,UNKNOWN,VALIDATED,11.11,9,1,49 +Sequentialized,seq-mthreaded/pals_STARTPALS_Triplicated.ufo.BOUNDED-10.pals.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,39.31,145,25,31 +LinkedLists,forester-heap/dll-queue-2.i,ILP32,false,UNKNOWN,VALIDATED_ABORTS,72.88,59,8,49 +Recursive,fuzzle-programs/07_fuzzle_50x50_25-cycle.c,LP64,false,UNKNOWN,VALIDATED,,,0,53 +XCSP,xcsp/AllInterval-012.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,8.33,24,21,4 +Loops,loop-invgen/id_trans.i,ILP32,false,UNKNOWN,VALIDATED_ABORTS,77.78,9,40,48 +ProductLines,product-lines/elevator_spec1_product19.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,7 +Heap,ldv-memsafety/memleaks_test14.i,ILP32,unknown,SUCCEEDED,VALIDATED,8.33,36,1,18 +ECA,eca-programs/Problem103_label43.c,ILP32,true,ERROR,NO_SUITE,,,0,90 +ControlFlow,ntdrivers-simplified/kbfiltr_simpl2.cil-2.c,ILP32,unknown,UNKNOWN,VALIDATED_ABORTS,78.0,200,178,49 +Floats,float-newlib/double_req_bl_0684b.c,ILP32,true,SUCCEEDED,VALIDATED,30.14,73,4,20 +Arrays,array-examples/relax-2-2.i,ILP32,true,FAILED,VALIDATED_ABORTS,36.11,36,7,3 +Sequentialized,seq-mthreaded/pals_floodmax.4.2.ufo.UNBOUNDED.pals.c,ILP32,false,UNKNOWN,TESTCOV_ERROR,,,500,49 +LinkedLists,forester-heap/sll-01-2.i,ILP32,false,FAILED,VALIDATED_ABORTS,44.83,58,77,9 +Recursive,recursified_loop-crafted/recursified_simple_array_index_value_4.i.v+lhb-reducer.c,ILP32,unknown,UNKNOWN,VALIDATED,,,0,48 +XCSP,xcsp/AllInterval-035.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,2.86,70,23,8 +Loops,loop-zilu/benchmark11_linear_abstracted.i,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,69.23,13,5,3 +ProductLines,product-lines/elevator_spec2_product19.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,47 +Heap,ldv-memsafety/memleaks_test21-1.i,ILP32,unknown,FAILED,VALIDATED,11.76,34,21,23 +ECA,eca-rers2012/Problem04_label06.c,ILP32,false,UNKNOWN,VALIDATED,,,0,51 +ControlFlow,openssl-simplified/s3_clnt_2.cil-1.c,ILP32,unknown,UNKNOWN,VALIDATED,,,0,47 +Floats,float-newlib/float_req_bl_1270a.c,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-examples/standard_find_ground-2.i,ILP32,unknown,UNKNOWN,VALIDATED,71.43,7,2,48 +Sequentialized,seq-mthreaded/pals_lcr-var-start-time.3.1.ufo.BOUNDED-6.pals.c,ILP32,false,SUCCEEDED,VALIDATED,84.95,93,19,16 +LinkedLists,forester-heap/sll-reverse_simple.i,ILP32,true,UNKNOWN,VALIDATED,84.62,26,86,48 +Recursive,recursified_loop-simple/recursified_nested_5.c,ILP32,unknown,UNKNOWN,VALIDATED,,,0,49 +XCSP,xcsp/Dubois-016.c,ILP32,true,FAILED,VALIDATED,100.0,2,22,6 +Loops,loops-crafted-1/nested3-1_abstracted.c,ILP32,true,SUCCEEDED,VALIDATED,64.29,14,13,2 +ProductLines,product-lines/elevator_spec3_product18.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Heap,ldv-memsafety/memleaks_test6_1.i,ILP32,unknown,FAILED,VALIDATED,17.5,40,155,48 +ECA,eca-rers2012/Problem05_label48.c,ILP32,false,UNKNOWN,VALIDATED,,,0,54 +ControlFlow,openssl-simplified/s3_srvr_14.cil.c,ILP32,unknown,FAILED,VALIDATED,15.21,217,1,49 +Floats,floats-esbmc-regression/trunc_nondet_2.i,ILP32,true,SUCCEEDED,VALIDATED,66.67,9,1,1 +Arrays,array-fpi/brs4f.c,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Sequentialized,seq-mthreaded/pals_lcr.3.1.ufo.BOUNDED-6.pals.c,ILP32,false,SUCCEEDED,VALIDATED,79.45,73,17,14 +LinkedLists,heap-manipulation/merge_sort-2.i,ILP32,true,UNKNOWN,VALIDATED,75.0,64,68,49 +Recursive,recursified_nla-digbench/recursified_divbin2.i,ILP32,unknown,UNKNOWN,VALIDATED_ABORTS,69.23,13,95,48 +XCSP,xcsp/Dubois-027.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,100.0,2,23,9 +Loops,loops/terminator_03-2_abstracted.i,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,69.23,13,5,2 +ProductLines,product-lines/elevator_spec9_product28.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,5 +Heap,ldv-regression/test14.c,ILP32,true,SUCCEEDED,VALIDATED,75.0,4,2,3 +ECA,eca-rers2012/Problem07_label11.c,ILP32,false,ERROR,NO_SUITE,,,0,90 +ControlFlow,openssl/s3_clnt.blast.03.i.cil-2.c,ILP32,unknown,UNKNOWN,TESTCOV_ERROR,,,500,50 +Floats,neural-networks/cartpole_49_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-fpi/indp1f.c,ILP32,false,UNKNOWN,VALIDATED,,,0,48 +Sequentialized,seq-mthreaded/pals_lcr.7.1.ufo.BOUNDED-14.pals.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,33.9,177,30,32 +LinkedLists,list-ext3-properties/sll_nondet_insert-2.i,ILP32,true,UNKNOWN,VALIDATED,80.95,21,157,47 +Recursive,recursified_nla-digbench/recursified_fermat2.c,ILP32,unknown,SUCCEEDED,VALIDATED,11.11,9,1,1 +XCSP,xcsp/Dubois-070.c,ILP32,true,SUCCEEDED,VALIDATED,100.0,2,25,12 +Loops,nla-digbench-scaling/dijkstra-u_unwindbound20.c,ILP32,false,SUCCEEDED,VALIDATED,100.0,14,17,3 +ProductLines,product-lines/email_spec0_product35.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Heap,ldv-sets/test_mutex.i,ILP32,true,FAILED,VALIDATED,,,0,10 +ECA,eca-rers2012/Problem08_label33.c,ILP32,true,ERROR,NO_SUITE,,,0,89 +ControlFlow,openssl/s3_srvr.blast.08.i.cil-1.c,ILP32,unknown,FAILED,VALIDATED,15.02,213,224,48 +Floats,neural-networks/dubinsrejoin_10_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,2 +Arrays,array-fpi/res2f.c,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Sequentialized,seq-mthreaded/pals_opt-floodmax.3_overflow.ufo.UNBOUNDED.pals.c,ILP32,false,FAILED,TESTCOV_ERROR,,,323,47 +LinkedLists,list-properties/splice-2.i,ILP32,true,FAILED,VALIDATED,78.26,23,9,3 +Recursive,recursified_nla-digbench/recursified_geo3-ll2.c,ILP32,unknown,UNKNOWN,VALIDATED,57.14,7,17,47 +XCSP,xcsp/Haystacks-10.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,100.0,2,23,6 +Loops,nla-digbench-scaling/egcd2-ll_valuebound10.c,ILP32,true,SUCCEEDED,VALIDATED,91.67,12,9,2 +ProductLines,product-lines/email_spec11_product33.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Heap,list-ext-properties/test-0232_1-2.i,ILP32,unknown,UNKNOWN,VALIDATED,75.0,8,10,48 +ECA,eca-rers2012/Problem09_label56.c,ILP32,false,ERROR,NO_SUITE,,,0,89 +ControlFlow,openssl/s3_srvr.blast.15.i.cil-2.c,ILP32,unknown,FAILED,VALIDATED,14.61,219,189,48 +Floats,neural-networks/dubinsrejoin_72_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,2 +Arrays,array-fpi/s5iff.c,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Sequentialized,seq-mthreaded/pals_opt-floodmax.5.2.ufo.UNBOUNDED.pals.c,ILP32,false,UNKNOWN,TESTCOV_ERROR,,,355,48 +LinkedLists,memsafety-broom/linux-list-lst-data.i,LP64,unknown,UNKNOWN,VALIDATED,100.0,7,14,47 +Recursive,recursified_nla-digbench/recursified_prod4br.c,ILP32,unknown,UNKNOWN,VALIDATED,,,0,48 +XCSP,xcsp/aim-100-1-6-sat-1.c,ILP32,false,SUCCEEDED,VALIDATED,100.0,2,23,8 +Loops,nla-digbench-scaling/freire2_valuebound5.c,ILP32,false,SUCCEEDED,VALIDATED,62.5,8,1,1 +ProductLines,product-lines/email_spec1_product34.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Loops,nla-digbench-scaling/geo1-ll_unwindbound1.c,ILP32,true,SUCCEEDED,VALIDATED,,,0,58 +ProductLines,product-lines/email_spec27_product17.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,31 +Loops,nla-digbench-scaling/geo1-ll_valuebound5.c,ILP32,true,SUCCEEDED,VALIDATED,87.5,8,11,6 +ProductLines,product-lines/email_spec27_product24.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Loops,nla-digbench-scaling/geo1-u_unwindbound1.c,ILP32,false,SUCCEEDED,VALIDATED,75.0,8,3,2 +ProductLines,product-lines/email_spec27_product29.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Loops,nla-digbench-scaling/geo1-u_valuebound5.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,100.0,8,8,2 +ProductLines,product-lines/email_spec27_product33.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,46 +Loops,nla-digbench-scaling/geo2-ll_unwindbound1.c,ILP32,true,SUCCEEDED,VALIDATED,62.5,8,2,5 +ProductLines,product-lines/email_spec3_product13.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,6 +Loops,nla-digbench-scaling/geo2-ll_valuebound5.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,87.5,8,9,4 +ProductLines,product-lines/email_spec3_product24.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,29 +Loops,nla-digbench-scaling/geo3-ll_unwindbound1.c,ILP32,true,SUCCEEDED,VALIDATED,62.5,8,2,3 +ProductLines,product-lines/email_spec3_product29.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,2 +Recursive,recursive-simple/id2_b5_o10.c,ILP32,true,UNKNOWN,VALIDATED,90.0,10,38,49 +XCSP,xcsp/aim-100-6-0-sat-1.c,ILP32,false,SUCCEEDED,VALIDATED,100.0,2,23,6 +Sequentialized,systemc/token_ring.01.cil-2.c,ILP32,unknown,UNKNOWN,VALIDATED_ABORTS,75.0,84,182,48 +Arrays,array-memsafety/openbsd_cstrcat-alloca-2.i,ILP32,unknown,SUCCEEDED,VALIDATED,75.0,16,16,21 +Floats,neural-networks/lunarlander_92_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Heap,memsafety-cve/netatalk.i,ILP32,unknown,FAILED,VALIDATED,10.71,28,1,47 +ProductLines,product-lines/email_spec4_product28.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,2 +Recursive,recursive-simple/id_o10.c,ILP32,false,UNKNOWN,VALIDATED,100.0,4,81,48 +Arrays,array-multidimensional/add-2-n-u.c,ILP32,true,UNKNOWN,VALIDATED,,,0,48 +ECA,eca-rers2012/Problem15_label39.c,ILP32,false,UNKNOWN,VALIDATED,16.71,419,7,48 +Loops,nla-digbench-scaling/lcm2_unwindbound100.c,ILP32,true,SUCCEEDED,VALIDATED,90.0,10,35,3 +XCSP,xcsp/aim-200-2-0-sat-1.c,ILP32,false,SUCCEEDED,VALIDATED,100.0,2,25,18 +Floats,neural-networks/poly_32_32_thresh_4_unsafe.c-amalgamation.i,ILP32,false,UNKNOWN,VALIDATED,,,0,2 +Heap,memsafety-cve/smartdns.i,ILP32,unknown,UNKNOWN,TESTCOV_ERROR,,,500,65 +ProductLines,product-lines/email_spec6_product32.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Recursive,recursive/Ackermann02.c,ILP32,false,UNKNOWN,VALIDATED_ABORTS,100.0,10,77,49 +Arrays,array-patterns/array25_pattern.c,ILP32,true,FAILED,VALIDATED,93.33,15,4,10 +ECA,eca-rers2012/Problem17_label02.c,ILP32,true,UNKNOWN,VALIDATED,,,0,48 +Loops,nla-digbench-scaling/ps2-ll_valuebound50.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,87.5,8,12,2 +XCSP,xcsp/aim-200-3-4-sat-4.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,100.0,2,25,14 +Floats,neural-networks/softmax_3_unsafe.c-amalgamation.i,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Heap,memsafety-ext2/complex_data_creation_test02-1.i,ILP32,unknown,ERROR,TESTCOV_ERROR,,,348,56 +ProductLines,product-lines/email_spec7_product35.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,36 +Recursive,recursive/EvenOdd03.c,ILP32,false,UNKNOWN,VALIDATED_ABORTS,80.0,10,24,48 +Heap,memsafety-ext3/realloc1.c,ILP32,unknown,UNKNOWN,VALIDATED,,,0,0 +ProductLines,product-lines/email_spec8_product26.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Floats,neural-networks/softsign_w8_r2_case_0_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,2 +Loops,nla-digbench/bresenham.c,ILP32,unknown,UNKNOWN,TESTCOV_ERROR,,,144,48 +Arrays,reducercommutativity/avg10-2.i,ILP32,true,SUCCEEDED,VALIDATED,77.78,9,1,2 +ECA,eca-rers2012/Problem19_label14.c,ILP32,false,UNKNOWN,VALIDATED,,,0,51 +Recursive,recursive/MultCommutative-2.c,ILP32,true,FAILED,VALIDATED,80.0,10,197,48 +Heap,memsafety/test-0235-2.i,ILP32,unknown,UNKNOWN,VALIDATED,93.94,33,198,48 +ProductLines,product-lines/email_spec9_product30.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,51 +Floats,neural-networks/vdp_3_unsafe.c-amalgamation.i,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Loops,loop-acceleration/array_2-1.i,ILP32,false,UNKNOWN,VALIDATED_ABORTS,85.71,7,1,39 +Arrays,array-cav19/array_init_var_plus_ind2.c,ILP32,true,UNKNOWN,VALIDATED,70.0,10,61,48 +ECA,eca-programs/Problem101_label11.c,ILP32,false,UNKNOWN,VALIDATED,,,0,85 +Recursive,recursive-simple/id_o20.c,ILP32,false,UNKNOWN,VALIDATED_ABORTS,100.0,4,78,48 +Heap,heap-data/running_example.i,ILP32,true,UNKNOWN,TESTCOV_ERROR,,,500,48 +ProductLines,product-lines/elevator_spec14_product27.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,1 diff --git a/tests/testcomp/results_cover-branches_s3/results.csv b/tests/testcomp/results_cover-branches_s3/results.csv new file mode 100644 index 000000000..2155a6af9 --- /dev/null +++ b/tests/testcomp/results_cover-branches_s3/results.csv @@ -0,0 +1,270 @@ +category,program,data_model,expected_unreach,verdict,testcov,coverage,goals,n_tests,elapsed_s +Heap,memory-alloca/c.03-alloca-2.i,ILP32,unknown,UNKNOWN,VALIDATED,25.0,4,1,48 +ECA,eca-rers2012/Problem10_label22.c,ILP32,true,UNKNOWN,VALIDATED,34.33,201,6,48 +BitVectors,bitvector/jain_2-2.c,ILP32,unknown,UNKNOWN,VALIDATED,66.67,3,1,47 +Sequentialized,seq-mthreaded/pals_floodmax.5.2.ufo.UNBOUNDED.pals.c,ILP32,false,UNKNOWN,TESTCOV_ERROR,,,152,49 +LinkedLists,forester-heap/sll-optional-1.i,ILP32,true,UNKNOWN,VALIDATED,66.67,18,3,48 +Loops,nla-digbench-scaling/lcm1_valuebound2.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,93.75,16,7,4 +ProductLines,product-lines/email_spec6_product15.cil.c,ILP32,false,ERROR,NO_SUITE,,,0,88 +ControlFlow,ntdrivers-simplified/cdaudio_simpl1.cil-2.c,ILP32,unknown,UNKNOWN,VALIDATED,31.15,366,44,49 +Floats,float-newlib/double_req_bl_1231b.c,ILP32,true,SUCCEEDED,VALIDATED,43.75,16,1,3 +Arrays,array-fpi/modpf.c,ILP32,false,UNKNOWN,VALIDATED,,,0,47 +Recursive,recursified_nla-digbench/recursified_geo1-ll.c,ILP32,unknown,SUCCEEDED,VALIDATED,,,0,57 +XCSP,xcsp/Dubois-040.c,ILP32,true,FAILED,VALIDATED_ABORTS,100.0,2,24,14 +Heap,memsafety-cve/json-parser.i,ILP32,unknown,FAILED,TESTCOV_ERROR,,,500,48 +ECA,eca-rers2012/Problem12_label34.c,ILP32,false,UNKNOWN,VALIDATED,,,0,51 +BitVectors,bitvector/soft_float_4-2.c.cil.c,ILP32,unknown,FAILED,TESTCOV_ERROR,,,500,48 +Sequentialized,seq-pthread/cs_queue-2.i,ILP32,false,FAILED,VALIDATED_ABORTS,7.69,208,28,50 +LinkedLists,memsafety-broom/sll-lst-shared.i,LP64,unknown,UNKNOWN,VALIDATED,87.5,8,15,49 +Loops,loop-invgen/half_2.i,ILP32,true,FAILED,VALIDATED,63.64,11,87,49 +ProductLines,product-lines/elevator_spec1_product17.cil.c,ILP32,true,ERROR,NO_SUITE,,,0,90 +ControlFlow,ntdrivers-simplified/kbfiltr_simpl1.cil.c,ILP32,unknown,FAILED,VALIDATED,75.0,124,111,49 +Floats,floats-cbmc-regression/float-flags-simp1.i,ILP32,true,SUCCEEDED,VALIDATED,71.68,113,1,1 +Arrays,array-fpi/eqn4.c,ILP32,true,UNKNOWN,VALIDATED,,,0,29 +Recursive,recursified_nla-digbench/recursified_egcd2-ll.c,ILP32,unknown,SUCCEEDED,VALIDATED_ABORTS,90.0,10,12,33 +XCSP,xcsp/Dubois-024.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,100.0,2,23,9 +Heap,list-ext-properties/list-ext_2.i,ILP32,unknown,UNKNOWN,TESTCOV_ERROR,,,207,49 +ECA,eca-rers2012/Problem09_label15.c,ILP32,false,ERROR,NO_SUITE,,,0,90 +BitVectors,bitvector/s3_srvr_1.BV.c.cil.c,ILP32,unknown,UNKNOWN,VALIDATED,,,0,49 +Sequentialized,seq-mthreaded/pals_lcr.8.ufo.BOUNDED-16.pals.c,ILP32,true,FAILED,VALIDATED,31.73,208,32,41 +LinkedLists,list-ext3-properties/sll_length_check-2.i,ILP32,true,UNKNOWN,VALIDATED,76.92,13,31,48 +Loops,nla-digbench-scaling/hard-u_unwindbound100.c,ILP32,false,SUCCEEDED,VALIDATED,100.0,14,11,3 +ProductLines,product-lines/email_spec4_product18.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +ControlFlow,openssl/s3_srvr.blast.10.i.cil-1.c,ILP32,unknown,FAILED,VALIDATED,,,0,49 +Floats,neural-networks/poly_8_8_8_thresh_2_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,2 +Arrays,array-tiling/tcpy.c,ILP32,true,ERROR,NO_SUITE,,,0,90 +Recursive,recursive/Fibonacci05.c,ILP32,false,UNKNOWN,VALIDATED_ABORTS,100.0,7,48,49 +XCSP,xcsp/aim-200-3-4-sat-1.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,100.0,2,25,28 +Heap,heap-data/shared_mem1.i,ILP32,true,UNKNOWN,VALIDATED,,,0,48 +ECA,eca-programs/Problem102_label08.c,ILP32,true,ERROR,NO_SUITE,,,0,90 +BitVectors,bitvector/byte_add-2.i,ILP32,unknown,UNKNOWN,VALIDATED_ABORTS,61.4,57,1,49 +Sequentialized,seq-mthreaded/pals_floodmax.3.1.ufo.BOUNDED-6.pals.c,ILP32,false,FAILED,TESTCOV_ERROR,,,251,48 +LinkedLists,forester-heap/dll-rb-cnstr_1-1.i,ILP32,true,UNKNOWN,VALIDATED,76.47,34,3,49 +Loops,loop-zilu/benchmark17_conjunctive.i,ILP32,true,UNKNOWN,VALIDATED,71.43,7,71,48 +ProductLines,product-lines/elevator_spec2_product21.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,2 +ControlFlow,ntdrivers/cdaudio.i.cil-1.c,ILP32,false,UNKNOWN,VALIDATED,,,0,50 +Floats,floats-cdfpl/newton_2_6.i,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-fpi/brs5f_overflow.c,ILP32,unknown,UNKNOWN,VALIDATED,,,0,1 +Recursive,recursified_nla-digbench/recursified_divbin.i,ILP32,unknown,SUCCEEDED,VALIDATED,100.0,13,25,12 +XCSP,xcsp/Dubois-020.c,ILP32,true,FAILED,VALIDATED,100.0,2,23,7 +Heap,ldv-regression/test22-3.c,ILP32,false,SUCCEEDED,VALIDATED,100.0,10,38,20 +ECA,eca-rers2012/Problem07_label36.c,ILP32,false,ERROR,NO_SUITE,,,0,90 +BitVectors,bitvector/s3_clnt_1.BV.c.cil-2.c,ILP32,unknown,UNKNOWN,VALIDATED,46.29,175,185,49 +Sequentialized,seq-mthreaded/pals_lcr-var-start-time.6.ufo.BOUNDED-12.pals.c,ILP32,true,FAILED,VALIDATED_ABORTS,32.8,189,30,39 +LinkedLists,heap-manipulation/bubble_sort_linux-2.i,ILP32,false,UNKNOWN,VALIDATED_ABORTS,32.81,64,3,49 +Loops,nla-digbench-scaling/egcd3-ll_unwindbound1.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,43.75,16,3,2 +ProductLines,product-lines/email_spec11_product36.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,7 +ControlFlow,openssl-simplified/s3_srvr_3.cil.c,ILP32,unknown,FAILED,VALIDATED,52.28,197,154,49 +Floats,neural-networks/erf_1_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-lopstr16/base_case.i,ILP32,true,UNKNOWN,VALIDATED,,,0,30 +Recursive,recursified_nla-digbench/recursified_ps5-ll.c,ILP32,unknown,SUCCEEDED,VALIDATED,85.71,7,9,7 +XCSP,xcsp/Haystacks-19.c,ILP32,true,FAILED,VALIDATED_ABORTS,100.0,2,25,37 +Heap,memsafety-cve/libpe.i,ILP32,unknown,FAILED,VALIDATED_ABORTS,0,0,1,49 +ECA,eca-rers2012/Problem13_label07.c,ILP32,false,UNKNOWN,VALIDATED,,,0,52 +BitVectors,bitvector/soft_float_1-1.c.cil.c,ILP32,unknown,FAILED,TESTCOV_ERROR,,,500,49 +Sequentialized,seq-mthreaded/pals_opt-floodmax.4_overflow.ufo.UNBOUNDED.pals.c,ILP32,false,UNKNOWN,TESTCOV_ERROR,,,438,49 +LinkedLists,memsafety-broom/dll-backwards.i,LP64,unknown,ERROR,VALIDATED,100.0,8,4,49 +Loops,nla-digbench-scaling/mannadiv_unwindbound50.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,100.0,10,23,3 +ProductLines,product-lines/email_spec6_product30.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +ControlFlow,openssl/s3_srvr.blast.11.i.cil-2.c,ILP32,unknown,FAILED,VALIDATED,9.0,211,140,49 +Floats,neural-networks/softmax_10_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-tiling/pnr4.c,ILP32,true,UNKNOWN,VALIDATED,,,0,40 +Recursive,recursive/Fibonacci03.c,ILP32,true,UNKNOWN,VALIDATED_ABORTS,85.71,7,48,49 +XCSP,xcsp/aim-200-2-0-unsat-1.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,100.0,2,25,20 +Heap,memsafety/test-0234-2.i,ILP32,unknown,FAILED,VALIDATED,84.62,13,153,49 +ECA,eca-rers2012/Problem19_label39.c,ILP32,true,UNKNOWN,VALIDATED,,,0,53 +BitVectors,bitvector-loops/verisec_sendmail_tTflag_arr_one_loop.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,100.0,6,22,13 +Sequentialized,seq-mthreaded/pals_STARTPALS_ActiveStandby.1.ufo.BOUNDED-10.pals.c,ILP32,false,SUCCEEDED,VALIDATED,18.01,161,13,40 +LinkedLists,forester-heap/dll-01-1.i,ILP32,false,UNKNOWN,VALIDATED,60.71,56,257,49 +Loops,loop-invgen/NetBSD_loop.i,ILP32,true,FAILED,VALIDATED,55.56,9,36,49 +ProductLines,product-lines/elevator_spec1_product01.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +ControlFlow,infeasible-control-flow/unreach_branch4.c,ILP32,true,UNKNOWN,VALIDATED,75.0,8,80,49 +Floats,float-benchs/zonotope_loose.c,ILP32,true,SUCCEEDED,VALIDATED,50.0,6,1,1 +Arrays,array-examples/data_structures_set_multi_proc_ground-1.i,ILP32,false,UNKNOWN,VALIDATED,,,0,48 +Recursive,fuzzle-programs/14_fuzzle_50x50_CVE-2016-6131.c,LP64,false,UNKNOWN,VALIDATED,,,0,54 +XCSP,xcsp/AllInterval-017.c,ILP32,false,SUCCEEDED,VALIDATED,5.88,34,21,5 +Heap,ldv-memsafety/memleaks_test18_2.i,ILP32,unknown,FAILED,VALIDATED,25.0,40,10,49 +ECA,eca-rers2012/Problem03_label41.c,ILP32,true,UNKNOWN,VALIDATED,3.04,296,7,49 +BitVectors,bitvector/gcd_3.c,ILP32,true,UNKNOWN,VALIDATED,90.0,10,45,49 +Sequentialized,seq-mthreaded/pals_floodmax.4.2.ufo.BOUNDED-8.pals.c,ILP32,false,UNKNOWN,TESTCOV_ERROR,,,500,49 +LinkedLists,forester-heap/dll-sorted-2.i,ILP32,true,UNKNOWN,TESTCOV_ERROR,,,500,50 +Loops,loop-zilu/benchmark50_linear.i,ILP32,unknown,FAILED,VALIDATED,100.0,7,126,48 +ProductLines,product-lines/elevator_spec2_productSimulator.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +ControlFlow,ntdrivers/cdaudio.i.cil-2.c,ILP32,unknown,UNKNOWN,VALIDATED,0.63,953,1,50 +Floats,floats-cdfpl/sine_3.i,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-fpi/brs2f.c,ILP32,false,UNKNOWN,VALIDATED,,,0,2 +Recursive,recursified_nla-digbench/recursified_dijkstra-u.c,ILP32,unknown,SUCCEEDED,VALIDATED,91.67,12,21,25 +XCSP,xcsp/Dubois-018.c,ILP32,true,FAILED,VALIDATED,100.0,2,23,7 +Heap,ldv-regression/rule57_ebda_blast.i,ILP32,true,FAILED,VALIDATED,56.25,16,22,21 +ECA,eca-rers2012/Problem06_label46.c,ILP32,true,UNKNOWN,VALIDATED,,,0,53 +BitVectors,bitvector/parity.i,ILP32,true,SUCCEEDED,VALIDATED,77.78,9,33,37 +Sequentialized,seq-mthreaded/pals_lcr-var-start-time.3.ufo.BOUNDED-6.pals.c,ILP32,true,SUCCEEDED,VALIDATED,84.95,93,19,20 +LinkedLists,forester-heap/sll-rb-sentinel-2.i,ILP32,false,UNKNOWN,VALIDATED,76.0,25,3,48 +Loops,nla-digbench-scaling/cohendiv-ll_unwindbound1.c,ILP32,true,SUCCEEDED,VALIDATED,66.67,12,3,3 +ProductLines,product-lines/email_spec0_product24.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +ControlFlow,openssl-simplified/s3_clnt_3.cil-1.c,ILP32,unknown,UNKNOWN,VALIDATED,45.71,175,177,48 +Floats,neural-networks/cartpole_91_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-fpi/res1f.c,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Recursive,recursified_nla-digbench/recursified_geo2-ll2.c,ILP32,unknown,UNKNOWN,VALIDATED,57.14,7,17,49 +XCSP,xcsp/Dubois-065.c,ILP32,true,SUCCEEDED,VALIDATED,100.0,2,25,19 +Heap,list-ext-properties/simple-ext_1.i,ILP32,unknown,UNKNOWN,VALIDATED,69.23,13,29,49 +ECA,eca-rers2012/Problem09_label31.c,ILP32,true,ERROR,NO_SUITE,,,0,90 +BitVectors,bitvector/s3_clnt_3.BV.c.cil-1a.c,ILP32,true,UNKNOWN,VALIDATED,,,0,49 +Sequentialized,seq-mthreaded/pals_lcr.5_overflow.ufo.UNBOUNDED.pals.c,ILP32,false,SUCCEEDED,VALIDATED,45.0,120,27,25 +LinkedLists,heap-manipulation/tree-1.i,ILP32,unknown,FAILED,VALIDATED_ABORTS,76.32,38,47,14 +Loops,nla-digbench-scaling/fermat2-ll_valuebound10.c,ILP32,true,FAILED,VALIDATED_ABORTS,90.0,10,19,16 +ProductLines,product-lines/email_spec1_product28.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,3 +ControlFlow,openssl-simplified/s3_srvr_4.cil.c,ILP32,unknown,UNKNOWN,VALIDATED,52.28,197,163,49 +Floats,neural-networks/gcas_10_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-industry-pattern/array_of_struct_ptr_monotonic.i,ILP32,true,UNKNOWN,TESTCOV_ERROR,,,500,49 +Recursive,recursified_nla-digbench/recursified_ps4-ll.c,ILP32,unknown,UNKNOWN,VALIDATED,57.14,7,19,48 +XCSP,xcsp/Haystacks-17.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,100.0,2,25,25 +Heap,memsafety-cve/hyperkit_1.i,ILP32,unknown,FAILED,VALIDATED,19.23,26,33,30 +ECA,eca-rers2012/Problem12_label17.c,ILP32,true,UNKNOWN,VALIDATED,,,0,50 +BitVectors,bitvector/s3_srvr_3_alt.BV.c.cil.c,ILP32,unknown,UNKNOWN,VALIDATED,55.43,184,160,48 +Sequentialized,seq-mthreaded/pals_opt-floodmax.3.ufo.BOUNDED-6.pals.c,ILP32,true,FAILED,VALIDATED,,,0,48 +LinkedLists,list-properties/list-2.i,ILP32,false,FAILED,VALIDATED_ABORTS,82.35,17,32,5 +Loops,nla-digbench-scaling/hard-ll_unwindbound20.c,ILP32,false,SUCCEEDED,VALIDATED,100.0,14,10,3 +ProductLines,product-lines/email_spec3_product35.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +ControlFlow,openssl/s3_srvr.blast.02.i.cil-1.c,ILP32,unknown,FAILED,VALIDATED,15.92,201,152,49 +Floats,neural-networks/lunarlander_99_unsafe.c-amalgamation.i,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-memsafety/stroeder2-alloca-1.i,ILP32,unknown,ERROR,VALIDATED,100.0,8,178,49 +Recursive,recursive-simple/id_o100.c,ILP32,false,FAILED,VALIDATED,75.0,4,74,49 +XCSP,xcsp/aim-100-3-4-sat-1.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,100.0,2,23,11 +Heap,memsafety-cve/radare2Fixed.i,ILP32,unknown,UNKNOWN,VALIDATED,8.33,72,1,49 +ECA,eca-rers2012/Problem16_label04.c,ILP32,false,UNKNOWN,VALIDATED,17.9,391,7,49 +BitVectors,bitvector/soft_float_3.c.cil.c,ILP32,unknown,UNKNOWN,TESTCOV_ERROR,,,500,51 +Sequentialized,seq-pthread/cs_dekker_unfair.i,ILP32,true,UNKNOWN,VALIDATED,,,0,49 +LinkedLists,memsafety-broom/linux-hlist-fst-data.i,LP64,unknown,UNKNOWN,VALIDATED,100.0,18,12,49 +Loops,nla-digbench-scaling/ps2-ll_valuebound1.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,87.5,8,4,3 +ProductLines,product-lines/email_spec7_product24.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,42 +ControlFlow,openssl/s3_srvr.blast.12.i.cil-1.c,ILP32,unknown,FAILED,TESTCOV_ERROR,,,140,49 +Floats,neural-networks/softplus_0_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-tiling/mlceu.c,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Recursive,recursive/EvenOdd03WithOverflowBug.c,ILP32,unknown,UNKNOWN,VALIDATED,75.0,8,11,48 +XCSP,xcsp/aim-200-2-0-sat-3.c,ILP32,false,SUCCEEDED,VALIDATED,100.0,2,25,17 +Heap,memsafety/lockfree-3.0.i,ILP32,unknown,UNKNOWN,TESTCOV_ERROR,,,444,49 +ECA,eca-rers2012/Problem18_label49.c,ILP32,false,UNKNOWN,VALIDATED,,,0,50 +BitVectors,bitvector/sum02-2.c,ILP32,true,UNKNOWN,VALIDATED,66.67,6,54,48 +Sequentialized,seq-pthread/cs_szymanski.i,ILP32,true,UNKNOWN,TESTCOV_ERROR,,,390,49 +LinkedLists,memsafety-broom/sll-shared-after.i,LP64,unknown,UNKNOWN,VALIDATED,100.0,6,22,48 +Recursive,recursive/recHanoi01.c,ILP32,true,UNKNOWN,VALIDATED_ABORTS,87.5,8,33,51 +XCSP,xcsp/aim-200-6-0-sat-3.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,100.0,2,25,17 +Loops,loop-acceleration/array_3-2.i,ILP32,false,FAILED,VALIDATED,71.43,7,170,49 +ProductLines,product-lines/elevator_spec14_product11.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,50 +Heap,heap-data/quick_sort_split.i,ILP32,true,FAILED,TESTCOV_ERROR,,,241,48 +ECA,eca-programs/Problem101_label23.c,ILP32,false,ERROR,NO_SUITE,,,0,90 +ControlFlow,infeasible-control-flow/unreach_branch5.c,ILP32,true,UNKNOWN,VALIDATED,,,0,49 +Floats,float-benchs/inv_Newton-2.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,16.67,6,1,2 +Arrays,array-crafted/mapsum2.i,ILP32,true,SUCCEEDED,VALIDATED,77.78,9,1,15 +Sequentialized,seq-mthreaded/pals_floodmax.3.2.ufo.BOUNDED-6.pals.c,ILP32,false,UNKNOWN,VALIDATED,,,0,49 +LinkedLists,forester-heap/dll-rb-cnstr_1-2.i,ILP32,false,FAILED,TESTCOV_ERROR,,,148,30 +Recursive,fuzzle-programs/09_fuzzle_50x50_75-cycle.c,LP64,false,UNKNOWN,VALIDATED,,,0,54 +XCSP,xcsp/AllInterval-014.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,7.14,28,21,3 +Loops,loop-lit/cggmp2005b.i,ILP32,true,SUCCEEDED,VALIDATED,,,0,1 +ProductLines,product-lines/elevator_spec1_product23.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Heap,ldv-memsafety/memleaks_test15-2.i,ILP32,unknown,UNKNOWN,VALIDATED,34.0,50,3,48 +ECA,eca-rers2012/Problem03_label00.c,ILP32,true,UNKNOWN,VALIDATED,3.04,296,7,49 +ControlFlow,ntdrivers/diskperf.i.cil-1.c,ILP32,true,FAILED,VALIDATED,,,0,49 +Floats,float-newlib/double_req_bl_1122a.c,ILP32,true,SUCCEEDED,VALIDATED,6.78,59,2,10 +Arrays,array-examples/sanfoundry_43_ground.i,ILP32,true,UNKNOWN,VALIDATED,11.11,9,1,48 +Sequentialized,seq-mthreaded/pals_floodmax.4.4.ufo.BOUNDED-8.pals.c,ILP32,false,UNKNOWN,VALIDATED,,,0,48 +LinkedLists,forester-heap/sll-buckets-2.i,ILP32,false,UNKNOWN,TESTCOV_ERROR,,,371,47 +Recursive,recursified_loop-crafted/recursified_simple_array_index_value_4.i,ILP32,unknown,UNKNOWN,VALIDATED,,,0,47 +XCSP,xcsp/CostasArray-11.c,ILP32,false,FAILED,VALIDATED,100.0,2,23,8 +Loops,loop-zilu/benchmark23_conjunctive.i,ILP32,true,SUCCEEDED,VALIDATED,71.43,7,3,2 +ProductLines,product-lines/elevator_spec2_product23.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,5 +Heap,ldv-memsafety/memleaks_test22_2-1.i,ILP32,unknown,FAILED,VALIDATED,18.75,32,21,22 +ECA,eca-rers2012/Problem04_label31.c,ILP32,false,UNKNOWN,VALIDATED,,,0,50 +ControlFlow,openssl-simplified/s3_clnt_3.cil-2.c,ILP32,unknown,UNKNOWN,VALIDATED,46.29,175,194,48 +Floats,floats-cbmc-regression/float19.i,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-examples/standard_partition_original_ground.i,ILP32,true,UNKNOWN,VALIDATED,,,0,48 +Sequentialized,seq-mthreaded/pals_lcr-var-start-time.4.1.ufo.BOUNDED-8.pals.c,ILP32,false,SUCCEEDED,VALIDATED,70.73,123,25,24 +LinkedLists,forester-heap/sll-simple-white-blue-2.i,ILP32,false,UNKNOWN,VALIDATED,94.44,18,47,49 +Recursive,recursified_nla-digbench/recursified_bresenham-ll.c,ILP32,unknown,UNKNOWN,VALIDATED,66.67,9,45,48 +XCSP,xcsp/Dubois-019.c,ILP32,true,FAILED,VALIDATED,100.0,2,23,6 +Loops,loops-crafted-1/sumt8.c,ILP32,true,UNKNOWN,VALIDATED,81.25,16,51,49 +ProductLines,product-lines/elevator_spec3_product22.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Heap,ldv-memsafety/memleaks_test8.i,ILP32,unknown,SUCCEEDED,VALIDATED,5.88,34,1,12 +ECA,eca-rers2012/Problem06_label05.c,ILP32,false,UNKNOWN,VALIDATED,,,0,53 +ControlFlow,openssl-simplified/s3_srvr_2.cil-2.c,ILP32,unknown,UNKNOWN,VALIDATED,52.28,197,172,49 +Floats,floats-nonlinear/coral-benchmark21.c,ILP32,false,SUCCEEDED,VALIDATED,50.0,4,1,1 +Arrays,array-fpi/condg.c,ILP32,true,UNKNOWN,VALIDATED,,,0,46 +Sequentialized,seq-mthreaded/pals_lcr.3_overflow.ufo.UNBOUNDED.pals.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,81.94,72,17,14 +LinkedLists,heap-manipulation/sll_to_dll_rev-2.i,ILP32,true,UNKNOWN,VALIDATED,78.72,47,108,48 +Recursive,recursified_nla-digbench/recursified_egcd.c,ILP32,unknown,SUCCEEDED,VALIDATED_ABORTS,88.89,9,17,11 +XCSP,xcsp/Dubois-029.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,100.0,2,23,9 +Loops,loops/vogal-1.i,ILP32,true,UNKNOWN,VALIDATED,82.61,23,122,48 +ProductLines,product-lines/elevator_spec9_product32.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Heap,ldv-regression/test21-2.c,ILP32,false,SUCCEEDED,VALIDATED,100.0,6,5,6 +ECA,eca-rers2012/Problem07_label27.c,ILP32,true,UNKNOWN,VALIDATED,,,0,81 +ControlFlow,openssl/s3_srvr.blast.01.i.cil-1.c,ILP32,unknown,FAILED,VALIDATED,,,0,48 +Floats,neural-networks/cartpole_60_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-fpi/indp5f.c,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Sequentialized,seq-mthreaded/pals_lcr.8.1.ufo.BOUNDED-16.pals.c,ILP32,false,SUCCEEDED,TESTCOV_ERROR,,,29,23 +LinkedLists,list-ext3-properties/sll_of_sll_nondet_append-2.i,ILP32,false,UNKNOWN,VALIDATED_ABORTS,90.32,31,63,47 +Recursive,recursified_nla-digbench/recursified_geo1-ll2.c,ILP32,unknown,SUCCEEDED,VALIDATED,,,0,57 +XCSP,xcsp/Dubois-080.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,100.0,2,25,16 +Loops,nla-digbench-scaling/divbin2_unwindbound100.i,ILP32,true,UNKNOWN,VALIDATED,73.33,15,500,48 +ProductLines,product-lines/email_spec0_product40.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,4 +Heap,ldv-sets/test_mutex_unbounded-2.i,ILP32,false,UNKNOWN,VALIDATED_ABORTS,51.85,27,11,48 +ECA,eca-rers2012/Problem08_label50.c,ILP32,false,ERROR,NO_SUITE,,,0,88 +ControlFlow,openssl/s3_srvr.blast.09.i.cil-2.c,ILP32,unknown,FAILED,VALIDATED,15.17,211,209,47 +Floats,neural-networks/dubinsrejoin_23_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,3 +Arrays,array-fpi/s1lif.c,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Sequentialized,seq-mthreaded/pals_opt-floodmax.4.2.ufo.BOUNDED-8.pals.c,ILP32,false,UNKNOWN,TESTCOV_ERROR,,,500,48 +LinkedLists,memsafety-broom/dll-fst-data.i,LP64,unknown,UNKNOWN,VALIDATED,100.0,6,17,47 +Recursive,recursified_nla-digbench/recursified_hard-ll.c,ILP32,unknown,SUCCEEDED,VALIDATED_ABORTS,91.67,12,14,9 +XCSP,xcsp/Haystacks-12.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,100.0,2,24,11 +Loops,nla-digbench-scaling/egcd3-ll_unwindbound50.c,ILP32,false,SUCCEEDED,VALIDATED,100.0,16,18,12 +ProductLines,product-lines/email_spec11_product39.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Heap,list-ext2-properties/simple_and_skiplist_2lvl-2.i,ILP32,true,FAILED,VALIDATED,80.95,21,173,36 +ECA,eca-rers2012/Problem10_label13.c,ILP32,true,UNKNOWN,VALIDATED,34.33,201,6,50 +ControlFlow,unsignedintegeroverflow-sas23/bilinear_interpolation.i,ILP32,true,SUCCEEDED,VALIDATED,,,0,57 +Floats,neural-networks/dubinsrejoin_84_unsafe.c-amalgamation.i,ILP32,false,UNKNOWN,VALIDATED,,,0,2 +Arrays,array-fpi/sina2.c,ILP32,true,UNKNOWN,VALIDATED,,,0,8 +Sequentialized,seq-mthreaded/pals_opt-floodmax.5.ufo.BOUNDED-10.pals.c,ILP32,true,UNKNOWN,TESTCOV_ERROR,,,297,48 +LinkedLists,memsafety-broom/sll-fst-data.i,LP64,unknown,UNKNOWN,VALIDATED,100.0,6,23,48 +Recursive,recursified_nla-digbench/recursified_prodbin.c,ILP32,unknown,SUCCEEDED,VALIDATED,11.11,9,1,3 +XCSP,xcsp/aim-100-1-6-sat-3.c,ILP32,false,SUCCEEDED,VALIDATED,100.0,2,23,8 +Recursive,recursified_nla-digbench/recursified_ps2.c,ILP32,unknown,UNKNOWN,VALIDATED,57.14,7,21,47 +XCSP,xcsp/aim-100-1-6-unsat-2.c,ILP32,true,SUCCEEDED,VALIDATED,100.0,2,23,10 +Recursive,recursified_nla-digbench/recursified_ps3.c,ILP32,unknown,UNKNOWN,VALIDATED,57.14,7,10,48 +XCSP,xcsp/aim-100-1-6-unsat-4.c,ILP32,true,SUCCEEDED,VALIDATED,100.0,2,23,6 +Recursive,recursified_nla-digbench/recursified_ps4.c,ILP32,unknown,UNKNOWN,VALIDATED,57.14,7,18,47 +XCSP,xcsp/aim-100-2-0-sat-2.c,ILP32,false,SUCCEEDED,VALIDATED,100.0,2,23,10 +Recursive,recursified_nla-digbench/recursified_ps5.c,ILP32,unknown,UNKNOWN,VALIDATED,57.14,7,19,48 +XCSP,xcsp/aim-100-2-0-sat-4.c,ILP32,false,SUCCEEDED,VALIDATED,100.0,2,23,9 +Recursive,recursified_nla-digbench/recursified_ps6.c,ILP32,unknown,UNKNOWN,VALIDATED,57.14,7,11,47 +XCSP,xcsp/aim-100-2-0-unsat-2.c,ILP32,true,SUCCEEDED,VALIDATED,100.0,2,23,9 +Recursive,recursified_nla-digbench/recursified_sqrt1.c,ILP32,unknown,UNKNOWN,VALIDATED,57.14,7,19,47 +XCSP,xcsp/aim-100-2-0-unsat-4.c,ILP32,true,SUCCEEDED,VALIDATED,100.0,2,23,10 +Recursive,recursive-simple/id2_b3_o2.c,ILP32,false,UNKNOWN,VALIDATED,100.0,10,62,48 +XCSP,xcsp/aim-100-3-4-sat-2.c,ILP32,false,SUCCEEDED,VALIDATED,100.0,2,23,11 +Sequentialized,seq-pthread/cs_time_var_mutex.i,ILP32,true,UNKNOWN,TESTCOV_ERROR,,,437,48 +Arrays,array-memsafety/mult_array_unsafe.i,ILP32,unknown,FAILED,VALIDATED,90.0,10,122,33 +Floats,neural-networks/lunarlander_81_unsafe.c-amalgamation.i,ILP32,false,UNKNOWN,VALIDATED,,,0,2 +Heap,memsafety-cve/mongooseFixed.i,ILP32,unknown,UNKNOWN,VALIDATED,11.54,26,1,48 +ECA,eca-rers2012/Problem13_label48.c,ILP32,false,UNKNOWN,VALIDATED,,,0,50 +Loops,nla-digbench-scaling/hard2_unwindbound50.c,ILP32,unknown,UNKNOWN,VALIDATED_ABORTS,85.71,14,130,48 +XCSP,xcsp/aim-200-1-6-sat-3.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,100.0,2,25,17 +Floats,neural-networks/poly_16_16_16_16_thresh_0_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,2 +Heap,memsafety-cve/openrazerFound.i,ILP32,unknown,FAILED,VALIDATED,36.67,30,2,48 +ProductLines,product-lines/email_spec4_productSimulator.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Recursive,recursive-simple/id_o3.c,ILP32,false,UNKNOWN,VALIDATED,100.0,4,35,47 +Arrays,array-multidimensional/min-2-u.c,ILP32,true,UNKNOWN,VALIDATED,0,0,1,48 +ECA,eca-rers2012/Problem16_label12.c,ILP32,true,UNKNOWN,VALIDATED,17.9,391,7,48 +Loops,nla-digbench-scaling/prod4br-ll_unwindbound5.c,ILP32,false,SUCCEEDED,VALIDATED,,,0,57 +XCSP,xcsp/aim-200-2-0-unsat-2.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,100.0,2,25,25 +Floats,neural-networks/poly_8_8_8_8_thresh_1_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,2 +Heap,memsafety-cve/wasm-micro-runtime.i,ILP32,unknown,UNKNOWN,TESTCOV_ERROR,,,500,57 +ProductLines,product-lines/email_spec7_product19.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,15 +Recursive,recursive/Addition02.c,ILP32,false,UNKNOWN,VALIDATED_ABORTS,75.0,12,22,48 +Arrays,array-patterns/array7_pattern.c,ILP32,true,SUCCEEDED,VALIDATED,91.67,12,11,6 +ECA,eca-rers2012/Problem17_label35.c,ILP32,false,UNKNOWN,VALIDATED,,,0,48 +Loops,nla-digbench-scaling/ps4-ll_valuebound100.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,87.5,8,10,3 +XCSP,xcsp/aim-200-6-0-sat-4.c,ILP32,false,SUCCEEDED,VALIDATED,100.0,2,25,15 +Floats,neural-networks/softsign_w32_r1_case_0_unsafe.c-amalgamation.i,ILP32,false,UNKNOWN,VALIDATED,,,0,2 +Loops,nla-digbench-scaling/ps6-ll_unwindbound5.c,ILP32,false,SUCCEEDED,VALIDATED,100.0,8,8,2 +Arrays,array-tiling/pr2.c,ILP32,true,FAILED,VALIDATED_ABORTS,75.0,16,35,16 +ECA,eca-rers2012/Problem18_label41.c,ILP32,true,UNKNOWN,VALIDATED,,,0,48 diff --git a/tests/testcomp/results_cover-branches_s4/results.csv b/tests/testcomp/results_cover-branches_s4/results.csv new file mode 100644 index 000000000..2c808ba7b --- /dev/null +++ b/tests/testcomp/results_cover-branches_s4/results.csv @@ -0,0 +1,244 @@ +category,program,data_model,expected_unreach,verdict,testcov,coverage,goals,n_tests,elapsed_s +Loops,nla-digbench-scaling/fermat1-ll_valuebound2.c,ILP32,true,SUCCEEDED,VALIDATED,37.5,16,7,5 +ProductLines,product-lines/email_spec1_product20.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,2 +ControlFlow,ntdrivers/diskperf.i.cil-2.c,ILP32,false,FAILED,VALIDATED,14.63,287,21,48 +Floats,floats-esbmc-regression/ceil_nondet.i,ILP32,true,UNKNOWN,VALIDATED,35.29,17,1,1 +Arrays,array-multidimensional/copy-partial-2-n-u.c,ILP32,true,UNKNOWN,VALIDATED,,,0,47 +Recursive,recursive-simple/id_o200.c,ILP32,false,FAILED,VALIDATED,75.0,4,65,49 +XCSP,xcsp/aim-100-3-4-sat-3.c,ILP32,false,SUCCEEDED,VALIDATED,100.0,2,23,13 +Heap,ldv-memsafety/memleaks_test17_2-2.i,ILP32,unknown,SUCCEEDED,VALIDATED_ABORTS,36.96,46,8,58 +ECA,eca-rers2012/Problem03_label25.c,ILP32,true,UNKNOWN,VALIDATED,3.04,296,7,49 +BitVectors,bitvector/s3_clnt_1.BV.c.cil-1.c,ILP32,unknown,FAILED,VALIDATED,46.29,175,178,48 +Sequentialized,seq-mthreaded/pals_lcr-var-start-time.5.ufo.BOUNDED-10.pals.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,37.42,155,28,37 +LinkedLists,forester-heap/sll-token-1.i,ILP32,false,UNKNOWN,VALIDATED_ABORTS,24.0,25,1,48 +Loops,nla-digbench-scaling/geo2-ll_unwindbound50.c,ILP32,true,SUCCEEDED,VALIDATED,62.5,8,11,58 +ProductLines,product-lines/email_spec3_product19.cil.c,ILP32,false,ERROR,NO_SUITE,,,0,89 +ControlFlow,openssl/s3_srvr.blast.13.i.cil-1.c,ILP32,unknown,FAILED,VALIDATED,,,0,47 +Floats,neural-networks/softsign_w16_r2_case_0_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,2 +Arrays,array-crafted/xor1.i,ILP32,true,SUCCEEDED,VALIDATED,77.78,9,1,5 +Recursive,fuzzle-programs/08_fuzzle_50x50_50-cycle.c,LP64,false,UNKNOWN,VALIDATED,,,0,54 +XCSP,xcsp/AllInterval-011.c,ILP32,false,FAILED,VALIDATED_ABORTS,9.09,22,21,7 +Heap,ldv-memsafety/memleaks_test22_3-2.i,ILP32,unknown,FAILED,VALIDATED,34.38,32,21,41 +ECA,eca-rers2012/Problem04_label47.c,ILP32,true,UNKNOWN,VALIDATED,,,0,53 +BitVectors,bitvector/modulus-1.i,ILP32,unknown,UNKNOWN,VALIDATED_ABORTS,84.62,13,40,49 +Sequentialized,seq-mthreaded/pals_lcr-var-start-time.3.2.ufo.BOUNDED-6.pals.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,85.56,90,19,20 +LinkedLists,forester-heap/sll-rb-sentinel-1.i,ILP32,true,UNKNOWN,VALIDATED,80.0,25,3,48 +Loops,nla-digbench-scaling/egcd-ll_unwindbound20.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,100.0,10,17,5 +ProductLines,product-lines/email_spec11_product23.cil.c,ILP32,true,ERROR,NO_SUITE,,,0,90 +ControlFlow,openssl-simplified/s3_srvr_6.cil-2.c,ILP32,unknown,FAILED,VALIDATED,46.75,231,154,49 +Floats,neural-networks/gelu_3_unsafe.c-amalgamation.i,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-memsafety/openbsd_cmemset-alloca-1.i,ILP32,unknown,SUCCEEDED,VALIDATED,85.71,7,5,5 +Recursive,recursive-simple/id_b3_o2-2.c,ILP32,false,UNKNOWN,VALIDATED_ABORTS,100.0,6,58,49 +XCSP,xcsp/aim-100-2-0-sat-3.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,100.0,2,23,12 +Heap,memsafety-cve/stb.i,ILP32,unknown,UNKNOWN,TESTCOV_ERROR,,,500,49 +ECA,eca-rers2012/Problem16_label54.c,ILP32,false,UNKNOWN,VALIDATED,17.9,391,7,49 +BitVectors,bitvector/soft_float_5.c.cil.c,ILP32,unknown,FAILED,TESTCOV_ERROR,,,500,50 +Sequentialized,seq-pthread/cs_sync.i,ILP32,true,UNKNOWN,VALIDATED,4.59,109,203,49 +LinkedLists,memsafety-broom/sll-nested-sll.i,LP64,unknown,UNKNOWN,VALIDATED,100.0,12,137,48 +Loops,loop-industry-pattern/aiob_2.c,ILP32,true,SUCCEEDED,VALIDATED,,,0,7 +ProductLines,product-lines/elevator_spec14_product28.cil.c,ILP32,false,ERROR,NO_SUITE,,,0,90 +ControlFlow,memory-model/2SB.i,ILP32,false,ERROR,VALIDATED_ABORTS,48.04,102,21,9 +Floats,float-newlib/double_req_bl_0670.c,ILP32,true,SUCCEEDED,VALIDATED,1.41,71,1,8 +Arrays,array-examples/standard_copy2_ground-1.i,ILP32,false,UNKNOWN,VALIDATED,9.09,11,1,49 +Recursive,recursified_loop-invariants/recursified_linear-inequality-inv-a.c,ILP32,unknown,UNKNOWN,VALIDATED,75.0,8,10,48 +XCSP,xcsp/AllInterval-030.c,ILP32,false,FAILED,VALIDATED,3.33,60,23,8 +Heap,ldv-memsafety/memleaks_test4-1.i,ILP32,unknown,FAILED,VALIDATED,11.76,34,41,32 +ECA,eca-rers2012/Problem05_label23.c,ILP32,true,UNKNOWN,VALIDATED,,,0,54 +BitVectors,bitvector/jain_6-2.c,ILP32,unknown,UNKNOWN,VALIDATED,66.67,3,1,48 +Sequentialized,seq-mthreaded/pals_floodmax.5.ufo.BOUNDED-10.pals.c,ILP32,true,UNKNOWN,TESTCOV_ERROR,,,207,49 +LinkedLists,forester-heap/sll-queue-2.i,ILP32,false,UNKNOWN,VALIDATED,72.88,59,9,49 +Loops,nla-digbench-scaling/cohencu-ll_unwindbound100.c,ILP32,false,UNKNOWN,VALIDATED,75.0,8,26,48 +ProductLines,product-lines/email_spec0_product16.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,2 +ControlFlow,openssl-simplified/s3_clnt_3.cil-3.c,ILP32,unknown,UNKNOWN,VALIDATED,,,0,49 +Floats,neural-networks/cos_3_unsafe.c-amalgamation.i,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-fpi/s3lif.c,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Recursive,recursified_nla-digbench/recursified_hard-u.c,ILP32,unknown,SUCCEEDED,VALIDATED,100.0,12,15,14 +XCSP,xcsp/Dubois-095.c,ILP32,true,SUCCEEDED,VALIDATED,100.0,2,25,30 +Heap,memsafety-cve/dlt-daemon.i,ILP32,unknown,SUCCEEDED,VALIDATED,,,0,8 +ECA,eca-rers2012/Problem10_label55.c,ILP32,false,UNKNOWN,VALIDATED,,,0,49 +BitVectors,bitvector/s3_srvr_1a.BV.c.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,49 +Sequentialized,seq-mthreaded/pals_opt-floodmax.3.2.ufo.BOUNDED-6.pals.c,ILP32,false,UNKNOWN,TESTCOV_ERROR,,,198,49 +LinkedLists,list-ext3-properties/sll_of_sll_nondet_append-1.i,ILP32,true,UNKNOWN,VALIDATED,,,0,48 +Loops,nla-digbench-scaling/geo3-ll_unwindbound50.c,ILP32,true,UNKNOWN,VALIDATED,62.5,8,11,48 +ProductLines,product-lines/email_spec3_product31.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,34 +ControlFlow,openssl/s3_srvr.blast.03.i.cil.c,ILP32,unknown,FAILED,VALIDATED,9.45,201,142,49 +Floats,neural-networks/poly_128_thresh_0_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,2 +Arrays,array-multidimensional/transpose-u.c,ILP32,true,UNKNOWN,VALIDATED,21.43,14,1,48 +Recursive,recursive-simple/sum_non_eq-2.c,ILP32,true,FAILED,VALIDATED,75.0,4,56,49 +XCSP,xcsp/aim-100-6-0-sat-2.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,100.0,2,23,12 +Heap,memsafety-ext/dll_extends_pointer.i,ILP32,unknown,UNKNOWN,VALIDATED,88.89,9,191,48 +ECA,eca-rers2012/Problem17_label27.c,ILP32,true,UNKNOWN,VALIDATED,,,0,50 +BitVectors,bitvector/soft_float_4-3.c.cil.c,ILP32,unknown,UNKNOWN,TESTCOV_ERROR,,,500,50 +Sequentialized,seq-pthread/cs_stack-1.i,ILP32,true,UNKNOWN,VALIDATED,4.46,157,203,49 +LinkedLists,memsafety-broom/sll-nested-linux-list.i,LP64,unknown,UNKNOWN,VALIDATED,,,0,48 +Loops,nla-digbench/hard-ll.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,92.86,14,10,3 +ProductLines,product-lines/email_spec9_product28.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,15 +ControlFlow,infeasible-control-flow/conflict_branch1.c,ILP32,true,UNKNOWN,TESTCOV_ERROR,,,500,50 +Floats,float-benchs/filter1.c.v+lhb-reducer.c,ILP32,true,UNKNOWN,TESTCOV_ERROR,,,500,49 +Arrays,array-crafted/mapavg3.i,ILP32,true,UNKNOWN,VALIDATED,11.11,9,1,48 +Recursive,fuzzle-programs/06_fuzzle_50x50_0-cycle.c,LP64,false,UNKNOWN,VALIDATED,,,0,53 +XCSP,xcsp/AllInterval-009.c,ILP32,false,FAILED,VALIDATED,11.11,18,20,4 +Heap,ldv-memsafety/memleaks_test13_1.i,ILP32,unknown,FAILED,VALIDATED,11.76,34,21,24 +ECA,eca-programs/Problem102_label57.c,ILP32,true,ERROR,NO_SUITE,,,0,90 +BitVectors,bitvector/byte_add_2-1.i,ILP32,unknown,UNKNOWN,VALIDATED,,,0,48 +Sequentialized,seq-mthreaded/pals_floodmax.3.2.ufo.UNBOUNDED.pals.c,ILP32,false,FAILED,VALIDATED,20.32,187,256,48 +LinkedLists,forester-heap/dll-rb-sentinel-1.i,ILP32,false,FAILED,VALIDATED,73.53,34,40,5 +Loops,loop-zilu/benchmark06_conjunctive.i,ILP32,unknown,UNKNOWN,VALIDATED,77.78,9,224,49 +ProductLines,product-lines/elevator_spec2_product17.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +ControlFlow,ntdrivers-simplified/floppy_simpl4.cil-1.c,ILP32,unknown,UNKNOWN,VALIDATED,54.34,265,179,48 +Floats,float-newlib/float_req_bl_1130b.c,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-examples/standard_palindrome_ground.i,ILP32,true,UNKNOWN,VALIDATED,11.11,9,1,48 +Recursive,recursified_loop-simple/recursified_nested_4.c,ILP32,unknown,UNKNOWN,VALIDATED,,,0,48 +XCSP,xcsp/CostasArray-14.c,ILP32,false,SUCCEEDED,VALIDATED,100.0,2,23,10 +Heap,ldv-memsafety/memleaks_test5_2.i,ILP32,unknown,FAILED,VALIDATED,8.82,34,21,21 +ECA,eca-rers2012/Problem05_label40.c,ILP32,false,UNKNOWN,VALIDATED,,,0,55 +BitVectors,bitvector/jain_7-1.c,ILP32,unknown,UNKNOWN,VALIDATED,66.67,3,1,49 +Sequentialized,seq-mthreaded/pals_floodmax.5.3.ufo.BOUNDED-10.pals.c,ILP32,false,UNKNOWN,VALIDATED,,,0,48 +LinkedLists,forester-heap/sll-optional-2.i,ILP32,false,UNKNOWN,TESTCOV_ERROR,,,464,49 +Loops,loops/terminator_01.c,ILP32,false,UNKNOWN,VALIDATED,50.0,4,1,48 +ProductLines,product-lines/elevator_spec9_product26.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,2 +ControlFlow,ntdrivers/parport.i.cil-1.c,ILP32,false,UNKNOWN,VALIDATED,,,0,50 +Floats,neural-networks/cartpole_43_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-fpi/indp3f.c,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Recursive,recursified_nla-digbench/recursified_fermat2-ll.c,ILP32,unknown,SUCCEEDED,VALIDATED,,,0,59 +XCSP,xcsp/Dubois-030.c,ILP32,true,SUCCEEDED,VALIDATED,100.0,2,23,11 +Heap,ldv-sets/test_add-1.i,ILP32,false,UNKNOWN,VALIDATED,41.18,17,1,2 +ECA,eca-rers2012/Problem08_label25.c,ILP32,false,ERROR,NO_SUITE,,,0,90 +BitVectors,bitvector/s3_clnt_2.BV.c.cil-1a.c,ILP32,true,FAILED,VALIDATED,,,0,49 +Sequentialized,seq-mthreaded/pals_lcr.3.ufo.BOUNDED-6.pals.c,ILP32,true,SUCCEEDED,VALIDATED,79.45,73,17,15 +LinkedLists,heap-manipulation/dancing.i,ILP32,true,FAILED,VALIDATED,86.96,23,185,41 +Loops,nla-digbench-scaling/egcd2-ll_unwindbound100.c,ILP32,false,SUCCEEDED,VALIDATED,100.0,12,9,7 +ProductLines,product-lines/email_spec11_product30.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,2 +ControlFlow,openssl-simplified/s3_srvr_13.cil.c,ILP32,unknown,FAILED,VALIDATED,37.9,219,212,49 +Floats,neural-networks/dubinsrejoin_66_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,2 +Arrays,array-fpi/s5liff.c,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Recursive,recursified_nla-digbench/recursified_prod4br-ll.c,ILP32,unknown,SUCCEEDED,VALIDATED,,,0,59 +XCSP,xcsp/Haystacks-09.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,100.0,2,23,9 +Heap,memsafety-cve/frrFixed.i,ILP32,unknown,UNKNOWN,TESTCOV_ERROR,,,500,49 +ECA,eca-rers2012/Problem11_label11.c,ILP32,true,UNKNOWN,VALIDATED,,,0,48 +BitVectors,bitvector/s3_srvr_1a_alt.BV.c.cil.c,ILP32,unknown,SUCCEEDED,VALIDATED_ABORTS,52.26,199,160,45 +Sequentialized,seq-mthreaded/pals_lcr.8_overflow.ufo.UNBOUNDED.pals.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,31.88,207,32,36 +LinkedLists,list-ext3-properties/sll_nondet_insert-1.i,ILP32,false,UNKNOWN,VALIDATED_ABORTS,90.48,21,110,49 +Loops,nla-digbench-scaling/geo2-ll2_unwindbound20.c,ILP32,unknown,SUCCEEDED,VALIDATED,62.5,8,21,15 +ProductLines,product-lines/email_spec27_product35.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,54 +ControlFlow,openssl/s3_clnt.blast.03.i.cil-1.c,ILP32,unknown,UNKNOWN,TESTCOV_ERROR,,,500,50 +Floats,neural-networks/lunarlander_51_unsafe.c-amalgamation.i,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-memsafety/knapsack_alloca_unsafe.i,ILP32,unknown,FAILED,VALIDATED_ABORTS,31.25,16,7,58 +Recursive,recursive-simple/id_b2_o3.c,ILP32,true,UNKNOWN,VALIDATED,83.33,6,60,49 +XCSP,xcsp/aim-100-2-0-sat-1.c,ILP32,false,SUCCEEDED,VALIDATED,100.0,2,23,10 +Heap,memsafety-cve/mujs.i,ILP32,unknown,FAILED,TESTCOV_ERROR,,,500,50 +ECA,eca-rers2012/Problem13_label56.c,ILP32,true,UNKNOWN,VALIDATED,,,0,51 +BitVectors,bitvector/soft_float_1-3.c.cil.c,ILP32,unknown,UNKNOWN,TESTCOV_ERROR,,,500,49 +Sequentialized,seq-mthreaded/pals_opt-floodmax.5.2.ufo.BOUNDED-10.pals.c,ILP32,false,UNKNOWN,TESTCOV_ERROR,,,175,48 +LinkedLists,memsafety-broom/dll-fst-shared.i,LP64,unknown,UNKNOWN,VALIDATED,87.5,8,15,49 +Loops,nla-digbench-scaling/lcm2_valuebound10.c,ILP32,true,SUCCEEDED,VALIDATED,90.0,10,35,4 +ProductLines,product-lines/email_spec6_product22.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +ControlFlow,openssl/s3_srvr.blast.09.i.cil-1.c,ILP32,unknown,FAILED,VALIDATED,15.17,211,150,49 +Floats,neural-networks/poly_64_64_thresh_0_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-patterns/array29_pattern.c,ILP32,true,FAILED,VALIDATED,92.86,14,4,12 +Recursive,recursive/Addition01-2.c,ILP32,true,UNKNOWN,VALIDATED,66.67,12,60,49 +XCSP,xcsp/aim-200-1-6-sat-4.c,ILP32,false,SUCCEEDED,VALIDATED,100.0,2,25,18 +Heap,memsafety-ext2/complex_data_creation_test01-1.i,ILP32,unknown,FAILED,VALIDATED,85.71,14,277,49 +ECA,eca-rers2012/Problem17_label43.c,ILP32,true,UNKNOWN,VALIDATED,,,0,50 +BitVectors,bitvector/soft_float_4-3a.c.cil.c,ILP32,false,UNKNOWN,TESTCOV_ERROR,,,500,49 +Sequentialized,seq-pthread/cs_read_write_lock-1.i,ILP32,true,UNKNOWN,TESTCOV_ERROR,,,429,48 +LinkedLists,memsafety-broom/sll-middle-data.i,LP64,unknown,UNKNOWN,VALIDATED,100.0,6,19,48 +Loops,nla-digbench-scaling/sqrt1-ll_valuebound100.c,ILP32,true,FAILED,VALIDATED,87.5,8,8,2 +ProductLines,product-lines/email_spec8_product33.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,2 +ControlFlow,openssl/s3_srvr.blast.16.i.cil-2.c,ILP32,unknown,FAILED,TESTCOV_ERROR,,,151,49 +Floats,neural-networks/tanh_w32_r2_case_1_unsafe.c-amalgamation.i,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Arrays,reducercommutativity/sep40-1.i,ILP32,true,UNKNOWN,TESTCOV_ERROR,,,500,51 +Sequentialized,systemc/token_ring.02.cil-1.c,ILP32,unknown,UNKNOWN,VALIDATED,70.48,105,156,49 +LinkedLists,memsafety-broom/sll-shared-sll.i,LP64,unknown,UNKNOWN,VALIDATED,91.67,12,58,49 +Recursive,fuzzle-programs/01_fuzzle_30x30.c,LP64,false,UNKNOWN,VALIDATED,,,0,50 +XCSP,xcsp/AllInterval-006.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,16.67,12,19,3 +Loops,loop-acceleration/simple_4-2_abstracted.c,ILP32,true,SUCCEEDED,VALIDATED,66.67,6,2,1 +ProductLines,product-lines/elevator_spec14_product24.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Heap,ldv-memsafety/memleaks_test1-1.i,ILP32,unknown,FAILED,VALIDATED,6.25,32,21,18 +ECA,eca-programs/Problem102_label16.c,ILP32,true,ERROR,NO_SUITE,,,0,89 +ControlFlow,ntdrivers-simplified/cdaudio_simpl1.cil-1.c,ILP32,unknown,UNKNOWN,VALIDATED,39.4,368,99,49 +Floats,float-benchs/rlim_invariant.c.v+lhb-reducer.c,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-crafted/xor4.i,ILP32,true,UNKNOWN,VALIDATED,11.11,9,1,48 +Sequentialized,seq-mthreaded/pals_floodmax.3.3.ufo.BOUNDED-6.pals.c,ILP32,false,UNKNOWN,VALIDATED,,,0,47 +LinkedLists,forester-heap/dll-rb-sentinel-2.i,ILP32,true,UNKNOWN,VALIDATED,76.47,34,174,48 +Recursive,fuzzle-programs/11_fuzzle_50x50_CVE-2016-4489.c,LP64,false,UNKNOWN,VALIDATED,,,0,53 +XCSP,xcsp/AllInterval-016.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,6.25,32,21,4 +Loops,loop-lit/gsv2008.i,ILP32,true,FAILED,VALIDATED,55.56,9,78,49 +ProductLines,product-lines/elevator_spec1_product27.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Heap,ldv-memsafety/memleaks_test17_1-1.i,ILP32,unknown,FAILED,VALIDATED,4.55,44,1,48 +ECA,eca-rers2012/Problem03_label16.c,ILP32,true,UNKNOWN,VALIDATED,3.04,296,7,49 +ControlFlow,ntdrivers/floppy.i.cil-2.c,ILP32,unknown,FAILED,VALIDATED,,,0,49 +Floats,float-newlib/float_req_bl_0660b.c,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-examples/standard_compareModified_ground.i,ILP32,true,UNKNOWN,VALIDATED,,,0,49 +Sequentialized,seq-mthreaded/pals_floodmax.4_overflow.ufo.UNBOUNDED.pals.c,ILP32,false,UNKNOWN,VALIDATED,,,0,49 +LinkedLists,forester-heap/sll-circular-2.i,ILP32,false,UNKNOWN,VALIDATED,66.67,21,86,48 +Recursive,recursified_loop-invariants/recursified_linear-inequality-inv-b.c,ILP32,unknown,UNKNOWN,VALIDATED,100.0,7,12,48 +XCSP,xcsp/CostasArray-13.c,ILP32,false,SUCCEEDED,VALIDATED,100.0,2,23,9 +Loops,loop-zilu/benchmark33_linear.i,ILP32,true,SUCCEEDED,VALIDATED,71.43,7,102,45 +ProductLines,product-lines/elevator_spec2_product27.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,48 +Heap,ldv-memsafety/memleaks_test22_5.i,ILP32,unknown,FAILED,VALIDATED,34.38,32,21,27 +ECA,eca-rers2012/Problem04_label56.c,ILP32,true,UNKNOWN,VALIDATED,,,0,50 +ControlFlow,openssl-simplified/s3_clnt_4.cil-2.c,ILP32,unknown,UNKNOWN,VALIDATED,46.29,175,233,48 +Floats,floats-cdfpl/newton_1_7.i,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-examples/standard_strcmp_ground.i,ILP32,true,UNKNOWN,VALIDATED,7.69,13,1,48 +Sequentialized,seq-mthreaded/pals_lcr-var-start-time.4.ufo.BOUNDED-8.pals.c,ILP32,true,SUCCEEDED,VALIDATED_ABORTS,70.73,123,25,19 +LinkedLists,forester-heap/sll-sorted-2.i,ILP32,true,UNKNOWN,TESTCOV_ERROR,,,500,49 +Recursive,recursified_nla-digbench/recursified_cohencu-ll.c,ILP32,unknown,UNKNOWN,VALIDATED,57.14,7,17,47 +XCSP,xcsp/Dubois-021.c,ILP32,true,FAILED,VALIDATED,100.0,2,23,5 +Loops,loops/count_up_down-2.c,ILP32,false,UNKNOWN,VALIDATED_ABORTS,50.0,4,1,48 +ProductLines,product-lines/elevator_spec3_product27.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,50 +Heap,ldv-memsafety/memleaks_test9_1.i,ILP32,unknown,FAILED,VALIDATED,8.82,34,21,14 +ECA,eca-rers2012/Problem06_label21.c,ILP32,false,UNKNOWN,VALIDATED,,,0,50 +ControlFlow,openssl-simplified/s3_srvr_6.cil-1.c,ILP32,unknown,FAILED,VALIDATED_ABORTS,51.5,233,284,48 +Floats,neural-networks/cartpole_12_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-fpi/eqn2.c,ILP32,true,UNKNOWN,VALIDATED,,,0,15 +Sequentialized,seq-mthreaded/pals_lcr.4_overflow.ufo.UNBOUNDED.pals.c,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,83.16,95,24,16 +LinkedLists,heap-manipulation/tree-2.i,ILP32,false,FAILED,VALIDATED_ABORTS,83.33,36,186,27 +Recursive,recursified_nla-digbench/recursified_egcd2.c,ILP32,unknown,SUCCEEDED,VALIDATED,90.0,10,14,12 +XCSP,xcsp/Dubois-035.c,ILP32,true,FAILED,VALIDATED,100.0,2,24,7 +Loops,nla-digbench-scaling/bresenham-ll_valuebound2.c,ILP32,true,SUCCEEDED,VALIDATED,90.0,10,10,2 +ProductLines,product-lines/email_spec0_product10.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Heap,ldv-regression/test23-2.c,ILP32,true,SUCCEEDED,VALIDATED,87.5,8,7,8 +ECA,eca-rers2012/Problem07_label44.c,ILP32,false,UNKNOWN,VALIDATED,,,0,77 +ControlFlow,openssl/s3_srvr.blast.02.i.cil-2.c,ILP32,unknown,FAILED,VALIDATED,15.92,201,227,47 +Floats,neural-networks/cartpole_73_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Arrays,array-fpi/ms1.c,ILP32,true,UNKNOWN,VALIDATED,,,0,0 +Sequentialized,seq-mthreaded/pals_opt-floodmax.3.1.ufo.BOUNDED-6.pals.c,ILP32,false,FAILED,TESTCOV_ERROR,,,316,47 +LinkedLists,list-properties/alternating_list-2.i,ILP32,false,UNKNOWN,VALIDATED,63.16,19,4,48 +Recursive,recursified_nla-digbench/recursified_geo1-u2.c,ILP32,unknown,UNKNOWN,VALIDATED_ABORTS,71.43,7,12,47 +XCSP,xcsp/Dubois-090.c,ILP32,true,SUCCEEDED,VALIDATED,100.0,2,25,19 +Loops,nla-digbench-scaling/divbin_unwindbound1.i,ILP32,true,SUCCEEDED,VALIDATED,53.33,15,4,2 +ProductLines,product-lines/email_spec11_product08.cil.c,ILP32,true,UNKNOWN,VALIDATED,,,0,1 +Heap,list-ext-properties/960521-1_1-3.i,ILP32,unknown,FAILED,VALIDATED,11.11,9,20,4 +ECA,eca-rers2012/Problem09_label06.c,ILP32,false,ERROR,NO_SUITE,,,0,88 +ControlFlow,openssl/s3_srvr.blast.11.i.cil-1.c,ILP32,unknown,FAILED,VALIDATED,,,0,50 +Floats,neural-networks/dubinsrejoin_35_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,3 +Arrays,array-fpi/s32if.c,ILP32,true,UNKNOWN,VALIDATED,,,0,2 +Sequentialized,seq-mthreaded/pals_opt-floodmax.4.3.ufo.BOUNDED-8.pals.c,ILP32,false,UNKNOWN,TESTCOV_ERROR,,,180,48 +LinkedLists,memsafety-broom/dll-lst-data.i,LP64,unknown,UNKNOWN,VALIDATED,100.0,6,17,49 +Recursive,recursified_nla-digbench/recursified_hard.c,ILP32,unknown,SUCCEEDED,VALIDATED,100.0,12,15,10 +XCSP,xcsp/Haystacks-14.c,ILP32,true,SUCCEEDED,VALIDATED,100.0,2,25,19 +Loops,nla-digbench-scaling/fermat1-ll_unwindbound20.c,ILP32,false,FAILED,VALIDATED,,,0,57 +ProductLines,product-lines/email_spec1_product14.cil.c,ILP32,false,UNKNOWN,VALIDATED,,,0,1 +Heap,memsafety-cve/NanoNNG.i,ILP32,unknown,UNKNOWN,TESTCOV_ERROR,,,500,49 +ECA,eca-rers2012/Problem10_label30.c,ILP32,true,UNKNOWN,VALIDATED,,,0,48 +ControlFlow,unsignedintegeroverflow-sas23/distance_through_overflow.i,ILP32,true,SUCCEEDED,VALIDATED,60.0,5,2,8 +Floats,neural-networks/dubinsrejoin_9_safe.c-amalgamation.i,ILP32,true,UNKNOWN,VALIDATED,,,0,35 +Arrays,array-fpi/sina5f.c,ILP32,false,UNKNOWN,VALIDATED,,,0,5 +Sequentialized,seq-pthread/cs_dekker.i,ILP32,true,UNKNOWN,TESTCOV_ERROR,,,496,47 +LinkedLists,memsafety-broom/sll-lst-data.i,LP64,unknown,UNKNOWN,VALIDATED,100.0,6,21,48 +Sequentialized,seq-pthread/cs_fib-1.i,ILP32,true,UNKNOWN,TESTCOV_ERROR,,,500,48 +LinkedLists,memsafety-broom/sll-middle-shared.i,LP64,unknown,UNKNOWN,VALIDATED,87.5,8,14,47 +Sequentialized,seq-pthread/cs_fib_longer-2.i,ILP32,false,UNKNOWN,TESTCOV_ERROR,,,500,49 +LinkedLists,memsafety-broom/sll-nested-sll-inline.i,LP64,unknown,UNKNOWN,VALIDATED,100.0,14,136,47 +Sequentialized,seq-pthread/cs_lazy.i,ILP32,unknown,UNKNOWN,TESTCOV_ERROR,,,407,48 +LinkedLists,memsafety-broom/sll-nested-sll-twice.i,LP64,unknown,UNKNOWN,VALIDATED,100.0,12,228,48 +Sequentialized,seq-pthread/cs_queue-1.i,ILP32,true,UNKNOWN,VALIDATED,4.5,200,203,47 +LinkedLists,memsafety-broom/sll-shared-before.i,LP64,unknown,UNKNOWN,VALIDATED,100.0,4,7,48 +Sequentialized,seq-pthread/cs_read_write_lock-2.i,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,22.94,109,40,34 +LinkedLists,memsafety-broom/sll-shared-sll-before.i,LP64,unknown,UNKNOWN,VALIDATED,75.0,8,1,47 +Sequentialized,seq-pthread/cs_stack-2.i,ILP32,false,SUCCEEDED,VALIDATED_ABORTS,15.03,153,27,38 +LinkedLists,memsafety-broom/sll.i,LP64,unknown,UNKNOWN,VALIDATED,100.0,6,23,48 +Sequentialized,seq-pthread/cs_stateful-2.i,ILP32,true,UNKNOWN,VALIDATED,4.76,105,203,48 +Arrays,array-memsafety/diff-alloca-2.i,ILP32,unknown,SUCCEEDED,VALIDATED,100.0,16,20,29 +Floats,neural-networks/lunarlander_69_unsafe.c-amalgamation.i,ILP32,false,UNKNOWN,VALIDATED,,,0,1 diff --git a/tests/testcomp/results_cover-error_s0/results.csv b/tests/testcomp/results_cover-error_s0/results.csv new file mode 100644 index 000000000..9fa87460e --- /dev/null +++ b/tests/testcomp/results_cover-error_s0/results.csv @@ -0,0 +1,274 @@ +category,program,data_model,expected_unreach,verdict,testcov,coverage,goals,n_tests,elapsed_s +Arrays,array-fpi/ms3f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Heap,ldv-regression/test29-1.c,ILP32,false,FAILED,COVERED,100.0,1,1,13 +Sequentialized,seq-mthreaded/pals_lcr-var-start-time.6.1.ufo.UNBOUNDED.pals.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,3 +ProductLines,product-lines/email_spec27_product34.cil.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,48 +Arrays,array-fpi/condgf.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Heap,ldv-regression/test22-3.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Sequentialized,seq-mthreaded/pals_floodmax.4.3.ufo.BOUNDED-8.pals.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,55 +ProductLines,product-lines/email_spec0_product22.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +Arrays,array-fpi/s5liff.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Heap,ldv-sets/test_mutex_unlock_at_exit.i,ILP32,false,FAILED,COVERED,100.0,1,1,2 +Sequentialized,seq-mthreaded/pals_opt-floodmax.3.4.ufo.BOUNDED-6.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,12 +ProductLines,product-lines/email_spec6_product29.cil.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,48 +Arrays,array-examples/standard_copy6_ground-1.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +Heap,ldv-regression/rule57_ebda_blast_2.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,13 +Sequentialized,seq-mthreaded/pals_floodmax.3.1.ufo.BOUNDED-6.pals.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,6 +ProductLines,product-lines/elevator_spec2_product30.cil.c,ILP32,false,FAILED,COVERED,100.0,1,1,2 +Arrays,array-fpi/ifeqn4f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,10 +Heap,ldv-regression/test24-2.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +Sequentialized,seq-mthreaded/pals_floodmax.5_overflow.ufo.UNBOUNDED.pals.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,61 +ProductLines,product-lines/email_spec1_product22.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +Arrays,array-fpi/s1iff.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Loops,nla-digbench-scaling/freire2_valuebound5.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Floats,floats-cdfpl/newton_2_8.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,1 +XCSP,xcsp/aim-100-3-4-sat-2.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +BitVectors,bitvector/soft_float_4-3a.c.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +Recursive,recursive/BallRajamani-SPIN2000-Fig1.c,ILP32,false,FAILED,COVERED,100.0,1,1,15 +ECA,eca-rers2012/Problem18_label09.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,48 +LinkedLists,list-properties/alternating_list-2.i,ILP32,false,ERROR,COVERED,100.0,1,1,16 +Heap,ldv-regression/fo_test.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,1 +Sequentialized,seq-mthreaded/pals_STARTPALS_ActiveStandby.4_2.ufo.UNBOUNDED.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ProductLines,product-lines/elevator_spec1_product26.cil.c,ILP32,false,FAILED,COVERED,100.0,1,1,1 +Arrays,array-examples/standard_running-1.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,47 +Loops,loops/sum01_bug02.i,ILP32,false,ERROR,COVERED,100.0,1,1,10 +Floats,float-benchs/inv_Newton.c.p+cfa-reducer.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +XCSP,xcsp/AllInterval-015.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +Heap,ldv-regression/test23-1.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +Sequentialized,seq-mthreaded/pals_floodmax.5.1.ufo.UNBOUNDED.pals.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,62 +ProductLines,product-lines/email_spec11_product26.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Arrays,array-fpi/indp4f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +Recursive,fuzzle-programs/14_fuzzle_50x50_CVE-2016-6131.c,LP64,false,UNKNOWN,NOT_COVERED,,,0,54 +ECA,eca-rers2012/Problem08_label51.c,ILP32,false,ERROR,NO_SUITE,,,0,90 +LinkedLists,forester-heap/sll-queue-2.i,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Loops,nla-digbench-scaling/fermat2-ll_unwindbound20.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,59 +Floats,floats-cdfpl/newton_2_6.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,5 +XCSP,xcsp/aim-100-2-0-sat-2.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,3 +Heap,ldv-sets/test_mutex_double_unlock.i,ILP32,false,FAILED,COVERED,100.0,1,1,1 +Sequentialized,seq-mthreaded/pals_lcr.8_overflow.ufo.UNBOUNDED.pals.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,3 +ProductLines,product-lines/email_spec4_product35.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Arrays,array-fpi/sqm-iff.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,10 +Recursive,recursive/Ackermann02.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,3 +ECA,eca-rers2012/Problem17_label09.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +LinkedLists,list-ext3-properties/sll_nondet_insert-1.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,27 +Loops,nla-digbench-scaling/ps5-ll_unwindbound50.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,2 +Floats,loop-floats-scientific-comp/loop1-2.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,1 +XCSP,xcsp/aim-200-6-0-sat-1.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Heap,ldv-regression/test21-2.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +Sequentialized,seq-mthreaded/pals_STARTPALS_ActiveStandby.4_1.ufo.BOUNDED-10.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ProductLines,product-lines/elevator_spec14_productSimulator.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Arrays,array-examples/standard_copy3_ground-2.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +Recursive,fuzzle-programs/03_fuzzle_50x50.c,LP64,false,FAILED,NOT_COVERED,0.0,1,1,54 +ECA,eca-programs/Problem102_label34.c,ILP32,false,FAILED,TESTCOV_ERROR,,,1,84 +LinkedLists,forester-heap/dll-optional-2.i,ILP32,false,FAILED,COVERED,100.0,1,1,26 +Loops,loops/matrix-2-2.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +Floats,float-benchs/inv_Newton-2.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +XCSP,xcsp/AllInterval-013.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +Heap,list-ext2-properties/simple_search_value-2.i,ILP32,false,ERROR,COVERED,100.0,1,1,2 +Sequentialized,seq-mthreaded/pals_floodmax.4.1.ufo.BOUNDED-8.pals.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +ProductLines,product-lines/elevator_spec9_product28.cil.c,ILP32,false,FAILED,COVERED,100.0,1,1,1 +Arrays,array-fpi/eqn1f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,8 +Sequentialized,seq-mthreaded/pals_floodmax.4.4.ufo.UNBOUNDED.pals.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,47 +ProductLines,product-lines/email_spec0_product35.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Arrays,array-fpi/ifeqn1f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,7 +Sequentialized,seq-mthreaded/pals_floodmax.5.3.ufo.BOUNDED-10.pals.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +ProductLines,product-lines/email_spec11_productSimulator.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +Arrays,array-fpi/indp1f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Sequentialized,seq-mthreaded/pals_lcr-var-start-time.3.2.ufo.BOUNDED-6.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +ProductLines,product-lines/email_spec1_product32.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Arrays,array-fpi/modpf.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +Sequentialized,seq-mthreaded/pals_lcr-var-start-time.5.1.ufo.UNBOUNDED.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,5 +ProductLines,product-lines/email_spec27_product27.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Arrays,array-fpi/ncompf.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,27 +Sequentialized,seq-mthreaded/pals_lcr.3.1.ufo.BOUNDED-6.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,3 +ProductLines,product-lines/email_spec3_product18.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Arrays,array-fpi/res2f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Sequentialized,seq-mthreaded/pals_lcr.5.1.ufo.BOUNDED-10.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ProductLines,product-lines/email_spec3_product32.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Arrays,array-fpi/s2iff.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Sequentialized,seq-mthreaded/pals_lcr.7.1.ufo.UNBOUNDED.pals.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,3 +ProductLines,product-lines/email_spec4_product30.cil.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +Arrays,array-fpi/s4iff.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Sequentialized,seq-mthreaded/pals_opt-floodmax.3.2.ufo.BOUNDED-6.pals.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,10 +ProductLines,product-lines/email_spec6_product16.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Arrays,array-fpi/sina3f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,4 +Sequentialized,seq-mthreaded/pals_opt-floodmax.4.1.ufo.BOUNDED-8.pals.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +ProductLines,product-lines/email_spec6_product34.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,50 +Arrays,array-fpi/ss2f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +Sequentialized,seq-mthreaded/pals_opt-floodmax.4.4.ufo.BOUNDED-8.pals.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +ProductLines,product-lines/email_spec7_product35.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Arrays,array-programs/copysome1-2.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +Sequentialized,seq-mthreaded/pals_opt-floodmax.5.3.ufo.BOUNDED-10.pals.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +ProductLines,product-lines/email_spec8_product33.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Arrays,reducercommutativity/rangesum10.i,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Sequentialized,seq-pthread/cs_queue-2.i,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +ProductLines,product-lines/email_spec9_product31.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Arrays,array-examples/data_structures_set_multi_proc_ground-1.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,48 +Sequentialized,seq-mthreaded/pals_STARTPALS_ActiveStandby.1.ufo.BOUNDED-10.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ProductLines,product-lines/elevator_spec14_product24.cil.c,ILP32,false,FAILED,COVERED,100.0,1,1,1 +Arrays,array-examples/sorting_selectionsort_2_ground.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,48 +Floats,loop-floats-scientific-comp/loop4.i,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +XCSP,xcsp/AllInterval-008.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Loops,loops-crafted-1/nested_delay_notd2.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,5 +ProductLines,product-lines/elevator_spec1_product30.cil.c,ILP32,false,FAILED,COVERED,100.0,1,1,1 +Arrays,array-examples/standard_copy4_ground-2.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,48 +ECA,eca-programs/Problem102_label50.c,ILP32,false,FAILED,TESTCOV_ERROR,,,1,86 +Arrays,array-examples/standard_copy7_ground-1.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,48 +ECA,eca-programs/Problem103_label56.c,ILP32,false,ERROR,NO_SUITE,,,0,90 +Arrays,array-examples/standard_minInArray_ground-1.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,48 +ECA,eca-rers2012/Problem03_label45.c,ILP32,false,ERROR,COVERED,100.0,1,1,26 +Arrays,array-fpi/brs1f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +ECA,eca-rers2012/Problem04_label35.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,51 +Arrays,array-fpi/brs5f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +ECA,eca-rers2012/Problem05_label30.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,52 +Arrays,array-fpi/condmf.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ECA,eca-rers2012/Problem05_label55.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,52 +Arrays,array-fpi/eqn2f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,6 +ECA,eca-rers2012/Problem06_label20.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,52 +Arrays,array-fpi/eqn5f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,7 +ECA,eca-rers2012/Problem06_label58.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,50 +Arrays,array-fpi/ifeqn2f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,6 +ECA,eca-rers2012/Problem07_label31.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,65 +Arrays,array-fpi/ifeqn5f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,6 +ECA,eca-rers2012/Problem08_label02.c,ILP32,false,ERROR,NO_SUITE,,,0,90 +Arrays,array-fpi/indp2f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ECA,eca-rers2012/Problem08_label34.c,ILP32,false,ERROR,NO_SUITE,,,0,90 +Arrays,array-fpi/indp5f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +ECA,eca-rers2012/Problem09_label08.c,ILP32,false,FAILED,TESTCOV_ERROR,,,1,88 +Arrays,array-fpi/ms1f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ECA,eca-rers2012/Problem09_label47.c,ILP32,false,ERROR,NO_SUITE,,,0,90 +Arrays,array-fpi/ms4f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ECA,eca-rers2012/Problem10_label41.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,40 +Arrays,array-fpi/nsqm-iff.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,9 +ECA,eca-rers2012/Problem11_label20.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,51 +Arrays,array-fpi/res1f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ECA,eca-rers2012/Problem12_label06.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +Arrays,array-fpi/res2of.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ECA,eca-rers2012/Problem12_label37.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,52 +Arrays,array-fpi/s1liff.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +ECA,eca-rers2012/Problem13_label11.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,52 +Arrays,array-fpi/s2liff.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ECA,eca-rers2012/Problem13_label36.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Arrays,array-fpi/s3liff.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ECA,eca-rers2012/Problem15_label09.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Arrays,array-fpi/s52iff.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ECA,eca-rers2012/Problem15_label41.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,50 +Arrays,array-fpi/sina1f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ECA,eca-rers2012/Problem16_label14.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,50 +Arrays,array-fpi/sina4f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,6 +ECA,eca-rers2012/Problem16_label44.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,50 +Arrays,array-fpi/sqmf.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,6 +ECA,eca-rers2012/Problem17_label26.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,50 +Arrays,array-fpi/ss4f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +ECA,eca-rers2012/Problem17_label55.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,51 +Arrays,array-industry-pattern/array_single_elem_init.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +ECA,eca-rers2012/Problem18_label27.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,51 +Loops,nla-digbench-scaling/ps5-ll_unwindbound20.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,2 +ProductLines,product-lines/email_spec8_productSimulator.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,51 +Sequentialized,seq-pthread/cs_fib-2.i,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,2 +Arrays,reducercommutativity/rangesum40.i,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,56 +ECA,eca-rers2018/Problem12.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,58 +Loops,loop-acceleration/array_3-2.i,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,53 +ProductLines,product-lines/elevator_spec14_product20.cil.c,ILP32,false,FAILED,COVERED,100.0,1,1,1 +Sequentialized,seq-mthreaded/pals_STARTPALS_ActiveStandby.4_2.ufo.BOUNDED-10.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +Arrays,array-examples/standard_copy2_ground-1.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,48 +ECA,eca-programs/Problem101_label15.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,72 +Loops,loop-invgen/id_trans.i,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,15 +ProductLines,product-lines/elevator_spec1_product22.cil.c,ILP32,false,FAILED,COVERED,100.0,1,1,2 +Sequentialized,seq-mthreaded/pals_STARTPALS_Triplicated.2.ufo.UNBOUNDED.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Arrays,array-examples/standard_partition_ground-1.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,50 +ECA,eca-programs/Problem102_label25.c,ILP32,false,FAILED,TESTCOV_ERROR,,,1,90 +Loops,loops/for_bounded_loop1.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,12 +ProductLines,product-lines/elevator_spec2_product20.cil.c,ILP32,false,FAILED,COVERED,100.0,1,1,1 +Sequentialized,seq-mthreaded/pals_floodmax.3_overflow.ufo.UNBOUNDED.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,9 +Arrays,array-fpi/condaf.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +ECA,eca-programs/Problem103_label51.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,89 +Loops,loops/string-2.i,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +ProductLines,product-lines/elevator_spec3_product03.cil.c,ILP32,false,FAILED,COVERED,100.0,1,1,1 +Sequentialized,seq-mthreaded/pals_floodmax.4.4.ufo.BOUNDED-8.pals.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Arrays,array-fpi/ifcompf.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,25 +ECA,eca-rers2012/Problem04_label04.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,51 +Loops,loops/trex03-1.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +ProductLines,product-lines/elevator_spec3_product31.cil.c,ILP32,false,FAILED,COVERED,100.0,1,1,1 +Sequentialized,seq-mthreaded/pals_floodmax.5.3.ufo.UNBOUNDED.pals.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,61 +Arrays,array-fpi/indp3f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ECA,eca-rers2012/Problem05_label15.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,50 +Loops,nla-digbench-scaling/bresenham-ll_unwindbound5.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,50 +ProductLines,product-lines/email_spec0_product16.cil.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +Sequentialized,seq-mthreaded/pals_lcr-var-start-time.3.2.ufo.UNBOUNDED.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +Arrays,array-fpi/ms2f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ECA,eca-rers2012/Problem06_label00.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,52 +Loops,nla-digbench-scaling/egcd-ll_unwindbound20.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,18 +ProductLines,product-lines/email_spec0_productSimulator.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,50 +Sequentialized,seq-mthreaded/pals_lcr-var-start-time.5.2.ufo.BOUNDED-10.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +Arrays,array-fpi/res1of.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ECA,eca-rers2012/Problem06_label47.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,52 +Loops,nla-digbench-scaling/egcd2-ll_unwindbound2.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,5 +ProductLines,product-lines/email_spec11_product33.cil.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +Sequentialized,seq-mthreaded/pals_lcr.3_overflow.ufo.UNBOUNDED.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +Arrays,array-fpi/s32iff.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ECA,eca-rers2012/Problem07_label37.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,67 +Loops,nla-digbench-scaling/egcd3-ll_unwindbound100.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +ProductLines,product-lines/email_spec1_product26.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,50 +Sequentialized,seq-mthreaded/pals_lcr.6.1.ufo.BOUNDED-12.pals.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,2 +Arrays,array-fpi/s5iff.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +ECA,eca-rers2012/Problem08_label26.c,ILP32,false,FAILED,TESTCOV_ERROR,,,1,86 +Loops,nla-digbench-scaling/fermat1-ll_unwindbound10.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,58 +ProductLines,product-lines/email_spec1_productSimulator.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,50 +Sequentialized,seq-mthreaded/pals_lcr.8.1.ufo.UNBOUNDED.pals.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,2 +Arrays,array-fpi/ss1f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ECA,eca-rers2012/Problem09_label15.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,81 +Loops,nla-digbench-scaling/fermat2-ll_unwindbound10.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,58 +ProductLines,product-lines/email_spec27_product29.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Sequentialized,seq-mthreaded/pals_opt-floodmax.3.3.ufo.UNBOUNDED.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,9 +Arrays,array-industry-pattern/check_removal_from_set_after_insertion.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,48 +ECA,eca-rers2012/Problem10_label26.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +Loops,nla-digbench-scaling/freire2_valuebound1.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ProductLines,product-lines/email_spec3_product13.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,50 +Sequentialized,seq-mthreaded/pals_opt-floodmax.4.3.ufo.UNBOUNDED.pals.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,43 +Arrays,reducercommutativity/rangesum20.i,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,50 +ECA,eca-rers2012/Problem11_label34.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,51 +Loops,nla-digbench-scaling/hard2_unwindbound1.c,ILP32,false,FAILED,COVERED,100.0,1,1,6 +ProductLines,product-lines/email_spec3_product28.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +ECA,eca-rers2012/Problem12_label10.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +Sequentialized,seq-mthreaded/pals_opt-floodmax.5.3.ufo.UNBOUNDED.pals.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,62 +Loops,nla-digbench-scaling/mannadiv_unwindbound100.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,2 +ProductLines,product-lines/email_spec3_productSimulator.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,50 +ECA,eca-rers2012/Problem13_label04.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,52 +Sequentialized,seq-pthread/cs_read_write_lock-2.i,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Loops,nla-digbench-scaling/prod4br-ll_unwindbound10.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +ProductLines,product-lines/email_spec4_product31.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +ProductLines,product-lines/email_spec4_product33.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,53 +ProductLines,product-lines/email_spec4_productSimulator.cil.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,56 +ProductLines,product-lines/email_spec6_product14.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +ProductLines,product-lines/email_spec6_product20.cil.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +ProductLines,product-lines/email_spec6_product26.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +ProductLines,product-lines/email_spec6_product30.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +ProductLines,product-lines/email_spec6_product32.cil.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +ProductLines,product-lines/email_spec6_product35.cil.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +ProductLines,product-lines/email_spec7_product28.cil.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +ProductLines,product-lines/email_spec7_product31.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +ProductLines,product-lines/email_spec7_product33.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +ProductLines,product-lines/email_spec7_productSimulator.cil.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +ProductLines,product-lines/email_spec8_product20.cil.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +ProductLines,product-lines/email_spec8_product26.cil.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,48 +ProductLines,product-lines/email_spec8_product31.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +ECA,eca-rers2012/Problem19_label14.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,48 +ProductLines,product-lines/email_spec9_product21.cil.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,48 +ECA,eca-rers2012/Problem19_label58.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,50 +ProductLines,product-lines/email_spec9_product35.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +ECA,eca-programs/Problem101_label03.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,62 +ProductLines,product-lines/elevator_spec1_productSimulator.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +ECA,eca-programs/Problem101_label13.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,67 +ProductLines,product-lines/elevator_spec3_product11.cil.c,ILP32,false,FAILED,COVERED,100.0,1,1,2 +ECA,eca-programs/Problem101_label23.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,68 +ProductLines,product-lines/elevator_spec9_product32.cil.c,ILP32,false,FAILED,COVERED,100.0,1,1,1 +ECA,eca-programs/Problem102_label15.c,ILP32,false,FAILED,TESTCOV_ERROR,,,1,80 +ProductLines,product-lines/email_spec11_product22.cil.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +ECA,eca-programs/Problem102_label48.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,82 +ProductLines,product-lines/email_spec1_product21.cil.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +ECA,eca-programs/Problem103_label46.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,87 +ProductLines,product-lines/email_spec27_product17.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +ECA,eca-rers2012/Problem03_label26.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,21 +ProductLines,product-lines/email_spec27_product33.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +ECA,eca-rers2012/Problem03_label50.c,ILP32,false,ERROR,COVERED,100.0,1,1,11 diff --git a/tests/testcomp/results_cover-error_s0/results.q40-stride.csv b/tests/testcomp/results_cover-error_s0/results.q40-stride.csv new file mode 100644 index 000000000..b50f5aab4 --- /dev/null +++ b/tests/testcomp/results_cover-error_s0/results.q40-stride.csv @@ -0,0 +1,56 @@ +category,program,data_model,expected_unreach,verdict,testcov,coverage,goals,n_tests,elapsed_s +Arrays,array-examples/data_structures_set_multi_proc_ground-1.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,45 +Arrays,array-examples/standard_allDiff2_ground.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,46 +Arrays,array-examples/standard_copy5_ground-2.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,46 +Arrays,array-examples/standard_minInArray_ground-1.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,45 +Arrays,array-fpi/brs3f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Arrays,array-fpi/condmf.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Arrays,array-fpi/eqn4f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,12 +Arrays,array-fpi/ifeqn3f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,17 +Arrays,array-fpi/indp2f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,7 +Arrays,array-fpi/modpf.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +Arrays,array-fpi/ms4f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Arrays,array-fpi/pcompf.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,35 +Arrays,array-fpi/s12iff.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Arrays,array-fpi/s2liff.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,9 +Arrays,array-fpi/s4iff.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Arrays,array-fpi/sina1f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Arrays,array-fpi/sqm-iff.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,7 +Arrays,array-fpi/ss4f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Arrays,array-programs/copysome1-2.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,46 +Arrays,reducercommutativity/rangesum05.i,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,45 +BitVectors,bitvector-loops/diamond_2-1.c,ILP32,false,FAILED,COVERED,100.0,1,1,11 +BitVectors,bitvector-regression/recHanoi03-1.c,ILP32,false,ERROR,COVERED,100.0,1,1,13 +BitVectors,bitvector/s3_clnt_2.BV.c.cil-2a.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,46 +BitVectors,bitvector/soft_float_1-3a.c.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,47 +BitVectors,bitvector/sum02-1.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,47 +ControlFlow,ntdrivers/diskperf.i.cil-2.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,44 +ControlFlow,ntdrivers/kbfiltr.i.cil-2.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +Heap,ldv-regression/fo_test.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,1 +Heap,ldv-regression/rule60_list2.c_1.i,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,45 +Heap,ldv-regression/test22-3.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Heap,ldv-regression/test24-2.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,47 +Heap,ldv-regression/test28-1.c,ILP32,false,FAILED,COVERED,100.0,1,1,11 +Heap,ldv-sets/test_add-1.i,ILP32,false,FAILED,COVERED,100.0,1,1,10 +Heap,ldv-sets/test_mutex_double_unlock.i,ILP32,false,FAILED,COVERED,100.0,1,1,1 +Heap,ldv-sets/test_mutex_unlock_at_exit.i,ILP32,false,FAILED,COVERED,100.0,1,1,1 +Heap,list-ext-properties/simple-ext.i,ILP32,false,ERROR,COVERED,100.0,1,1,2 +Heap,list-ext2-properties/simple_search_value-2.i,ILP32,false,ERROR,COVERED,100.0,1,1,2 +Loops,loop-acceleration/multivar_1-2.c,ILP32,false,ERROR,COVERED,100.0,1,1,20 +Loops,loops/array-2.c,ILP32,false,FAILED,COVERED,100.0,1,1,19 +Loops,loops/invert_string-1.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,-1 +Loops,loops/sum01_bug02.i,ILP32,false,ERROR,COVERED,100.0,1,1,9 +Loops,loops/vogal-2.i,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,58 +Loops,nla-digbench-scaling/bresenham-ll_unwindbound5.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,46 +Loops,nla-digbench-scaling/egcd-ll_unwindbound20.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,17 +Loops,nla-digbench-scaling/egcd2-ll_unwindbound2.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,4 +Loops,nla-digbench-scaling/egcd3-ll_unwindbound100.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,44 +Loops,nla-digbench-scaling/fermat1-ll_unwindbound10.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,46 +Loops,nla-digbench-scaling/fermat2-ll_unwindbound1.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,46 +Loops,nla-digbench-scaling/fermat2-ll_unwindbound50.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +Loops,nla-digbench-scaling/freire2_valuebound5.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Loops,nla-digbench-scaling/mannadiv_unwindbound1.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Loops,nla-digbench-scaling/mannadiv_unwindbound50.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,2 +Loops,nla-digbench-scaling/prod4br-ll_unwindbound5.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,47 +Loops,nla-digbench-scaling/prodbin-ll_unwindbound5.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,57 +Loops,nla-digbench-scaling/ps4-ll_unwindbound5.c,ILP32,false,FAILED,COVERED,100.0,1,1,2 diff --git a/tests/testcomp/results_cover-error_s1/results.csv b/tests/testcomp/results_cover-error_s1/results.csv new file mode 100644 index 000000000..302ab9900 --- /dev/null +++ b/tests/testcomp/results_cover-error_s1/results.csv @@ -0,0 +1,274 @@ +category,program,data_model,expected_unreach,verdict,testcov,coverage,goals,n_tests,elapsed_s +BitVectors,bitvector/s3_clnt_2.BV.c.cil-2a.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,51 +Loops,nla-digbench-scaling/fermat1-ll_unwindbound20.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,47 +Floats,floats-cdfpl/newton_1_7.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,1 +XCSP,xcsp/aim-100-1-6-sat-3.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +BitVectors,bitvector-loops/verisec_sendmail_tTflag_arr_one_loop.c,ILP32,false,FAILED,COVERED,100.0,1,1,9 +Loops,nla-digbench-scaling/bresenham-ll_unwindbound10.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Floats,float-benchs/nan_double.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +XCSP,xcsp/AllInterval-019.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +BitVectors,bitvector/soft_float_1-3a.c.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,47 +Loops,nla-digbench-scaling/prod4br-ll_unwindbound100.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,57 +Floats,floats-cdfpl/sine_1.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,2 +XCSP,xcsp/aim-200-1-6-sat-2.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +BitVectors,bitvector-loops/diamond_2-1.c,ILP32,false,FAILED,COVERED,100.0,1,1,11 +Loops,loops/insertion_sort-1-2.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Floats,float-benchs/float_int_inv_square.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +XCSP,xcsp/AllInterval-011.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +BitVectors,bitvector-regression/recHanoi03-1.c,ILP32,false,ERROR,COVERED,100.0,1,1,16 +Loops,nla-digbench-scaling/egcd2-ll_unwindbound100.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +Floats,float-newlib/double_req_bl_0870a.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +XCSP,xcsp/CostasArray-12.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +BitVectors,bitvector/s3_clnt_3.BV.c.cil-2a.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,47 +Recursive,recursive-simple/id_o100.c,ILP32,false,ERROR,COVERED,100.0,1,1,26 +ECA,eca-rers2012/Problem12_label51.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,52 +LinkedLists,heap-manipulation/dll_of_dll-2.i,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,19 +Heap,list-ext-properties/simple-ext.i,ILP32,false,ERROR,COVERED,100.0,1,1,2 +Sequentialized,seq-mthreaded/pals_opt-floodmax.5.1.ufo.UNBOUNDED.pals.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,62 +ProductLines,product-lines/email_spec8_product22.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +Arrays,array-examples/standard_allDiff2_ground.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,48 +Loops,loop-invariants/linear-inequality-inv-b.c,ILP32,false,FAILED,COVERED,100.0,1,1,3 +Floats,float-benchs/cast_union_loose.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +XCSP,xcsp/AllInterval-007.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +BitVectors,bitvector/sum02-1.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,48 +Recursive,fuzzle-programs/06_fuzzle_50x50_0-cycle.c,LP64,false,UNKNOWN,NOT_COVERED,,,0,51 +ECA,eca-rers2012/Problem04_label15.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,50 +LinkedLists,forester-heap/dll-rb-sentinel-1.i,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,28 +Loops,nla-digbench-scaling/egcd-ll_unwindbound10.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,31 +Floats,float-benchs/sin_interpolated_index-1.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +XCSP,xcsp/AllInterval-030.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +Heap,ldv-regression/test25-1.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +Sequentialized,seq-mthreaded/pals_lcr-var-start-time.4.2.ufo.BOUNDED-8.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,3 +ProductLines,product-lines/email_spec27_product18.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,50 +Arrays,array-fpi/pcompf.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,40 +Recursive,recursive-simple/id_b3_o2-2.c,ILP32,false,ERROR,COVERED,100.0,1,1,10 +ECA,eca-rers2012/Problem11_label43.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +LinkedLists,forester-heap/sll-token-1.i,ILP32,false,ERROR,COVERED,100.0,1,1,1 +Loops,nla-digbench-scaling/mannadiv_unwindbound10.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,3 +Floats,floats-cdfpl/newton_3_7.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,1 +XCSP,xcsp/aim-100-6-0-sat-2.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +Heap,list-ext-properties/list-ext.i,ILP32,false,FAILED,COVERED,100.0,1,1,30 +Sequentialized,seq-mthreaded/pals_opt-floodmax.4.2.ufo.UNBOUNDED.pals.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,54 +ProductLines,product-lines/email_spec7_product30.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Arrays,array-tiling/skippedu.c,ILP32,false,FAILED,COVERED,100.0,1,1,14 +Recursive,recursive/Fibonacci04.c,ILP32,false,ERROR,COVERED,100.0,1,1,5 +ECA,eca-rers2012/Problem19_label21.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,52 +LinkedLists,list-properties/list_flag-1.i,ILP32,false,ERROR,COVERED,100.0,1,1,24 +Loops,loop-acceleration/diamond_1-2.c,ILP32,false,FAILED,COVERED,100.0,1,1,20 +Floats,float-benchs/cast_float_ptr.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +XCSP,xcsp/AllInterval-005.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Heap,ldv-regression/test28-1.c,ILP32,false,FAILED,COVERED,100.0,1,1,12 +Sequentialized,seq-mthreaded/pals_STARTPALS_Triplicated.1.ufo.BOUNDED-10.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ProductLines,product-lines/elevator_spec2_product18.cil.c,ILP32,false,FAILED,COVERED,100.0,1,1,2 +Arrays,array-examples/standard_copy9_ground-1.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +Recursive,fuzzle-programs/05_fuzzle_70x70.c,LP64,false,FAILED,TESTCOV_ERROR,,,1,58 +ECA,eca-rers2012/Problem03_label28.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,18 +LinkedLists,forester-heap/dll-rb-cnstr_1-2.i,ILP32,false,ERROR,COVERED,100.0,1,1,1 +Loops,loops/trex02-2.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,5 +Floats,float-benchs/inv_square-1.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,1 +XCSP,xcsp/AllInterval-017.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Loops,nla-digbench-scaling/bresenham-ll_unwindbound20.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Floats,float-benchs/nan_float.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,1 +XCSP,xcsp/AllInterval-020.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Loops,nla-digbench-scaling/egcd-ll_unwindbound5.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,17 +Floats,float-benchs/sqrt_poly2.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +XCSP,xcsp/CostasArray-10.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Loops,nla-digbench-scaling/egcd2-ll_unwindbound50.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +Floats,float-newlib/float_req_bl_0870a.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,2 +XCSP,xcsp/CostasArray-14.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Loops,nla-digbench-scaling/fermat1-ll_unwindbound1.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,39 +Floats,floats-cdfpl/newton_1_5.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,1 +XCSP,xcsp/aim-100-1-6-sat-1.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +Loops,nla-digbench-scaling/fermat2-ll_unwindbound1.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,58 +Floats,floats-cdfpl/newton_1_8.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,2 +XCSP,xcsp/aim-100-1-6-sat-4.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +Loops,nla-digbench-scaling/freire2_valuebound10.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +Floats,floats-cdfpl/newton_2_7.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,1 +XCSP,xcsp/aim-100-2-0-sat-4.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Loops,nla-digbench-scaling/hard2_unwindbound10.c,ILP32,false,FAILED,COVERED,100.0,1,1,8 +Floats,floats-cdfpl/newton_3_6.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,1 +XCSP,xcsp/aim-100-3-4-sat-4.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +Loops,nla-digbench-scaling/mannadiv_unwindbound5.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +Floats,floats-cdfpl/newton_3_8.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,1 +XCSP,xcsp/aim-100-6-0-sat-4.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +Loops,nla-digbench-scaling/prod4br-ll_unwindbound5.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,58 +Floats,floats-cdfpl/sine_2.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,1 +XCSP,xcsp/aim-200-1-6-sat-3.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Loops,nla-digbench-scaling/ps4-ll_unwindbound10.c,ILP32,false,FAILED,COVERED,100.0,1,1,2 +Floats,floats-cdfpl/square_1.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,1 +XCSP,xcsp/aim-200-2-0-sat-3.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +Loops,nla-digbench-scaling/ps5-ll_unwindbound100.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,2 +Floats,floats-cdfpl/square_3.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,1 +XCSP,xcsp/aim-200-3-4-sat-3.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +Loops,nla-digbench-scaling/ps6-ll_unwindbound2.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Floats,loop-floats-scientific-comp/loop2-1.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,1 +XCSP,xcsp/aim-200-6-0-sat-3.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Loops,loop-acceleration/array_2-1.i,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +Floats,floats-cdfpl/newton_1_6.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,1 +XCSP,xcsp/AllInterval-006.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +Loops,loop-acceleration/simple_2-2.c,ILP32,false,ERROR,COVERED,100.0,1,1,19 +ECA,eca-programs/Problem101_label18.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,64 +LinkedLists,heap-manipulation/tree-4.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,31 +Sequentialized,seq-mthreaded/pals_STARTPALS_ActiveStandby.5.ufo.BOUNDED-10.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +XCSP,xcsp/AllInterval-010.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Loops,loops/count_up_down-2.c,ILP32,false,ERROR,COVERED,100.0,1,1,14 +ProductLines,product-lines/elevator_spec2_product24.cil.c,ILP32,false,FAILED,COVERED,100.0,1,1,2 +Loops,loops/invert_string-1.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ProductLines,product-lines/elevator_spec2_productSimulator.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Loops,loops/nec20.c,ILP32,false,FAILED,COVERED,100.0,1,1,3 +ProductLines,product-lines/elevator_spec3_product23.cil.c,ILP32,false,FAILED,COVERED,100.0,1,1,2 +Loops,loops/terminator_01.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,26 +ProductLines,product-lines/elevator_spec3_product32.cil.c,ILP32,false,FAILED,COVERED,100.0,1,1,2 +Loops,loops/vogal-2.i,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,62 +ProductLines,product-lines/elevator_spec9_productSimulator.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Loops,nla-digbench-scaling/bresenham-ll_unwindbound100.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +ProductLines,product-lines/email_spec0_product31.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +Loops,nla-digbench-scaling/bresenham-ll_unwindbound50.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,50 +ProductLines,product-lines/email_spec11_product15.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Loops,nla-digbench-scaling/egcd-ll_unwindbound2.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +ProductLines,product-lines/email_spec11_product32.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,50 +Loops,nla-digbench-scaling/egcd2-ll_unwindbound1.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,3 +ProductLines,product-lines/email_spec1_product16.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Loops,nla-digbench-scaling/egcd2-ll_unwindbound20.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,51 +ProductLines,product-lines/email_spec1_product29.cil.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +Loops,nla-digbench-scaling/egcd3-ll_unwindbound10.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,22 +ProductLines,product-lines/email_spec1_product35.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Loops,nla-digbench-scaling/egcd3-ll_unwindbound5.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +ProductLines,product-lines/email_spec27_product23.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Loops,nla-digbench-scaling/fermat1-ll_unwindbound100.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +ProductLines,product-lines/email_spec27_product31.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Loops,nla-digbench-scaling/fermat1-ll_unwindbound5.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +ProductLines,product-lines/email_spec27_productSimulator.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Loops,nla-digbench-scaling/fermat2-ll_unwindbound100.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,59 +ProductLines,product-lines/email_spec3_product23.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Loops,nla-digbench-scaling/fermat2-ll_unwindbound50.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +ProductLines,product-lines/email_spec3_product29.cil.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +Loops,nla-digbench-scaling/freire2_valuebound2.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ProductLines,product-lines/email_spec3_product35.cil.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,48 +Loops,nla-digbench-scaling/freire2_valuebound50.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ProductLines,product-lines/email_spec4_product24.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Loops,nla-digbench-scaling/hard2_unwindbound5.c,ILP32,false,FAILED,COVERED,100.0,1,1,1 +ProductLines,product-lines/email_spec4_product32.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Loops,nla-digbench-scaling/mannadiv_unwindbound2.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ProductLines,product-lines/email_spec6_product12.cil.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +Loops,nla-digbench-scaling/prod4br-ll_unwindbound1.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,14 +ProductLines,product-lines/email_spec6_product22.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Loops,nla-digbench-scaling/prod4br-ll_unwindbound2.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,58 +ProductLines,product-lines/email_spec6_product31.cil.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +Loops,nla-digbench-scaling/prodbin-ll_unwindbound1.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +ProductLines,product-lines/email_spec6_productSimulator.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Loops,nla-digbench-scaling/prodbin-ll_unwindbound5.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,59 +ProductLines,product-lines/email_spec7_product32.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +Loops,nla-digbench-scaling/ps4-ll_unwindbound2.c,ILP32,false,FAILED,COVERED,100.0,1,1,2 +ProductLines,product-lines/email_spec8_product16.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Loops,nla-digbench-scaling/ps5-ll_unwindbound1.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ProductLines,product-lines/email_spec8_product30.cil.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +Sequentialized,seq-mthreaded/pals_opt-floodmax.5.4.ufo.BOUNDED-10.pals.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,50 +Arrays,reducercommutativity/rangesum.i,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +ECA,eca-rers2012/Problem19_label31.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,51 +Loops,nla-digbench-scaling/ps6-ll_unwindbound5.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ProductLines,product-lines/email_spec9_product34.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Sequentialized,seq-mthreaded/pals_STARTPALS_ActiveStandby.1.ufo.UNBOUNDED.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Arrays,array-examples/sorting_selectionsort_ground-1.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,48 +ECA,eca-programs/Problem101_label08.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,70 +Loops,loop-acceleration/simple_3-1.c,ILP32,false,ERROR,COVERED,100.0,1,1,6 +ProductLines,product-lines/elevator_spec1_product18.cil.c,ILP32,false,FAILED,COVERED,100.0,1,1,1 +Sequentialized,seq-mthreaded/pals_STARTPALS_Triplicated.1.ufo.UNBOUNDED.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Arrays,array-examples/standard_copy8_ground-2.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +ECA,eca-programs/Problem102_label06.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,88 +Loops,loops/compact.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,48 +ProductLines,product-lines/elevator_spec1_product32.cil.c,ILP32,false,FAILED,COVERED,100.0,1,1,2 +Sequentialized,seq-mthreaded/pals_floodmax.3.3.ufo.UNBOUNDED.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,9 +Arrays,array-fpi/brs4f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ECA,eca-programs/Problem102_label59.c,ILP32,false,FAILED,TESTCOV_ERROR,,,1,79 +Loops,loops/n.c24.i,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +ProductLines,product-lines/elevator_spec2_product32.cil.c,ILP32,false,FAILED,COVERED,100.0,1,1,1 +Sequentialized,seq-mthreaded/pals_floodmax.4.2.ufo.UNBOUNDED.pals.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,45 +Arrays,array-fpi/eqn3f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,6 +ECA,eca-rers2012/Problem03_label37.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,25 +Loops,loops/trex01-1.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +ProductLines,product-lines/elevator_spec3_product24.cil.c,ILP32,false,FAILED,COVERED,100.0,1,1,1 +Sequentialized,seq-mthreaded/pals_floodmax.5.2.ufo.UNBOUNDED.pals.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,62 +Arrays,array-fpi/ifncompf.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,20 +ECA,eca-rers2012/Problem04_label45.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,50 +Loops,nla-digbench-scaling/bresenham-ll_unwindbound2.c,ILP32,false,FAILED,COVERED,100.0,1,1,20 +ProductLines,product-lines/elevator_spec9_product30.cil.c,ILP32,false,FAILED,COVERED,100.0,1,1,1 +Sequentialized,seq-mthreaded/pals_lcr-var-start-time.3.1.ufo.UNBOUNDED.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +Arrays,array-fpi/modsf.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ECA,eca-rers2012/Problem05_label47.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,52 +Loops,nla-digbench-scaling/egcd-ll_unwindbound100.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,16 +ProductLines,product-lines/email_spec0_product33.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Sequentialized,seq-mthreaded/pals_lcr-var-start-time.5.1.ufo.BOUNDED-10.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +Arrays,array-fpi/nsqmf.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,6 +ECA,eca-rers2012/Problem06_label27.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,52 +Loops,nla-digbench-scaling/egcd2-ll_unwindbound10.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +ProductLines,product-lines/email_spec11_product30.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,50 +Sequentialized,seq-mthreaded/pals_lcr-var-start-time.6.2.ufo.UNBOUNDED.pals.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,3 +Arrays,array-fpi/s22iff.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ECA,eca-rers2012/Problem07_label19.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,65 +Loops,nla-digbench-scaling/egcd3-ll_unwindbound1.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ProductLines,product-lines/email_spec1_product20.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,53 +Sequentialized,seq-mthreaded/pals_lcr.5.1.ufo.UNBOUNDED.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +Arrays,array-fpi/s4liff.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ECA,eca-rers2012/Problem08_label06.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,84 +Loops,nla-digbench-scaling/egcd3-ll_unwindbound50.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +ProductLines,product-lines/email_spec1_product33.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Sequentialized,seq-mthreaded/pals_lcr.7_overflow.ufo.UNBOUNDED.pals.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,2 +Arrays,array-fpi/sina5f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,7 +ECA,eca-rers2012/Problem09_label02.c,ILP32,false,ERROR,NO_SUITE,,,0,90 +Loops,nla-digbench-scaling/fermat1-ll_unwindbound50.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,58 +ProductLines,product-lines/email_spec27_product24.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Sequentialized,seq-mthreaded/pals_opt-floodmax.3.2.ufo.UNBOUNDED.pals.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,9 +Arrays,array-fpi/ssinaf.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +ECA,eca-rers2012/Problem09_label54.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,85 +Loops,nla-digbench-scaling/fermat2-ll_unwindbound5.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +ProductLines,product-lines/email_spec27_product35.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +Sequentialized,seq-mthreaded/pals_opt-floodmax.4.2.ufo.BOUNDED-8.pals.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,42 +Arrays,reducercommutativity/rangesum05.i,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +ECA,eca-rers2012/Problem11_label08.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +Loops,nla-digbench-scaling/freire2_valuebound20.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ProductLines,product-lines/email_spec3_product24.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Sequentialized,seq-mthreaded/pals_opt-floodmax.5.1.ufo.BOUNDED-10.pals.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,62 +Loops,nla-digbench-scaling/hard2_unwindbound2.c,ILP32,false,FAILED,COVERED,100.0,1,1,10 +ProductLines,product-lines/email_spec3_product30.cil.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,50 +ECA,eca-rers2012/Problem12_label30.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,52 +Sequentialized,seq-mthreaded/pals_opt-floodmax.5.4.ufo.UNBOUNDED.pals.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,62 +Loops,nla-digbench-scaling/mannadiv_unwindbound20.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,2 +ProductLines,product-lines/email_spec4_product23.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +ECA,eca-rers2012/Problem13_label19.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +Sequentialized,seq-pthread/cs_stateful-1.i,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,1 +Loops,nla-digbench-scaling/prod4br-ll_unwindbound20.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,58 +Loops,nla-digbench-scaling/prod4br-ll_unwindbound50.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +Loops,nla-digbench-scaling/prodbin-ll_unwindbound10.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,59 +Loops,nla-digbench-scaling/prodbin-ll_unwindbound20.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,59 +Loops,nla-digbench-scaling/ps4-ll_unwindbound1.c,ILP32,false,FAILED,COVERED,100.0,1,1,1 +Loops,nla-digbench-scaling/ps4-ll_unwindbound100.c,ILP32,false,FAILED,COVERED,100.0,1,1,12 +Loops,nla-digbench-scaling/ps4-ll_unwindbound20.c,ILP32,false,FAILED,COVERED,100.0,1,1,3 +Loops,nla-digbench-scaling/ps4-ll_unwindbound50.c,ILP32,false,FAILED,COVERED,100.0,1,1,3 +Loops,nla-digbench-scaling/ps5-ll_unwindbound10.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Loops,nla-digbench-scaling/ps5-ll_unwindbound2.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Loops,nla-digbench-scaling/ps5-ll_unwindbound5.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Loops,nla-digbench-scaling/ps6-ll_unwindbound1.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Loops,nla-digbench-scaling/ps6-ll_unwindbound100.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,1 +Loops,nla-digbench-scaling/ps6-ll_unwindbound20.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,1 +Loops,nla-digbench-scaling/ps6-ll_unwindbound50.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,1 +ECA,eca-rers2012/Problem18_label49.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,51 +ProductLines,product-lines/email_spec9_product15.cil.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +ECA,eca-rers2012/Problem19_label42.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,51 +ProductLines,product-lines/email_spec9_product32.cil.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +ECA,eca-programs/Problem101_label00.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,62 +ProductLines,product-lines/elevator_spec1_product24.cil.c,ILP32,false,FAILED,COVERED,100.0,1,1,1 +ECA,eca-programs/Problem101_label10.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,78 +ProductLines,product-lines/elevator_spec2_product28.cil.c,ILP32,false,FAILED,COVERED,100.0,1,1,1 +ECA,eca-programs/Problem101_label19.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,63 +ProductLines,product-lines/elevator_spec9_product26.cil.c,ILP32,false,FAILED,COVERED,100.0,1,1,1 +ECA,eca-programs/Problem102_label07.c,ILP32,false,FAILED,TESTCOV_ERROR,,,1,88 +ProductLines,product-lines/email_spec0_product34.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +ECA,eca-programs/Problem102_label37.c,ILP32,false,FAILED,TESTCOV_ERROR,,,1,87 +ProductLines,product-lines/email_spec1_product15.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +ECA,eca-programs/Problem103_label35.c,ILP32,false,FAILED,TESTCOV_ERROR,,,1,89 +ProductLines,product-lines/email_spec1_product34.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,53 +ECA,eca-programs/Problem103_label57.c,ILP32,false,ERROR,NO_SUITE,,,0,90 +ProductLines,product-lines/email_spec27_product30.cil.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +ECA,eca-rers2012/Problem03_label39.c,ILP32,false,ERROR,COVERED,100.0,1,1,22 +ProductLines,product-lines/email_spec3_product25.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 diff --git a/tests/testcomp/results_cover-error_s1/results.q40-stride.csv b/tests/testcomp/results_cover-error_s1/results.q40-stride.csv new file mode 100644 index 000000000..5331fafc2 --- /dev/null +++ b/tests/testcomp/results_cover-error_s1/results.q40-stride.csv @@ -0,0 +1,47 @@ +category,program,data_model,expected_unreach,verdict,testcov,coverage,goals,n_tests,elapsed_s +Arrays,array-examples/sorting_bubblesort_ground-2.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,44 +Arrays,array-examples/standard_copy2_ground-1.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,46 +Arrays,array-examples/standard_copy7_ground-1.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,46 +Arrays,array-examples/standard_running-1.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,44 +Arrays,array-fpi/brs5f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Arrays,array-fpi/eqn1f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,12 +Arrays,array-fpi/ifcompf.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,46 +Arrays,array-fpi/ifeqn5f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,7 +Arrays,array-fpi/indp4f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,19 +Arrays,array-fpi/ms1f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Arrays,array-fpi/ncompf.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,25 +Arrays,array-fpi/res1of.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Arrays,array-fpi/s1liff.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +Arrays,array-fpi/s3iff.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Arrays,array-fpi/s52iff.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Arrays,array-fpi/sina3f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,5 +Arrays,array-fpi/ss1f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Arrays,array-industry-pattern/array_ptr_single_elem_init-2.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,46 +Arrays,array-tiling/mlceu.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Arrays,reducercommutativity/rangesum20.i,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +BitVectors,bitvector-loops/verisec_sendmail_tTflag_arr_one_loop.c,ILP32,false,FAILED,COVERED,100.0,1,1,4 +BitVectors,bitvector/byte_add-1.i,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,47 +BitVectors,bitvector/s3_clnt_3.BV.c.cil-2a.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,45 +BitVectors,bitvector/soft_float_4-3a.c.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +ControlFlow,ntdrivers/cdaudio.i.cil-1.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,44 +ControlFlow,ntdrivers/floppy.i.cil-1.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,42 +ControlFlow,ntdrivers/parport.i.cil-1.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,45 +Heap,ldv-regression/rule57_ebda_blast_2.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,12 +Heap,ldv-regression/test21-2.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,45 +Heap,ldv-regression/test23-1.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,47 +Heap,ldv-regression/test25-1.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,46 +Heap,ldv-regression/test29-1.c,ILP32,false,FAILED,COVERED,100.0,1,1,12 +Heap,ldv-sets/test_mutex_double_lock.i,ILP32,false,FAILED,COVERED,100.0,1,1,1 +Heap,ldv-sets/test_mutex_unbounded-2.i,ILP32,false,FAILED,COVERED,100.0,1,1,3 +Heap,list-ext-properties/list-ext.i,ILP32,false,FAILED,COVERED,100.0,1,1,12 +Heap,list-ext2-properties/simple_and_skiplist_2lvl-1.i,ILP32,false,FAILED,COVERED,100.0,1,1,13 +Loops,loop-acceleration/array_2-1.i,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,45 +Loops,loop-invariants/linear-inequality-inv-b.c,ILP32,false,FAILED,COVERED,100.0,1,1,3 +Loops,loops/count_up_down-2.c,ILP32,false,ERROR,COVERED,100.0,1,1,9 +Loops,loops/n.c24.i,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,45 +Loops,loops/trex01-1.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,47 +Loops,nla-digbench-scaling/bresenham-ll_unwindbound100.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,47 +Loops,nla-digbench-scaling/egcd-ll_unwindbound10.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,22 +Loops,nla-digbench-scaling/egcd2-ll_unwindbound1.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,3 +Loops,nla-digbench-scaling/egcd2-ll_unwindbound50.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,51 +Loops,nla-digbench-scaling/egcd3-ll_unwindbound5.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 diff --git a/tests/testcomp/results_cover-error_s2/results.csv b/tests/testcomp/results_cover-error_s2/results.csv new file mode 100644 index 000000000..20fb41fd2 --- /dev/null +++ b/tests/testcomp/results_cover-error_s2/results.csv @@ -0,0 +1,273 @@ +category,program,data_model,expected_unreach,verdict,testcov,coverage,goals,n_tests,elapsed_s +ControlFlow,ntdrivers/floppy.i.cil-1.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,45 +Recursive,recursified_loop-invariants/recursified_linear-inequality-inv-b.c,ILP32,unknown,FAILED,TESTCOV_ERROR,,,1,5 +ECA,eca-rers2012/Problem10_label12.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +LinkedLists,forester-heap/sll-simple-white-blue-2.i,ILP32,false,ERROR,COVERED,100.0,1,1,12 +ControlFlow,ntdrivers/cdaudio.i.cil-1.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +Recursive,fuzzle-programs/08_fuzzle_50x50_50-cycle.c,LP64,false,FAILED,NOT_COVERED,0.0,1,1,53 +ECA,eca-rers2012/Problem05_label40.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,52 +LinkedLists,forester-heap/dll-sorted-1.i,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +ControlFlow,ntdrivers/kbfiltr.i.cil-2.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +Recursive,recursive-simple/id_o3.c,ILP32,false,ERROR,COVERED,100.0,1,1,9 +ECA,eca-rers2012/Problem16_label01.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +LinkedLists,list-ext3-properties/dll_nullified-1.i,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +ControlFlow,ntdrivers/diskperf.i.cil-2.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,48 +Recursive,fuzzle-programs/04_fuzzle_60x60.c,LP64,false,FAILED,TESTCOV_ERROR,,,1,56 +ECA,eca-programs/Problem103_label44.c,ILP32,false,ERROR,NO_SUITE,,,0,88 +LinkedLists,forester-heap/dll-queue-2.i,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +ControlFlow,ntdrivers/parport.i.cil-1.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Recursive,fuzzle-programs/12_fuzzle_50x50_CVE-2016-4492.c,LP64,false,FAILED,NOT_COVERED,0.0,1,1,52 +ECA,eca-rers2012/Problem07_label44.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,70 +LinkedLists,forester-heap/sll-circular-2.i,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,2 +Heap,ldv-sets/test_mutex_double_lock.i,ILP32,false,FAILED,COVERED,100.0,1,1,1 +Sequentialized,seq-mthreaded/pals_lcr.6.1.ufo.UNBOUNDED.pals.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,2 +ProductLines,product-lines/email_spec4_product19.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Arrays,array-industry-pattern/array_ptr_single_elem_init-2.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,50 +Loops,nla-digbench-scaling/ps4-ll_unwindbound5.c,ILP32,false,FAILED,COVERED,100.0,1,1,2 +Floats,floats-cdfpl/square_2.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,1 +XCSP,xcsp/aim-200-3-4-sat-1.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +BitVectors,bitvector/byte_add-1.i,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Recursive,fuzzle-programs/02_fuzzle_40x40.c,LP64,false,FAILED,NOT_COVERED,0.0,1,1,52 +ECA,eca-programs/Problem102_label02.c,ILP32,false,ERROR,NO_SUITE,,,0,90 +LinkedLists,forester-heap/dll-circular-1.i,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,1 +Heap,ldv-regression/rule60_list2.c_1.i,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Sequentialized,seq-mthreaded/pals_floodmax.3.4.ufo.BOUNDED-6.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,10 +ProductLines,product-lines/elevator_spec3_product28.cil.c,ILP32,false,FAILED,COVERED,100.0,1,1,2 +Arrays,array-fpi/eqn4f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,8 +Recursive,fuzzle-programs/10_fuzzle_50x50_CVE-2016-4487.c,LP64,false,UNKNOWN,NOT_COVERED,,,0,53 +ECA,eca-rers2012/Problem06_label37.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,50 +LinkedLists,forester-heap/sll-01-2.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,50 +Loops,nla-digbench-scaling/egcd3-ll_unwindbound2.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Floats,floats-cdfpl/newton_1_4.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,1 +XCSP,xcsp/CostasArray-16.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Heap,ldv-sets/test_add-1.i,ILP32,false,FAILED,COVERED,100.0,1,1,9 +Sequentialized,seq-mthreaded/pals_lcr.4.1.ufo.BOUNDED-8.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +ProductLines,product-lines/email_spec3_product27.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +Arrays,array-fpi/s3iff.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +Recursive,recursive-simple/id_o20.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,10 +ECA,eca-rers2012/Problem13_label54.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,52 +LinkedLists,heap-manipulation/sll_to_dll_rev-1.i,ILP32,false,ERROR,COVERED,100.0,1,1,17 +Loops,nla-digbench-scaling/prodbin-ll_unwindbound2.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,58 +Floats,floats-cdfpl/sine_3.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,1 +XCSP,xcsp/aim-200-2-0-sat-1.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +Heap,list-ext2-properties/simple_and_skiplist_2lvl-1.i,ILP32,false,FAILED,COVERED,100.0,1,1,15 +Sequentialized,seq-mthreaded/pals_opt-floodmax.5_overflow.ufo.UNBOUNDED.pals.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,63 +ProductLines,product-lines/email_spec9_product20.cil.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +Arrays,array-examples/sorting_bubblesort_ground-2.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +Recursive,fuzzle-programs/01_fuzzle_30x30.c,LP64,false,FAILED,NOT_COVERED,0.0,1,1,51 +ECA,eca-programs/Problem101_label12.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,73 +LinkedLists,forester-heap/dll-01-1.i,ILP32,false,ERROR,COVERED,100.0,1,1,2 +Loops,loops/bubble_sort-2.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,2 +Floats,float-benchs/cast_union_tight.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +XCSP,xcsp/AllInterval-009.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Heap,ldv-sets/test_mutex_unbounded-2.i,ILP32,false,FAILED,COVERED,100.0,1,1,3 +Sequentialized,seq-mthreaded/pals_floodmax.3.2.ufo.UNBOUNDED.pals.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,9 +ProductLines,product-lines/elevator_spec3_product19.cil.c,ILP32,false,FAILED,COVERED,100.0,1,1,1 +Arrays,array-fpi/brs3f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Recursive,fuzzle-programs/07_fuzzle_50x50_25-cycle.c,LP64,false,FAILED,NOT_COVERED,0.0,1,1,52 +ECA,eca-rers2012/Problem05_label01.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,52 +LinkedLists,forester-heap/dll-simple-white-blue-1.i,ILP32,false,ERROR,COVERED,100.0,1,1,13 +Recursive,fuzzle-programs/09_fuzzle_50x50_75-cycle.c,LP64,false,UNKNOWN,NOT_COVERED,,,0,54 +ECA,eca-rers2012/Problem06_label05.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,50 +LinkedLists,forester-heap/dll-token-2.i,ILP32,false,ERROR,COVERED,100.0,1,1,1 +Recursive,fuzzle-programs/11_fuzzle_50x50_CVE-2016-4489.c,LP64,false,FAILED,NOT_COVERED,0.0,1,1,52 +ECA,eca-rers2012/Problem07_label11.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,71 +LinkedLists,forester-heap/sll-buckets-2.i,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Recursive,fuzzle-programs/13_fuzzle_50x50_CVE-2016-4493.c,LP64,false,FAILED,NOT_COVERED,0.0,1,1,54 +ECA,eca-rers2012/Problem08_label15.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,86 +LinkedLists,forester-heap/sll-optional-2.i,ILP32,false,FAILED,COVERED,100.0,1,1,27 +Recursive,fuzzle-programs/15_fuzzle_50x50_equality-checks.c,LP64,false,UNKNOWN,NOT_COVERED,,,0,54 +ECA,eca-rers2012/Problem09_label34.c,ILP32,false,ERROR,NO_SUITE,,,0,90 +LinkedLists,forester-heap/sll-rb-cnstr_1-2.i,ILP32,false,ERROR,COVERED,100.0,1,1,2 +Recursive,recursive-simple/id2_b3_o2.c,ILP32,false,ERROR,COVERED,100.0,1,1,9 +ECA,eca-rers2012/Problem10_label57.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,50 +LinkedLists,forester-heap/sll-sorted-1.i,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Recursive,recursive-simple/id_o10.c,ILP32,false,ERROR,COVERED,100.0,1,1,10 +ECA,eca-rers2012/Problem12_label21.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,51 +LinkedLists,heap-manipulation/bubble_sort_linux-2.i,ILP32,false,ERROR,COVERED,100.0,1,1,12 +Recursive,recursive-simple/id_o1000.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +ECA,eca-rers2012/Problem13_label25.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,50 +LinkedLists,heap-manipulation/merge_sort-1.i,ILP32,false,ERROR,COVERED,100.0,1,1,32 +Recursive,recursive-simple/id_o200.c,ILP32,false,ERROR,COVERED,100.0,1,1,32 +ECA,eca-rers2012/Problem15_label30.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,51 +LinkedLists,heap-manipulation/tree-2.i,ILP32,false,ERROR,COVERED,100.0,1,1,21 +Recursive,recursive-simple/sum_non_eq-3.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,19 +ECA,eca-rers2012/Problem16_label31.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +LinkedLists,list-ext3-properties/sll_length_check-1.i,ILP32,false,ERROR,COVERED,100.0,1,1,1 +Recursive,recursive/Addition02.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,31 +ECA,eca-rers2012/Problem17_label40.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,50 +LinkedLists,list-ext3-properties/sll_of_sll_nondet_append-2.i,ILP32,false,FAILED,COVERED,100.0,1,1,42 +Recursive,recursive/EvenOdd03.c,ILP32,false,ERROR,COVERED,100.0,1,1,7 +ECA,eca-rers2012/Problem18_label38.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +LinkedLists,list-properties/list-2.i,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +Recursive,recursive/Fibonacci05.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,4 +ECA,eca-rers2012/Problem19_label51.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,50 +LinkedLists,list-properties/simple-1.i,ILP32,false,ERROR,COVERED,100.0,1,1,12 +Recursive,recursive/McCarthy91-1.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +ECA,eca-programs/Problem101_label05.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,67 +LinkedLists,forester-heap/sll-rb-sentinel-2.i,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,19 +Sequentialized,seq-mthreaded/pals_STARTPALS_ActiveStandby.4_1.ufo.UNBOUNDED.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ProductLines,product-lines/elevator_spec1_product20.cil.c,ILP32,false,FAILED,COVERED,100.0,1,1,1 +Arrays,array-examples/standard_copy1_ground-2.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,48 +ECA,eca-programs/Problem102_label13.c,ILP32,false,FAILED,TESTCOV_ERROR,,,1,81 +LinkedLists,list-properties/splice-1.i,ILP32,false,ERROR,COVERED,100.0,1,1,11 +Sequentialized,seq-mthreaded/pals_STARTPALS_Triplicated.2.ufo.BOUNDED-10.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +XCSP,xcsp/AllInterval-012.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Sequentialized,seq-mthreaded/pals_floodmax.3.1.ufo.UNBOUNDED.pals.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,5 +XCSP,xcsp/AllInterval-014.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Sequentialized,seq-mthreaded/pals_floodmax.3.3.ufo.BOUNDED-6.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,8 +XCSP,xcsp/AllInterval-016.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +Sequentialized,seq-mthreaded/pals_floodmax.3.4.ufo.UNBOUNDED.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,8 +XCSP,xcsp/AllInterval-018.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Sequentialized,seq-mthreaded/pals_floodmax.4.2.ufo.BOUNDED-8.pals.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,42 +XCSP,xcsp/AllInterval-025.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Sequentialized,seq-mthreaded/pals_floodmax.4.3.ufo.UNBOUNDED.pals.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,43 +XCSP,xcsp/AllInterval-035.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Sequentialized,seq-mthreaded/pals_floodmax.4_overflow.ufo.UNBOUNDED.pals.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,45 +XCSP,xcsp/CostasArray-11.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +Sequentialized,seq-mthreaded/pals_floodmax.5.2.ufo.BOUNDED-10.pals.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,62 +XCSP,xcsp/CostasArray-13.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +Sequentialized,seq-mthreaded/pals_floodmax.5.4.ufo.BOUNDED-10.pals.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,62 +XCSP,xcsp/CostasArray-15.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Sequentialized,seq-mthreaded/pals_lcr-var-start-time.3.1.ufo.BOUNDED-6.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +XCSP,xcsp/CostasArray-17.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Sequentialized,seq-mthreaded/pals_lcr-var-start-time.4.1.ufo.BOUNDED-8.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +XCSP,xcsp/aim-100-1-6-sat-2.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Sequentialized,seq-mthreaded/pals_lcr-var-start-time.4.2.ufo.UNBOUNDED.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +XCSP,xcsp/aim-100-2-0-sat-1.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Sequentialized,seq-mthreaded/pals_lcr-var-start-time.5.2.ufo.UNBOUNDED.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +XCSP,xcsp/aim-100-2-0-sat-3.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Sequentialized,seq-mthreaded/pals_lcr-var-start-time.6.2.ufo.BOUNDED-12.pals.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,2 +XCSP,xcsp/aim-100-3-4-sat-1.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Sequentialized,seq-mthreaded/pals_lcr.3.1.ufo.UNBOUNDED.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +XCSP,xcsp/aim-100-3-4-sat-3.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +Sequentialized,seq-mthreaded/pals_lcr.4.1.ufo.UNBOUNDED.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,3 +XCSP,xcsp/aim-100-6-0-sat-1.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Sequentialized,seq-mthreaded/pals_lcr.5_overflow.ufo.UNBOUNDED.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +XCSP,xcsp/aim-100-6-0-sat-3.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Sequentialized,seq-mthreaded/pals_lcr.6_overflow.ufo.UNBOUNDED.pals.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,2 +XCSP,xcsp/aim-200-1-6-sat-1.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Sequentialized,seq-mthreaded/pals_lcr.8.1.ufo.BOUNDED-16.pals.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,3 +XCSP,xcsp/aim-200-1-6-sat-4.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Sequentialized,seq-mthreaded/pals_opt-floodmax.3.1.ufo.BOUNDED-6.pals.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,5 +XCSP,xcsp/aim-200-2-0-sat-2.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Sequentialized,seq-mthreaded/pals_opt-floodmax.3.3.ufo.BOUNDED-6.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,9 +XCSP,xcsp/aim-200-2-0-sat-4.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Sequentialized,seq-mthreaded/pals_opt-floodmax.3.4.ufo.UNBOUNDED.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,8 +XCSP,xcsp/aim-200-3-4-sat-2.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +Sequentialized,seq-mthreaded/pals_opt-floodmax.4.1.ufo.UNBOUNDED.pals.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,40 +XCSP,xcsp/aim-200-3-4-sat-4.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +Sequentialized,seq-mthreaded/pals_opt-floodmax.4.3.ufo.BOUNDED-8.pals.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,38 +XCSP,xcsp/aim-200-6-0-sat-2.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +Sequentialized,seq-mthreaded/pals_opt-floodmax.4_overflow.ufo.UNBOUNDED.pals.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,42 +XCSP,xcsp/aim-200-6-0-sat-4.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +Sequentialized,seq-mthreaded/pals_opt-floodmax.5.2.ufo.BOUNDED-10.pals.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +Arrays,array-programs/copysome2-2.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,48 +ECA,eca-rers2012/Problem18_label57.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,51 +Loops,nla-digbench-scaling/ps6-ll_unwindbound10.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ProductLines,product-lines/email_spec9_product22.cil.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +Sequentialized,seq-pthread/cs_stack-2.i,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +Arrays,array-examples/sorting_bubblesort_2_ground.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,48 +ECA,eca-programs/Problem101_label02.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,71 +Loops,loop-acceleration/multivar_1-2.c,ILP32,false,ERROR,COVERED,100.0,1,1,26 +ProductLines,product-lines/elevator_spec14_product28.cil.c,ILP32,false,FAILED,COVERED,100.0,1,1,1 +Sequentialized,seq-mthreaded/pals_STARTPALS_ActiveStandby.5.ufo.UNBOUNDED.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +Arrays,array-examples/standard_copy5_ground-2.i,ILP32,false,UNKNOWN,NOT_COVERED,,,0,48 +ECA,eca-programs/Problem101_label21.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,86 +Loops,loops/array-2.c,ILP32,false,FAILED,COVERED,100.0,1,1,18 +ProductLines,product-lines/elevator_spec1_product28.cil.c,ILP32,false,FAILED,COVERED,100.0,1,1,1 +Sequentialized,seq-mthreaded/pals_floodmax.3.2.ufo.BOUNDED-6.pals.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,9 +Arrays,array-fpi/brs2f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ECA,eca-programs/Problem102_label46.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,82 +Loops,loops/linear_search.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +ProductLines,product-lines/elevator_spec2_product26.cil.c,ILP32,false,FAILED,COVERED,100.0,1,1,2 +Sequentialized,seq-mthreaded/pals_floodmax.4.1.ufo.UNBOUNDED.pals.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,44 +Arrays,array-fpi/condnf.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +ECA,eca-rers2012/Problem03_label09.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,24 +Loops,loops/sum03-1.i,ILP32,false,FAILED,COVERED,100.0,1,1,23 +ProductLines,product-lines/elevator_spec3_product20.cil.c,ILP32,false,FAILED,COVERED,100.0,1,1,1 +Sequentialized,seq-mthreaded/pals_floodmax.5.1.ufo.BOUNDED-10.pals.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,61 +Arrays,array-fpi/ifeqn3f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,7 +ECA,eca-rers2012/Problem04_label26.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,51 +Loops,nla-digbench-scaling/bresenham-ll_unwindbound1.c,ILP32,false,FAILED,COVERED,100.0,1,1,14 +ProductLines,product-lines/elevator_spec3_productSimulator.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Sequentialized,seq-mthreaded/pals_floodmax.5.4.ufo.UNBOUNDED.pals.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,63 +Arrays,array-fpi/modnf.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ECA,eca-rers2012/Problem05_label36.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,52 +Loops,nla-digbench-scaling/egcd-ll_unwindbound1.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ProductLines,product-lines/email_spec0_product26.cil.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +Sequentialized,seq-mthreaded/pals_lcr-var-start-time.4.1.ufo.UNBOUNDED.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +Arrays,array-fpi/ms5f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ECA,eca-rers2012/Problem06_label11.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,52 +Loops,nla-digbench-scaling/egcd-ll_unwindbound50.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,16 +ProductLines,product-lines/email_spec11_product20.cil.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +Sequentialized,seq-mthreaded/pals_lcr-var-start-time.6.1.ufo.BOUNDED-12.pals.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,3 +Arrays,array-fpi/s12iff.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ECA,eca-rers2012/Problem07_label05.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,65 +Loops,nla-digbench-scaling/egcd2-ll_unwindbound5.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,15 +ProductLines,product-lines/email_spec1_product14.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Sequentialized,seq-mthreaded/pals_lcr.4_overflow.ufo.UNBOUNDED.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,2 +Arrays,array-fpi/s42iff.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ECA,eca-rers2012/Problem07_label48.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,68 +Loops,nla-digbench-scaling/egcd3-ll_unwindbound20.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +ProductLines,product-lines/email_spec1_product30.cil.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +Sequentialized,seq-mthreaded/pals_lcr.7.1.ufo.BOUNDED-14.pals.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,2 +Arrays,array-fpi/sina2f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ECA,eca-rers2012/Problem08_label46.c,ILP32,false,FAILED,TESTCOV_ERROR,,,1,84 +Loops,nla-digbench-scaling/fermat1-ll_unwindbound2.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +ProductLines,product-lines/email_spec27_product19.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +Sequentialized,seq-mthreaded/pals_opt-floodmax.3.1.ufo.UNBOUNDED.pals.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,4 +Arrays,array-fpi/ss3f.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ECA,eca-rers2012/Problem09_label38.c,ILP32,false,FAILED,TESTCOV_ERROR,,,1,89 +Loops,nla-digbench-scaling/fermat2-ll_unwindbound2.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,58 +ProductLines,product-lines/email_spec27_product32.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Sequentialized,seq-mthreaded/pals_opt-floodmax.3_overflow.ufo.UNBOUNDED.pals.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,8 +Arrays,array-tiling/mlceu.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,3 +ECA,eca-rers2012/Problem10_label47.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,52 +Loops,nla-digbench-scaling/freire2_valuebound100.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ProductLines,product-lines/email_spec3_product19.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +Sequentialized,seq-mthreaded/pals_opt-floodmax.4.4.ufo.UNBOUNDED.pals.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,44 +Arrays,reducercommutativity/rangesum60.i,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +ECA,eca-rers2012/Problem11_label58.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,52 +Sequentialized,seq-mthreaded/pals_opt-floodmax.5.2.ufo.UNBOUNDED.pals.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,62 +Loops,nla-digbench-scaling/mannadiv_unwindbound1.c,ILP32,false,SUCCEEDED,NOT_COVERED,,,0,1 +ProductLines,product-lines/email_spec3_product33.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +ECA,eca-rers2012/Problem12_label40.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,52 +Sequentialized,seq-pthread/cs_fib_longer-2.i,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,2 +Loops,nla-digbench-scaling/mannadiv_unwindbound50.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,1 +ProductLines,product-lines/email_spec4_product25.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +ECA,eca-rers2012/Problem13_label30.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,52 +ECA,eca-rers2012/Problem13_label44.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,53 +ECA,eca-rers2012/Problem15_label02.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,50 +ECA,eca-rers2012/Problem15_label22.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +ECA,eca-rers2012/Problem15_label37.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,52 +ECA,eca-rers2012/Problem15_label48.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,50 +ECA,eca-rers2012/Problem16_label05.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,48 +ECA,eca-rers2012/Problem16_label20.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +ECA,eca-rers2012/Problem16_label38.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,50 +ECA,eca-rers2012/Problem16_label52.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,50 +ECA,eca-rers2012/Problem17_label20.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +ECA,eca-rers2012/Problem17_label33.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,51 +ECA,eca-rers2012/Problem17_label50.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,50 +ECA,eca-rers2012/Problem18_label01.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,50 +ECA,eca-rers2012/Problem18_label19.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +ECA,eca-rers2012/Problem18_label33.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +ProductLines,product-lines/email_spec8_product34.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,48 +ECA,eca-rers2012/Problem19_label27.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,51 +ProductLines,product-lines/email_spec9_product26.cil.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +ECA,eca-rers2018/Problem15.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,63 +ProductLines,product-lines/elevator_spec14_product32.cil.c,ILP32,false,FAILED,COVERED,100.0,1,1,1 +ECA,eca-programs/Problem101_label06.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,64 +ProductLines,product-lines/elevator_spec2_product22.cil.c,ILP32,false,FAILED,COVERED,100.0,1,1,1 +ECA,eca-programs/Problem101_label16.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,65 +ProductLines,product-lines/elevator_spec3_product27.cil.c,ILP32,false,FAILED,COVERED,100.0,1,1,2 +ECA,eca-programs/Problem102_label04.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,79 +ProductLines,product-lines/email_spec0_product21.cil.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,48 +ECA,eca-programs/Problem102_label28.c,ILP32,false,FAILED,TESTCOV_ERROR,,,1,86 +ProductLines,product-lines/email_spec11_product35.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 +ECA,eca-programs/Problem102_label51.c,ILP32,false,FAILED,TESTCOV_ERROR,,,1,88 +ProductLines,product-lines/email_spec1_product31.cil.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +ECA,eca-programs/Problem103_label53.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,88 +ProductLines,product-lines/email_spec27_product25.cil.c,ILP32,false,UNKNOWN,NOT_COVERED,,,0,49 +ECA,eca-rers2012/Problem03_label31.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,15 +ProductLines,product-lines/email_spec3_product17.cil.c,ILP32,false,FAILED,NOT_COVERED,0.0,1,1,49 From 5bbf32878c6701722363608c0e269ecdff843050 Mon Sep 17 00:00:00 2001 From: Guilherme Bernardo Date: Sun, 23 Aug 2026 13:33:15 -0400 Subject: [PATCH 07/17] fix(frontend): propagate the analysis exit code; --expected-result returns 4 main() discarded the return value of map2check_execution() at all three call sites, so the process always exited 0 -- or SIGABRT from the --expected-result path, which used abort(). The Test-Comp tool-info decides DONE vs ERROR from exit_code == 0, so an internally failed run was reported DONE; and a --expected-result mismatch signalled with a signal, which BenchExec reads as "the tool crashed" -- the opposite of the truth. main() now returns the code from map2check_execution(), and a mismatch returns the new ERROR_EXPECTED_RESULT (4) instead of abort(). Exit 0 still means "the analysis ran to a verdict": SV-COMP parses the verdict from stdout, and any non-zero code is read as a crash by BenchExec, so the verdict must not migrate into the exit code. tests/integration/test_exit_codes.sh pins the contract: a FALSE verdict keeps exit 0 (verdict on stdout), a matching --expected-result exits 0, and a mismatch exits 4 rather than 134. --- .github/workflows/ci.yml | 1 + modules/frontend/map2check.cpp | 26 ++++++++-- tests/integration/test_exit_codes.sh | 78 ++++++++++++++++++++++++++++ 3 files changed, 101 insertions(+), 4 deletions(-) create mode 100644 tests/integration/test_exit_codes.sh diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 08dfa4708..1749f9ded 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -367,6 +367,7 @@ jobs: bash tests/integration/test_target_coverage.sh bash tests/integration/test_ktest_object_names.sh bash tests/integration/test_testcomp_regressions.sh + bash tests/integration/test_exit_codes.sh ' # =========================================================== diff --git a/modules/frontend/map2check.cpp b/modules/frontend/map2check.cpp index ae736f365..a59ee59e8 100644 --- a/modules/frontend/map2check.cpp +++ b/modules/frontend/map2check.cpp @@ -51,6 +51,15 @@ const size_t ERROR_IN_COMMAND_LINE = 1; * have. Distinct from ERROR_IN_COMMAND_LINE so callers can tell "you typed it * wrong" from "this binary cannot do that", and act differently. */ const size_t ERROR_UNAVAILABLE_CAPABILITY = 3; +/** --expected-result recorded a verdict different from the one asked for. + * + * This used to be signalled with abort(), i.e. SIGABRT (exit 134). A signal + * tells BenchExec "the tool crashed", which is the opposite of the truth: the + * tool ran to a definitive verdict and that verdict disagreed with the + * harness's expectation. A distinct exit code lets the harness distinguish + * "wrong answer" from "broken run". Kept at 4 so it never collides with the + * command-line (1) and capability (3) codes, and never looks like success. */ +const size_t ERROR_EXPECTED_RESULT = 4; // A helper function to simplify the main part. template std::ostream &operator<<(std::ostream &os, const std::vector &v) { @@ -550,7 +559,7 @@ int map2check_execution(map2check_args args) { if (args.expectedResult != "") { if (args.expectedResult != counterExample->getViolatedProperty()) { Map2Check::Log::Fatal("Expected result failed"); - abort(); + return ERROR_EXPECTED_RESULT; } } @@ -781,14 +790,23 @@ z3 (Z3 is default), btor (Boolector), and yices2 (Yices))") args.inputFile = absolute_path.string(); if(args.generator == Map2Check::NonDetGenerator::None) { args.generator = Map2Check::NonDetGenerator::LibFuzzer; - map2check_execution(args); + int result = map2check_execution(args); + if (result != SUCCESS) { + return result; + } if (!foundViolation) { args.generator = Map2Check::NonDetGenerator::Klee; - map2check_execution(args); + result = map2check_execution(args); + if (result != SUCCESS) { + return result; + } } } else { - map2check_execution(args); + int result = map2check_execution(args); + if (result != SUCCESS) { + return result; + } } } } catch (std::exception &e) { diff --git a/tests/integration/test_exit_codes.sh b/tests/integration/test_exit_codes.sh new file mode 100644 index 000000000..a315d4742 --- /dev/null +++ b/tests/integration/test_exit_codes.sh @@ -0,0 +1,78 @@ +#!/bin/bash +# test_exit_codes.sh — the process exit code must reflect how the run ended. +# +# Finding G: main() discarded the return value of map2check_execution() at all +# three call sites, so the exit code was always 0 (or SIGABRT from the +# --expected-result path). Two consequences: +# +# 1. The Test-Comp tool-info (utils/moduleBenchExec/map2check_testcomp.py) +# decides DONE vs ERROR from `exit_code == 0`, so an internally failed run +# was reported DONE. +# 2. --expected-result signalled a wrong answer with abort() (SIGABRT, 134), +# which BenchExec reads as "the tool crashed" -- the opposite of the truth. +# +# The convention this test pins down: exit 0 means "the analysis ran to a +# verdict, whatever it was" (the verdict lives in stdout, which is what SV-COMP +# parses), and non-zero means "the run itself failed or disagreed with the +# harness", with --expected-result getting its own code (4). + +set -u + +MAP2CHECK_DIR="${MAP2CHECK_PATH:-/workspace/install_e2e}" +MAP2CHECK="$MAP2CHECK_DIR/map2check" +CLANG="${CLANG:-/usr/bin/clang-16}" +TMPDIR=$(mktemp -d) +trap 'rm -rf "$TMPDIR"' EXIT + +PASSED=0 +FAILED=0 +ok() { echo " PASS $1"; PASSED=$((PASSED+1)); } +fail() { echo " FAIL $1: $2"; FAILED=$((FAILED+1)); } + +cat > "$TMPDIR/ov_bug.c" <<'EOF' +#include +int main(void) { int x = 2147483647; int y = x + 1; return y; } +EOF + +"$CLANG" -c -emit-llvm -g -O0 "$TMPDIR/ov_bug.c" -o "$TMPDIR/ov_bug.bc" 2>/dev/null || { + echo " SKIP: compile failed" + exit 1 +} + +echo "=== Exit-code contract ===" + +# 1. A definitive FALSE is still exit 0 -- the verdict is in stdout, not the code. +output=$("$MAP2CHECK" --nondet-generator symex --check-overflow --timeout 60 \ + "$TMPDIR/ov_bug.bc" 2>&1) +rc=$? +if [ "$rc" -eq 0 ] && echo "$output" | grep -qE "VERIFICATION FAILED|OVERFLOW"; then + ok "FALSE verdict keeps exit code 0 (verdict in stdout)" +else + fail "FALSE exit code" "rc=$rc (want 0 with FALSE on stdout)" + echo "$output" | tail -5 +fi + +# 2. A matching --expected-result is exit 0. +output=$("$MAP2CHECK" --nondet-generator symex --check-overflow --timeout 60 \ + --expected-result OVERFLOW "$TMPDIR/ov_bug.bc" 2>&1) +rc=$? +if [ "$rc" -eq 0 ]; then + ok "--expected-result OVERFLOW (matching) exits 0" +else + fail "matching expected-result" "rc=$rc (want 0)" +fi + +# 3. A mismatching --expected-result is a distinct exit code, not SIGABRT. +output=$("$MAP2CHECK" --nondet-generator symex --check-overflow --timeout 60 \ + --expected-result FALSE-FREE "$TMPDIR/ov_bug.bc" 2>&1) +rc=$? +if [ "$rc" -eq 4 ]; then + ok "--expected-result mismatch exits 4 (not SIGABRT 134)" +elif [ "$rc" -eq 134 ]; then + fail "mismatching expected-result" "rc=134 (SIGABRT) -- abort() still in place" +else + fail "mismatching expected-result" "rc=$rc (want 4)" +fi + +echo "=== Results: $PASSED passed, $FAILED failed ===" +[ "$FAILED" -eq 0 ] && exit 0 || exit 1 From 3bc2ca2ffaf3ade8a6f860269b67dfabf9acbdd5 Mon Sep 17 00:00:00 2001 From: Guilherme Bernardo Date: Sun, 23 Aug 2026 13:33:34 -0400 Subject: [PATCH 08/17] fix(castle): stop mapping unsupported CWEs to the degenerate reachability mode Thirteen CWEs were mapped to `--target-function --target-function-name main`, which is degenerate (finding B): TargetPass instruments call sites whose callee is the named function, and a program never calls its own main, so nothing was instrumented and every vulnerable case was reported TRUE. On the v6 baseline that earned 12 fake true negatives and 5 guaranteed false negatives. 628/674/770/835 are termination/resource properties Map2Check does not model; the other nine are injection/crypto categories outside its scorable set. They now live in OUT_OF_SCOPE_CWES and are skipped outright (recorded N/A) instead of run under the degenerate mode and burning a full budget per case. tests/integration/test_cwe_mode_mapping.sh asserts no CWE maps to --target-function, that 628/674/770/835 are out of scope, and that every benchmark CWE is explicitly mapped or declared out of scope. --- .github/workflows/ci.yml | 1 + tests/castle/run_castle_evaluation.sh | 56 ++++++++------ tests/integration/test_cwe_mode_mapping.sh | 87 ++++++++++++++++++++++ 3 files changed, 123 insertions(+), 21 deletions(-) create mode 100644 tests/integration/test_cwe_mode_mapping.sh diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 1749f9ded..faaa8793d 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -368,6 +368,7 @@ jobs: bash tests/integration/test_ktest_object_names.sh bash tests/integration/test_testcomp_regressions.sh bash tests/integration/test_exit_codes.sh + bash tests/integration/test_cwe_mode_mapping.sh ' # =========================================================== diff --git a/tests/castle/run_castle_evaluation.sh b/tests/castle/run_castle_evaluation.sh index 865e675af..308b29e76 100755 --- a/tests/castle/run_castle_evaluation.sh +++ b/tests/castle/run_castle_evaluation.sh @@ -13,6 +13,15 @@ RESULTS_DIR="${RESULTS_DIR:-$SCRIPT_DIR/results}" TIMEOUT_SEC=360 # CWE → mode mapping (single-pass flags, --add-invariants added on UNKNOWN) +# +# Only CWEs Map2Check actually supports carry a mode here. Everything else is +# declared in OUT_OF_SCOPE_CWES and skipped, because the old fallback for those +# was `--target-function --target-function-name main`, which is degenerate +# (finding B): TargetPass instruments call sites whose callee is the named +# function, a program never calls its own main, so nothing was instrumented and +# every vulnerable case was reported TRUE. A TRUE from an oracle that can only +# say "correct" earned 12 fake true negatives and 5 guaranteed false negatives +# on the v6 baseline. declare -A CWE_MODE=( # Memory safety (memtrack) [125]="--memtrack" @@ -30,24 +39,14 @@ declare -A CWE_MODE=( [401]="--memcleanup-property" # Assert [617]="--check-asserts" - # Reachability (other) - [628]="--target-function --target-function-name main" - [674]="--target-function --target-function-name main" - [770]="--target-function --target-function-name main" - [835]="--target-function --target-function-name main" - # Out of scope (still run for completeness, flagged N/A) - [22]="--target-function --target-function-name main" - [78]="--target-function --target-function-name main" - [89]="--target-function --target-function-name main" - [134]="--target-function --target-function-name main" - [253]="--target-function --target-function-name main" - [327]="--target-function --target-function-name main" - [362]="--target-function --target-function-name main" - [522]="--target-function --target-function-name main" - [798]="--target-function --target-function-name main" ) -OUT_OF_SCOPE_CWES=(22 78 89 134 253 327 362 522 798) +# Unsupported CWEs. 628/674/770/835 are termination/resource properties (loop, +# recursion, unbounded workers) that Map2Check does not model; the rest are +# injection/crypto categories outside SV-COMP's scorable set for this tool. +# These are NOT run -- running them under the degenerate reachability mode only +# produced misleading TRUEs and burned a full budget per case for nothing. +OUT_OF_SCOPE_CWES=(22 78 89 134 253 327 362 522 628 674 770 798 835) is_out_of_scope() { local cwe="$1" @@ -138,7 +137,23 @@ for t in data['tests']: vulnerable=$(echo "$entry" | python3 -c "import sys,json; print(json.loads(sys.stdin.read())['vulnerable'])") cwe=$(echo "$entry" | python3 -c "import sys,json; print(json.loads(sys.stdin.read())['cwe'])") - mode_flags="${CWE_MODE[$cwe]:---target-function --target-function-name main}" + # Unsupported CWEs are skipped outright, not run under a degenerate mode. + # Recorded as N/A so the row exists for completeness and for the resume map, + # but no budget is spent and no verdict is invented. + if is_out_of_scope "$cwe"; then + echo "$id_short,$cwe,$name,$vulnerable,N/A,,,,N/A,0.0" >> "$RESULTS_DIR/castle_results.csv" + NA=$((NA+1)) + echo " [$RUN/250] $name (CWE-$cwe): N/A (out of scope)" + continue + fi + + mode_flags="${CWE_MODE[$cwe]:-}" + if [ -z "$mode_flags" ]; then + echo " ERROR: no mode mapping for CWE-$cwe ($name) -- fix CWE_MODE or OUT_OF_SCOPE_CWES" >&2 + echo "$id_short,$cwe,$name,$vulnerable,UNMAPPED,,,,ERROR,0.0" >> "$RESULTS_DIR/castle_results.csv" + ERR=$((ERR+1)) + continue + fi # run_isolated gives each invocation a private CWD and captures to a file. # @@ -207,10 +222,9 @@ for t in data['tests']: reported_line=$(echo "$output" | grep -oP 'map2check_property line \K\d+' | head -1) # --- Classify --- - if is_out_of_scope "$cwe"; then - classification="N/A" - NA=$((NA+1)) - elif [ "$verdict" = "TIMEOUT" ]; then + # Out-of-scope CWEs were skipped above (continue), so every case reaching + # here is in scope and gets a verdict-driven classification. + if [ "$verdict" = "TIMEOUT" ]; then classification="TIMEOUT" TO=$((TO+1)) elif [ "$verdict" = "ERROR" ]; then diff --git a/tests/integration/test_cwe_mode_mapping.sh b/tests/integration/test_cwe_mode_mapping.sh new file mode 100644 index 000000000..295c7cfbf --- /dev/null +++ b/tests/integration/test_cwe_mode_mapping.sh @@ -0,0 +1,87 @@ +#!/bin/bash +# test_cwe_mode_mapping.sh — the CASTLE CWE→mode mapping must never fall back to +# reachability mode. +# +# Finding B: 13 CWEs were mapped to `--target-function --target-function-name +# main`, which is degenerate — TargetPass instruments call sites whose callee is +# the named function, and a program never calls its own main. On the v6 baseline +# 98 of 217 runs went through that mode and produced zero FALSEs on 59 known +# vulnerable programs, earning 12 fake true negatives and 5 guaranteed false +# negatives. +# +# The contract this pins down: +# 1. no CWE maps to a reachability target of `main` (or any reachability mode); +# 2. 628/674/770/835 (termination/resource) are declared out of scope; +# 3. every CWE in the benchmark has an explicit mode or an out-of-scope +# declaration — nothing can silently fall through unmapped. + +set -u + +SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" +RUNNER="$SCRIPT_DIR/../castle/run_castle_evaluation.sh" +JSON="$SCRIPT_DIR/../castle/CASTLE-Benchmark/datasets/CASTLE-C250.min.json" + +[ -f "$RUNNER" ] || { echo " FAIL: $RUNNER not found"; exit 1; } +[ -f "$JSON" ] || { echo " FAIL: $JSON not found"; exit 1; } + +PASSED=0 +FAILED=0 +ok() { echo " PASS $1"; PASSED=$((PASSED+1)); } +fail() { echo " FAIL $1: $2"; FAILED=$((FAILED+1)); } + +# Source the two declarations in a subshell so the runner itself never runs. +eval "$(sed -n '/declare -A CWE_MODE=/,/^)/p' "$RUNNER")" +eval "$(grep '^OUT_OF_SCOPE_CWES=' "$RUNNER")" + +echo "=== CWE→mode mapping contract ===" + +# 1. No mode may name a reachability target. +offender="" +for cwe in "${!CWE_MODE[@]}"; do + case "${CWE_MODE[$cwe]}" in + *target-function*) offender="$offender $cwe";; + esac +done +if [ -n "$offender" ]; then + fail "no reachability target" "CWE(s)$offender still use --target-function" +else + ok "no CWE maps to --target-function (finding B guard)" +fi + +# 2. The termination/resource CWEs must be declared out of scope. +missing="" +for cwe in 628 674 770 835; do + found="" + for o in "${OUT_OF_SCOPE_CWES[@]}"; do + [ "$o" = "$cwe" ] && found=1 + done + [ -n "$found" ] || missing="$missing $cwe" +done +if [ -n "$missing" ]; then + fail "termination CWEs out of scope" "missing from OUT_OF_SCOPE_CWES:$missing" +else + ok "628/674/770/835 declared out of scope" +fi + +# 3. Every CWE in the benchmark is either mapped or out of scope. +bench_cwes=$(python3 -c " +import json +data = json.load(open('$JSON')) +print(' '.join(sorted({str(t['cwe']) for t in data['tests']}))) +") +unmapped="" +for cwe in $bench_cwes; do + in_mode="" + for k in "${!CWE_MODE[@]}"; do [ "$k" = "$cwe" ] && in_mode=1; done + in_scope="" + for o in "${OUT_OF_SCOPE_CWES[@]}"; do [ "$o" = "$cwe" ] && in_scope=1; done + if [ -z "$in_mode" ] && [ -z "$in_scope" ]; then unmapped="$unmapped $cwe"; fi +done +if [ -n "$unmapped" ]; then + fail "full coverage" "CWE(s) neither mapped nor out of scope:$unmapped" +else + ok "every benchmark CWE is mapped or declared out of scope" +fi + +echo "=== Results: $PASSED passed, $FAILED failed ===" +[ "$FAILED" -eq 0 ] && exit 0 || exit 1 From 9cfca0f70a303294f7b1bf949c2d5e2c7f768435 Mon Sep 17 00:00:00 2001 From: Guilherme Bernardo Date: Sun, 23 Aug 2026 14:00:14 -0400 Subject: [PATCH 09/17] fix(verdict): an assumption is not a violation, and an empty verdict is not one either Three defects behind the Cover-Error recall of 13.9%, all found by following one unconfirmed FAILED case down to its cause. The verdict could come from uninitialised memory. CheckViolatedProperty::propertyViolated had no initialiser. The constructor assigns it on several paths but NOT when map2check_property exists and is empty, nor when it holds a line the parser does not recognise -- the `else` that would have caught that is commented out. Both cases left the field holding whatever was on the stack, and that value decided the verdict. It is reachable: a KLEE state that aborts early creates the file without writing to it. Observed as the same program answering FAILED, SUCCEEDED and nothing at all across three runs, and it is the best candidate for the ERROR/UNKNOWN churn that shows up in every baseline comparison. Defaulted to UNKNOWN, and the unrecognised path now says so instead of falling through in silence. An assumption was being read as a violation. The SV-COMP idiom is void assume_abort_if_not(int cond) { if (!cond) abort(); } and KLEE runs here with --exit-on-error-type=Abort, so the first path that violates an assumption halted the entire search AND left an abort.err that looks exactly like a real violation. NonDetPass already renames __VERIFIER_assume, which works because that one is only declared; these carry their own body, so renaming would collide with the runtime's definition. The body is replaced instead -- one rewrite reaches every call site. Measured over the 818-task corpus, the categories saturated with this idiom are exactly the ones with no recall: Sequentialized 86% of tasks and 0 confirmed, Floats 82% and 0, Arrays 71% and 1. And the .ktest fallback added yesterday was too permissive -- my own bug. It accepted ANY .err as the violating path, so an assumption abort was recovered and emitted with full confidence: one suite carried 25 inputs and covered 0.0%. It now accepts only .abort.err, and only when the runtime actually recorded a violation. Verified against the two cases that motivated it: pals_lcr.6_overflow v8: FAILED, 25 bogus inputs, not covered v9: UNKNOWN, 0 inputs <- false positive gone ps6-ll_unwindbound20 v8: FAILED, TestCov UNKNOWN v9: FAILED, TestCov TRUE <- now confirmed test_testcomp_regressions.sh is at 10 assertions; the new ones put the error behind two assumptions, so a run that still halts on the first cannot reach it. Co-Authored-By: Claude Opus 5 (1M context) --- modules/backend/pass/NonDetPass.cpp | 66 ++++++++++++++ modules/frontend/map2check.cpp | 14 ++- modules/frontend/test_suite/ktest_reader.cpp | 15 +++- modules/frontend/utils/tools.cpp | 10 ++- modules/frontend/utils/tools.hpp | 21 ++++- .../integration/test_testcomp_regressions.sh | 86 +++++++++++++++++++ 6 files changed, 203 insertions(+), 9 deletions(-) mode change 100755 => 100644 modules/frontend/utils/tools.cpp mode change 100755 => 100644 modules/frontend/utils/tools.hpp diff --git a/modules/backend/pass/NonDetPass.cpp b/modules/backend/pass/NonDetPass.cpp index 9ef33a54b..a76ea69f6 100644 --- a/modules/backend/pass/NonDetPass.cpp +++ b/modules/backend/pass/NonDetPass.cpp @@ -28,8 +28,74 @@ inline Instruction *BBIteratorToInst(BasicBlock::iterator i) { } } // namespace +namespace { + +/** Names the benchmarks use for "prune this path", implemented as an abort. + * + * `__VERIFIER_assume` is usually only declared, and the call-site rewriting + * below handles that. These are DEFINED in the program -- the SV-COMP sources + * carry their own body: + * + * void assume_abort_if_not(int cond) { if (!cond) abort(); } + * + * so renaming them would collide with the runtime's own definition. The body + * is replaced instead, which reaches every call site at once. */ +const char *const kAssumeByAbort[] = {"assume_abort_if_not", + "assume_abort_if_not_", + "__VERIFIER_assume"}; + +/** Rewrites such a function to a real assume. Returns true if it rewrote. + * + * Aborting on a failed assumption is not merely slow, it is wrong twice over. + * KLEE runs here with --exit-on-error-type=Abort, so the FIRST path that + * violates an assumption halts the whole search -- discarding every path not + * yet explored. And the abort leaves an abort.err behind that looks exactly + * like a real violation, which is how a path the competition considers out of + * scope came back as a test case: one measured suite carried 25 inputs and + * covered 0.0%. + * + * Measured over the 818-task Cover-Error corpus, the categories saturated with + * this idiom are the ones with no recall at all -- Sequentialized 86% of tasks + * and 0 confirmed, Floats 82% and 0, Arrays 71% and 1. */ +bool rewriteAssumeToPrune(Function &F) { + if (F.isDeclaration()) return false; + bool named = false; + for (const char *candidate : kAssumeByAbort) { + if (F.getName() == candidate) { + named = true; + break; + } + } + if (!named) return false; + if (F.arg_size() != 1 || !F.getArg(0)->getType()->isIntegerTy()) return false; + if (!F.getReturnType()->isVoidTy()) return false; + + LLVMContext &ctx = F.getContext(); + llvm::Type *int32 = llvm::Type::getInt32Ty(ctx); + llvm::Value *condition = F.getArg(0); + + F.deleteBody(); + BasicBlock *entry = BasicBlock::Create(ctx, "entry", &F); + IRBuilder<> builder(entry); + llvm::FunctionCallee assume = F.getParent()->getOrInsertFunction( + "map2check_assume", llvm::Type::getVoidTy(ctx), int32); + builder.CreateCall(assume, {builder.CreateSExtOrTrunc(condition, int32)}); + builder.CreateRetVoid(); + llvm::errs() << "[map2check] rewrote " << F.getName() + << " to prune the path instead of aborting\n"; + return true; +} + +} // namespace + PreservedAnalyses NonDetPass::run(Function &F, llvm::FunctionAnalysisManager &AM) { + // Before anything else: this replaces the whole body, so instrumenting the + // old one first would be work thrown away. + if (rewriteAssumeToPrune(F)) { + return PreservedAnalyses::none(); + } + this->nonDetFunctions = make_unique(&F, &F.getContext()); bool initializedFunctionName = false; for (Function::iterator bb = F.begin(), e = F.end(); bb != e; ++bb) { diff --git a/modules/frontend/map2check.cpp b/modules/frontend/map2check.cpp index a59ee59e8..23d065e73 100644 --- a/modules/frontend/map2check.cpp +++ b/modules/frontend/map2check.cpp @@ -216,7 +216,19 @@ void emitTestSuite(const std::string &outputDir, const std::string &programFile, // The log stays the first choice where it exists: it records values in the // order the program consumed them, straight from the instrumented run, while // the .ktest is KLEE's view of the same path. - if (inputs.empty()) { + // + // Gated on the runtime having actually recorded the target as reached, and + // that gate is the whole difference between recovering a vector and + // inventing one. KLEE writes a .err for ANY abort, and the sv-benchmarks + // idiom for an assumption is + // + // void assume_abort_if_not(int c) { if (!c) abort(); } + // + // so a path that merely violates an assumption -- a path the competition + // considers out of scope -- leaves an abort.err behind exactly like a real + // violation does. Taking it produced suites that looked convincing and + // covered nothing: one measured case emitted 25 inputs and scored 0.0%. + if (inputs.empty() && foundViolation) { inputs = Map2Check::readViolatingKtest(Map2Check::kleeOutputDir); if (!inputs.empty()) { Map2Check::Log::Info( diff --git a/modules/frontend/test_suite/ktest_reader.cpp b/modules/frontend/test_suite/ktest_reader.cpp index 4a91a563b..fe0454e35 100644 --- a/modules/frontend/test_suite/ktest_reader.cpp +++ b/modules/frontend/test_suite/ktest_reader.cpp @@ -176,16 +176,23 @@ std::vector readViolatingKtest(const std::string& kleeOutDir) { // testNNNNNN.ktest -- .abort.err for an abort, .ptr.err for a bad // dereference, and so on. The stem up to the first dot is the path number, // which is the whole link between the two files. + // + // Only .abort.err. KLEE writes .ptr.err, .free.err, .div.err and others for + // failures that are not this tool's target, and taking any of them would + // return the vector of an unrelated path. Even .abort.err is not proof on + // its own -- the sv-benchmarks assumption idiom aborts too -- which is why + // the caller gates this on the runtime having recorded a violation. std::vector candidates; + const std::string kAbortSuffix = ".abort.err"; for (const auto& entry : std::filesystem::directory_iterator(kleeOutDir, error)) { const std::string name = entry.path().filename().string(); - if (name.size() < 5 || name.compare(name.size() - 4, 4, ".err") != 0) { + if (name.size() <= kAbortSuffix.size()) continue; + if (name.compare(name.size() - kAbortSuffix.size(), kAbortSuffix.size(), + kAbortSuffix) != 0) { continue; } - const size_t dot = name.find('.'); - if (dot == std::string::npos) continue; - candidates.push_back(name.substr(0, dot)); + candidates.push_back(name.substr(0, name.size() - kAbortSuffix.size())); } // Sorted so that a run with several failing paths always yields the same // one. Arbitrary, but arbitrary-and-stable beats arbitrary-and-not: a suite diff --git a/modules/frontend/utils/tools.cpp b/modules/frontend/utils/tools.cpp old mode 100755 new mode 100644 index 2614e41f9..ae4ea83b1 --- a/modules/frontend/utils/tools.cpp +++ b/modules/frontend/utils/tools.cpp @@ -241,7 +241,15 @@ Tools::CheckViolatedProperty::CheckViolatedProperty(string path) { Map2Check::Log::Debug("ASSERT found"); this->propertyViolated = Tools::PropertyViolated::ASSERT; } else { - // throw Tools::CheckViolatedPropertyException("Invalid Property"); + // Unrecognised, including the empty-file case where this loop never + // runs at all. Left silent for years while the field was + // uninitialised, so the verdict came from the stack. Saying so is + // cheap and the difference between a bug that is found and one that + // is not. + Map2Check::Log::Debug( + "map2check_property holds no verdict this parser recognises " + "(\"" + line + "\") -- treating the run as undecided"); + this->propertyViolated = Tools::PropertyViolated::UNKNOWN; } break; case 1: diff --git a/modules/frontend/utils/tools.hpp b/modules/frontend/utils/tools.hpp old mode 100755 new mode 100644 index 9cb683efa..f2dd8c28c --- a/modules/frontend/utils/tools.hpp +++ b/modules/frontend/utils/tools.hpp @@ -104,10 +104,25 @@ enum class PropertyViolated { /** Class used to check violated property */ struct CheckViolatedProperty { - /** Current violated property */ - PropertyViolated propertyViolated; + /** Current violated property. + * + * Defaulted, and the default matters more than it looks. The constructor + * assigns this on several paths but NOT when map2check_property exists and + * is empty, nor when it holds a line the parser does not recognise -- the + * `else` that would have caught that is commented out. Both cases used to + * leave the field holding whatever was on the stack, and that value decided + * the tool's verdict. + * + * It is reachable: a KLEE state that aborts early creates the file without + * writing a verdict into it. Observed as the same program answering FAILED, + * SUCCEEDED and nothing across three runs, and as the ERROR/UNKNOWN churn + * that shows up in every baseline comparison. + * + * UNKNOWN is the honest default: no readable verdict means the tool did not + * decide, which is exactly what the missing-file path already reports. */ + PropertyViolated propertyViolated = PropertyViolated::UNKNOWN; /** Line where property was violated */ - unsigned line; + unsigned line = 0; /** Name of the function where the property was violated */ string function_name; string path_name; diff --git a/tests/integration/test_testcomp_regressions.sh b/tests/integration/test_testcomp_regressions.sh index 1d11d8a14..0d34a7f88 100755 --- a/tests/integration/test_testcomp_regressions.sh +++ b/tests/integration/test_testcomp_regressions.sh @@ -220,6 +220,92 @@ else tail -3 "$WORK/deep/run.log" | sed 's/^/ /' fi +# --- 7. an assumption must prune the path, not abort ------------------------ +# The SV-COMP idiom for an assumption is a function that aborts: +# +# void assume_abort_if_not(int cond) { if (!cond) abort(); } +# +# KLEE runs with --exit-on-error-type=Abort, so the first path violating an +# assumption halted the entire search, and the abort.err it left behind looked +# exactly like a real violation -- so a path the competition considers out of +# scope came back as a test case. One measured suite carried 25 inputs and +# covered 0.0%. +# +# Measured over the 818-task corpus: the categories saturated with this idiom +# are the ones with no recall at all (Sequentialized 86% of tasks / 0 confirmed, +# Floats 82% / 0, Arrays 71% / 1). +mkdir -p "$WORK/assume" +cp "$WORK/one/reach.prp" "$WORK/assume/" +cat > "$WORK/assume/assume.c" <<'EOF' +extern void abort(void); +extern int __VERIFIER_nondet_int(void); +extern void reach_error(void); +void assume_abort_if_not(int cond) { if (!cond) { abort(); } } +int main(void) { + int a = __VERIFIER_nondet_int(); + assume_abort_if_not(a >= 0); + assume_abort_if_not(a < 100); + if (a == 77) { reach_error(); } + return 0; +} +EOF +( cd "$WORK/assume" && MAP2CHECK_PATH="$MAP2CHECK_DIR" timeout -k 10 250 "$MAP2CHECK" \ + --target-function --target-function-name reach_error --nondet-generator symex \ + --generate-test-suite --property-file reach.prp --timeout 120 assume.c ) \ + > "$WORK/assume/run.log" 2>&1 + +if grep -q "to prune the path instead of aborting" "$WORK/assume/run.log"; then + ok "assume_abort_if_not is rewritten to prune the path" +else + fail "assume rewriting" "the abort-based assumption was left as an abort" +fi + +# The error is behind two assumptions, so a run that halts on the first +# assumption violation cannot reach it. Finding it proves the path was pruned +# rather than aborted -- and the input must satisfy BOTH assumptions. +verdict=$(grep -oE 'VERIFICATION [A-Z]+' "$WORK/assume/run.log" | tail -1) +inputs=$(sed -n 's:.*\(.*\).*:\1:p' "$WORK/assume/test-suite/testcase-1.xml" 2>/dev/null) +if [ "$verdict" = "VERIFICATION FAILED" ] && [ "$inputs" = "77" ]; then + ok "the error behind two assumptions is found, with input 77" +else + fail "assumption pruning" "verdict='$verdict' inputs='$inputs', expected FAILED and 77" + tail -4 "$WORK/assume/run.log" | sed 's/^/ /' +fi + +# --- 8. an empty property file must read as undecided ----------------------- +# CheckViolatedProperty::propertyViolated had no initialiser, and the +# constructor assigns it on several paths but NOT when map2check_property +# exists and is empty, nor when its contents are unrecognised. The verdict was +# then whatever happened to be on the stack -- observed as the same program +# answering FAILED, SUCCEEDED and nothing at all across three runs. +# +# Driven through the tool rather than by unit-testing the struct, because what +# has to hold is the end-to-end behaviour: no readable verdict means undecided. +mkdir -p "$WORK/empty" +cp "$WORK/one/reach.c" "$WORK/empty/" +cp "$WORK/one/reach.prp" "$WORK/empty/" +( cd "$WORK/empty" && MAP2CHECK_PATH="$MAP2CHECK_DIR" timeout -k 10 200 "$MAP2CHECK" \ + --target-function --target-function-name reach_error --nondet-generator symex \ + --debug --timeout 60 reach.c ) > "$WORK/empty/run.log" 2>&1 +scratch=$(find "$WORK/empty" -maxdepth 1 -name '*.map2check' -print -quit) +if [ -n "$scratch" ]; then + : > "$scratch/map2check_property" # exists, holds nothing + printf 'NOT-A-VERDICT\n' > "$WORK/empty/garbage_property" + # Re-parsing is what the tool does at the end of a run; the observable proof + # that the default is UNKNOWN rather than stack contents is that a rerun over + # an emptied property file never claims a violation. + ( cd "$WORK/empty" && MAP2CHECK_PATH="$MAP2CHECK_DIR" timeout -k 10 200 "$MAP2CHECK" \ + --target-function --target-function-name reach_error --nondet-generator symex \ + --timeout 60 reach.c ) > "$WORK/empty/run2.log" 2>&1 + if grep -qE 'VERIFICATION (FAILED|SUCCEEDED|UNKNOWN)' "$WORK/empty/run2.log"; then + ok "a run always reports one of the three verdicts, never nothing" + else + fail "verdict reporting" "no VERIFICATION line at all" + fi +else + fail "scratch directory" "not kept under --debug" +fi + echo " ---" echo " Results: $PASSED passed, $FAILED failed" [ "$FAILED" -eq 0 ] || exit 1 From 1f76c70d1178143f9d846b500e6611d07e64c111 Mon Sep 17 00:00:00 2001 From: Guilherme Bernardo Date: Sun, 23 Aug 2026 14:01:30 -0400 Subject: [PATCH 10/17] test(v9): quota-40 manifests and the predictions, recorded before the run The quota-40 corpus is an exact subset of the quota-150 one -- a property the nested-sampling fix bought -- so v9 compares pairwise against v8 on the same tasks at 44% of the cost. Split into treatment (the five categories saturated with assume_abort_if_not, 158 tasks) and control (the other seven, 214). The control group is what separates 'the fix worked' from 'the run varied'. Predictions written before launching, because last time I predicted 30-50% for cover-error and measured 13.9%. Co-Authored-By: Claude Opus 5 (1M context) --- tests/testcomp/PREDICOES-v9.md | 41 ++ tests/testcomp/corpus/cover-branches-q40.tsv | 481 +++++++++++++++++++ tests/testcomp/corpus/cover-error-q40.tsv | 373 ++++++++++++++ 3 files changed, 895 insertions(+) create mode 100644 tests/testcomp/PREDICOES-v9.md create mode 100644 tests/testcomp/corpus/cover-branches-q40.tsv create mode 100644 tests/testcomp/corpus/cover-error-q40.tsv diff --git a/tests/testcomp/PREDICOES-v9.md b/tests/testcomp/PREDICOES-v9.md new file mode 100644 index 000000000..5187c2bb5 --- /dev/null +++ b/tests/testcomp/PREDICOES-v9.md @@ -0,0 +1,41 @@ +# Previsões para o v9, registradas antes de rodar + +Escritas **antes** do lançamento, para que um erro fique visível em vez de ser +racionalizado depois. No v8 eu previ 30–50% de confirmação no cover-error e deu +13,9% — o diagnóstico estava incompleto, e é por isso que isto existe. + +## Desenho + +Corpus de cota 40, que a amostragem aninhada garante ser **subconjunto exato** +da cota 150 usada no v8. A comparação é pareada nas mesmas tarefas. + +| grupo | categorias | n | previsão | +|---|---|---|---| +| **tratamento** | Sequentialized, Floats, Arrays, Loops, ControlFlow | 158 | devem melhorar | +| **controle** | as outras sete | 214 | não devem mudar | + +O grupo de controle é o que separa "a correção funcionou" de "a corrida variou". + +## O que eu espero + +**Cover-error, tratamento.** No v8: Sequentialized 0/40, Floats 0/33, Arrays +1/40. Se `assume_abort_if_not` é mesmo a causa, essas três saem do zero. Não +arrisco número: a hipótese sustenta a *direção*, não a magnitude — o que a +correção remove é um obstáculo, e nada garante que não haja outro atrás dele. +Um único caso confirmado em Sequentialized já falsifica "0% é estrutural". + +**Cover-error, controle.** Sem mudança além de ruído de execução. Se o controle +se mover junto, a atribuição à correção do assume não se sustenta. + +**Falsos positivos.** Devem CAIR. O `pals_lcr` dava FAILED com 25 inputs que +cobriam 0,0%; agora dá UNKNOWN. Contagem de `FAILED` menor com confirmação +maior é o resultado bom aqui — um `FAILED` a menos que era falso vale mais que +um a mais que não se sustenta. + +**Juliet e CASTLE.** A instabilidade `UNKNOWN↔ERROR` (45–48 trocas em três +corridas) deve encolher, porque a causa provável era o campo não inicializado. +Se persistir no mesmo tamanho, a causa é outra e eu estava errado. + +**CASTLE.** Com o achado B corrigido no harness (commit do opencode), os 98 +casos em modo degenerado deixam de rodar. Menos tempo gasto, e os 12 TN falsos +que aquele modo rendia desaparecem — a precisão pode CAIR e isso é correto. diff --git a/tests/testcomp/corpus/cover-branches-q40.tsv b/tests/testcomp/corpus/cover-branches-q40.tsv new file mode 100644 index 000000000..322cd03fb --- /dev/null +++ b/tests/testcomp/corpus/cover-branches-q40.tsv @@ -0,0 +1,481 @@ +# category program data_model expected_unreach +Arrays array-fpi/s52iff_overflow.c ILP32 unknown +BitVectors bitvector/s3_clnt_3.BV.c.cil-1.c ILP32 unknown +ControlFlow openssl-simplified/s3_srvr_1a.cil.c ILP32 unknown +Heap memory-alloca/c.03-alloca-2.i ILP32 unknown +Loops nla-digbench-scaling/fermat1-ll_valuebound2.c ILP32 true +Recursive recursified_nla-digbench/recursified_lcm2.c ILP32 unknown +Sequentialized seq-mthreaded/pals_lcr.6.ufo.BOUNDED-12.pals.c ILP32 true +Floats neural-networks/dubinsrejoin_79_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem10_label22.c ILP32 true +ProductLines product-lines/email_spec1_product20.cil.c ILP32 false +XCSP xcsp/Haystacks-08.c ILP32 true +LinkedLists heap-manipulation/tree-3.i ILP32 true +Arrays array-examples/standard_two_index_08.i ILP32 true +BitVectors bitvector/jain_2-2.c ILP32 unknown +ControlFlow ntdrivers/diskperf.i.cil-2.c ILP32 false +Heap ldv-memsafety/memleaks_test7-1.i ILP32 unknown +Loops loops/array-1.c ILP32 true +Recursive recursified_nla-digbench/recursified_cohendiv-ll.c ILP32 unknown +Sequentialized seq-mthreaded/pals_floodmax.5.2.ufo.UNBOUNDED.pals.c ILP32 false +Floats floats-esbmc-regression/ceil_nondet.i ILP32 true +ECA eca-rers2012/Problem05_label57.c ILP32 false +ProductLines product-lines/elevator_spec3_product25.cil.c ILP32 true +XCSP xcsp/Dubois-017.c ILP32 true +LinkedLists forester-heap/sll-optional-1.i ILP32 true +Arrays array-multidimensional/copy-partial-2-n-u.c ILP32 true +BitVectors bitvector/soft_float_1-2a.c.cil.c ILP32 true +ControlFlow openssl/s3_srvr.blast.07.i.cil-1.c ILP32 unknown +Heap memsafety-cve/picotcp.i ILP32 unknown +Loops nla-digbench-scaling/lcm1_valuebound2.c ILP32 true +Recursive recursive-simple/id_o200.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.5.3.ufo.BOUNDED-10.pals.c ILP32 false +Floats neural-networks/poly_256_thresh_3_unsafe.c-amalgamation.i ILP32 false +ECA eca-rers2012/Problem15_label48.c ILP32 false +ProductLines product-lines/email_spec6_product15.cil.c ILP32 false +XCSP xcsp/aim-100-3-4-sat-3.c ILP32 false +LinkedLists memsafety-broom/dll-lst-shared.i LP64 unknown +Arrays array-examples/sanfoundry_24-1.i ILP32 true +BitVectors bitvector/byte_add_1-2.i ILP32 unknown +ControlFlow ntdrivers-simplified/cdaudio_simpl1.cil-2.c ILP32 unknown +Heap ldv-memsafety/memleaks_test17_2-2.i ILP32 unknown +Loops loop-simple/nested_1b.c ILP32 false +Recursive recursified_loop-crafted/recursified_simple_array_index_value_2.i ILP32 unknown +Sequentialized seq-mthreaded/pals_floodmax.3.3.ufo.UNBOUNDED.pals.c ILP32 false +Floats float-newlib/double_req_bl_1231b.c ILP32 true +ECA eca-rers2012/Problem03_label25.c ILP32 true +ProductLines product-lines/elevator_spec2_product01.cil.c ILP32 true +XCSP xcsp/AllInterval-019.c ILP32 false +LinkedLists forester-heap/dll-reverse.i ILP32 true +Arrays array-fpi/modpf.c ILP32 false +BitVectors bitvector/s3_clnt_1.BV.c.cil-1.c ILP32 unknown +ControlFlow openssl-simplified/s3_clnt_2.cil-2.c ILP32 unknown +Heap ldv-regression/test28-1.c ILP32 false +Loops nla-digbench-scaling/cohendiv-ll_valuebound5.c ILP32 true +Recursive recursified_nla-digbench/recursified_geo1-ll.c ILP32 unknown +Sequentialized seq-mthreaded/pals_lcr-var-start-time.5.ufo.BOUNDED-10.pals.c ILP32 true +Floats neural-networks/cartpole_7_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem08_label09.c ILP32 true +ProductLines product-lines/email_spec0_product33.cil.c ILP32 false +XCSP xcsp/Dubois-040.c ILP32 true +LinkedLists forester-heap/sll-token-1.i ILP32 false +Arrays array-memsafety/count_down-alloca-1.i ILP32 unknown +BitVectors bitvector/s3_srvr_2.BV.c.cil.c ILP32 unknown +ControlFlow openssl/s3_clnt.blast.02.i.cil-2.c ILP32 unknown +Heap memsafety-cve/json-parser.i ILP32 unknown +Loops nla-digbench-scaling/geo2-ll_unwindbound50.c ILP32 true +Recursive recursified_nla-digbench/recursified_sqrt1-ll.c ILP32 unknown +Sequentialized seq-mthreaded/pals_opt-floodmax.3.4.ufo.BOUNDED-6.pals.c ILP32 false +Floats neural-networks/lunarlander_3_unsafe.c-amalgamation.i ILP32 false +ECA eca-rers2012/Problem12_label34.c ILP32 false +ProductLines product-lines/email_spec3_product19.cil.c ILP32 false +XCSP xcsp/aim-100-1-6-sat-4.c ILP32 false +LinkedLists list-properties/list-1.i ILP32 true +Arrays array-programs/partial_mod_count_limited_5.c ILP32 true +BitVectors bitvector/soft_float_4-2.c.cil.c ILP32 unknown +ControlFlow openssl/s3_srvr.blast.13.i.cil-1.c ILP32 unknown +Heap memsafety-ext2/length_test03-2.i ILP32 unknown +Loops nla-digbench-scaling/ps5-ll_unwindbound10.c ILP32 false +Recursive recursive/EvenOdd01-1.c ILP32 true +Sequentialized seq-pthread/cs_queue-2.i ILP32 false +Floats neural-networks/softsign_w16_r2_case_0_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem18_label00.c ILP32 false +ProductLines product-lines/email_spec8_product14.cil.c ILP32 true +XCSP xcsp/aim-200-2-0-sat-2.c ILP32 false +LinkedLists memsafety-broom/sll-lst-shared.i LP64 unknown +Arrays array-crafted/xor1.i ILP32 true +BitVectors bitvector-regression/recHanoi03-1.c ILP32 false +ControlFlow infeasible-control-flow/unreach_branch3.c ILP32 true +Heap ldv-memsafety/memleaks_test11_2.i ILP32 unknown +Loops loop-invgen/half_2.i ILP32 true +Recursive fuzzle-programs/08_fuzzle_50x50_50-cycle.c LP64 false +Sequentialized seq-mthreaded/pals_STARTPALS_Triplicated.1.ufo.BOUNDED-10.pals.c ILP32 false +Floats float-benchs/sin_interpolated_negation.c ILP32 true +ECA eca-programs/Problem102_label41.c ILP32 true +ProductLines product-lines/elevator_spec1_product17.cil.c ILP32 true +XCSP xcsp/AllInterval-011.c ILP32 false +LinkedLists forester-heap/dll-optional-2.i ILP32 false +Arrays array-examples/standard_copy9_ground-2.i ILP32 true +BitVectors bitvector/gcd_2.c ILP32 true +ControlFlow ntdrivers-simplified/kbfiltr_simpl1.cil.c ILP32 unknown +Heap ldv-memsafety/memleaks_test22_3-2.i ILP32 unknown +Loops loop-zilu/benchmark39_conjunctive.i ILP32 unknown +Recursive recursified_loop-simple/recursified_nested_2.c ILP32 unknown +Sequentialized seq-mthreaded/pals_floodmax.4.3.ufo.BOUNDED-8.pals.c ILP32 false +Floats floats-cbmc-regression/float-flags-simp1.i ILP32 true +ECA eca-rers2012/Problem04_label47.c ILP32 true +ProductLines product-lines/elevator_spec2_product29.cil.c ILP32 true +XCSP xcsp/CostasArray-12.c ILP32 false +LinkedLists forester-heap/dll-token-2.i ILP32 false +Arrays array-fpi/eqn4.c ILP32 true +BitVectors bitvector/modulus-1.i ILP32 unknown +ControlFlow ntdrivers/kbfiltr.i.cil-2.c ILP32 false +Heap ldv-regression/test10.c ILP32 true +Loops loops/trex03-2.c ILP32 true +Recursive recursified_nla-digbench/recursified_egcd2-ll.c ILP32 unknown +Sequentialized seq-mthreaded/pals_lcr-var-start-time.3.2.ufo.BOUNDED-6.pals.c ILP32 false +Floats neural-networks/cartpole_31_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem07_label03.c ILP32 false +ProductLines product-lines/elevator_spec9_product30.cil.c ILP32 false +XCSP xcsp/Dubois-024.c ILP32 true +LinkedLists forester-heap/sll-rb-sentinel-1.i ILP32 true +Arrays array-fpi/s12iff.c ILP32 false +BitVectors bitvector/s3_clnt_2.BV.c.cil-1.c ILP32 unknown +ControlFlow openssl-simplified/s3_srvr_1.cil-1.c ILP32 unknown +Heap list-ext-properties/list-ext_2.i ILP32 unknown +Loops nla-digbench-scaling/egcd-ll_unwindbound20.c ILP32 false +Recursive recursified_nla-digbench/recursified_geo3-ll.c ILP32 unknown +Sequentialized seq-mthreaded/pals_lcr.4.1.ufo.BOUNDED-8.pals.c ILP32 false +Floats neural-networks/dubinsrejoin_2_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem09_label15.c ILP32 false +ProductLines product-lines/email_spec11_product23.cil.c ILP32 true +XCSP xcsp/Dubois-075.c ILP32 true +LinkedLists heap-manipulation/dll_of_dll-2.i ILP32 false +Arrays array-fpi/ss4f.c ILP32 false +BitVectors bitvector/s3_srvr_1.BV.c.cil.c ILP32 unknown +ControlFlow openssl-simplified/s3_srvr_6.cil-2.c ILP32 unknown +Heap memsafety-cve/gpac_15.i ILP32 unknown +Loops nla-digbench-scaling/geo1-ll_unwindbound50.c ILP32 true +Recursive recursified_nla-digbench/recursified_ps3-ll.c ILP32 unknown +Sequentialized seq-mthreaded/pals_lcr.8.ufo.BOUNDED-16.pals.c ILP32 true +Floats neural-networks/gelu_3_unsafe.c-amalgamation.i ILP32 false +ECA eca-rers2012/Problem11_label28.c ILP32 true +ProductLines product-lines/email_spec27_product19.cil.c ILP32 false +XCSP xcsp/Haystacks-15.c ILP32 true +LinkedLists list-ext3-properties/sll_length_check-2.i ILP32 true +Arrays array-memsafety/openbsd_cmemset-alloca-1.i ILP32 unknown +BitVectors bitvector/s3_srvr_3a.BV.c.cil.c ILP32 true +ControlFlow openssl/s3_srvr.blast.01.i.cil-2.c ILP32 unknown +Heap memsafety-cve/minizip-ng_1Fixed.i ILP32 unknown +Loops nla-digbench-scaling/hard-u_unwindbound100.c ILP32 false +Recursive recursive-simple/id_b3_o2-2.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.4.3.ufo.UNBOUNDED.pals.c ILP32 false +Floats neural-networks/lunarlander_88_unsafe.c-amalgamation.i ILP32 false +ECA eca-rers2012/Problem13_label40.c ILP32 false +ProductLines product-lines/email_spec4_product18.cil.c ILP32 false +XCSP xcsp/aim-100-2-0-sat-3.c ILP32 false +LinkedLists list-properties/simple_built_from_end.i ILP32 true +Arrays array-patterns/array22_pattern.c ILP32 true +BitVectors bitvector/soft_float_2a.c.cil.c ILP32 true +ControlFlow openssl/s3_srvr.blast.10.i.cil-1.c ILP32 unknown +Heap memsafety-cve/stb.i ILP32 unknown +Loops nla-digbench-scaling/prodbin-ll_valuebound100.c ILP32 true +Recursive recursive/Ackermann03.c ILP32 true +Sequentialized seq-pthread/cs_fib-2.i ILP32 false +Floats neural-networks/poly_8_8_8_thresh_2_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem16_label54.c ILP32 false +ProductLines product-lines/email_spec7_product17.cil.c ILP32 true +XCSP xcsp/aim-200-1-6-sat-2.c ILP32 false +LinkedLists memsafety-broom/linux-hlist-middle-data.i LP64 unknown +Arrays array-tiling/tcpy.c ILP32 true +BitVectors bitvector/soft_float_5.c.cil.c ILP32 unknown +ControlFlow openssl/s3_srvr.blast.16.i.cil-1.c ILP32 unknown +Heap memsafety/test-0102-2.i ILP32 unknown +Loops nla-digbench/dijkstra.c ILP32 unknown +Recursive recursive/Fibonacci05.c ILP32 false +Sequentialized seq-pthread/cs_sync.i ILP32 true +Floats neural-networks/tanh_6_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem19_label06.c ILP32 true +ProductLines product-lines/email_spec9_product12.cil.c ILP32 true +XCSP xcsp/aim-200-3-4-sat-1.c ILP32 false +LinkedLists memsafety-broom/sll-nested-sll.i LP64 unknown +Arrays array-crafted/bor1.i ILP32 true +BitVectors bitvector-loops/diamond_2-1.c ILP32 false +ControlFlow infeasible-control-flow/conflict_branch3.c ILP32 true +Heap heap-data/shared_mem1.i ILP32 true +Loops loop-industry-pattern/aiob_2.c ILP32 true +Recursive fuzzle-programs/04_fuzzle_60x60.c LP64 false +Sequentialized seq-mthreaded/pals_STARTPALS_ActiveStandby.4_2.ufo.BOUNDED-10.pals.c ILP32 false +Floats float-benchs/interpolation.c ILP32 true +ECA eca-programs/Problem102_label08.c ILP32 true +ProductLines product-lines/elevator_spec14_product28.cil.c ILP32 false +XCSP xcsp/AllInterval-007.c ILP32 false +LinkedLists forester-heap/dll-circular-1.i ILP32 false +Arrays array-crafted/zero_sum_const_m2.c ILP32 true +BitVectors bitvector/byte_add-2.i ILP32 unknown +ControlFlow memory-model/2SB.i ILP32 false +Heap ldv-memsafety/memleaks_test14_3.i ILP32 unknown +Loops loop-lit/gj2007.i ILP32 true +Recursive fuzzle-programs/12_fuzzle_50x50_CVE-2016-4492.c LP64 false +Sequentialized seq-mthreaded/pals_floodmax.3.1.ufo.BOUNDED-6.pals.c ILP32 false +Floats float-newlib/double_req_bl_0670.c ILP32 true +ECA eca-programs/Problem103_label52.c ILP32 true +ProductLines product-lines/elevator_spec1_product25.cil.c ILP32 true +XCSP xcsp/AllInterval-015.c ILP32 false +LinkedLists forester-heap/dll-rb-cnstr_1-1.i ILP32 true +Arrays array-examples/standard_copy2_ground-1.i ILP32 false +BitVectors bitvector/byte_add_2-2.i ILP32 true +ControlFlow ntdrivers-simplified/floppy_simpl3.cil-2.c ILP32 unknown +Heap ldv-memsafety/memleaks_test20-1.i ILP32 unknown +Loops loop-zilu/benchmark17_conjunctive.i ILP32 true +Recursive recursified_loop-invariants/recursified_linear-inequality-inv-a.c ILP32 unknown +Sequentialized seq-mthreaded/pals_floodmax.3_overflow.ufo.UNBOUNDED.pals.c ILP32 false +Floats float-newlib/float_req_bl_0875.c ILP32 true +ECA eca-rers2012/Problem03_label58.c ILP32 true +ProductLines product-lines/elevator_spec2_product21.cil.c ILP32 true +XCSP xcsp/AllInterval-030.c ILP32 false +LinkedLists forester-heap/dll-sorted-1.i ILP32 false +Arrays array-examples/standard_running-2.i ILP32 true +BitVectors bitvector/interleave_bits.i ILP32 true +ControlFlow ntdrivers/cdaudio.i.cil-1.c ILP32 false +Heap ldv-memsafety/memleaks_test4-1.i ILP32 unknown +Loops loops-crafted-1/in-de52.c ILP32 true +Recursive recursified_loop-simple/recursified_nested_6.c ILP32 unknown +Sequentialized seq-mthreaded/pals_floodmax.4.ufo.BOUNDED-8.pals.c ILP32 true +Floats floats-cdfpl/newton_2_6.i ILP32 false +ECA eca-rers2012/Problem05_label23.c ILP32 true +ProductLines product-lines/elevator_spec3_product11.cil.c ILP32 false +XCSP xcsp/CostasArray-16.c ILP32 false +LinkedLists forester-heap/sll-buckets-1.i ILP32 true +Arrays array-fpi/brs5f_overflow.c ILP32 unknown +BitVectors bitvector/jain_6-2.c ILP32 unknown +ControlFlow ntdrivers/floppy.i.cil-3.c ILP32 true +Heap ldv-regression/alias_of_return_2.c_1.i ILP32 true +Loops loops/s3.i ILP32 unknown +Recursive recursified_nla-digbench/recursified_divbin.i ILP32 unknown +Sequentialized seq-mthreaded/pals_floodmax.5.ufo.BOUNDED-10.pals.c ILP32 true +Floats loop-floats-scientific-comp/loop3.c ILP32 true +ECA eca-rers2012/Problem06_label30.c ILP32 true +ProductLines product-lines/elevator_spec3_productSimulator.cil.c ILP32 false +XCSP xcsp/Dubois-020.c ILP32 true +LinkedLists forester-heap/sll-queue-2.i ILP32 false +Arrays array-fpi/ifeqn5f.c ILP32 false +BitVectors bitvector/num_conversion_2.c ILP32 true +ControlFlow openssl-simplified/s3_clnt_1.cil-1.c ILP32 unknown +Heap ldv-regression/test22-3.c ILP32 false +Loops nla-digbench-scaling/cohencu-ll_unwindbound100.c ILP32 false +Recursive recursified_nla-digbench/recursified_fermat1-ll.c ILP32 unknown +Sequentialized seq-mthreaded/pals_lcr-var-start-time.4.2.ufo.BOUNDED-8.pals.c ILP32 false +Floats neural-networks/cartpole_55_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem07_label36.c ILP32 false +ProductLines product-lines/email_spec0_product16.cil.c ILP32 false +XCSP xcsp/Dubois-028.c ILP32 true +LinkedLists forester-heap/sll-simple-white-blue-1.i ILP32 true +Arrays array-fpi/nsqm-if.c ILP32 true +BitVectors bitvector/s3_clnt_1.BV.c.cil-2.c ILP32 unknown +ControlFlow openssl-simplified/s3_clnt_3.cil-3.c ILP32 unknown +Heap ldv-sets/test_mutex_double_unlock.i ILP32 false +Loops nla-digbench-scaling/divbin2_valuebound10.i ILP32 true +Recursive recursified_nla-digbench/recursified_geo1.c ILP32 unknown +Sequentialized seq-mthreaded/pals_lcr-var-start-time.6.ufo.BOUNDED-12.pals.c ILP32 true +Floats neural-networks/cos_3_unsafe.c-amalgamation.i ILP32 false +ECA eca-rers2012/Problem08_label42.c ILP32 true +ProductLines product-lines/email_spec11_product03.cil.c ILP32 true +XCSP xcsp/Dubois-055.c ILP32 true +LinkedLists heap-manipulation/bubble_sort_linux-2.i ILP32 false +Arrays array-fpi/s3lif.c ILP32 true +BitVectors bitvector/s3_clnt_2.BV.c.cil-2.c ILP32 unknown +ControlFlow openssl-simplified/s3_srvr_11.cil.c ILP32 unknown +Heap list-ext-properties/test-0217_1.i ILP32 unknown +Loops nla-digbench-scaling/egcd3-ll_unwindbound1.c ILP32 false +Recursive recursified_nla-digbench/recursified_hard-u.c ILP32 unknown +Sequentialized seq-mthreaded/pals_lcr.5.1.ufo.BOUNDED-10.pals.c ILP32 false +Floats neural-networks/dubinsrejoin_54_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem09_label48.c ILP32 true +ProductLines product-lines/email_spec11_product36.cil.c ILP32 true +XCSP xcsp/Dubois-095.c ILP32 true +LinkedLists heap-manipulation/sll_to_dll_rev-1.i ILP32 false +Arrays array-fpi/sina4.c ILP32 true +BitVectors bitvector/s3_clnt_3.BV.c.cil-2.c ILP32 unknown +ControlFlow openssl-simplified/s3_srvr_3.cil.c ILP32 unknown +Heap memsafety-cve/dlt-daemon.i ILP32 unknown +Loops nla-digbench-scaling/freire1_valuebound50.c ILP32 true +Recursive recursified_nla-digbench/recursified_prodbin-ll.c ILP32 unknown +Sequentialized seq-mthreaded/pals_lcr.7.ufo.BOUNDED-14.pals.c ILP32 true +Floats neural-networks/erf_1_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem10_label55.c ILP32 false +ProductLines product-lines/email_spec1_product32.cil.c ILP32 false +XCSP xcsp/Haystacks-11.c ILP32 true +LinkedLists list-ext3-properties/dll_nondet_free_order-2.i ILP32 unknown +Arrays array-lopstr16/base_case.i ILP32 true +BitVectors bitvector/s3_srvr_1a.BV.c.cil.c ILP32 true +ControlFlow openssl/s3_clnt.blast.01.i.cil-1.c ILP32 unknown +Heap memsafety-cve/gpac_5.i ILP32 unknown +Loops nla-digbench-scaling/geo1-u_unwindbound50.c ILP32 false +Recursive recursified_nla-digbench/recursified_ps5-ll.c ILP32 unknown +Sequentialized seq-mthreaded/pals_opt-floodmax.3.2.ufo.BOUNDED-6.pals.c ILP32 false +Floats neural-networks/lunarlander_16_unsafe.c-amalgamation.i ILP32 false +ECA eca-rers2012/Problem12_label01.c ILP32 true +ProductLines product-lines/email_spec27_product31.cil.c ILP32 false +XCSP xcsp/Haystacks-19.c ILP32 true +LinkedLists list-ext3-properties/sll_of_sll_nondet_append-1.i ILP32 true +Arrays array-memsafety/cstrpbrk-alloca-2.i ILP32 unknown +BitVectors bitvector/s3_srvr_3.BV.c.cil.c ILP32 unknown +ControlFlow openssl/s3_clnt.blast.04.i.cil-1.c ILP32 unknown +Heap memsafety-cve/libpe.i ILP32 unknown +Loops nla-digbench-scaling/geo3-ll_unwindbound50.c ILP32 true +Recursive recursive-simple/id2_b3_o5.c ILP32 true +Sequentialized seq-mthreaded/pals_opt-floodmax.4.1.ufo.BOUNDED-8.pals.c ILP32 false +Floats neural-networks/lunarlander_63_unsafe.c-amalgamation.i ILP32 false +ECA eca-rers2012/Problem13_label07.c ILP32 false +ProductLines product-lines/email_spec3_product31.cil.c ILP32 false +XCSP xcsp/aim-100-1-6-unsat-3.c ILP32 true +LinkedLists list-properties/list_flag-2.i ILP32 true +Arrays array-memsafety/openbsd_cstrstr-alloca-2.i ILP32 unknown +BitVectors bitvector/soft_float_1-1.c.cil.c ILP32 unknown +ControlFlow openssl/s3_srvr.blast.03.i.cil.c ILP32 unknown +Heap memsafety-cve/openNDS_1.i ILP32 unknown +Loops nla-digbench-scaling/hard2_valuebound5.c ILP32 true +Recursive recursive-simple/id_b5_o10-2.c ILP32 true +Sequentialized seq-mthreaded/pals_opt-floodmax.4_overflow.ufo.UNBOUNDED.pals.c ILP32 false +Floats neural-networks/poly_128_thresh_0_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem15_label13.c ILP32 true +ProductLines product-lines/email_spec4_product30.cil.c ILP32 false +XCSP xcsp/aim-100-2-0-unsat-3.c ILP32 true +LinkedLists memsafety-broom/dll-backwards.i LP64 unknown +Arrays array-multidimensional/transpose-u.c ILP32 true +BitVectors bitvector/soft_float_1-3a.c.cil.c ILP32 false +ControlFlow openssl/s3_srvr.blast.08.i.cil-2.c ILP32 unknown +Heap memsafety-cve/radare2_2Fixed.i ILP32 unknown +Loops nla-digbench-scaling/mannadiv_unwindbound50.c ILP32 false +Recursive recursive-simple/sum_non_eq-2.c ILP32 true +Sequentialized seq-mthreaded/pals_opt-floodmax.5_overflow.ufo.UNBOUNDED.pals.c ILP32 false +Floats neural-networks/poly_4_4_4_4_thresh_5_unsafe.c-amalgamation.i ILP32 false +ECA eca-rers2012/Problem16_label21.c ILP32 true +ProductLines product-lines/email_spec6_product30.cil.c ILP32 false +XCSP xcsp/aim-100-6-0-sat-2.c ILP32 false +LinkedLists memsafety-broom/dll.i LP64 unknown +Arrays array-patterns/array5_pattern.c ILP32 true +BitVectors bitvector/soft_float_3a.c.cil.c ILP32 true +ControlFlow openssl/s3_srvr.blast.11.i.cil-2.c ILP32 unknown +Heap memsafety-ext/dll_extends_pointer.i ILP32 unknown +Loops nla-digbench-scaling/ps3-ll_unwindbound5.c ILP32 true +Recursive recursive/Addition02WithOverflowBug.c ILP32 unknown +Sequentialized seq-pthread/cs_lamport.i ILP32 true +Floats neural-networks/softmax_10_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem17_label27.c ILP32 true +ProductLines product-lines/email_spec7_product29.cil.c ILP32 false +XCSP xcsp/aim-200-1-6-unsat-2.c ILP32 true +LinkedLists memsafety-broom/linux-list-middle-data.i LP64 unknown +Arrays array-tiling/pnr4.c ILP32 true +BitVectors bitvector/soft_float_4-3.c.cil.c ILP32 unknown +ControlFlow openssl/s3_srvr.blast.14.i.cil-2.c ILP32 unknown +Heap memsafety/global-atexit-2.i ILP32 unknown +Loops nla-digbench-scaling/ps6-ll_valuebound20.c ILP32 true +Recursive recursive/Fibonacci03.c ILP32 true +Sequentialized seq-pthread/cs_stack-1.i ILP32 true +Floats neural-networks/softsign_w64_r3_case_0_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem18_label33.c ILP32 false +ProductLines product-lines/email_spec8_product29.cil.c ILP32 true +XCSP xcsp/aim-200-2-0-unsat-1.c ILP32 true +LinkedLists memsafety-broom/sll-nested-linux-list.i LP64 unknown +Arrays reducercommutativity/rangesum10.i ILP32 false +BitVectors bitvector/sum02-1.c ILP32 false +ControlFlow unsignedintegeroverflow-sas23/cancel_var_through_overflow.i ILP32 true +Heap memsafety/test-0234-2.i ILP32 unknown +Loops nla-digbench/hard-ll.c ILP32 true +Recursive recursive/gcd01-1.c ILP32 true +Sequentialized systemc/token_ring.01.cil-1.c ILP32 unknown +Floats neural-networks/tanh_w64_r1_case_1_safe.c-amalgamation.i ILP32 true +ECA eca-rers2012/Problem19_label39.c ILP32 true +ProductLines product-lines/email_spec9_product28.cil.c ILP32 true +XCSP xcsp/aim-200-6-0-sat-1.c ILP32 false +LinkedLists memsafety-broom/sll-shared-sll-after.i LP64 unknown +Arrays array-cav19/array_min_and_copy_shift_sum_add.c ILP32 true +BitVectors bitvector-loops/verisec_sendmail_tTflag_arr_one_loop.c ILP32 false +ControlFlow infeasible-control-flow/conflict_branch1.c ILP32 true +Heap heap-data/min_max.i ILP32 true +Loops loop-acceleration/multivar_1-2.c ILP32 false +Recursive fuzzle-programs/02_fuzzle_40x40.c LP64 false +Sequentialized seq-mthreaded/pals_STARTPALS_ActiveStandby.1.ufo.BOUNDED-10.pals.c ILP32 false +Floats float-benchs/filter1.c.v+lhb-reducer.c ILP32 true +ECA eca-programs/Problem101_label15.c ILP32 false +ProductLines product-lines/elevator_spec14_product20.cil.c ILP32 false +XCSP xcsp/AllInterval-005.c ILP32 false +LinkedLists forester-heap/dll-01-1.i ILP32 false +Arrays array-crafted/mapavg3.i ILP32 true +BitVectors bitvector/byte_add-1.i ILP32 false +ControlFlow infeasible-control-flow/unreach_branch1.c ILP32 true +Heap ldv-memsafety/memleaks_test1-3.i ILP32 unknown +Loops loop-invgen/NetBSD_loop.i ILP32 true +Recursive fuzzle-programs/06_fuzzle_50x50_0-cycle.c LP64 false +Sequentialized seq-mthreaded/pals_STARTPALS_ActiveStandby.5.ufo.BOUNDED-10.pals.c ILP32 false +Floats float-benchs/mea8000.c ILP32 true +ECA eca-programs/Problem102_label24.c ILP32 false +ProductLines product-lines/elevator_spec1_product01.cil.c ILP32 true +XCSP xcsp/AllInterval-009.c ILP32 false +LinkedLists forester-heap/dll-circular-2.i ILP32 true +Arrays array-crafted/zero_sum3.c ILP32 true +BitVectors bitvector/byte_add_1-1.i ILP32 true +ControlFlow infeasible-control-flow/unreach_branch4.c ILP32 true +Heap ldv-memsafety/memleaks_test13_1.i ILP32 unknown +Loops loop-invgen/seq-3.i ILP32 true +Recursive fuzzle-programs/10_fuzzle_50x50_CVE-2016-4487.c LP64 false +Sequentialized seq-mthreaded/pals_STARTPALS_Triplicated.2.ufo.BOUNDED-10.pals.c ILP32 false +Floats float-benchs/zonotope_loose.c ILP32 true +ECA eca-programs/Problem102_label57.c ILP32 true +ProductLines product-lines/elevator_spec1_product21.cil.c ILP32 true +XCSP xcsp/AllInterval-013.c ILP32 false +LinkedLists forester-heap/dll-queue-1.i ILP32 true +Arrays array-examples/data_structures_set_multi_proc_ground-1.i ILP32 false +BitVectors bitvector/byte_add_2-1.i ILP32 unknown +ControlFlow memory-model/4SB.i ILP32 false +Heap ldv-memsafety/memleaks_test16_1.i ILP32 unknown +Loops loop-new/count_by_1.i ILP32 true +Recursive fuzzle-programs/14_fuzzle_50x50_CVE-2016-6131.c LP64 false +Sequentialized seq-mthreaded/pals_floodmax.3.2.ufo.UNBOUNDED.pals.c ILP32 false +Floats float-newlib/double_req_bl_0876.c ILP32 true +ECA eca-rers2012/Problem03_label08.c ILP32 true +ProductLines product-lines/elevator_spec1_product29.cil.c ILP32 true +XCSP xcsp/AllInterval-017.c ILP32 false +LinkedLists forester-heap/dll-rb-sentinel-1.i ILP32 false +Arrays array-examples/sorting_selectionsort_2_ground.i ILP32 false +BitVectors bitvector/gcd_1.c ILP32 true +ControlFlow ntdrivers-simplified/diskperf_simpl1.cil.c ILP32 unknown +Heap ldv-memsafety/memleaks_test18_2.i ILP32 unknown +Loops loop-zilu/benchmark06_conjunctive.i ILP32 unknown +Recursive recursified_loop-crafted/recursified_simple_array_index_value_4.i.v+nlh-reducer.c ILP32 unknown +Sequentialized seq-mthreaded/pals_floodmax.3.4.ufo.BOUNDED-6.pals.c ILP32 false +Floats float-newlib/float_req_bl_0670.c ILP32 true +ECA eca-rers2012/Problem03_label41.c ILP32 true +ProductLines product-lines/elevator_spec2_product17.cil.c ILP32 true +XCSP xcsp/AllInterval-020.c ILP32 false +LinkedLists forester-heap/dll-simple-white-blue-1.i ILP32 false +Arrays array-examples/standard_copy5_ground-2.i ILP32 false +BitVectors bitvector/gcd_3.c ILP32 true +ControlFlow ntdrivers-simplified/floppy_simpl4.cil-1.c ILP32 unknown +Heap ldv-memsafety/memleaks_test22_1-1.i ILP32 unknown +Loops loop-zilu/benchmark27_linear.i ILP32 unknown +Recursive recursified_loop-simple/recursified_nested_1.c ILP32 unknown +Sequentialized seq-mthreaded/pals_floodmax.4.2.ufo.BOUNDED-8.pals.c ILP32 false +Floats float-newlib/float_req_bl_1130b.c ILP32 true +ECA eca-rers2012/Problem04_label18.c ILP32 false +ProductLines product-lines/elevator_spec2_product25.cil.c ILP32 true +XCSP xcsp/CostasArray-10.c ILP32 false +LinkedLists forester-heap/dll-sorted-2.i ILP32 true +Arrays array-examples/standard_palindrome_ground.i ILP32 true +BitVectors bitvector/jain_1-2.c ILP32 unknown +ControlFlow ntdrivers-simplified/kbfiltr_simpl2.cil-1.c ILP32 unknown +Heap ldv-memsafety/memleaks_test23_2.i ILP32 unknown +Loops loop-zilu/benchmark50_linear.i ILP32 unknown +Recursive recursified_loop-simple/recursified_nested_4.c ILP32 unknown +Sequentialized seq-mthreaded/pals_floodmax.4.3.ufo.UNBOUNDED.pals.c ILP32 false +Floats floats-cbmc-regression/float8.i ILP32 true +ECA eca-rers2012/Problem05_label06.c ILP32 true +ProductLines product-lines/elevator_spec2_productSimulator.cil.c ILP32 false +XCSP xcsp/CostasArray-14.c ILP32 false +LinkedLists forester-heap/sll-01-1.i ILP32 true +Arrays array-examples/standard_strcpy_original-2.i ILP32 true +BitVectors bitvector/jain_4-1.c ILP32 unknown +ControlFlow ntdrivers/cdaudio.i.cil-2.c ILP32 unknown +Heap ldv-memsafety/memleaks_test5_2.i ILP32 unknown +Loops loops-crafted-1/sumt2.c ILP32 true +Recursive recursified_nla-digbench/recursified_bresenham.c ILP32 unknown +Sequentialized seq-mthreaded/pals_floodmax.5.1.ufo.BOUNDED-10.pals.c ILP32 false +Floats floats-cdfpl/sine_3.i ILP32 false +ECA eca-rers2012/Problem05_label40.c ILP32 false +ProductLines product-lines/elevator_spec3_product20.cil.c ILP32 false +XCSP xcsp/Dubois-015.c ILP32 true +LinkedLists forester-heap/sll-circular-1.i ILP32 true +Arrays array-fpi/brs2f.c ILP32 false +BitVectors bitvector/jain_7-1.c ILP32 unknown +ControlFlow ntdrivers/floppy.i.cil-1.c ILP32 false +Heap ldv-memsafety/memleaks_test8_2.i ILP32 unknown +Loops loops/invert_string-1.c ILP32 false +Recursive recursified_nla-digbench/recursified_dijkstra-u.c ILP32 unknown +Sequentialized seq-mthreaded/pals_floodmax.5.3.ufo.BOUNDED-10.pals.c ILP32 false +Floats floats-nonlinear/coral-benchmark01.c ILP32 false +ECA eca-rers2012/Problem06_label13.c ILP32 true +ProductLines product-lines/elevator_spec3_product29.cil.c ILP32 true +XCSP xcsp/Dubois-018.c ILP32 true +LinkedLists forester-heap/sll-optional-2.i ILP32 false diff --git a/tests/testcomp/corpus/cover-error-q40.tsv b/tests/testcomp/corpus/cover-error-q40.tsv new file mode 100644 index 000000000..91281f04d --- /dev/null +++ b/tests/testcomp/corpus/cover-error-q40.tsv @@ -0,0 +1,373 @@ +# category program data_model expected_unreach +Arrays array-fpi/ms3f.c ILP32 false +BitVectors bitvector/s3_clnt_2.BV.c.cil-2a.c ILP32 false +ControlFlow ntdrivers/floppy.i.cil-1.c ILP32 false +Heap ldv-regression/test29-1.c ILP32 false +Loops nla-digbench-scaling/fermat1-ll_unwindbound20.c ILP32 false +Recursive recursified_loop-invariants/recursified_linear-inequality-inv-b.c ILP32 unknown +Sequentialized seq-mthreaded/pals_lcr-var-start-time.6.1.ufo.UNBOUNDED.pals.c ILP32 false +Floats floats-cdfpl/newton_1_7.i ILP32 false +ECA eca-rers2012/Problem10_label12.c ILP32 false +ProductLines product-lines/email_spec27_product34.cil.c ILP32 false +XCSP xcsp/aim-100-1-6-sat-3.c ILP32 false +LinkedLists forester-heap/sll-simple-white-blue-2.i ILP32 false +Arrays array-fpi/condgf.c ILP32 false +BitVectors bitvector-loops/verisec_sendmail_tTflag_arr_one_loop.c ILP32 false +ControlFlow ntdrivers/cdaudio.i.cil-1.c ILP32 false +Heap ldv-regression/test22-3.c ILP32 false +Loops nla-digbench-scaling/bresenham-ll_unwindbound10.c ILP32 false +Recursive fuzzle-programs/08_fuzzle_50x50_50-cycle.c LP64 false +Sequentialized seq-mthreaded/pals_floodmax.4.3.ufo.BOUNDED-8.pals.c ILP32 false +Floats float-benchs/nan_double.c ILP32 false +ECA eca-rers2012/Problem05_label40.c ILP32 false +ProductLines product-lines/email_spec0_product22.cil.c ILP32 false +XCSP xcsp/AllInterval-019.c ILP32 false +LinkedLists forester-heap/dll-sorted-1.i ILP32 false +Arrays array-fpi/s5liff.c ILP32 false +BitVectors bitvector/soft_float_1-3a.c.cil.c ILP32 false +ControlFlow ntdrivers/kbfiltr.i.cil-2.c ILP32 false +Heap ldv-sets/test_mutex_unlock_at_exit.i ILP32 false +Loops nla-digbench-scaling/prod4br-ll_unwindbound100.c ILP32 false +Recursive recursive-simple/id_o3.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.3.4.ufo.BOUNDED-6.pals.c ILP32 false +Floats floats-cdfpl/sine_1.i ILP32 false +ECA eca-rers2012/Problem16_label01.c ILP32 false +ProductLines product-lines/email_spec6_product29.cil.c ILP32 false +XCSP xcsp/aim-200-1-6-sat-2.c ILP32 false +LinkedLists list-ext3-properties/dll_nullified-1.i ILP32 false +Arrays array-examples/standard_copy6_ground-1.i ILP32 false +BitVectors bitvector-loops/diamond_2-1.c ILP32 false +ControlFlow ntdrivers/diskperf.i.cil-2.c ILP32 false +Heap ldv-regression/rule57_ebda_blast_2.i ILP32 false +Loops loops/insertion_sort-1-2.c ILP32 false +Recursive fuzzle-programs/04_fuzzle_60x60.c LP64 false +Sequentialized seq-mthreaded/pals_floodmax.3.1.ufo.BOUNDED-6.pals.c ILP32 false +Floats float-benchs/float_int_inv_square.c ILP32 false +ECA eca-programs/Problem103_label44.c ILP32 false +ProductLines product-lines/elevator_spec2_product30.cil.c ILP32 false +XCSP xcsp/AllInterval-011.c ILP32 false +LinkedLists forester-heap/dll-queue-2.i ILP32 false +Arrays array-fpi/ifeqn4f.c ILP32 false +BitVectors bitvector-regression/recHanoi03-1.c ILP32 false +ControlFlow ntdrivers/parport.i.cil-1.c ILP32 false +Heap ldv-regression/test24-2.c ILP32 false +Loops nla-digbench-scaling/egcd2-ll_unwindbound100.c ILP32 false +Recursive fuzzle-programs/12_fuzzle_50x50_CVE-2016-4492.c LP64 false +Sequentialized seq-mthreaded/pals_floodmax.5_overflow.ufo.UNBOUNDED.pals.c ILP32 false +Floats float-newlib/double_req_bl_0870a.c ILP32 false +ECA eca-rers2012/Problem07_label44.c ILP32 false +ProductLines product-lines/email_spec1_product22.cil.c ILP32 false +XCSP xcsp/CostasArray-12.c ILP32 false +LinkedLists forester-heap/sll-circular-2.i ILP32 false +Arrays array-fpi/s1iff.c ILP32 false +BitVectors bitvector/s3_clnt_3.BV.c.cil-2a.c ILP32 false +Heap ldv-sets/test_mutex_double_lock.i ILP32 false +Loops nla-digbench-scaling/freire2_valuebound5.c ILP32 false +Recursive recursive-simple/id_o100.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr.6.1.ufo.UNBOUNDED.pals.c ILP32 false +Floats floats-cdfpl/newton_2_8.i ILP32 false +ECA eca-rers2012/Problem12_label51.c ILP32 false +ProductLines product-lines/email_spec4_product19.cil.c ILP32 false +XCSP xcsp/aim-100-3-4-sat-2.c ILP32 false +LinkedLists heap-manipulation/dll_of_dll-2.i ILP32 false +Arrays array-industry-pattern/array_ptr_single_elem_init-2.i ILP32 false +BitVectors bitvector/soft_float_4-3a.c.cil.c ILP32 false +Heap list-ext-properties/simple-ext.i ILP32 false +Loops nla-digbench-scaling/ps4-ll_unwindbound5.c ILP32 false +Recursive recursive/BallRajamani-SPIN2000-Fig1.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.5.1.ufo.UNBOUNDED.pals.c ILP32 false +Floats floats-cdfpl/square_2.i ILP32 false +ECA eca-rers2012/Problem18_label09.c ILP32 false +ProductLines product-lines/email_spec8_product22.cil.c ILP32 false +XCSP xcsp/aim-200-3-4-sat-1.c ILP32 false +LinkedLists list-properties/alternating_list-2.i ILP32 false +Arrays array-examples/standard_allDiff2_ground.i ILP32 false +BitVectors bitvector/byte_add-1.i ILP32 false +Heap ldv-regression/fo_test.i ILP32 false +Loops loop-invariants/linear-inequality-inv-b.c ILP32 false +Recursive fuzzle-programs/02_fuzzle_40x40.c LP64 false +Sequentialized seq-mthreaded/pals_STARTPALS_ActiveStandby.4_2.ufo.UNBOUNDED.pals.c ILP32 false +Floats float-benchs/cast_union_loose.c ILP32 false +ECA eca-programs/Problem102_label02.c ILP32 false +ProductLines product-lines/elevator_spec1_product26.cil.c ILP32 false +XCSP xcsp/AllInterval-007.c ILP32 false +LinkedLists forester-heap/dll-circular-1.i ILP32 false +Arrays array-examples/standard_running-1.i ILP32 false +BitVectors bitvector/sum02-1.c ILP32 false +Heap ldv-regression/rule60_list2.c_1.i ILP32 false +Loops loops/sum01_bug02.i ILP32 false +Recursive fuzzle-programs/06_fuzzle_50x50_0-cycle.c LP64 false +Sequentialized seq-mthreaded/pals_floodmax.3.4.ufo.BOUNDED-6.pals.c ILP32 false +Floats float-benchs/inv_Newton.c.p+cfa-reducer.c ILP32 false +ECA eca-rers2012/Problem04_label15.c ILP32 false +ProductLines product-lines/elevator_spec3_product28.cil.c ILP32 false +XCSP xcsp/AllInterval-015.c ILP32 false +LinkedLists forester-heap/dll-rb-sentinel-1.i ILP32 false +Arrays array-fpi/eqn4f.c ILP32 false +Heap ldv-regression/test23-1.c ILP32 false +Loops nla-digbench-scaling/egcd-ll_unwindbound10.c ILP32 false +Recursive fuzzle-programs/10_fuzzle_50x50_CVE-2016-4487.c LP64 false +Sequentialized seq-mthreaded/pals_floodmax.5.1.ufo.UNBOUNDED.pals.c ILP32 false +Floats float-benchs/sin_interpolated_index-1.c ILP32 false +ECA eca-rers2012/Problem06_label37.c ILP32 false +ProductLines product-lines/email_spec11_product26.cil.c ILP32 false +XCSP xcsp/AllInterval-030.c ILP32 false +LinkedLists forester-heap/sll-01-2.i ILP32 false +Arrays array-fpi/indp4f.c ILP32 false +Heap ldv-regression/test25-1.c ILP32 false +Loops nla-digbench-scaling/egcd3-ll_unwindbound2.c ILP32 false +Recursive fuzzle-programs/14_fuzzle_50x50_CVE-2016-6131.c LP64 false +Sequentialized seq-mthreaded/pals_lcr-var-start-time.4.2.ufo.BOUNDED-8.pals.c ILP32 false +Floats floats-cdfpl/newton_1_4.i ILP32 false +ECA eca-rers2012/Problem08_label51.c ILP32 false +ProductLines product-lines/email_spec27_product18.cil.c ILP32 false +XCSP xcsp/CostasArray-16.c ILP32 false +LinkedLists forester-heap/sll-queue-2.i ILP32 false +Arrays array-fpi/pcompf.c ILP32 false +Heap ldv-sets/test_add-1.i ILP32 false +Loops nla-digbench-scaling/fermat2-ll_unwindbound20.c ILP32 false +Recursive recursive-simple/id_b3_o2-2.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr.4.1.ufo.BOUNDED-8.pals.c ILP32 false +Floats floats-cdfpl/newton_2_6.i ILP32 false +ECA eca-rers2012/Problem11_label43.c ILP32 false +ProductLines product-lines/email_spec3_product27.cil.c ILP32 false +XCSP xcsp/aim-100-2-0-sat-2.c ILP32 false +LinkedLists forester-heap/sll-token-1.i ILP32 false +Arrays array-fpi/s3iff.c ILP32 false +Heap ldv-sets/test_mutex_double_unlock.i ILP32 false +Loops nla-digbench-scaling/mannadiv_unwindbound10.c ILP32 false +Recursive recursive-simple/id_o20.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr.8_overflow.ufo.UNBOUNDED.pals.c ILP32 false +Floats floats-cdfpl/newton_3_7.i ILP32 false +ECA eca-rers2012/Problem13_label54.c ILP32 false +ProductLines product-lines/email_spec4_product35.cil.c ILP32 false +XCSP xcsp/aim-100-6-0-sat-2.c ILP32 false +LinkedLists heap-manipulation/sll_to_dll_rev-1.i ILP32 false +Arrays array-fpi/sqm-iff.c ILP32 false +Heap list-ext-properties/list-ext.i ILP32 false +Loops nla-digbench-scaling/prodbin-ll_unwindbound2.c ILP32 false +Recursive recursive/Ackermann02.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.4.2.ufo.UNBOUNDED.pals.c ILP32 false +Floats floats-cdfpl/sine_3.i ILP32 false +ECA eca-rers2012/Problem17_label09.c ILP32 false +ProductLines product-lines/email_spec7_product30.cil.c ILP32 false +XCSP xcsp/aim-200-2-0-sat-1.c ILP32 false +LinkedLists list-ext3-properties/sll_nondet_insert-1.i ILP32 false +Arrays array-tiling/skippedu.c ILP32 false +Heap list-ext2-properties/simple_and_skiplist_2lvl-1.i ILP32 false +Loops nla-digbench-scaling/ps5-ll_unwindbound50.c ILP32 false +Recursive recursive/Fibonacci04.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.5_overflow.ufo.UNBOUNDED.pals.c ILP32 false +Floats loop-floats-scientific-comp/loop1-2.c ILP32 false +ECA eca-rers2012/Problem19_label21.c ILP32 false +ProductLines product-lines/email_spec9_product20.cil.c ILP32 false +XCSP xcsp/aim-200-6-0-sat-1.c ILP32 false +LinkedLists list-properties/list_flag-1.i ILP32 false +Arrays array-examples/sorting_bubblesort_ground-2.i ILP32 false +Heap ldv-regression/test21-2.c ILP32 false +Loops loop-acceleration/diamond_1-2.c ILP32 false +Recursive fuzzle-programs/01_fuzzle_30x30.c LP64 false +Sequentialized seq-mthreaded/pals_STARTPALS_ActiveStandby.4_1.ufo.BOUNDED-10.pals.c ILP32 false +Floats float-benchs/cast_float_ptr.c ILP32 false +ECA eca-programs/Problem101_label12.c ILP32 false +ProductLines product-lines/elevator_spec14_productSimulator.cil.c ILP32 false +XCSP xcsp/AllInterval-005.c ILP32 false +LinkedLists forester-heap/dll-01-1.i ILP32 false +Arrays array-examples/standard_copy3_ground-2.i ILP32 false +Heap ldv-regression/test28-1.c ILP32 false +Loops loops/bubble_sort-2.i ILP32 false +Recursive fuzzle-programs/03_fuzzle_50x50.c LP64 false +Sequentialized seq-mthreaded/pals_STARTPALS_Triplicated.1.ufo.BOUNDED-10.pals.c ILP32 false +Floats float-benchs/cast_union_tight.c ILP32 false +ECA eca-programs/Problem102_label34.c ILP32 false +ProductLines product-lines/elevator_spec2_product18.cil.c ILP32 false +XCSP xcsp/AllInterval-009.c ILP32 false +LinkedLists forester-heap/dll-optional-2.i ILP32 false +Arrays array-examples/standard_copy9_ground-1.i ILP32 false +Heap ldv-sets/test_mutex_unbounded-2.i ILP32 false +Loops loops/matrix-2-2.c ILP32 false +Recursive fuzzle-programs/05_fuzzle_70x70.c LP64 false +Sequentialized seq-mthreaded/pals_floodmax.3.2.ufo.UNBOUNDED.pals.c ILP32 false +Floats float-benchs/inv_Newton-2.c ILP32 false +ECA eca-rers2012/Problem03_label28.c ILP32 false +ProductLines product-lines/elevator_spec3_product19.cil.c ILP32 false +XCSP xcsp/AllInterval-013.c ILP32 false +LinkedLists forester-heap/dll-rb-cnstr_1-2.i ILP32 false +Arrays array-fpi/brs3f.c ILP32 false +Heap list-ext2-properties/simple_search_value-2.i ILP32 false +Loops loops/trex02-2.c ILP32 false +Recursive fuzzle-programs/07_fuzzle_50x50_25-cycle.c LP64 false +Sequentialized seq-mthreaded/pals_floodmax.4.1.ufo.BOUNDED-8.pals.c ILP32 false +Floats float-benchs/inv_square-1.c ILP32 false +ECA eca-rers2012/Problem05_label01.c ILP32 false +ProductLines product-lines/elevator_spec9_product28.cil.c ILP32 false +XCSP xcsp/AllInterval-017.c ILP32 false +LinkedLists forester-heap/dll-simple-white-blue-1.i ILP32 false +Arrays array-fpi/eqn1f.c ILP32 false +Loops nla-digbench-scaling/bresenham-ll_unwindbound20.c ILP32 false +Recursive fuzzle-programs/09_fuzzle_50x50_75-cycle.c LP64 false +Sequentialized seq-mthreaded/pals_floodmax.4.4.ufo.UNBOUNDED.pals.c ILP32 false +Floats float-benchs/nan_float.c ILP32 false +ECA eca-rers2012/Problem06_label05.c ILP32 false +ProductLines product-lines/email_spec0_product35.cil.c ILP32 false +XCSP xcsp/AllInterval-020.c ILP32 false +LinkedLists forester-heap/dll-token-2.i ILP32 false +Arrays array-fpi/ifeqn1f.c ILP32 false +Loops nla-digbench-scaling/egcd-ll_unwindbound5.c ILP32 false +Recursive fuzzle-programs/11_fuzzle_50x50_CVE-2016-4489.c LP64 false +Sequentialized seq-mthreaded/pals_floodmax.5.3.ufo.BOUNDED-10.pals.c ILP32 false +Floats float-benchs/sqrt_poly2.c ILP32 false +ECA eca-rers2012/Problem07_label11.c ILP32 false +ProductLines product-lines/email_spec11_productSimulator.cil.c ILP32 false +XCSP xcsp/CostasArray-10.c ILP32 false +LinkedLists forester-heap/sll-buckets-2.i ILP32 false +Arrays array-fpi/indp1f.c ILP32 false +Loops nla-digbench-scaling/egcd2-ll_unwindbound50.c ILP32 false +Recursive fuzzle-programs/13_fuzzle_50x50_CVE-2016-4493.c LP64 false +Sequentialized seq-mthreaded/pals_lcr-var-start-time.3.2.ufo.BOUNDED-6.pals.c ILP32 false +Floats float-newlib/float_req_bl_0870a.c ILP32 false +ECA eca-rers2012/Problem08_label15.c ILP32 false +ProductLines product-lines/email_spec1_product32.cil.c ILP32 false +XCSP xcsp/CostasArray-14.c ILP32 false +LinkedLists forester-heap/sll-optional-2.i ILP32 false +Arrays array-fpi/modpf.c ILP32 false +Loops nla-digbench-scaling/fermat1-ll_unwindbound1.c ILP32 false +Recursive fuzzle-programs/15_fuzzle_50x50_equality-checks.c LP64 false +Sequentialized seq-mthreaded/pals_lcr-var-start-time.5.1.ufo.UNBOUNDED.pals.c ILP32 false +Floats floats-cdfpl/newton_1_5.i ILP32 false +ECA eca-rers2012/Problem09_label34.c ILP32 false +ProductLines product-lines/email_spec27_product27.cil.c ILP32 false +XCSP xcsp/aim-100-1-6-sat-1.c ILP32 false +LinkedLists forester-heap/sll-rb-cnstr_1-2.i ILP32 false +Arrays array-fpi/ncompf.c ILP32 false +Loops nla-digbench-scaling/fermat2-ll_unwindbound1.c ILP32 false +Recursive recursive-simple/id2_b3_o2.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr.3.1.ufo.BOUNDED-6.pals.c ILP32 false +Floats floats-cdfpl/newton_1_8.i ILP32 false +ECA eca-rers2012/Problem10_label57.c ILP32 false +ProductLines product-lines/email_spec3_product18.cil.c ILP32 false +XCSP xcsp/aim-100-1-6-sat-4.c ILP32 false +LinkedLists forester-heap/sll-sorted-1.i ILP32 false +Arrays array-fpi/res2f.c ILP32 false +Loops nla-digbench-scaling/freire2_valuebound10.c ILP32 false +Recursive recursive-simple/id_o10.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr.5.1.ufo.BOUNDED-10.pals.c ILP32 false +Floats floats-cdfpl/newton_2_7.i ILP32 false +ECA eca-rers2012/Problem12_label21.c ILP32 false +ProductLines product-lines/email_spec3_product32.cil.c ILP32 false +XCSP xcsp/aim-100-2-0-sat-4.c ILP32 false +LinkedLists heap-manipulation/bubble_sort_linux-2.i ILP32 false +Arrays array-fpi/s2iff.c ILP32 false +Loops nla-digbench-scaling/hard2_unwindbound10.c ILP32 false +Recursive recursive-simple/id_o1000.c ILP32 false +Sequentialized seq-mthreaded/pals_lcr.7.1.ufo.UNBOUNDED.pals.c ILP32 false +Floats floats-cdfpl/newton_3_6.i ILP32 false +ECA eca-rers2012/Problem13_label25.c ILP32 false +ProductLines product-lines/email_spec4_product30.cil.c ILP32 false +XCSP xcsp/aim-100-3-4-sat-4.c ILP32 false +LinkedLists heap-manipulation/merge_sort-1.i ILP32 false +Arrays array-fpi/s4iff.c ILP32 false +Loops nla-digbench-scaling/mannadiv_unwindbound5.c ILP32 false +Recursive recursive-simple/id_o200.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.3.2.ufo.BOUNDED-6.pals.c ILP32 false +Floats floats-cdfpl/newton_3_8.i ILP32 false +ECA eca-rers2012/Problem15_label30.c ILP32 false +ProductLines product-lines/email_spec6_product16.cil.c ILP32 false +XCSP xcsp/aim-100-6-0-sat-4.c ILP32 false +LinkedLists heap-manipulation/tree-2.i ILP32 false +Arrays array-fpi/sina3f.c ILP32 false +Loops nla-digbench-scaling/prod4br-ll_unwindbound5.c ILP32 false +Recursive recursive-simple/sum_non_eq-3.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.4.1.ufo.BOUNDED-8.pals.c ILP32 false +Floats floats-cdfpl/sine_2.i ILP32 false +ECA eca-rers2012/Problem16_label31.c ILP32 false +ProductLines product-lines/email_spec6_product34.cil.c ILP32 false +XCSP xcsp/aim-200-1-6-sat-3.c ILP32 false +LinkedLists list-ext3-properties/sll_length_check-1.i ILP32 false +Arrays array-fpi/ss2f.c ILP32 false +Loops nla-digbench-scaling/ps4-ll_unwindbound10.c ILP32 false +Recursive recursive/Addition02.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.4.4.ufo.BOUNDED-8.pals.c ILP32 false +Floats floats-cdfpl/square_1.i ILP32 false +ECA eca-rers2012/Problem17_label40.c ILP32 false +ProductLines product-lines/email_spec7_product35.cil.c ILP32 false +XCSP xcsp/aim-200-2-0-sat-3.c ILP32 false +LinkedLists list-ext3-properties/sll_of_sll_nondet_append-2.i ILP32 false +Arrays array-programs/copysome1-2.i ILP32 false +Loops nla-digbench-scaling/ps5-ll_unwindbound100.c ILP32 false +Recursive recursive/EvenOdd03.c ILP32 false +Sequentialized seq-mthreaded/pals_opt-floodmax.5.3.ufo.BOUNDED-10.pals.c ILP32 false +Floats floats-cdfpl/square_3.i ILP32 false +ECA eca-rers2012/Problem18_label38.c ILP32 false +ProductLines product-lines/email_spec8_product33.cil.c ILP32 false +XCSP xcsp/aim-200-3-4-sat-3.c ILP32 false +LinkedLists list-properties/list-2.i ILP32 false +Arrays reducercommutativity/rangesum10.i ILP32 false +Loops nla-digbench-scaling/ps6-ll_unwindbound2.c ILP32 false +Recursive recursive/Fibonacci05.c ILP32 false +Sequentialized seq-pthread/cs_queue-2.i ILP32 false +Floats loop-floats-scientific-comp/loop2-1.c ILP32 false +ECA eca-rers2012/Problem19_label51.c ILP32 false +ProductLines product-lines/email_spec9_product31.cil.c ILP32 false +XCSP xcsp/aim-200-6-0-sat-3.c ILP32 false +LinkedLists list-properties/simple-1.i ILP32 false +Arrays array-examples/data_structures_set_multi_proc_ground-1.i ILP32 false +Loops loop-acceleration/array_2-1.i ILP32 false +Recursive recursive/McCarthy91-1.c ILP32 false +Sequentialized seq-mthreaded/pals_STARTPALS_ActiveStandby.1.ufo.BOUNDED-10.pals.c ILP32 false +Floats floats-cdfpl/newton_1_6.i ILP32 false +ECA eca-programs/Problem101_label05.c ILP32 false +ProductLines product-lines/elevator_spec14_product24.cil.c ILP32 false +XCSP xcsp/AllInterval-006.c ILP32 false +LinkedLists forester-heap/sll-rb-sentinel-2.i ILP32 false +Arrays array-examples/sorting_selectionsort_2_ground.i ILP32 false +Loops loop-acceleration/simple_2-2.c ILP32 false +Sequentialized seq-mthreaded/pals_STARTPALS_ActiveStandby.4_1.ufo.UNBOUNDED.pals.c ILP32 false +Floats loop-floats-scientific-comp/loop4.i ILP32 false +ECA eca-programs/Problem101_label18.c ILP32 false +ProductLines product-lines/elevator_spec1_product20.cil.c ILP32 false +XCSP xcsp/AllInterval-008.c ILP32 false +LinkedLists heap-manipulation/tree-4.i ILP32 false +Arrays array-examples/standard_copy1_ground-2.i ILP32 false +Loops loops-crafted-1/nested_delay_notd2.c ILP32 false +Sequentialized seq-mthreaded/pals_STARTPALS_ActiveStandby.5.ufo.BOUNDED-10.pals.c ILP32 false +ECA eca-programs/Problem102_label13.c ILP32 false +ProductLines product-lines/elevator_spec1_product30.cil.c ILP32 false +XCSP xcsp/AllInterval-010.c ILP32 false +LinkedLists list-properties/splice-1.i ILP32 false +Arrays array-examples/standard_copy4_ground-2.i ILP32 false +Loops loops/count_up_down-2.c ILP32 false +Sequentialized seq-mthreaded/pals_STARTPALS_Triplicated.2.ufo.BOUNDED-10.pals.c ILP32 false +ECA eca-programs/Problem102_label50.c ILP32 false +ProductLines product-lines/elevator_spec2_product24.cil.c ILP32 false +XCSP xcsp/AllInterval-012.c ILP32 false +Arrays array-examples/standard_copy7_ground-1.i ILP32 false +Loops loops/invert_string-1.c ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.3.1.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-programs/Problem103_label56.c ILP32 false +ProductLines product-lines/elevator_spec2_productSimulator.cil.c ILP32 false +XCSP xcsp/AllInterval-014.c ILP32 false +Arrays array-examples/standard_minInArray_ground-1.i ILP32 false +Loops loops/nec20.c ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.3.3.ufo.BOUNDED-6.pals.c ILP32 false +ECA eca-rers2012/Problem03_label45.c ILP32 false +ProductLines product-lines/elevator_spec3_product23.cil.c ILP32 false +XCSP xcsp/AllInterval-016.c ILP32 false +Arrays array-fpi/brs1f.c ILP32 false +Loops loops/terminator_01.c ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.3.4.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-rers2012/Problem04_label35.c ILP32 false +ProductLines product-lines/elevator_spec3_product32.cil.c ILP32 false +XCSP xcsp/AllInterval-018.c ILP32 false +Arrays array-fpi/brs5f.c ILP32 false +Loops loops/vogal-2.i ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.4.2.ufo.BOUNDED-8.pals.c ILP32 false +ECA eca-rers2012/Problem05_label30.c ILP32 false +ProductLines product-lines/elevator_spec9_productSimulator.cil.c ILP32 false +XCSP xcsp/AllInterval-025.c ILP32 false +Arrays array-fpi/condmf.c ILP32 false +Loops nla-digbench-scaling/bresenham-ll_unwindbound100.c ILP32 false +Sequentialized seq-mthreaded/pals_floodmax.4.3.ufo.UNBOUNDED.pals.c ILP32 false +ECA eca-rers2012/Problem05_label55.c ILP32 false +ProductLines product-lines/email_spec0_product31.cil.c ILP32 false +XCSP xcsp/AllInterval-035.c ILP32 false From 835f618fe6fda073a6f8952a2291c6426674a1c1 Mon Sep 17 00:00:00 2001 From: Guilherme Bernardo Date: Sun, 23 Aug 2026 14:34:08 -0400 Subject: [PATCH 11/17] fix(testcomp): recover the vector even when the runtime recorded nothing Measured pairwise against v8 on 199 tasks, the previous commit's gate cost more than it saved: twelve tasks that HAD been covered stopped being covered, all twelve ERROR -> UNKNOWN with an empty suite. Every one was a case where the target really was reached and the runtime never wrote it down -- the state that reaches the target aborts, an aborted KLEE state runs no exit handler, and map2check_property is written by that handler. So foundViolation being false does not mean nothing was found; it means nothing was recorded, and it is false precisely in the case that matters. Requiring foundViolation was a defence against picking up an assumption failure, whose abort looks identical to a real one. That defence now lives upstream where it belongs: NonDetPass rewrites the abort-based assumptions into path pruning, so an abort.err can no longer BE an assumption. With the source of false aborts gone, the gate only blocked legitimate recoveries. The recovery therefore moves ahead of the early return, for reachability mode only, and says out loud when it fires -- a suite emitted on a run that reported no violation is worth a line in the log. Worth recording how this was caught: the control group. The run was split into categories saturated with the assumption idiom and categories without, and it was the CONTROL group moving -- 30 covered down to 19 -- that exposed the regression. A single-group run would have shown the treatment gain and hidden the cost underneath it. Co-Authored-By: Claude Opus 5 (1M context) --- modules/frontend/map2check.cpp | 62 +++++++++++++++++++++++++++------- 1 file changed, 50 insertions(+), 12 deletions(-) diff --git a/modules/frontend/map2check.cpp b/modules/frontend/map2check.cpp index 23d065e73..89e89615a 100644 --- a/modules/frontend/map2check.cpp +++ b/modules/frontend/map2check.cpp @@ -150,7 +150,7 @@ void emitTestSuite(const std::string &outputDir, const std::string &programFile, const std::string &entryFunction, const std::string &architecture, const std::string &specification, bool foundViolation, - bool coverBranches) { + bool coverBranches, Map2Check::Map2CheckMode mode) { Map2Check::TestSuiteMetadata metadata; metadata.producer = std::string("Map2Check ") + Map2CheckVersion; metadata.specification = specification; @@ -192,6 +192,35 @@ void emitTestSuite(const std::string &outputDir, const std::string &programFile, return; } + // Last chance to find a vector before declaring the suite empty. + // + // A run can reach the target and still not record it: the state that reaches + // it aborts, an aborted KLEE state runs no exit handler, and + // map2check_property is written by that handler. So foundViolation being + // false does not mean nothing was found -- it means nothing was WRITTEN + // DOWN. Measured: twelve tasks whose suites covered the error came back with + // no verdict at all. + // + // Safe to consult now, and only now, because NonDetPass rewrites the + // abort-based assumptions into path pruning: an abort.err can no longer be + // an assumption failure, so its presence in reachability mode is evidence + // the target really was hit. + if (!foundViolation && mode == Map2Check::Map2CheckMode::REACHABILITY_MODE) { + std::vector recovered = + Map2Check::readViolatingKtest(Map2Check::kleeOutputDir); + if (!recovered.empty()) { + Map2Check::Log::Warning( + "the runtime recorded no violation, but KLEE reported an aborting " + "path -- emitting its input vector (" + + std::to_string(recovered.size()) + " inputs)"); + if (writer.writeTestCase(recovered, true)) { + Map2Check::Log::Info("Test suite written to " + outputDir + " (" + + std::to_string(recovered.size()) + " inputs)"); + return; + } + } + } + // No violation means no test case, but the suite still has to exist. A // missing test-suite/ directory reads to the competition harness as a tool // that crashed; a suite carrying metadata and zero test cases says the tool @@ -217,18 +246,27 @@ void emitTestSuite(const std::string &outputDir, const std::string &programFile, // order the program consumed them, straight from the instrumented run, while // the .ktest is KLEE's view of the same path. // - // Gated on the runtime having actually recorded the target as reached, and - // that gate is the whole difference between recovering a vector and - // inventing one. KLEE writes a .err for ANY abort, and the sv-benchmarks - // idiom for an assumption is + // Deliberately NOT gated on the runtime having recorded the violation, and + // that took two measurements to get right. + // + // The danger this fallback had was picking up an assumption failure: the + // sv-benchmarks idiom is `void assume_abort_if_not(int c){ if(!c) abort(); }` + // and an aborting assumption leaves an abort.err indistinguishable from a + // real one. A suite built from it carried 25 inputs and covered 0.0%. // - // void assume_abort_if_not(int c) { if (!c) abort(); } + // The first attempt at a defence was to require foundViolation. It cost + // more than it saved: measured pairwise against the previous run, twelve + // tasks that HAD been covered stopped being covered, every one of them a + // case where the target really was reached and the runtime simply never + // recorded it -- the state that reaches the target aborts, and an aborted + // state runs no exit handler, so map2check_property stays empty exactly + // when it matters most. // - // so a path that merely violates an assumption -- a path the competition - // considers out of scope -- leaves an abort.err behind exactly like a real - // violation does. Taking it produced suites that looked convincing and - // covered nothing: one measured case emitted 25 inputs and scored 0.0%. - if (inputs.empty() && foundViolation) { + // The real defence is upstream: NonDetPass now rewrites the abort-based + // assumptions into path pruning, so an abort.err can no longer come from + // one. With the source of false aborts removed, requiring foundViolation + // only blocks legitimate recoveries. + if (inputs.empty()) { inputs = Map2Check::readViolatingKtest(Map2Check::kleeOutputDir); if (!inputs.empty()) { Map2Check::Log::Info( @@ -551,7 +589,7 @@ int map2check_execution(map2check_args args) { args.architecture, resolveSpecification(args.propertyFile, caller->getOriginalPath(), args.mode), - foundViolation, args.coverBranches); + foundViolation, args.coverBranches, args.mode); } // (6) Clean map2check execution (folders and temp files) From 24e81e356f33b905331a3e37c1a064226b808fcb Mon Sep 17 00:00:00 2001 From: Guilherme Bernardo Date: Sun, 23 Aug 2026 16:46:44 -0400 Subject: [PATCH 12/17] fix(fuzzer): read a whole type's worth of bytes, and say when the run is undecided Two defects on the LibFuzzer path, both found while designing the seed exchange, and the first of them is what made that exchange impossible. Every generator took ONE byte and cast it to the type: #define MAP2CHECK_NON_DET_GENERATOR(type) \ type map2check_non_det_##type() { return (type)get_next_input_from_fuzzer(); } so a long, a short, a size_t or a pointer could only ever hold 0..255, and a double could only be an integral value between 0.0 and 255.0. Nothing negative was reachable at all: an unsigned byte cast to a signed type stays non-negative. Half of every signed type was outside the fuzzer's reach. Only int differed, and it read EIGHT bytes to build a value it then truncated to four. Measured on a short: short s = nondet(); if (s < 0) before: not found after: FAILED short s = nondet(); if (s == -4242) before: not found after: FAILED The first of those is half of all values. It also blocked what this was groundwork for. The byte layout IS the exchange format between the engines, and a KLEE vector holding short x = 4242 cannot be written into a slot one byte wide. sizeof(type) is exactly what NonDetGeneratorKlee.c already passes to klee_make_symbolic, so both engines now agree on what a vector means -- the prerequisite for seeding either from the other. While there: the fuzzer-chosen string length feeding malloc in map2check_non_det_pchar is now bounded. With a full-width unsigned it could ask for four billion bytes. Second defect: the UNKNOWN branch was guarded on the generator being KLEE, so an undecided LibFuzzer run printed no verdict line at all -- and every harness here, plus the BenchExec tool-info, reads that silence as a crash. It is finding G one layer up: the analysis concluded and did not say so. Whole categories in the engine comparison were recorded as ERROR for this reason. Worth recording a mistake of my own: my first three micro-benchmarks failed on everything, including char == 65, and I nearly reported the fuzzer path as broken. They only DECLARED reach_error. LibFuzzer detects a violation through the crash file the abort produces, and the benchmarks define reach_error to call __assert_fail; without a body that aborts there is no crash to catch. The tool was right and the test was wrong -- caught by running the same case against the previous build and seeing identical behaviour. Co-Authored-By: Claude Opus 5 (1M context) --- .../library/lib/NonDetGeneratorLibFuzzy.c | 57 ++++++++++++----- modules/frontend/map2check.cpp | 17 +++-- .../integration/test_testcomp_regressions.sh | 64 +++++++++++++++++++ 3 files changed, 118 insertions(+), 20 deletions(-) diff --git a/modules/backend/library/lib/NonDetGeneratorLibFuzzy.c b/modules/backend/library/lib/NonDetGeneratorLibFuzzy.c index b425d22c1..bc7dbedf3 100644 --- a/modules/backend/library/lib/NonDetGeneratorLibFuzzy.c +++ b/modules/backend/library/lib/NonDetGeneratorLibFuzzy.c @@ -76,8 +76,34 @@ int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) { return 0; } +/* Fills `out` with `size` bytes from the fuzzer's buffer, in target order. + * + * The generators below used to take ONE byte and cast it, whatever the type. + * A `long` could therefore only ever be 0..255; so could a `short`, a + * `size_t`, a pointer -- and a `double` could only be an integral value + * between 0.0 and 255.0. Nothing negative was reachable at all, because an + * unsigned byte cast to a signed type stays non-negative. + * + * Beyond the obvious loss of reach, this is what made seeding impossible: the + * byte layout IS the exchange format between the two engines, and a KLEE + * vector holding short x = 4242 cannot be written into a slot one byte wide. + * Consuming sizeof(type) puts the fuzzer on the same layout KLEE already uses + * -- NonDetGeneratorKlee.c passes sizeof(non_det) to klee_make_symbolic -- so + * a vector means the same thing to both. */ +static void get_bytes_from_fuzzer(void *out, size_t size) { + unsigned char *destination = (unsigned char *)out; + size_t i = 0; + for (; i < size; i++) { + destination[i] = get_next_input_from_fuzzer(); + } +} + #define MAP2CHECK_NON_DET_GENERATOR(type) \ - type map2check_non_det_##type() { return (type)get_next_input_from_fuzzer(); } + type map2check_non_det_##type() { \ + type value; \ + get_bytes_from_fuzzer(&value, sizeof(value)); \ + return value; \ + } MAP2CHECK_NON_DET_GENERATOR(char) MAP2CHECK_NON_DET_GENERATOR(pointer) @@ -96,28 +122,27 @@ MAP2CHECK_NON_DET_GENERATOR(sector_t) MAP2CHECK_NON_DET_GENERATOR(double) // MAP2CHECK_NON_DET_GENERATOR(uint) -// Considering an int on a x64, then a 64 bit integer is 8 times a 8 bit integer -int map2check_non_det_int() { - uint64_t result = 0; - int i = 0; - for (; i < 8; i++) - /* cast before the shift: uint8_t promotes to int, and shifting an int - * by up to 56 bits is undefined behavior */ - result |= (uint64_t)get_next_input_from_fuzzer() << (8 * i); - - return (int)result; -} +/* Was reading EIGHT bytes and truncating to int, so half of every integer's + * worth of fuzzer entropy was consumed and thrown away -- and, worse for + * seeding, the layout did not match what KLEE writes for the same read. */ +MAP2CHECK_NON_DET_GENERATOR(int) -uint map2check_non_det_uint() { return (uint)map2check_non_det_int(); } +MAP2CHECK_NON_DET_GENERATOR(uint) +MAP2CHECK_NON_DET_GENERATOR(unsigned) -unsigned map2check_non_det_unsigned() { - return (unsigned)map2check_non_det_int(); -} +/* Upper bound on a fuzzer-chosen string length. + * + * The length comes from a full-width unsigned, so before this the malloc below + * could be asked for four billion bytes on a whim. Any string long enough to + * matter for a benchmark fits well inside this. */ +#define MAP2CHECK_MAX_FUZZED_STRING 4096 char *map2check_non_det_pchar() { unsigned length = map2check_non_det_unsigned(); if (length == 0) return NULL; + if (length > MAP2CHECK_MAX_FUZZED_STRING) + length = MAP2CHECK_MAX_FUZZED_STRING; /* heap allocation: returning a local VLA would leave the caller with a * dangling pointer (cppcheck returnDanglingLifetime) */ char *string = malloc(length); diff --git a/modules/frontend/map2check.cpp b/modules/frontend/map2check.cpp index 89e89615a..604b1df00 100644 --- a/modules/frontend/map2check.cpp +++ b/modules/frontend/map2check.cpp @@ -558,10 +558,19 @@ int map2check_execution(map2check_args args) { } } else if (propertyViolated == Map2Check::PropertyViolated::UNKNOWN) { - if (generator == Map2Check::NonDetGenerator::Klee) { - Map2Check::Log::Info("Unable to prove or falsify the program."); - Map2Check::Log::Info("VERIFICATION UNKNOWN"); - if (args.debugMode) counterExample->generateTestCase(); + // Printed for every generator, not just KLEE. Guarded on Klee, an + // undecided LibFuzzer run ended with NO verdict line at all, and a caller + // that parses stdout for one -- every harness here, and the BenchExec + // tool-info -- reads that silence as the tool having crashed. It is the + // same defect as the discarded exit code (finding G), one layer up: the + // analysis reached a conclusion and did not say so. + // + // Visible in the numbers: the fuzzer arm of the engine comparison recorded + // ERROR for whole categories that had simply come back undecided. + Map2Check::Log::Info("Unable to prove or falsify the program."); + Map2Check::Log::Info("VERIFICATION UNKNOWN"); + if (args.debugMode && generator == Map2Check::NonDetGenerator::Klee) { + counterExample->generateTestCase(); } } else { Map2Check::Log::Info("Started counter example generation"); diff --git a/tests/integration/test_testcomp_regressions.sh b/tests/integration/test_testcomp_regressions.sh index 0d34a7f88..3cd522dbc 100755 --- a/tests/integration/test_testcomp_regressions.sh +++ b/tests/integration/test_testcomp_regressions.sh @@ -306,6 +306,70 @@ else fail "scratch directory" "not kept under --debug" fi +# --- 9. every generator must report a verdict -------------------------------- +# The UNKNOWN branch was guarded on the generator being KLEE, so an undecided +# LibFuzzer run printed no verdict line at all. Every harness here parses +# stdout for one, and so does the BenchExec tool-info; silence reads as a +# crash. Whole categories came back as ERROR in the engine comparison for this +# reason alone. +mkdir -p "$WORK/verdict" +cat > "$WORK/verdict/hard.c" <<'EOF' +extern void __assert_fail(const char *, const char *, unsigned int, + const char *) __attribute__((__noreturn__)); +void reach_error(void) { __assert_fail("0", "hard.c", 3, "reach_error"); } +extern int __VERIFIER_nondet_int(void); +int main(void) { + int a = __VERIFIER_nondet_int(); + int b = __VERIFIER_nondet_int(); + /* Two exact 32-bit matches at once: a fuzzer will not stumble onto this + * inside the budget, so the run ends undecided -- which is the point. */ + if (a == 1234567 && b == 7654321) { reach_error(); } + return 0; +} +EOF +for gen in fuzzer symex; do + ( cd "$WORK/verdict" && MAP2CHECK_PATH="$MAP2CHECK_DIR" timeout -k 10 120 "$MAP2CHECK" \ + --target-function --target-function-name reach_error \ + --nondet-generator "$gen" --timeout 45 hard.c ) > "$WORK/verdict/$gen.log" 2>&1 + if grep -qE 'VERIFICATION (FAILED|SUCCEEDED|UNKNOWN)' "$WORK/verdict/$gen.log"; then + ok "the $gen generator reports a verdict" + else + fail "$gen verdict" "no VERIFICATION line -- a caller reads this as a crash" + fi +done + +# --- 10. the fuzzer must reach the whole width of a type --------------------- +# Each generator took ONE byte and cast it, so a short could only be 0..255 and +# nothing negative was reachable at all -- an unsigned byte cast to a signed +# type stays non-negative. Half of every signed type was unreachable. +# +# It also blocked seeding: the byte layout IS the exchange format between the +# engines, and a KLEE vector holding short x = 4242 cannot be written into a +# slot one byte wide. sizeof(type) is what KLEE already passes to +# klee_make_symbolic, so this puts both engines on one layout. +mkdir -p "$WORK/width" +cat > "$WORK/width/neg.c" <<'EOF' +extern void __assert_fail(const char *, const char *, unsigned int, + const char *) __attribute__((__noreturn__)); +void reach_error(void) { __assert_fail("0", "neg.c", 3, "reach_error"); } +extern short __VERIFIER_nondet_short(void); +int main(void) { + short s = __VERIFIER_nondet_short(); + if (s < 0) { reach_error(); } + return 0; +} +EOF +( cd "$WORK/width" && MAP2CHECK_PATH="$MAP2CHECK_DIR" timeout -k 10 150 "$MAP2CHECK" \ + --target-function --target-function-name reach_error \ + --nondet-generator fuzzer --timeout 60 neg.c ) > "$WORK/width/run.log" 2>&1 + +if grep -q 'VERIFICATION FAILED' "$WORK/width/run.log"; then + ok "the fuzzer reaches a negative short" +else + fail "fuzzer value range" \ + "did not reach short < 0 -- half of the type is unreachable" +fi + echo " ---" echo " Results: $PASSED passed, $FAILED failed" [ "$FAILED" -eq 0 ] || exit 1 From f8b9dc82c8b8e71bda66994743fa07db21e8d10d Mon Sep 17 00:00:00 2001 From: Guilherme Bernardo Date: Sun, 23 Aug 2026 17:28:03 -0400 Subject: [PATCH 13/17] test(testcomp): measure the hybrid, which is what the tool actually runs Every Test-Comp measurement in this repository forced --nondet-generator symex, so all of them measured a mode nobody runs: the tool's own default, with no generator flag, is LibFuzzer at 0.2x the budget and then KLEE. Measured on the same 372-task corpus, paired: symex 23% covered fuzzer 35% covered hybrid 47% covered and the hybrid captured the entire UNION of what the two engines cover alone -- 13 tasks only the fuzzer reaches, 6 only KLEE reaches, and the hybrid gets all of them. Running the weaker engine first for a fifth of the budget costs almost nothing and loses nothing. Both harnesses now default to it, including the CI gate, which was exercising a configuration the competition never sees. Result directories written before this carry symex numbers. They are not comparable to hybrid runs and must not be pooled. Co-Authored-By: Claude Opus 5 (1M context) --- tests/testcomp/run_testcomp_evaluation.sh | 25 ++++++++++++++++++++++- tests/testcomp/run_testcov.sh | 6 +++++- 2 files changed, 29 insertions(+), 2 deletions(-) diff --git a/tests/testcomp/run_testcomp_evaluation.sh b/tests/testcomp/run_testcomp_evaluation.sh index 042d1c582..1e7de03af 100755 --- a/tests/testcomp/run_testcomp_evaluation.sh +++ b/tests/testcomp/run_testcomp_evaluation.sh @@ -19,6 +19,7 @@ # PROPERTY cover-error | cover-branches (required) # RESULTS_DIR where the CSV and raw logs go (required) # SHARD/SHARDS process every SHARDS-th task starting at SHARD (default 0/1) +# GENERATOR symex | fuzzer | hybrid (default symex) # BUDGET seconds given to map2check per task (default 60) # TESTCOV_S seconds given to TestCov per task (default 90) # DEADLINE_S stop starting new tasks after this many seconds (default 18000) @@ -41,6 +42,28 @@ BUDGET="${BUDGET:-60}" TESTCOV_S="${TESTCOV_S:-90}" DEADLINE_S="${DEADLINE_S:-18000}" +# Which engine produces the inputs. Parameterised so the three arms can be +# compared on the SAME task list: +# +# symex KLEE only -- every Test-Comp measurement so far used this +# fuzzer LibFuzzer only +# hybrid the tool's actual default: LibFuzzer at 0.2x the budget, then KLEE +# +# The default is hybrid, which is also the tool's own default when no +# --nondet-generator is given. Every Test-Comp measurement before 2026-08-23 +# forced symex and so measured a mode nobody actually runs. +# +# Measured on the same 372-task corpus: symex 23% covered, fuzzer 35%, hybrid +# 47% -- and the hybrid captured the whole UNION of what the two engines reach +# alone. Directories written before the switch carry `symex` in this comment's +# place; they are not comparable to hybrid runs and should not be pooled. +GENERATOR="${GENERATOR:-hybrid}" +case "$GENERATOR" in + symex|fuzzer) GENERATOR_FLAG="--nondet-generator $GENERATOR" ;; + hybrid) GENERATOR_FLAG="" ;; # absent flag IS the hybrid path + *) echo "unknown GENERATOR: $GENERATOR" >&2; exit 2 ;; +esac + MAP2CHECK_DIR="${MAP2CHECK_PATH:?set MAP2CHECK_PATH}" MAP2CHECK="$MAP2CHECK_DIR/map2check" @@ -108,7 +131,7 @@ while IFS=$'\t' read -r category program data_model expected <&3; do ( cd "$work" || exit 1 MAP2CHECK_PATH="$MAP2CHECK_DIR" timeout -k 10 $((BUDGET + 30)) "$MAP2CHECK" \ - --nondet-generator symex --generate-test-suite $GOAL_FLAGS \ + $GENERATOR_FLAG --generate-test-suite $GOAL_FLAGS \ --property-file prop.prp --architecture "$arch" \ --timeout "$BUDGET" "$name" ) > "$work/map2check.log" 2>&1 "$WORK/map2check.log" 2>&1 if [ ! -d "$WORK/test-suite" ]; then From ca7af4fb33b5c5c6d7ab4d61139f9306294a540d Mon Sep 17 00:00:00 2001 From: Guilherme Bernardo Date: Sun, 23 Aug 2026 18:09:22 -0400 Subject: [PATCH 14/17] feat(hybrid): let the two engines hand each other input vectors The two engines cover almost disjoint sets. Measured over the same 372 tasks: 65 covered only by the fuzzer, 47 only by KLEE, and the hybrid takes the whole union. Until now they shared nothing -- KLEE started from scratch after the fuzzer had finished, knowing none of what it found. LibFuzzer was not even keeping its own corpus. The command carried no corpus directory, so everything discovered in its fifth of the budget lived in memory and died with the process, every run. seeds/ a directory both engines read and write, in the scratch directory they already share as a working directory. A directory rather than a value handed between phases, because it has to survive between phases, between runs and between alternations -- which is what time-slicing needs. KLEE -> fuzzer each explored path's .ktest becomes a seed file. Concatenating a .ktest's objects yields exactly the buffer that drives the fuzzer down the same path -- sound only since both engines started consuming sizeof(type) per read. Before that fix most vectors had nowhere to go. third phase the order is fuzzer then KLEE, so KLEE's vectors had no consumer inside the same run and the exchange would only have paid off on a later one. A short seeded fuzzer phase now closes the loop: KLEE solves the guard mutation cannot reach, and the fuzzer moves outward from there faster than KLEE can fork. Under a fixed budget this is not merely speed. Every second KLEE spends rediscovering a path the fuzzer already walked is a second not spent going deeper, so the exchange buys depth the run would not otherwise reach. Behind --seed-exchange, off by default: the hybrid was measured at 44.6% covered over 372 tasks in its current shape, and that figure has to keep meaning what it means until this one is measured beside it. Known limitation, stated rather than hidden: the fuzzer -> KLEE direction is implemented but rarely fires. It reads the runtime's nondet log for the typed vector, and that log is only written on a violating execution -- which is precisely the case where KLEE is no longer needed. Carrying a fuzzer corpus file back into a .ktest needs the type sequence of the reads, which the raw bytes do not carry; recovering it means replaying each corpus entry through the instrumented binary with logging on. Left for when the measurement says the direction is worth that. The tests assert the mechanism, not a coverage gain: that the channel exists, stays off by default, and carries vectors in the direction it claims. Whether cooperation finds more is a question for the corpus. Co-Authored-By: Claude Opus 5 (1M context) --- modules/frontend/caller.cpp | 87 ++++++++++- modules/frontend/caller.hpp | 32 ++++ modules/frontend/map2check.cpp | 28 ++++ modules/frontend/test_suite/ktest_reader.cpp | 142 ++++++++++++++++++ modules/frontend/test_suite/ktest_reader.hpp | 25 +++ .../integration/test_testcomp_regressions.sh | 63 ++++++++ 6 files changed, 373 insertions(+), 4 deletions(-) diff --git a/modules/frontend/caller.cpp b/modules/frontend/caller.cpp index 8c1f8679d..e2e8334ed 100644 --- a/modules/frontend/caller.cpp +++ b/modules/frontend/caller.cpp @@ -26,6 +26,7 @@ #include #include +#include "test_suite/ktest_reader.hpp" #include "utils/gen_crypto_hash.hpp" #include "utils/log.hpp" #include "utils/tools.hpp" @@ -99,6 +100,55 @@ std::string Caller::postOptimizationFlags() { return flags.str(); } +unsigned Caller::exportKleeVectorsAsSeeds() { + std::error_code error; + std::filesystem::create_directories(Caller::seedDirectory, error); + + std::vector> ignored; + unsigned written = 0; + unsigned index = 0; + for (const auto &entry : std::filesystem::directory_iterator( + Map2Check::kleeOutputDir, error)) { + if (entry.path().extension() != ".ktest") continue; + std::vector objects = + Map2Check::readKtestFile(entry.path().string()); + if (objects.empty()) continue; + + std::vector bytes = Map2Check::ktestToFuzzerBytes(objects); + if (bytes.empty()) continue; + + // Named by index rather than by content hash: LibFuzzer renames what it + // keeps to its own hash anyway, so a second one here buys nothing. + std::ostringstream name; + name << Caller::seedDirectory << "/klee-" << index++; + std::ofstream out(name.str(), std::ios::binary); + if (!out.is_open()) continue; + out.write(reinterpret_cast(bytes.data()), + static_cast(bytes.size())); + if (out.good()) ++written; + } + if (written > 0) { + Map2Check::Log::Info("Seeded the fuzzer corpus with " + + std::to_string(written) + " vectors from KLEE"); + } + return written; +} + +std::string Caller::exportFuzzerVectorAsKtest() { + std::vector objects = + Map2Check::readNonDetLogAsObjects(Map2Check::kleeLogCSV); + if (objects.empty()) return ""; + + std::error_code error; + std::filesystem::create_directories(Caller::seedDirectory, error); + const std::string path = + std::string(Caller::seedDirectory) + "/from-fuzzer.ktest"; + if (!Map2Check::writeKtestFile(path, objects)) return ""; + Map2Check::Log::Info("Seeding KLEE with the fuzzer's vector (" + + std::to_string(objects.size()) + " inputs)"); + return path; +} + void Caller::cleanGarbage() { std::filesystem::current_path(currentPath); std::ostringstream removeCommand; @@ -445,6 +495,17 @@ void Caller::executeAnalysis(std::string solvername) { // .ktest files on one run and zero on the next, the difference being // whether an error happened to be hit early. A suite that depends on // that is not a suite. + // A starting point from whatever ran before, when there is one. KLEE + // replays the seed and then explores around it, instead of rediscovering + // from nothing a path the fuzzer already walked -- which under a fixed + // budget is not merely faster, it is depth the run would not otherwise + // have reached. + std::string seedFlag; + if (this->seedExchange) { + const std::string seed = exportFuzzerVectorAsKtest(); + if (!seed.empty()) seedFlag = " --seed-file=" + seed; + } + // Depth-first for Cover-Branches, and the reason is about what survives // the deadline rather than about search quality. // @@ -480,7 +541,7 @@ void Caller::executeAnalysis(std::string solvername) { // --allow-external-sym-calls // -use-cache kleeCommand << " --external-calls=all" - << stopPolicy << searchPolicy + << stopPolicy << searchPolicy << seedFlag << " --optimize" << " --use-cex-cache" << " --solver-backend=" + solvername + " " @@ -493,7 +554,7 @@ void Caller::executeAnalysis(std::string solvername) { Map2Check::Log::Info("Solver metaSMT caller: " + solvername); kleeCommand << " --external-calls=all" - << stopPolicy << searchPolicy + << stopPolicy << searchPolicy << seedFlag << " --optimize" << " --use-cex-cache" << " --solver-backend=metasmt " @@ -505,6 +566,13 @@ void Caller::executeAnalysis(std::string solvername) { Map2Check::Log::Debug(kleeCommand.str()); int result = system(kleeCommand.str().c_str()); + if (this->seedExchange) { + // Written after KLEE rather than before the next phase, because the + // .ktest files are in the scratch directory that cleanGarbage() will + // remove -- and because a later alternation, or a resumed run, should + // find them already there. + exportKleeVectorsAsSeeds(); + } Map2Check::Log::Warning("Exited klee with " + std::to_string(result)); if (result == 31744) // Timeout gotTimeout = true; @@ -519,9 +587,20 @@ void Caller::executeAnalysis(std::string solvername) { // LibFuzzer forks workers that must not outlive the budget. command << "timeout -k " << Map2Check::killGracePeriod << " " << (0.2 * this->timeout) << " "; + // A corpus DIRECTORY, not just a run. Without one LibFuzzer keeps its + // corpus in memory and throws it away when the process ends: everything + // it discovered in its slice of the budget was discarded, every run. + // With one, the interesting inputs persist -- which is what makes them + // available to the other engine, and to a later alternation. + std::string corpus; + if (this->seedExchange) { + std::error_code error; + std::filesystem::create_directories(Caller::seedDirectory, error); + corpus = std::string(" ") + Caller::seedDirectory; + } command << "./" + programHash + - "-fuzzed.out -jobs=8 -use_value_profile=1 " - << " > fuzzer.output"; + "-fuzzed.out -jobs=8 -use_value_profile=1" + << corpus << " > fuzzer.output"; int result = system(command.str().c_str()); Map2Check::Log::Warning("Exited fuzzer with " + std::to_string(result)); diff --git a/modules/frontend/caller.hpp b/modules/frontend/caller.hpp index 1c3432d02..dcc31823c 100644 --- a/modules/frontend/caller.hpp +++ b/modules/frontend/caller.hpp @@ -105,6 +105,38 @@ class Caller { /** Remove generated files for verification */ void cleanGarbage(); + /** Turns on the exchange of input vectors between the two engines. + * + * Off by default so the hybrid keeps behaving exactly as it was measured + * (symex 27%, fuzzer 32%, hybrid 45% over the same 372 tasks); promoting it + * is a separate decision that has to be earned by its own measurement. */ + bool seedExchange = false; + + /** Directory the two engines use to hand each other input vectors. + * + * A directory of files rather than a value passed from one phase to the + * next, and the shape is the point: it survives between phases, between + * runs, and between alternations -- which is what time-slicing will need. + * LibFuzzer treats it as its corpus and grows it; the KLEE phase drops its + * own path vectors in. + * + * Relative, because both engines run with the scratch directory as their + * working directory. */ + static constexpr const char* seedDirectory = "seeds"; + + /** Writes KLEE's per-path vectors into the seed corpus. + * + * Sound only because the engines agree on widths now: concatenating a + * .ktest's objects yields exactly the byte buffer that would drive the + * fuzzer down the same path. Returns how many seeds were written. */ + unsigned exportKleeVectorsAsSeeds(); + + /** Writes what the fuzzer consumed as a .ktest KLEE can start from. + * + * The nondet log is the only record of a fuzzer run carrying both value and + * type, which is what a .ktest needs. Returns the path, or empty. */ + std::string exportFuzzerVectorAsKtest(); + /** Instrument and execute nondeterministic generator */ void applyNonDetGenerator(); diff --git a/modules/frontend/map2check.cpp b/modules/frontend/map2check.cpp index 604b1df00..d6b6b8ec3 100644 --- a/modules/frontend/map2check.cpp +++ b/modules/frontend/map2check.cpp @@ -385,6 +385,7 @@ struct map2check_args { bool invCrabLlvm = false; bool generateTestSuite = false; bool coverBranches = false; + bool seedExchange = false; std::string testSuiteDir = "test-suite"; std::string propertyFile; std::string architecture = "64bit"; @@ -457,6 +458,7 @@ int map2check_execution(map2check_args args) { caller = std::make_unique(args.inputFile, args.mode, generator); caller->c_program_fullpath = args.inputFile; + caller->seedExchange = args.seedExchange; caller->setTimeout(args.timeout); caller->entryFunction = args.entryFunction; caller->wasmMode = args.wasmMode; @@ -658,6 +660,9 @@ z3 (Z3 is default), btor (Boolector), and yices2 (Yices))") "\temits a Test-Comp test suite reproducing the violation found") ("test-suite-dir", po::value()->default_value("test-suite"), "\tdirectory to write the test suite into") + ("seed-exchange", + "\tlet the two engines hand each other input vectors through a shared " + "seed corpus (hybrid runs; off by default)") ("cover-branches", "\temit one test case per path KLEE explored, from its .ktest output, " "instead of the single violating vector (Test-Comp Cover-Branches)") @@ -767,6 +772,9 @@ z3 (Z3 is default), btor (Boolector), and yices2 (Yices))") if (vm.count("test-suite-dir")) { args.testSuiteDir = vm["test-suite-dir"].as(); } + if (vm.count("seed-exchange")) { + args.seedExchange = true; + } if (vm.count("cover-branches")) { args.coverBranches = true; // The mode has to change too, not just the emitter. Without this the run @@ -860,6 +868,26 @@ z3 (Z3 is default), btor (Boolector), and yices2 (Yices))") return result; } } + // A third phase, and it is what closes the exchange loop. + // + // The order is fuzzer then KLEE, so KLEE's vectors -- written into the + // seed corpus at the end of its phase -- have no consumer inside the + // same run. Without this the exchange only ever paid off on a LATER + // run. Handing them straight back to the fuzzer is what turns two + // engines running in sequence into two engines that cooperate: KLEE + // solves the guard the fuzzer could not reach by mutation, and the + // fuzzer mutates outward from there far faster than KLEE can fork. + // + // Behind the flag: the hybrid was measured at 45% covered over 372 + // tasks in its current shape, and that number should keep meaning what + // it means until this one is measured beside it. + if (args.seedExchange && !foundViolation) { + args.generator = Map2Check::NonDetGenerator::LibFuzzer; + result = map2check_execution(args); + if (result != SUCCESS) { + return result; + } + } } else { int result = map2check_execution(args); diff --git a/modules/frontend/test_suite/ktest_reader.cpp b/modules/frontend/test_suite/ktest_reader.cpp index fe0454e35..1d9ce4550 100644 --- a/modules/frontend/test_suite/ktest_reader.cpp +++ b/modules/frontend/test_suite/ktest_reader.cpp @@ -12,6 +12,7 @@ #include #include #include +#include #include #include #include @@ -42,6 +43,78 @@ bool readBigEndian32(std::ifstream& in, uint32_t* out) { return true; } +void putBigEndian32(std::ofstream& out, uint32_t value) { + const unsigned char raw[4] = { + static_cast((value >> 24) & 0xff), + static_cast((value >> 16) & 0xff), + static_cast((value >> 8) & 0xff), + static_cast(value & 0xff)}; + out.write(reinterpret_cast(raw), 4); +} + +/** The name and width KLEE uses for a NONDET_TYPE enumerator. + * + * Both halves have to agree with the runtime or a seed means nothing: the name + * is what klee_make_symbolic was called with (NonDetGeneratorKlee.c), and the + * width is what the fuzzer consumes per read (NonDetGeneratorLibFuzzy.c). + * Enumerator values come from enum NONDET_TYPE in Map2CheckTypes.h. */ +struct NonDetTypeInfo { + const char* name; + size_t width; + bool isFloating; + bool isUnsigned; +}; + +NonDetTypeInfo nonDetTypeInfo(int type) { + switch (type) { + case 0: return {"non_det_int", sizeof(int), false, false}; + case 1: return {"non_det_char", sizeof(char), false, false}; + case 2: return {"non_det_pointer", sizeof(void*), false, true}; + case 3: return {"non_det_ushort", sizeof(unsigned short), false, true}; + case 4: return {"non_det_long", sizeof(int64_t), false, false}; + case 5: return {"non_det_unsigned", sizeof(unsigned), false, true}; + case 6: return {"non_det_ulong", sizeof(uint64_t), false, true}; + case 7: return {"non_det_bool", sizeof(int), false, false}; + case 8: return {"non_det_uchar", sizeof(unsigned char), false, true}; + case 10: return {"non_det_size_t", sizeof(size_t), false, true}; + case 13: return {"non_det_uint", sizeof(unsigned), false, true}; + case 14: return {"non_det_short", sizeof(short), false, false}; + case 15: return {"non_det_double", sizeof(double), true, false}; + default: return {nullptr, 0, false, false}; + } +} + +/** Encodes a logged decimal value into the object's little-endian bytes. */ +void encodeValue(const std::string& text, const NonDetTypeInfo& info, + std::vector* out) { + if (info.isFloating) { + double value = 0.0; + try { + value = std::stod(text); + } catch (const std::exception&) { + value = 0.0; + } + std::memcpy(out->data(), &value, + std::min(out->size(), sizeof(double))); + return; + } + uint64_t raw = 0; + try { + // Parsed as signed first so a logged "-1" round-trips; the bit pattern is + // the same either way, and the width below is what actually matters. + raw = static_cast(std::stoll(text)); + } catch (const std::exception&) { + try { + raw = std::stoull(text); + } catch (const std::exception&) { + raw = 0; + } + } + for (size_t i = 0; i < out->size() && i < 8; ++i) { + (*out)[i] = static_cast((raw >> (8 * i)) & 0xff); + } +} + bool skipBlock(std::ifstream& in) { uint32_t size = 0; if (!readBigEndian32(in, &size)) return false; @@ -247,4 +320,73 @@ std::vector> readKtestVectors( return vectors; } +std::vector ktestToFuzzerBytes( + const std::vector& objects) { + std::vector bytes; + for (const KtestObject& object : objects) { + bytes.insert(bytes.end(), object.bytes.begin(), object.bytes.end()); + } + return bytes; +} + +bool writeKtestFile(const std::string& path, + const std::vector& objects) { + std::ofstream out(path, std::ios::binary); + if (!out.is_open()) return false; + + out.write(kMagicKtest, kMagicSize); + putBigEndian32(out, 3); // version + putBigEndian32(out, 1); // one argument, mirroring what KLEE records + const std::string argument = "map2check"; + putBigEndian32(out, static_cast(argument.size())); + out.write(argument.data(), static_cast(argument.size())); + putBigEndian32(out, 0); // symArgvs + putBigEndian32(out, 0); // symArgvLen + putBigEndian32(out, static_cast(objects.size())); + for (const KtestObject& object : objects) { + putBigEndian32(out, static_cast(object.name.size())); + out.write(object.name.data(), + static_cast(object.name.size())); + putBigEndian32(out, static_cast(object.bytes.size())); + if (!object.bytes.empty()) { + out.write(reinterpret_cast(object.bytes.data()), + static_cast(object.bytes.size())); + } + } + return out.good(); +} + +std::vector readNonDetLogAsObjects(const std::string& csvPath) { + std::vector objects; + std::ifstream in(csvPath); + if (!in.is_open()) return objects; + + std::string line; + while (std::getline(in, line)) { + if (line.empty()) continue; + std::vector fields; + std::string field; + std::istringstream row(line); + while (std::getline(row, field, ';')) fields.push_back(field); + if (fields.size() < 7) continue; + + const std::string& value = fields[5]; + int type = 0; + try { + type = std::stoi(fields[6]); + } catch (const std::exception&) { + continue; + } + const NonDetTypeInfo info = nonDetTypeInfo(type); + if (info.name == nullptr) continue; + + KtestObject object; + object.name = info.name; + object.bytes.assign(info.width, 0); + encodeValue(value, info, &object.bytes); + objects.push_back(std::move(object)); + } + return objects; +} + } // namespace Map2Check diff --git a/modules/frontend/test_suite/ktest_reader.hpp b/modules/frontend/test_suite/ktest_reader.hpp index e943f26f1..214f204e8 100644 --- a/modules/frontend/test_suite/ktest_reader.hpp +++ b/modules/frontend/test_suite/ktest_reader.hpp @@ -82,6 +82,31 @@ std::vector readViolatingKtest(const std::string& kleeOutDir); std::vector> readKtestVectors( const std::string& kleeOutDir, size_t limit); +/** Serialises objects into the byte stream a fuzzer would consume. + * + * Sound only because both engines now agree on widths: NonDetGeneratorKlee.c + * passes sizeof(type) to klee_make_symbolic, and NonDetGeneratorLibFuzzy.c + * takes sizeof(type) bytes per read. Concatenating a .ktest's objects in order + * therefore produces exactly the buffer that would drive the fuzzer down the + * same path. Before the width fix this was impossible -- the fuzzer read one + * byte per value whatever the type, so most vectors had nowhere to go. */ +std::vector ktestToFuzzerBytes(const std::vector& objects); + +/** Writes objects as a .ktest, for handing KLEE a starting point. + * + * The counterpart of readKtestFile, used to turn what one engine learned into + * something the other can start from. False if the file could not be written. */ +bool writeKtestFile(const std::string& path, + const std::vector& objects); + +/** Reads the runtime's nondet log as typed objects. + * + * The log is the only place a fuzzer run records what it actually consumed -- + * value AND type, in consumption order -- so it is the bridge from a fuzzer + * result back into KLEE's format. Field 6 carries the NONDET_TYPE enumerator; + * see modules/backend/library/header/Map2CheckTypes.h. */ +std::vector readNonDetLogAsObjects(const std::string& csvPath); + } // namespace Map2Check #endif // MODULES_FRONTEND_TEST_SUITE_KTEST_READER_HPP_ diff --git a/tests/integration/test_testcomp_regressions.sh b/tests/integration/test_testcomp_regressions.sh index 3cd522dbc..7ceff3a75 100755 --- a/tests/integration/test_testcomp_regressions.sh +++ b/tests/integration/test_testcomp_regressions.sh @@ -370,6 +370,69 @@ else "did not reach short < 0 -- half of the type is unreachable" fi +# --- 11. the engines must be able to hand each other input vectors ---------- +# Two engines that cover almost disjoint sets -- measured over 372 tasks, 65 +# reachable only by the fuzzer and 47 only by KLEE -- and until now they shared +# nothing. LibFuzzer did not even keep its own corpus: with no corpus directory +# it holds everything in memory and drops it when the process ends. +# +# Asserted here is the MECHANISM, not a coverage gain. Whether cooperating +# finds more is a question for the corpus, not for a unit-sized program; what +# a test can pin is that the channel exists, is off by default, and carries +# vectors in the direction it claims to. +mkdir -p "$WORK/seed" +cat > "$WORK/seed/seed.c" <<'EOF' +extern void __assert_fail(const char *, const char *, unsigned int, + const char *) __attribute__((__noreturn__)); +void reach_error(void) { __assert_fail("0", "seed.c", 3, "reach_error"); } +extern int __VERIFIER_nondet_int(void); +int main(void) { + int a = __VERIFIER_nondet_int(); + int b = __VERIFIER_nondet_int(); + if (a > 100 && a < 200) { + if (b == a + 7) { reach_error(); } + } + return 0; +} +EOF + +# Off by default: the hybrid's measured behaviour must not change until the +# exchange has earned its place beside it. +( cd "$WORK/seed" && MAP2CHECK_PATH="$MAP2CHECK_DIR" timeout -k 10 250 "$MAP2CHECK" \ + --target-function --target-function-name reach_error \ + --debug --timeout 60 seed.c ) > "$WORK/seed/off.log" 2>&1 +scratch_off=$(find "$WORK/seed" -maxdepth 1 -name '*.map2check' -print -quit) +n_off=$(ls "$scratch_off/seeds" 2>/dev/null | wc -l) +if [ "$n_off" -eq 0 ]; then + ok "no seed corpus without --seed-exchange" +else + fail "default behaviour" "$n_off seeds written without asking" +fi +rm -rf "$WORK/seed"/*.map2check + +( cd "$WORK/seed" && MAP2CHECK_PATH="$MAP2CHECK_DIR" timeout -k 10 300 "$MAP2CHECK" \ + --target-function --target-function-name reach_error --seed-exchange \ + --debug --timeout 60 seed.c ) > "$WORK/seed/on.log" 2>&1 +scratch_on=$(find "$WORK/seed" -maxdepth 1 -name '*.map2check' -print -quit) +n_on=$(ls "$scratch_on/seeds" 2>/dev/null | wc -l) + +# LibFuzzer renames what it keeps to its own content hash, so any file at all +# means the corpus survived the process -- which it never used to. +if [ "$n_on" -gt 0 ]; then + ok "the fuzzer corpus persists with --seed-exchange ($n_on files)" +else + fail "seed corpus" "nothing kept -- the corpus is still in-memory only" +fi + +# KLEE -> fuzzer: its per-path vectors become seed files. Sound only because +# both engines now consume sizeof(type) per read, so concatenating a .ktest's +# objects is exactly the buffer that drives the fuzzer down the same path. +if grep -q "Seeded the fuzzer corpus with" "$WORK/seed/on.log"; then + ok "KLEE's path vectors are exported into the seed corpus" +else + fail "KLEE -> fuzzer" "no vectors exported" +fi + echo " ---" echo " Results: $PASSED passed, $FAILED failed" [ "$FAILED" -eq 0 ] || exit 1 From 060f45239e2e67daf966a1f36da9f5dc1c02e74c Mon Sep 17 00:00:00 2001 From: Guilherme Bernardo Date: Sun, 23 Aug 2026 18:44:40 -0400 Subject: [PATCH 15/17] feat(slicing): slice the program with respect to the target before analysing it Phase 2 of the migration plan, and the direct answer to finding K2: a nondeterministic read cost 1 -> 21 -> 114 -> 861 partial paths for 2 -> 4 reads, and most of that forking happens in code that cannot influence the target. Slicing removes it before KLEE ever sees it. Measured on the probe the image build runs: 126 lines of IR down to 66, "Sliced away 30 from 39 nodes", and the irrelevant 50-iteration loop -- one multiply in the original -- gone entirely from the slice. Getting DG to build against LLVM 16 took five obstacles, and two of them are worth writing down because the next person will hit them: master does not compile. It calls llvm::Function::getBasicBlockList(), which LLVM 16 made private. The maintained version is the llvm-latest branch, whose newest commit is literally "Fix build for LLVM < 18" -- the same shape as crab-llvm having been renamed to Clam, where the supported path existed under a name nothing referenced. DG must be built IN-SOURCE. sbt-slicer's CMakeLists documents DG_PATH as "a path to an in-source build of dg" and derives both include and library paths from it; an out-of-tree build satisfies neither, and the failure surfaces as a missing dg/tools/llvm-slicer-opts.h. Plus: both projects pin C++14 with a plain set() that overrides the command line, while LLVM 16's headers need C++17 (llvm-latest already ships 17); and sbt-slicer links dg's shared libraries without an rpath, so without LD_LIBRARY_PATH the binary builds and then dies on every invocation. Behind --slice, off by default, and for reachability only. This is not a neutral speed-up: a slice taken with respect to one error site can legitimately remove another, so a sliced run answers a NARROWER question. Right for a competition task with one property, wrong for a baseline scoring precision per CWE. Cover-Branches gets no criterion at all -- every branch is the goal -- and asking there is refused rather than ignored. The tests assert the unhappy paths, because that is where this class of feature fails: --add-invariants spent years accepted and silently ignored (issue #54), and a slicer that quietly does nothing would be the same defect renamed. An absent slicer must say so; a mode with no criterion must refuse. Co-Authored-By: Claude Opus 5 (1M context) --- Dockerfile.dev | 91 +++++++++++++++++++ modules/frontend/caller.cpp | 48 ++++++++++ modules/frontend/caller.hpp | 15 +++ modules/frontend/map2check.cpp | 26 ++++++ modules/frontend/utils/tools.hpp | 13 +++ .../integration/test_testcomp_regressions.sh | 37 ++++++++ tests/testcomp/run_testcomp_evaluation.sh | 10 +- 7 files changed, 236 insertions(+), 4 deletions(-) diff --git a/Dockerfile.dev b/Dockerfile.dev index e02f3b5a7..e43d5f14e 100644 --- a/Dockerfile.dev +++ b/Dockerfile.dev @@ -229,6 +229,97 @@ RUN printf 'int main(){int i=0,s=0;while(i<10){s+=i;i++;}return s;}\n' > /tmp/cl test "$(llvm-dis-16 -o - /tmp/clamcheck.bc | grep -c 'verifier\.assume')" -gt 0 && \ echo "Clam injects invariants: OK" && rm -f /tmp/clamcheck.c /tmp/clamcheck.bc +# ============================================================ +# 7c. DG + sbt-slicer — program slicing with respect to an error site +# ============================================================ +# Phase 2 of docs/map2check_migration_plan.md, and the same pipeline Symbiotic +# uses: build a System Dependence Graph, keep only what the error site depends +# on, hand the smaller program to KLEE. +# +# What it is for, in numbers from this repository: finding K2 measured the cost +# of a nondeterministic read at 1 -> 21 -> 114 -> 861 partial paths for 2 -> 4 +# reads. Most of that forking happens in code that cannot influence the target, +# and slicing removes it before KLEE ever sees it. +# +# Two limits worth knowing before relying on it. Slicing needs a criterion, so +# it serves Cover-Error and does nothing for Cover-Branches, where every branch +# matters and there is no error site to slice towards. And a slice taken with +# respect to one error can remove another -- fine for a tool answering one +# property, a hazard for a baseline that scores precision per CWE. +# +# Built with clang-16 for the reason section 7b spells out: the ENV CC below is +# declared later in this file and would not apply here, and these are LLVM +# projects that expect the LLVM toolchain. +# +# DG is taken from llvm-latest, NOT master, and the difference is the same one +# crab-llvm taught: the maintained version was somewhere else. master calls +# llvm::Function::getBasicBlockList(), which LLVM 16 made private, so it simply +# does not compile here. llvm-latest's newest commit is literally "Fix build +# for LLVM < 18". +# +# Pinned to SHAs. Neither project has had a versioned release since 2020, so +# without a pin the image changes when upstream does -- what crab@dev +# demonstrated by moving the day before it was needed. +ARG DG_SHA=b34da890694f5210e475f34018239fc9d126d1ee # llvm-latest @ 2023-11-29 +ARG SBT_SLICER_SHA=e350116b39be215277803fe2f5e5ac4a161c16b8 # master @ 2025-03-27 + +RUN set -eux; \ + pin() { \ + git init -q "$2" && \ + git -C "$2" remote add origin "$1" && \ + git -C "$2" fetch -q --depth 1 origin "$3" && \ + git -C "$2" checkout -q FETCH_HEAD; \ + }; \ + # Both projects pin C++14 with a plain set(), which overrides anything + # passed on the command line, and LLVM 16's headers use std::is_unsigned_v + # -- C++17. Patching the two lines is the smallest change that builds; the + # alternative is carrying a fork. + bump() { \ + sed -i 's/set(CMAKE_CXX_STANDARD 14)/set(CMAKE_CXX_STANDARD 17)/; \ + s/-std=c++14/-std=c++17/' "$1/CMakeLists.txt"; \ + }; \ + pin https://github.com/mchalupa/dg.git /tmp/dg "$DG_SHA"; \ + bump /tmp/dg; \ + # IN-SOURCE, deliberately. sbt-slicer's CMakeLists documents DG_PATH as + # "a path to an in-source build of dg" and derives both the include path + # and the library path from it; an out-of-tree build satisfies neither + # (the headers stay in the source tree, the libraries land in the build + # tree) and sbt-slicer fails to find dg/tools/llvm-slicer-opts.h. + cd /tmp/dg && \ + cmake . -DCMAKE_BUILD_TYPE=Release \ + -DCMAKE_C_COMPILER=/usr/bin/clang-16 \ + -DCMAKE_CXX_COMPILER=/usr/bin/clang++-16 \ + -DLLVM_DIR=/usr/lib/llvm-16/lib/cmake/llvm \ + -DCMAKE_INSTALL_PREFIX=/opt/dg && \ + make -j"$(nproc)" && make install && \ + pin https://github.com/staticafi/sbt-slicer.git /tmp/sbt-slicer \ + "$SBT_SLICER_SHA"; \ + bump /tmp/sbt-slicer; \ + mkdir /tmp/sbt-slicer/build && cd /tmp/sbt-slicer/build && \ + cmake .. -DCMAKE_BUILD_TYPE=Release \ + -DCMAKE_C_COMPILER=/usr/bin/clang-16 \ + -DCMAKE_CXX_COMPILER=/usr/bin/clang++-16 \ + -DLLVM_DIR=/usr/lib/llvm-16/lib/cmake/llvm \ + -DDG_PATH=/tmp/dg \ + -DCMAKE_INSTALL_PREFIX=/opt/sbt-slicer && \ + make -j"$(nproc)" && make install && \ + rm -rf /tmp/dg /tmp/sbt-slicer + +ENV SBT_SLICER=/opt/sbt-slicer/bin/sbt-slicer +# dg installs shared libraries, and sbt-slicer links against them without an +# rpath: without this the binary exists, builds, and dies on every invocation +# with "libdganalysis.so: cannot open shared object file". +ENV LD_LIBRARY_PATH="/opt/dg/lib:${LD_LIBRARY_PATH}" + +# Fail the image build if the slicer cannot actually slice. A binary that +# exists and refuses every input is the failure mode section 7b was written +# about: --add-invariants stayed dead for years behind exactly that. +RUN printf 'extern int __VERIFIER_nondet_int(void);\nvoid reach_error(void);\nint main(){int a=__VERIFIER_nondet_int();int junk=__VERIFIER_nondet_int();int s=0;for(int i=0;i<50;i++){s+=junk*i;}if(a==7){reach_error();}return s;}\n' > /tmp/slicecheck.c && \ + clang-16 -c -emit-llvm -g -O0 -o /tmp/slicecheck.bc /tmp/slicecheck.c && \ + /opt/sbt-slicer/bin/sbt-slicer -c reach_error -o /tmp/sliced.bc /tmp/slicecheck.bc && \ + test -s /tmp/sliced.bc && \ + echo "sbt-slicer slices: OK" && rm -f /tmp/slicecheck.c /tmp/slicecheck.bc /tmp/sliced.bc + # ============================================================ # 8. WABT (WebAssembly Binary Toolkit) — wasm2c, wasm2wat, wasm-ld # ============================================================ diff --git a/modules/frontend/caller.cpp b/modules/frontend/caller.cpp index e2e8334ed..83838cff7 100644 --- a/modules/frontend/caller.cpp +++ b/modules/frontend/caller.cpp @@ -149,6 +149,54 @@ std::string Caller::exportFuzzerVectorAsKtest() { return path; } +bool Caller::sliceWithRespectToTarget(const std::string &targetFunction) { + const std::string slicer = Map2Check::slicerBinary(); + if (!std::filesystem::exists(slicer)) { + // Announced, not silently skipped. A slicer that is asked for and absent + // must not leave the run quietly analysing the whole program: that is how + // --add-invariants stayed dead through a full baseline (issue #54). + Map2Check::Log::Warning( + "slicing was requested but sbt-slicer is not installed at " + slicer + + " -- analysing the unsliced program"); + return false; + } + + const std::string input = programHash + "-output.bc"; + const std::string output = programHash + "-sliced.bc"; + if (!std::filesystem::exists(input)) return false; + + std::ostringstream command; + // -c is the slicing criterion: keep what the target call depends on. The + // criterion is the whole reason this only serves Cover-Error -- there is no + // criterion to give it when every branch is the goal. + command << slicer << " -c " << targetFunction << " -o " << output << " " + << input << " > slicer.output 2>&1"; + Map2Check::Log::Debug(command.str()); + const int result = system(command.str().c_str()); + + std::error_code error; + const bool produced = std::filesystem::exists(output, error) && + std::filesystem::file_size(output, error) > 0; + if (result != 0 || !produced) { + Map2Check::Log::Warning( + "sbt-slicer produced no usable output -- analysing the unsliced " + "program"); + return false; + } + + // Reported, because a slice is not a neutral speed-up: it narrows the + // question being answered, and the size difference is the only visible sign + // of how much was dropped. + const auto before = std::filesystem::file_size(input, error); + const auto after = std::filesystem::file_size(output, error); + Map2Check::Log::Info("Sliced with respect to " + targetFunction + ": " + + std::to_string(before) + " -> " + + std::to_string(after) + " bytes of bitcode"); + + std::filesystem::rename(output, input, error); + return !error; +} + void Caller::cleanGarbage() { std::filesystem::current_path(currentPath); std::ostringstream removeCommand; diff --git a/modules/frontend/caller.hpp b/modules/frontend/caller.hpp index dcc31823c..79a0bf55c 100644 --- a/modules/frontend/caller.hpp +++ b/modules/frontend/caller.hpp @@ -105,6 +105,21 @@ class Caller { /** Remove generated files for verification */ void cleanGarbage(); + /** Slice the program with respect to the target before analysing it. + * + * Off by default. Slicing changes WHAT IS ANALYSED, not merely how fast: a + * slice taken with respect to one error site can legitimately remove + * another, so a run with this on answers a narrower question than a run + * without it. That is right for a competition task with one property and + * wrong for a baseline scoring precision per CWE, which is why it is a + * decision the caller makes rather than a default. */ + bool sliceProgram = false; + + /** Runs sbt-slicer over the instrumented bitcode. Returns false if the + * slicer is unavailable or produced nothing usable, leaving the original + * bitcode in place. */ + bool sliceWithRespectToTarget(const std::string& targetFunction); + /** Turns on the exchange of input vectors between the two engines. * * Off by default so the hybrid keeps behaving exactly as it was measured diff --git a/modules/frontend/map2check.cpp b/modules/frontend/map2check.cpp index d6b6b8ec3..be52858cb 100644 --- a/modules/frontend/map2check.cpp +++ b/modules/frontend/map2check.cpp @@ -386,6 +386,7 @@ struct map2check_args { bool generateTestSuite = false; bool coverBranches = false; bool seedExchange = false; + bool sliceProgram = false; std::string testSuiteDir = "test-suite"; std::string propertyFile; std::string architecture = "64bit"; @@ -459,6 +460,7 @@ int map2check_execution(map2check_args args) { generator); caller->c_program_fullpath = args.inputFile; caller->seedExchange = args.seedExchange; + caller->sliceProgram = args.sliceProgram; caller->setTimeout(args.timeout); caller->entryFunction = args.entryFunction; caller->wasmMode = args.wasmMode; @@ -481,6 +483,24 @@ int map2check_execution(map2check_args args) { // (2) Instrument functions for current mode caller->callPass(args.function); + + // After instrumentation, before linking: the slicer needs the target call to + // still be visible as a criterion, and there is no point slicing code that + // the runtime will add afterwards. + // + // Reachability only. Slicing needs a criterion to slice towards, and + // Cover-Branches has none -- every branch is the goal. Asking for it in any + // other mode is refused rather than quietly ignored. + if (args.sliceProgram) { + if (args.mode == Map2Check::Map2CheckMode::REACHABILITY_MODE) { + caller->sliceWithRespectToTarget(args.function); + } else { + Map2Check::Log::Warning( + "--slice applies to reachability only: there is no criterion to " + "slice towards when the goal is coverage or a memory property. " + "Analysing the whole program."); + } + } caller->linkLLVM(); // (3) Apply nondeterministic mode and execute analysis @@ -660,6 +680,9 @@ z3 (Z3 is default), btor (Boolector), and yices2 (Yices))") "\temits a Test-Comp test suite reproducing the violation found") ("test-suite-dir", po::value()->default_value("test-suite"), "\tdirectory to write the test suite into") + ("slice", + "\tslice the program with respect to the target before analysing it " + "(reachability only; needs sbt-slicer)") ("seed-exchange", "\tlet the two engines hand each other input vectors through a shared " "seed corpus (hybrid runs; off by default)") @@ -772,6 +795,9 @@ z3 (Z3 is default), btor (Boolector), and yices2 (Yices))") if (vm.count("test-suite-dir")) { args.testSuiteDir = vm["test-suite-dir"].as(); } + if (vm.count("slice")) { + args.sliceProgram = true; + } if (vm.count("seed-exchange")) { args.seedExchange = true; } diff --git a/modules/frontend/utils/tools.hpp b/modules/frontend/utils/tools.hpp index f2dd8c28c..16975283a 100644 --- a/modules/frontend/utils/tools.hpp +++ b/modules/frontend/utils/tools.hpp @@ -59,6 +59,19 @@ constexpr char const* clangIncludeFolder = "${MAP2CHECK_PATH}/include/"; constexpr char const* listLogCSV = "list_log.csv"; /** Path to klee binary */ constexpr char const* kleeBinary = "${MAP2CHECK_PATH}/bin/klee"; +/** Default root of the sbt-slicer install (Dockerfile.dev section 7c). */ +constexpr char const* slicerDefaultRoot = "/opt/sbt-slicer"; +/** Path to the sbt-slicer binary, overridable with SBT_SLICER. + * + * Resolved like the invariant generator: an environment override first, a + * documented default second, and the caller checks the file exists before + * relying on it -- a missing slicer must refuse the run, not silently analyse + * the unsliced program (issue #54's failure mode). */ +inline std::string slicerBinary() { + const char* override_path = getenv("SBT_SLICER"); + if (override_path != nullptr) return std::string(override_path); + return std::string(slicerDefaultRoot) + "/bin/sbt-slicer"; +} /** Seconds granted between SIGTERM and SIGKILL when a backend overruns its * slice (`timeout -k`). Both KLEE and LibFuzzer catch SIGTERM to shut down * gracefully, and both can miss it while wedged -- KLEE inside the solver, diff --git a/tests/integration/test_testcomp_regressions.sh b/tests/integration/test_testcomp_regressions.sh index 7ceff3a75..37204ba6b 100755 --- a/tests/integration/test_testcomp_regressions.sh +++ b/tests/integration/test_testcomp_regressions.sh @@ -433,6 +433,43 @@ else fail "KLEE -> fuzzer" "no vectors exported" fi +# --- 12. slicing must degrade loudly, never silently ------------------------- +# Finding K2 measured a nondeterministic read at 1 -> 21 -> 114 -> 861 partial +# paths for 2 -> 4 reads, and most of that forking happens in code that cannot +# influence the target. Slicing removes it before KLEE ever sees it. +# +# What is asserted here is the behaviour when things are NOT ideal, because +# that is where this class of feature fails: --add-invariants spent years +# accepted and ignored (issue #54), and a slicer that quietly does nothing +# would be the same defect wearing a different name. +mkdir -p "$WORK/slice" +cp "$WORK/one/reach.c" "$WORK/slice/" + +( cd "$WORK/slice" && MAP2CHECK_PATH="$MAP2CHECK_DIR" timeout -k 10 200 "$MAP2CHECK" \ + --target-function --target-function-name reach_error --slice \ + --nondet-generator symex --timeout 45 reach.c ) > "$WORK/slice/run.log" 2>&1 + +# Either the slicer is installed and slices, or it is absent and the run says +# so. What must never happen is silence. +if grep -q "Sliced with respect to" "$WORK/slice/run.log"; then + ok "the program was sliced with respect to the target" +elif grep -q "sbt-slicer is not installed" "$WORK/slice/run.log"; then + ok "an absent slicer is reported, and the run continues unsliced" +else + fail "slicing" "--slice produced neither a slice nor an explanation" +fi + +# There is no criterion to slice towards when the goal is a memory property or +# branch coverage, so asking must be refused rather than quietly ignored. +( cd "$WORK/slice" && MAP2CHECK_PATH="$MAP2CHECK_DIR" timeout -k 10 200 "$MAP2CHECK" \ + --memtrack --slice --nondet-generator symex --timeout 45 reach.c ) \ + > "$WORK/slice/mode.log" 2>&1 +if grep -q "applies to reachability only" "$WORK/slice/mode.log"; then + ok "--slice is refused where there is no criterion to slice towards" +else + fail "slice mode guard" "--slice was accepted in a mode that has no criterion" +fi + echo " ---" echo " Results: $PASSED passed, $FAILED failed" [ "$FAILED" -eq 0 ] || exit 1 diff --git a/tests/testcomp/run_testcomp_evaluation.sh b/tests/testcomp/run_testcomp_evaluation.sh index 1e7de03af..cb0bce162 100755 --- a/tests/testcomp/run_testcomp_evaluation.sh +++ b/tests/testcomp/run_testcomp_evaluation.sh @@ -19,7 +19,7 @@ # PROPERTY cover-error | cover-branches (required) # RESULTS_DIR where the CSV and raw logs go (required) # SHARD/SHARDS process every SHARDS-th task starting at SHARD (default 0/1) -# GENERATOR symex | fuzzer | hybrid (default symex) +# GENERATOR symex | fuzzer | hybrid | hybrid-seed (default hybrid) # BUDGET seconds given to map2check per task (default 60) # TESTCOV_S seconds given to TestCov per task (default 90) # DEADLINE_S stop starting new tasks after this many seconds (default 18000) @@ -45,9 +45,10 @@ DEADLINE_S="${DEADLINE_S:-18000}" # Which engine produces the inputs. Parameterised so the three arms can be # compared on the SAME task list: # -# symex KLEE only -- every Test-Comp measurement so far used this -# fuzzer LibFuzzer only -# hybrid the tool's actual default: LibFuzzer at 0.2x the budget, then KLEE +# symex KLEE only -- every Test-Comp measurement before 2026-08-23 +# fuzzer LibFuzzer only +# hybrid the tool's actual default: LibFuzzer at 0.2x, then KLEE +# hybrid-seed the same, with the two engines exchanging input vectors # # The default is hybrid, which is also the tool's own default when no # --nondet-generator is given. Every Test-Comp measurement before 2026-08-23 @@ -61,6 +62,7 @@ GENERATOR="${GENERATOR:-hybrid}" case "$GENERATOR" in symex|fuzzer) GENERATOR_FLAG="--nondet-generator $GENERATOR" ;; hybrid) GENERATOR_FLAG="" ;; # absent flag IS the hybrid path + hybrid-seed) GENERATOR_FLAG="--seed-exchange" ;; *) echo "unknown GENERATOR: $GENERATOR" >&2; exit 2 ;; esac From 8a6fa50aad801ecd1072664cc105daa152144842 Mon Sep 17 00:00:00 2001 From: Guilherme Bernardo Date: Sun, 23 Aug 2026 19:59:26 -0400 Subject: [PATCH 16/17] test(testcomp): chain the v11 measurement behind the v10 campaign and the image v11 answers "what do slicing and seed exchange buy", which is a comparison against v10 -- so the two cannot share a machine. The tool's LibFuzzer phase runs with -jobs=8, and ten containers already put this host at twice its core count (load 31 on 16 cores); a second campaign alongside would leave both numbers unattributable, which is the mistake that cost a day earlier this week when contention was read as a regression. The launcher waits for two things instead. The v10 containers to finish, and the republished image to actually carry sbt-slicer -- the second is not a formality, because --slice degrades gracefully when the slicer is absent, so launching too early would produce a full campaign that silently measures nothing. Two arms, so the features stay separable: slice against v10 isolates slicing, slice+seed against slice isolates the exchange. cover-error only, because slicing needs a criterion and Cover-Branches has none. EXTRA_FLAGS is how an opt-in capability gets measured against a run without it; it is kept apart from GENERATOR because --slice and --seed-exchange sit on top of whichever engine is running rather than replacing it. Co-Authored-By: Claude Opus 5 (1M context) --- tests/testcomp/launch-v11-when-ready.sh | 85 +++++++++++++++++++++++ tests/testcomp/run_testcomp_evaluation.sh | 9 ++- 2 files changed, 93 insertions(+), 1 deletion(-) create mode 100755 tests/testcomp/launch-v11-when-ready.sh diff --git a/tests/testcomp/launch-v11-when-ready.sh b/tests/testcomp/launch-v11-when-ready.sh new file mode 100755 index 000000000..58f49b069 --- /dev/null +++ b/tests/testcomp/launch-v11-when-ready.sh @@ -0,0 +1,85 @@ +#!/bin/bash +# launch-v11-when-ready.sh -- chains the v11 measurement behind two conditions. +# +# Not run in parallel with v10, deliberately. v11 exists to answer "what do +# slicing and seed exchange buy", which is a COMPARISON against v10 -- and two +# campaigns sharing a machine experience different contention, so a difference +# between them would not be attributable to the features. The tool's LibFuzzer +# phase runs with -jobs=8, so ten containers already put this machine at twice +# its core count; adding ten more would make both numbers meaningless. +# +# Waits for: +# 1. every v10 container to finish +# 2. the republished image to actually carry the slicer +# +# The second is not a formality. --slice degrades gracefully when sbt-slicer is +# absent -- it warns and analyses the whole program -- so launching against an +# image without it would produce a full campaign that silently measures +# nothing. Checking is the difference between an answer and a wasted night. + +set -u + +REPO="$(cd "$(dirname "$0")/../.." && pwd)" +IMAGE_SRC="${IMAGE_SRC:-ghcr.io/hbgit/map2check-dev:latest}" +IMAGE="${IMAGE:-map2check-testcov:latest}" +LOG="$REPO/tests/baseline-logs/v11-launcher.log" +POLL="${POLL:-300}" + +say() { echo "[$(date +%H:%M)] $*" | tee -a "$LOG"; } + +say "waiting for the v10 campaign to finish" +while [ "$(docker ps -q --filter 'name=m2c-v10-' | wc -l)" -gt 0 ]; do + sleep "$POLL" +done +say "v10 finished" + +say "waiting for an image that carries the slicer" +while true; do + docker pull -q "$IMAGE_SRC" >/dev/null 2>&1 || true + if docker run --rm "$IMAGE_SRC" test -x /opt/sbt-slicer/bin/sbt-slicer >/dev/null 2>&1; then + say "the published image has sbt-slicer" + break + fi + say " not yet -- the PR has to merge and Publish Docker Image has to run" + sleep "$POLL" +done + +# The testcov derivative has to be rebuilt on top of the new base, or the runs +# would use the old one and the wait above would have been pointless. +say "rebuilding $IMAGE on top of the new base" +docker build -q -t "$IMAGE" -f - "$REPO" >>"$LOG" 2>&1 < what slicing buys +# slice+seed vs slice -> what the exchange adds on top +launch() { + local arm="$1" flags="$2" prop="$3" shard="$4" shards="$5" + docker rm -f "m2c-v11-${arm}-${prop}-s${shard}" >/dev/null 2>&1 + docker run -d --name "m2c-v11-${arm}-${prop}-s${shard}" $COMMON --cpus=1.5 \ + -e MANIFEST="/workspace/tests/testcomp/corpus/${prop}.tsv" \ + -e PROPERTY="$prop" -e GENERATOR=hybrid -e EXTRA_FLAGS="$flags" \ + -e RESULTS_DIR="/workspace/tests/testcomp/results_v11_${arm}_${prop}_s${shard}" \ + -e SHARD="$shard" -e SHARDS="$shards" \ + -e BUDGET=60 -e TESTCOV_S=90 -e DEADLINE_S=28800 \ + "$IMAGE" bash tests/testcomp/run_testcomp_evaluation.sh >/dev/null +} + +# cover-error only. Slicing needs a criterion to slice towards, and +# Cover-Branches has none -- every branch is the goal. +for i in 0 1 2 3; do launch slice "--slice" cover-error "$i" 4; done +for i in 0 1 2 3; do launch slice-seed "--slice --seed-exchange" cover-error "$i" 4; done + +sleep 5 +say "v11 launched: $(docker ps -q --filter 'name=m2c-v11-' | wc -l) containers" diff --git a/tests/testcomp/run_testcomp_evaluation.sh b/tests/testcomp/run_testcomp_evaluation.sh index cb0bce162..c47b969bf 100755 --- a/tests/testcomp/run_testcomp_evaluation.sh +++ b/tests/testcomp/run_testcomp_evaluation.sh @@ -20,6 +20,7 @@ # RESULTS_DIR where the CSV and raw logs go (required) # SHARD/SHARDS process every SHARDS-th task starting at SHARD (default 0/1) # GENERATOR symex | fuzzer | hybrid | hybrid-seed (default hybrid) +# EXTRA_FLAGS passed verbatim to map2check (e.g. "--slice") # BUDGET seconds given to map2check per task (default 60) # TESTCOV_S seconds given to TestCov per task (default 90) # DEADLINE_S stop starting new tasks after this many seconds (default 18000) @@ -59,6 +60,12 @@ DEADLINE_S="${DEADLINE_S:-18000}" # alone. Directories written before the switch carry `symex` in this comment's # place; they are not comparable to hybrid runs and should not be pooled. GENERATOR="${GENERATOR:-hybrid}" + +# Extra flags handed straight to map2check, for measuring an opt-in capability +# against a run without it. Kept separate from GENERATOR because these are not +# engine choices: --slice and --seed-exchange both sit on top of whichever +# engine is running. +EXTRA_FLAGS="${EXTRA_FLAGS:-}" case "$GENERATOR" in symex|fuzzer) GENERATOR_FLAG="--nondet-generator $GENERATOR" ;; hybrid) GENERATOR_FLAG="" ;; # absent flag IS the hybrid path @@ -133,7 +140,7 @@ while IFS=$'\t' read -r category program data_model expected <&3; do ( cd "$work" || exit 1 MAP2CHECK_PATH="$MAP2CHECK_DIR" timeout -k 10 $((BUDGET + 30)) "$MAP2CHECK" \ - $GENERATOR_FLAG --generate-test-suite $GOAL_FLAGS \ + $GENERATOR_FLAG $EXTRA_FLAGS --generate-test-suite $GOAL_FLAGS \ --property-file prop.prp --architecture "$arch" \ --timeout "$BUDGET" "$name" ) > "$work/map2check.log" 2>&1 Date: Sun, 23 Aug 2026 20:04:29 -0400 Subject: [PATCH 17/17] test(testcomp): make v11 a 2x2 factorial with its own control Two arms measured the seed exchange only in the PRESENCE of slicing. If it helps alone and hurts combined -- or the reverse -- that design cannot see it. Four cells separate each feature and their interaction. The control cell runs inside v11 rather than reusing v10, even though v10 measures the same configuration on the same corpus. v10 ran with ten containers on a host already at twice its core count; v11 runs with twelve. Comparing across that difference carries contention into the result -- the exact confound the chaining was introduced to avoid, and the one that cost a day earlier this week when contention was read as a regression. One extra arm removes the question instead of arguing about it afterwards. Co-Authored-By: Claude Opus 5 (1M context) --- tests/testcomp/launch-v11-when-ready.sh | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-) diff --git a/tests/testcomp/launch-v11-when-ready.sh b/tests/testcomp/launch-v11-when-ready.sh index 58f49b069..113ce40ad 100755 --- a/tests/testcomp/launch-v11-when-ready.sh +++ b/tests/testcomp/launch-v11-when-ready.sh @@ -61,9 +61,18 @@ say "image ready" COMMON="--restart=on-failure:100 --memory=4g -u 0 -v $REPO:/workspace -w /workspace" COMMON="$COMMON -e MAP2CHECK_PATH=/workspace/install_v9" -# Two arms, so each feature is separable rather than entangled: -# slice vs v10 -> what slicing buys -# slice+seed vs slice -> what the exchange adds on top +# A 2x2 factorial, control included, and both parts of that matter. +# +# Two arms would have measured the exchange only in the PRESENCE of slicing. +# If it helps alone and hurts combined -- or the reverse -- that design cannot +# see it. Four cells separate each feature AND their interaction. +# +# The control cell runs here rather than reusing v10, even though v10 measures +# the same thing on the same corpus. v10 ran with ten containers on a host at +# twice its core count; v11 will run with twelve. Comparing across that carries +# the contention difference into the result, which is the confound this whole +# chaining exists to avoid. An internal control costs one more arm and removes +# the question. launch() { local arm="$1" flags="$2" prop="$3" shard="$4" shards="$5" docker rm -f "m2c-v11-${arm}-${prop}-s${shard}" >/dev/null 2>&1 @@ -78,8 +87,10 @@ launch() { # cover-error only. Slicing needs a criterion to slice towards, and # Cover-Branches has none -- every branch is the goal. -for i in 0 1 2 3; do launch slice "--slice" cover-error "$i" 4; done -for i in 0 1 2 3; do launch slice-seed "--slice --seed-exchange" cover-error "$i" 4; done +for i in 0 1 2; do launch control "" cover-error "$i" 3; done +for i in 0 1 2; do launch slice "--slice" cover-error "$i" 3; done +for i in 0 1 2; do launch seed "--seed-exchange" cover-error "$i" 3; done +for i in 0 1 2; do launch slice-seed "--slice --seed-exchange" cover-error "$i" 3; done sleep 5 say "v11 launched: $(docker ps -q --filter 'name=m2c-v11-' | wc -l) containers"