Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement a new rewrite engine based on rewriting theory #1717

Open
wants to merge 138 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
138 commits
Select commit Hold shift + click to select a range
b6e1aa6
Made the Rewriter class non copyable.
mlaveaux Apr 25, 2019
40179f0
Removed commented code.
mlaveaux Apr 26, 2019
a76df75
Added innermost as a new rewriter implementation.
mlaveaux Apr 26, 2019
269c6e6
Removed an obsolete comment block.
mlaveaux Apr 26, 2019
a38be7b
Added several functions implemented according to the pseudocode.
mlaveaux Apr 26, 2019
c787008
Added a constexpr constant to indicate that a variable is undefined.
mlaveaux Apr 26, 2019
88499dc
Implemented a substitution function, abstraction case not yet finished.
mlaveaux Apr 28, 2019
4e7754e
The match function applies the substitution to the right-hand side.
mlaveaux Apr 28, 2019
1469a28
Fixed an issue with the capture avoiding substitution, updated comments.
mlaveaux Apr 28, 2019
5e66d8e
Only print one message per tried (succesful or failing) rule.
mlaveaux Apr 28, 2019
361fece
Optimized matching by only trying equations that start with the same …
mlaveaux Apr 28, 2019
dece028
Renamed defined to count to match the standard map interface.
mlaveaux Apr 28, 2019
8c68d0a
Made match_lhs a free function (accepting any substitution template).
mlaveaux Apr 28, 2019
6ce5550
Keep track of sub-terms that are already in normal form.
mlaveaux Apr 28, 2019
87cab79
Fixed a mistake induced by merging.
mlaveaux Apr 28, 2019
0c23440
Use an unordered_map_large to store the matching substitution.
mlaveaux Apr 28, 2019
fe3f214
Moved the implementation of recording normal forms to helper functions.
mlaveaux Apr 29, 2019
4d6dabc
Added comment to matching_sigma.
mlaveaux Apr 29, 2019
a8e6405
Added computing of conditions similar to the jitty rewriter.
mlaveaux Apr 29, 2019
a866054
Fixed the rewriting of individual function symbols.
mlaveaux Apr 29, 2019
abe76e2
Enable an option to toggle normal form tracking.
mlaveaux Apr 29, 2019
fac96a0
Fixed a mistake with the normal form tracking.
mlaveaux Apr 29, 2019
dbfc5b4
Moved static functions to a single utility header.
mlaveaux Apr 29, 2019
8dd11d1
Smaller fixed, the substitution does not have to be applied again.
mlaveaux Apr 29, 2019
655e6b7
Added a possible implementation of the construction stacks.
mlaveaux Apr 30, 2019
53f00fd
Optimized construction_stacks by removing the temporary argument vector.
mlaveaux Apr 30, 2019
8ba7f1f
The matching substitution is used recursively and thus can not be saved.
mlaveaux Apr 30, 2019
1eac228
No longer rewrite in the ConstructionStack.
mlaveaux Apr 30, 2019
1c41eaf
Moved the choice for using the construction stack to a helper function.
mlaveaux Apr 30, 2019
75ff951
Small documentation fixes and removed an extraneous semicolon.
mlaveaux May 2, 2019
8c52e40
Optimized ConstructionStack by reusing an existing argument stack.
mlaveaux May 2, 2019
fc0da27
Use get_nested_head to determine the head of a data_expression.
mlaveaux May 2, 2019
85bb11a
Fixed smaller documentation issues.
mlaveaux May 2, 2019
37e80e5
Use the stack_array to store arguments during rewriting and substitut…
mlaveaux May 2, 2019
145b9b6
Only try to perform a match when the application is not in normal form.
mlaveaux May 2, 2019
034dc88
Check for the given term being in normal form in the match function.
mlaveaux May 2, 2019
9d9416f
Replaced the unordered_map by a mapping using the index of the head s…
mlaveaux May 2, 2019
5da8d71
Only insert normal forms when they are not already marked as such.
mlaveaux May 2, 2019
d96229e
Made the m_local_sigma (was m_matching_sigma) and the argument stack …
mlaveaux May 3, 2019
aaaf752
When terms are equivalent they must match by definition.
mlaveaux May 3, 2019
3812d37
Added caching for results of rewriting function applications.
mlaveaux May 3, 2019
c816d50
Use the fixed_size_cache with a size of 1000 elements.
mlaveaux May 3, 2019
3b7d504
Updated the comments and removed some unnecessary printing.
mlaveaux May 3, 2019
9ee1c80
Increased default fixed_size_cache to 1024 elements, use emplace in i…
mlaveaux May 3, 2019
b969e2e
Moved caching to before the call to rewrite_application.
mlaveaux May 4, 2019
f3f43df
Fixed some warnings, improved documentation and changed caching behav…
mlaveaux May 4, 2019
d3e29f2
Applied the fifo_policy for the rewrite cache and only cache results …
mlaveaux May 6, 2019
9eacd65
Added a compile time option to count the number of applications per r…
mlaveaux May 6, 2019
2b99802
Added a cache_metric object to track cache hits and misses.
mlaveaux May 6, 2019
df75cca
Print cached rewrite steps and rewrite cache metrics.
mlaveaux May 9, 2019
9a4c5fa
Added rewriting lambda abstractions by renaming all bound variables.
mlaveaux May 9, 2019
000f6f6
Moved applying the substitution to a different header file.
mlaveaux May 10, 2019
5f52123
Print alpha-conversion before rewriting the body.
mlaveaux May 10, 2019
8d2c500
Moved the implementation of the construction stack to its own header.
mlaveaux May 14, 2019
d1d6b53
Moved (naive) matching to its own class and implementation.
mlaveaux May 14, 2019
ab8bbd9
The substitution must be mutable when applying due to renaming bound …
mlaveaux May 22, 2019
a7c88c7
Renamed all variables in rewrite rules to avoid conflicts.
mlaveaux May 23, 2019
dfda949
Print the single-step rewrite steps performed during rewriting.
mlaveaux May 23, 2019
051f76c
Made the unprotected_aterm constructor explicit.
mlaveaux May 23, 2019
98db751
The substitution must be stored when rewriting conditions.
mlaveaux May 23, 2019
5eb3b99
Added a virtual base class for matching algorithms.
mlaveaux Jun 12, 2019
0cab73a
Added an implementation of an pattern match automaton based matcher.
mlaveaux Jun 13, 2019
7931826
Fixed several compilation errors after rebasing.
mlaveaux Aug 23, 2019
78d5b63
Changed the interface of Matcher to have less impact on performance.
mlaveaux Aug 23, 2019
d2f8907
Added the automaton matcher as default.
mlaveaux Aug 26, 2019
533c7cf
Moved the stopwatch class to the mcrl2::utilities namespace.
mlaveaux Sep 5, 2019
852a7a1
Simplified the matching implementation.
mlaveaux Sep 6, 2019
dccbce1
Added naive handling of non-linear patterns.
mlaveaux Sep 7, 2019
a8b756c
Moved the linearisation of a TRS to a separate header.
mlaveaux Sep 18, 2019
bab53cb
Do not redefine equivalence_classes in the automaton_matcher header.
mlaveaux Sep 25, 2019
8cbda58
Moved is_consistent to the consistency header.
mlaveaux Sep 25, 2019
06bf21a
Moved the implementation of the underlying automaton to a separate cl…
mlaveaux Sep 26, 2019
1c0618b
Changed the underlying automaton to an indexed version.
mlaveaux Sep 26, 2019
1ae0280
Rename TRS variables such that their position is the identifier (and …
mlaveaux Sep 26, 2019
dfe0170
Moved some implementations to headers and added the AdaptiveMatcher.
mlaveaux Sep 26, 2019
d5e2521
Use a vector for the argument stack that is preserved between calls.
mlaveaux Sep 26, 2019
b2565be
Worked on the implementation of the APMA construction.
mlaveaux Oct 1, 2019
53b7927
Worked on the construction of the EAPMA.
mlaveaux Oct 2, 2019
2e5789c
Removed the automaton_matcher as it is subsumed by the adaptive_matcher.
mlaveaux Oct 4, 2019
f4eb0be
Simplified and fixed several parts of the EPMA evaluation.
mlaveaux Oct 4, 2019
b972be1
Fixed some performance issues in the AdaptiveMatcher.
mlaveaux Oct 5, 2019
0f57375
Enabled a choice for on-the-fly or naive consistency checking in Naiv…
mlaveaux Oct 20, 2019
932f0f6
Fixed several issues, implemented select and restrict.
mlaveaux Oct 20, 2019
0a44354
Changed the interface of the matching algorithms to perform better.
mlaveaux Oct 21, 2019
6ffede6
Print additional debug and performance metrics, added options for opt…
mlaveaux Oct 21, 2019
217f605
Moved all substitution, on right hand sides exclusively to the final …
mlaveaux Oct 21, 2019
25a4d00
Moved the substitution to the final state to avoid unnecessary substi…
mlaveaux Oct 21, 2019
f90044b
Added an option for greedy matching (choose first available match).
mlaveaux Oct 21, 2019
fe36e98
Fixed some issues, replaced mutable_indexed_substitution by a map sub…
mlaveaux Oct 21, 2019
80b6847
Removed an unnecessary include.
mlaveaux Oct 21, 2019
92c442b
Moved the definition and functions related to positions to a differen…
mlaveaux Oct 23, 2019
28a9fd1
Implemented linearisation using positions, added a naive check based …
mlaveaux Oct 23, 2019
1829776
Added variables of the condition to be substituted.
mlaveaux Oct 23, 2019
f44cb32
Added a consistency check based on the indices stored for matching.
mlaveaux Oct 23, 2019
4643168
Fixed various issues.
mlaveaux Oct 23, 2019
2116640
Changed the greedy algorithm slightly.
mlaveaux Oct 23, 2019
acb3b33
Check for consistency on-the-fly enabled first.
mlaveaux Oct 23, 2019
cdea9f9
Changed the matching interface to return an iterator over all matches.
mlaveaux Oct 24, 2019
cf654b1
Simplified the cases for conditional rewriting.
mlaveaux Oct 24, 2019
6133d00
The matching substitution must be preserved for the adaptive automata.
mlaveaux Oct 24, 2019
d2361ab
Fixed some issues with rewriting conditions, added more logging.
mlaveaux Oct 25, 2019
910db10
Fixed an issue with progressing when the current index is not consist…
mlaveaux Nov 2, 2019
c71d66b
Added an optional on-the-fly check for non confluent rules.
mlaveaux Nov 2, 2019
47c9a86
Print additional information about the rewriter.
mlaveaux Nov 3, 2019
9f60fac
Brute force optimize for the smallest automaton.
mlaveaux Nov 4, 2019
cd92d9a
Fixed compilation issues under MSVC.
mlaveaux Apr 15, 2020
d42df56
Implemented enumeration of quantifiers and rewriting of where clauses.
mlaveaux Apr 16, 2020
8e9a6f7
Preparation for reusing indices in the automaton matcher.
mlaveaux Apr 16, 2020
9c1cfb5
Implemented tracking normal forms using tags.
mlaveaux Apr 16, 2020
3703655
Commented the confluence detection out.
mlaveaux Apr 17, 2020
bf04658
Added a sequence_substitution to store the matching substitution.
mlaveaux Apr 18, 2020
0f24528
Tracking normal forms using tags is now more efficient than set based.
mlaveaux Apr 18, 2020
3a5e707
Added the missing sequence_substitution header, removed unused code.
mlaveaux Apr 18, 2020
da496da
Fixed several MSVC warnings.
mlaveaux Apr 18, 2020
53003e3
Optimized the sequence substitution by reusing a single allocated queue.
mlaveaux Apr 20, 2020
bdfac65
Improved the interface of the iterator returned by the matching algor…
mlaveaux Apr 24, 2020
1645942
Added the original equation to a linear_data_equation for printing.
mlaveaux Apr 25, 2020
bee4e81
Fixed an error in the sequence_substitution.
mlaveaux Apr 25, 2020
97bd6be
Introduced function symbols for consistency checking checkmark and cr…
mlaveaux May 8, 2020
370cbce
Added consistency states.
mlaveaux May 8, 2020
93b9185
Added the evaluation of a consistency state.
mlaveaux May 8, 2020
50eebe8
Added the consistency part of the construction.
mlaveaux May 8, 2020
0b84214
Fixed a bug with the computation of workC.
mlaveaux May 8, 2020
6bd7e03
Removed on-the-fly consistency checking from the iterator.
mlaveaux May 8, 2020
3c1f406
Added a printer for expressions in the rewriter.
mlaveaux Jun 2, 2020
c7f589e
Do not reset the rewrite rule statistics for every top-level rewrite.
mlaveaux Jun 2, 2020
fad7a9c
Made jitty the default rewriter.
mlaveaux Jun 19, 2020
e577094
Fixed an issue with the percentage calculation in cache_metric.
mlaveaux Jun 19, 2020
a2c7560
Collect information about the number of rewrite calls.
mlaveaux Jun 19, 2020
22c3d46
Fixed some compiler warnings.
mlaveaux Jun 19, 2020
88d085a
Minor changes.
mlaveaux Jul 21, 2020
962b8a4
Added the REC benchmarks to benchmark the rewriters.
mlaveaux Jul 18, 2023
bb05945
Resolved more compilation errors.
mlaveaux Jul 18, 2023
ab49534
Fixed more compilation errors.
mlaveaux Jul 18, 2023
8a6e328
Fixed compilation error.
mlaveaux Dec 5, 2023
9ac173f
Fixed the namespace of stopwatch.
mlaveaux Mar 6, 2024
2b94aa6
Fixed the namespace in the tools.
mlaveaux Mar 8, 2024
29a6dfd
These benchmarks were already moved upstream
mlaveaux Mar 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 1 addition & 40 deletions libraries/atermpp/benchmark/benchmark_shared.h
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ using namespace atermpp;
template<typename F>
void benchmark_threads(std::size_t number_of_threads, F f)
{
stopwatch timer;
mcrl2::utilities::stopwatch timer;

// Initialize a number of threads.
std::vector<std::thread> threads(number_of_threads - 1);
Expand All @@ -40,45 +40,6 @@ void benchmark_threads(std::size_t number_of_threads, F f)
std::cerr << "time: " << timer.seconds() << std::endl;
}

/// \brief Create a nested function application f_depth. Where f_0 = c and f_i = f(f_i-1,...,f_i-1).
template<bool with_converter = false>
aterm_appl create_nested_function(const std::string& function_name, const std::string& leaf_name, std::size_t number_of_arguments, std::size_t depth)
{
// Create a suitable function application.
function_symbol f(function_name, number_of_arguments);
function_symbol c(leaf_name, 0);

aterm_appl c_term(c);

// Initialize a wide function application.
std::vector<aterm> arguments(f.arity());
for (std::size_t i = 0; i < arguments.size(); ++i)
{
arguments[i] = c_term;
}
aterm_appl f_term(f, arguments.begin(), arguments.end());

for (std::size_t j = 0; j < depth; ++j)
{
// Create a very wide nested function application
for (std::size_t k = 0; k < arguments.size(); ++k)
{
arguments[k] = f_term;
}

if (with_converter)
{
make_term_appl(f_term, f, arguments.begin(), arguments.end(), detail::do_not_convert_term<aterm>());
}
else
{
make_term_appl(f_term, f, arguments.begin(), arguments.end());
}
}

return f_term;
}

/// \brief Create a nested function application f_depth. Where f_0 = c and f_i = f(f_i-1,...,f_i-1).
/// However, this function uses the fixed arity constructor of length N which should be faster.
template<std::size_t N>
Expand Down
2 changes: 1 addition & 1 deletion libraries/atermpp/include/mcrl2/atermpp/aterm.h
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ class unprotected_aterm

/// \brief Constructor.
/// \param term The term from which the new term is constructed.
unprotected_aterm(const detail::_aterm* term) noexcept
explicit unprotected_aterm(const detail::_aterm* term) noexcept
: m_term(term)
{}

Expand Down
9 changes: 6 additions & 3 deletions libraries/atermpp/include/mcrl2/atermpp/detail/aterm_hash.h
Original file line number Diff line number Diff line change
Expand Up @@ -163,10 +163,13 @@ std::size_t aterm_hasher<N>::operator()(const _aterm& term) const noexcept
const std::size_t arity = (N == DynamicNumberOfArguments) ? f.arity() : N;

// This is a function application with arguments, hash each argument and combine the result.
const _aterm_appl<>& term_appl = static_cast<const _aterm_appl<>&>(term);
for (std::size_t i = 0; i < arity; ++i)
if constexpr (N > 0)
{
hnr = combine(hnr, term_appl.arg(i));
const _aterm_appl<>& term_appl = static_cast<const _aterm_appl<>&>(term);
for (std::size_t i = 0; i < arity; ++i)
{
hnr = combine(hnr, term_appl.arg(i));
}
}

return hnr;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -412,7 +412,7 @@ typename ATERM_POOL_STORAGE::iterator ATERM_POOL_STORAGE::destroy(iterator it)
const Element& term = *it;

// Trigger the deletion hook before the term is actually destroyed.
call_deletion_hook(&term);
call_deletion_hook(unprotected_aterm(&term));

// Remove them from the hash table, will also destroy terms with fixed arity.
return m_term_set.erase(it);
Expand Down
3 changes: 3 additions & 0 deletions libraries/data/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,12 @@ mcrl2_add_library(mcrl2_data
source/data_specification.cpp
source/typecheck.cpp
source/detail/prover/smt_lib_solver.cpp
source/detail/rewrite/innermost.cpp
source/detail/rewrite/jitty.cpp
source/detail/rewrite/rewrite.cpp
source/detail/rewrite/strategy.cpp
source/detail/match/naive_matcher.cpp
source/detail/match/adaptive_matcher.cpp
${COMPILING_REWRITER_SRC}
EXCLUDE_HEADERTEST
mcrl2/data/detail/rewrite/jittycpreamble.h
Expand Down
15 changes: 15 additions & 0 deletions libraries/data/include/mcrl2/data/data_equation.h
Original file line number Diff line number Diff line change
Expand Up @@ -163,5 +163,20 @@ std::set<data::function_symbol> find_function_symbols(const data::data_equation&

} // namespace mcrl2

namespace std
{

template<>
struct hash<mcrl2::data::data_equation>
{
std::size_t operator()(const mcrl2::data::data_equation& v) const
{
const hash<atermpp::aterm> hasher;
return hasher(v);
}
};

} // namespace std

#endif // MCRL2_DATA_DATA_EQUATION_H

154 changes: 154 additions & 0 deletions libraries/data/include/mcrl2/data/detail/match/adaptive_matcher.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,154 @@
// Author(s): Maurice Laveaux
// Copyright: see the accompanying file COPYING or copy at
// https://github.com/mCRL2org/mCRL2/blob/master/COPYING
//
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE_1_0.txt or copy at
// http://www.boost.org/LICENSE_1_0.txt)
//

#ifndef MCRL2_DATA_DETAIL_ADAPTIVE_MATCHER_H
#define MCRL2_DATA_DETAIL_ADAPTIVE_MATCHER_H

#include "mcrl2/data/data_expression.h"
#include "mcrl2/data/data_equation.h"
#include "mcrl2/data/detail/match/matcher.h"
#include "mcrl2/data/detail/match/automaton.h"
#include "mcrl2/data/detail/match/position.h"
#include "mcrl2/data/detail/match/automaton_consistency.h"
#include "mcrl2/utilities/indexed_set.h"

#include <vector>

namespace mcrl2::data::detail
{

/// \brief Represents the unnamed variable omega.
class not_equal : public function_symbol
{
public:
not_equal()
: function_symbol(core::identifier_string("@@not_equal"), data::sort_expression())
{}
};

/// \returns True iff the given data expression is of type omega.
inline bool is_not_equal(const data_expression& expression)
{
return expression == not_equal();
}

/// \brief An adaptive non-linear pattern matching automata
template<typename Substitution>
class AdaptiveMatcher final
{
public:
/// \brief Initialize the automaton matcher with a number of equations.
AdaptiveMatcher(const data_equation_vector& equations);
virtual ~AdaptiveMatcher() {}

/// \brief A (simplistic) iterator over the matching results.
class const_iterator
{
public:
const_iterator(Substitution& sigma)
: m_matching_sigma(sigma)
{}

/// \brief Construct an iterator over the match_set, computing the consistent patterns from the subterm indexing
const_iterator(std::vector<indexed_linear_data_equation>* match_set,
Substitution& sigma)
: m_match_set(match_set),
m_matching_sigma(sigma)
{}

void operator++()
{
// Find the next consistent match.
++m_current_index;
}

const extended_data_equation& operator*()
{
return (*m_match_set)[m_current_index];
}

bool operator!=(std::nullptr_t) const
{
return (m_match_set != nullptr && m_current_index < m_match_set->size());
}

private:
std::vector<indexed_linear_data_equation>* m_match_set = nullptr;
Substitution& m_matching_sigma;

std::vector<bool> m_consistent;
std::size_t m_current_index = 0;
};

// Matcher interface.

/// \details We assume that the given substitution is the identity.
const_iterator match(const data_expression& term, Substitution& matching_sigma);

private:

/// \brief Each state in the pattern match automaton is labelled with a set of variables.
struct apma_state
{
bool is_consistency_state() const { return compare.first != compare.second; }

bool is_matching_state() const { return position != std::numeric_limits<std::size_t>::max(); }

// Matching states:
std::size_t position = std::numeric_limits<std::size_t>::max(); ///< L, the index of the position to be inspected.
std::vector<std::size_t> argument_positions; ///< These are the positions where arguments must be stored in the subterm table.

// Consistency states:
std::pair<std::size_t, std::size_t> compare = std::make_pair(0,0); ///< L, the pair of positions that must be compared.

// Final states:
std::vector<indexed_linear_data_equation> match_set; ///< L, the equations that matched.
std::vector<std::pair<data::variable, std::size_t>> assignments; ///< P, the variables that must be assigned and their subterm position.
};

// The underlying automaton.
using Automaton = IndexedAutomaton<apma_state>;

/// \returns A subautomaton for the given state and prefix.
Automaton construct_apma(data_expression pref,
const std::vector<indexed_linear_data_equation>& L,
const std::set<std::pair<std::size_t, std::size_t> >& E,
const std::set<std::pair<std::size_t, std::size_t> >& N
);

/// \returns A subset of the given positions, filtering out unecessary positions.
std::set<position> restrict(const std::set<position>& positions, const std::vector<indexed_linear_data_equation>& L);

/// \returns A single position selected from a set of positions.
position select(const std::set<position>& positions, const std::vector<indexed_linear_data_equation>& L);

/// \returns A unique and consistent index for the given position.
std::size_t position_index(const position& positions);

// Information about the underlying automata.

Automaton m_automaton;

mcrl2::utilities::indexed_set<position> m_positions;

std::size_t m_dontcare_index = 0; ///< The index of the dontcare data_expression.
std::size_t m_equal_index = 0;
std::size_t m_not_equal_index = 0;

// Store (temporary) information about the match that can be invalidated.

std::vector<atermpp::unprotected_aterm> m_subterms; ///< A mapping from indices to subterms.
};

} // namespace mcrl2::data::detail

// Explicit instantiations.


#endif // MCRL2_DATA_DETAIL_ADAPTIVE_MATCHER_H
Loading