The difficulty of predicting behavior on stochastic local search algorithms

Abstract

Identifying key properties of Stochastic Local Search (SLS) algorithms, such as convergence to optimal solutions, is essential. Unfortunately, due to their Turing-completeness and Rice’s theorem, their non-trivial semantic properties are generally undecidable. Therefore, most convergence results are achieved by abusing properties that ultimately depict them as simple (probabilistic) exhaustive search algorithms. We show that the general difficulty to prove properties of SLS algorithms has a strong theoretical basis: even when SLS algorithms are deterministic and their memory is linearly bounded, finding out their output from their input configuration is PSPACE-hard — and thus intractable if P PSPACE. This is proven by translating the PSPACE-hard DLBA-ACCEPT problem (i.e. given a Deterministic Linear Bounded Automaton and a word, checking whether the automaton accepts the word) into an instance of the tile-matching problem MPCP such that its solution denotes the configurations traversed by the DLBA during its execution. Simple SLS algorithms can obtain increasing partial solutions for these MPCP instances and provide the answer of the original DLBA-ACCEPT instances. It is also shown that finding out whether an SLS algorithm using linear memory fulfills any non-trivial semantic property is PSPACE-hard. An adaptation of Rice’s theorem dealing with computation artifacts running with linear space is introduced for that purpose. In order to provide an intuitive test of PSPACE-hardness for SLS algorithms, examples of how our criteria is applied to several heuristics, such as depth-first search and genetic algorithms, are shown.