Squeeziness for Non-Deterministic Systems

Abstract

Context: Failed Error Propagation greatly reduces the effectiveness of Software Testing by masking faults present in the code. This situation happens when the System Under Test executes a faulty statement, the state of the system is affected by this fault, but the expected output is observed. Therefore, it is a must to assess its impact in the testing process. Squeeziness has been shown to be a useful measure to assess the likelihood of fault masking in deterministic systems. Objective: The main goal of this paper is to define a new Squeeziness notion that can be used in a scenario where we may have non-deterministic behaviours. The new notion should be a conservative extension of the previous one. In addition, it would be necessary to evaluate whether the new notion appropriately estimates the likelihood that a component of a system introduces Failed Error Propagation. Method: We defined our black-box scenario where non-deterministic behaviours might appear. Next, we presented a new Squeeziness notion that can be used in this scenario. Finally, we carried out different experiments to evaluate the usefulness of our proposal as an appropriate estimation of the likelihood of Failed Error Propagation. Results: We found a high correlation between our new Squeeziness notion and the likelihood of Failed Error Propagation in non-deterministic systems. We also found that the extra computation time with respect to the deterministic version of Squeeziness was negligible. Conclusion: Our new Squeeziness notion is a good measure to estimate the likelihood of Failed Error Propagation being introduced by a component of a system (potentially) showing non-deterministic behaviours. Since it is a conservative extension of the original notion and the extra computation time needed to compute it, with respect to the time needed to compute the former notion, is very small, we conclude that the new notion can be safely used to assess the likelihood of fault masking in deterministic systems.