A survey on formal active and passive testing with applications to the cloud

Abstract

Testing has become an integral part of innovation, production and operation of systems. The activity of testing is already a flourishing area with the active participation of a large community of researchers and experts who are highly aware of its importance and impact for the future deployment and use of software and systems. Formal methods have proved to be very promising for developing automated and generic testing methods. Actually,the combination of formal methods and testing is currently well understood and tools to automate testing activities are widely available. (Formal) testing is the assessment, by means of experiments, that a product conforms to its (formal) requirements. Test cases are designed to test particular aspects of the system, called test objectives. In order to formally obtain testing objectives, it is necessary to provide mathematical models for the semantic of the studied system, formal frameworks for testing, languages to describe the expected properties or requirements of the system in a precise and unambiguous way and methods to generate the expected test cases. In addition, active methods require deploying a test environment(test architecture) to execute test cases and to observe implementation reactions. They may also interrupt the system normal functioning arbitrarily, for example by resetting it after each test case execution. However,when a system is deployed in an integrated environment,it becomes quite difficult to access it. Moreover,active methods may disturb the natural operation of the implementation under test. So, these ones may not be suitable in regards to the tested system. Passive testing represents another interesting alternative, which offers several advantages, for example, to not disturb the system while testing. In this paper we present a survey covering the main approaches to formal testing. We have divided the survey in three parts. First, we distinguish between active and passive approaches to formal testing. Next, we review some of the work on testing the cloud and on testing in the cloud.