CloudExpert: An intelligent system for selecting cloud system simulators

Abstract

During the last decade, the research community has developed different simulation tools to model and study cloud systems. However, current cloud simulators focus on specific features that typically do not fully cover all aspects of the cloud infrastructure. The ever-growing number of existing simulators increases the difficulty to properly choose the most appropriate one. Moreover, in certain situations, these simulators must be combined to analyze the features required by the user, which leads to investing a considerable time and effort for their selection. In this paper, we propose CloudExpert, an intelligent system based on metamorphic testing that selects the most appropriate simulator covering the features of interest for the user. In contrast to our previous work, where metamorphic testing is applied to improve models representing a cloud, in this work we analyse the underlying features of several well-known cloud simulators to generate metamorphic rules, which are applied to represent the properties of the simulator. To show the applicability of CloudExpert, we conducted an empirical study where the adequacy of six well-known cloud simulators was analyzed. In this experiment, CloudExpert recommended the most appropriate simulator for eight scenarios involving different aspects of the cloud (energy, storage, network, memory, CPU) and simulator performance; and could also identify strengths and weaknesses of these simulators. Then, we further validated CloudExpert in two different ways. Firstly, the effectiveness of CloudExpert was measured using different faulty cloud simulators. Secondly, we designed a questionnaire based on the results provided by CloudExpert for some of the scenarios of the first experiment. The questionnaire was answered by eight experts in cloud simulation, confirming the usefulness of the tool.