Uma linguagem específica de domínio para geração de testes de performance

Abstract

This work presents GenMeter, a tool composed of: (i) a domain-specific language (DSL) used to describe textually performance tests; and (ii) a component that uses those described test specifications to generate projects in different performance test execution platforms. The purpose is to use concepts defined in the language to abstract the concepts of each platform, which are often modeled differently regarding nomenclature and/or structure and even dependent on the tool rather than just the domain. The proposed tool supports SOAP, REST and web applications performance tests to JMeter and Silk Performer. It also allows customization to new test types and target platforms. Studies were conducted to evaluate the use of the tool: 3 tests of web applications, REST and SOAP services have been rewritten in the DSL and then were generated projects to the target platforms, to be executed. After the adaptation and new implementations necessary for the project generation, we obtained feedback regarding the ability to customize the tool for the applications types and platforms and organizations features. Moreover, the scripts were also evaluated for their conciseness: tests were created with Gatling.io tool (also based on the company’s test) to compare with existing DSL and Silk Performer tests. Our study also compared the total number of words needed to define each test and the relation between the number of reserved words and the total number of words; and the relationship between the number of reserved words out of context, and the total of reserved words. Tests created with GenMeter have, on average, 59,15% less words in relation to Silk Performer tests and 39,43% in relation to Gatling.io tests, except by one test type, where GenMeter’s tests get little more than the double (138,35%) of words. In second comparison, on average, tests with the GenMeter presented a percentage of 56.33% of reserved words in relation to the total, against 39.98% from Silk Performer and 67.03% from Gatling.io. This first comparison can be interpreted as the amount of additional information that the user needs to provide for each language, in addition to the structures provided by them. In the third comparison, which can be interpreted as how the syntax of the host language may interfere with viewing of the test information, GenMeter had an average of 23.57% of reserved words out of context relative to the total of reserved words against 53.38% from Silk Performer and 54.60% from Gatling. Thus, it was possible to observe the benefits of using DSL for different types of applications, customizing it according to certain concepts and features platforms and organizations.