The use of combinatorial data is part of a set of advanced test generation concepts which generally leads to longer test generation. Note that adding combinatorial data coverage also rarely helps to improve coverage for models. To work most effectively and simplify model and test analysis, model combinatorial data coverage should only be used in the final stages of model development.
Recommendation: Model combinatorial data only after you have completed modelling the entire targeted functionality and achieved 100% coverage in test generation. Similarly, disable combinatorial data coverage settings if you maintain or significantly extend a model from a previous delivery and only bring back the settings once you have achieved 100% coverage with the modified model. |
As explained in detail in the separate document “Combinatorial Testing with Conformiq Creator”, (see References) multiple settings combinatorial data coverage settings multiply the effect of combinatorial data on test generation which not only increases test generation time and memory usage but also to very many tests that need to be validated. A common misunderstanding is that setting data coverage in multiple actions that data of these actions would be combined – the only way this can be modeled is using one combinatorial action. The “exploratory” settings for “Data Coverage” options should only be used by expert users.
Recommendation: Limit the use of combinatorial action (and combinatorial “Data Coverage” options) to a few uses per model, i.e., not every input action of your model. In particular, limit the use of the “exploratory” setting in data coverage options to no more than one or two places per model. Validate that use of these options really adds value to your generated tests, e.g., if the use of “exploratory” generates “too many” tests for you then remove or reduce the setting. |