Sometimes the duration of automated tests written with ScalaTests library can be reduced in times easly.
Here are the few steps I found to make it fast and easier.
1. Maven: use the latest version of maven itself 3.5.x and scala-maven-plugin 3.4.x Be careful with other plugins aren't supporting parallel execution, if you see this message in build log, carefully review the plugins list
The best solution is usage of shared fixtures or test scopes. If you don't want to refactor your test in BigBang approach it's easier to extend the Suits/Specs with org.scalatest.OneInstancePerTest.
If you applied the forking policy from second recommendation to at most once - than all your tests are running in the same JVM and it's possible to share some class instances.
There are some cons with following to this approach - tests aren't independent anymore. Every test should be implemented with keeping in mind that resources are mutated in parallel.
For example Unit tests that counts the final number of rows in table - should be updated to check by the number of rows by condition. In meantime in before[Each/All] should not clear the whole state of shared resource but the parts are related to current test case only. Here is example how to share the embedded cassandra instance between the Specs:
5. Migrate to async. In case of testing the async services/functionality instead of blocking the test for checking the results use *Async implementations for the Specs: AsyncFlatSpec, AsyncFunSpec etc. More details are here.
6. Review your before/after[Each/All] methods. Sometimes the execution of those makes major influence into whole test run. If your close the resources - it could be not mandatory to do it - because of short running JVM process is spawned only for test run. If your delete all the data to clean up the state before the tests then think on how it's possible to isolate the tests - via UUID or unique table names etc, this is mandatory step for recommendation #3
7. Tune the JVM params, test goal in maven is short duration Java process and these JVM args are representing this fact:
Here are the few steps I found to make it fast and easier.
1. Maven: use the latest version of maven itself 3.5.x and scala-maven-plugin 3.4.x Be careful with other plugins aren't supporting parallel execution, if you see this message in build log, carefully review the plugins list
[WARNING] *****************************************************************
[WARNING] * Your build is requesting parallel execution, but project *
[WARNING] * contains the following plugin(s) that have goals not marked *
[WARNING] * as @threadSafe to support parallel building. *
2. Switch on parallel mode and disable JVM forking:
Maven and sbt:
ScalaTest's normal approach for running suites of tests in parallel is to run different suites in parallel, but the tests of any one suite sequentially. This approach should provide sufficient distribution of the work load in most cases, but some suites may encapsulate multiple long-running tests. Such suites may dominate the execution time of the run. If so, mixing in this trait into just those suites will allow their long-running tests to run in parallel with each other, thereby helping to reduce the total time required to run an entire run.3. Analyse the shared state. Switching the tests into parallel mode sometimes introduces race conditions. If test methods are sharing mutable state in class fields. For example:
The best solution is usage of shared fixtures or test scopes. If you don't want to refactor your test in BigBang approach it's easier to extend the Suits/Specs with org.scalatest.OneInstancePerTest.
Trait that facilitates a style of testing in which each test is run in its own instance of the suite class to isolate each test from the side effects of the other tests in the suite. If you mix this trait into a Suite, you can initialize shared reassignable fixture variables as well as shared mutable fixture objects in the constructor of the class. Because each test will run in its own instance of the class, each test will get a fresh copy of the instance variables. This is the approach to test isolation taken, for example, by the JUnit framework.4. Reduce the number of heavy resources via re-sharing them. Sometimes there could be great performance boost when some state of tests is shared between the suits. For example embedded db instance - if it's started before each test and switched off after - it could require more time than test execution itself.
If you applied the forking policy from second recommendation to at most once - than all your tests are running in the same JVM and it's possible to share some class instances.
There are some cons with following to this approach - tests aren't independent anymore. Every test should be implemented with keeping in mind that resources are mutated in parallel.
For example Unit tests that counts the final number of rows in table - should be updated to check by the number of rows by condition. In meantime in before[Each/All] should not clear the whole state of shared resource but the parts are related to current test case only. Here is example how to share the embedded cassandra instance between the Specs:
5. Migrate to async. In case of testing the async services/functionality instead of blocking the test for checking the results use *Async implementations for the Specs: AsyncFlatSpec, AsyncFunSpec etc. More details are here.
6. Review your before/after[Each/All] methods. Sometimes the execution of those makes major influence into whole test run. If your close the resources - it could be not mandatory to do it - because of short running JVM process is spawned only for test run. If your delete all the data to clean up the state before the tests then think on how it's possible to isolate the tests - via UUID or unique table names etc, this is mandatory step for recommendation #3
7. Tune the JVM params, test goal in maven is short duration Java process and these JVM args are representing this fact:
-XX:+TieredCompilation -XX:TieredStopAtLevel=1
No comments:
Post a Comment