End-to-end testing Java web applications with an embedded Tomcat

In this instalment on assembly testing [1] I’ll talk about how automated system tests run over my Java web applications. The set-up is rather cheap, not very labour-intensive and can incrementally evolve from a simple assembly test (aka smoke test) to a full-fledged system test. Since this post is rather concrete naming tools and frameworks, I’ll present the actors up-front: I’ll show how to run end-to-end tests on a Spring web application during a Maven build with the tests involving the user interface (HTML + Javascript), the HTTP communication with the Java backend, the Spring application consisting of controllers and services and finally the database. A quick note on naming: in this post the terms “end-to-end tests” and “system tests” are synonyms.

The accompanying source code for this project is on Github [9] at https://github.com/ggeorgovassilis/test-with-embedded-tomcat

Running end-to-end tests during the build cycle

The action plan is straight-forward and aimed at running end-to-end tests on a Java web application from within a Maven build. It’s helpful to re-read the Maven build lifecycle [2] if you’re not up-to-date on it.

Maven first resolves dependencies and compiles the application
It then runs unit tests in the test phase
The application WAR is built in the package phase
Integration tests are then run in the integration-test phase. That’s where we’ll run an embedded Tomcat and interact with the application through the HTMLUnit [8] browser.
Since the scope of this document is end-to-end testing we’ll not look at anything that happens after the testing phase like deployment and report generation

Maven interestingly has a second test phase, the integration-test phase where it executes selective tests after the application has been compiled and packaged. After the package phase the project’s target folder contains another folder (usually named similarly to applicationname-war) which is the exact contents of the WAR file deployed on the file system. The idea is to run an embedded Tomcat server and tell it to serve the web application from that folder. Integration tests will then use an HTMLUnit browser (if you haven’t heard about it before: it is a web browser written in Java) to interact with the application just like a human user would do.

Getting Maven dependencies right and declaring integration tests in pom.xml

Dependencies for this endeavour are surprisingly hard to get right since most modules have runtime dependencies on each other but don’t explicitly declare them in the project’s pom.xml (for an up-to-date version go to github). We’ll need the embedded Tomcat, Htmlunit and also will have to sort out logging.

Next, we tell Maven to run all tests during the normal testing phase except those in the integration package and only run the integration tests during the integration-test phase:

 <plugin>
    <!-- Separates the unit tests from the integration tests. -->
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.20</version>
    <configuration>
        <skip>true</skip>
        <trimStackTrace>false</trimStackTrace>
        <configuration>
            <useSystemClassLoader>${useSystemClassloader}</useSystemClassLoader>
        </configuration>
    </configuration>
    <executions>
        <execution>
            <id>unit-tests</id>
            <phase>test</phase>
            <goals>
                <goal>test</goal>
            </goals>
            <configuration>
                <skip>false</skip>
                <configuration>
                    <useSystemClassLoader>${useSystemClassloader}</useSystemClassLoader>
                </configuration>
                <includes>
                    <include>**/*Test.java</include>
                </includes>
                <excludes>
                    <exclude>**/integrationtests/*</exclude>
                </excludes>
            </configuration>
        </execution>
        <execution>
            <id>integration-tests</id>
            <phase>integration-test</phase>
            <goals>
                <goal>test</goal>
            </goals>
            <configuration>
                <skip>false</skip>
                <configuration>
                    <useSystemClassLoader>${useSystemClassloader}</useSystemClassLoader>
                </configuration>
                <includes>
                    <include>**/integrationtests/*Test.java</include>
                </includes>
            </configuration>
        </execution>
    </executions>
</plugin>

The “useSystemClassloader” setting is surefire a property which comes in handy when solving class loading issues.

System test starts Tomcat and runs HTMLUnit

System tests are a bit tricky because they interfere which each other. They operate on a stateful application, their changes are persisted in the application’s database so that a test run modifies the application which may affect subsequent tests. The only way to avoid test interference is to either build a backdoor feature toggle [5] into the application that resets the application state or restart the server for each test. In the scope of this post I’ll go for the easy, second option where each test restarts the application server, trading speed for simplicity.

We’ll write a base integration test which takes care of running Tomcat, the web application and the HTMLUnit browser. Concrete tests can extend the base test class and interact with the web application. The complete code for the base class is on Github, the interesting parts:

Configures Tomcat to load the application from where maven copied it to and assigns a temporary location for the workspace.

String webappDirLocation = "target/test-with-embedded-tomcat-0.0.2-SNAPSHOT";
tomcat = new Tomcat();
File temp = File.createTempFile("integration-test", ".tmp");
tomcat.setBaseDir(temp.getParent());
tomcat.setPort(SERVER_PORT);

Starting with Tomcat 8.0.34 JAR scanning changed; if the below code is omitted then Tomcat will show some (harmless) exceptions saying it can’t scan JAR files like xerxesImpl (which it looks for at wrong locations). Some JAR files, however, it finds and needs, like JSTL JARs. You probably need to experiment here:

// disable scanning of dependencies introduced by maven
StandardJarScanner discardingJarScanner = new StandardJarScanner();
discardingJarScanner.setJarScanFilter(new JarScanFilter() {
            
    public boolean check(JarScanType jarScanType, String jarName) {
         return jarName.contains("jstl-");
    }
});
context.setJarScanner(discardingJarScanner);

Last not least, there’s a way to inject server-provided JNDI resources such as data sources:

 // enable JNDI, e.g. for server-provided data sources
 tomcat.enableNaming();
 ContextResource resource = new ContextResource();
 resource.setName("jdbc/twetDataSource");
 resource.setAuth("Container");
 resource.setType(javax.sql.DataSource.class.getName());
 resource.setScope("Sharable");
 resource.setProperty("driverClassName", "org.hsqldb.jdbc.JDBCDriver");
 resource.setProperty("url", "jdbc:hsqldb:mem:testdb");
 context.getNamingResources().addResource(resource);
 tomcat.getServer().getGlobalNamingResources().addResource(resource);

Let’s look a bit at this beast… the test is a JUnit 4 test with two setup methods. setupServer locates the maven target folder and starts the embedded Tomcat server at port 7777. The setup can become more complicated if the application needs JNDI and resources provided by the application server such as datasources or JMS queues, there is more at [3] on the topic. The setup method also installs a system property which would cause a Spring application (almost all my Java applications use Spring these days) to pick the “integrationtest” profile [4] which can be used to adapt the application to different environments, such as production, unit tests and integration tests.

setupBrowser creates an HTMLUnit instance that should behave similarily to a Chrome browser. The applications I usually test are Javascript-heavy, so there is a bit of fiddling required to get the test browser to behave in a useful (I purposefully avoid the term “correct”) way. Symmetrically to the setup methods, there is a tear-down method after which stops the Tomcat server and disposes of the web browser instance.

A concrete test: visiting the home page

This test extends the BaseIntegrationTest, reads the application’s home page and expects to find a login link there and a DIV with the current date:

@Category(IntegrationTest.class)
public class TestLoginForm extends BaseIntegrationTest{
  final String homePage = "http://localhost:"+SERVER_PORT+"/mywebapp/";
  
  @Test
  public void test_homepage_correct_date_and_headlines() throws Exception{
    Date now = new Date();
    SimpleDateFormat sdf = new SimpleDateFormat("dd-MM-yyyy");
    String expectedDate = sdf.format(now);
    HtmlPage page = webClient.getPage(baseUrl);
    List list = page.getByXPath("//div[@class='today']");
    assertEquals(1, list.size());
    HtmlDivision div = (HtmlDivision)list.get(0);
    assertTrue(div.asText().contains(expectedDate));
    list = page.getByXPath("//div[contains(@class,'headline')]/b");
    assertEquals("Twetland election polls", ((HtmlElement)list.get(0)).getTextContent());
    }
  }

The test can obviously be extended towards more complex interactions with the web application.

Things to watch out for

Classpath issues because of Maven-provided JARs: the web application would normally find its own dependencies, namely everything in the WEB-INF/lib path. Unfortunately all these dependencies are already in the classpath of the unit test which interferes in an especially annoying way with annotations and JAR scanning where classes are found and instantiated twice. Workarounds may vary from explicitly setting a custom class loader ( tomcat.getServer().getParentClassLoader() ) to instructing Surefire to remove certain packages from the classpath [6].

Related to the classpath topic is spring finding persistence.xml twice [7] in the classpath (Conflicting persistence unit definitions for name…). Apart from the general solution to classpath resolution, the particular issue can be fixed by telling the LocalEntityManagerFactoryBean the exact location of persistence.xml

IDEs and Maven: the integration testing approach described here vitally depends on the application being properly built and deployed in Maven’s target path – many IDEs, including Eclipse, use their own compilation and deployment mechanism which does not update the target folder. Thus, while unit tests will work fine, integration tests will yield wrong results unless a maven clean install is first executed.

Last not least, the inclusion of the embedded Tomcat introduces several classpath dependencies like logging libraries or Tomcat’s EL implementation which may affect the application’s behaviour. I noticed a slightly different behaviour in resource mappings when upgrading from 8.0.28 to 8.0.30.

Resources

[1] Assembly testing
http://georgovassilis.blogspot.com/2016/01/assembly-tests.html

[2] Maven build lifecycle
https://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html

[3] Adding datasource programmaticaly to JNDI context in embedded tomcat 7
http://stackoverflow.com/questions/20860283/adding-datasource-programmaticaly-to-jndi-context-in-embedded-tomcat-7

[4] Spring profiles
https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-profiles.html

[5] Feature toggles
http://martinfowler.com/bliki/FeatureToggle.html

[6] Configuring Surefire
http://maven.apache.org/surefire/maven-surefire-plugin/examples/configuring-classpath.html

[7] Duplicate persistence unit
http://stackoverflow.com/questions/11656848/conflicting-persistence-unit-definitions

[8] HTMLUnit
http://htmlunit.sourceforge.net/

[9] test-with-embedded-tomcat
https://github.com/ggeorgovassilis/test-with-embedded-tomcat

4 thoughts on “End-to-end testing Java web applications with an embedded Tomcat”

George Georgovassilis says:

14 Feb 2016 at 10:28

Some updates to data source handling

LikeLike

Nick says:

26 Mar 2016 at 21:10

I think most of these stuff nowadays if you use spring-boot-test and spring-boot you have far more cleaner setup and utilizing MockMVC simplifies your life…

LikeLike

George Georgovassilis says:

27 Mar 2016 at 9:37

Absolutely! TDD is a really good technique with many successful tools around it like the ones you listed that cover test design, test automation, dependency management and dependency mocking. This post is just a technical implementation of an older post [1] where I tried to make the point that if a project doesn't want to afford decent testing it should at least implement a smoke test since the investment is minimal and the return is huge.

[1] http://georgovassilis.blogspot.com/2016/01/assembly-tests.html

LikeLike

George Georgovassilis says:

27 Mar 2016 at 9:46

By the way, my favourite tool for integration tests is Arquillian http://arquillian.org/

LikeLike