Testing

This section provides an overview of how testing in GeoServer works and a brief guide for developers to help with the process of writing tests.

Libraries

GeoServer utilizes a number of commonly used testing libraries.

JUnit

The well known JUnit framework is the primary test library used in GeoServer. The current version used is Junit 4.x. While it is possible to continue to write JUnit 3.x style tests with JUnit 4, new tests should be written in the JUnit4 style with annotations.

Current version: 4.11

XMLUnit

The XMLUnit library provides a convenient way to make test assertions about the structure of XML documents. Since many components and services in GeoServer output XML, XMLUnit is a very useful library.

Current version: 1.3

MockRunner

The MockRunner framework provides a set of classes that implement the various interfaces of the J2EE and Java Servlet apis. It is typically used to create HttpServletRequest , HttpServletResponse, etc... objects for testing servlet based components.

Current version: 0.3.6

EasyMock

The EasyMock library is a mocking framework that is used to simulate various objects without actually creating a real version of them. This is an extremely useful tool when developing unit tests for a component A, that requires component B when component B may not be so easy to create from scratch.

Current version: 2.5.2

Testing Categories and Terminology

Software testing falls into many different categories and the GeoServer code base is no exception. In the GeoServer code base one may find different types of tests.

Unit

Tests that exercise a particular method/class/component in complete isolation. In GeoServer these are tests that typically don’t extend from any base class and look like what one would typically expect a unit test to look like.

Integration/Mock

Tests that exercise a component by that must integrate with another component to operate. In GeoServer these are tests that somehow mock up the dependencies for the component under test either by creating it directly or via a mocking library.

System

Tests that exercise a component or set of components by testing it in a fully running system. In GeoServer these are tests that create a fully functional GeoServer system, including a data directory/configuration and a spring context.

Helper classes are provided to help inject your classes into the system configuration including GeoServerExtensionsHelper.

Writing System Tests

System tests are the most common type of test case in GeoServer, primarily because they are the easiest tests to write. However they come with a cost of performance. The GeoServer system test framework provides a fully functional GeoServer system. Creating this system is an expensive operation so a full system test should be used only as a last resort. Developers are encouraged to consider a straight unit or mock tests before resorting to a full system test.

In GeoServer system tests extend from the org.geoserver.test.GeoServerSystemTestSupport class. The general lifecycle of a system test goes through the following states:

  1. System initialization
  2. System creation
  3. Test execution
  4. System destruction

Phases 1 and 2 are referred to as the setup phase. It is during this phase that two main operations are performed. The first is the creation of the GeoServer data directory on disk. The second is the creation of the spring application context.

Single vs Repeated Setup

By default, for performance reasons, the setup phase is executed only once for a system test. This can however be configured by annotating the test class with a special annotation named TestSetup. For example, to specify that the setup should be executed many times, for each test method of the class:

@TestSetup(run=TestSetupFrequency.REPEAT)
public class MyTestCase extends GeoServerSystemTestSupport {
   ...
}

This however should be used only as a last resort since as mentioned before a repeated setup makes the test execute very slowly. An alternative to a repeated setup is to have the test case revert any changes that it makes during its execution, so that every test method can execute in a consistent state. The GeoServerSystemTestSupport contains a number of convenience methods for doing this. Consider the following test:

public class MyTestCase extends GeoServerSystemTestSupport {

   @Before
   public void revertChanges() {
       //roll back any changes made
       revertLayer("foo");
   }

   @Test
   public void testThatChangesLayerFoo() {
      //change layer foo in some way
   }
}

The test makes some changes to a particular layer but uses a before hook to revert any such changes. In general this is the recommended pattern for system tests that must are not read-only and must modify configuration or data to execute.

Method Level SetUp

A third method of controlling test setup frequency is available at the test case level. Annotating a test method with the RunTestSetup annotation will cause the test setup to be run before the test method is executed. For example:

public class MyTestCase extends GeoServerSystemTestSupport {

   @Before
   public void revertChanges() {
       //roll back any changes made
       revertLayer("foo");
   }

   @Test
   public void test1() {
   }

   @Test
   public void test2() {
   }

   @Test
   @RunTestSetup
   public void test3() {

   }

   @Test
   public void test4() {
   }

}

In the above method the test setup will be run twice. Once before the entire test class is run, and again before the test3 method is executed.

Setup/Teardown Hooks

There are a number of ways to hook into test lifecycle to provide setup and tear down functionality.

JUnit @Before, @After, @BeforeClass, @AfterClass

As with any JUnit test various annotations are available to perform tasks at various points of the test life cycle. However with a GeoServer system test one must be wary of the task having a dependency on the system state. For this reason the GeoServerSystemTestSupport class provides its own callbacks.

setUpTestData

This callback method is invoked before the system has been created. It is meant to provide the test with a way to configure what configuration gets created in the GeoServer data directory for the test. By default the test setup will create a standard set of vector layers. This method is where that should be changed, for instance to indicate that the test requires that raster layers be created as well. For example:

public class MySystemTest extends GeoServerSystemTestBase {

   protected void setUpTestData(SystemTestData testData) {
      // do the default by calling super
      super.setUpTestData(testData);

      // add raster layers
      testData.setUpDefaultRasterLayers();
   }
}

Depending on whether the test uses a single or repeated setup this method will be called once or many times.

onSetUp

This callback method is invoked after the system has been created. It is meant for standard post system initialization tasks. Like for instance changing some service configuration, adding new layers, etc...

Depending on whether the test uses a single or repeated setup this method will be called once or many times. For this reason this method can not be used to simply initialize fields of the test class. For instance, consider the following:

public class MySystemTest extends GeoServerSystemTestBase {

    Catalog catalog;

    @Override
    protected void onTestSetup(SystemTestData testData) throws Exception {
       // add a layer named foo to the catalog
       Catalog catalog = getCatalog();
       catalog.addLayer(new Layer("foo"));

       // initialize the catalog field
       this.catalog = catalog;
    }

    @Test
    public void test1() {
       catalog.getLayerByName("foo");
    }

    @Test
    public void test2() {
       catalog.getLayerByName("foo");
    }
}

Since this is a one time setup, the onSetUp method is only executed once, before the test1 method. When the test2 method is executed it is actually a new instance of the test class, but the onTestSetup is not re-executed. The proper way to this initialization would be:

public class MySystemTest extends GeoServerSystemTestBase {

    Catalog catalog;

    @Override
    protected void onTestSetup(SystemTestData testData) throws Exception {
       // add a layer named foo to the catalog
       Catalog catalog = getCatalog();
       catalog.addLayer(new Layer("foo"));

       // initialize the catalog field
       this.catalog = catalog;
    }

    @Before
    public void initCatalog() {
        this.catalog = getCatalog();
    }
}

System Test Data

The GeoServer system test will create a data directory with a standard set of vector layers. The contents of this data directory are as follows:

Stores and Layers

Workspace Store Layer Name Default Style
cdf cdf Deletes Default
cdf cdf Fifteen Default
cdf cdf Inserts Default
cdf cdf Locks Default
cdf cdf Nulls Default
cdf cdf Other Default
cdf cdf Seven Default
cdf cdf Updates Default
cgf cgf Lines Default
cgf cgf MLines Default
cgf cgf MPoints Default
cgf cgf MPolygons Default
cgf cgf Points Default
cgf cgf Polygons Default
cite cite BasicPolygons BasicPolygons
cite cite Bridges Bridges
cite cite Buildings Buildings
cite cite DividedRoutes DividedRoutes
cite cite Forests Forests
cite cite Geometryless Default
cite cite Lakes Lakes
cite cite MapNeatline MapNeatLine
cite cite NamedPlaces NamedPlaces
cite cite Ponds Ponds
cite cite RoadSegments RoadSegments
cite cite Streams Streams
sf sf AgregateGeoFeature Default
sf sf GenericEntity Default
sf sf PrimitiveGeoFeature Default

Note

The gs workspace contains no layers. It is typically used as the workspace for layers that are added by test cases.

Writing Mock Tests

Mock tests, also referred to as integration tests, are a good way to test a component that has dependencies on other components. It is often not simple to create the dependent component with the correct configuration.

A mock test is just a regular unit test that uses functions from the EasyMock library to create mock objects. There is however a base class named GeoServerMockTestSupport that is designed to provide a pre-created set of mock objects. These pre-created mock objects are designed to mimic the objects as they would be found in an actual running system. For example:

public class MyMockTest extends GeoServerMockTestSupport {

   @Test
   public void testFoo() {
      //get the mock catalog
      Catalog catalog = getCatalog();

      //create the object we actually want to test
      Foo foo = new Foo(catalog);
   }
}

Like system tests, mock tests do a one-time setup with the same setUpTestData and onSetUp callbacks.

The benefit of mock tests over system tests is the setup cost. Mock tests essentially have no setup cost which means they can execute very quickly, which helps to keep overall build times down.

EasyMock Class Extension

By default EasyMock can only mock up interfaces. To mock up classes requires the EasyMock classextension jar and also the cglib library. These can be declared in a maven pom like so:

<dependency>
  <groupId>org.easymock</groupId>
  <artifactId>easymockclassextension</artifactId>
  <scope>test</scope>
</dependency>
<dependency>
  <groupId>cglib</groupId>
  <artifactId>cglib-nodep</artifactId>
  <scope>test</scope>
</dependency>

The change is mostly transparent, however rather than importing org.easymock.EasyMock one must import org.easymock.classextension.EasyMock.

Maven Dependencies

All of the GeoServer base test classes live in the gs-main module. However since they live in the test packages a special dependency must be set up in the pom of the module depending on main. This looks like:

<dependency>
  <groupId>org.geoserver</groupId>
  <artifactId>gs-main</artifactId>
  <version>${project.version}</version>
  <classifier>tests</classifier>
  <scope>test</scope>
</dependency>

Furthermore, in maven test scope dependencies are not transitive in the same way that regular dependencies are. Therefore some additional dependencies must also be declared:

<dependency>
 <groupId>com.mockrunner</groupId>
 <artifactId>mockrunner</artifactId>
 <scope>test</scope>
</dependency>
<dependency>
 <groupId>xmlunit</groupId>
 <artifactId>xmlunit</artifactId>
 <scope>test</scope>
</dependency>
<dependency>
 <groupId>org.easymock</groupId>
 <artifactId>easymock</artifactId>
 <scope>test</scope>
</dependency>

Online Tests

Often a test requires some external resource such as a database or a server to operate. Such tests should never assume that resource will be available and should skip test execution, rather than fail, when the test is not available.

JUnit4 provides a handy way to do this with the org.junit.Asssume class. Methods of the class are called from a @Before hook or from a test method. For example consider the common case of connecting to a database:

public class MyTest {

    Connection connect() {
        //create a connection to the database
        try {
           Conection cx = ...
           return cx;
        }
        catch(Exception e) {
           LOGGER.log(Level.WARNING, "Connection failed", e);
           return null;
        }
    }

    @Before
    public void testConnection() {
        Connection cx = connect();
        org.junit.Assume.assumeNotNull(cx);
        cx.close();
    }

    @Test
    public void test1() {
        // test something
    }
}

In the above example the assumeNotNull method will throw back an exception telling JUnit to skip execution of the test.