Fabio Maffioletti | An alternative approach on using MockRestServiceServer and WireMock in a Spring Boot application

28 Sep 2016

An alternative approach on using MockRestServiceServer and WireMock in a Spring Boot application

Suppose you are developing a Spring Boot client application that needs to communicate with a remote server API. You are for sure going to write a communication layer (let's call it Repository) between your application and the server, and you are going to test it in some way. So you need to simulate the server behavior i.e. responses and you have some ways to do this in a unit test:

you can use Mockito to mock the RestTemplate
you can use Mockito to mock the Repositories, even if in this case you are not calling the RestTemplate so you overtake some of the application logic

But if you want to load the Spring context and build a sort of integration test, you have these alternatives:

use the @MockBean annotation to mock the Repository response, but also in this case you are not calling the RestTemplate and thus overtaking some of the application logic
start a mock server like WireMock or MockServer and mock the responses there, even if this causes problems if you need to refresh the Spring context, so you need to manage the start/stop/restart phases by yourself
use the Spring's MockRestServiceServer class

I decided to exclude the Mockito approach because I wanted to build an integration test and I had time constraints so I could not spend much time in mocking the RestTemplate responses. So I went for the MockServer solution, also because I wanted to be able to launch the client application on my local machine and be able to open webpages and see some real mock data inside. So, basically I used the same approach both for my test and local profiles. The problem with this is that I needed to have two different configurations for the same bean representing the remote server, and they both had to stay in the src/main/java directory to be able to also run the application on the local machine, for example:

@Component("remoteServer")
@Profile("!test")
public class RealRemoteServer {

    @Value("${remote.server.base.url}")
    private String remoteBaseURL;

    @PostConstruct
    public void init() {
        log.info("Remote server at " + remoteBaseURL)
        // nothing else to do here
    }
	
}

@Component("remoteServer")
@Profile("test")
public class MockRemoteServer {

    @Value("${remote.server.base.url}")
    private String remoteBaseURL;

    @PostConstruct
    public void init() {
    	log.info("Remote server at " + remoteBaseURL)
    	// start MockServer and define mock responses
    }

    @PreDestroy
    public void destroy() {
        // shut down MockServer
    }
	
}

Moreover, to be able to run the MockServer also on the local machine, I needed to have it as a non test scoped dependency in my pom file. This had to be changed, and this post is about the solution I found.

The MockRestServiceServer way

I created this project as an example, so we have two profiles test and production, a repository supposed to communicate with a remote server API and the test class.

So, let's say that we have a ProductRepository that defines a findAll and a findOne methods, like:

package me.fabiomaffioletti.msc.repository;

import java.util.List;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.core.ParameterizedTypeReference;
import org.springframework.http.HttpMethod;
import org.springframework.http.ResponseEntity;
import org.springframework.stereotype.Repository;
import org.springframework.web.client.RestTemplate;

import lombok.RequiredArgsConstructor;
import lombok.extern.log4j.Log4j;
import me.fabiomaffioletti.msc.util.HttpEntityBuilder;
import me.fabiomaffioletti.msc.dto.ProductDTO;

@Log4j
@RequiredArgsConstructor(onConstructor = @__(@Autowired))
@Repository
public class ProductRepositoryImpl implements ProductRepository {

    private final RestTemplate restTemplate;

    @Value("${remote.server.uri.products}")
    private String productsURI;

    @Override
    public ResponseEntity<List<ProductDTO>> findAll() {
        HttpEntityBuilder<Void> httpEntityBuilder = new HttpEntityBuilder<>();
        return restTemplate.exchange(productsURI, HttpMethod.GET, httpEntityBuilder.build(), new ParameterizedTypeReference<List<ProductDTO>>() {});
    }

    @Override
    public ResponseEntity<ProductDTO> findOne(Long id) {
        HttpEntityBuilder<Void> httpEntityBuilder = new HttpEntityBuilder<>();
        return restTemplate.exchange(productsURI + "/{id}", HttpMethod.GET, httpEntityBuilder.build(), ProductDTO.class, id);
    }

}

And the related test class made like this:

package me.fabiomaffioletti.msc.repository;

import java.util.List;

import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.test.context.junit4.SpringRunner;

import me.fabiomaffioletti.msc.MockserverCondensedTestConfiguration;
import me.fabiomaffioletti.msc.dto.ProductDTO;

import static org.hamcrest.Matchers.equalTo;
import static org.hamcrest.Matchers.hasSize;
import static org.hamcrest.Matchers.is;
import static org.junit.Assert.assertThat;

@RunWith(SpringRunner.class)
@MockserverCondensedTestConfiguration
public class ProductRepositoryTest {

    @Autowired
    private ProductRepository productRepository;

    @Test
    public void testItShouldRetrieveTheListOfProducts() {
        ResponseEntity<List<ProductDTO>> responseEntity = productRepository.findAll();
        assertThat(responseEntity.getStatusCode(), is(HttpStatus.OK));
        assertThat(responseEntity.getBody(), hasSize(3));
    }

    @Test
    public void testItShouldRetrieveAProductWithTheGivenId() {
        ResponseEntity<ProductDTO> responseEntity = productRepository.findOne(1L);
        assertThat(responseEntity.getStatusCode(), is(HttpStatus.OK));
        assertThat(responseEntity.getBody().getId(), is(1L));
        assertThat(responseEntity.getBody().getName(), is(equalTo("product one")));
    }

}

Having @MockserverCondensedTestConfiguration specifying the test intent:

package me.fabiomaffioletti.msc;

import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.ActiveProfiles;

@SpringBootTest
@ActiveProfiles("test")
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.TYPE)
public @interface MockserverCondensedTestConfiguration {

}

While in my application.properties file I have defined the remote server endpoints, with a the base url as a variable defined in each profiled properties file:

remote.server.uri.products=${remote.server.base.url}/products

# in application-test.properties
remote.server.base.url=http://localhost:9000

# in application-prod.properties
remote.server.base.url=http://remote-server-real-url

So now, if you try to run the tests, you will get this error:

org.springframework.web.client.ResourceAccessException: I/O error on GET request for "http://localhost:9000/products": Connection refused; nested exception is java.net.ConnectException: Connection refused

That's because there is nothing, real nor mock, listening to that endpoint, and responding with something. To mock the response it is possible to use the MockRestServiceServer class, contained in the Spring test dependencies. Let's see an example and how I decided to change the approach of using it in a particular case: the ProductRepositoryTest class becomes like this one:

package me.fabiomaffioletti.msc.repository;

import java.io.IOException;
import java.util.List;

import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.http.HttpMethod;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.test.context.junit4.SpringRunner;
import org.springframework.test.web.client.MockRestServiceServer;
import org.springframework.test.web.client.response.DefaultResponseCreator;
import org.springframework.web.client.RestTemplate;

import me.fabiomaffioletti.msc.MockserverCondensedTestConfiguration;
import me.fabiomaffioletti.msc.dto.ProductDTO;

import static java.nio.file.Files.readAllBytes;
import static java.nio.file.Paths.get;
import static org.hamcrest.Matchers.equalTo;
import static org.hamcrest.Matchers.hasSize;
import static org.hamcrest.Matchers.is;
import static org.junit.Assert.assertThat;
import static org.springframework.test.web.client.ExpectedCount.times;
import static org.springframework.test.web.client.match.MockRestRequestMatchers.method;
import static org.springframework.test.web.client.match.MockRestRequestMatchers.requestTo;
import static org.springframework.test.web.client.response.MockRestResponseCreators.withStatus;

@RunWith(SpringRunner.class)
@MockserverCondensedTestConfiguration
public class ProductRepositoryTest {

    @Autowired
    private ProductRepository productRepository;

    @Autowired
    private RestTemplate restTemplate;

    @Value("${remote.server.uri.products}")
    private String productsURI;

    private MockRestServiceServer mockRestServiceServer;

    @Before
    public void setUp() {
        mockRestServiceServer = MockRestServiceServer.bindTo(restTemplate).build();
    }

    @Test
    public void testItShouldRetrieveTheListOfProducts() throws IOException {
        byte[] responseBody = readAllBytes(get("src", "test", "resources", "mock", "response", "remote.server.response.products.findAll.200.json"));
        DefaultResponseCreator responseCreator = withStatus(HttpStatus.OK).body(responseBody).contentType(MediaType.APPLICATION_JSON_UTF8);
        mockRestServiceServer.expect(times(1), requestTo(productsURI)).andExpect(method(HttpMethod.GET)).andRespond(responseCreator);

        ResponseEntity<List<ProductDTO>> responseEntity = productRepository.findAll();
        assertThat(responseEntity.getStatusCode(), is(HttpStatus.OK));
        assertThat(responseEntity.getBody(), hasSize(3));

        mockRestServiceServer.verify();
    }

    @Test
    public void testItShouldRetrieveAProductWithTheGivenId() throws IOException {
        byte[] responseBody = readAllBytes(get("src", "test", "resources", "mock", "response", "remote.server.response.products.findOne.1.200.json"));
        DefaultResponseCreator responseCreator = withStatus(HttpStatus.OK).body(responseBody).contentType(MediaType.APPLICATION_JSON_UTF8);
        mockRestServiceServer.expect(times(1), requestTo(productsURI + "/1")).andExpect(method(HttpMethod.GET)).andRespond(responseCreator);

        responseBody = readAllBytes(get("src", "test", "resources", "mock", "response", "remote.server.response.products.findOne.2.200.json"));
        responseCreator = withStatus(HttpStatus.OK).body(responseBody).contentType(MediaType.APPLICATION_JSON_UTF8);
        mockRestServiceServer.expect(times(1), requestTo(productsURI + "/2")).andExpect(method(HttpMethod.GET)).andRespond(responseCreator);

        ResponseEntity<ProductDTO> responseEntity = productRepository.findOne(1L);
        assertThat(responseEntity.getStatusCode(), is(HttpStatus.OK));
        assertThat(responseEntity.getBody().getId(), is(1L));
        assertThat(responseEntity.getBody().getName(), is(equalTo("product one")));

        responseEntity = productRepository.findOne(2L);
        assertThat(responseEntity.getStatusCode(), is(HttpStatus.OK));
        assertThat(responseEntity.getBody().getId(), is(2L));
        assertThat(responseEntity.getBody().getName(), is(equalTo("product two")));

        mockRestServiceServer.verify();
    }

}

So for each test, you need to define the mock server expectations, and optionally you can define how many times the remote endpoint gets called in the single test. Tests are now green. This is already a big step ahead compared to the previous approach. But this is an example project, it contains just one repository and one test class with two tests. In my use case I needed to answer these two questions:

what if I need to introduce this approach in an existing application containing several repositories and many test classes?
what if, in addition to repositories, I need to test the upper application layer, i.e. services, which use repositories? And controllers which use services which use repositories?

It became clear that I could not change each test and set expectations in each test. So I came up with a different approach.

The MockRestServiceServer alternative way

Basically I wanted to define the expectations in just one place, like it happened with the MockServer. So I created this class, in the src/test/java directory, that is only active when using the test profile. Moreover I added this class to the one picked up by Spring when loading the context, so the @Bean is configured. Here are the resulting classes:

package me.fabiomaffioletti.msc;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Profile;
import org.springframework.http.HttpMethod;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.test.web.client.MockRestServiceServer;
import org.springframework.test.web.client.response.DefaultResponseCreator;
import org.springframework.web.client.RestTemplate;

import lombok.extern.log4j.Log4j;

import static java.nio.file.Files.readAllBytes;
import static java.nio.file.Paths.get;
import static org.springframework.test.web.client.ExpectedCount.times;
import static org.springframework.test.web.client.match.MockRestRequestMatchers.method;
import static org.springframework.test.web.client.match.MockRestRequestMatchers.requestTo;
import static org.springframework.test.web.client.response.MockRestResponseCreators.withStatus;

@Log4j
@Profile("test")
public class MockRemoteServerConfiguration {

    @Autowired
    private RestTemplate restTemplate;

    @Value("${remote.server.uri.products}")
    private String productsURI;

    @Bean
    public MockRestServiceServer mockRestServiceServer() throws Exception {
        MockRestServiceServer mockRestServiceServer = MockRestServiceServer.bindTo(restTemplate).build();

        byte[] responseBody = readAllBytes(get("src", "test", "resources", "mock", "response", "remote.server.response.products.findAll.200.json"));
        DefaultResponseCreator responseCreator = withStatus(HttpStatus.OK).body(responseBody).contentType(MediaType.APPLICATION_JSON_UTF8);
        mockRestServiceServer.expect(times(1), requestTo(productsURI)).andExpect(method(HttpMethod.GET)).andRespond(responseCreator);

        responseBody = readAllBytes(get("src", "test", "resources", "mock", "response", "remote.server.response.products.findOne.1.200.json"));
        responseCreator = withStatus(HttpStatus.OK).body(responseBody).contentType(MediaType.APPLICATION_JSON_UTF8);
        mockRestServiceServer.expect(times(1), requestTo(productsURI + "/1")).andExpect(method(HttpMethod.GET)).andRespond(responseCreator);

        responseBody = readAllBytes(get("src", "test", "resources", "mock", "response", "remote.server.response.products.findOne.2.200.json"));
        responseCreator = withStatus(HttpStatus.OK).body(responseBody).contentType(MediaType.APPLICATION_JSON_UTF8);
        mockRestServiceServer.expect(times(1), requestTo(productsURI + "/2")).andExpect(method(HttpMethod.GET)).andRespond(responseCreator);

        return mockRestServiceServer;
    }

}

package me.fabiomaffioletti.msc;

import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.ActiveProfiles;

@SpringBootTest(classes = {MockserverCondensedApplication.class, MockRemoteServerConfiguration.class})
@ActiveProfiles("test")
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.TYPE)
public @interface MockserverCondensedTestConfiguration {

}

Of course I also removed the MockRestServiceServer stuff from the test class, so it is exactly as the beginning. This means that there is no need to change the tests already written previously, which, as I wrote before, could be hundreds or thousands. The resulting test class becomes:

package me.fabiomaffioletti.msc.repository;

import java.io.IOException;
import java.util.List;

import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.test.context.junit4.SpringRunner;

import me.fabiomaffioletti.msc.MockserverCondensedTestConfiguration;
import me.fabiomaffioletti.msc.dto.ProductDTO;

import static org.hamcrest.Matchers.equalTo;
import static org.hamcrest.Matchers.hasSize;
import static org.hamcrest.Matchers.is;
import static org.junit.Assert.assertThat;

@RunWith(SpringRunner.class)
@MockserverCondensedTestConfiguration
public class ProductRepositoryTest {

    @Autowired
    private ProductRepository productRepository;

    @Test
    public void testItShouldRetrieveTheListOfProducts() throws IOException {
        ResponseEntity<List<ProductDTO>> responseEntity = productRepository.findAll();
        assertThat(responseEntity.getStatusCode(), is(HttpStatus.OK));
        assertThat(responseEntity.getBody(), hasSize(3));
    }

    @Test
    public void testItShouldRetrieveAProductWithTheGivenId() throws IOException {
        ResponseEntity<ProductDTO> responseEntity = productRepository.findOne(1L);
        assertThat(responseEntity.getStatusCode(), is(HttpStatus.OK));
        assertThat(responseEntity.getBody().getId(), is(1L));
        assertThat(responseEntity.getBody().getName(), is(equalTo("product one")));

        responseEntity = productRepository.findOne(2L);
        assertThat(responseEntity.getStatusCode(), is(HttpStatus.OK));
        assertThat(responseEntity.getBody().getId(), is(2L));
        assertThat(responseEntity.getBody().getName(), is(equalTo("product two")));
    }

}

Now, if you run the tests, you will... get an error, like this:

java.lang.AssertionError: Request URI 
Expected :http://localhost:9000/products
Actual   :http://localhost:9000/products/1

This happend because the MockRestServiceServer expectations have an order, that is following the creation and addition order specified in the MockRemoteServerConfiguration definition. You can pass an argument to the MockRestServiceServer build method, to specify which "expectation manager" you want to use. Spring gives you two: SimpleRequestExpectationManager which is the deafult one, and UnorderedRequestExpectationManager. Both count the number of expected calls to the mock server expectation and try to match it with the actual number of calls happened during the test. The difference between the two is that UnorderedRequestExpectationManager ignores the declaration order.

So, if you know how many calls are made to the mock server in the whole test suite, you can try to use UnorderedRequestExpectationManager and specify, for each expectation, the number of expected calls, in the times() method. Unfortunately this is not my case because I don't know how many calls are made to each endpoint and in which order they are made, so I made my own implementation of the AbstractRequestExpectationManager which is actually an extension of the UnorderedRequestExpectationManager.

package me.fabiomaffioletti.msc;

import org.springframework.test.web.client.UnorderedRequestExpectationManager;

public class NoResetRequestExpectationManager extends UnorderedRequestExpectationManager {

    @Override
    public void reset() {
        // do not reset or clear the expectation list
    }

}

As you see, if you compare it with the UnorderedRequestExpectationManager reset method, this new one does not do anything, so basically it it saying that we don't care about the order of the calls and we don't care about the number of calls executed. The last things to do are:

changing the declaration of the MockRestServiceServer to build with the NoResetRequestExpectationManager
changing the times(x) to min(1) for each expectation, so we grant that the minimum number of calls to each endpoint is 1

Here we go with the final class:

package me.fabiomaffioletti.msc;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Profile;
import org.springframework.http.HttpMethod;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.test.web.client.MockRestServiceServer;
import org.springframework.test.web.client.response.DefaultResponseCreator;
import org.springframework.web.client.RestTemplate;

import lombok.extern.log4j.Log4j;

import static java.nio.file.Files.readAllBytes;
import static java.nio.file.Paths.get;
import static org.springframework.test.web.client.ExpectedCount.min;
import static org.springframework.test.web.client.match.MockRestRequestMatchers.method;
import static org.springframework.test.web.client.match.MockRestRequestMatchers.requestTo;
import static org.springframework.test.web.client.response.MockRestResponseCreators.withStatus;

@Log4j
@Profile("test")
public class MockRemoteServerConfiguration {

    @Autowired
    private RestTemplate restTemplate;

    @Value("${remote.server.uri.products}")
    private String productsURI;

    @Bean
    public MockRestServiceServer mockRestServiceServer() throws Exception {
        MockRestServiceServer mockRestServiceServer = MockRestServiceServer.bindTo(restTemplate).build(new NoResetRequestExpectationManager());

        byte[] responseBody = readAllBytes(get("src", "test", "resources", "mock", "response", "remote.server.response.products.findAll.200.json"));
        DefaultResponseCreator responseCreator = withStatus(HttpStatus.OK).body(responseBody).contentType(MediaType.APPLICATION_JSON_UTF8);
        mockRestServiceServer.expect(min(1), requestTo(productsURI)).andExpect(method(HttpMethod.GET)).andRespond(responseCreator);

        responseBody = readAllBytes(get("src", "test", "resources", "mock", "response", "remote.server.response.products.findOne.1.200.json"));
        responseCreator = withStatus(HttpStatus.OK).body(responseBody).contentType(MediaType.APPLICATION_JSON_UTF8);
        mockRestServiceServer.expect(min(1), requestTo(productsURI + "/1")).andExpect(method(HttpMethod.GET)).andRespond(responseCreator);

        responseBody = readAllBytes(get("src", "test", "resources", "mock", "response", "remote.server.response.products.findOne.2.200.json"));
        responseCreator = withStatus(HttpStatus.OK).body(responseBody).contentType(MediaType.APPLICATION_JSON_UTF8);
        mockRestServiceServer.expect(min(1), requestTo(productsURI + "/2")).andExpect(method(HttpMethod.GET)).andRespond(responseCreator);

        return mockRestServiceServer;
    }

}

Tests are now green. Here is the structure of the project at this point:

What we got:

we moved away from the src/main/java directory everything related to tests
we don't have any external server running
we used a Spring component, which is always a good idea in a Spring's application

But we still have a problem, i.e. running the application on the local machine, with mock data, so avoiding the use of a real instance of the remote server. For this I will use WireMock in a docker container.

Using WireMock for mocking responses on the local machine

First thing to do is to pull the WireMock docker image: docker pull rodolpheche/wiremock. Then, of course, we need to create a local profile configuration (application-local.properties), that could be something like:

remote.server.base.url=http://localhost:1888

This means that we intend to run the WireMock on localhost and port 1888. That's why, from the project root, we need to run the docker image with this command:

docker run -d -v $PWD/stub:/home/wiremock --name mockserver -p 1888:8080 -p 1881:8081 rodolpheche/wiremock

Now if you go to http://localhost:1888/__admin/ you should see an empty mappings json. And in your project root directory there will be a stub directory containing mappings and __files: the first one will hold the mappings definitions, the second one the response bodies. For example, the mapping file for the findAll products endpoint could be something like this:

{
    "request": {
        "method": "GET",
        "urlPath": "/products"
    },
    "response": {
        "status": 200,
        "bodyFileName": "remote.server.response.products.findAll.200.json",
        "headers": {
            "Content-Type": "application/json"
        }
    }
}

And the response, in the __files directory, something like:

[
  {
    "id": 1,
    "name": "product one"
  },
  {
    "id": 2,
    "name": "product two"
  },
  {
    "id": 3,
    "name": "product three"
  },
  {
    "id": 4,
    "name": "product four"
  },
  {
    "id": 5,
    "name": "product five"
  }
]

Now, if the docker is restarted with docker restart mockserver and you navigate to http://localhost:1888/__admin/, you should see all the mappings defined. How to test that everything will be ok when the application will be running with the local profile? We need a controller to do that (let me skip the service layer and autowire the repository directly):

package me.fabiomaffioletti.msc.controller;

import java.util.List;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RestController;

import lombok.RequiredArgsConstructor;
import lombok.extern.log4j.Log4j;
import me.fabiomaffioletti.msc.dto.ProductDTO;
import me.fabiomaffioletti.msc.repository.ProductRepository;

@Log4j
@RequiredArgsConstructor(onConstructor = @__(@Autowired))
@RestController
public class ProductController {

    private final ProductRepository productRepository;

    @GetMapping("/products")
    public ResponseEntity<List<ProductDTO>> findAll() {
        return productRepository.findAll();
    }

    @GetMapping("/products/{id}")
    public ResponseEntity<ProductDTO> findOne(@PathVariable Long id) {
        return productRepository.findOne(id);
    }

}

If you start the application and navigate to http://localhost:8080/products you should see the response coming from the WireMock server on Docker. Here is the resulting structure:

Resources

You can find the example project here