Tuesday, November 19, 2019

Better integration tests with WireMock

No matter if you follow the classical test pyramid or one of the newer approaches like the Testing Honeycomb you should start writing integration tests at some point during development.
There are different types of integration tests you can write. Starting with persistence tests, you can check the interaction between your components or you can simulate calling external services. This article will be about the latter case.
Let us start with a motivating example before talking about WireMock.

The ChuckNorrisFact service

The complete example can be found on GitHub.
You might have seen me using the Chuck Norris fact API in a previous blog post. The API will serve us as an example for another service that our implementation depends on.
We have a simple ChuckNorrisFactController as the API for manual testing. Next to the “business” classes there is the ChuckNorrisService that does the call to the external API. It uses Spring’s RestTemplate. Nothing special.
What I have seen many times are tests that mock the RestTemplate and return some pre-canned answer. The implementation could look like this:
@Service
public class ChuckNorrisService{
...
  public ChuckNorrisFact retrieveFact() {
    ResponseEntity<ChuckNorrisFactResponse> response = restTemplate.getForEntity(url, ChuckNorrisFactResponse.class);
    return Optional.ofNullable(response.getBody()).map(ChuckNorrisFactResponse::getFact).orElse(BACKUP_FACT);
  }
 ...
 }
Next to the usual unit tests checking for the success cases there would be at least one test covering the error case, i.e. a 4xx or 5xx status code:
  @Test
  public void shouldReturnBackupFactInCaseOfError() {
    String url = "http://localhost:8080";
    RestTemplate mockTemplate = mock(RestTemplate.class);
    ResponseEntity<ChuckNorrisFactResponse> responseEntity = new ResponseEntity<>(HttpStatus.SERVICE_UNAVAILABLE);
    when(mockTemplate.getForEntity(url, ChuckNorrisFactResponse.class)).thenReturn(responseEntity);
    var service = new ChuckNorrisService(mockTemplate, url);

    ChuckNorrisFact retrieved = service.retrieveFact();

    assertThat(retrieved).isEqualTo(ChuckNorrisService.BACKUP_FACT);
  }
Doesn’t look bad, right? The response entity returns a 503 error code and our service will not crash. All tests are green and we can deploy our application.
Unfortunately, Spring’s RestTemplate does not work like this. The method signature of getForEntity gives us a very small hint. It states throws RestClientException. And this is where the mocked RestTemplate differs from the actual implementation. We will never receive a ResponseEntity with a 4xx or 5xx status code. The RestTemplate will throw a subclass of RestClientException. Looking at the class hierarchy we can get a good impression of what could be thrown:
Therefore, lets see how we can make this test better.

WireMock to the rescue

WireMock simulates web services by starting a mock server and returning answers you configured it to return. It is easy to integrate into your tests and mocking requests is also simple thanks to a nice DSL.
For JUnit 4 there is a WireMockRule that helps with starting an stopping the server. For JUnit 5 you will have to do it yourself. When you check the example project you can find the ChuckNorrisServiceIntegrationTest. It is a SpringBoot test based on JUnit 4. Let’s take a look at it.
The most important part is the ClassRule:
  @ClassRule
  public static WireMockRule wireMockRule = new WireMockRule();
As mentioned before, this will start and stop the WireMock server. You could also use the rule as normal Rule to start and stop the server for each test. For our test this isn’t necessary.
Next, you can see several configureWireMockFor... methods. These contain the instructions for WireMock when to return what answer. Splitting the WireMock configuration into several methods and calling them from the tests is my approach to using WireMock. Of course you could set up all possbile requests in an @Before method. For the success case we do:
  public void configureWireMockForOkResponse(ChuckNorrisFact fact) throws JsonProcessingException {
    ChuckNorrisFactResponse chuckNorrisFactResponse = new ChuckNorrisFactResponse("success", fact);
    stubFor(get(urlEqualTo("/jokes/random"))
        .willReturn(okJson(OBJECT_MAPPER.writeValueAsString(chuckNorrisFactResponse))));
  }
All methods are imported statically from com.github.tomakehurst.wiremock.client.WireMock. As you can see, we stub an HTTP GET to a path /jokes/random and return a JSON object. The okJson() method is just shorthand for a 200 response with JSON content. For the error case the code is even more simple:
  private void configureWireMockForErrorResponse() {
    stubFor(get(urlEqualTo("/jokes/random"))
        .willReturn(serverError()));
  }
As you can see, the DSL makes it easy to read the instructions.
Having WireMock in place we can see that our previous implementation does not work since the RestTemplate throws an exception. Therefore, we gotta adjust our code:
  public ChuckNorrisFact retrieveFact() {
    try {
      ResponseEntity<ChuckNorrisFactResponse> response = restTemplate.getForEntity(url, ChuckNorrisFactResponse.class);
      return Optional.ofNullable(response.getBody()).map(ChuckNorrisFactResponse::getFact).orElse(BACKUP_FACT);
    } catch (HttpStatusCodeException e){
      return BACKUP_FACT;
    }
  }
This already covers WireMock’s basic use-cases. Configure an answer for a request, execute the test, check the results. It’s as simple as that.
Still, there is one problem you will usually encounter when you run your tests in a cloud environment. Let’s see what we can do.

WireMock on a dynamic port

You might have noticed that the integration test in the project contains an ApplicationContextInitializer class and that its @TestPropertySource annotation overwrites the URL of the actual API. That is because I wanted to start WireMock on a random port. Of course you can configure a fixed port for WireMock and use this one as hard-coded value in your tests. But if your tests are running on some cloud providers infrastructure you cannot be sure that the port is free. Therefore, I think a random port is better.
Still, when using properties in a Spring application we have to pass the random port somehow to our service. Or, as you can see in the example, overwrite the URL. That is why we use the ApplicationContextInitializer. We add the dynamically assigned port to the application context and then we can refer to it by using the property ${wiremock.port}. The only disadvantage here is that we now have to use a ClassRule. Else, we couldn’t access the port before the Spring application is being initialized.
Having solved this problem, let’s take a look at one common problem when it comes to HTTP calls.

Timeouts

WireMock offers many more possibilities for responses than just simple answers to GET requests. Another test case that is often forgotten is testing timeouts. Developers tend to forget to set timeouts on the RestTemplate or even on URLConnections. Without timeouts both will wait for an infinite amount of time for responses. In the best case you will not notice, in the worst case all your threads wait for a response that will never arrive.
Therefore, we should add a test that simulates a timeout. Of course, we can also create a delay with e.g. a Mockito mock, but in that case we would guess again how the RestTemplate behaves. Simulating a delay with WireMock is pretty easy:
  private void configureWireMockForSlowResponse() throws JsonProcessingException {
    ChuckNorrisFactResponse chuckNorrisFactResponse = new ChuckNorrisFactResponse("success", new ChuckNorrisFact(1L, ""));
    stubFor(get(urlEqualTo("/jokes/random"))
        .willReturn(
            okJson(OBJECT_MAPPER.writeValueAsString(chuckNorrisFactResponse))
                .withFixedDelay((int) Duration.ofSeconds(10L).toMillis())));
  }
withFixedDelay() expects an int value representing milliseconds. I prefer using Duration or at least a constant that indicates that the parameter represents milliseconds without having to read the JavaDoc every time.
After setting a timeout on our RestTemplate and adding the test for the slow response we can see that the RestTemplate throws a ResourceAccessException. So we can either adjust the catch block to catch this exception and the HttpStatusCodeException or just catch the superclass of both:
  public ChuckNorrisFact retrieveFact() {
    try {
      ResponseEntity<ChuckNorrisFactResponse> response = restTemplate.getForEntity(url, ChuckNorrisFactResponse.class);
      return Optional.ofNullable(response.getBody()).map(ChuckNorrisFactResponse::getFact).orElse(BACKUP_FACT);
    } catch (RestClientException e){
      return BACKUP_FACT;
    }
  }
Now we have nicely covered the most common cases when doing HTTP requests and we can be sure that we are testing close to real world conditions.

Why not Hoverfly?

Another choice for HTTP integration tests is Hoverfly. It works similar to WireMock but I have come to prefer the latter. The reason is that WireMock is also quite useful when running end-to-end tests that include a browser. Hoverfly (at least the Java library) is limited by using JVM proxies. This might make it faster than WireMock but when e.g. some JavaScript code comes into play it does not work at all. The fact that WireMock starts a webserver is very useful when your browser code also calls some other services directly. You can then mock those with WireMock, too, and write e.g. your Selenium tests.

Conclusion

I hope this article could show you two things:
  1. the importance of integration tests
  2. that WireMock is pretty nice
Of course, both topics could fill many more articles. Still, I wanted to give you a feeling of how to use WireMock and what it is capable of. Feel free to check their documentation and try many more things. As an example, testing authentication with WireMock is also possible.

Monday, June 25, 2018

Book review: Programming Beyond Practices by Gregory T. Brown


Picture by chimp CC BY 3.0 license

I have recently finished the book "Programming Beyond Practices" by Gregory T. Brown and want to give you a short book review.

Despite the title containing the word "programming" the book does not contain any code at all and this totally fits the intention of the book. In eigth chapters the author shows us things we have to take care of next to writing code. The subtitle "Be more than just a code monkey" emphasizes this even more.

When I received the book I was surprised that it is relatively short. Having round about 120 pages it is one of the shortest books I own. Initially, I thought that the short size would be a disadvantage. Nevertheless, after having finished the book, I think it is an advantage. One can easily read it again and again without having to expect two weeks of reading. Just doing a recap of a certain aspect of the book or of all chapters can be done quickly. Still, the chapters contain all of the information necessary and I never felt that the author missed something or kept a chapter artificially short. The only time I wanted to read more was the second last chapter where I wanted to know how the company described would proceed. But that was just my interest in the well written story.

This leads me to the style of writing. The author chose a good way to share his knowledge: every chapter describes a short story of an imaginary project, software, lecture, etc., which contains some interwoven dialogs. These stories make the book easy to read and make the learnings tangible. The chapters all start with a short, general introduction and finish with a summary. Additionally, the author added some questions and exercises to the chapters to force the reader to think a little bit more about the topic presented. Although I did not really do the exercises, I think they can be of great use if one reads this book with others.

Lastly, the author added three riddles to the book. I could not find the solution to any of them but if anyone managed to solve them, please share the solution with me. Next to that some additional materials can be found on the authors website.

All in all I can recommend the book and suggest you to read it, too. The price might seem a little bit high for the number of pages but regarding the knowledge it contains it is definitely worth it.

And what did I learn? Being someone who loves his job for the technical aspects I will try to concentrate more on the problem-solving aspect in the future. Especially with regards to the people involved in the project/software.

Saturday, May 19, 2018

Gatling-JDBC on Maven Central

Some time ago I wrote a blog post on the codecentric blog about how to extend Gatling. As accompanying code example I created a small library on GitHub: Gatling-JDBC

Finally, after keeping the library untouched for several months, I performed the necessary steps to publish it on Maven Central! This means you can now use de.codecentric.gatling-jdbc version 1.0.0 in your Gatling load tests. I upgraded its dependencies so that is compatible with the latest Gatling version 2.3.1.

Its usage is described in the blog post or you can take a look at the different simulations in the test directory. If you encounter any problems or would like to suggest any improvements feel free to open an issue on GitHub.

Tuesday, May 15, 2018

Improve your test structure with Lambdas and Mockito’s Answer

Refactoring is an important task that should be done from time to time. No matter if you call it technical debt or give it any other way. We sometimes implement features quick and dirty or while implementing we learn better ways how we could achieve our goal.
While refactoring is often applied to business logic or the infrastructure related to it, it seems that doing it for tests is often neglected. Therefore, you want to show you in this blog post on the codecentric blog, how lambdas and Mockito's answer can help your tests.

Tuesday, May 8, 2018

DRY in the 21st century

Due to some discussion arising regarding the DRY principle, especially regarding Microservices and the consideration to either duplicate code or create a library project, I collected my thoughts about this topic. Please find the article in the codecentric blog. If you want to add your thoughts about this topic, feel free to use the comment section there.

Tuesday, May 1, 2018

Gatling Load Testing Part 2 - Extending Gatling

Although the blog post has been published already some time ago, I would like to point out to the second article related to Gatling Load Testing. Again, published here, in the codecentric blog.
You can find the related project on GitHub.

Wednesday, November 22, 2017

Dynamic Validation with Spring Boot Validation

After it has been quite for a while, I have written a new blog post about dynamic validation with Spring Boot validation. The post has been published on the codecentric blog. You can find it here. Enjoy it!

 

Copyright @ 2013 Wrong tracks of a developer.

Designed by Templateiy