The Testcontainers extension does not integrate with Spring in a way that ensures the container starts early enough for Spring and overrides the Spring properties (e.g., spring.data.jdbc.url).

While searching for a solution, I came across several articles that seemed somewhat outdated. Therefore, I wanted to share my solution here. The articles I found were:

• Bootstrap project with Spring Boot, Kotest, Testcontainers & MongoDB | Jakub Prądzyński's Blog

• Using Testcontainers with Micronaut and Kotest | @akobor

• TestContainers Extention and Spring boot startup timing · Issue #1649 · kotest/kotest

What all of those solutions have in common is that they wrap the container and then control its lifecycle. This could be achieved through extension functions, static methods, or an abstract test superclass. Having all the lifecycle callbacks, you can then start and stop the container before the Spring application context starts and override the properties.

Although these approaches work, they seemed unnecessarily complex to me.

Starting point

What I started with was a common Spring JUnit test with Testcontainers:

@ActiveProfiles("default", "test")
@Testcontainers
@SpringBootTest
class SomeIntegrationTest() {
    companion object {
        @Container
        private val postgres = PostgreSQLContainer("postgres:16.1")

        @JvmStatic
        @DynamicPropertySource
        fun properties(registry: DynamicPropertyRegistry) {
            with(registry) {
                add("spring.flyway.url") { postgres.jdbcUrl }
                add("spring.flyway.user") { postgres.username }
                add("spring.flyway.password") { postgres.password }
                add("spring.r2dbc.url") {
                    postgres.jdbcUrl.replaceFirst("jdbc", "r2dbc")
                }
                add("spring.r2dbc.username") { postgres.username }
                add("spring.r2dbc.password") { postgres.password }
            }
        }
    }
// setup, teardown and tests go here
...
}

While I could keep the @SpringBootTest annotation due to the Kotest extension @Testcontainers , which helps with the container lifecycle, does not work with Kotest.

The Kotest version

What my predecessors did not have at their time was the @ServiceConnection annotation. You can read more about it here. Given a standard Postgres container, it should be possible to use this annotation. So, I experimented and rewrote the test like this:

import io.kotest.extensions.testcontainers.perSpec

@ActiveProfiles("default", "test")
@SpringBootTest
class SomeIntegrationTest : StringSpec() {

    companion object {
        @ServiceConnection
        private val postgres = PostgreSQLContainer("postgres:16.1")
            .apply { this.start() }
    }

    init {
        listener(postgres.perSpec())
        // setup, teardown and tests go here
        ...
    }
}

I also encountered the issue where the container started too late for Spring. To address this, I start the container explicitly in the companion object. Registering the container as a listener using perSpec() then managed the rest of the lifecycle.

With spring-boot-testcontainers in the classpath, there’s no need to override properties manually anymore. We can just add the @ServiceConnection annotation.

Conclusion

So, all in all, instead of having to write our own lifecycle wrapper we just use perSpec() and the manual start with .apply { this.start() } and have a nicely working integration test. The @ServiceConnection annotation could also have been applied to the original test, but it’s great to see that it works seamlessly with the Kotest Spring extension.

What makes those matchers special for me is that they are my first contribution to the Kotest project. So, “achievement unlocked”, I could say.

The YAML matchers currently cover two aspects:

• valid YAML

• equal YAML

and for those, you have four methods:

• shouldBeValidYaml

• shouldNotBeValidYaml

• shouldEqualYaml

• shouldNotEqualYaml

The usage is quite simple. All of the methods are extension functions on String. Therefore, when importing from io.kotest.assertions.yaml you can call those on any String. E.g. when checking for valid YAML:

"""
foo: bar
""".shouldBeValidYaml()

Equally simple, when comparing YAML (notice the infix notation):

"""
key: value
""" shouldEqualYaml "key: value"

Hopefully, this will help others improve their tests. Additionally, I’d be happy to add more YAML matchers if others have suggestions. Lastly, the whole feature is multiplatform ready, so you can use it in your KMP project.

Implementation details

If anyone is interested, I can go more into the details. It is pretty simple actually. The base for the matchers is the KAML library. It is used for parsing the YAML. Parsing the YAML is already everything that is needed for checking the validity of the YAML. If KAML throws an exception we know that the YAML isn’t valid and the check fails (or passed if you check for not valid).

When checking for equality, we also start with parsing the YAML. Then, we compare the contents of the YAML, again relying on KAML.

And that’s all of the magic in the background.

Multiplatform and Java 8

What had been quite a journey for me was the multiplatform and Java 8 compatibility. Since Kotest supports all multiplatform targets until tier 3 I tried to also cover those targets (I could’ve chosen an easier way but that was my personal challenge). I then found out that KAML does not support all multiplatform targets. Changing this was a quite easy change.

Unfortunately, SnakeYAML Engine KMP, on which KAML is based, did not support Java 8. Therefore, it was incompatible with Kotest. In order to have Java 8 compatibility there, I had to go one level deeper (insert random Inception GIF here).

SnakeYAML uses an urlencoder and this one was the last link in the chain. So, in order to finish my YAML matchers, I had to start here. So I adjusted the urlencoder, in order to modify SnakeYAML, to have update it’s version in KAML. Finally, I could implement the YAML matchers with full multiplatform support.

Of course, I didn’t do everything by myself. I received much help by the maintainers of the respective projects. I'd like to add that everyone was super helpful. It was quite a very nice open source experience.

Although it took longer than expected and more hops than ever expected, I’m glad that I could finish this feature.

Beginning in December, I had the chance to participate in a Bertelsmann scholarship. I was allowed to start the Udacity nanodegree "AI Programming with Python" for free. Last month, I finished the nanodegree successfully, and therefore I wanted to write a short review about the course.

2. Udacity Overview

If you do not know Udacity, let me give you a short overview. Udacity provides online classes similar to Coursera or Udemy. As far as I know, most of the classes cover technical or management content. What makes Udacity unique is that they provide so-called "nanodegrees." Basically, this means the classes are more in-depth, and your projects will be checked by humans.

If you want to know more about nanodegrees, check out this blog post: Nanodegree 101: What is a Nanodegree Program? | Udacity

3. Description of the Course

As I already mentioned, I chose "AI Programming with Python" as nanodegree. At first I thought that I could easily finish the class by Spring but I have to admit that I really needed nearly the whole time we were given.

The nanodegree is split into seven courses:

1. Introduction to AI programming

2. Introduction to Python for AI Programmers

3. Numpy, Pandas, Matplotlib

4. Linear Algebra Essentials

5. Calculus Essentials

6. Neural Networks - AI Programming with Python

7. Create Your Own Image Classifier (the final project)

As you can see, the course starts with basic Python and progresses to more advanced libraries. Then there is a big part covering the math behind neural networks. Lastly, you finally program your own neural network.

To summarize, the nanodegree consists of math and Python. 😂

4. Course Content Review

I am not a Python developer, but I had tried some Python before. Therefore, the language basics covered at the beginning were quite easy for me. The parts that covered the libraries more in-depth (e.g., Pandas and Pytorch) were more complicated but manageable. Most of the programming took place inside the browser using a virtual machine that either ran a simple text editor or a Jupyter notebook. For the short exercises and the first project, this was sufficient. For the final project, I decided to use my local IDE and GitHub, which is also supported.

The courses covered the math basics, which was okay. The math behind neural networks was more challenging for me. It had been a while since I had my last math lecture. Still, especially the videos provided by 3Blue1Brown were easy to understand.

Lastly, the final project also took some time to finish. I had to look up previous lessons and read the Pytorch documentation to find what I needed. I suppose for somebody who is not a developer, this must be much harder.

5. Assessment & Feedback

I was surprised by the detailed assessment when I received the feedback for my first project. I thought that some automated checks would run over the code (e.g., linting and unit tests) after turning it in. Instead, I received detailed feedback from a human reviewer who mentioned errors but also suggested optional improvements. For the second and larger project, the feedback was as good as before.

In addition to the "official" project feedback, a Udacity community was provided. In my case, it consisted only of other Bertelsmann scholarship recipients and Udacity mentors/moderators. I was skeptical at first but later used it as a quick way to ask questions about the projects or things unclear from the classes. The mentors usually responded within a day and solved all my issues.

6. Outcome & Impact

So far, I have not been able to put my newly learned skills to use. Nevertheless, knowing how a neural network works internally and knowing that one can take already trained networks to adjust them to other use-cases helps me keep my eyes open for opportunities. Additionally, since AI is a very common topic these days, it helps me distinguish what's possible and what's not.

Besides the main topic, I gained a lot of experience with Jupyter notebooks. This might be useful in the future, especially since Kotlin notebooks are now available as well.

Repeating a lot of math wasn't my favorite part of the class, but I'm sure that the repetition won't hurt 😅

7. Summary & Recommendation

All in all, the nanodegree was quite challenging, but I am happy that I finished it. If you consider taking this class, you should be aware that it is not a general AI overview. It covers only neural networks, but those in-depth. If this is interesting for you or you plan to use a neural network soon, then this class might be well suited for you.

What I cannot judge is the pricing of the nanodegree, since I received a scholarship.

Without getting too much into details, I would like to sketch my use case. I had to consume a single topic which contained a lot of different user events. Based on a single type of event I had to collect all of them belonging to the same "resource" (e.g. a webpage). Having identified those, I had to bring them into a time-based order and see if a continuous interaction happened. If the interaction reached a certain threshold (e.g. the user spent a certain time on a webpage) I had to forward a single event but no more. I tried to sketch it in the following picture:

This is the event stream for a single user. The events arrive out of order and are mixed. I am only interested in the square events. Additionally, I have to wait until at least two of the same colour arrive, that's my threshold. More than two yellow squares should not lead to more output.

Since the requirement involved some grouping, my initial idea was to use a state store, but...

StateStore is primarily local

You can use a StateStore to store data and compare it e.g. to other incoming events. The stores use a Kafka Topic (a topic with the changelog postfix) for fault tolerance and recreating the store in case of a restart (see also the good explanation here). But although this topic is replicated in the Kafka cluster, it's being filled primarily locally and stored on local disk. I started with this code:

        streamsBuilder
            .addStateStore(
                Stores.keyValueStoreBuilder(
                    Stores.persistentKeyValueStore(STATE_STORE_NAME),
                    Serdes.String(),
                    valueSerde
                )
            )
            .stream<String, GenericRecord>(topic)
            .filter { _, value -> ... }
            .process(::MyProcessor, STATE_STORE_NAME)

But in my case it didn't work because I needed a global view of all the stored events. Therefore, a state store was the wrong choice. But, it wasn't only due to the local StateStore that this was problematic, but also due to my source. Before we talk about this, a note on load.

If you filter incorrectly, you might end up duplicating your source topic

As already mentioned, my application consumed a rather large topic and because I used the StateStore wrongly, we ended up nearly duplicating the incoming messages (the grouping that was part of the requirement did not work). The wrong grouping wasn't simply a coding error, there were integration tests for this. It was due to a logical error because I lacked Kafka knowledge and was directly connected to the next problem (message order). How could you avoid this? Well, I would say there is no general solution. Testing more thoroughly in Preprod would have at least avoided some problems in Prod.

Check your source's message order/key

This was a big beginner's mistake. When Kafka distributes messages across partitions it does so based on the key. I knew this when we started but somehow assumed that the source topic uses a key so that all relevant messages arrive in the order I need (in this case, all messages for one user arrive in order in the same node). Assuming this was a huge mistake. Our source used random keys, i.e. all messages were distributed randomly across all replicas. The providing team's goal was to distribute the load evenly, not group messages.

After finding out about the upstream topic's key, there was no way this would change. The other team wouldn't and couldn't change their partitioning. Repartitioning with Kafka Streams is not a problem. We can use a group-by for this:

        streamsBuilder
            .stream<String, GenericRecord>(topic)
            .filter { _, value -> ... }
            .groupBy({ _, value -> """${value.some}${value.thing}""" }, Grouped.`as`("group-by-something"))

Still, ordering messages is problematic. Simply repartitioning does not bring the messages in order.

Ordering out-of-order messages is hard

For my use case, I had two choices to get the messages back in order:

1. Store the messages locally in a StateStore and push them out in order from time to time (see ReorderIntegrationTest in this PR)

2. Use a session window to group together events based on their timestamp

Luckily, the session window perfectly fit my use case:

        streamsBuilder
            // add the TimeStampExtractor
            .stream<String, GenericRecord>(topic, Consumed.with(timestampExtractor))
            .filter { _, value -> ... }
            .groupBy({ _, value -> """${value.some}${value.thing}""" }, Grouped.`as`("group-by-something"))
            .windowedBy(
                // Define the session window according to the use-case
                SessionWindows.ofInactivityGapAndGrace(
                    Duration.ofSeconds(eventIntervalSeconds),
                    Duration.ofSeconds(gracePeriodSeconds)
                )
            )
            // aggregate all related events in some intermediate format
            .aggregate(
                // initial
                ::AggregationRoot,
                // aggregation
                { _, value, aggregationRoot->
                    AggregationRoot(
                        aggregationRoot.events + value
                    )
                },
                // merge
                { _, left, right->
                    left + right
                },
                Materialized.with(Serdes.String(), JsonSerde(...))
            )
            // only pass on finished sessions, not intermediate results
            .suppress(untilWindowCloses(unbounded()))

Session windows end when a message for a new window arrives

This was a little bit tricky to find out but could be replicated in an integration test. A session window consists of all events that are no more than a certain duration apart. When using session windows you can provide a TimestampExtractor so that Kafka Streams knows which timestamp to use for session assignment. But to know when a session ends, one event has to arrive which starts a new session. This is due to the fact that Kafka Streams doesn't use a wall clock internally but only relies on the event timestamps.

Now that I had a way to group and aggregate my events, I ran into another problem.

Message size during aggregation

When aggregating session windows and collecting all messages the session might grow considerably large. Setting max.request.size can help but considering that a session can grow as large as your typical user interaction, it is better to find a way to reduce the amount of messages somehow in either the aggregation or the merging. In my case, the merge of two sessions contained a lot of duplicates, so it helped to use a set.

Final thoughts

All the points I mentioned here were the problems that I encountered. Besides that, I want to state that after having fixed those, the Kafka Streams application runs smoothly and has a great performance. Also, the fluent stream definition is quite readable. Therefore, don't be afraid to use Kafka Streams if the use case fits.

Initially, I was planning to give you some specific examples of things that do not work. Like this annotation or that one. But while doing my research I came to a different conclusion. It is not about specifics that do not work well with Coroutines. I came to a general conclusion:

Expect that anything that uses annotation magic does not work

There might be exceptions, since I am pretty sure that I do not know every single annotation, but regarding the ones I use most often, nothing works with Coroutines out of the box.

Let's take a look at some examples.

Cacheable

If you want to annotate a suspend function with @Cacheable you will be quite disappointed. Due to the fact that the Kotlin compiler will modify the method signature and add a Continuation parameter. Basically, the Continuation will mess with the caching interceptor. There is a pretty good SoF answer regarding this.

Workaround

The general solution for all incompatibilities presented here is to go back to the basics, i.e., we code manually what the annotations do for us automagically. In the case of the @Cacheable annotation we must use the CacheManager ourselves. E.g., when using Caffeine, we need a CacheManager bean:

    @Bean
    fun cacheManager(caffeine: Caffeine<Any, Any>): CacheManager =
        CaffeineCacheManager().apply {
            setCaffeine(Caffeine.newBuilder().maximumSize(10_000))
        }

And then we can use the Bean to create Caches when needed:

    private val cache: Cache = cacheManager.getCache("cacheName")!!

Lastly, we check presence and insert into the cache when necessary:

        return if (cache[key] != null) {
            cache.get(key, MyClass::class.java)
        } else {
            ...
            cache.put(key, myClass)
        }

Additionally, as you might have read in the SoF answer, you could also fall back to using a Deferred as return value. But using a Deferred here might bring other problems since we are no longer invoking a method but rather starting a job.

Transactional

Currently, if you do not happen to use the R2DB driver, there is no ReactiveTransactionManager in your application autoconfigured. This means that reactive methods, like suspend functions, cannot be used with the @Transactional annotation.

As far as I know, it is problematic to stretch a transaction across several threads. And in case of suspend functions, you might switch the thread several times.

Workaround

If you want to stick to suspend functions, you will have to use the TransactionTemplate. You should be able to inject an org.springframework.transaction.PlatformTransactionManager and then create a TransactionTemplate:

val transactionTemplate = TransactionTemplate(platformTransactionManager)

You can then call the executeWithoutResult or execute methods. But be careful. I did not test this. You should avoid calling other suspend functions without runBlocking. The way I see it, everything you do within one of those two execute functions should be non-reactive due to the reason mentioned above.

CircuitBreaker

As you can see from this issue (effective 11.10.2022) there is no support for the @CircuitBreaker annotation. Nevertheless, Resilience4J supports Kotlin Coroutines.

Workaround

As you can read in the documentation you may execute or decorate the suspend functions. Something like this should work:

    private suspend fun <T> makeCall(yourLambda: suspend () -> T): T =
            circuitBreaker.executeSuspendFunction {
                timeLimiter.executeSuspendFunction(yourLambda)
            }

FeignClient

If you are a fan of the declarative FeignClient I have to disappoint you, too. There is currently no support for reactive methods.

Workaround

There is a community project called "Feign Reactive" which you can find here. It allows you to use Mono or Flux as return types. You could then transform those into Coroutines by using the extension methods from kotlinx-coroutines-reactor like await().

Bean

Not even the @Bean annotation works with suspend functions. Somehow, the inserted Continuation also messes with Spring's ConstructorResolver.

Okay, this is quite an artificial example since there should be no suspend functions involved when creating a bean. Still, it shows that there are some problems left to be solved.

Final remarks

As always with new technologies the surrounding ecosystem must catch up. Though, Kotlin and its Coroutines aren't that new, a framework as big a Spring cannot change quickly.

There was a project that tried to overcome the limitations but it hasn't received any updates in a while (https://github.com/konrad-kaminski/spring-kotlin-coroutine). So, I suppose it is EOL.

Personally, I hope that with Spring 6 and Spring Boot 3 there will be better support out-of-the-box for all the things I have shown in this article. We will know more at the end of the year.

Today, I want to present to you the book "Effective Kotlin" by Marcin Moskala. A friend and former colleague of mine recommended it to me (thank you Patrick). I must admit that I was skeptical after looking at the book before ordering it. Some time ago I stopped reading books that are solely on a "code" level. Programming languages, their compilers or even what is considered best practice (looking at you Scala 2) change so rapidly these days that such books are obsolete a year or two after publication. There are some evergreens of course, e.g., Java Concurrency in Practice, but those are rare in my opinion. "Effective Kotlin" was published in November 2019. According to my definition this means it is already old (for comparison November 2019 Kotlin 1.3.60 was released, today we use Kotlin 1.7.10).

Still, since it was a recommendation, I gave it a try.

The strucuture

The book contains different levels of recommendations. Some cover basics that are not related to Kotlin (e.g., "Design for Readability") and others are very specific details (e.g., "Avoid member extensions"). Additionally, chapters covering general aspects are labeled for better orientation. The book is separated into three parts that center around a certain aspect. Within each part there are several chapters. The chapters within each part get more complicated. This helps when one has to look up some specific advice or searches for a code example.

The content

As already mentioned, the information varies from OOP best practices to very Kotlin specific aspects. Every chapter contains code examples illustrating what is being presented. The examples are easy to understand since they are written in a readable way. I read the printed edition; therefore, the code is well formatted. Some online reviews mention that the formatting is not good when using an e-book reader.

The code is mostly unrelated to a specific platform. From time-to-time specific aspects are pointed out that become relevant when one is not using Kotlin on the JVM. Additionally, some examples show Android code. Nevertheless, one does not need to know the Android platform.

How did the book help me during my daily work?

What I will try sometime soon is to introduce more inline value classes (the ones with the JvmInline annotation and the value keyword). Additionally, what I pay more attention to after having read the book is the usage of sequences when chaining many map, flatMap, filter, etc. transformations.

But the real value is not that I learned this certain pattern or that method, it is more that I have a reference book to check when I get the feeling that there should be a way to write some better code. Each time I get a feeling that there should be a better way I can grab the book and quickly check.

Conclusion

All in all, I have to admit that the book is still relevant these days and that I learned quite a lot. I am not sure if it will become one of those evergreen books, but I would recommend you give it a try.

If Marcin Moskala ever reads this article, here is my recommendation for the next edition: Add an alphabetical index to the end of the book. It would be much easier to search for certain things by looking up a keyword. Actually, that this is missing, is the biggest weakness of the book.

Using a non-reactive database

Basically, I use the same example as in the previous article. There is a controller, a repository, and an entity. To better highlight the difference, I also introduced a service class.

The repository

The repository is as expected:

interface EntityRepository: JpaRepository<TestEntity, UUID>

A simple JpaRepository. No additional methods required.

The controller

Basically, the controller stayed the same:

@RestController
class TestController(private val service: TestService) {

    @GetMapping(path = ["/entity/{id}"])
    suspend fun getEntity(@PathVariable id: UUID) = service.findById(id)

}

The only difference is that we use the service and do not call the repository directly.

The service

Let's start with the way we shouldn't do it:

@Service
class TestService(private val repository: EntityRepository) {
    suspend fun findById(id: UUID): TestEntity? = repository.findById(id).orElse(null)
}

If we try this our code would compile and our tests would pass. So, what's the problem you ask? IntelliJ shows a warning:

"Thread starvation", interesting, but what does this mean? We can find a good definition in the Java SE documentation:

Starvation describes a situation where a thread is unable to gain regular access to shared resources and is unable to make progress

Our coroutine calls a blocking part of the code. Because it has to block and wait for the (database) response the thread cannot perform other work. If the database never answers, the thread is always occupied and the thread-pool, where our thread originated from, lost one thread. What's even worse is that we lose our coroutine advantage. Coroutines should be non-blocking. When they wait for something, they are supposed to return the thread and wait in the background (simply put).

Let's take a look at the better solution.

IntelliJ already told us what we have to do. After applying the quick fix our service looks like this:

@Service
class TestService(private val repository: EntityRepository) {
    suspend fun findById(id: UUID): TestEntity? {
        return withContext(Dispatchers.IO) {
            repository.findById(id)
        }.orElse(null)
    }
}

But what does this do? We can add some logging (I like to use io.github.microutils:kotlin-logging) to get some insights. The important part here is the thread name. Therefore, I had to adjust the logback layout a little.

When we use the non-optimal approach, we'll see this output in the service's findById() method:

2022-07-14 | 06:26:09.967 | reactor-http-nio-4 @coroutine#1 |  INFO | d.c.c.TestService | Hello from findById

The important part is the third column. Our code is running in a reactor thread on coroutine number one. When using the suggested solution we get the following output:

2022-07-14 | 06:25:05.536 | reactor-http-nio-4 @coroutine#1 |  INFO | d.c.c.TestService | Hello from findById outside context
2022-07-14 | 06:25:05.544 | DefaultDispatcher-worker-1 @coroutine#1 |  INFO | d.c.c.TestService | Hello from findById inside context

As we can see, by switching the context, we switched the threadpool and the coroutine. By calling withContext we "returned" the Reactor coroutine. It is now free to handle other incoming HTTP requests. Our blocking code now uses a thread from the IO Dispatcher whose purpose it is to handle blocking calls. You may wonder why the DefaultDispatcher appears though we used Dispatchers.IO. We can find the solution in the Dispatcher.IO Javadoc:

This dispatcher and its views share threads with the Default dispatcher, [...]

Now that we use the new context our tests will still pass. Nevertheless, we now have a coroutine optimal solution.

Conclusion

Our main conclusion is that we should do what IntelliJ demands :-D JetBrains, being the company behind IntelliJ and Kotlin obviously knows what works best and what doesn't. Besides that, we can see that switching from non-blocking to blocking code is not that difficult with coroutines within Spring. Finally, stay tuned for my next article featuring coroutines.

As usual, you can find the complete example on GitHub.

I had a problem, so I decided to use threads. tNwoowp rIo bhlaevmes.

Still, it is surprisingly easy to get started with Coroutines in Spring. When using Webflux, we get Coroutine support. One disclaimer about Coroutines has to be mentioned here: Coroutines won't make our code automagically thread safe. We still have to consider possible concurrency issues. Anyways, working with Coroutines, the code looks more familiar than when working e.g., with Webflux/Reactor because the code looks like sequential code and is not cluttered with all the map(), flatMap(), switchIfEmpty().... calls.

To encourage the usage of Coroutines I decided to write some small blog posts, highlighting my experiences. Additionally, in my projects, I could see a substantial performance boost after switching to Coroutines. In this part we will cover the easiest case, staying reactive from the controller to the repository.

Reactive from the controller to the repository

The easiest way to get started with Coroutines in Spring is when you have spring-boot-starter-webflux in your classpath and your database driver is also reactive. In that case most of the work is just adding the suspend keyword. But let's take a look at an example.

The RestController

The controller is nothing special, it just needs the suspend keyword:

@RestController
class TestController(private val repository: EntityRepository) {

    @GetMapping(path = ["/entity/{id}"])
    suspend fun getEntity(@PathVariable id: String) = repository.findById(ObjectId(id))

}

When you set a breakpoint in the method you can see that Spring, using Reactor, started a Coroutine for us. We do not have to handle any contexts.

To keep the example short, I did not create a Service class. So next, let's take a look at the repository.

The repository

The repository looks slightly different than usual:

interface EntityRepository: CoroutineCrudRepository<TestEntity, ObjectId>

Spring introduced a CoroutineCrudRepository some time ago, in which all methods are suspend functions. In my example I use the reactive MongoDB driver. Therefore, everything is reactive by default and I do not have to add any additional configuration. Furthermore, my code looks like the usual sequential code one is used to read.

Conclusion

As you can see from the code, we hardly recognize that we are using Coroutines (still, keep in mind that you code must be thread-safe!). Of course, the example is very simple and nowhere close to the complexity of a real-world project. Still, it is a starting point. In the following article we will see how we can bridge the gap to a non-reactive database driver.

You can find the complete example on GitHub.

The ChuckNorrisFact service

The complete example can be found on GitHub.
You might have seen me using the Chuck Norris fact API in a previous blog post. The API will serve us as an example for another service that our implementation depends on.
We have a simple ChuckNorrisFactController as the API for manual testing. Next to the “business” classes there is the ChuckNorrisService that does the call to the external API. It uses Spring’s RestTemplate. Nothing special.
What I have seen many times are tests that mock the RestTemplate and return some pre-canned answer. The implementation could look like this:

@Service
public class ChuckNorrisService{
...
  public ChuckNorrisFact retrieveFact() {
    ResponseEntity<ChuckNorrisFactResponse> response = restTemplate.getForEntity(url, ChuckNorrisFactResponse.class);
    return Optional.ofNullable(response.getBody()).map(ChuckNorrisFactResponse::getFact).orElse(BACKUP_FACT);
  }
 ...
 }

Next to the usual unit tests checking for the success cases there would be at least one test covering the error case, i.e. a 4xx or 5xx status code:

  @Test
  public void shouldReturnBackupFactInCaseOfError() {
    String url = "http://localhost:8080";
    RestTemplate mockTemplate = mock(RestTemplate.class);
    ResponseEntity<ChuckNorrisFactResponse> responseEntity = new ResponseEntity<>(HttpStatus.SERVICE_UNAVAILABLE);
    when(mockTemplate.getForEntity(url, ChuckNorrisFactResponse.class)).thenReturn(responseEntity);
    var service = new ChuckNorrisService(mockTemplate, url);

    ChuckNorrisFact retrieved = service.retrieveFact();

    assertThat(retrieved).isEqualTo(ChuckNorrisService.BACKUP_FACT);
  }

Doesn’t look bad, right? The response entity returns a 503 error code and our service will not crash. All tests are green and we can deploy our application.
Unfortunately, Spring’s RestTemplate does not work like this. The method signature of getForEntity gives us a very small hint. It states throws RestClientException. And this is where the mocked RestTemplate differs from the actual implementation. We will never receive a ResponseEntity with a 4xx or 5xx status code. The RestTemplate will throw a subclass of RestClientException. Looking at the class hierarchy we can get a good impression of what could be thrown:

Therefore, lets see how we can make this test better.

WireMock to the rescue

WireMock simulates web services by starting a mock server and returning answers you configured it to return. It is easy to integrate into your tests and mocking requests is also simple thanks to a nice DSL.
For JUnit 4 there is a WireMockRule that helps with starting an stopping the server. For JUnit 5 you will have to do it yourself. When you check the example project you can find the ChuckNorrisServiceIntegrationTest. It is a SpringBoot test based on JUnit 4. Let’s take a look at it.
The most important part is the ClassRule:

      @ClassRule  public static WireMockRule wireMockRule = new WireMockRule();

As mentioned before, this will start and stop the WireMock server. You could also use the rule as normal Rule to start and stop the server for each test. For our test this isn’t necessary.
Next, you can see several configureWireMockFor... methods. These contain the instructions for WireMock when to return what answer. Splitting the WireMock configuration into several methods and calling them from the tests is my approach to using WireMock. Of course you could set up all possbile requests in an @Before method. For the success case we do:

  public void configureWireMockForOkResponse(ChuckNorrisFact fact) throws JsonProcessingException {
    ChuckNorrisFactResponse chuckNorrisFactResponse = new ChuckNorrisFactResponse("success", fact);
    stubFor(get(urlEqualTo("/jokes/random"))
        .willReturn(okJson(OBJECT_MAPPER.writeValueAsString(chuckNorrisFactResponse))));
  }

All methods are imported statically from com.github.tomakehurst.wiremock.client.WireMock. As you can see, we stub an HTTP GET to a path /jokes/random and return a JSON object. The okJson() method is just shorthand for a 200 response with JSON content. For the error case the code is even more simple:

  private void configureWireMockForErrorResponse() {
    stubFor(get(urlEqualTo("/jokes/random"))
        .willReturn(serverError()));
  }

As you can see, the DSL makes it easy to read the instructions.
Having WireMock in place we can see that our previous implementation does not work since the RestTemplate throws an exception. Therefore, we gotta adjust our code:

  public ChuckNorrisFact retrieveFact() {
    try {
      ResponseEntity<ChuckNorrisFactResponse> response = restTemplate.getForEntity(url, ChuckNorrisFactResponse.class);
      return Optional.ofNullable(response.getBody()).map(ChuckNorrisFactResponse::getFact).orElse(BACKUP_FACT);
    } catch (HttpStatusCodeException e){
      return BACKUP_FACT;
    }
  }

This already covers WireMock’s basic use-cases. Configure an answer for a request, execute the test, check the results. It’s as simple as that.
Still, there is one problem you will usually encounter when you run your tests in a cloud environment. Let’s see what we can do.

WireMock on a dynamic port

You might have noticed that the integration test in the project contains an ApplicationContextInitializer class and that its @TestPropertySource annotation overwrites the URL of the actual API. That is because I wanted to start WireMock on a random port. Of course you can configure a fixed port for WireMock and use this one as hard-coded value in your tests. But if your tests are running on some cloud providers infrastructure you cannot be sure that the port is free. Therefore, I think a random port is better.
Still, when using properties in a Spring application we have to pass the random port somehow to our service. Or, as you can see in the example, overwrite the URL. That is why we use the ApplicationContextInitializer. We add the dynamically assigned port to the application context and then we can refer to it by using the property ${wiremock.port}. The only disadvantage here is that we now have to use a ClassRule. Else, we couldn’t access the port before the Spring application is being initialized.
Having solved this problem, let’s take a look at one common problem when it comes to HTTP calls.

Timeouts

WireMock offers many more possibilities for responses than just simple answers to GET requests. Another test case that is often forgotten is testing timeouts. Developers tend to forget to set timeouts on the RestTemplate or even on URLConnections. Without timeouts both will wait for an infinite amount of time for responses. In the best case you will not notice, in the worst case all your threads wait for a response that will never arrive.
Therefore, we should add a test that simulates a timeout. Of course, we can also create a delay with e.g. a Mockito mock, but in that case we would guess again how the RestTemplate behaves. Simulating a delay with WireMock is pretty easy:

  private void configureWireMockForSlowResponse() throws JsonProcessingException {
    ChuckNorrisFactResponse chuckNorrisFactResponse = new ChuckNorrisFactResponse("success", new ChuckNorrisFact(1L, ""));
    stubFor(get(urlEqualTo("/jokes/random"))
        .willReturn(
            okJson(OBJECT_MAPPER.writeValueAsString(chuckNorrisFactResponse))
                .withFixedDelay((int) Duration.ofSeconds(10L).toMillis())));
  }

withFixedDelay() expects an int value representing milliseconds. I prefer using Duration or at least a constant that indicates that the parameter represents milliseconds without having to read the JavaDoc every time.
After setting a timeout on our RestTemplate and adding the test for the slow response we can see that the RestTemplate throws a ResourceAccessException. So we can either adjust the catch block to catch this exception and the HttpStatusCodeException or just catch the superclass of both:

  public ChuckNorrisFact retrieveFact() {
    try {
      ResponseEntity<ChuckNorrisFactResponse> response = restTemplate.getForEntity(url, ChuckNorrisFactResponse.class);
      return Optional.ofNullable(response.getBody()).map(ChuckNorrisFactResponse::getFact).orElse(BACKUP_FACT);
    } catch (RestClientException e){
      return BACKUP_FACT;
    }
  }

Now we have nicely covered the most common cases when doing HTTP requests and we can be sure that we are testing close to real world conditions.

Why not Hoverfly?

Another choice for HTTP integration tests is Hoverfly. It works similar to WireMock but I have come to prefer the latter. The reason is that WireMock is also quite useful when running end-to-end tests that include a browser. Hoverfly (at least the Java library) is limited by using JVM proxies. This might make it faster than WireMock but when e.g. some JavaScript code comes into play it does not work at all. The fact that WireMock starts a webserver is very useful when your browser code also calls some other services directly. You can then mock those with WireMock, too, and write e.g. your Selenium tests.

Conclusion

I hope this article could show you two things:

1. the importance of integration tests
2. that WireMock is pretty nice

Of course, both topics could fill many more articles. Still, I wanted to give you a feeling of how to use WireMock and what it is capable of. Feel free to check their documentation and try many more things. As an example, testing authentication with WireMock is also possible.

I have recently finished the book "Programming Beyond Practices" by Gregory T. Brown and want to give you a short book review.

Despite the title containing the word "programming" the book does not contain any code at all and this totally fits the intention of the book. In eigth chapters the author shows us things we have to take care of next to writing code. The subtitle "Be more than just a code monkey" emphasizes this even more.

When I received the book I was surprised that it is relatively short. Having round about 120 pages it is one of the shortest books I own. Initially, I thought that the short size would be a disadvantage. Nevertheless, after having finished the book, I think it is an advantage. One can easily read it again and again without having to expect two weeks of reading. Just doing a recap of a certain aspect of the book or of all chapters can be done quickly. Still, the chapters contain all of the information necessary and I never felt that the author missed something or kept a chapter artificially short. The only time I wanted to read more was the second last chapter where I wanted to know how the company described would proceed. But that was just my interest in the well written story.

This leads me to the style of writing. The author chose a good way to share his knowledge: every chapter describes a short story of an imaginary project, software, lecture, etc., which contains some interwoven dialogs. These stories make the book easy to read and make the learnings tangible. The chapters all start with a short, general introduction and finish with a summary. Additionally, the author added some questions and exercises to the chapters to force the reader to think a little bit more about the topic presented. Although I did not really do the exercises, I think they can be of great use if one reads this book with others.

Lastly, the author added three riddles to the book. I could not find the solution to any of them but if anyone managed to solve them, please share the solution with me. Next to that some additional materials can be found on the authors website.

All in all I can recommend the book and suggest you to read it, too. The price might seem a little bit high for the number of pages but regarding the knowledge it contains it is definitely worth it.

And what did I learn? Being someone who loves his job for the technical aspects I will try to concentrate more on the problem-solving aspect in the future. Especially with regards to the people involved in the project/software.