All Articles

Testing Spring WebClient in Kotlin, with JUnit or Integration Tests

HTTP
Sometimes, you need information from other sources through http

In this previous article, we have introduced the WebClient as a way of making request to third party applications. However, we did not add tests. This will be covered in this article.

There are three ways of testing (in Unit tests) that the WebClient is working:

  • Unit Tests with a mocked WebClient
  • Integration Tests against a MockServer
  • Integration Tests against the actual API (which is strongly discouraged, as we do not want tests to be dependant on other systems!)

This article will cover both the UnitTest option using JUnit and MockK, and also integration tests using @SpringBootTest with a MockWebServer.

Setup

First, here is a recap of the setup we are using. The API is the following.

data class CryptoAssetDto(
    val id: String,
    val symbol: String,
    val name: String,
    @JsonProperty("current_price")
    val price: Double,
    @JsonProperty("market_cap_rank")
    val marketCapRank: Int
)

The WebClient is configured as follows:

@Configuration
class WebClientConfiguration {

    @Bean
    fun coinGeckoWebClient(@Value("\${service.coingecko.base-url}") baseUrl: String): WebClient {
        return WebClient.builder()
            .baseUrl(baseUrl)
            .clientConnector(ReactorClientHttpConnector(httpClient()))
            .defaultHeader(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE)
            .defaultHeader(HttpHeaders.ACCEPT, "${MediaType.APPLICATION_JSON}")
            .defaultHeader(HttpHeaders.ACCEPT_CHARSET, StandardCharsets.UTF_8.toString())
            .build()
    }

    @Bean
    fun httpClient(): HttpClient {
        return HttpClient.create()
            .option(ChannelOption.CONNECT_TIMEOUT_MILLIS, 5000)
            .responseTimeout(Duration.ofMillis(5000))
            .doOnConnected { conn ->
                conn.addHandlerLast(ReadTimeoutHandler(5000, TimeUnit.MILLISECONDS))
                    .addHandlerLast(WriteTimeoutHandler(5000, TimeUnit.MILLISECONDS))
            }
    }

The WebClient is injected into the gateway which coordinates the call:

@Component
class CoinGeckoGateway @Inject constructor(private val coinGeckoWebClient: WebClient) {

    fun loadGreatestCoins(page: Int): Flux<CryptoAsset> {

        return coinGeckoWebClient.get().uri { uriBuilder ->
            uriBuilder
                .path("/coins/markets")
                .queryParam("vs_currency", "usd")
                .queryParam("order", "market_cap_desc")
                .queryParam("per_page", "250")
                .queryParam("sparkline", "false")
                .queryParam("page", page)
                .build()
        }
            .retrieve()
            .bodyToFlux(CryptoAssetDto::class.java)
            .map { it.toDomain() }
    }
}

For the response, I have created a Fixture that will return a response containing the objects we require:

class CryptoAssetDtoFixture {
    companion object {
        fun cryptoAssetDto(
            id: String = "BTC",
            symbol: String = "BTC",
            name: String = "Bitcoin",
            price: Double = 10.0,
            marketCapRank: Int = 1
        ): CryptoAssetDto {
            return CryptoAssetDto(
                id = id,
                name = name,
                symbol = symbol,
                price = price,
                marketCapRank = marketCapRank
            )
        }

        fun cryptoList(): List<CryptoAssetDto> {
            return listOf(
                cryptoAssetDto(),
                cryptoAssetDto(id = "ETH", symbol = "ETH", marketCapRank = 2, price = 5.0, name = "Ethereum"),
                cryptoAssetDto(id = "BNB", symbol = "BNB", marketCapRank = 2, price = 3.0, name = "Binance Coin")
            )
        }
    }
}

Unit Test

As I’ve stated in the article about testing, the preferred way of testing should always be unit tests, with the respective mocks handling everything from third party classes.

There are limitations to this however. In fact, our gateway uses the WebClient, and if this is mocked, every single object that follows has to be mocked as well. This would look as follows:

@ExtendWith(MockKExtension::class)
internal class CoinGeckoGatewayTest {

    @MockK
    lateinit var webClient: WebClient

    @MockK
    lateinit var requestBodyUriSpec: WebClient.RequestBodyUriSpec

    @MockK
    lateinit var requestBodySpec: WebClient.RequestBodySpec

    @MockK
    lateinit var responseSpec: WebClient.ResponseSpec

    @InjectMockKs
    lateinit var coinGeckoGateway: CoinGeckoGateway

    @Test
    fun loadGreatestCoins() {
        every { webClient.get() } returns requestBodyUriSpec
        every { requestBodyUriSpec.uri(any<java.util.function.Function<UriBuilder, URI>>()) } returns requestBodySpec
        every { requestBodySpec.retrieve() } returns responseSpec
        every { responseSpec.bodyToFlux(CryptoAssetDto::class.java) } returns Flux.fromIterable(cryptoList())

        val greatestCoins = coinGeckoGateway.loadGreatestCoins(0)
        val coinsAsList = greatestCoins.collectList().block()

        assertAll(
            { assertThat(coinsAsList!!.size).isEqualTo(3) },
            { assertThat(coinsAsList!![0]).isInstanceOf(CryptoAsset::class.java) },
            { assertThat(coinsAsList!![0].id).isEqualTo("BTC") }
        )
    }
}

Now, this is fairly ugly. One could even state that we are not really testing anything. In fact, we are really only testing these two items:

  • the WebClient is actually called (otherwise MockK would throw an exception for functions unnecessarily mocked)
  • The Mapping at the end is correct

Also, note that some items are not even easy to mock. In fact, the .uri() method is expecting a lambda, so we cannot even simply use any(), but instead need to define the class in much more detail. Simply using any() will result in an ambiguous declaration, and the test will not run.

However, for the WebClient, since everything is mocked, we do not really verify that the query parameters are set, that the call is actually made, or anything that we’d really like to verify.

So, how could we test this more elegantly?

Integration Test

The response is to have actual integration tests. Now, we’d like to avoid integration tests that make the actual call to the real API, as we do not want to make useless requests, or have failing tests if that API is down.

However, we could make use of integration tests where we have our own mocked WebServer. Spring would instantiate a temporary WebServer against which our gateway and the WebClient could make the request.

In order to do so, we first need to add some imports, as this is not included in the standard Spring jars.

Imports

<dependency>
    <groupId>com.squareup.okhttp3</groupId>
    <artifactId>mockwebserver</artifactId>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>com.squareup.okhttp3</groupId>
    <artifactId>okhttp</artifactId>
    <scope>test</scope>
</dependency>

Test

First of all, here is the test, before I go into a bit more detail what it all means:

@SpringBootTest
@ActiveProfiles("test")
internal class CoinGeckoGatewayIntegrationTest {

    private lateinit var server: MockWebServer

    @Inject
    private lateinit var coinGeckoGateway: CoinGeckoGateway

    private val objectMapper = ObjectMapper()

    @BeforeEach
    fun setUp() {
        server = MockWebServer()
        server.start(12345)
    }

    @Test
    fun loadGreatestCoins() {
        val cryptos = cryptoList()
        server.enqueue(
            MockResponse().setBody(
                objectMapper.writeValueAsString(cryptos)
            )
                .setResponseCode(200)
                .setHeader("Content-Type", "application/json")
        )
        val greatestCoins = coinGeckoGateway.loadGreatestCoins(1)

        val coinsAsList = greatestCoins.collectList().block()
        val request = server.takeRequest()

        assertAll(
            { assertThat(coinsAsList!!.size).isEqualTo(3) },
            { assertThat(coinsAsList!![0]).isInstanceOf(CryptoAsset::class.java) },
            { assertThat(coinsAsList!![0].id).isEqualTo("BTC") },
            { assertThat(request.getHeader(HttpHeaders.CONTENT_TYPE)).isEqualTo(MediaType.APPLICATION_JSON_VALUE) },
            { assertThat(request.getHeader(HttpHeaders.ACCEPT)).isEqualTo("${MediaType.APPLICATION_JSON}") },
            { assertThat(request.getHeader(HttpHeaders.ACCEPT_CHARSET)).isEqualTo(StandardCharsets.UTF_8.toString()) },
            { assertThat(request.requestUrl.toString()).isEqualTo("http://localhost:12345/coins/markets?vs_currency=usd&order=market_cap_desc&per_page=250&sparkline=false&page=1") }
        )
    }

    @AfterEach
    fun tearDown() {
        server.shutdown()
    }
}

Setup

In the Setup, we define all that’s required for the Test to run successfully. This entails the following items:

  • Injecting the actual Gateway
  • Setting up the MockWebServer on a defined port
  • Shutting down the MockWebServer after running the test
    private lateinit var server: MockWebServer

    @Inject
    private lateinit var coinGeckoGateway: CoinGeckoGateway

    @BeforeEach
    fun setUp() {
        server = MockWebServer()
        server.start(12345)
    }
    
    @AfterEach
    fun tearDown() {
        server.shutdown()
    }

Additionally, the application-test.yaml contains the value for the baseUrl:

service:
  coingecko:
    base-url: http://localhost:12345

Mocking the response

After the setup, we would already have a running MockWebServer that could serve requests. However, it would not know what to return on the endpoint, and so we’d end up with a ReadTimeoutException.

In order to avoid the Exception, we can now create the Response we’d like to have from the server:

        val cryptos = cryptoList()
        server.enqueue(
            MockResponse().setBody(
                objectMapper.writeValueAsString(cryptos)
            )
                .setResponseCode(200)
                .setHeader("Content-Type", "application/json")
        )

The response can be much more detailed, but this is pretty much everything that’s required. By default, it would return status 200, but I like to explicitly state the status, as there may be some specific handling for statuses that are still successful, but may be special, such as 207.

Actual call and Assertions

Finally, we can execute the code that would make the actual request. Actually, would is incorrect, as it really does make the request. It’s just against our own defined WebServer.

Since this call is now using the actual WebClient, the full configuration of it will be correctly set, and we can make all the assertions for headers, path, URL, etc.

We can also still make the same assertions for the mapping of the response as we would in the previous Unit Test. However, the amount of more useful assertions that can be covered by the integration test makes this a much more viable option!

val greatestCoins = coinGeckoGateway.loadGreatestCoins(1)

        val coinsAsList = greatestCoins.collectList().block()
        val request = server.takeRequest()

        assertAll(
            { assertThat(coinsAsList!!.size).isEqualTo(3) },
            { assertThat(coinsAsList!![0]).isInstanceOf(CryptoAsset::class.java) },
            { assertThat(coinsAsList!![0].id).isEqualTo("BTC") },
            { assertThat(request.getHeader(HttpHeaders.CONTENT_TYPE)).isEqualTo(MediaType.APPLICATION_JSON_VALUE) },
            { assertThat(request.getHeader(HttpHeaders.ACCEPT)).isEqualTo("${MediaType.APPLICATION_JSON}") },
            { assertThat(request.getHeader(HttpHeaders.ACCEPT_CHARSET)).isEqualTo(StandardCharsets.UTF_8.toString()) },
            { assertThat(request.requestUrl.toString()).isEqualTo("http://localhost:12345/coins/markets?vs_currency=usd&order=market_cap_desc&per_page=250&sparkline=false&page=1") }
        )

I am aware that people may think that this is quite some work for testing outgoing API calls. However, with RestTemplate going into maintenance mode, more and more people/projects will need to be aware of these testing options.

As you see now, integration tests can be extremely useful. They may make the build of the application a bit slower, but in a case such as here, this tradeoff is well worth it!