How to get Rid of Boilerplate in Integration Tests with DBUnit and Custom Annotation.

Hi! My name is Viacheslav Aksenov and I am a backend developer specializing in developing complex backend systems in Java and Kotlin. Also, I write a lot of code for myself, which you can find on my GitHub:

What is this article about

During my work, I often see the integration of a service and a database. And specifically - with testing this integration. During integration testing involving a database, each time you need to do the following:

• Set the initial state of tables
• Run a test with a specific request
• Assert response
• Assert the content of tables after passing tests

In this article, I would like to offer a solution that I applied to my last working project, which turned out to be great. And this reduced the number of boilerplates dozens of times.

An example of an experimental service

Let's look at an example based on production code. There is a Spring Boot web service written in Java. The controller layer is based on the usual implementation through the @RestController annotation Postgresql is used as the database. To integrate the service with the database, a Spring Data Jpa wrapper over JDBC is used for faster interaction.

Controller:

@PutMapping(ManagerApi.CONTACT)
@Operation(summary = "Update contact")
public ContactResponse updateContact(
   @RequestBody UpsertContactRequest request,
   @RequestParam UUID id
) {
   return contactService.update(id, request);
}

Models:

public record ContactResponse(
   UUID id,
   String name,
   String phoneNumber
) {
}

public record UpsertContactRequest(
   @NonNull String name,
   @NonNull String phoneNumber
) {
}

Database entity:

@Entity
@Table(name = "contacts", schema = "cms")
@Builder
@Setter
public class ContactEntity extends AbstractEntity {
 
   @Id
   @GeneratedValue(generator = "UUID")
   @GenericGenerator(name = "UUID", strategy = "org.hibernate.id.UUIDGenerator")
   @Column(updatable = false, nullable = false)
   protected UUID id;
  
   @Column
   private String name;
 
   @Column(name = "phone_number", length = 10)
   private String phoneNumber;
 
}

Repository:

@Repository
public interface ContactJpaRepository
   extends JpaRepository<ContactEntity, UUID> {
}

How can we test it?

JUnit + mockMvc + Sprint Test

Use repositories from the repository layer directly in tests set the state of the database in the test, generate a model for request as DTO, make a call through mockMvc, then assert the response and the state of the database using JUnit.

Test example:

@SpringBootTest
public class SimpleContactTest {

    @Autowired
    private MockMvc mockMvc;

    @Autowired
    private ContactJpaRepository contactJpaRepository;

    @Autowired
    private ObjectMapper objectMapper;

    @Test
    @SneakyThrows
    public void updateContactTest() {
        var oldEntity = ContactEntity.builder()
            .name("old name")
            .phoneNumber("0000000000")
            .build();
        var savedEntity = contactJpaRepository.save(oldEntity);
        var request = new CreateContactRequest("name", "1112223344");

        var contentAsString = mockMvc.perform(
                MockMvcRequestBuilders.put(
                        ManagerApi.CONTACT,, savedEntity.getId())
                    .header(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE)
                    .content(objectMapper.writeValueAsBytes(request)))
            .andExpect(MockMvcResultMatchers.status().isOk())
            .andReturn()
            .getResponse()
            .getContentAsString();

        var response = objectMapper.readValue(contentAsString, ContactResponse.class);

        Assertions.assertEquals(request.name(), response.name());
        Assertions.assertEquals(request.phoneNumber(), response.phoneNumber());
        Assertions.assertNotNull(response.id());

        Optional<ContactEntity> entity = contactJpaRepository.findById(response.id());
        Assertions.assertTrue(entity.isPresent());
        Assertions.assertEquals(response.name(), entity.get().getName());
        Assertions.assertEquals(response.phoneNumber(), entity.get().getPhoneNumber());
    }

}

The pros of this approach: complete transparency of what is happening, the absence of additional librariesCons: the test is simply huge. Verbosity, which grows exponentially, makes it impossible to develop quickly with a quality result. There is also a need to ensure that the test after or before its work clears the data from the database if it is static. However, with all its simplicity and clarity, this is an extremely verbose approach. If you use it in large projects, you risk wasting a lot of time.

Optimize regularly used logic - this will save you an enormous amount of time in the future.

Let’s improve it with the popular library - DBUnit.

DBUnit is an open-source framework that helps with solving problems such as filling databases, and tables, and comparing tables and datasets with a database. It is also an extension for JUnit. In contrast to the previous method - preparing the state of the database and checking for a match with the required state is performed by DBUnit.

Test example:

@DBUnit(
    caseInsensitiveStrategy = Orthography.LOWERCASE,
    batchedStatements = true,
    allowEmptyFields = true,
    schema = "cms"
)
@SpringBootTest
public class SimpleContactTest {

    @Autowired
    private MockMvc mockMvc;

    @Autowired
    private ObjectMapper objectMapper;

    @Test
    @SneakyThrows
    @DataSet(
        cleanBefore = true,
        value = "controller/contact/update.success/dataset.json"
    )
    @ExpectedDataSet("controller/contact/update.success/dataset-expected.json")
    public void updateContactTest() {
        var request = new CreateContactRequest("name", "1112223344");

        var contentAsString = mockMvc.perform(
                MockMvcRequestBuilders.put(
                        ManagerApi.CONTACT,
                        "7624f434-cbc5-11ec-9d64-0242ac120002")
                    .header(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE)
                    .content(objectMapper.writeValueAsBytes(request)))
            .andExpect(MockMvcResultMatchers.status().isOk())
            .andReturn()
            .getResponse()
            .getContentAsString();

        var response = objectMapper.readValue(contentAsString, ContactResponse.class);

        Assertions.assertEquals(request.name(), response.name());
        Assertions.assertEquals(request.phoneNumber(), response.phoneNumber());
        Assertions.assertNotNull(response.id());
    }
}

Additional files describing the state of the database:

## controller/contact/update.success/dataset.json
 
{
   "contacts": [
       {
           "id": "7624f434-cbc5-11ec-9d64-0242ac120002",
           "name": "old name",
           "phone_number": "0000000000"
       }
   ]
}
 
## controller/contact/update.success/dataset-expected.json
 
{
   "contacts": [
       {
           "id": "7624f434-cbc5-11ec-9d64-0242ac120002",
           "name": "name",
           "phone_number": "1112223344"
       }
   ]
}

As we can see, we managed to completely exclude work with the controller layer and the database from the tests. This greatly reduces the boilerplate code. Cons: There is an additional addiction that you need to learn to work with. The complexity of the library configuration for highly custom scripts. Added additional configuration files

Let’s add the custom annotation to specify the request/response

In order to refuse to describe models in java code for requests and checking responses, you can write your own annotation that will use the already described models from the file and use the @ParametrizedTest annotation.

@Target({ElementType.ANNOTATION_TYPE, ElementType.METHOD, })
@Retention(RetentionPolicy.RUNTIME)
@Documented
@ArgumentsSource(JsonFileSourceProvider.class)
public @interface JsonFileSource {
   String file() default "";
 
   String expectFile() default "";
}

A class that implements reading from a file:

public class JsonFileSourceProvider
   implements AnnotationConsumer<JsonFileSource>, ArgumentsProvider {
 
   private final List<String> resources = new ArrayList<>();
 
   @Override
   public void accept(JsonFileSource jsonFileSource) {
       addResource(jsonFileSource.file());
       addResource(jsonFileSource.expectFile());
   }
 
   @Override
   public Stream<? extends Arguments> provideArguments(ExtensionContext context) {
       return Stream.of(resources)
           .map(
               r -> r.stream()
                   .map(this::getJsonResource)
                   .toArray()
           )
           .map(Arguments::of);
   }
 
   private void addResource(String resource) {
       if (!resource.isEmpty()) {
           this.resources.add(resource);
       }
   }
 
   private String getJsonResource(String file) {
       try {
           return new String(
               Files.readAllBytes(
                   ResourceUtils.getFile(String.format("classpath:%s", file)).toPath()
               )
           );
       } catch (final IOException err) {
           return null;
       }
   }
}

With this annotation, the test starts to look a lot simpler:

@DBUnit(
   caseInsensitiveStrategy = Orthography.LOWERCASE,
   batchedStatements = true,
   allowEmptyFields = true,
   schema = "cms"
)
@SpringBootTest
public class SimpleContactTest {
 
   @Autowired
   private MockMvc mockMvc;
 
   @SneakyThrows
   @ParameterizedTest
   @DataSet(
       cleanBefore = true,
       value = "controller/contact/update.success/dataset.json"
   )
   @ExpectedDataSet("controller/contact/update.success/dataset-expected.json")
   @JsonFileSource(
       file = "controller/contact/update.success/request.json",
       expectFile = "controller/contact/update.success/response.json"
   )
   public void updateContactTest(String request, String response) {
       mockMvc.perform(
               MockMvcRequestBuilders.put(
                       ManagerApi.CONTACT, "7624f434-cbc5-11ec-9d64-0242ac120002")
                   .header(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE)
                   .content(request))
           .andExpect(MockMvcResultMatchers.status().isOk())
           .andExpect(MockMvcResultMatchers.content().json(response));
   }
}

And the entire description of the request and response models is stored in the files:

## controller/contact/update.success/request.json
 
{
  "name": "old name",
  "phone_number": "0000000000"
}
 
 
## controller/contact/update.success/response.json
 
{
  "name": "name",
  "phone_number": "1112223344"
}

The advantages of this approach are difficult to exaggerate. We see the almost complete exclusion of the boilerplate from the test body. This becomes extremely noticeable on a production project, where the number of tests is in the hundreds. The test became at least three times smaller. It also becomes well structured.Cons: Requires writing an annotation and a class to read files. And also the files themselves need to be stored in the project resources and structured well.

Conclusion:

Thus, by using the DBUnit library, as well as adding our own annotation for reading models from resource files, we managed to reduce the volume of the test body by more than three times. As well as files describing requests, service responses and the state of the database become very visual and easy to navigate.

The next step to reduce the boilerplate could be to separate mockMvc into a separate abstract class. But that's another story. I hope the tips in this article will be helpful and inspire you to improve the conditions for writing integration tests in your large and small Spring applications. Thank you for your attention!

Photo by on