How to use jOOQ with Testcontainers and, Flyway

When using a database the question is how to manage the versions of the schema and how to use the database in testing. Container technologies became a defacto standard. Why not use containers for development and your database?

Testcontainers is a Java library that supports JUnit tests, providing lightweight, throwaway instances of common databases, Selenium web browsers, or anything else that can run in a Docker container.

Sounds perfect, right? And what about database versioning?

Flyway. Version control for your database. Robust schema evolution across all environments. With ease, pleasure, and plain SQL. (Source:

Awesome! Now let’s have a look at how we can use these two technologies in combination with jOOQ to generate the Java code and test the application. The example project uses MariaDB but would work with any database. You’ll find the source code on GitHub:

The Build

The example uses Maven as the build tool. And the process is as follows

  1. Start MariaDB
    A Testcontainer with MariaDB is started using the Groovy plugin
  2. Execute Flyway migrations
    Flyway migrations are applied to the MariaDB
  3. Generate jOOQ Code
    jOOQ generates code from the MariaDB objects
  4. Compile
    Code and generated classes are compiled
  5. Run Tests
    For each test, Testcontainers starts a MariaDB container

Bellow, you can see how the Maven plugins are configured.

<!-- Start Testcontainer -->
                    db = new org.testcontainers.containers.MariaDBContainer("mariadb:10.7.3")
          'db.url', db.getJdbcUrl())
<!-- Migrate schema -->
<!-- Generate jOOQ code -->

That’s it. Easy, isn’t it?

If you are new to jOOQ don’t miss my introduction video:

Visualization of the Message Flow between Business Functions with Vaadin and Neo4j

The Project

Analyzing and visualizing the message flow between business functions was the goal of my current project. At first, we considered using a UML tool for this job, but we came to the conclusion that it might not be as flexible as we need it to be. Finally, I’ve got the assignment to create a custom web application.

Since business functions and messages are related to each other, it made sense to represent them as a graph. That’s why I chose Neo4j as the database. Now the question was how to manage and visualize the graph. As I’m experienced with the Vaadin framework I want to use it also in this project. 

Vaadin has a lot of great UI components but in my case, there was no match. Finally, I’ve found vis.js. The network diagram seemed appropriate for the visualization. Luckily Vaadin provides the Vaadin Directory, a place to publish 3rd party components. From the Vaadin directory, a component called vis-network-vaadin is available that provides a Java API on top of vis.js

The Graph

The graph below is a simplified model of what my client wants to manged in the application. A business function can send many messages and a message can be received by many business functions.

The Implementation

First I created a Vaadin project on and added the vis-network-vaadin dependency for the visualization. As Vaadin uses Spring Boot by default I could just add spring-boot-starter-data-neo4j for the data access.

Data Access

Spring Data Neo4j provides easy access to Neo4j. As I already know Spring Data JPA and the programming model are very similar it was easy to get started. First I’ve mapped the nodes and defined the relationships using the Neo4j annotations.

public class BusinessFunction {

    private Long id;

    private String nameDE;
    private String actorsDE;
    private String descriptionDE;
public class Message {

    private Long id;

    private String nameDE;
    private String descriptionDE;

    @Relationship(type = "SENDS", direction = Relationship.Direction.INCOMING)
    private Set<BusinessFunction> senders = new HashSet<>();

    @Relationship(type = "RECEIVES")
    private Set<BusinessFunction> receivers = new HashSet<>();

To read and write the data you can use repositories and make use of interface methods that will be used to generate the queries for you. Remark: I didn’t care about the performance so the generated queries were good enough in the first phase.

public interface BusinessFunctionRepository extends Neo4jRepository<BusinessFunction, Long> {

    Optional<BusinessFunction> findByNameDE(String name);

    List<BusinessFunction> findAllByNameDELike(String name, Pageable pageable);
public interface MessageRepository extends Neo4jRepository<Message, Long> {

    Optional<Message> findByNameDE(String name);

    List<Message> findAllByNameDELike(String name, Pageable pageable);


Finally, I had to visualize the graph with a network diagram. Using the vis-network-vaadin API made it quite simple. I just had to map BusinessFunction and Message to nodes and create edges from the relationships.

var networkDiagram = new NetworkDiagram(Options.builder().build());

var businessFunctionNodes = businessFunctionRepository.findAll().stream()
        .map(businessFunction -&gt; createNode("b-", businessFunction.getId(), businessFunction.getNameDE(), "DodgerBlue"))
var nodes = new ArrayList<>(businessFunctionNodes);

var messages = messageRepository.findAll();
var messageNodes =
        .map(message -&gt; createNode("m-", message.getId(), message.getNameDE(), "Orange"))


var dataProvider = new ListDataProvider<>(nodes);

var edges = new ArrayList<Edge>();
for (Message message : messages) {
    for (BusinessFunction sender : message.getSenders()) {
        edges.add(createEdge("b-", sender.getId(), "m-" + message.getId().toString(), getTranslation("sends")));
    for (BusinessFunction receiver : message.getReceivers()) {
        edges.add(createEdge("m-", message.getId(), "b-" + receiver.getId(), getTranslation("receives")));


These are the helper methods to create nodes and edges:

private Edge createEdge(String prefix, Long id, String name, String label) {
    var edge = new Edge(prefix + id.toString(), name);
    edge.setArrows(new Arrows(new ArrowHead(1, Arrows.Type.arrow)));
    return edge;

private Node createNode(String prefix, Long id, String name, String color) {
    var node = new Node(prefix + id, name);
    node.setWidthConstraint(new WidthConstraint(100, 100));
    return node;

Finally, the graph is displayed in the application.


The application is still in an early stage. The graph will be extended and the diagram must be improved. Especially the behavior when dragging around the edges seems to be quite tricky and vis.js provides a lot of configuration.

As a Java developer creating UIs with Vaadin makes it very efficient. There are even 3rd party libraries that wrap components in a Java API. On the other side, I was impressed by how easy it is to start with Neo4j and to integrate it into a Spring Boot application.

Btw. If you want to learn more about Spring Boot check my video below.

Vaadin and jOOQ: Match Made in Heaven

Have you ever had to write an application that didn’t do much other than display and change data? And did you use a Single Page Application framework like Angular, a REST API, and Hibernate?

How about if that was a lot easier?

Introducing Vaadin and jOOQ

Vaadin exists already for 20 years but was completely overhauled three years ago. Vaadin Flow is a web framework that allows the development of a web application in Java. It’s based on web components and the state is synchronized between browser and server.

What makes Vaadin a perfect match for data-centric applications are the components. We will see how simple it is to display data in a grid.

The second framework we will use is jOOQ for database access. jOOQ consists of two parts: a code generator and a DSL.

With the code generator, you can generate Java code for all database objects including tables, views, procedures, user-defined types, and many more. The DSL allows you to write type-safe, compile-time-checked SQL in Java using the generated Java classes.

Displaying a List of Data

Data Providers

To fetch data from a backend Vaadin provides data providers. Data providers are used when a component displays a collection of data. For example, in Grids or ComboBoxes.

There are two types of data providers: InMemoryDataProvider and CallbackDataProvider. An InMemoryDataProvider is used if you only have a handful of records to display whereas the CallBackDataProvider provides lazy loading and paging.

Creating the Grid

We want to create a grid to display the revenue of customers for that we use a Java Record:

public record CustomerInfo(Integer id, String lastname, String firstname, double revenue) {

This Java Record can then be used to define the type of the grid and add the columns. The addColumn is an overloaded method and we use to pass a method reference that returns the value that must be displayed in the grid cell.

Grid<CustomerInfo> grid = new Grid<>();
grid.addColumn(CustomerInfo::firstname).setHeader("First Name");
grid.addColumn(CustomerInfo::lastname).setHeader("Last Name");

Fetching the Data

As we have many customers we want to use paging. The Grid has an overloaded method setItems where we only have to pass a callback that is used to fetch the data. Internally a CallbackDataProvider is created.

grid.setItems(query -> dsl

To fetch the data we use the jOOQ DSL with the generated Java code. There are Java classes for the tables (CUSTOMER, PURCHASE_ORDER, ORDER_ITEM, and PRODUCT) that contain the columns (CUSTOMER.ID, CUSTOMER.LASTNAME, etc). Finally, we fetch the result into the CustomerInfo record.

We use offset and limit from the query object that is passed to the fetch callback method from Vaadin so the grid will have infinite scrolling and will call the callback method to fetch data when needed.

As you see in the example the code looks like SQL but due to the generated code, it is fully type-safe and is checked at compile time. If you have breaking database changes your code may no longer compile after regenerating the Java code.

And finally, this is what the grid looks like. There are many more features in grid-like filtering and sorting. You can find more information about these features and also about all other Vaadin components in the components directory.


This introduction covered only a small part of the feature set of both frameworks but I hope the example gives you an impression of how easy it is to create data-centric web applications with the combination of Vaadin and jOOQ.

If you’d like to know more check out the jOOQ documentation and tutorials and watch my Vaadin quick start tutorial to learn more about Vaadin:

Java 16 Records with JPA and jOOQ

The new Java version 16 includes a new feature: Records “Enhance the Java programming language with records, which are classes that act as transparent carriers for immutable data. Records can be thought of as nominal tuples.”

Let’s try Java records with JPA and jOOQ.

JPA Constructor Expression

One way to use projection in JPA queries is using the constructor expression. The name constructor expression implies that the constructor is called with the fields from the projection.

select new com.demo.dto.EmployeeDTO(, from Employee e

In the example we have a DTO called EmployeeDTO and the constructor takes two Strings as parameters.

With Java before Java 16 we would create a class like this:

public final class EmployeeDTO {

    private final String employeeName;
    private final String departmentName;

    public EmployeeDTO(String employeeName, String departmentName) {
        this.employeeName = employeeName;
        this.departmentName = departmentName;

    public String employeeName() {
        return employeeName;

    public String departmentName() {
        return departmentName;

    public boolean equals(Object obj) {
        if (obj == this) return true;
        if (obj == null || obj.getClass() != this.getClass()) return false;
        var that = (EmployeeDTO) obj;
        return Objects.equals(this.employeeName, that.employeeName) &&
                Objects.equals(this.departmentName, that.departmentName);

    public int hashCode() {
        return Objects.hash(employeeName, departmentName);

    public String toString() {
        return "EmployeeDTO[" +
                "employeeName=" + employeeName + ", " +
                "departmentName=" + departmentName + ']';

Thanks to Java 16 Records this is now much simpler:

public record EmployeeDTO(String employeeName, String departmentName) {

This Record will contain the required constructor and also the methods to get the employeeName and the departmentName so it’s a perfect fit for JPAs constructor expression!

jOOQ SQL Projection

Besides JPA there is another great solution for accessing relational database systems: jOOQ

With jOOQ, we can write type-safe SQL in Java. And very often we also want DTOs as a result. Also here Java Records shine:

List<EmployeeDTO> employees = dsl


Java Records are a great addition to the Java language and a great fit to use with persistence technologies like JPA or jOOQ.

If you want to try it on your own, please find the example code on GitHub:

Use the Power of your Database: XML and JSON

Today databases have a lot of functionality that is often not used by software developers because they simply do not know that this exists. But knowing about these features can save a lot of time because you may write less code.

One of these hidden gems is the ability to produce XML or JSON data using a SQL SELECT statement.
In the examples, Oracle Database is used but these features are also available in other Databases like PostgreSQL or SQL Server.


Let’s assume we have the following model and we want to have employees with their phones.


In Oracle Database producing XML would look like this.

select xmlelement(
        name "employees",
        xmlagg(xmlelement(name "employee",
            xmlattributes(EMPLOYEE.ID, EMPLOYEE.NAME),
            xmlelement(name "phones", (select xmlagg(xmlelement(name "phone",
                                              xmlattributes(PHONE.PHONENUMBER, PHONE.TYPE)))
                                       from PHONE
                                       where PHONE.EMPLOYEE_ID = EMPLOYEE.ID)))))

The result of the query will be this XML:

    <employee ID="1" NAME="Ursula Friedman">
            <phone PHONENUMBER="031 333 11 12" TYPE="WORK"/>
    <employee ID="2" NAME="Hans Boss">
            <phone PHONENUMBER="031 333 11 01" TYPE="HOME"/>
            <phone PHONENUMBER="032 311 43 12" TYPE="WORK"/>


For sure this also works for JSON in Oracle Database.

			KEY 'id' value EMPLOYEE.ID,
			KEY 'name' value EMPLOYEE.NAME,
			KEY 'phones' value json_arrayagg(json_object(KEY 'number' value PHONE.PHONENUMBER, KEY 'type' value PHONE.TYPE))
		) employee

The query produces this result:

    "id": 1,
    "name": "Ursula Friedman",
    "phones": [
        "number": "031 333 11 12",
        "type": "WORK"
    "id": 2,
    "name": "Hans Boss",
    "phones": [
        "number": "031 333 11 01",
        "type": "HOME"
        "number": "032 311 43 12",
        "type": "WORK"


Knowing the features of your database can save you time.
Start reading the manual of your database today. Have fun!

Type Safe SQL in Java

No matter if you are using frameworks like JPA, MyBatis or Spring Data JDBC you always end up declaring the SQL statements as a Java String.

Strings, String, Strings

No matter if you are using frameworks like JPAMyBatis or Spring Data JDBC you always end up declaring the SQL statements as a Java String.
The problem with this approach is that you have to write tests for every statement to make sure that it is even valid SQL. There is no compile time guarantee that the SQL statement will execute.

Get rid of the Strings!

Embedded SQL

I started professional software development in 1995 on IBM mainframe computers programming in COBOL. To access the database we used something called “Embedded SQL”:

SELECT lastname, firstname
INTO :lastname, :firstname
FROM employee
WHERE id = :id

The cool thing about Embedded SQL was that a pre-compiler was checking every SQL statement and only if it was valid the code compiled.
Bellow you can see the compile steps. (Source:


When I first met Java and JDBC in 2000 I was confused that nothing similar existed. I found out that there was an initiative called SQLJ started in 1997 but never took off. I don’t have an idea why, maybe because this was hard to integrate for IDE vendors and pre-compilers where not very common for Java. At least the compile steps are similar to Embedded SQL:

When comparing JDBC and SQLJ we can see that there is not much difference from the amount of code you have to write but everything after #sql is type safe because the pre-compiler checks the syntax where as with JDBC there is a String that could contain any error and the error will happen late in production.

And then I found jOOQ!

Ten years ago Lukas Eder release the first version of jOOQ. According to the website is jOOQ “The easiest way to write SQL in Java”

Let’s try to write the same query as above with jOOQ:

List<EmployeeDTO> records = create
         .where(EMPLOYEE.SALARY.between(80000, 100000))

Pretty cool, isn’t it? Yes – but how does it work?

1. Code Generator

jOOQ uses a code generator to generate Java classes from database objects.

For example this is an extract of the class generated by jOOQ for the table EMPLOYEE:

public class Employee extends TableImpl<EmployeeRecord> {

    public static final Employee EMPLOYEE = new Employee();

    public final TableField<EmployeeRecord, Integer> ID = createField("ID", org.jooq.impl.SQLDataType.INTEGER.nullable(false).identity(true), this, "");
    public final TableField<EmployeeRecord, String> LASTNAME = createField("LASTNAME", org.jooq.impl.SQLDataType.VARCHAR(50).nullable(false), this, "");
    public final TableField<EmployeeRecord, String> FIRSTNAME = createField("FIRSTNAME", org.jooq.impl.SQLDataType.VARCHAR(50).nullable(false), this, "");
    public final TableField<EmployeeRecord, Integer> SALARY = createField("SALARY", org.jooq.impl.SQLDataType.INTEGER, this, "");
    public final TableField<EmployeeRecord, Integer> DEPARTMENT_ID = createField("DEPARTMENT_ID", org.jooq.impl.SQLDataType.INTEGER.nullable(false), this, "");
    public final TableField<EmployeeRecord, Integer> MANAGER_ID = createField("MANAGER_ID", org.jooq.impl.SQLDataType.INTEGER, this, "");

There are constants for the table and all the columns. Thanks to these meta data classes it’s not possible to use a type in a SQL statement that does not exists in the database. And because you can generate the meta data every time, the database model changes your code will not compile if there are breaking changes.

How to configure the generator and what input formats for the generator are possible will be described in a future post. (Stay tuned)

2. Domain Specific Language

The second part of jOOQ is the DSL (Domain Specific Language) that allows to write SQL code in Java.
And in contrast to SQL in Strings the DSL forces me to write valid SQL!


So let’s see some more examples. The examples are based on this data model:


What’s next?

That was just a short introduction. In the next blog post we will have a deeper look at all the features jOOQ provides.

In the meanwhile you can checkout the code here:

Java Persistence Done Right

When it comes to accessing relational databases with Java, people usually think of two options:

1. SQL (Structured Query Language)
2. ORM (Object Relational Mapping)

Because the usage of SQL with the Java API JDBC (Java Database Connectivity) is painful and error-prone the first choice is usually an ORM like JPA/Hibernate.


Let’s have a look at the definition of ORM on Wikipedia:

Object-relational mapping (ORM, O/RM, and O/R mapping tool) in computer science is a programming technique for converting data between incompatible type systems using object-oriented programming languages. This creates, in effect, a “virtual object database” that can be used from within the programming language.

The idea behind an ORM framework is to hide the database access from the user. Another goal is to introduce the capability to the database access layer that does not exist in a relational database like inheritance. But this abstraction is leaky and leads to the so-called impedance mismatch:

The Impedance Mismatch

The object-relational impedance mismatch is a set of conceptual and technical difficulties that are often encountered when a relational database management system (RDBMS) is being served by an application program (or multiple application programs) written in an object-oriented programming language or style, particularly because objects or class definitions must be mapped to database tables defined by a relational schema. Source: Wikipedia

The problem with ORM is that the user by default does not have full control over the database access and this can cause several problems. The most common problem is poor performance caused by the fact that developers usually don’t deep dive into the details of a framework. This naive approach usually leads to too many SQL statements executed by the ORM framework.

The fact that it’s possible to define parent-child relationships in ORM raises the question when and how the children are loaded. By default, this is done in a lazy way. So let’s assume that we have a customer order with many items and we want to fetch the customer orders the generated SQL statements will look like this:

select * from customer_order;

The above query returns all customer orders (e.g. 1,2,3,4). If the program accesses the children the ORM framework will produce a query per customer order:

select * from item where customer_order_id = 1;
select * from item where customer_order_id = 2;
select * from item where customer_order_id = 3;
select * from item where customer_order_id = 4;

This problem is called n+1 select problem and happens in every application that uses ORM. The ORM usually provides techniques to overcome this problem but as initially said developers usually are not ORM experts.

What are the alternatives?

As you can see ORM may not be the silver bullet you’re looking for. But as initially mentioned using SQL can be very painful. Luckily there are two popular alternatives.

1. MyBatis (former iBatis)
2. jOOQ


MyBatis was first released in 2001 under the name iBatis and the idea behind this framework is to map SQL statements to Java objects. In contrast to ORM where the SQL statements are generated by the framework, you have full control over the SQL statements because you have to write them on your own.

The code examples show how you write the SQL statement in an annotation (it also supports XML) and that the method returns a Java object and not a JDBC ResultSet:

public interface DepartmentMapper {

    @Select("select id, name from department WHERE name = #{name}")
    Department findByName(String name);

The downside of MyBatis is that there is a lot of mapping work to do. To overcome this disadvantage there is a generator that can help with this task.
But the biggest disadvantage of MyBatis is the lack of type safety. SQL statements are written in Strings and also the mapping is just strings this may cause problems during runtime because the mapping and the SQL are not checked during compile time.


jOOQ is a framework that embraces SQL and makes SQL the primary language to speak to the database from Java in a typesafe and fluent way. jOOQ provides a domain-specific language (DSL). All the artifacts you use with this DSL are generated from the database meta-model.

The difference to MyBatis is that you don’t write SQL in plain text and therefore the compiler can check your SQL statements and you have full code completion in your IDE.

DepartmentDTO department = dsl

As you can see in the above example you really write SQL! The types in capital letters are constants that are generated from the database meta-model and provide the type-safety with the DSL.

Should I still use ORM?

As usual, the answer is “it depends”. But because of the impedance mismatch and the fact you have to be an expert in ORM and SQL you really should think twice if it’s worth investing in this technology.
With jOOQ you will get a great alternative plus full control over the database access!

What’s Next?

In the next blog post, I will introduce jOOQ as the best way to use SQL in Java applications.
Stay tuned!

Introduction to Database Migrations with Spring Boot and Flyway

If you are working with Spring Boot there are several ways to initialize the database.

You could create a schema.sql, add it to the classpath, and set spring.datasource.initialization-mode=true. Spring will execute the SQL script on startup.

Or you could use the Hibernate built-in functionality and set spring.jpa.hibernate.ddl-auto to create or update.

But both approaches are very limited and if you have to deal with an evolving database you may look for something more sophisticated.
That’s where database migration tools fill the gap.

Database migration tools

In the Java environment, there are two open-source projects available: Flyway and Liquibase. The difference between these two is that Flyway uses SQL for the migrations and Liquibase supports additional abstractions in XML, YAML, or JSON.

Both tools are very well integrated with Spring Boot. You simply have to add the right dependency in your pom.xml or Gradle build file.
Read more about why you should care about database migrations here.

Flyway with Spring Boot

In my example, I use Flyway with Spring Boot. To use it simply add this dependency and Spring Boot will auto-configure Flyway right away.


Now you’re ready to start adding your migration SQL files to your classpath in the path classpath:db/migration

Naming Conventions

To have Flyway execute your SQL scripts you have to follow some naming conventions.

  • Prefix (V, U or R)V = Versioned migrations that are executed only once
  • U = Undo migrations (only supported in the commercial version)
  • R = Repeatable migrations. Scripts that could run with every execution
  • Version
  • Separator __
  • Description
  • Suffix: (.sql)

Example: V1__Init.sql

Example Migration

V1__Init.sql could look like

CREATE TABLE address (
    id integer NOT NULL,
    city character varying(255),
    state character varying(255),
    street character varying(255),
    zip character varying(255),
    created_date timestamp,
    last_modified_date timestamp

How it Works

Flyway uses a table to store the version of the database. The default name of this table is flyway_schema_history and the content looks like this:

As you can see Flyway stores the actual version and is, therefore, able to know which migrations scripts have to be executed.

Further Reading

That was just a brief introduction. Flyway has way more to offer and you will find all you need to know in the official documentation on