Practice of Programming (daily): Journey of Spring Boot upgrade, Jackson and Lombok

February 8, 2018

Journey of Spring Boot upgrade, Jackson and Lombok

Recently I have started the process of upgrading from Spring Boot 1.3.2 into the latest (at the moment of writing) Spring Boot 1.5.10 version.

And I faced the couple of interesting issues on the way (changes in Spring Boot testing classes are not counting, which are well documented [1] and [2]). In our case it was related to the version upgrade of Jackson and Lombok.

So, if you are using these 2 libraries and planning (or doing) the upgrade to the Spring Boot 1.5.10 it is worth continue reading.

Intro

I will demonstrate the problem on the sample class below:

import lombok.Value;

@Value
public class Address {
    String country;
    String city;
}

and the test class (we are using Spock here, see [3]):

import com.fasterxml.jackson.databind.ObjectMapper
import spock.lang.Shared
import spock.lang.Specification

class AddressSpec extends Specification {
    @Shared
    def mapper = new ObjectMapper()

    def "deserialize address"() {
        expect:
        mapper.readValue("""{"country": "X", "city": "Y"}""", Address) == new Address("X", "Y")
    }
}

Problem 1

Spring Boot 1.3.2, Jackson 2.7.5, Lombok 1.16.18

Note: Spring Boot 1.3.2 actually uses Jackson 2.6.5 version, although in the project we used 2.7.5, this is important. Although if you migrate from Spring Boot 1.4.x you are already using Jackson 2.7.x version. So the problem case is applicable there.

So, when using the above versions the test above runs fine without any problems. This was the current state of the app.

Spring Boot 1.5.10, Jackson 2.8.10, Lombok 1.16.20

Note: the versions above are the versions that Spring Boot 1.5.10 comes with [4]. And it will be yours default versions unless you change/specify them explicitly in your Maven POM.

Now, if you run the test it will fail with the following error:

Condition failed with Exception:

mapper.readValue("""{"country": "X", "city": "Y"}""", Address) == new Address("X", "Y")
|      |
|      com.fasterxml.jackson.databind.JsonMappingException: Can not construct instance of blog.Address: no suitable constructor found, can not deserialize from Object value (missing default constructor or creator, or perhaps need to add/enable type information?)
|       at [Source: {"country": "X", "city": "Y"}; line: 1, column: 2]
com.fasterxml.jackson.databind.ObjectMapper@71def8f8


	at blog.AddressSpec.deserialize address(AddressSpec.groovy:13)
Caused by: com.fasterxml.jackson.databind.JsonMappingException: Can not construct instance of blog.Address: no suitable constructor found, can not deserialize from Object value (missing default constructor or creator, or perhaps need to add/enable type information?)
 at [Source: {"country": "X", "city": "Y"}; line: 1, column: 2]
	at com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:270)
	at com.fasterxml.jackson.databind.DeserializationContext.instantiationException(DeserializationContext.java:1456)
	at com.fasterxml.jackson.databind.DeserializationContext.handleMissingInstantiator(DeserializationContext.java:1012)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1206)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:314)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:148)
	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3814)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2858)
	... 1 more

Why is that? See that the main problem is the following:

no suitable constructor found, can not deserialize from Object value (missing default constructor or creator, or perhaps need to add/enable type information?)

But then how it has worked before?

After doing some research I found the following:

Jackson 2.7 introduced support for @ConstructorProperties [5] (specifically [6])
@ConstructorProperties was generated by default by Lombok up to the version 1.16.18. While in the version 1.16.20 they disabled this behaviour and don't generate @ConstructorProperties by default on constructor anymore [7].

So this explains why it has stopped working right now.

All fine, but then how to fix it?

There are multiple solutions available:

Solution 1

Write constructor explicitly for every @Value or @Data class and add @JsonCreator and @JsonProperty there explicitly. It was not the option in our case, as there are a lot of Lombok @Value powered classes, and it goes against the purpose of Lombok. But that could be different in your project.

Solution 2

Annotate Lombok classes with @AllArgsConstructor(onConstructor_={@ConstructorProperties({"country", "city"})}), which will make now deserialization work, but again lots of work, and could be error prone and brittle in the case of any refactor.

One could also try to use @AllArgsConstructor(onConstructor_={@JsonCreator}), unfortunately this will not work, and fails with the following error:

Condition failed with Exception:

mapper.readValue("""{"country": "X", "city": "Y"}""", Address) == new Address("X", "Y")
|      |
|      com.fasterxml.jackson.databind.JsonMappingException: Argument #0 of constructor [constructor for blog.Address, annotations: {interface com.fasterxml.jackson.annotation.JsonCreator=@com.fasterxml.jackson.annotation.JsonCreator(mode=DEFAULT)}] has no property name annotation; must have name when multiple-parameter constructor annotated as Creator
|       at [Source: {"country": "X", "city": "Y"}; line: 1, column: 1]
com.fasterxml.jackson.databind.ObjectMapper@1e683a3e


	at blog.AddressSpec.deserialize address(AddressSpec.groovy:13)
Caused by: com.fasterxml.jackson.databind.JsonMappingException: Argument #0 of constructor [constructor for blog.Address, annotations: {interface com.fasterxml.jackson.annotation.JsonCreator=@com.fasterxml.jackson.annotation.JsonCreator(mode=DEFAULT)}] has no property name annotation; must have name when multiple-parameter constructor annotated as Creator
 at [Source: {"country": "X", "city": "Y"}; line: 1, column: 1]
	at com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:305)
	at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:268)
	at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:244)
	at com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:142)
	at com.fasterxml.jackson.databind.DeserializationContext.findRootValueDeserializer(DeserializationContext.java:476)
	at com.fasterxml.jackson.databind.ObjectMapper._findRootDeserializer(ObjectMapper.java:3915)
	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3810)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2858)
	... 1 more

As indeed explicit @JsonProperty is required for the constructor arguments.

Solution 3

...
def setupSpec() {
    mapper.registerModule(new ParameterNamesModule())
}
...

And this was the option we have chosen.

For that option if you use IntelliJ IDEA you need to set -parameters for the Java Compiler like in the image below:

And for the Maven add the corresponding option to the Maven compiler plugin:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-compiler-plugin</artifactId>
    <configuration>
        <parameters>true</parameters>        
    </configuration>
    <version>3.7.0</version>
</plugin>

Although as mentioned in [8] there are couple of preconditions to make this work, specifically (copied from that page):

class Person must be compiled with Java 8 compliant compiler with option to store formal parameter names turned on (-parameters option). For more information about Java 8 API for accessing parameter names at runtime see this
if there are multiple visible constructors and there is no default constructor, @JsonCreator is required to select constructor for deserialization
if used with jackson-databind lower than 2.6.0, @JsonCreator is required. In practice, @JsonCreator is also often required with 2.6.0+ due to issues with constructor discovery that will be resolved in 2.7.
if class Person has a single argument constructor, its argument needs to be annotated with @JsonProperty("propertyName"). This is to preserve legacy behavior, see FasterXML/jackson-databind/#1498

So, in the case if the constructor has only one argument, unfortunately we have to write the constructor explicitly and annotate it with @JsonCreator and argument with @JsonProperty.

All good the problem was fixed. That could be it for your scenario. Although in our case we had another problem now.

Problem 2

There were couple of tests which produced the JSON value for the request object, and the request class had explicit @JsonCreator and @JsonProperty in it. Although the JSON produced in the tests didn't match the property names, but instead matched the fields.

Let me show you the example (will use the same Address class):

import com.fasterxml.jackson.annotation.JsonCreator;
import com.fasterxml.jackson.annotation.JsonProperty;
import lombok.Value;

@Value
public class Address {
    String country;
    String city;

    @JsonCreator
    public Address(@JsonProperty("cntr") final String country,
                   @JsonProperty("ct") final String city) {
        this.country = country;
        this.city = city;
    }
}

So, as you see above the arguments in the constructor have explicit @JsonProperty, and the names are cntr and ct accordingly. It means that we expected the following JSON to be passed: {"cntr": "country name", "ct": "city name"}. Although in the test we generated {"country": "country name", "city": "city name"} instead. Again this is just to demonstrate the problem, as in the project the classes were different and they were the request objects.

And as you can see in the image below Address constructor gets called by Jackson, but nulls are passed there for every argument.

So I guess Jackson then removes the final modifier on the fields, and assigns the values there directly "by passing" the constructor. Which also means that if you have any validation logic in your constructor, based on the values of the arguments it will no longer work, as the values will be null there.

Although!, and it is quite interesting with the ParameterNamesModule module registered it forces Jackson to respect the names defined in the constructor, so with the corresponding module registered (and we have done that per Solution #3), the tests started to fail:

Condition failed with Exception:

mapper.readValue("""{"country": "X", "city": "Y"}""", Address) == new Address("X", "Y")
|      |
|      com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "country" (class blog.Address), not marked as ignorable (2 known properties: "cntr", "ct"])
|       at [Source: {"country": "X", "city": "Y"}; line: 1, column: 30] (through reference chain: blog.Address["country"])
com.fasterxml.jackson.databind.ObjectMapper@6253c26


	at blog.AddressSpec.deserialize address(AddressSpec.groovy:18)
Caused by: com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "country" (class blog.Address), not marked as ignorable (2 known properties: "cntr", "ct"])
 at [Source: {"country": "X", "city": "Y"}; line: 1, column: 30] (through reference chain: blog.Address["country"])
	at com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:62)
	at com.fasterxml.jackson.databind.DeserializationContext.handleUnknownProperty(DeserializationContext.java:834)
	at com.fasterxml.jackson.databind.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:1093)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownProperty(BeanDeserializerBase.java:1489)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownProperties(BeanDeserializerBase.java:1443)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:487)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1191)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:314)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:148)
	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3814)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2858)
	... 1 more

Which is something I would expect to happen also when explicit @JacksonCreator and @JsonProperty are used.

So I opened the ticket [10].

The solution would be to modify the JSON produced in the tests to match the @JsonProperty-ies, although again this is something I would expect to fail in the first place even without ParameterNamesModule module registered.

Overall it was an interesting experience and interesting research to make to understand why things stopped working and how they worked before.

[1]: https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-1.4-Release-Notes#test-utilities-and-classes

[2]: https://spring.io/blog/2016/04/15/testing-improvements-in-spring-boot-1-4

[3]: Setup and testing with Spock

[4]: https://github.com/spring-projects/spring-boot/milestone/96?closed=1&page=1

[5]: https://github.com/FasterXML/jackson/wiki/Jackson-Release-2.7#databind

[6]: https://github.com/FasterXML/jackson-databind/issues/905

[7]: https://projectlombok.org/changelog

[8]: https://github.com/FasterXML/jackson-modules-java8/tree/master/parameter-names

[9]: https://docs.oracle.com/javase/tutorial/reflect/member/methodparameterreflection.html

[10]: https://github.com/FasterXML/jackson-databind/issues/1924

Tags: development