For an absent property Jackson injects `NullNode` instead of `null` to a JsonNode-typed constructor argument of a `@ConstructorProperties`-annotated constructor

[robvarga]

Describe the bug
When using @ConstructorProperties-annotated constructor to deserialize type, for an attribute of type JsonNode which is absent from the deserialized JSON, a NullJsonNode is passed to the constructor instead of null reference.

I would like to know if there is a workaround (e.g. some annotation indicating what to do for absent properties, or configuring some alternative deserializer on a per-property basis which is able to deal with this) for this issue which works with a constructor and final attributes instead of having to resort to non-final attributes and setters.
Since our project is on 2.7.5 at the moment, I would particularly be intereted if there is a workaround for 2.7.x which allows me to inject null instead of this NullNode at least on a per object basis if not on a per-property basis.

If there is no workaround, then this should probably be fixed or a workaround should be provided which works with final attributes and @ConstructorProperties-annotated constructor.

Version information
Which Jackson version(s) was this for?
Reproduced with 2.12.4, 2.9.8 and 2.7.5

To Reproduce
If you have a way to reproduce this with:

package jacksontests;
import java.beans.ConstructorProperties;
import org.junit.Assert;
import org.junit.Test;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;

public class JsonNodeNullTest {

    public static class TestClass {

        private final JsonNode nodeFromConstructor;
        private JsonNode nodeFromSetter;

        @ConstructorProperties("nodeFromConstructor")
        public TestClass(JsonNode nodeFromConstructor) {
            this.nodeFromConstructor = nodeFromConstructor;
        }

        public JsonNode getNodeFromSetter() {
            return nodeFromSetter;
        }

        public void setNodeFromSetter(JsonNode nodeFromSetter) {
            this.nodeFromSetter = nodeFromSetter;
        }

        public JsonNode getNodeFromConstructor() {
            return nodeFromConstructor;
        }
    }

    @Test
    public void testJsonNodeNulls() throws Exception {
        TestClass res = new ObjectMapper().readValue("{}", TestClass.class);
        Assert.assertNotNull(res);
        Assert.assertNull(res.getNodeFromSetter());
        Assert.assertNull(res.getNodeFromConstructor());
    }
}

Expected behavior
The unit test fails as res.getNodeFromConstructor() returns a NullNode instead of null.
It should return null identically to the property which is setter-populated. (The setter does not get called).

Additional context

[cowtowncoder]

I cannot think of a way to change this behavior, except for custom JsonNode deserializer (which would work but requires some effort).
Given that this behavior has existed for a long time (based on versions mentioned), I doubt change in behavior is possible for patches, although could perhaps be considered for a minor version.

But I hope to have a look at this before 2.13.0 is released to double-check my understanding. Null replacement itself is needed mostly for primitive values; I think I agree that it should not be used for JsonNode.

[robvarga]

Ok, the following JsonDeserializer referenced with @JsonDeserialize#using seems to do the trick:

public class AbsentAsNullJsonNodeDeserializer extends JsonNodeDeserializer {

    /**
     * This I believe is required with Jackson 2.9 upwards in order for #getNullValue(DeserializationContext) to be invoked on every property and with a non-null <code>ctxt</code> parameter.
     * @since 2.9 
     * /
    @Override
    public AccessPattern getNullAccessPattern() {
        return AccessPattern.DYNAMIC;
    }

    @Override
    public JsonNode getNullValue(DeserializationContext ctxt) {
        return ctxt.getParser().hasToken(JsonToken.END_OBJECT) ? null : super.getNullValue(ctxt);
    }
}

Thanks for the clue that it should be possible with JsonDeserializer, I looked at it once but didn't succeed to get it done at that time.

Tested with Jackson 2.7.5 and 2.12.4

[robvarga]

A unit test to cover the main possible cases:

public class JsonNodeNullTest {

    public static class TestClass {

        private final JsonNode nodeFromConstructor;
        private final JsonNode nodeFromConstructor2;
        private JsonNode nodeFromSetter;
        private JsonNode nodeFromSetter2;

        @ConstructorProperties({ "nodeFromConstructor", "nodeFromConstructor2" })
        public TestClass(@JsonDeserialize(using = AbsentAsNullJsonNodeDeserializer.class) JsonNode nodeFromConstructor,
                         @JsonDeserialize(using = AbsentAsNullJsonNodeDeserializer.class) JsonNode nodeFromConstructor2) {
            this.nodeFromConstructor = nodeFromConstructor;
            this.nodeFromConstructor2 = nodeFromConstructor2;
        }

        public JsonNode getNodeFromSetter() {
            return nodeFromSetter;
        }

        public void setNodeFromSetter(JsonNode nodeFromSetter) {
            this.nodeFromSetter = nodeFromSetter;
        }

        public JsonNode getNodeFromSetter2() {
            return nodeFromSetter2;
        }

        public void setNodeFromSetter2(JsonNode nodeFromSetter2) {
            this.nodeFromSetter2 = nodeFromSetter2;
        }

        public JsonNode getNodeFromConstructor() {
            return nodeFromConstructor;
        }

        public JsonNode getNodeFromConstructor2() {
            return nodeFromConstructor2;
        }
    }

    @Test
    public void testJsonNodeNulls() throws Exception {
        TestClass res = new ObjectMapper().readValue("{\"nodeFromConstructor2\" : null, \"nodeFromSetter2\" : null }", TestClass.class);
        Assert.assertNotNull(res);
        Assert.assertNull(res.getNodeFromSetter());
        Assert.assertThat(res.getNodeFromSetter2(), CoreMatchers.instanceOf(NullNode.class));
        Assert.assertNull(res.getNodeFromConstructor());
        Assert.assertThat(res.getNodeFromConstructor2(), CoreMatchers.instanceOf(NullNode.class));
    }
}

[cowtowncoder]

I will have to think about this a bit more -- may consider change in 2.14, but too late for anything in 2.13.

[robvarga]

Just don't break the workaround, please :-)

[cowtowncoder]

@robvarga :)

Usage as shown should still work.

Another work-around, if you don't mind using Optional, would be using Optional<JsonNode> -- I think that would pass absent Optional as expected.

[novtor]

We have the same issue starting from 2.11. Koltin class containing nullable property is desirialized as NullNode instead of null. For example:
data class AModel(val aProp: com.fasterxml.jackson.databind.JsonNode?)
when desirializing from {} string results in aProp to be NullNode.instance

[cowtowncoder]

One problem here wrt Kotlin is just that Kotlin has additional nullability metadata over Java.

So the question of null mapping into JsonNode is one thing, but whether to, and how, consider Kotlin metadata is similarly tricky problem.

[novtor]

sure, @cowtowncoder, finally I open a PR in jackson-databind-kotlin: FasterXML/jackson-module-kotlin#491.
I think that if a value is nullable and missing in the payload, it should not be assigned any null-like value

[cowtowncoder]

I'll let Kotlin module maintainers to decide that: I think it is something worth documenting. The original use case for getNullValue() really was about primitive values (for which null does not work) and creator methods.
But it then extended a bit on "reference types" (Optional of Java 8, Guava, Option of Scala), and semantics there get bit muddier, since null-avoidance is sort of what these types do. Whether same should apply to JsonNode is debatable.

[novtor]

Agree, so let us debate 😄
If we look at it from the e2e serialize/deserialize point of view: null value is serialized as null (or absent) and should be deserialized as null. Currently, missing or null payload is deserialized as NullNode.
Another point is that it is like this only for kotlin classes, not java, java field is deserialized as null
I think there is some inconsistency