MappingCouchbaseConverter.read() is not recursive

[JesusTheHun]

Using Cluster.query() I'm parsing the result into classes that have child object.
A quick exemple to visualise :

@Data
public class A {
    String someProp;
    SubA nestedObject;

    @Data
    public static SubA() {
        String someNestedProp;
    }
}

Doing converter.read(A.class, document) leads to nestedObject being a HashMap<String, String> instead of a SubA as anyone would expect. Am I doing this wrong or are my expectations correct ?

[mikereiche]

I made an ARepsitory and did a save followed by a find.

git clone git@github.com:spring-projects/spring-data-couchbase.git

package org.springframework.data.couchbase.domain;

import lombok.AllArgsConstructor;
import lombok.Data;
import org.springframework.data.annotation.Id;
import org.springframework.data.couchbase.core.mapping.Document;

@Document
@Data
@AllArgsConstructor
public class A {
  @Id
  String someProp;
  SubA nestedObject;

  public String getId() {
    return someProp;
  }

  public String toString(){
    return "A{ someProp="+someProp+" nestedObject="+nestedObject+"}";
  }

  @Data
  @AllArgsConstructor
  public static class SubA {
    String someNestedProp;
    public String toString(){
      return "SubA{ someNestedProp="+someNestedProp+"}";
    }
  }

}

package org.springframework.data.couchbase.repository;

import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.couchbase.config.AbstractCouchbaseConfiguration;
import org.springframework.data.couchbase.domain.A;
import org.springframework.data.couchbase.domain.ARepository;
import org.springframework.data.couchbase.repository.config.EnableCouchbaseRepositories;
import org.springframework.data.couchbase.util.ClusterAwareIntegrationTests;
import org.springframework.data.couchbase.util.ClusterType;
import org.springframework.data.couchbase.util.IgnoreWhen;
import org.springframework.test.context.junit.jupiter.SpringJUnitConfig;

import java.lang.reflect.InvocationTargetException;
import java.util.Optional;

import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertTrue;

/**
 * Repository KV tests
 *
 * @author Michael Nitschinger
 * @author Michael Reiche
 */
@SpringJUnitConfig(CouchbaseARepositoryKeyValueIntegrationTests.Config.class)
@IgnoreWhen(clusterTypes = ClusterType.MOCKED)
public class CouchbaseARepositoryKeyValueIntegrationTests extends ClusterAwareIntegrationTests {

  @Autowired
  ARepository aRepository;

  @Test
  @IgnoreWhen(clusterTypes = ClusterType.MOCKED)
  void saveAndFindById() throws NoSuchMethodException, InvocationTargetException, IllegalAccessException {
    A.SubA subA = new A.SubA("this is someNestedProp");
    A a = new A("this is someProp", subA);
    a = aRepository.save(a);
    Optional<A> found = aRepository.findById(a.getId());
    assertTrue(found.isPresent());
    assertEquals(a,
        found.get());
    System.out.println("a="+a);
    System.out.println("a.suba="+a.getSomeProp());
    aRepository.delete(a);
  }


  @Configuration
  @EnableCouchbaseRepositories("org.springframework.data.couchbase")
  static class Config extends AbstractCouchbaseConfiguration {

    @Override
    public String getConnectionString() {
      return connectionString();
    }

    @Override
    public String getUserName() {
      return config().adminUsername();
    }

    @Override
    public String getPassword() {
      return config().adminPassword();
    }

    @Override
    public String getBucketName() {
      return bucketName();
    }

  }

}

src/test/resources/integration.properties

# Couchbase Integration-Test Default Properties
# If set to false, it is assumed that the host is managing the cluster and
# as a result no containers or anything will be spun up.
# Options: containerized, mocked, unmanaged
cluster.type=unmanaged <<<< change to unmanaged
# Default configs for both cases
cluster.adminUsername=Administrator
cluster.adminPassword=password
# Default configs for the mocked environment
cluster.mocked.numNodes=1
cluster.mocked.numReplicas=1
# Entry point configuration if not managed
# value of hostname and ns_server port
cluster.unmanaged.seed=10.144.220.101:8091 <<<< my server
cluster.unmanaged.numReplicas=0

I get the expected result:

a=A{ someProp=this is someProp nestedObject=SubA{ someNestedProp=this is someNestedProp}}
a.suba=this is someProp

From CouchbaseTemplate.support().decodeEntity()

T readEntity = converter.read(entityClass, (CouchbaseDocument) translationService.decode(source, converted));

[mikereiche]

the stacktrace from decodeEntity() will be this. See the two calls at read:243, MappingCouchbaseConverter? That's the recursion. The second call is initiated from read:Value:846 because the 'value' is a(nother) CouchbaseDocument.

readValue:845, MappingCouchbaseConverter (org.springframework.data.couchbase.core.convert) [2]
access$300:85, MappingCouchbaseConverter (org.springframework.data.couchbase.core.convert)
getPropertyValue:964, MappingCouchbaseConverter$CouchbasePropertyValueProvider (org.springframework.data.couchbase.core.convert)
getValueInternal:308, MappingCouchbaseConverter (org.springframework.data.couchbase.core.convert)
doWithPersistentProperty:276, MappingCouchbaseConverter$1 (org.springframework.data.couchbase.core.convert)
doWithPersistentProperty:268, MappingCouchbaseConverter$1 (org.springframework.data.couchbase.core.convert)
doWithProperties:360, BasicPersistentEntity (org.springframework.data.mapping.model)
read:268, MappingCouchbaseConverter (org.springframework.data.couchbase.core.convert)
read:243, MappingCouchbaseConverter (org.springframework.data.couchbase.core.convert)
readValue:846, MappingCouchbaseConverter (org.springframework.data.couchbase.core.convert) [1]
access$300:85, MappingCouchbaseConverter (org.springframework.data.couchbase.core.convert)
getPropertyValue:964, MappingCouchbaseConverter$CouchbasePropertyValueProvider (org.springframework.data.couchbase.core.convert)
getPropertyValue:913, MappingCouchbaseConverter$CouchbasePropertyValueProvider (org.springframework.data.couchbase.core.convert)
getParameterValue:74, PersistentEntityParameterValueProvider (org.springframework.data.mapping.model)
getParameterValue:53, SpELExpressionParameterValueProvider (org.springframework.data.mapping.model)
extractInvocationArguments:276, ClassGeneratingEntityInstantiator (org.springframework.data.mapping.model)
createInstance:248, ClassGeneratingEntityInstantiator$EntityInstantiatorAdapter (org.springframework.data.mapping.model)
createInstance:89, ClassGeneratingEntityInstantiator (org.springframework.data.mapping.model)
read:265, MappingCouchbaseConverter (org.springframework.data.couchbase.core.convert)
read:243, MappingCouchbaseConverter (org.springframework.data.couchbase.core.convert)
read:200, MappingCouchbaseConverter (org.springframework.data.couchbase.core.convert)
read:85, MappingCouchbaseConverter (org.springframework.data.couchbase.core.convert)
decodeEntity:106, CouchbaseTemplateSupport (org.springframework.data.couchbase.core)

btw - The CouchbaseTemplateSupport object and its decodeEntity() method are public, so you shouldn't have to dig into MappingCouchbaseConverter.read(). #1210

[mikereiche]

If you post your code, I'll run it in the debugger to see what's going on.

[JesusTheHun]

Here is the code :

Class definition

@Data
abstract public class BaseManageableEntity {
    @Id
    protected String id;

    @Field("dag_paths")
    protected List<DagPath> dagPaths = new ArrayList<>();

    abstract public ManageableType getManageableType();

    @Data
    @AllArgsConstructor
    @EqualsAndHashCode
    public static class DagPath {
        private LinkType pathType;
        private List<DagNode> path;

        @Data
        @EqualsAndHashCode
        public static class DagNode {
            private String id;
            private ManageableType type;

            private DagNode() {}

            public static DagNode of(BaseManageableEntity manageable) {
                if (manageable.getId() == null) throw new IllegalArgumentException();
                if (manageable.getManageableType() == null) throw new IllegalArgumentException();

                var node = new DagNode();
                node.id = manageable.getId();
                node.type = manageable.getManageableType();

                return node;
            }
        }
    }
}

@Document
@Data
@EqualsAndHashCode(callSuper = true)
public class Foo extends BaseManageableEntity {
    String title;
    public ManageableType getManageableType() {
        return ManageableType. ManageableTypeOne;
    }
}

public enum ManageableType {
    ManageableTypeOne, ManageableTypeTwo
}

public enum LinkType {
    LinkTypeOne, LinkTypeTwo
}

// Utility functions
public <T> T toDomainEntity(byte[] obj, Class<T> clazz) {
    JsonObject json = serializer.deserialize(JsonObject.class, obj);
    Class<?> targetClass = Class.forName(json.get("_class").toString());

    if (!clazz.isAssignableFrom(targetClass)) {
        throw new IllegalArgumentException(String.format("Trying to assign %s to %s", targetClass, clazz));
    }

    CouchbaseDocument document = new CouchbaseDocument()
            .setId(json.get(TemplateUtils.SELECT_ID).toString())
            .setContent(json);
    return (T) couchbaseTemplate.getConverter().read(targetClass, document);
}

public <T> List<T> toDomainEntities(List<byte[]> rows, Class<T> clazz) {
    return rows.stream().map(obj -> toDomainEntity(obj, clazz)).collect(Collectors.toList());
}

// Execution
QueryResult results = cluster.query(localQuery, QueryOptions.queryOptions().parameters(JsonObject.from(localMap)).adhoc(isAdHoc()));
return toDomainEntities(results.rowsAs(byte[].class), clazz);

[mikereiche]

Could you provide an input data? Thanks.

[mikereiche]

The difference could be that the fields are lists of sub-documents. But it should still work

[JesusTheHun]

I have edited the code in my previous comment to be more complete.
Very basic stuff for data input:

var a = new Foo();
a.setId("a");

var b = new Foo();
b.setId("b");

var dagPath = new BaseManageableEntity.DagPath(LinkType.LinkTypeOne, List.of(BaseManageableEntity.DagPath.DagNode.of(a), BaseManageableEntity.DagPath.DagNode.of(b)));
b.getDagPaths().add(dagPath);

[mikereiche]

Foo doesn't implement getManageableType() from BaseManageableEntity.

[JesusTheHun]

I've updated the code.

[mikereiche]

couchbaseTemplate.insertById()/findById() (which uses CouchbaseTemplateSupport.decodeEntity) seems to work:

  @Test
  @IgnoreWhen(clusterTypes = ClusterType.MOCKED)
  void saveAndFindById2() {

    Foo a = new Foo();
    a.setId("a");

    var b = new Foo();
    b.setId("b");

    BaseManageableEntity.DagPath dagPath = new BaseManageableEntity.DagPath(LinkType.LinkTypeOne,
        List.of(BaseManageableEntity.DagPath.DagNode.of(a),
            BaseManageableEntity.DagPath.DagNode.of(b)));
    b.getDagPaths()
        .add(dagPath);

    try {
      couchbaseTemplate.removeById(Foo.class).one(b.getId());
    } catch(Exception e){
      //System.err.println(e);
    }
    couchbaseTemplate.insertById(Foo.class).one(b);

    var c = couchbaseTemplate.findById(Foo.class).one(b.getId());
    System.err.println("Foo: " +c);
    System.err.println("DagPath: "+c.getDagPaths());

    couchbaseTemplate.removeById(Foo.class).one(b.getId());
  }

Foo: Foo(title=null, mt=ManageableTypeOne)
DagPath: [BaseManageableEntity.DagPath(pathType=LinkTypeOne, path=[BaseManageableEntity.DagPath.DagNode(id=a, type=ManageableTypeOne), BaseManageableEntity.DagPath.DagNode(id=b, type=ManageableTypeOne)])]

Where does 'serializer' in toDomainEntity come from?

[JesusTheHun]

Where does 'serializer' in toDomainEntity come from?

It comes from cluster.environment().jsonSerializer().

[JesusTheHun]

I tried to use CouchbaseTemplateSupport but for that I needed to upgrade to 4.3.1 and then I met the breaking change exposed in #1315

[mikereiche]

The issue here (in 1312) is that in the CouchbaseDocument Foo, the List<> dagPath is an ArrayList (maybe it should be a CouchbaseList?) and it falls through to the else. I'll look into it.

In the meantime, can you use either the template or repository? Or maybe we can skip the CouchbaseDocument and go from use a String as the source.

	private <R> R readValue(Object value, TypeInformation<?> type, Object parent) {
		Class<?> rawType = type.getType();

		if (conversions.hasCustomReadTarget(value.getClass(), rawType)) {
			return (R) conversionService.convert(value, rawType);
		} else if (value instanceof CouchbaseDocument) {
			return (R) read(type, (CouchbaseDocument) value, parent);
		} else if (value instanceof CouchbaseList) {
			return (R) readCollection(type, (CouchbaseList) value, parent);
		} else {
			return (R) getPotentiallyConvertedSimpleRead(value, rawType);
		}
	}