Application migration from version 1.3.2 to version 2.3.4

[dcdh]

Describe the bug
I am migrating an application using an old version of morphia to Quarkus.
I have notice a breaking change regarding the index validation on @reference property.
Before it was possible to référence them inside the index but now it is no more possible : a ValidationException is thrown instead.

To Reproduce
Steps to reproduce the behavior:

Have an entity with a property annotated with @reference
Reference the property inside @Index

Sample:

@Entity(value = "Child")
public class Child {
    @Id
    ObjectId id;
}

@Entity(value = "Parent")
@Indexes({
    @Index(fields = @Field(value = "child.$id"), options = @IndexOptions(disableValidation = true))
})
public class Parent {
    @Reference
    Child child;
}

The change has been made on the resolve method in PathTarget when it checks if an element is present and if the field is a reference. Line 175.
Does this behavior is expected ?
Maybe a check can be added if we want to validateNames or not ?

Expected behavior
Application should start.
Index should be created.

** Please complete the following information: **

• Server Version: quarkus-morphia 0.5.0
• Driver Version:
• Morphia Version:

Additional context
Any example models, queries, and executable test cases you can supply will greatly help debugging your issue:

[evanchooly]

Which part is it complaining about? Which element can it not find? I think i know where the error likely is. If you don't have existing data, you could always set that reference to use idOnly and just index the child field.

[dcdh]

Hello Evan, sorry for the late reply.

In fact the situation is worst :( It is not possible to query a field referencing another collection even if the validation is disabled.
I've attached a reproducer developed in Quarkus with Morphia. One of the test is failing with this issue:

org.opentest4j.AssertionFailedError: Unexpected exception thrown: dev.morphia.query.ValidationException: Could not resolve path 'child.$id' against 'org.acme.Parent'.  Unknown path element: 'child'.

	at org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:152)
	at org.junit.jupiter.api.AssertDoesNotThrow.createAssertionFailedError(AssertDoesNotThrow.java:84)
	at org.junit.jupiter.api.AssertDoesNotThrow.assertDoesNotThrow(AssertDoesNotThrow.java:53)
	at org.junit.jupiter.api.AssertDoesNotThrow.assertDoesNotThrow(AssertDoesNotThrow.java:36)
	at org.junit.jupiter.api.Assertions.assertDoesNotThrow(Assertions.java:3168)
	at org.acme.ParentRepositoryTest.shouldQueryParentHavingSpecificChild(ParentRepositoryTest.java:104)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at io.quarkus.test.junit.QuarkusTestExtension.runExtensionMethod(QuarkusTestExtension.java:1013)
	at io.quarkus.test.junit.QuarkusTestExtension.interceptTestMethod(QuarkusTestExtension.java:827)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
Caused by: dev.morphia.query.ValidationException: Could not resolve path 'child.$id' against 'org.acme.Parent'.  Unknown path element: 'child'.
	at dev.morphia.internal.PathTarget.failValidation(PathTarget.java:155)
	at dev.morphia.internal.PathTarget.resolve(PathTarget.java:176)
	at dev.morphia.internal.PathTarget.translatedPath(PathTarget.java:124)
	at dev.morphia.query.filters.Filter.path(Filter.java:167)

Could you please have a look on it ? Many thanks.

[dcdh]

I cannot push files from my organisation :(
Here are them

package org.acme;

import dev.morphia.annotations.Entity;
import dev.morphia.annotations.Id;
import org.bson.types.ObjectId;

import java.util.Objects;

@Entity(value = "child", useDiscriminator = false)
public class Child {
    @Id
    ObjectId id;

    public Child() {
    }

    public Child(final ObjectId id) {
        this.id = Objects.requireNonNull(id);
    }
}

package org.acme;

import dev.morphia.annotations.Entity;
import dev.morphia.annotations.Id;
import dev.morphia.annotations.Reference;
import org.bson.types.ObjectId;

import java.util.Objects;

@Entity(value = "parent", useDiscriminator = false)
public class Parent {
    @Id
    ObjectId id;
    @Reference(lazy = true)
    Child child;

    public Parent() {
    }

    public Parent(final ObjectId id, final Child child) {
        this.id = Objects.requireNonNull(id);
        this.child = Objects.requireNonNull(child);
    }
}

package org.acme;

import dev.morphia.Datastore;
import dev.morphia.query.filters.Filters;
import jakarta.enterprise.context.ApplicationScoped;
import org.bson.types.ObjectId;

import java.util.Objects;

@ApplicationScoped
public class ParentRepository {

    private final Datastore store;

    public ParentRepository(final Datastore store) {
        this.store = Objects.requireNonNull(store);
    }

    public Child store(final Child child) {
        store.insert(child);
        return child;
    }

    public Parent store(final Parent parent) {
        store.insert(parent);
        return parent;
    }

    public Parent findByChildId(final ObjectId objectId) {
        return store.find(Parent.class).disableValidation()
                .filter(Filters.eq("child.$id", objectId))
                .first();
    }
}

package org.acme;

import com.mongodb.DBRefCodecProvider;
import com.mongodb.client.MongoClient;
import com.mongodb.client.model.Filters;
import io.quarkus.test.junit.QuarkusTest;
import jakarta.inject.Inject;
import org.bson.Document;
import org.bson.UuidRepresentation;
import org.bson.codecs.*;
import org.bson.types.ObjectId;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
import org.skyscreamer.jsonassert.JSONAssert;

import java.util.List;

import static java.util.Arrays.asList;
import static org.bson.codecs.configuration.CodecRegistries.fromProviders;
import static org.bson.codecs.configuration.CodecRegistries.withUuidRepresentation;
import static org.junit.jupiter.api.Assertions.*;

@QuarkusTest
public class ParentRepositoryTest {
    private static final Codec<Document> DEFAULT_CODEC = withUuidRepresentation(fromProviders(asList(new ValueCodecProvider(),
            new CollectionCodecProvider(), new IterableCodecProvider(),
            new BsonValueCodecProvider(), new DocumentCodecProvider(),
            new DBRefCodecProvider(), // FIX here
            new MapCodecProvider())), UuidRepresentation.STANDARD).get(Document.class);
    @Inject
    ParentRepository parentRepository;
    @Inject
    MongoClient mongoClient;

    @Test
    public void shouldStoreChild() {
        assertDoesNotThrow(() -> {
            final Child child = new Child();
            parentRepository.store(child);
        });
    }

    @Test
    public void shouldStoreParent() {
        assertDoesNotThrow(() -> {
            final Parent parent = new Parent();
            parentRepository.store(parent);
        });
    }

    @Test
    public void shouldStoreParentWithChild() {
        final Child child = new Child(new ObjectId("65fd462a48d840308ed68001"));
        parentRepository.store(child);
        final Parent parent = new Parent(
                new ObjectId("65fd461548d840308ed68000"),
                child);
        parentRepository.store(parent);

        assertAll(
                () -> JSONAssert.assertEquals(
                        //language=json
                        """
                                {
                                    "_id": {
                                        "$oid": "65fd462a48d840308ed68001"
                                    }
                                }
                                """,
                        mongoClient.getDatabase("sample").getCollection("child").find().first().toJson(DEFAULT_CODEC),
                        true
                ),
                () -> JSONAssert.assertEquals(
                        //language=json
                        """
                                {
                                    "_id": {
                                        "$oid": "65fd461548d840308ed68000"
                                    },
                                    "child": {
                                        "$ref": "child",
                                        "$id": {
                                            "$oid": "65fd462a48d840308ed68001"
                                        }
                                    }
                                }
                                """,
                        mongoClient.getDatabase("sample").getCollection("parent").find().first().toJson(DEFAULT_CODEC),
                        true
                )
        );
    }

    @Test
    public void shouldQueryParentHavingSpecificChild() {
        final Child child = new Child(new ObjectId("65fd462a48d840308ed68001"));
        parentRepository.store(child);
        final Parent parent = new Parent(
                new ObjectId("65fd461548d840308ed68000"),
                child);
        parentRepository.store(parent);

        assertDoesNotThrow(() -> {
            parentRepository.findByChildId(new ObjectId("65fd462a48d840308ed68001"));
        });
                assertNotNull(parentRepository.findByChildId(new ObjectId("65fd462a48d840308ed68001")));

    }

    @AfterEach
    public void tearDown() {
        List.of("child", "parent")
                .forEach(collection -> mongoClient.getDatabase("sample").getCollection(collection)
                        .deleteMany(Filters.empty()));
    }
}

quarkus.mongodb.database=sample
quarkus.morphia.database=sample
quarkus.morphia.packages=org.acme
quarkus.morphia.map-sub-packages=true

<?xml version="1.0"?>
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <modelVersion>4.0.0</modelVersion>
  <groupId>org.acme</groupId>
  <artifactId>morphiua-sample-parent-children</artifactId>
  <version>1.0.0-SNAPSHOT</version>
  <properties>
    <compiler-plugin.version>3.12.1</compiler-plugin.version>
    <maven.compiler.release>17</maven.compiler.release>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
    <quarkus.platform.artifact-id>quarkus-bom</quarkus.platform.artifact-id>
    <quarkus.platform.group-id>io.quarkus.platform</quarkus.platform.group-id>
    <quarkus.platform.version>3.8.3</quarkus.platform.version>
    <skipITs>true</skipITs>
    <surefire-plugin.version>3.2.5</surefire-plugin.version>
  </properties>
  <dependencyManagement>
    <dependencies>
      <dependency>
        <groupId>${quarkus.platform.group-id}</groupId>
        <artifactId>${quarkus.platform.artifact-id}</artifactId>
        <version>${quarkus.platform.version}</version>
        <type>pom</type>
        <scope>import</scope>
      </dependency>
    </dependencies>
  </dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>io.quarkus</groupId>
      <artifactId>quarkus-arc</artifactId>
    </dependency>
    <dependency>
      <groupId>io.quarkus</groupId>
      <artifactId>quarkus-resteasy-reactive</artifactId>
    </dependency>
      <dependency>
          <groupId>io.quarkiverse.morphia</groupId>
          <artifactId>quarkus-morphia</artifactId>
          <version>0.5.0</version>
      </dependency>
      <dependency>
          <groupId>org.skyscreamer</groupId>
          <artifactId>jsonassert</artifactId>
          <version>1.5.1</version>
          <scope>test</scope>
      </dependency>
    <dependency>
      <groupId>io.quarkus</groupId>
      <artifactId>quarkus-junit5</artifactId>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>io.rest-assured</groupId>
      <artifactId>rest-assured</artifactId>
      <scope>test</scope>
    </dependency>
  </dependencies>
  <build>
    <plugins>
      <plugin>
        <groupId>${quarkus.platform.group-id}</groupId>
        <artifactId>quarkus-maven-plugin</artifactId>
        <version>${quarkus.platform.version}</version>
        <extensions>true</extensions>
        <executions>
          <execution>
            <goals>
              <goal>build</goal>
              <goal>generate-code</goal>
              <goal>generate-code-tests</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
      <plugin>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>${compiler-plugin.version}</version>
        <configuration>
          <compilerArgs>
            <arg>-parameters</arg>
          </compilerArgs>
        </configuration>
      </plugin>
      <plugin>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>${surefire-plugin.version}</version>
        <configuration>
          <systemPropertyVariables>
            <java.util.logging.manager>org.jboss.logmanager.LogManager</java.util.logging.manager>
            <maven.home>${maven.home}</maven.home>
          </systemPropertyVariables>
        </configuration>
      </plugin>
      <plugin>
        <artifactId>maven-failsafe-plugin</artifactId>
        <version>${surefire-plugin.version}</version>
        <executions>
          <execution>
            <goals>
              <goal>integration-test</goal>
              <goal>verify</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <systemPropertyVariables>
            <native.image.path>${project.build.directory}/${project.build.finalName}-runner</native.image.path>
            <java.util.logging.manager>org.jboss.logmanager.LogManager</java.util.logging.manager>
            <maven.home>${maven.home}</maven.home>
          </systemPropertyVariables>
        </configuration>
      </plugin>
    </plugins>
  </build>
  <profiles>
    <profile>
      <id>native</id>
      <activation>
        <property>
          <name>native</name>
        </property>
      </activation>
      <properties>
        <skipITs>false</skipITs>
        <quarkus.package.type>native</quarkus.package.type>
      </properties>
    </profile>
  </profiles>
</project>

[dcdh]

This exception occurs at the same place in PathTarget

                if (hasNext() && property.isReference()) {
                    failValidation(segment);
                }

https://github.com/MorphiaOrg/morphia/blob/r2.3.4/core/src/main/java/dev/morphia/internal/PathTarget.java#L175

Quarkus is using version 2.3.4 of Morphia.

The check on reference was not present in version 1.3.2

https://github.com/MorphiaOrg/morphia/blob/r1.3.2/morphia/src/main/java/org/mongodb/morphia/internal/PathTarget.java#L116

[dcdh]

Hope you enjoyed well your travel :)

In fact I went further regarding this issue. It is more broader than index because it impact executing query on dbref fields. In this sample I am no more able to filter on child.$id

So this issue impact:

• indexation of document using dbref;
• filtering on documents using dbref.

Source of this issue located in PathTarget.

[evanchooly]

correct. you wouldn't query against child.$id, you'd just query against child. The fact that the structure in the database is, potentially, a DBRef should not factor in to your queries. You're querying against the child property regardless of how the reference is stored in the database.

[dcdh]

Thanks for your reply.

I've just updated the test.
By changing the method findByChildId to get the child to this one:

    public Parent findByChildId(final ObjectId objectId) {
        return store.find(Parent.class).disableValidation()
                .filter(Filters.eq("child", objectId))
                .first();
    }

No more exception. But the method now return null :(
How should I write it ?

[dcdh]

Ok I need to initialize a Child Object with only the ObjectId and it is working.

Done this way:

    public Parent findByChildId(final ObjectId objectId) {
        return store.find(Parent.class).disableValidation()
                .filter(Filters.eq("child", new Child(objectId)))
                .first();
    }

Can it be possible to update the documentation ?

Many thanks.

[evanchooly]

That query should work with just the ID, but I'm not in front of the code to dig into it just now.

[vsevel]

hello @evanchooly , any news on this?