-
-
Notifications
You must be signed in to change notification settings - Fork 958
feat(doctrine): uuid filter #7628
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
f80b922 to
7f6964d
Compare
soyuka
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice, also note I'm working on a range filter that can be used with many different types (soyuka@c74675d)
b7e3778 to
100a022
Compare
|
|
||
| public function getSchema(Parameter $parameter): array | ||
| { | ||
| return self::UUID_SCHEMA; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
API Platform not automatically adds a Symfony Uuid/Ulid constraint, I could create another PR for that
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| return $databaseValues; | ||
| } | ||
|
|
||
| return $values; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we should not need this this is done by Doctrine when applying parameters no? If this is really needed could you tell me why? Also you could have a guard clause in the callee and reduce the function complexity.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Doctrine handles type conversion automatically when you set the third argument in setParameter(). However, if the type is not explicitly set, UUIDs are treated as strings by default. Depending on the database platform, this can cause issues because if the platform natively supports UUIDs, the value should be converted to a binary representation.
Doctrine Documentation:
Calling setParameter() automatically infers which type you are setting as value.
This works for integers, arrays of strings/integers, DateTime instances, and for managed entities. If you want to set a type explicitly, you can call the third argument to setParameter() explicitly. It accepts either a DBALDoctrine\DBAL\ParameterType::*or a DBAL Type name for conversion.
Symfony Documentation:
When using built-in Doctrine repository methods (e.g.
findOneBy()), Doctrine knows how to convert these UUID types to build the SQL query. However, when using DQL queries or building the query yourself, you'll need to setuuidas the type of the UUID parameters.
In the application of fixture:
We have the configuration in config_common.yml:
https://github.com/aaa2000/core/blob/b8397aeec444d94d34e3631961e693560622a933/tests/Fixtures/app/config/config_common.yml#L12
doctrine:
dbal:
driver: 'pdo_sqlite'
charset: 'UTF8'
types:
uuid: Ramsey\Uuid\Doctrine\UuidType
uuid_binary: Ramsey\Uuid\Doctrine\UuidBinaryType
symfony_uuid: Symfony\Bridge\Doctrine\Types\UuidTypeSo, the third argument should be uuid or uuid_binary or symfony_uuid, and the array of uuid strings should be transformed into an array of objects (Symfony uuid, Ramsey uuid...). Note: we can use the convertToPHPValue method of the Doctrine type for this https://github.com/symfony/symfony/blob/8.0/src/Symfony/Bridge/Doctrine/Types/AbstractUidType.php#L43
For WHERE ... IN() clauses, we cannot set the parameter type with uuid or uuid_binary or symfony_uuid. ArrayParameterType only supports INTEGER, STRING, ASCII, and BINARY.
UUIDs are stored as CHAR(36) by default, or in a platform-specific datatype if the database supports native UUIDs. PostgreSQL and SQL Server have native UUID support: PostgreSQLPlatform, SQLServerPlatform See also Doctrine DBAL: Mapping Matrix
For this reason, it seemed simpler to convert the UUIDs into their database representation.
Note that Symfony does the same thing in ORMQueryBuilderLoader https://github.com/symfony/symfony/blob/f31baa789d552e6cd3b349ea35aad6b67508a599/src/Symfony/Bridge/Doctrine/Form/ChoiceList/ORMQueryBuilderLoader.php#L70
|
|
||
| $values = (array) $value; | ||
| $associations = []; | ||
| if ($this->isPropertyNested($property, $resourceClass)) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
At some point I'd like to share the relation filtering logic into other filters, I'm still wondering how we'd make this good enough.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm really interested by the nested property filters soyuka so I would happy to help.
cf #7554 (comment) Currently I have a hack NestedFilter used this way
'target' => new QueryParameter(filter: new NestedFilter(new ExactFilter()), property: 'languagePair.target'),
but I think it would be awesome to support it natively.
Looking at your comments
- Filtering on Nested Properties does not work with QueryParameter #7526 (comment)
- feat(doctrine): new search filters #7121 (comment)
I thought that you didn't want to support nested properties. So I'd like to understand what changed in this AbstractUuidFilter ? If it's supported for AbstractUuidFilter can it be for ExactFilter and PartialSearchFilter ? If not, why the bug you wanted to avoid on those filter cannot happen on this filter ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the bug is about the nested property not being of the type of the filter (main issue with the SearchFilter). We're working with @Maxcastel on providing the update for relations and nested props on all the filters. Can you show your NestedFilter ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We're working with @Maxcastel on providing the update for relations and nested props on all the filters.
I'm happy to hear it, don't hesitate to ping me on your discussion if I can help :)
Can you show your NestedFilter ?
I'm not sure my NestedFilter will help cause I think it's kinda hacky but I can still share it
use OrmPropertyHelperTrait;
use PropertyHelperTrait;
public function __construct(
private readonly FilterInterface $filter,
) {
}
/**
* @param array<string, mixed> $context
*/
public function apply(QueryBuilder $queryBuilder, QueryNameGeneratorInterface $queryNameGenerator, string $resourceClass, ?Operation $operation = null, array $context = []): void
{
if ($this->filter instanceof ManagerRegistryAwareInterface) {
$this->filter->setManagerRegistry($this->getManagerRegistry());
}
if ($this->filter instanceof LoggerAwareInterface) {
$this->filter->setLogger($this->getLogger());
}
/** @var Parameter $parameter */
$parameter = $context['parameter'];
$property = $parameter->getProperty();
if (null === $property) {
throw new \InvalidArgumentException('Nested filter only supports single property.');
}
[$alias, $property] = $this->resolveNestedJoins(
$queryBuilder,
$queryNameGenerator,
$property,
$resourceClass,
);
$qb = clone $queryBuilder;
$qb->resetDQLPart('from');
$qb->from(self::class, $alias); // Override the root alias of the queryBuilder
$qb->resetDQLPart('where');
$qb->setParameters(new ArrayCollection());
$newParameter = $parameter->withProperty($property)->withProperties([$property]);
$this->filter->apply(
$qb,
$queryNameGenerator,
$resourceClass,
$operation,
['parameter' => $newParameter] + $context
);
$queryBuilder->andWhere($qb->getDQLPart('where'));
$parameters = $queryBuilder->getParameters();
foreach ($qb->getParameters() as $p) {
$parameters->add($p);
}
$queryBuilder->setParameters($parameters);
}
and then I'm using it with Filter which does not supports nested property, this way
'target' => new QueryParameter(filter: new NestedFilter(new ExactFilter()), property: 'languagePair.target'),
b8397ae to
5cb992d
Compare
soyuka
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you target main ?
| } | ||
|
|
||
| $values = $this->convertValuesToTheDatabaseRepresentationIfNecessary($queryBuilder, $doctrineTypeField, $values); | ||
| $this->addWhere($queryBuilder, $queryNameGenerator, $associationAlias, $associationField, $values); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| $this->addWhere($queryBuilder, $queryNameGenerator, $associationAlias, $associationField, $values); | |
| if ($values) { | |
| $this->addWhere($queryBuilder, $queryNameGenerator, $associationAlias, $associationField, $values); | |
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This cannot be done because if there are multiple invalid UUIDs, there is no filter and all resources are fetched. So that doesn't seem okay to me.
If I add automatically a Symfony Uuid/Ulid constraint in ParameterValidationConstraints or if I rethrow the ConversionException, I could add it.
@see testSearchFilterOnManyToOneRelationByInvalidUuids
5cb992d to
80c9d45
Compare
80c9d45 to
a6b7796
Compare
| "illuminate/support": "^11.0 || ^12.0", | ||
| "jangregor/phpstan-prophecy": "^2.1.11", | ||
| "justinrainbow/json-schema": "^5.2.11", | ||
| "justinrainbow/json-schema": "^6.5.2", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
cherry pick of #7619 to fix the behat tests on main branch
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've merged 4.2 onto main just now
4733b7d to
3719b08
Compare
soyuka
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice work! a few small comments, let me know if you think its fine like this, I'm concerned about the addWhere being called with a null value.
110cbaf to
860bbed
Compare
|
@soyuka I updated the PR, I throw an InvalidArgumentException if there is a conversion exception. When the uuid is invalid, the response is now a bad request response instead of a success response with 0 items. |
a2b33c1 to
9f982d1
Compare
|
Thanks! I should've been more specific about the schema/openapi, I took the liberty of adding a new commit for this at Could you tell me if you're fine with my changes? |
177fc5f to
b4fd07a
Compare
b4fd07a to
1b68192
Compare
|
@soyuka Yes, everything looks good to me. Thanks for addressing this! The jod |
|
thanks a lot for tackling this issue! thats lots of PR actually to close :D next big work on filters is continuing soyuka@c74675d that way we'll have uuid ranges available :) (feel free to grab the work if you're interested) |
Add Symfony uuid, Ramsey uuid and Ramsey uuid binary filters.
The PR avoids modifying the search filter to prevent regressions. Furthermore, supporting partial strategies for UUID requires more work. In PostgreSQL, I think we need to cast the UUID to a string in the SQL query.
Possible improvements: