Geolocation 'search_phase_execution_exception' error 'failed to find geo field [location]')

[millerf]

A bit of background

I am merely trying to run a request that will let me get some documents are located around a certain point. I keep getting that error from the dataset, and the issue eludes me...

I have been stuck on this issue for a while. Tried changing versions for each of the packages but to no avail..

Versions used

Django==4.1
django-elasticsearch-dsl==7.2.2
django-elasticsearch-dsl-drf==0.22.5
djangorestframework==3.13.1
elasticsearch==7.17.4
elasticsearch-dsl==7.3.0

Elastic search running in docker

  elasticsearch:
    restart: always
    image: elasticsearch:7.17.4
    environment:
      - discovery.type=single-node
      - xpack.security.enabled=false

    env_file: .env.local
    volumes:
        - /usr/share/elasticsearch/data
    ports:
      - "9200:9200"

Documents and datasets

I am removing as much useless information as possible here:

Document

@registry.register_document
class PlaceDocument(Document):
    class Index:
        name = 'places'
        settings = {
            'number_of_shards': 1,
            'number_of_replicas': 1
        }

    location = fields.GeoPointField(attr='location_field_indexing')

    class Django:
         model = Place
         fields = [
             'id',
         ]
         related_models = [Service]  # Optional: to ensure the Car will be re-saved when Service is updated

    def get_instances_from_related(self, related_instance):
        """If related_models is set, define how to retrieve the Car instance(s) from the related model.
        The related_models option should be used with caution because it can lead in the index
        to the updating of a lot of items.
        """
        if isinstance(related_instance, Service):
            return related_instance.place

Serializer

class PlaceDocumentSimpleSerializer(DocumentSerializer):
    """Serializer for the Place document."""

    class Meta:
        """Meta options."""

        document = PlaceDocument
        fields = (
            'id',
            'location'
        )

View

class PlaceDocumentView(DocumentViewSet):
    """The PlaceDocumentView view."""

    document = PlaceDocument
    serializer_class = PlaceDocumentSimpleSerializer
    lookup_field = 'id'
    filter_backends = [
        `GeoSpatialFilteringFilterBackend`,
    ]
    
    # Define geo-spatial filtering fields
    geo_spatial_filter_fields = {
        'location': {
            'lookups': [
                LOOKUP_FILTER_GEO_DISTANCE,
            ],
        },
    }

Results

I am sure the indexing is done properly: when removing the GeoSpatialFilteringFilterBackend from the filter_backends, the request always returns a list of objects such as

[OrderedDict([('id', 1), ('location', {'lat': 1.0, 'lon': 1.0})])]

But when GeoSpatialFilteringFilterBackend is active I get

RequestError(400, 'search_phase_execution_exception', 'failed to find geo field [location]')

Possible solution

I am wondering if it is a version mismatch, but as I use Django 4, django-elasticsearch-dsl-drf seems to pass the tests (see this PR), and this is the only version compatible with django-elasticsearch-dsl....

Can anyone guide me towards some leads? I am not a ES guru and I have no idea how to proceed from there....

I tried going down to ES6 but nothing... And again, I am no sure how is DSL version-compatible....

BTW, it looks like the publisher view from the docs is also missing GeoSpatialFilteringFilterBackend

Thanks!

[millerf]

Issue 300!!

[millerf]

Not working either with django==3.2.15

[barseghyanartur]

Does example project work for you? There are geo fields there.

[millerf]

Does example project work for you? There are geo fields there.

The live version does not have any data and the instructions for the local version seem outdated... I am also in a region with very bad internet and I struggle to download new versions of ES :(

[millerf]

I have issues running the example locally:

Installing env

I did:

$> pip install -r examples/requirements/django_3_2_and_elastic_7x.txt && docker pull elasticsearch:7.17.4 && docker run -p 9200:9200 -e "discovery.type=single-node" -e "xpack.security.enabled=false" elasticsearch:7.17.4

(using a ES version I already have downloaded)

Creating test data:

I had to create this folder:

$> mkdir /examples/logs

Then I had to install pip install django_elasticsearch_dsl_drf

$>  pip install django_elasticsearch_dsl_drf

Then running

./scripts/create_test_data.sh
Traceback (most recent call last):
  File "./manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "django-elasticsearch-dsl-drf-master/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
    utility.execute()
  File "django-elasticsearch-dsl-drf-master/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 413, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "django-elasticsearch-dsl-drf-master/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 257, in fetch_command
    klass = load_command_class(app_name, subcommand)
  File "django-elasticsearch-dsl-drf-master/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 39, in load_command_class
    module = import_module('%s.management.commands.%s' % (app_name, name))
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "django-elasticsearch-dsl-drf-master/examples/simple/books/management/commands/books_create_test_data.py", line 5, in <module>
    import factories
  File "django-elasticsearch-dsl-drf-master/examples/simple/factories/__init__.py", line 1, in <module>
    from .auth_user import *
  File "django-elasticsearch-dsl-drf-master/examples/simple/factories/auth_user.py", line 3, in <module>
    from factory import (
ModuleNotFoundError: No module named 'factory'

And I am stuck here for now....

[millerf]

I found the issue: the mapping is not referenced as geopoint

 curl -XGET 'http://127.0.0.1:9200/_mapping'


{"places":{"mappings":{"properties":{"id":{"type":"long"},"location":{"properties":{"lat":{"type":"float"},"lon":{"type":"float"}}}

Even if the document clearly states

    location = fields.GeoPointField(attr='location_field_indexing')

I'll dig deeper into it.

[millerf]

It looks like the mapping is based on the model, and not on the document.

Because of this:

@property
    def location_field_indexing(self):
        """Location for indexing.
        Used in Elasticsearch indexing/tests of `geo_distance` native filter.
        """
        return {
            'lat': self.latitude,
            'lon': self.longitude,
        }

The mapping is "location":{"properties":{"lat":{"type":"float"},"lon":{"type":"float"}}
instead of "location":{"type":"geopoint"}...

Any idea @barseghyanartur ?

[millerf]

I solved it:
Basing myself on this [django-elasticsearch-dsl])(django-es/django-elasticsearch-dsl#195) issue.

The fix was to prepare the data in the document instead of the model.

In PlaceDocument

   location = fields.GeoPointField()  # Without the `attr` attribute

 def prepare_location(self, instance: Place) -> dict:
        return {
            'lon': instance.longitude,
            'lat': instance.latitude
        }

And remove from the model

@property
def location_field_indexing(self):
  """Location for indexing.
  Used in Elasticsearch indexing/tests of `geo_distance` native filter.
  """
  return {
     'lat': self.latitude,
     'lon': self.longitude,
  }

[millerf]

For the record, the issue seems to also be linked with a bug that does not get the right mappings unless the full index is rebuild.