[BUG] IK Analyzer missing in ElasticSearch started by run-es #672
Description
Describe the bug
The application requires the IK Analyzer plugin for text analysis in ElasticSearch. However, the ElasticSearch instance started using the run-es command does not include the IK Analyzer plugin, leading to issues when attempting to use features that depend on it.
Reproduce Step
Create any collection
Celery Error Log
[2025-04-07 07:06:16,704: INFO/MainProcess] Task aperag.tasks.collection.init_collection_task[ac7e4599-a638-48fb-8846-55ba93fdf0c3] received
[2025-04-07 07:06:17,314: INFO/MainProcess] HTTP Request: GET http://127.0.0.1:6333/collections/colf89336ef549acbd2 "HTTP/1.1 404 Not Found"
[2025-04-07 07:06:17,319: INFO/MainProcess] HTTP Request: DELETE http://127.0.0.1:6333/collections/colf89336ef549acbd2 "HTTP/1.1 200 OK"
[2025-04-07 07:06:17,411: INFO/MainProcess] HTTP Request: PUT http://127.0.0.1:6333/collections/colf89336ef549acbd2 "HTTP/1.1 200 OK"
[2025-04-07 07:06:17,432: INFO/MainProcess] HTTP Request: GET http://127.0.0.1:6333/collections/colf89336ef549acbd2-qa "HTTP/1.1 404 Not Found"
[2025-04-07 07:06:17,435: INFO/MainProcess] HTTP Request: DELETE http://127.0.0.1:6333/collections/colf89336ef549acbd2-qa "HTTP/1.1 200 OK"
[2025-04-07 07:06:17,542: INFO/MainProcess] HTTP Request: PUT http://127.0.0.1:6333/collections/colf89336ef549acbd2-qa "HTTP/1.1 200 OK"
[2025-04-07 07:06:17,625: INFO/MainProcess] HEAD http://localhost:9200/colf89336ef549acbd2 [status:404 duration:0.082s]
[2025-04-07 07:06:17,716: INFO/MainProcess] PUT http://localhost:9200/colf89336ef549acbd2 [status:400 duration:0.091s]
[2025-04-07 07:06:17,726: ERROR/MainProcess] Task aperag.tasks.collection.init_collection_task[ac7e4599-a638-48fb-8846-55ba93fdf0c3] raised unexpected: UnpickleableExceptionWrapper('elasticsearch', 'BadRequestError', ('mapper_parsing_exception',), "BadRequestError('mapper_parsing_exception', meta=ApiResponseMeta(status=400, http_version='1.1', headers={'X-elastic-product': 'Elasticsearch', 'content-type': 'application/vnd.elasticsearch+json;compatible-with=8', 'content-length': '410'}, duration=0.09117984771728516, node=NodeConfig(scheme='http', host='localhost', port=9200, path_prefix='', headers={'user-agent': 'elasticsearch-py/8.11.1 (Python/3.11.9; elastic-transport/8.11.0)'}, connections_per_node=10, request_timeout=10.0, http_compress=False, verify_certs=True, ca_certs=None, client_cert=None, client_key=None, ssl_assert_hostname=None, ssl_assert_fingerprint=None, ssl_version=None, ssl_context=None, ssl_show_warn=True, _extras={})), body={'error': {'root_cause': [{'type': 'mapper_parsing_exception', 'reason': 'Failed to parse mapping: analyzer [ik_smart] has not been configured in mappings'}], 'type': 'mapper_parsing_exception', 'reason': 'Failed to parse mapping: analyzer [ik_smart] has not been configured in mappings', 'caused_by': {'type': 'illegal_argument_exception', 'reason': 'analyzer [ik_smart] has not been configured in mappings'}}, 'status': 400})")
Traceback (most recent call last):
File "/Users/earayu/Library/Caches/pypoetry/virtualenvs/aperag-1nZxLWz5-py3.11/lib/python3.11/site-packages/celery/app/trace.py", line 477, in trace_task
R = retval = fun(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^
File "/Users/earayu/Library/Caches/pypoetry/virtualenvs/aperag-1nZxLWz5-py3.11/lib/python3.11/site-packages/celery/app/trace.py", line 760, in __protected_call__
return self.run(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/earayu/Documents/GitHub/apecloud/ApeRAG/aperag/tasks/collection.py", line 59, in init_collection_task
create_index(index_name)
File "/Users/earayu/Documents/GitHub/apecloud/ApeRAG/aperag/context/full_text.py", line 101, in create_index
es.indices.create(index=index, body={"mappings": mapping})
File "/Users/earayu/Library/Caches/pypoetry/virtualenvs/aperag-1nZxLWz5-py3.11/lib/python3.11/site-packages/elasticsearch/_sync/client/utils.py", line 402, in wrapped
return api(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^
File "/Users/earayu/Library/Caches/pypoetry/virtualenvs/aperag-1nZxLWz5-py3.11/lib/python3.11/site-packages/elasticsearch/_sync/client/indices.py", line 493, in create
return self.perform_request( # type: ignore[return-value]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/earayu/Library/Caches/pypoetry/virtualenvs/aperag-1nZxLWz5-py3.11/lib/python3.11/site-packages/elasticsearch/_sync/client/_base.py", line 389, in perform_request
return self._client.perform_request(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/earayu/Library/Caches/pypoetry/virtualenvs/aperag-1nZxLWz5-py3.11/lib/python3.11/site-packages/elasticsearch/_sync/client/_base.py", line 320, in perform_request
raise HTTP_EXCEPTIONS.get(meta.status, ApiError)(
celery.utils.serialization.UnpickleableExceptionWrapper: BadRequestError('mapper_parsing_exception', meta=ApiResponseMeta(status=400, http_version='1.1', headers={'X-elastic-product': 'Elasticsearch', 'content-type': 'application/vnd.elasticsearch+json;compatible-with=8', 'content-length': '410'}, duration=0.09117984771728516, node=NodeConfig(scheme='http', host='localhost', port=9200, path_prefix='', headers={'user-agent': 'elasticsearch-py/8.11.1 (Python/3.11.9; elastic-transport/8.11.0)'}, connections_per_node=10, request_timeout=10.0, http_compress=False, verify_certs=True, ca_certs=None, client_cert=None, client_key=None, ssl_assert_hostname=None, ssl_assert_fingerprint=None, ssl_version=None, ssl_context=None, ssl_show_warn=True, _extras={})), body={'error': {'root_cause': [{'type': 'mapper_parsing_exception', 'reason': 'Failed to parse mapping: analyzer [ik_smart] has not been configured in mappings'}], 'type': 'mapper_parsing_exception', 'reason': 'Failed to parse mapping: analyzer [ik_smart] has not been configured in mappings', 'caused_by': {'type': 'illegal_argument_exception', 'reason': 'analyzer [ik_smart] has not been configured in mappings'}}, 'status': 400})