[PERF] Improve persistence storage performance (#5217)

# Pull Request Template
<!-- Please include a summary of the changes and the related issue.
Please also include relevant motivation and context. List any
dependencies that are required for this change. -->

Closes #<issue_number>

**Type of change**
<!-- Please delete options that are not relevant. Remember to title the
PR according to the type of change -->

- Bug fix (non-breaking change which fixes an issue)
- New feature (non-breaking change which adds functionality)
- Breaking change (fix or feature that would cause existing
functionality to not work as expected)
- Refactor (change restructuring the codebase without changing
functionality)
- Improvement (change adding some improvement to an existing
functionality)
- Documentation update

**How Has This Been Tested**
<!-- Please add some reference about how your feature has been tested.
-->

**Checklist**
<!-- Please go over the list and make sure you've taken everything into
account -->

- I added relevant documentation
- I followed the style guidelines of this project
- I did a self-review of my code
- I made corresponding changes to the documentation
- I confirm My changes generate no new warnings
- I have added tests that prove my fix is effective or that my feature
works
- I have added relevant notes to the CHANGELOG.md file (See
https://keepachangelog.com/)

Author: frascuchon

argilla-server/docker/quickstart/Dockerfile

@@ -5,6 +5,12 @@ FROM ${ARGILLA_SERVER_IMAGE}:${ARGILLA_VERSION}
 
 USER root
 
+# Copy Argilla distribution files
+COPY scripts/start_quickstart_argilla.sh /home/argilla
+COPY scripts/start_argilla_server.sh /home/argilla
+COPY Procfile /home/argilla
+COPY requirements.txt /packages/requirements.txt
+
 RUN apt-get update && apt-get install -y \
     apt-transport-https \
     gnupg \
@@ -16,12 +22,6 @@ RUN wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | gpg --dearmo
 # Add Elasticsearch repository
 RUN echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" | tee /etc/apt/sources.list.d/elastic-8.x.list
 
-# Copy Argilla distribution files
-COPY scripts/start_quickstart_argilla.sh /home/argilla
-COPY scripts/start_argilla_server.sh /home/argilla
-COPY Procfile /home/argilla
-COPY requirements.txt /packages/requirements.txt
-
 RUN \
     # Indicate that this is a quickstart deployment
     echo -e "{  \"deployment\":  \"quickstart\" }" > /opt/venv/lib/python3.10/site-packages/argilla_server/static/deployment.json && \
@@ -70,6 +70,4 @@ ENV ARGILLA_WORKSPACE=$ADMIN_USERNAME
 
 ENV UVICORN_PORT=6900
 
-ENV REINDEX_DATASETS=0
-
 CMD ["/bin/bash", "start_quickstart_argilla.sh"]

argilla-server/docker/quickstart/README.md

@@ -132,8 +132,3 @@ This will run the latest quickstart docker image with 3 users `owner`, `admin`,
 - `ANNOTATOR_PASSWORD`: This sets a custom password for login into the app with the `argilla` username. The default password
   is `12345678`. By setting up a custom password you can use your own password to login into the app.
 - `ARGILLA_WORKSPACE`: The name of a workspace that will be created and used by default for admin and annotator users. The default value will be the one defined by `ADMIN_USERNAME` environment variable.
-- `LOAD_DATASETS`: This variables will allow you to load sample datasets. The default value will be `full`. The
-  supported values for this variable is as follows:
-    1. `single`: Load single datasets for Feedback task.
-    2. `full`: Load all the sample datasets for NLP tasks (Feedback, TokenClassification, TextClassification, Text2Text)
-    3. `none`: No datasets being loaded.

argilla-server/docker/quickstart/config/elasticsearch.yml

@@ -1,6 +1,7 @@
 cluster.name: "docker-cluster"
 network.host: 0.0.0.0
-path.data: "/data/elasticsearch"
+path.data: "/usr/share/elasticsearch/data"
+path.logs: "/usr/share/elasticsearch/logs"
 discovery.type: single-node
 xpack.security.enabled: false
 xpack.security.transport.ssl.enabled: false

argilla-server/docker/quickstart/scripts/start_argilla_server.sh

@@ -31,10 +31,9 @@ python -m argilla_server database users create \
 	--role annotator \
 	--workspace "$ARGILLA_WORKSPACE"
 
-if [ "$REINDEX_DATASETS" == "true" ] || [ "$REINDEX_DATASETS" == "1" ]; then
-  echo "Reindexing existing datasets"
-  python -m argilla_server search-engine reindex
-fi
+# Forcing reindex on restart since elasticsearch data could be allocated in a non-persistent volume
+echo "Reindexing existing datasets"
+python -m argilla_server search-engine reindex
 
 # Start Argilla
 echo "Starting Argilla"