Automated Integration Test Goldens Update from CI #5800
Google Cloud Build / website-pull-request-nl (datcom-ci)
succeeded
Dec 5, 2025 in 4m 47s
Summary
Build Information
| Trigger | website-pull-request-nl |
| Build | 99be4723-379d-43e1-9d9c-9e702abc9fbd |
| Start | 2025-12-04T18:58:08-08:00 |
| Duration | 3m57.944s |
| Status | SUCCESS |
Steps
| Step | Status | Duration |
|---|---|---|
| setup_python | SUCCESS | 2m14.693s |
| explore_tests | SUCCESS | 1m33.397s |
| nl_tests | SUCCESS | 1m21.843s |
Details
starting build "99be4723-379d-43e1-9d9c-9e702abc9fbd"
FETCHSOURCE
From https://github.com/datacommonsorg/website
* branch b9eb6b58f5f10d6d804f4f7c68078e2f7effbfdb -> FETCH_HEAD
HEAD is now at b9eb6b5 feat: Update goldens from Cloud Build workflow (build e7750a0e-7bd5-448e-93d4-24eb330fcbfa)
GitCommit:
b9eb6b58f5f10d6d804f4f7c68078e2f7effbfdb
BUILD
Starting Step #0 - "setup_python"
Step #0 - "setup_python": Pulling image: python:3.11.3
Step #0 - "setup_python": 3.11.3: Pulling from library/python
Step #0 - "setup_python": bd73737482dd: Pulling fs layer
Step #0 - "setup_python": 6710592d62aa: Pulling fs layer
Step #0 - "setup_python": 75256935197e: Pulling fs layer
Step #0 - "setup_python": c1e5026c6457: Pulling fs layer
Step #0 - "setup_python": f0016544b8b9: Pulling fs layer
Step #0 - "setup_python": 1d58eee51ff2: Pulling fs layer
Step #0 - "setup_python": 93dc7b704cd1: Pulling fs layer
Step #0 - "setup_python": caefdefa531e: Pulling fs layer
Step #0 - "setup_python": f0016544b8b9: Verifying Checksum
Step #0 - "setup_python": f0016544b8b9: Download complete
Step #0 - "setup_python": 6710592d62aa: Verifying Checksum
Step #0 - "setup_python": 6710592d62aa: Download complete
Step #0 - "setup_python": bd73737482dd: Verifying Checksum
Step #0 - "setup_python": bd73737482dd: Download complete
Step #0 - "setup_python": 75256935197e: Verifying Checksum
Step #0 - "setup_python": 75256935197e: Download complete
Step #0 - "setup_python": 93dc7b704cd1: Download complete
Step #0 - "setup_python": caefdefa531e: Verifying Checksum
Step #0 - "setup_python": caefdefa531e: Download complete
Step #0 - "setup_python": 1d58eee51ff2: Verifying Checksum
Step #0 - "setup_python": 1d58eee51ff2: Download complete
Step #0 - "setup_python": c1e5026c6457: Verifying Checksum
Step #0 - "setup_python": c1e5026c6457: Download complete
Step #0 - "setup_python": bd73737482dd: Pull complete
Step #0 - "setup_python": 6710592d62aa: Pull complete
Step #0 - "setup_python": 75256935197e: Pull complete
Step #0 - "setup_python": c1e5026c6457: Pull complete
Step #0 - "setup_python": f0016544b8b9: Pull complete
Step #0 - "setup_python": 1d58eee51ff2: Pull complete
Step #0 - "setup_python": 93dc7b704cd1: Pull complete
Step #0 - "setup_python": caefdefa531e: Pull complete
Step #0 - "setup_python": Digest: sha256:3a619e3c96fd4c5fc5e1998fd4dcb1f1403eb90c4c6409c70d7e80b9468df7df
Step #0 - "setup_python": Status: Downloaded newer image for python:3.11.3
Step #0 - "setup_python": docker.io/library/python:3.11.3
Step #0 - "setup_python": --setup_python ### Set up python environment
Step #0 - "setup_python": installing server/requirements.txt
Step #0 - "setup_python": DEPRECATION: langdetect is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559
Step #0 - "setup_python": DEPRECATION: data-gemma is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559
Step #0 - "setup_python": DEPRECATION: flask_testing is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559
Step #0 - "setup_python":
Step #0 - "setup_python": [notice] A new release of pip available: 22.3.1 -> 25.3
Step #0 - "setup_python": [notice] To update, run: pip install --upgrade pip
Step #0 - "setup_python": Looking in indexes: https://pypi.org/simple, https://download.pytorch.org/whl/cpu
Step #0 - "setup_python": Collecting torch==2.2.2
Step #0 - "setup_python": Downloading https://download.pytorch.org/whl/cpu/torch-2.2.2%2Bcpu-cp311-cp311-linux_x86_64.whl (186.8 MB)
Step #0 - "setup_python": ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 186.8/186.8 MB 11.9 MB/s eta 0:00:00
Step #0 - "setup_python": Collecting filelock
Step #0 - "setup_python": Downloading filelock-3.20.0-py3-none-any.whl (16 kB)
Step #0 - "setup_python": Requirement already satisfied: typing-extensions>=4.8.0 in ./.venv/lib/python3.11/site-packages (from torch==2.2.2) (4.12.2)
Step #0 - "setup_python": Collecting sympy
Step #0 - "setup_python": Obtaining dependency information for sympy from https://download.pytorch.org/whl/sympy-1.14.0-py3-none-any.whl.metadata
Step #0 - "setup_python": Downloading https://download.pytorch.org/whl/sympy-1.14.0-py3-none-any.whl.metadata (12 kB)
Step #0 - "setup_python": Collecting networkx
Step #0 - "setup_python": Downloading networkx-3.6-py3-none-any.whl (2.1 MB)
Step #0 - "setup_python": ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 80.8 MB/s eta 0:00:00
Step #0 - "setup_python": Requirement already satisfied: jinja2 in ./.venv/lib/python3.11/site-packages (from torch==2.2.2) (3.1.6)
Step #0 - "setup_python": Collecting fsspec
Step #0 - "setup_python": Downloading fsspec-2025.12.0-py3-none-any.whl (201 kB)
Step #0 - "setup_python": ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 201.4/201.4 kB 37.3 MB/s eta 0:00:00
Step #0 - "setup_python": Requirement already satisfied: MarkupSafe>=2.0 in ./.venv/lib/python3.11/site-packages (from jinja2->torch==2.2.2) (2.1.2)
Step #0 - "setup_python": Collecting mpmath<1.4,>=1.1.0
Step #0 - "setup_python": Downloading https://download.pytorch.org/whl/mpmath-1.3.0-py3-none-any.whl (536 kB)
Step #0 - "setup_python": ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 536.2/536.2 kB 52.4 MB/s eta 0:00:00
Step #0 - "setup_python": Downloading https://download.pytorch.org/whl/sympy-1.14.0-py3-none-any.whl (6.3 MB)
Step #0 - "setup_python": ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.3/6.3 MB 118.1 MB/s eta 0:00:00
Step #0 - "setup_python": Using cached https://download.pytorch.org/whl/sympy-1.14.0-py3-none-any.whl (6.3 MB)
Step #0 - "setup_python": Installing collected packages: mpmath, sympy, networkx, fsspec, filelock, torch
Step #0 - "setup_python": Successfully installed filelock-3.20.0 fsspec-2025.12.0 mpmath-1.3.0 networkx-3.6 sympy-1.14.0 torch-2.2.2+cpu
Step #0 - "setup_python":
Step #0 - "setup_python": [notice] A new release of pip available: 22.3.1 -> 25.3
Step #0 - "setup_python": [notice] To update, run: pip install --upgrade pip
Step #0 - "setup_python": installing nl_server/requirements.txt
Step #0 - "setup_python":
Step #0 - "setup_python": [notice] A new release of pip available: 22.3.1 -> 25.3
Step #0 - "setup_python": [notice] To update, run: pip install --upgrade pip
Finished Step #0 - "setup_python"
Starting Step #1 - "explore_tests"
Starting Step #2 - "nl_tests"
Step #1 - "explore_tests": Already have image (with digest): python:3.11.3
Step #2 - "nl_tests": Already have image (with digest): python:3.11.3
Step #2 - "nl_tests": --nl ### Running nl page integration tests
Step #2 - "nl_tests": Using ENV_PREFIX=Staging
Step #2 - "nl_tests": Starting servers using run_servers.sh...
Step #2 - "nl_tests": FLASK_ENV=integration_test
Step #2 - "nl_tests": GOOGLE_CLOUD_PROJECT=datcom-website-staging
Step #2 - "nl_tests": ENABLE_MODEL=true
Step #2 - "nl_tests": Starting NL Server...
Step #2 - "nl_tests": Starting Website server...
Step #1 - "explore_tests": --explore ### Running explore page integration tests
Step #1 - "explore_tests": Using ENV_PREFIX=Staging
Step #1 - "explore_tests": Starting servers using run_servers.sh...
Step #1 - "explore_tests": FLASK_ENV=integration_test
Step #1 - "explore_tests": GOOGLE_CLOUD_PROJECT=datcom-website-staging
Step #1 - "explore_tests": ENABLE_MODEL=true
Step #1 - "explore_tests": Starting NL Server...
Step #1 - "explore_tests": Starting Website server...
Step #2 - "nl_tests": [03:00:41][INFO ][config_reader.py:120] Loading index and model catalog from: /workspace/nl_server/../deploy/helm_charts/dc_website/nl/catalog.yaml
Step #2 - "nl_tests": [03:00:41][INFO ][config_reader.py:86] server config:
Step #2 - "nl_tests": {
Step #2 - "nl_tests": "version": "1",
Step #2 - "nl_tests": "default_indexes": [
Step #2 - "nl_tests": "base_uae_mem"
Step #2 - "nl_tests": ],
Step #2 - "nl_tests": "indexes": {
Step #2 - "nl_tests": "base_uae_mem": {
Step #2 - "nl_tests": "store_type": "MEMORY",
Step #2 - "nl_tests": "source_path": "/workspace/tools/nl/embeddings/input/base",
Step #2 - "nl_tests": "model": "uae-large-v1-model",
Step #2 - "nl_tests": "healthcheck_query": "Life expectancy",
Step #2 - "nl_tests": "embeddings_path": "gs://datcom-nl-models/base_uae_mem_2025_11_03_07_10_42/embeddings.csv"
Step #2 - "nl_tests": },
Step #2 - "nl_tests": "bio_ft": {
Step #2 - "nl_tests": "store_type": "MEMORY",
Step #2 - "nl_tests": "source_path": "/workspace/tools/nl/embeddings/input/bio",
Step #2 - "nl_tests": "model": "ft-final-v20230717230459-all-MiniLM-L6-v2",
Step #2 - "nl_tests": "healthcheck_query": "Gene",
Step #2 - "nl_tests": "embeddings_path": "gs://datcom-nl-models/bio_ft_2024_11_08_19_00_38/embeddings.csv"
Step #2 - "nl_tests": },
Step #2 - "nl_tests": "medium_ft": {
Step #2 - "nl_tests": "store_type": "MEMORY",
Step #2 - "nl_tests": "source_path": null,
Step #2 - "nl_tests": "model": "ft-final-v20230717230459-all-MiniLM-L6-v2",
Step #2 - "nl_tests": "healthcheck_query": "Life expectancy",
Step #2 - "nl_tests": "embeddings_path": "gs://datcom-nl-models/embeddings_medium_2024_05_09_18_01_32.ft_final_v20230717230459.all-MiniLM-L6-v2.csv"
Step #2 - "nl_tests": },
Step #2 - "nl_tests": "base_mistral_mem": {
Step #2 - "nl_tests": "store_type": "MEMORY",
Step #2 - "nl_tests": "source_path": "/workspace/tools/nl/embeddings/input/base",
Step #2 - "nl_tests": "model": "sfr-embedding-mistral-model",
Step #2 - "nl_tests": "healthcheck_query": "Life expectancy",
Step #2 - "nl_tests": "embeddings_path": "gs://datcom-nl-models/base_mistral_mem_2024_07_01_10_23_43/embeddings.csv"
Step #2 - "nl_tests": },
Step #2 - "nl_tests": "sdg_ft": {
Step #2 - "nl_tests": "store_type": "MEMORY",
Step #2 - "nl_tests": "source_path": "/workspace/tools/nl/embeddings/input/sdg",
Step #2 - "nl_tests": "model": "ft-final-v20230717230459-all-MiniLM-L6-v2",
Step #2 - "nl_tests": "healthcheck_query": "Hunger",
Step #2 - "nl_tests": "embeddings_path": "gs://datcom-nl-models/sdg_ft_2024_06_24_23_45_46/embeddings.csv"
Step #2 - "nl_tests": },
Step #2 - "nl_tests": "undata_ft": {
Step #2 - "nl_tests": "store_type": "MEMORY",
Step #2 - "nl_tests": "source_path": "/workspace/tools/nl/embeddings/input/undata",
Step #2 - "nl_tests": "model": "ft-final-v20230717230459-all-MiniLM-L6-v2",
Step #2 - "nl_tests": "healthcheck_query": "Hunger",
Step #2 - "nl_tests": "embeddings_path": "gs://datcom-nl-models/undata_ft_2024_06_24_23_47_04/embeddings.csv"
Step #2 - "nl_tests": },
Step #2 - "nl_tests": "undata_ilo_ft": {
Step #2 - "nl_tests": "store_type": "MEMORY",
Step #2 - "nl_tests": "source_path": "/workspace/tools/nl/embeddings/input/undata_ilo",
Step #2 - "nl_tests": "model": "ft-final-v20230717230459-all-MiniLM-L6-v2",
Step #2 - "nl_tests": "healthcheck_query": "Employment",
Step #2 - "nl_tests": "embeddings_path": "gs://datcom-nl-models/undata_ilo_ft_2024_10_14_13_45_50/embeddings.csv"
Step #2 - "nl_tests": }
Step #2 - "nl_tests": },
Step #2 - "nl_tests": "models": {
Step #2 - "nl_tests": "cross-encoder-ms-marco-miniilm-l6-v2": {
Step #2 - "nl_tests": "type": "VERTEXAI",
Step #2 - "nl_tests": "usage": "RERANKING",
Step #2 - "nl_tests": "score_threshold": null,
Step #2 - "nl_tests": "project_id": "datcom-website-dev",
Step #2 - "nl_tests": "location": "us-central1",
Step #2 - "nl_tests": "prediction_endpoint_id": "3977846152316846080"
Step #2 - "nl_tests": },
Step #2 - "nl_tests": "cross-encoder-mxbai-rerank-base-v1": {
Step #2 - "nl_tests": "type": "VERTEXAI",
Step #2 - "nl_tests": "usage": "RERANKING",
Step #2 - "nl_tests": "score_threshold": null,
Step #2 - "nl_tests": "project_id": "datcom-website-dev",
Step #2 - "nl_tests": "location": "us-central1",
Step #2 - "nl_tests": "prediction_endpoint_id": "284894457873039360"
Step #2 - "nl_tests": },
Step #2 - "nl_tests": "uae-large-v1-model": {
Step #2 - "nl_tests": "type": "VERTEXAI",
Step #2 - "nl_tests": "usage": "EMBEDDINGS",
Step #2 - "nl_tests": "score_threshold": 0.7,
Step #2 - "nl_tests": "project_id": "datcom-nl",
Step #2 - "nl_tests": "location": "us-central1",
Step #2 - "nl_tests": "prediction_endpoint_id": "8110162693219942400"
Step #2 - "nl_tests": },
Step #2 - "nl_tests": "ft-final-v20230717230459-all-MiniLM-L6-v2": {
Step #2 - "nl_tests": "type": "LOCAL",
Step #2 - "nl_tests": "usage": "EMBEDDINGS",
Step #2 - "nl_tests": "score_threshold": 0.5,
Step #2 - "nl_tests": "gcs_folder": "gs://datcom-nl-models/ft_final_v20230717230459.all-MiniLM-L6-v2"
Step #2 - "nl_tests": },
Step #2 - "nl_tests": "sfr-embedding-mistral-model": {
Step #2 - "nl_tests": "type": "VERTEXAI",
Step #2 - "nl_tests": "usage": "EMBEDDINGS",
Step #2 - "nl_tests": "score_threshold": 0.5,
Step #2 - "nl_tests": "project_id": "datcom-website-dev",
Step #2 - "nl_tests": "location": "us-central1",
Step #2 - "nl_tests": "prediction_endpoint_id": "224012300019826688"
Step #2 - "nl_tests": }
Step #2 - "nl_tests": },
Step #2 - "nl_tests": "enable_reranking": true
Step #2 - "nl_tests": }
Step #1 - "explore_tests": [03:00:42][INFO ][config_reader.py:120] Loading index and model catalog from: /workspace/nl_server/../deploy/helm_charts/dc_website/nl/catalog.yaml
Step #1 - "explore_tests": [03:00:42][INFO ][config_reader.py:86] server config:
Step #1 - "explore_tests": {
Step #1 - "explore_tests": "version": "1",
Step #1 - "explore_tests": "default_indexes": [
Step #1 - "explore_tests": "base_uae_mem"
Step #1 - "explore_tests": ],
Step #1 - "explore_tests": "indexes": {
Step #1 - "explore_tests": "base_uae_mem": {
Step #1 - "explore_tests": "store_type": "MEMORY",
Step #1 - "explore_tests": "source_path": "/workspace/tools/nl/embeddings/input/base",
Step #1 - "explore_tests": "model": "uae-large-v1-model",
Step #1 - "explore_tests": "healthcheck_query": "Life expectancy",
Step #1 - "explore_tests": "embeddings_path": "gs://datcom-nl-models/base_uae_mem_2025_11_03_07_10_42/embeddings.csv"
Step #1 - "explore_tests": },
Step #1 - "explore_tests": "bio_ft": {
Step #1 - "explore_tests": "store_type": "MEMORY",
Step #1 - "explore_tests": "source_path": "/workspace/tools/nl/embeddings/input/bio",
Step #1 - "explore_tests": "model": "ft-final-v20230717230459-all-MiniLM-L6-v2",
Step #1 - "explore_tests": "healthcheck_query": "Gene",
Step #1 - "explore_tests": "embeddings_path": "gs://datcom-nl-models/bio_ft_2024_11_08_19_00_38/embeddings.csv"
Step #1 - "explore_tests": },
Step #1 - "explore_tests": "medium_ft": {
Step #1 - "explore_tests": "store_type": "MEMORY",
Step #1 - "explore_tests": "source_path": null,
Step #1 - "explore_tests": "model": "ft-final-v20230717230459-all-MiniLM-L6-v2",
Step #1 - "explore_tests": "healthcheck_query": "Life expectancy",
Step #1 - "explore_tests": "embeddings_path": "gs://datcom-nl-models/embeddings_medium_2024_05_09_18_01_32.ft_final_v20230717230459.all-MiniLM-L6-v2.csv"
Step #1 - "explore_tests": },
Step #1 - "explore_tests": "base_mistral_mem": {
Step #1 - "explore_tests": "store_type": "MEMORY",
Step #1 - "explore_tests": "source_path": "/workspace/tools/nl/embeddings/input/base",
Step #1 - "explore_tests": "model": "sfr-embedding-mistral-model",
Step #1 - "explore_tests": "healthcheck_query": "Life expectancy",
Step #1 - "explore_tests": "embeddings_path": "gs://datcom-nl-models/base_mistral_mem_2024_07_01_10_23_43/embeddings.csv"
Step #1 - "explore_tests": },
Step #1 - "explore_tests": "sdg_ft": {
Step #1 - "explore_tests": "store_type": "MEMORY",
Step #1 - "explore_tests": "source_path": "/workspace/tools/nl/embeddings/input/sdg",
Step #1 - "explore_tests": "model": "ft-final-v20230717230459-all-MiniLM-L6-v2",
Step #1 - "explore_tests": "healthcheck_query": "Hunger",
Step #1 - "explore_tests": "embeddings_path": "gs://datcom-nl-models/sdg_ft_2024_06_24_23_45_46/embeddings.csv"
Step #1 - "explore_tests": },
Step #1 - "explore_tests": "undata_ft": {
Step #1 - "explore_tests": "store_type": "MEMORY",
Step #1 - "explore_tests": "source_path": "/workspace/tools/nl/embeddings/input/undata",
Step #1 - "explore_tests": "model": "ft-final-v20230717230459-all-MiniLM-L6-v2",
Step #1 - "explore_tests": "healthcheck_query": "Hunger",
Step #1 - "explore_tests": "embeddings_path": "gs://datcom-nl-models/undata_ft_2024_06_24_23_47_04/embeddings.csv"
Step #1 - "explore_tests": },
Step #1 - "explore_tests": "undata_ilo_ft": {
Step #1 - "explore_tests": "store_type": "MEMORY",
Step #1 - "explore_tests": "source_path": "/workspace/tools/nl/embeddings/input/undata_ilo",
Step #1 - "explore_tests": "model": "ft-final-v20230717230459-all-MiniLM-L6-v2",
Step #1 - "explore_tests": "healthcheck_query": "Employment",
Step #1 - "explore_tests": "embeddings_path": "gs://datcom-nl-models/undata_ilo_ft_2024_10_14_13_45_50/embeddings.csv"
Step #1 - "explore_tests": }
Step #1 - "explore_tests": },
Step #1 - "explore_tests": "models": {
Step #1 - "explore_tests": "cross-encoder-ms-marco-miniilm-l6-v2": {
Step #1 - "explore_tests": "type": "VERTEXAI",
Step #1 - "explore_tests": "usage": "RERANKING",
Step #1 - "explore_tests": "score_threshold": null,
Step #1 - "explore_tests": "project_id": "datcom-website-dev",
Step #1 - "explore_tests": "location": "us-central1",
Step #1 - "explore_tests": "prediction_endpoint_id": "3977846152316846080"
Step #1 - "explore_tests": },
Step #1 - "explore_tests": "cross-encoder-mxbai-rerank-base-v1": {
Step #1 - "explore_tests": "type": "VERTEXAI",
Step #1 - "explore_tests": "usage": "RERANKING",
Step #1 - "explore_tests": "score_threshold": null,
Step #1 - "explore_tests": "project_id": "datcom-website-dev",
Step #1 - "explore_tests": "location": "us-central1",
Step #1 - "explore_tests": "prediction_endpoint_id": "284894457873039360"
Step #1 - "explore_tests": },
Step #1 - "explore_tests": "uae-large-v1-model": {
Step #1 - "explore_tests": "type": "VERTEXAI",
Step #1 - "explore_tests": "usage": "EMBEDDINGS",
Step #1 - "explore_tests": "score_threshold": 0.7,
Step #1 - "explore_tests": "project_id": "datcom-nl",
Step #1 - "explore_tests": "location": "us-central1",
Step #1 - "explore_tests": "prediction_endpoint_id": "8110162693219942400"
Step #1 - "explore_tests": },
Step #1 - "explore_tests": "ft-final-v20230717230459-all-MiniLM-L6-v2": {
Step #1 - "explore_tests": "type": "LOCAL",
Step #1 - "explore_tests": "usage": "EMBEDDINGS",
Step #1 - "explore_tests": "score_threshold": 0.5,
Step #1 - "explore_tests": "gcs_folder": "gs://datcom-nl-models/ft_final_v20230717230459.all-MiniLM-L6-v2"
Step #1 - "explore_tests": },
Step #1 - "explore_tests": "sfr-embedding-mistral-model": {
Step #1 - "explore_tests": "type": "VERTEXAI",
Step #1 - "explore_tests": "usage": "EMBEDDINGS",
Step #1 - "explore_tests": "score_threshold": 0.5,
Step #1 - "explore_tests": "project_id": "datcom-website-dev",
Step #1 - "explore_tests": "location": "us-central1",
Step #1 - "explore_tests": "prediction_endpoint_id": "224012300019826688"
Step #1 - "explore_tests": }
Step #1 - "explore_tests": },
Step #1 - "explore_tests": "enable_reranking": true
Step #1 - "explore_tests": }
Step #2 - "nl_tests": [03:00:43][INFO ][gcs.py:50] Download datcom-nl-models/ft_final_v20230717230459.all-MiniLM-L6-v2 to /tmp/datcom-nl-models/ft_final_v20230717230459.all-MiniLM-L6-v2
Step #1 - "explore_tests": [03:00:45][INFO ][gcs.py:50] Download datcom-nl-models/ft_final_v20230717230459.all-MiniLM-L6-v2 to /tmp/datcom-nl-models/ft_final_v20230717230459.all-MiniLM-L6-v2
Step #2 - "nl_tests": [03:00:46][INFO ][SentenceTransformer.py:113] Load pretrained SentenceTransformer: /tmp/datcom-nl-models/ft_final_v20230717230459.all-MiniLM-L6-v2
Step #2 - "nl_tests": [03:00:46][INFO ][SentenceTransformer.py:219] Use pytorch device_name: cpu
Step #2 - "nl_tests": [03:00:46][INFO ][gcs.py:50] Download datcom-nl-models/base_uae_mem_2025_11_03_07_10_42/embeddings.csv to /tmp/datcom-nl-models/base_uae_mem_2025_11_03_07_10_42/embeddings.csv
Step #1 - "explore_tests": [03:00:47][INFO ][SentenceTransformer.py:113] Load pretrained SentenceTransformer: /tmp/datcom-nl-models/ft_final_v20230717230459.all-MiniLM-L6-v2
Step #1 - "explore_tests": [03:00:48][INFO ][SentenceTransformer.py:219] Use pytorch device_name: cpu
Step #1 - "explore_tests": [03:00:48][INFO ][gcs.py:50] Download datcom-nl-models/base_uae_mem_2025_11_03_07_10_42/embeddings.csv to /tmp/datcom-nl-models/base_uae_mem_2025_11_03_07_10_42/embeddings.csv
Step #2 - "nl_tests": [03:00:49][INFO ][memory.py:67] Loading embeddings file: /tmp/datcom-nl-models/base_uae_mem_2025_11_03_07_10_42/embeddings.csv
Step #2 - "nl_tests":
Generating train split: 0 examples [00:00, ? examples/s][03:00:49][INFO ][util.py:607] ['http://localhost:6070/healthz'] not ready, waiting for 5 seconds
Step #1 - "explore_tests": [03:00:51][INFO ][memory.py:67] Loading embeddings file: /tmp/datcom-nl-models/base_uae_mem_2025_11_03_07_10_42/embeddings.csv
Step #1 - "explore_tests": [03:00:51][INFO ][util.py:607] ['http://localhost:6070/healthz'] not ready, waiting for 5 seconds
Step #2 - "nl_tests":
Generating train split: 7305 examples [00:04, 1503.81 examples/s]
Generating train split: 7305 examples [00:04, 1502.74 examples/s]
Step #2 - "nl_tests": [03:00:54][INFO ][gcs.py:50] Download datcom-nl-models/bio_ft_2024_11_08_19_00_38/embeddings.csv to /tmp/datcom-nl-models/bio_ft_2024_11_08_19_00_38/embeddings.csv
Step #1 - "explore_tests": [03:00:54][INFO ][gcs.py:50] Download datcom-nl-models/bio_ft_2024_11_08_19_00_38/embeddings.csv to /tmp/datcom-nl-models/bio_ft_2024_11_08_19_00_38/embeddings.csv
Step #2 - "nl_tests": [03:00:54][INFO ][memory.py:67] Loading embeddings file: /tmp/datcom-nl-models/bio_ft_2024_11_08_19_00_38/embeddings.csv
Step #1 - "explore_tests": [03:00:54][INFO ][memory.py:67] Loading embeddings file: /tmp/datcom-nl-models/bio_ft_2024_11_08_19_00_38/embeddings.csv
Step #2 - "nl_tests":
Generating train split: 0 examples [00:00, ? examples/s]
Generating train split: 92 examples [00:00, 1411.93 examples/s]
Step #2 - "nl_tests": [03:00:54][INFO ][util.py:607] ['http://localhost:6070/healthz'] not ready, waiting for 5 seconds
Step #2 - "nl_tests": [03:00:54][INFO ][gcs.py:50] Download datcom-nl-models/embeddings_medium_2024_05_09_18_01_32.ft_final_v20230717230459.all-MiniLM-L6-v2.csv to /tmp/datcom-nl-models/embeddings_medium_2024_05_09_18_01_32.ft_final_v20230717230459.all-MiniLM-L6-v2.csv
Step #1 - "explore_tests": [03:00:55][INFO ][gcs.py:50] Download datcom-nl-models/embeddings_medium_2024_05_09_18_01_32.ft_final_v20230717230459.all-MiniLM-L6-v2.csv to /tmp/datcom-nl-models/embeddings_medium_2024_05_09_18_01_32.ft_final_v20230717230459.all-MiniLM-L6-v2.csv
Step #2 - "nl_tests": ============================= test session starts ==============================
Step #2 - "nl_tests": platform linux -- Python 3.11.3, pytest-9.0.1, pluggy-1.6.0 -- /workspace/.venv/bin/python3
Step #2 - "nl_tests": cachedir: .pytest_cache
Step #2 - "nl_tests": rootdir: /workspace
Step #2 - "nl_tests": configfile: pytest.ini
Step #2 - "nl_tests": plugins: rerunfailures-10.2, flakefinder-1.1.0, xdist-3.2.1, anyio-4.12.0
Step #2 - "nl_tests": gw0 I / gw1 I / gw2 I / gw3 I / gw4 I / gw5 I / gw6 I / gw7 I / gw8 I / gw9 I / gw10 I / gw11 I / gw12 I / gw13 I / gw14 I / gw15 I / gw16 I / gw17 I / gw18 I / gw19 I / gw20 I / gw21 I / gw22 I / gw23 I / gw24 I / gw25 I / gw26 I / gw27 I / gw28 I / gw29 I / gw30 I / gw31 I
Step #2 - "nl_tests":
[gw0] linux Python 3.11.3 cwd: /workspace
Step #2 - "nl_tests":
[gw1] linux Python 3.11.3 cwd: /workspace
Step #2 - "nl_tests":
[gw2] linux Python 3.11.3 cwd: /workspace
Step #2 - "nl_tests":
[gw3] linux Python 3.11.3 cwd: /workspace
Step #2 - "nl_tests":
[gw4] linux Python 3.11.3 cwd: /workspace
Step #2 - "nl_tests":
[gw5] linux Python 3.11.3 cwd: /workspace
Step #2 - "nl_tests":
[gw6] linux Python 3.11.3 cwd: /workspace
Step #2 - "nl_tests":
[gw7] linux Python 3.11.3 cwd: /workspace
Step #2 - "nl_tests":
[gw8] linux Python 3.11.3 cwd: /workspace
Step #2 - "nl_tests":
[gw9] linux Python 3.11.3 cwd: /workspace
Step #2 - "nl_tests":
[gw10] linux Python 3.11.3 cwd: /workspace
Step #2 - "nl_tests":
[gw11] linux Python 3.11.3 cwd: /workspace
Step #2 - "nl_tests":
[gw12] linux Python 3.11.3 cwd: /workspace
Step #2 - "nl_tests":
[gw13] linux Python 3.11.3 cwd: /workspace
Step #2 - "nl_tests":
[gw14] linux Python 3.11.3 cwd: /workspace
Step #2 - "nl_tests":
[gw15] linux Python 3.11.3 cwd: /workspace
Step #2 - "nl_tests":
[gw16] linux Python 3.11.3 cwd: /workspace
Step #2 - "nl_tests":
[gw17] linux Python 3.11.3 cwd: /workspace
Step #2 - "nl_tests":
[gw18] linux Python 3.11.3 cwd: /workspace
Step #2 - "nl_tests":
[gw19] linux Python 3.11.3 cwd: /workspace
Step #2 - "nl_tests":
[gw20] linux Python 3.11.3 cwd: /workspace
Step #2 - "nl_tests":
[gw21] linux Python 3.11.3 cwd: /workspace
Step #2 - "nl_tests":
[gw22] linux Python 3.11.3 cwd: /workspace
Step #2 - "nl_tests":
[gw23] linux Python 3.11.3 cwd: /workspace
Step #2 - "nl_tests":
[gw24] linux Python 3.11.3 cwd: /workspace
Step #2 - "nl_tests":
[gw25] linux Python 3.11.3 cwd: /workspace
Step #2 - "nl_tests":
[gw26] linux Python 3.11.3 cwd: /workspace
Step #2 - "nl_tests":
[gw27] linux Python 3.11.3 cwd: /workspace
Step #2 - "nl_tests":
[gw28] linux Python 3.11.3 cwd: /workspace
Step #2 - "nl_tests":
[gw29] linux Python 3.11.3 cwd: /workspace
Step #2 - "nl_tests":
[gw30] linux Python 3.11.3 cwd: /workspace
Step #2 - "nl_tests":
[gw31] linux Python 3.11.3 cwd: /workspace
Step #2 - "nl_tests":
[gw0] Python 3.11.3 (main, May 23 2023, 13:25:46) [GCC 10.2.1 20210110]
Step #2 - "nl_tests":
[gw1] Python 3.11.3 (main, May 23 2023, 13:25:46) [GCC 10.2.1 20210110]
Step #2 - "nl_tests":
[gw3] Python 3.11.3 (main, May 23 2023, 13:25:46) [GCC 10.2.1 20210110]
Step #2 - "nl_tests":
[gw2] Python 3.11.3 (main, May 23 2023, 13:25:46) [GCC 10.2.1 20210110]
Step #2 - "nl_tests":
[gw4] Python 3.11.3 (main, May 23 2023, 13:25:46) [GCC 10.2.1 20210110]
Step #2 - "nl_tests":
[gw6] Python 3.11.3 (main, May 23 2023, 13:25:46) [GCC 10.2.1 20210110]
Step #2 - "nl_tests":
[gw5] Python 3.11.3 (main, May 23 2023, 13:25:46) [GCC 10.2.1 20210110]
Step #2 - "nl_tests":
[gw7] Python 3.11.3 (main, May 23 2023, 13:25:46) [GCC 10.2.1 20210110]
Step #2 - "nl_tests":
[gw8] Python 3.11.3 (main, May 23 2023, 13:25:46) [GCC 10.2.1 20210110]
Step #2 - "nl_tests":
[gw9] Python 3.11.3 (main, May 23 2023, 13:25:46) [GCC 10.2.1 20210110]
Step #2 - "nl_tests":
[gw11] Python 3.11.3 (main, May 23 2023, 13:25:46) [GCC 10.2.1 20210110]
Step #2 - "nl_tests":
[gw10] Python 3.11.3 (main, May 23 2023, 13:25:46) [GCC 10.2.1 20210110]
Step #2 - "nl_tests":
[gw12] Python 3.11.3 (main, May 23 2023, 13:25:46) [GCC 10.2.1 20210110]
Step #2 - "nl_tests":
[gw13] Python 3.11.3 (main, May 23 2023, 13:25:46) [GCC 10.2.1 20210110]
Step #2 - "nl_tests":
[gw14] Python 3.11.3 (main, May 23 2023, 13:25:46) [GCC 10.2.1 20210110]
Step #2 - "nl_tests":
[gw15] Python 3.11.3 (main, May 23 2023, 13:25:46) [GCC 10.2.1 20210110]
Step #2 - "nl_tests":
[gw16] Python 3.11.3 (main, May 23 2023, 13:25:46) [GCC 10.2.1 20210110]
Step #2 - "nl_tests":
[gw17] Python 3.11.3 (main, May 23 2023, 13:25:46) [GCC 10.2.1 20210110]
Step #2 - "nl_tests":
[gw18] Python 3.11.3 (main, May 23 2023, 13:25:46) [GCC 10.2.1 20210110]
Step #2 - "nl_tests":
[gw20] Python 3.11.3 (main, May 23 2023, 13:25:46) [GCC 10.2.1 20210110]
Step #2 - "nl_tests":
[gw19] Python 3.11.3 (main, May 23 2023, 13:25:46) [GCC 10.2.1 20210110]
Step #2 - "nl_tests":
[gw21] Python 3.11.3 (main, May 23 2023, 13:25:46) [GCC 10.2.1 20210110]
Step #2 - "nl_tests":
[gw22] Python 3.11.3 (main, May 23 2023, 13:25:46) [GCC 10.2.1 20210110]
Step #2 - "nl_tests":
[gw24] Python 3.11.3 (main, May 23 2023, 13:25:46) [GCC 10.2.1 20210110]
Step #2 - "nl_tests":
[gw23] Python 3.11.3 (main, May 23 2023, 13:25:46) [GCC 10.2.1 20210110]
Step #2 - "nl_tests":
[gw26] Python 3.11.3 (main, May 23 2023, 13:25:46) [GCC 10.2.1 20210110]
Step #2 - "nl_tests":
[gw25] Python 3.11.3 (main, May 23 2023, 13:25:46) [GCC 10.2.1 20210110]
Step #2 - "nl_tests":
[gw27] Python 3.11.3 (main, May 23 2023, 13:25:46) [GCC 10.2.1 20210110]
Step #2 - "nl_tests":
[gw28] Python 3.11.3 (main, May 23 2023, 13:25:46) [GCC 10.2.1 20210110]
Step #2 - "nl_tests":
[gw29] Python 3.11.3 (main, May 23 2023, 13:25:46) [GCC 10.2.1 20210110]
Step #2 - "nl_tests":
[gw30] Python 3.11.3 (main, May 23 2023, 13:25:46) [GCC 10.2.1 20210110]
Step #2 - "nl_tests":
[gw31] Python 3.11.3 (main, May 23 2023, 13:25:46) [GCC 10.2.1 20210110]
Step #2 - "nl_tests": gw0 [14] / gw1 [14] / gw2 [14] / gw3 [14] / gw4 [14] / gw5 [14] / gw6 [14] / gw7 [14] / gw8 [14] / gw9 [14] / gw10 [14] / gw11 [14] / gw12 [14] / gw13 [14] / gw14 [14] / gw15 [14] / gw16 [14] / gw17 [14] / gw18 [14] / gw19 [14] / gw20 [14] / gw21 [14] / gw22 [14] / gw23 [14] / gw24 [14] / gw25 [14] / gw26 [14] / gw27 [14] / gw28 [14] / gw29 [14] / gw30 [14] / gw31 [14]
Step #2 - "nl_tests":
Step #2 - "nl_tests": scheduling tests via LoadScheduling
Step #2 - "nl_tests":
Step #2 - "nl_tests": server/integration_tests/nl_test.py::NLTestMisc::test_international
Step #2 - "nl_tests": server/integration_tests/nl_test.py::NLTestDemo::test_demo_cities_feb2023
Step #2 - "nl_tests": server/integration_tests/nl_test.py::NLTestDemo::test_textbox_sample
Step #2 - "nl_tests": server/integration_tests/nl_test.py
...
[Logs truncated due to log size limitations. For full logs, see https://console.cloud.google.com/cloud-build/builds/99be4723-379d-43e1-9d9c-9e702abc9fbd?project=879489846695.]
...
ings_path": "gs://datcom-nl-models/embeddings_medium_2024_05_09_18_01_32.ft_final_v20230717230459.all-MiniLM-L6-v2.csv"
Step #1 - "explore_tests": },
Step #1 - "explore_tests": "base_mistral_mem": {
Step #1 - "explore_tests": "store_type": "MEMORY",
Step #1 - "explore_tests": "source_path": "/workspace/tools/nl/embeddings/input/base",
Step #1 - "explore_tests": "model": "sfr-embedding-mistral-model",
Step #1 - "explore_tests": "healthcheck_query": "Life expectancy",
Step #1 - "explore_tests": "embeddings_path": "gs://datcom-nl-models/base_mistral_mem_2024_07_01_10_23_43/embeddings.csv"
Step #1 - "explore_tests": },
Step #1 - "explore_tests": "sdg_ft": {
Step #1 - "explore_tests": "store_type": "MEMORY",
Step #1 - "explore_tests": "source_path": "/workspace/tools/nl/embeddings/input/sdg",
Step #1 - "explore_tests": "model": "ft-final-v20230717230459-all-MiniLM-L6-v2",
Step #1 - "explore_tests": "healthcheck_query": "Hunger",
Step #1 - "explore_tests": "embeddings_path": "gs://datcom-nl-models/sdg_ft_2024_06_24_23_45_46/embeddings.csv"
Step #1 - "explore_tests": },
Step #1 - "explore_tests": "undata_ft": {
Step #1 - "explore_tests": "store_type": "MEMORY",
Step #1 - "explore_tests": "source_path": "/workspace/tools/nl/embeddings/input/undata",
Step #1 - "explore_tests": "model": "ft-final-v20230717230459-all-MiniLM-L6-v2",
Step #1 - "explore_tests": "healthcheck_query": "Hunger",
Step #1 - "explore_tests": "embeddings_path": "gs://datcom-nl-models/undata_ft_2024_06_24_23_47_04/embeddings.csv"
Step #1 - "explore_tests": },
Step #1 - "explore_tests": "undata_ilo_ft": {
Step #1 - "explore_tests": "store_type": "MEMORY",
Step #1 - "explore_tests": "source_path": "/workspace/tools/nl/embeddings/input/undata_ilo",
Step #1 - "explore_tests": "model": "ft-final-v20230717230459-all-MiniLM-L6-v2",
Step #1 - "explore_tests": "healthcheck_query": "Employment",
Step #1 - "explore_tests": "embeddings_path": "gs://datcom-nl-models/undata_ilo_ft_2024_10_14_13_45_50/embeddings.csv"
Step #1 - "explore_tests": }
Step #1 - "explore_tests": },
Step #1 - "explore_tests": "models": {
Step #1 - "explore_tests": "cross-encoder-ms-marco-miniilm-l6-v2": {
Step #1 - "explore_tests": "type": "VERTEXAI",
Step #1 - "explore_tests": "usage": "RERANKING",
Step #1 - "explore_tests": "score_threshold": null,
Step #1 - "explore_tests": "project_id": "datcom-website-dev",
Step #1 - "explore_tests": "location": "us-central1",
Step #1 - "explore_tests": "prediction_endpoint_id": "3977846152316846080"
Step #1 - "explore_tests": },
Step #1 - "explore_tests": "cross-encoder-mxbai-rerank-base-v1": {
Step #1 - "explore_tests": "type": "VERTEXAI",
Step #1 - "explore_tests": "usage": "RERANKING",
Step #1 - "explore_tests": "score_threshold": null,
Step #1 - "explore_tests": "project_id": "datcom-website-dev",
Step #1 - "explore_tests": "location": "us-central1",
Step #1 - "explore_tests": "prediction_endpoint_id": "284894457873039360"
Step #1 - "explore_tests": },
Step #1 - "explore_tests": "uae-large-v1-model": {
Step #1 - "explore_tests": "type": "VERTEXAI",
Step #1 - "explore_tests": "usage": "EMBEDDINGS",
Step #1 - "explore_tests": "score_threshold": 0.7,
Step #1 - "explore_tests": "project_id": "datcom-nl",
Step #1 - "explore_tests": "location": "us-central1",
Step #1 - "explore_tests": "prediction_endpoint_id": "8110162693219942400"
Step #1 - "explore_tests": },
Step #1 - "explore_tests": "ft-final-v20230717230459-all-MiniLM-L6-v2": {
Step #1 - "explore_tests": "type": "LOCAL",
Step #1 - "explore_tests": "usage": "EMBEDDINGS",
Step #1 - "explore_tests": "score_threshold": 0.5,
Step #1 - "explore_tests": "gcs_folder": "gs://datcom-nl-models/ft_final_v20230717230459.all-MiniLM-L6-v2"
Step #1 - "explore_tests": },
Step #1 - "explore_tests": "sfr-embedding-mistral-model": {
Step #1 - "explore_tests": "type": "VERTEXAI",
Step #1 - "explore_tests": "usage": "EMBEDDINGS",
Step #1 - "explore_tests": "score_threshold": 0.5,
Step #1 - "explore_tests": "project_id": "datcom-website-dev",
Step #1 - "explore_tests": "location": "us-central1",
Step #1 - "explore_tests": "prediction_endpoint_id": "224012300019826688"
Step #1 - "explore_tests": }
Step #1 - "explore_tests": },
Step #1 - "explore_tests": "enable_reranking": true
Step #1 - "explore_tests": }
Step #2 - "nl_tests": [03:01:15][INFO ][config_reader.py:120] Loading index and model catalog from: /workspace/nl_server/../deploy/helm_charts/dc_website/nl/catalog.yaml
Step #2 - "nl_tests": [03:01:15][INFO ][config_reader.py:86] server config:
Step #2 - "nl_tests": {
Step #2 - "nl_tests": "version": "1",
Step #2 - "nl_tests": "default_indexes": [
Step #2 - "nl_tests": "base_uae_mem"
Step #2 - "nl_tests": ],
Step #2 - "nl_tests": "indexes": {
Step #2 - "nl_tests": "base_uae_mem": {
Step #2 - "nl_tests": "store_type": "MEMORY",
Step #2 - "nl_tests": "source_path": "/workspace/tools/nl/embeddings/input/base",
Step #2 - "nl_tests": "model": "uae-large-v1-model",
Step #2 - "nl_tests": "healthcheck_query": "Life expectancy",
Step #2 - "nl_tests": "embeddings_path": "gs://datcom-nl-models/base_uae_mem_2025_11_03_07_10_42/embeddings.csv"
Step #2 - "nl_tests": },
Step #2 - "nl_tests": "bio_ft": {
Step #2 - "nl_tests": "store_type": "MEMORY",
Step #2 - "nl_tests": "source_path": "/workspace/tools/nl/embeddings/input/bio",
Step #2 - "nl_tests": "model": "ft-final-v20230717230459-all-MiniLM-L6-v2",
Step #2 - "nl_tests": "healthcheck_query": "Gene",
Step #2 - "nl_tests": "embeddings_path": "gs://datcom-nl-models/bio_ft_2024_11_08_19_00_38/embeddings.csv"
Step #2 - "nl_tests": },
Step #2 - "nl_tests": "medium_ft": {
Step #2 - "nl_tests": "store_type": "MEMORY",
Step #2 - "nl_tests": "source_path": null,
Step #2 - "nl_tests": "model": "ft-final-v20230717230459-all-MiniLM-L6-v2",
Step #2 - "nl_tests": "healthcheck_query": "Life expectancy",
Step #2 - "nl_tests": "embeddings_path": "gs://datcom-nl-models/embeddings_medium_2024_05_09_18_01_32.ft_final_v20230717230459.all-MiniLM-L6-v2.csv"
Step #2 - "nl_tests": },
Step #2 - "nl_tests": "base_mistral_mem": {
Step #2 - "nl_tests": "store_type": "MEMORY",
Step #2 - "nl_tests": "source_path": "/workspace/tools/nl/embeddings/input/base",
Step #2 - "nl_tests": "model": "sfr-embedding-mistral-model",
Step #2 - "nl_tests": "healthcheck_query": "Life expectancy",
Step #2 - "nl_tests": "embeddings_path": "gs://datcom-nl-models/base_mistral_mem_2024_07_01_10_23_43/embeddings.csv"
Step #2 - "nl_tests": },
Step #2 - "nl_tests": "sdg_ft": {
Step #2 - "nl_tests": "store_type": "MEMORY",
Step #2 - "nl_tests": "source_path": "/workspace/tools/nl/embeddings/input/sdg",
Step #2 - "nl_tests": "model": "ft-final-v20230717230459-all-MiniLM-L6-v2",
Step #2 - "nl_tests": "healthcheck_query": "Hunger",
Step #2 - "nl_tests": "embeddings_path": "gs://datcom-nl-models/sdg_ft_2024_06_24_23_45_46/embeddings.csv"
Step #2 - "nl_tests": },
Step #2 - "nl_tests": "undata_ft": {
Step #2 - "nl_tests": "store_type": "MEMORY",
Step #2 - "nl_tests": "source_path": "/workspace/tools/nl/embeddings/input/undata",
Step #2 - "nl_tests": "model": "ft-final-v20230717230459-all-MiniLM-L6-v2",
Step #2 - "nl_tests": "healthcheck_query": "Hunger",
Step #2 - "nl_tests": "embeddings_path": "gs://datcom-nl-models/undata_ft_2024_06_24_23_47_04/embeddings.csv"
Step #2 - "nl_tests": },
Step #2 - "nl_tests": "undata_ilo_ft": {
Step #2 - "nl_tests": "store_type": "MEMORY",
Step #2 - "nl_tests": "source_path": "/workspace/tools/nl/embeddings/input/undata_ilo",
Step #2 - "nl_tests": "model": "ft-final-v20230717230459-all-MiniLM-L6-v2",
Step #2 - "nl_tests": "healthcheck_query": "Employment",
Step #2 - "nl_tests": "embeddings_path": "gs://datcom-nl-models/undata_ilo_ft_2024_10_14_13_45_50/embeddings.csv"
Step #2 - "nl_tests": }
Step #2 - "nl_tests": },
Step #2 - "nl_tests": "models": {
Step #2 - "nl_tests": "cross-encoder-ms-marco-miniilm-l6-v2": {
Step #2 - "nl_tests": "type": "VERTEXAI",
Step #2 - "nl_tests": "usage": "RERANKING",
Step #2 - "nl_tests": "score_threshold": null,
Step #2 - "nl_tests": "project_id": "datcom-website-dev",
Step #2 - "nl_tests": "location": "us-central1",
Step #2 - "nl_tests": "prediction_endpoint_id": "3977846152316846080"
Step #2 - "nl_tests": },
Step #2 - "nl_tests": "cross-encoder-mxbai-rerank-base-v1": {
Step #2 - "nl_tests": "type": "VERTEXAI",
Step #2 - "nl_tests": "usage": "RERANKING",
Step #2 - "nl_tests": "score_threshold": null,
Step #2 - "nl_tests": "project_id": "datcom-website-dev",
Step #2 - "nl_tests": "location": "us-central1",
Step #2 - "nl_tests": "prediction_endpoint_id": "284894457873039360"
Step #2 - "nl_tests": },
Step #2 - "nl_tests": "uae-large-v1-model": {
Step #2 - "nl_tests": "type": "VERTEXAI",
Step #2 - "nl_tests": "usage": "EMBEDDINGS",
Step #2 - "nl_tests": "score_threshold": 0.7,
Step #2 - "nl_tests": "project_id": "datcom-nl",
Step #2 - "nl_tests": "location": "us-central1",
Step #2 - "nl_tests": "prediction_endpoint_id": "8110162693219942400"
Step #2 - "nl_tests": },
Step #2 - "nl_tests": "ft-final-v20230717230459-all-MiniLM-L6-v2": {
Step #2 - "nl_tests": "type": "LOCAL",
Step #2 - "nl_tests": "usage": "EMBEDDINGS",
Step #2 - "nl_tests": "score_threshold": 0.5,
Step #2 - "nl_tests": "gcs_folder": "gs://datcom-nl-models/ft_final_v20230717230459.all-MiniLM-L6-v2"
Step #2 - "nl_tests": },
Step #2 - "nl_tests": "sfr-embedding-mistral-model": {
Step #2 - "nl_tests": "type": "VERTEXAI",
Step #2 - "nl_tests": "usage": "EMBEDDINGS",
Step #2 - "nl_tests": "score_threshold": 0.5,
Step #2 - "nl_tests": "project_id": "datcom-website-dev",
Step #2 - "nl_tests": "location": "us-central1",
Step #2 - "nl_tests": "prediction_endpoint_id": "224012300019826688"
Step #2 - "nl_tests": }
Step #2 - "nl_tests": },
Step #2 - "nl_tests": "enable_reranking": true
Step #2 - "nl_tests": }
Step #1 - "explore_tests": [03:01:16][INFO ][SentenceTransformer.py:113] Load pretrained SentenceTransformer: /tmp/datcom-nl-models/ft_final_v20230717230459.all-MiniLM-L6-v2
Step #2 - "nl_tests": [03:01:16][INFO ][SentenceTransformer.py:113] Load pretrained SentenceTransformer: /tmp/datcom-nl-models/ft_final_v20230717230459.all-MiniLM-L6-v2
Step #2 - "nl_tests": [03:01:16][INFO ][SentenceTransformer.py:219] Use pytorch device_name: cpu
Step #2 - "nl_tests": [03:01:16][INFO ][memory.py:67] Loading embeddings file: /tmp/datcom-nl-models/base_uae_mem_2025_11_03_07_10_42/embeddings.csv
Step #1 - "explore_tests": [03:01:16][INFO ][SentenceTransformer.py:219] Use pytorch device_name: cpu
Step #1 - "explore_tests": [03:01:16][INFO ][memory.py:67] Loading embeddings file: /tmp/datcom-nl-models/base_uae_mem_2025_11_03_07_10_42/embeddings.csv
Step #2 - "nl_tests": [03:01:16][INFO ][memory.py:67] Loading embeddings file: /tmp/datcom-nl-models/bio_ft_2024_11_08_19_00_38/embeddings.csv
Step #1 - "explore_tests": [03:01:16][INFO ][memory.py:67] Loading embeddings file: /tmp/datcom-nl-models/bio_ft_2024_11_08_19_00_38/embeddings.csv
Step #1 - "explore_tests": [03:01:17][INFO ][memory.py:67] Loading embeddings file: /tmp/datcom-nl-models/embeddings_medium_2024_05_09_18_01_32.ft_final_v20230717230459.all-MiniLM-L6-v2.csv
Step #2 - "nl_tests": [03:01:17][INFO ][memory.py:67] Loading embeddings file: /tmp/datcom-nl-models/embeddings_medium_2024_05_09_18_01_32.ft_final_v20230717230459.all-MiniLM-L6-v2.csv
Step #2 - "nl_tests": [03:01:17][INFO ][memory.py:67] Loading embeddings file: /tmp/datcom-nl-models/base_mistral_mem_2024_07_01_10_23_43/embeddings.csv
Step #1 - "explore_tests": [03:01:17][INFO ][memory.py:67] Loading embeddings file: /tmp/datcom-nl-models/base_mistral_mem_2024_07_01_10_23_43/embeddings.csv
Step #2 - "nl_tests": [03:01:18][INFO ][memory.py:67] Loading embeddings file: /tmp/datcom-nl-models/sdg_ft_2024_06_24_23_45_46/embeddings.csv
Step #1 - "explore_tests": [03:01:18][INFO ][memory.py:67] Loading embeddings file: /tmp/datcom-nl-models/sdg_ft_2024_06_24_23_45_46/embeddings.csv
Step #2 - "nl_tests": [03:01:18][INFO ][memory.py:67] Loading embeddings file: /tmp/datcom-nl-models/undata_ft_2024_06_24_23_47_04/embeddings.csv
Step #1 - "explore_tests": [03:01:18][INFO ][memory.py:67] Loading embeddings file: /tmp/datcom-nl-models/undata_ft_2024_06_24_23_47_04/embeddings.csv
Step #2 - "nl_tests": [03:01:18][INFO ][memory.py:67] Loading embeddings file: /tmp/datcom-nl-models/undata_ilo_ft_2024_10_14_13_45_50/embeddings.csv
Step #1 - "explore_tests": [03:01:18][INFO ][memory.py:67] Loading embeddings file: /tmp/datcom-nl-models/undata_ilo_ft_2024_10_14_13_45_50/embeddings.csv
Step #2 - "nl_tests": [03:01:18][INFO ][flask.py:79] NL Server Flask app initialized
Step #2 - "nl_tests": [03:01:18][INFO ][nl_app.py:27] Run nl server in local mode (host=localhost), port=6070
Step #2 - "nl_tests": [03:01:18][WARNING ][_internal.py:97] * Debugger is active!
Step #1 - "explore_tests": [03:01:18][INFO ][flask.py:79] NL Server Flask app initialized
Step #1 - "explore_tests": [03:01:18][INFO ][nl_app.py:27] Run nl server in local mode (host=localhost), port=6070
Step #1 - "explore_tests": [03:01:18][WARNING ][_internal.py:97] * Debugger is active!
Step #2 - "nl_tests": [03:01:19][INFO ][util.py:593] http://localhost:6070/healthz is up running
Step #1 - "explore_tests": [03:01:19][INFO ][util.py:593] http://localhost:6070/healthz is up running
Step #2 - "nl_tests": [03:01:19][INFO ][gcs.py:50] Download datcom-website-config/nl_bad_words.txt to /tmp/datcom-website-config/nl_bad_words.txt
Step #1 - "explore_tests": [03:01:19][INFO ][gcs.py:50] Download datcom-website-config/nl_bad_words.txt to /tmp/datcom-website-config/nl_bad_words.txt
Step #2 - "nl_tests": [03:01:19][INFO ][util.py:593] https://staging.api.datacommons.org/version is up running
Step #2 - "nl_tests": [03:01:19][INFO ][web_app.py:27] Run web server in local mode
Step #2 - "nl_tests": * Serving Flask app 'server.__init__'
Step #2 - "nl_tests": * Debug mode: on
Step #1 - "explore_tests": [03:01:20][INFO ][util.py:593] https://staging.api.datacommons.org/version is up running
Step #1 - "explore_tests": [03:01:20][INFO ][web_app.py:27] Run web server in local mode
Step #1 - "explore_tests": * Serving Flask app 'server.__init__'
Step #1 - "explore_tests": * Debug mode: on
Step #1 - "explore_tests": [03:01:27][INFO ][util.py:593] http://localhost:6070/healthz is up running
Step #2 - "nl_tests": [03:01:27][INFO ][util.py:593] http://localhost:6070/healthz is up running
Step #1 - "explore_tests": [03:01:28][INFO ][util.py:593] https://staging.api.datacommons.org/version is up running
Step #1 - "explore_tests": [03:01:28][INFO ][web_app.py:27] Run web server in local mode
Step #2 - "nl_tests": [03:01:28][INFO ][util.py:593] https://staging.api.datacommons.org/version is up running
Step #2 - "nl_tests": [03:01:28][INFO ][web_app.py:27] Run web server in local mode
Step #1 - "explore_tests": [03:01:28][WARNING ][_internal.py:97] * Debugger is active!
Step #2 - "nl_tests": [03:01:28][WARNING ][_internal.py:97] * Debugger is active!
Step #1 - "explore_tests":
Step #2 - "nl_tests":
Step #1 - "explore_tests": [gw26] [ 2%] SKIPPED server/integration_tests/explore_test.py::ExploreTestEE1::test_e2e_answer_places
Step #1 - "explore_tests": [gw0] [ 4%] PASSED server/integration_tests/explore_test.py::ExploreTestDetection::test_detection_basic
Step #1 - "explore_tests": server/integration_tests/explore_test.py::ExploreTestEE1::test_e2e_sdg
Step #1 - "explore_tests": [gw9] [ 6%] PASSED server/integration_tests/explore_test.py::ExploreTestDetection::test_detection_bugs
Step #1 - "explore_tests": server/integration_tests/explore_test.py::ExploreTestEE2::test_e2e_fallbacks
Step #1 - "explore_tests": [gw21] [ 8%] PASSED server/integration_tests/explore_test.py::ExploreTestFulfillment::test_fulfillment_sdg_global
Step #1 - "explore_tests": [gw3] [ 10%] PASSED server/integration_tests/explore_test.py::ExploreTestDetection::test_detection_basic_sfr
Step #1 - "explore_tests": server/integration_tests/explore_test.py::ExploreTestEE1::test_e2e_undata
Step #1 - "explore_tests": [gw22] [ 12%] PASSED server/integration_tests/explore_test.py::ExploreTestFulfillment::test_fulfillment_sdg_global_specialvars
Step #1 - "explore_tests": [gw23] [ 14%] PASSED server/integration_tests/explore_test.py::ExploreTestFulfillment::test_fulfillment_sdg_specialvars
Step #1 - "explore_tests": [gw4] [ 16%] PASSED server/integration_tests/explore_test.py::ExploreTestDetection::test_detection_basic_uae
Step #1 - "explore_tests": server/integration_tests/explore_test.py::ExploreTestEE1::test_e2e_us_demo
Step #1 - "explore_tests": [gw25] [ 18%] PASSED server/integration_tests/explore_test.py::ExploreTestFulfillment::test_fulfillment_undata
Step #1 - "explore_tests": [gw20] [ 20%] PASSED server/integration_tests/explore_test.py::ExploreTestFulfillment::test_fulfillment_sdg
Step #1 - "explore_tests": [gw15] [ 22%] PASSED server/integration_tests/explore_test.py::ExploreTestFulfillment::test_fulfillment_basic
Step #1 - "explore_tests": server/integration_tests/explore_test.py::ExploreTestEE2::test_e2e_toolformer_rag_mode
Step #1 - "explore_tests": [gw5] [ 24%] PASSED server/integration_tests/explore_test.py::ExploreTestDetection::test_detection_basic_undata
Step #1 - "explore_tests": server/integration_tests/explore_test.py::ExploreTestEE1::test_filter_query_disabled
Step #1 - "explore_tests": [gw2] [ 26%] PASSED server/integration_tests/explore_test.py::ExploreTestDetection::test_detection_basic_sdg
Step #1 - "explore_tests": [gw6] [ 28%] PASSED server/integration_tests/explore_test.py::ExploreTestDetection::test_detection_basic_undata_ilo
Step #1 - "explore_tests": [gw12] [ 30%] PASSED server/integration_tests/explore_test.py::ExploreTestDetection::test_detection_sdg
Step #1 - "explore_tests": server/integration_tests/explore_test.py::ExploreTestEE2::test_e2e_strict_default_place
Step #1 - "explore_tests": [gw7] [ 32%] PASSED server/integration_tests/explore_test.py::ExploreTestDetection::test_detection_basic_undata_dev
Step #1 - "explore_tests": server/integration_tests/explore_test.py::ExploreTestEE2::test_e2e_date_range
Step #1 - "explore_tests": server/integration_tests/explore_test.py::ExploreTestEE1::test_e2e_translate
Step #1 - "explore_tests": server/integration_tests/explore_test.py::ExploreTestEE2::test_e2e_correlation_simple_place
Step #1 - "explore_tests": [gw1] [ 34%] PASSED server/integration_tests/explore_test.py::ExploreTestDetection::test_detection_basic_bio
Step #1 - "explore_tests": server/integration_tests/explore_test.py::ExploreTestEE1::test_e2e_superlatives
Step #1 - "explore_tests": [gw13] [ 36%] PASSED server/integration_tests/explore_test.py::ExploreTestDetection::test_detection_statvars
Step #2 - "nl_tests": [gw10] [ 7%] PASSED server/integration_tests/nl_test.py::NLTestMisc::test_strict_low_confidence
Step #2 - "nl_tests": [gw12] [ 14%] PASSED server/integration_tests/nl_test.py::NLTestMisc::test_strict_multi_verb
Step #2 - "nl_tests": [gw6] [ 21%] PASSED server/integration_tests/nl_test.py::NLTestDemo::test_textbox_sample
Step #2 - "nl_tests": [gw13] [ 28%] PASSED server/integration_tests/nl_test.py::NLTestMisc::test_translate
Step #1 - "explore_tests": server/integration_tests/explore_test.py::ExploreTestEE2::test_e2e_strict_multi_sv
Step #1 - "explore_tests": [gw14] [ 38%] PASSED server/integration_tests/explore_test.py::ExploreTestDetection::test_detection_translate
Step #2 - "nl_tests": [gw11] [ 35%] PASSED server/integration_tests/nl_test.py::NLTestMisc::test_strict_default_place
Step #2 - "nl_tests": [03:01:34][INFO ][llm_api.py:87] Gemini model used for LLM API: gemini-2.5-flash
Step #2 - "nl_tests": [03:01:34][INFO ][models.py:4993] AFC is enabled with max remote calls: 10.
Step #1 - "explore_tests": server/integration_tests/explore_test.py::ExploreTestEE2::test_e2e_strict_multi_verb
Step #1 - "explore_tests": [gw8] [ 40%] PASSED server/integration_tests/explore_test.py::ExploreTestDetection::test_detection_bio
Step #1 - "explore_tests": server/integration_tests/explore_test.py::ExploreTestEE2::test_e2e_default_place
Step #1 - "explore_tests": [gw19] [ 42%] PASSED server/integration_tests/explore_test.py::ExploreTestFulfillment::test_fulfillment_explore_more
Step #1 - "explore_tests": [gw13] [ 44%] PASSED server/integration_tests/explore_test.py::ExploreTestEE2::test_e2e_strict_multi_sv
Step #1 - "explore_tests": [gw24] [ 46%] PASSED server/integration_tests/explore_test.py::ExploreTestFulfillment::test_fulfillment_statvars
Step #2 - "nl_tests": [gw1] [ 42%] PASSED server/integration_tests/nl_test.py::NLTestDemo::test_demo_climatetrace
Step #1 - "explore_tests": [gw18] [ 48%] PASSED server/integration_tests/explore_test.py::ExploreTestFulfillment::test_fulfillment_nl_size
Step #1 - "explore_tests": [gw17] [ 50%] PASSED server/integration_tests/explore_test.py::ExploreTestFulfillment::test_fulfillment_correlation
Step #1 - "explore_tests": server/integration_tests/explore_test.py::ExploreTestEE2::test_e2e_triple
Step #1 - "explore_tests": [gw27] [ 52%] PASSED server/integration_tests/explore_test.py::ExploreTestEE1::test_e2e_correlation_bugs
Step #1 - "explore_tests": [03:01:37][INFO ][llm_api.py:87] Gemini model used for LLM API: gemini-2.5-flash
Step #1 - "explore_tests": [03:01:37][INFO ][models.py:4993] AFC is enabled with max remote calls: 10.
Step #1 - "explore_tests": [03:01:37][INFO ][llm_api.py:87] Gemini model used for LLM API: gemini-2.5-flash
Step #1 - "explore_tests": [03:01:37][INFO ][models.py:4993] AFC is enabled with max remote calls: 10.
Step #1 - "explore_tests": [03:01:37][INFO ][llm_api.py:87] Gemini model used for LLM API: gemini-2.5-flash
Step #1 - "explore_tests": [03:01:37][INFO ][models.py:4993] AFC is enabled with max remote calls: 10.
Step #1 - "explore_tests": [gw2] [ 54%] PASSED server/integration_tests/explore_test.py::ExploreTestEE1::test_e2e_translate
Step #1 - "explore_tests": [gw16] [ 56%] PASSED server/integration_tests/explore_test.py::ExploreTestFulfillment::test_fulfillment_comparison
Step #1 - "explore_tests": server/integration_tests/explore_test.py::ExploreTestEE2::test_e2e_toolformer_rig_mode
Step #2 - "nl_tests": [03:01:38][INFO ][_client.py:1025] HTTP Request: POST https://generativelanguage.googleapis.com/v1/models/gemini-2.5-flash:generateContent "HTTP/1.1 200 OK"
Step #1 - "explore_tests": [gw0] [ 58%] PASSED server/integration_tests/explore_test.py::ExploreTestEE1::test_e2e_sdg
Step #2 - "nl_tests": [gw9] [ 50%] PASSED server/integration_tests/nl_test.py::NLTestMisc::test_sdg
Step #1 - "explore_tests": [gw12] [ 60%] PASSED server/integration_tests/explore_test.py::ExploreTestEE2::test_e2e_strict_default_place
Step #1 - "explore_tests": [gw14] [ 62%] PASSED server/integration_tests/explore_test.py::ExploreTestEE2::test_e2e_strict_multi_verb
Step #2 - "nl_tests": [gw5] [ 57%] PASSED server/integration_tests/nl_test.py::NLTestMisc::test_demo_usa_map_types
Step #1 - "explore_tests": [03:01:40][INFO ][_client.py:1025] HTTP Request: POST https://generativelanguage.googleapis.com/v1/models/gemini-2.5-flash:generateContent "HTTP/1.1 200 OK"
Step #1 - "explore_tests": [gw5] [ 64%] PASSED server/integration_tests/explore_test.py::ExploreTestEE1::test_filter_query_disabled
Step #1 - "explore_tests": [03:01:41][INFO ][_client.py:1025] HTTP Request: POST https://generativelanguage.googleapis.com/v1/models/gemini-2.5-flash:generateContent "HTTP/1.1 200 OK"
Step #1 - "explore_tests": [03:01:43][INFO ][_client.py:1025] HTTP Request: POST https://generativelanguage.googleapis.com/v1/models/gemini-2.5-flash:generateContent "HTTP/1.1 200 OK"
Step #2 - "nl_tests": [gw4] [ 64%] PASSED server/integration_tests/nl_test.py::NLTestDemo::test_demo_multisv
Step #1 - "explore_tests": [gw3] [ 66%] PASSED server/integration_tests/explore_test.py::ExploreTestEE1::test_e2e_undata
Step #1 - "explore_tests": [gw11] [ 68%] RERUN server/integration_tests/explore_test.py::ExploreTestDetection::test_detection_multivar
Step #1 - "explore_tests": server/integration_tests/explore_test.py::ExploreTestDetection::test_detection_multivar
Step #1 - "explore_tests": [gw10] [ 70%] PASSED server/integration_tests/explore_test.py::ExploreTestDetection::test_detection_context
Step #1 - "explore_tests": [03:01:45][INFO ][llm_api.py:87] Gemini model used for LLM API: gemini-2.5-flash
Step #1 - "explore_tests": [03:01:45][INFO ][models.py:4993] AFC is enabled with max remote calls: 10.
Step #1 - "explore_tests": server/integration_tests/explore_test.py::ExploreTestEE2::test_e2e_high_sv_threshold
Step #2 - "nl_tests": [gw8] [ 71%] PASSED server/integration_tests/nl_test.py::NLTestMisc::test_place_detection_e2e_dc
Step #2 - "nl_tests": [gw7] [ 78%] PASSED server/integration_tests/nl_test.py::NLTestMisc::test_international
Step #2 - "nl_tests": [gw3] [ 85%] PASSED server/integration_tests/nl_test.py::NLTestDemo::test_demo_fallback
Step #1 - "explore_tests": [gw7] [ 72%] PASSED server/integration_tests/explore_test.py::ExploreTestEE2::test_e2e_correlation_simple_place
Step #1 - "explore_tests": [03:01:47][INFO ][llm_api.py:87] Gemini model used for LLM API: gemini-2.5-flash
Step #1 - "explore_tests": [03:01:47][INFO ][models.py:4993] AFC is enabled with max remote calls: 10.
Step #1 - "explore_tests": [gw1] [ 74%] PASSED server/integration_tests/explore_test.py::ExploreTestEE1::test_e2e_superlatives
Step #1 - "explore_tests": [gw10] [ 76%] PASSED server/integration_tests/explore_test.py::ExploreTestEE2::test_e2e_high_sv_threshold
Step #1 - "explore_tests": [03:01:48][INFO ][_client.py:1025] HTTP Request: POST https://generativelanguage.googleapis.com/v1/models/gemini-2.5-flash:generateContent "HTTP/1.1 200 OK"
Step #1 - "explore_tests": [gw31] [ 78%] PASSED server/integration_tests/explore_test.py::ExploreTestEE1::test_e2e_india_demo
Step #2 - "nl_tests": [gw0] [ 92%] PASSED server/integration_tests/nl_test.py::NLTestDemo::test_demo_cities_feb2023
Step #1 - "explore_tests": [gw30] [ 80%] PASSED server/integration_tests/explore_test.py::ExploreTestEE1::test_e2e_electrification_demo
Step #1 - "explore_tests": [03:01:49][INFO ][_client.py:1025] HTTP Request: POST https://generativelanguage.googleapis.com/v1/models/gemini-2.5-flash:generateContent "HTTP/1.1 200 OK"
Step #1 - "explore_tests": [gw9] [ 82%] PASSED server/integration_tests/explore_test.py::ExploreTestEE2::test_e2e_fallbacks
Step #1 - "explore_tests": [gw16] [ 84%] PASSED server/integration_tests/explore_test.py::ExploreTestEE2::test_e2e_toolformer_rig_mode
Step #1 - "explore_tests": [gw28] [ 86%] PASSED server/integration_tests/explore_test.py::ExploreTestEE1::test_e2e_edge_cases
Step #2 - "nl_tests": [gw2] [100%] PASSED server/integration_tests/nl_test.py::NLTestDemo::test_demo_feb2023
Step #2 - "nl_tests":
Step #2 - "nl_tests": =============================== warnings summary ===============================
Step #2 - "nl_tests": .venv/lib/python3.11/site-packages/flask_babel/__init__.py:183: 32 warnings
Step #2 - "nl_tests": /workspace/.venv/lib/python3.11/site-packages/flask_babel/__init__.py:183: DeprecationWarning: 'locked_cached_property' is deprecated and will be removed in Flask 2.4. Use a lock inside the decorated function if locking is needed.
Step #2 - "nl_tests": @locked_cached_property
Step #2 - "nl_tests":
Step #2 - "nl_tests": -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
Step #2 - "nl_tests": ================== 14 passed, 32 warnings in 76.73s (0:01:16) ==================
Step #1 - "explore_tests": [03:01:52][INFO ][llm_api.py:87] Gemini model used for LLM API: gemini-2.5-flash
Step #1 - "explore_tests": [03:01:52][INFO ][models.py:4993] AFC is enabled with max remote calls: 10.
Step #1 - "explore_tests": [gw15] [ 88%] PASSED server/integration_tests/explore_test.py::ExploreTestEE2::test_e2e_toolformer_rag_mode
Step #1 - "explore_tests": [gw11] [ 88%] PASSED server/integration_tests/explore_test.py::ExploreTestDetection::test_detection_multivar
Finished Step #2 - "nl_tests"
Step #1 - "explore_tests": server/integration_tests/explore_test.py::ExploreTestEE2::test_e2e_single_date
Step #1 - "explore_tests": [03:01:54][INFO ][_client.py:1025] HTTP Request: POST https://generativelanguage.googleapis.com/v1/models/gemini-2.5-flash:generateContent "HTTP/1.1 200 OK"
Step #1 - "explore_tests": [gw4] [ 90%] PASSED server/integration_tests/explore_test.py::ExploreTestEE1::test_e2e_us_demo
Step #1 - "explore_tests": [gw8] [ 92%] PASSED server/integration_tests/explore_test.py::ExploreTestEE2::test_e2e_default_place
Step #1 - "explore_tests": [gw29] [ 94%] PASSED server/integration_tests/explore_test.py::ExploreTestEE1::test_e2e_edge_cases2
Step #1 - "explore_tests": [gw17] [ 96%] PASSED server/integration_tests/explore_test.py::ExploreTestEE2::test_e2e_triple
Step #1 - "explore_tests": [gw6] [ 98%] PASSED server/integration_tests/explore_test.py::ExploreTestEE2::test_e2e_date_range
Step #1 - "explore_tests": [gw11] [100%] PASSED server/integration_tests/explore_test.py::ExploreTestEE2::test_e2e_single_date
Step #1 - "explore_tests":
Step #1 - "explore_tests": =============================== warnings summary ===============================
Step #1 - "explore_tests": .venv/lib/python3.11/site-packages/flask_babel/__init__.py:183: 32 warnings
Step #1 - "explore_tests": /workspace/.venv/lib/python3.11/site-packages/flask_babel/__init__.py:183: DeprecationWarning: 'locked_cached_property' is deprecated and will be removed in Flask 2.4. Use a lock inside the decorated function if locking is needed.
Step #1 - "explore_tests": @locked_cached_property
Step #1 - "explore_tests":
Step #1 - "explore_tests": -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
Step #1 - "explore_tests": ======== 49 passed, 1 skipped, 32 warnings, 1 rerun in 88.58s (0:01:28) ========
Finished Step #1 - "explore_tests"
PUSH
DONE
Loading