Skip to content

Code imported from https://github.com/neptune-ai/neptune-fetcher#8

Merged
gabrys merged 9 commits intomainfrom
imported
Aug 18, 2025
Merged

Code imported from https://github.com/neptune-ai/neptune-fetcher#8
gabrys merged 9 commits intomainfrom
imported

Conversation

@gabrys
Copy link
Contributor

@gabrys gabrys commented Aug 18, 2025

No description provided.

michalsosn and others added 9 commits August 13, 2025 10:46
* feat: assume types for known sys attributes

* fix: rebase to neptune_query

---------

Co-authored-by: Michał Sośnicki <michal.sosnicki@neptune.ai>
Co-authored-by: Piotr Gabryjeluk <piotr.gabryjeluk@neptune.ai>
The optimization tries to avoid multi-threading for fetching attribute definitions if only one Filter is passed to `_fetch_attribute_definitions`.

However it doesn't make the code any faster and introduces unnecessary code paths.

Updated unit tests to match the simplified code.

Tested that avoiding calling `concurrency.generate_concurrently` with one item doesn't gain any measurable performance, so I'm removing the extra code from `internal/composition/attributes.py` for simplicity.
Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 4 to 5.
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](actions/download-artifact@v4...v5)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-version: '5'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
… (#409)

* PY-195 Remove mentions of set_context and set_project from exceptions

These were removed when switching from fetcher to query

* Update src/neptune_query/exceptions.py

* Update src/neptune_query/exceptions.py

---------

Co-authored-by: Sabine Ståhlberg <sabine.stahlberg@neptune.ai>
* PY-172 Improve the UX of download_files

Internally:

1. Replace the single File.label field with experiment_name and run_id attributes, enforcing mutual exclusivity.
2. Encode project_identifier in the File object.
3. Shorten File's `repr`
4. Infer the column name for the output of `download_files` from the passed files rather than rely on which version of download_files is called (`nq.download_files` or `nq_runs.download_files`)

* Address code review notes

1. Don't allow mixing Files coming from experiment and run APIs
2. Fix bug calculating missing file paths
3. Adding a TODO for future update to `convert_table_to_dataframe`
4. Make create_files_dataframe accept `container_type` rather than `index_column_name`

* Update attribute_paths in file_object_* fixtures
* Rename `NEPTUNE_FETCHER_QUERY_SIZE_LIMIT` env to `NEPTUNE_FETCHER_MAX_REQUEST_SIZE`

* PY-175 Rename envs from NEPTUNE_FETCHER_* to NEPTUNE_QUERY_*

* Sort env variables

* Bump default NEPTUNE_QUERY_MAX_WORKERS to 32 (from 10) (#413)

The workers don't do much CPU work, but rather wait for network IO and 32 was
shown to improve performance of fetching files. Other than a slight increase
in the memory usage, there's no known downside to increasing this number to 32.
* PY-180 Change type inference in neptune-query for better UX

1. If the type cannot be inferred because the attribute doesn't exist in the project -> show a warning and infer as string
2. If the type cannot be inferred because the attribute has multiple types across the runs and experiments of the project -> raise an exception prompting the user to specify the type explicitly
  * don't try to be smart and limit the attribute search to only runs/experiments matching some filters, etc

* Emit inference warnings for list_containers and list_attributes.

* Test warning is emmitted for missing attributes
@gabrys gabrys merged commit 23e3e30 into main Aug 18, 2025
24 checks passed
@gabrys gabrys deleted the imported branch August 19, 2025 11:16
@gabrys gabrys restored the imported branch August 19, 2025 11:16
@gabrys gabrys deleted the imported branch August 19, 2025 11:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants