Skip to content

CHANGELOG.md

Latest commit

 

History

History
269 lines (163 loc) · 7.56 KB

CHANGELOG.md

File metadata and controls

269 lines (163 loc) · 7.56 KB

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[2.5.1] - OPEN

Added

Changed

Fixed

  • Slurm job information is now fetched from both the Slurm DB and the Slurm queue allowing to include ineligible jobs' data.
  • Fixed issue with health check liveness at deployment time.

[2.5.0]

Added

Changed

  • ⚠️ Configuration Breaking data_operation and data_transfer settings are now configurable independently for each cluster.
  • ⚠️ Configuration Breaking datatransfer_jobs_directives setting is now under data_operation.

Fixed

  • ⚠️ API Breaking Fix transfer directives serialization, now properties names are properly camelcased (see issue: #162).
  • Returns an error if the transfer_method chosen for large data transfer is not available.
  • Documentation about streamer and wormhole data transfer methods.
  • buffer_limit for /filesystem/<system>/ops/* operation is now adapted to the value of settings.data_operation.max_ops_file_size (it was set to the value by default of 5MB).
  • Updated Demo launcher configuration.
  • Fix error for PBS jobs when no nodes are assigned to it.
  • Handle job arrays in PBS. Job IDs will be strings, and not integers anymore in the API responses.
  • Remove hardcoded jfrog link from the wormhole download endpoint.
  • Customizable Response's headers tracing log.

[2.4.2]

Added

  • File target check in S3 file transfer job for external file upload.
  • cluster.scheduler.connection_mode setting to configure how the client connects to the scheduler backend (ssh, rest or hybrid)
  • Get jobs now allows to specify the account parameter.
  • Fine grained probing services per cluster.

Changed

  • Large file download via s3 no longer appends uuid to file name.

Fixed

  • File transfer examples with .NET

[2.4.1]

Added

  • Auto-selection of expected signature algorithm when alg attribute is not set in IdP JWKs endpoint.
  • Firecrest streamer now computes checksums to validate data transfer
  • Firecrest streamer transfer end-point returns immediate errors if target paths have issues.

Changed

  • Firecrest Streamer no longer allows to overwrite existing files.
  • Get jobs shows jobs up to one week.
  • Firecrest streamer improved error handling, including remote errors.
  • SLURM RESTAPI now validates the username claim on the access token from the auth configuration.

Fixed

  • Demo Launcher now is adapted correctly to the data_operation setup introduced in 2.4.0
  • Old link to the installation documentation
  • Timeout value for commands executed via filesystems/ops and status is now configured with command execution timeout setting
  • Stdout, stderr path are now fully expanded
  • probing configuration is optional now for the clusters settings
  • Updated documentation for large data upload

[2.4.0]

Added

  • Compress and Extract end-points now support multiple compression types (none, bz2, gzip, and xz).
  • Support for Magic Wormhole data transfer method

Changed

  • The System Name path parameter and the corresponding Cluster name configuration are case insensitive.

Fixed

  • Fixed Slurm sacct integration and data parsing.

  • Docker Compose startup: Added dependency for Slurm to wait for Keycloak health check before starting, preventing JWT certificate download failures.

  • Upload and Download transfer endpoints now require to specify transfer directives

  • Installation docs:

    • Helm charts: FirecREST settings are all included in values.yaml file
    • Changed documentation name from Deployment to Install

  • Improved direct upload endpoints size limit checks

[2.3.1]

Added

Changed

  • New data_operation setting to replace storage. The new setting decouples the max_ops_file_size parameter from the data_transfer settings that are now a child parameter of data_operation. Also, data_transfer can be of multiple types.
  • Certificates debug information when SSH connection fails

Fixed

[2.3.0]

Added

  • Add support for the OpenPBS scheduler.
  • Add support for the DeiC ssh certificate authority.
  • Allows to set the JWT claim that contains the username.

Changed

  • /filesystem/cluster-slurm-ssh/ops/view endpoint now accepts size and offset parameters to read an arbitrary chunk of a file

Fixed

[2.2.8]

Added

  • Refactor FastAPI models for Slurm, in order to make it easier to add new schedulers.
  • Make cp recursive so that directories can also be copied and add the option to keep symbolic links.
  • Support for clusters configuration files on option in Helm Chart: if enabled the a firecrest-cluster-configs ConfigMap is expected to expose YAML files for clusters configuration.

Changed

Fixed

  • Fixed Slurm timestamps parsing issues, timezone was not properly handled.

[2.2.7]

Added

  • Query parameter allusers in GET /compute/jobs to show all visible jobs for the user in the scheduler
  • Environment variable UVICORN_LOG_CONFIG to enable Uvicorn log configuration file path (analog to --log-config)

Changed

Fixed

  • Show nodes from hidden partitions using Slurm CLI
  • Fixed reservation start and end datatime parsing.
  • Handles instances where no Job exit status is provided.
  • Fixed unnecessary user keys retrieval with SSH connection pool.
  • Fixed proper SSH process termination on timeout.
  • UVICORN_LOG_CONFIG value on helm chart

[2.2.6]

Added

  • account optional parameter to job submission request
  • script_path optional parameter for submitting jobs from a remote file
  • JupyterHub example
  • Documentation for logging architecture
  • Workflow orchestrator example
  • UI browser app example
  • POST and PUT bodies request examples
  • Documentation and examples in C# .NET

Changed

  • Documentation for logging architecture
  • Images for documentation
  • Description of API definition

Fixed

[2.2.5]

Added

  • Log for request and command execution tracing

Changed

Fixed

  • Fix health check for older versions of Slurm REST API (< v0.0.42)

[2.2.4]

Added

Changed

  • Slurm health check now uses "scontrol ping"

Fixed

  • Disabled cluster health checks won't cause errors
  • Github pages changed to allow mkdocs syntax for notes and code samples

[2.2.3]

Added

  • New /status/liveness end-point (no auth is required)

Changed

Fixed

  • Improved health checker reliability
  • Fixed Demo launcher when no public certificate is provided

[2.2.2]

Added

Changed

Fixed

  • Demo launcher ssh login node checks socket connection instead executing a ping
  • Removed deprecated keycloak configuration from docker dev environment

[2.2.1]

Added

  • FirecREST Web UI has been added to the demo image.

Changed

Fixed

  • Templates for upload and download using filesystems/transfer endpoint.
  • Return error code 408 when basic commands timeout on the cluster.

[2.2.0]

Added

  • Added /filesystem/{system_name}/transfer/compress and /filesystem/{system_name}/transfer/extract
    • compress operations (on transfer and ops endpoints) accept match_pattern parameter to compress files using regex syntax.
  • Added new FirecREST demo image.
  • Added support for private key passphrase.

Changed

  • Images are now built for multiple platforms: linux/amd64, linux/arm64

Fixed

[2.1.4]

Fixed

Helm Chart now allows to dynamically set volumes and annotations.

[2.1.3]

Added

Initial release.