feat(deployment): Add Spider integration in k8s helm.

[sitaowang1998]

Description

This PR integrates Spider into k8s helm chart by:

• Bump spider and spider-py versions for support of SPIDER_STORAGE_URL environment variable.
• Update ClpConfig and job orchestration codes to use the environment variable.
• Add spider database name and credentials and compression type in helm values, config map and database secrets.
• Update db creator, compression scheduler and compression worker charts to use the new values.
• Add spider scheduler chart.

Checklist

• The PR satisfies the contribution guidelines.
• This is a breaking change and that has been indicated in the PR title, OR this isn't a
  breaking change.
• Necessary docs have been updated, OR no docs need to be updated.

Validation performed

• Compression and search works with clp-json using Spider for compression.
• Compression and search works with clp-text using Spider for compression.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}