Skip to content
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ All notable changes to this project will be documented in this file.
- Run a `containerdebug` process in the background of each HDFS container to collect debugging information ([#629]).
- Support configuring JVM arguments ([#636]).
- Aggregate emitted Kubernetes events on the CustomResources ([#643]).
- Add support for version `3.4.1` ([#656]).

### Changed

Expand All @@ -27,6 +28,7 @@ All notable changes to this project will be documented in this file.
[#640]: https://github.com/stackabletech/hdfs-operator/pull/640
[#643]: https://github.com/stackabletech/hdfs-operator/pull/643
[#655]: https://github.com/stackabletech/hdfs-operator/pull/655
[#656]: https://github.com/stackabletech/hdfs-operator/pull/656

## [24.11.1] - 2025-01-10

Expand Down
2 changes: 1 addition & 1 deletion docs/modules/hdfs/examples/getting_started/hdfs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ metadata:
name: simple-hdfs
spec:
image:
productVersion: 3.4.0
productVersion: 3.4.1
clusterConfig:
zookeeperConfigMapName: simple-hdfs-znode
dfsReplication: 1
Expand Down
2 changes: 1 addition & 1 deletion docs/modules/hdfs/examples/getting_started/hdfs.yaml.j2
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ metadata:
name: simple-hdfs
spec:
image:
productVersion: 3.4.0
productVersion: 3.4.1
clusterConfig:
zookeeperConfigMapName: simple-hdfs-znode
dfsReplication: 1
Expand Down
8 changes: 4 additions & 4 deletions docs/modules/hdfs/pages/usage-guide/upgrading.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,12 @@
IMPORTANT: HDFS upgrades are experimental, and details may change at any time.

HDFS currently requires a manual process to upgrade.
This guide takes you through an example case, upgrading an example cluster (from our xref:getting_started/index.adoc[Getting Started] guide) from HDFS 3.3.6 to 3.4.0.
This guide takes you through an example case, upgrading an example cluster (from our xref:getting_started/index.adoc[Getting Started] guide) from HDFS 3.3.6 to 3.4.1.

== Preparing for the worst

Upgrades can fail, and it is important to prepare for when that happens.
Apache HDFS supports https://hadoop.apache.org/docs/r3.4.0/hadoop-project-dist/hadoop-hdfs/HdfsRollingUpgrade.html#Downgrade_and_Rollback[two ways to revert an upgrade]:
Apache HDFS supports https://hadoop.apache.org/docs/r3.4.1/hadoop-project-dist/hadoop-hdfs/HdfsRollingUpgrade.html#Downgrade_and_Rollback[two ways to revert an upgrade]:

Rollback:: Reverts all user data to the pre-upgrade state. Requires taking the cluster offline.
Downgrade:: Downgrades the HDFS software but preserves all changes made by users. Can be performed as a rolling change, keeping the cluster online.
Expand Down Expand Up @@ -69,7 +69,7 @@ Once HDFS is ready to upgrade, the HdfsCluster can be updated with the new produ

[source,shell]
----
$ kubectl patch hdfs/simple-hdfs --patch '{"spec": {"image": {"productVersion": "3.4.0"}}}' --type=merge
$ kubectl patch hdfs/simple-hdfs --patch '{"spec": {"image": {"productVersion": "3.4.1"}}}' --type=merge
hdfscluster.hdfs.stackable.tech/simple-hdfs patched
----

Expand Down Expand Up @@ -103,7 +103,7 @@ Finally, mark the cluster as upgraded:

[source,shell]
----
$ kubectl patch hdfs/simple-hdfs --subresource=status --patch '{"status": {"deployedProductVersion": "3.4.0"}}' --type=merge
$ kubectl patch hdfs/simple-hdfs --subresource=status --patch '{"status": {"deployedProductVersion": "3.4.1"}}' --type=merge
hdfscluster.hdfs.stackable.tech/simple-hdfs patched
----

Expand Down
3 changes: 2 additions & 1 deletion docs/modules/hdfs/partials/supported-versions.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
// This is a separate file, since it is used by both the direct HDFS-Operator documentation, and the overarching
// Stackable Platform documentation.

- 3.4.0 (LTS)
- 3.4.1 (LTS)
- 3.4.0 (deprecated)
- 3.3.6 - Please note that there is a https://github.com/stackabletech/hdfs-operator/issues/440[known issue] related to NameNode bootstrapping which can happen in rare cases.
- 3.3.4
4 changes: 2 additions & 2 deletions rust/operator-binary/src/config/jvm.rs
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ mod tests {
name: hdfs
spec:
image:
productVersion: 3.4.0
productVersion: 3.4.1
clusterConfig:
zookeeperConfigMapName: hdfs-zk
nameNodes:
Expand Down Expand Up @@ -149,7 +149,7 @@ mod tests {
name: hdfs
spec:
image:
productVersion: 3.4.0
productVersion: 3.4.1
clusterConfig:
zookeeperConfigMapName: hdfs-zk
nameNodes:
Expand Down
2 changes: 1 addition & 1 deletion rust/operator-binary/src/crd/affinity.rs
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ metadata:
name: simple-hdfs
spec:
image:
productVersion: 3.4.0
productVersion: 3.4.1
clusterConfig:
zookeeperConfigMapName: hdfs-zk
journalNodes:
Expand Down
16 changes: 8 additions & 8 deletions rust/operator-binary/src/crd/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1508,7 +1508,7 @@ metadata:
name: hdfs
spec:
image:
productVersion: 3.4.0
productVersion: 3.4.1
clusterConfig:
zookeeperConfigMapName: hdfs-zk
dataNodes:
Expand Down Expand Up @@ -1543,7 +1543,7 @@ metadata:
name: hdfs
spec:
image:
productVersion: 3.4.0
productVersion: 3.4.1
clusterConfig:
zookeeperConfigMapName: hdfs-zk
dataNodes:
Expand Down Expand Up @@ -1578,7 +1578,7 @@ metadata:
name: hdfs
spec:
image:
productVersion: 3.4.0
productVersion: 3.4.1
clusterConfig:
zookeeperConfigMapName: hdfs-zk
dataNodes:
Expand Down Expand Up @@ -1608,7 +1608,7 @@ metadata:
name: hdfs
spec:
image:
productVersion: 3.4.0
productVersion: 3.4.1
clusterConfig:
zookeeperConfigMapName: hdfs-zk
rackAwareness:
Expand Down Expand Up @@ -1673,7 +1673,7 @@ metadata:
name: hdfs
spec:
image:
productVersion: 3.4.0
productVersion: 3.4.1
clusterConfig:
zookeeperConfigMapName: hdfs-zk
dataNodes:
Expand Down Expand Up @@ -1729,7 +1729,7 @@ metadata:
name: hdfs
spec:
image:
productVersion: 3.4.0
productVersion: 3.4.1
clusterConfig:
zookeeperConfigMapName: hdfs-zk
dataNodes:
Expand Down Expand Up @@ -1785,7 +1785,7 @@ metadata:
name: hdfs
spec:
image:
productVersion: 3.4.0
productVersion: 3.4.1
clusterConfig:
zookeeperConfigMapName: hdfs-zk
dataNodes:
Expand All @@ -1812,7 +1812,7 @@ metadata:
name: hdfs
spec:
image:
productVersion: 3.4.0
productVersion: 3.4.1
clusterConfig:
zookeeperConfigMapName: hdfs-zk
rackAwareness:
Expand Down
5 changes: 3 additions & 2 deletions tests/test-definition.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,12 @@ dimensions:
- 3.3.4
- 3.3.6
- 3.4.0
- 3.4.1
# To use a custom image, add a comma and the full name after the product version
# - 3.4.0,oci.stackable.tech/sandbox/hadoop:3.4.0-stackable0.0.0-dev
- name: hadoop-latest
values:
- 3.4.0
- 3.4.1
# To use a custom image, add a comma and the full name after the product version
# - 3.4.0,oci.stackable.tech/sandbox/hadoop:3.4.0-stackable0.0.0-dev
- name: hadoop-external-client-docker-image
Expand All @@ -20,7 +21,7 @@ dimensions:
- 3.9.2
- name: zookeeper-latest
values:
- 3.9.2
- 3.9.3
- name: krb5
values:
- 1.21.1
Expand Down
Loading