[Bug] repaired not adhering placement ledger feature can not repair opened ledger

[TakaHiR07]

BUG REPORT

I also do a disaster recovery test, and try to use this feature to repair ledger. #3359

But it can not repair the ledger of pulsar "_change_event" topic, because its ledger is opened status.

Expected behavior

Now my solution is also repair the opened ledger, and ensure not write the opened ledger by do not modify the topic metadata.

But I wonder if there have better way ?

By the way, now we can not repair not adhering RegionAware placement ledger. Can we add a config minNumRegionsPerWriteQuorum, just as minNumRacksPerWriteQuorum, to ensure quorum must be distributed in at least 3 regions ?

[hangc0276]

@horizonzy Please help take a look, thanks.

[horizonzy]

@horizonzy Please help take a look, thanks.

Fine

[horizonzy]

Now my solution is also repair the opened ledger, and ensure not write the opened ledger by do not modify the topic metadata.

Could you please tell me how you implemented it? Now we check the ledger placement policy when its metadata is closed. Did you change the source code to implement it?

bookkeeper/bookkeeper-server/src/main/java/org/apache/bookkeeper/replication/AuditorPlacementPolicyCheckTask.java

Line 260 in 575a81c

if (metadata.isClosed()) {

[TakaHiR07]

Could you please tell me how you implemented it? Now we check the ledger placement policy when its metadata is closed. Did you change the source code to implement it?

bookkeeper/bookkeeper-server/src/main/java/org/apache/bookkeeper/replication/AuditorPlacementPolicyCheckTask.java

Line 260 in 575a81c

if (metadata.isClosed()) {

I change two part of source code, remove "if (!metadata.isClosed())"

bookkeeper/bookkeeper-server/src/main/java/org/apache/bookkeeper/replication/AuditorPlacementPolicyCheckTask.java

Line 260 in 575a81c

if (metadata.isClosed()) {

bookkeeper/bookkeeper-server/src/main/java/org/apache/bookkeeper/replication/ReplicationWorker.java

Line 391 in 575a81c

if (!metadata.isClosed()) {

[horizonzy]

Could you please tell me how you implemented it? Now we check the ledger placement policy when its metadata is closed. Did you change the source code to implement it?

bookkeeper/bookkeeper-server/src/main/java/org/apache/bookkeeper/replication/AuditorPlacementPolicyCheckTask.java

Line 260 in 575a81c

if (metadata.isClosed()) {

I change two part of source code, remove "if (!metadata.isClosed())"

bookkeeper/bookkeeper-server/src/main/java/org/apache/bookkeeper/replication/AuditorPlacementPolicyCheckTask.java

Line 260 in 575a81c

if (metadata.isClosed()) {

bookkeeper/bookkeeper-server/src/main/java/org/apache/bookkeeper/replication/ReplicationWorker.java

Line 391 in 575a81c

if (!metadata.isClosed()) {

Make sense. There is one thing need to pay attention: the last ensemble should not be checked.

[horizonzy]

I change two part of source code, remove "if (!metadata.isClosed())"

bookkeeper/bookkeeper-server/src/main/java/org/apache/bookkeeper/replication/AuditorPlacementPolicyCheckTask.java

Line 260 in 575a81c

if (metadata.isClosed()) {

bookkeeper/bookkeeper-server/src/main/java/org/apache/bookkeeper/replication/ReplicationWorker.java

Line 391 in 575a81c

if (!metadata.isClosed()) {

Could you please submit a PR?

[TakaHiR07]

Could you please submit a PR?

ok

[horizonzy]

By the way, now we can not repair not adhering RegionAware placement ledger. Can we add a config minNumRegionsPerWriteQuorum, just as minNumRacksPerWriteQuorum, to ensure quorum must be distributed in at least 3 regions ?

And for this part, we didn't have a strong demand for it. If you need this feature, you can refer the org.apache.bookkeeper.client.RackawareEnsemblePlacementPolicyImpl#replaceToAdherePlacementPolicy to implement it in the RegionAwareEnsemblePlacementPolicy. I can help you to review it.