Skip to content

Error on starting pulsar standalone using Pulsar 2.2 #2020

@van-scott

Description

@van-scott
  1. It was okay to start pulsar standalone using bin/pulsar-daemon start standalone
  2. The machine was crashed due to other issues.
  3. After the machine is rebooted, run bin/pulsar-daemon start standalone again. It failed to start due to following errors:
dir /home/work/service/apache-pulsar-2.2.0/data/bookkeeper/ranges/ranges/000000000000000000/000000000000000001/000000000000000000
org.rocksdb.RocksDBException: While lock file: /home/work/service/apache-pulsar-2.2.0/data/bookkeeper/ranges/ranges/000000000000000000/000000000000000001/000000000000000000/current/LOCK: 没有可用的锁
	at org.rocksdb.RocksDB.open(Native Method) ~[org.rocksdb-rocksdbjni-5.13.3.jar:?]
	at org.rocksdb.RocksDB.open(RocksDB.java:286) ~[org.rocksdb-rocksdbjni-5.13.3.jar:?]
	at org.apache.bookkeeper.statelib.impl.kv.RocksdbKVStore.openRocksdb(RocksdbKVStore.java:323) ~[org.apache.bookkeeper-statelib-4.7.2.jar:4.7.2]
	at org.apache.bookkeeper.statelib.impl.kv.RocksdbKVStore.openLocalDB(RocksdbKVStore.java:298) ~[org.apache.bookkeeper-statelib-4.7.2.jar:4.7.2]
	at org.apache.bookkeeper.statelib.impl.kv.RocksdbKVStore.openRocksdb(RocksdbKVStore.java:288) ~[org.apache.bookkeeper-statelib-4.7.2.jar:4.7.2]
	at org.apache.bookkeeper.statelib.impl.kv.RocksdbKVStore.init(RocksdbKVStore.java:233) ~[org.apache.bookkeeper-statelib-4.7.2.jar:4.7.2]
	at org.apache.bookkeeper.statelib.impl.journal.AbstractStateStoreWithJournal.lambda$initializeLocalStore$5(AbstractStateStoreWithJournal.java:202) ~[org.apache.bookkeeper-statelib-4.7.2.jar:4.7.2]
	at org.apache.bookkeeper.statelib.impl.journal.AbstractStateStoreWithJournal.lambda$executeIO$16(AbstractStateStoreWithJournal.java:471) ~[org.apache.bookkeeper-statelib-4.7.2.jar:4.7.2]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_161]
	at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:111) [org.apache.pulsar-pulsar-functions-metrics-2.2.0.jar:?]
	at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:58) [org.apache.pulsar-pulsar-functions-metrics-2.2.0.jar:?]
	at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:75) [org.apache.pulsar-pulsar-functions-metrics-2.2.0.jar:?]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_161]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_161]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_161]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [?:1.8.0_161]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_161]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_161]
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [io.netty-netty-all-4.1.22.Final.jar:4.1.22.Final]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_161]
17:18:16.374 [io-write-scheduler-OrderedScheduler-1-0] WARN  org.apache.bookkeeper.stream.storage.impl.sc.StorageContainerRegistryImpl - De-registered StorageContainer ('0') when failed to start
java.util.concurrent.CompletionException: org.apache.bookkeeper.statelib.api.exceptions.StateStoreException: Failed to restore rocksdb 000000000000000000/000000000000000000/000000000000000000
	at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292) ~[?:1.8.0_161]
	at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308) ~[?:1.8.0_161]
	at java.util.concurrent.CompletableFuture.uniCompose(CompletableFuture.java:943) ~[?:1.8.0_161]
	at java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:926) ~[?:1.8.0_161]
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) ~[?:1.8.0_161]
	at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977) ~[?:1.8.0_161]
	at org.apache.bookkeeper.statelib.impl.journal.AbstractStateStoreWithJournal.lambda$executeIO$16(AbstractStateStoreWithJournal.java:474) ~[org.apache.bookkeeper-statelib-4.7.2.jar:4.7.2]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_161]
	at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:111) [org.apache.pulsar-pulsar-functions-metrics-2.2.0.jar:?]
	at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:58) [org.apache.pulsar-pulsar-functions-metrics-2.2.0.jar:?]
	at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:75) [org.apache.pulsar-pulsar-functions-metrics-2.2.0.jar:?]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_161]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_161]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_161]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [?:1.8.0_161]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_161]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_161]
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [io.netty-netty-all-4.1.22.Final.jar:4.1.22.Final]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_161]
Caused by: org.apache.bookkeeper.statelib.api.exceptions.StateStoreException: Failed to restore rocksdb 000000000000000000/000000000000000000/000000000000000000
	at org.apache.bookkeeper.statelib.impl.rocksdb.checkpoint.RocksCheckpointer.restore(RocksCheckpointer.java:84) ~[org.apache.bookkeeper-statelib-4.7.2.jar:4.7.2]
	at org.apache.bookkeeper.statelib.impl.kv.RocksdbKVStore.loadRocksdbFromCheckpointStore(RocksdbKVStore.java:161) ~[org.apache.bookkeeper-statelib-4.7.2.jar:4.7.2]
	at org.apache.bookkeeper.statelib.impl.kv.RocksdbKVStore.init(RocksdbKVStore.java:223) ~[org.apache.bookkeeper-statelib-4.7.2.jar:4.7.2]
	at org.apache.bookkeeper.statelib.impl.journal.AbstractStateStoreWithJournal.lambda$initializeLocalStore$5(AbstractStateStoreWithJournal.java:202) ~[org.apache.bookkeeper-statelib-4.7.2.jar:4.7.2]
	at org.apache.bookkeeper.statelib.impl.journal.AbstractStateStoreWithJournal.lambda$executeIO$16(AbstractStateStoreWithJournal.java:471) ~[org.apache.bookkeeper-statelib-4.7.2.jar:4.7.2]
	... 12 more
Caused by: java.io.FileNotFoundException: 
  1. bin/pulsar-daemon start standalone -nss can bypass the issue.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions