Skip to content

RENAME crashes with "External string not supported yet" when source key is offloaded to tiered storage #6163

@vyavdoshenko

Description

@vyavdoshenko

RENAME command causes server crash when the source key's value has been offloaded to tiered storage (External encoding).

Steps to Reproduce:

  1. Start Dragonfly with tiered storage enabled:
./dragonfly \                                 
  --tiered_prefix=/tmp/tiered_test \
  --tiered_offload_threshold=1.0 \
  --tiered_upload_threshold=0.0 \
  --tiered_experimental_cooling=false \
  --proactor_threads=3 \
  --logtostderr

Create a large key (>3KB):

redis-cli SET x "$(python3 -c "print('A'*3000)")"

Wait for offloading to complete (~3 seconds)

Execute RENAME:

redis-cli RENAME x b

Expected Behavior:
RENAME should succeed, moving the key from x to b.

Actual Behavior:
Server crashes with:

F20251204 17:21:01.817420 198536 rdb_save.cc:240] External string not supported yet
*** Check failure stack trace: ***
    @     0x5589bad10629  google::LogMessage::Fail()
    @     0x5589bad1056f  google::LogMessage::SendToLog()
    @     0x5589bad0fd22  google::LogMessage::Flush()
    @     0x5589bad13e1a  google::LogMessageFatal::~LogMessageFatal()
    @     0x5589ba2d1be6  _ZN4dfly13RdbSerializer9SaveValueERKNS_10CompactObjE.localalias
    @     0x5589ba2d9bd1  dfly::SerializerBase::DumpObject()
    @     0x5589ba2da04f  dfly::SerializerBase::DumpObject()
    @     0x5589b9fa20a9  dfly::(anonymous namespace)::Renamer::SerializeSrc()
    @     0x5589b9fa18b0  _ZZN4dfly12_GLOBAL__N_17Renamer9FetchDataEvENKUlPNS_11TransactionEPNS_11EngineShardEE_clES3_S5_
    @     0x5589b9fb9761  _ZSt13__invoke_implIN6facade8OpStatusERKZN4dfly12_GLOBAL__N_17Renamer9FetchDataEvEUlPNS2_11TransactionEPNS2_11EngineShardEE_JS6_S8_EET_St14__invoke_otherOT0_DpOT1_
    @     0x5589b9fb7ccf  _ZSt8__invokeIRKZN4dfly12_GLOBAL__N_17Renamer9FetchDataEvEUlPNS0_11TransactionEPNS0_11EngineShardEE_JS4_S6_EENSt15__invoke_resultIT_JDpT0_EE4typeEOSB_DpOSC_
    @     0x5589b9fb6938  _ZSt6invokeIRKZN4dfly12_GLOBAL__N_17Renamer9FetchDataEvEUlPNS0_11TransactionEPNS0_11EngineShardEE_JS4_S6_EENSt13invoke_resultIT_JDpT0_EE4typeEOSB_DpOSC_
    @     0x5589b9fb50c6  _ZN4absl12lts_2025051219functional_internal12InvokeObjectIZN4dfly12_GLOBAL__N_17Renamer9FetchDataEvEUlPNS3_11TransactionEPNS3_11EngineShardEE_NS6_14RunnableResultEJS7_S9_EEET0_NS1_7VoidPtrEDpNS1_8ForwardTIT1_E4typeE
    @     0x5589ba72619c  absl::lts_20250512::FunctionRef<>::operator()()
    @     0x5589ba712b23  dfly::Transaction::RunCallback()
    @     0x5589ba711f9e  dfly::Transaction::RunInShard()
    @     0x5589b9e4f8d7  _ZZN4dfly11EngineShard13PollExecutionEPKcPNS_11TransactionEENKUlS4_bE_clES4_b
    @     0x5589b9e50676  dfly::EngineShard::PollExecution()
    @     0x5589ba7150f3  _ZZN4dfly11Transaction11DispatchHopEvENKUlvE_clEv
    @     0x5589ba72352e  _ZSt13__invoke_implIvRZN4dfly11Transaction11DispatchHopEvEUlvE_JEET_St14__invoke_otherOT0_DpOT1_
    @     0x5589ba722f5d  _ZSt10__invoke_rIvRZN4dfly11Transaction11DispatchHopEvEUlvE_JEENSt9enable_ifIX16is_invocable_r_vIT_T0_DpT1_EES5_E4typeEOS6_DpOS7_
    @     0x5589ba722a82  _ZNSt17_Function_handlerIFvvEZN4dfly11Transaction11DispatchHopEvEUlvE_E9_M_invokeERKSt9_Any_data
    @     0x5589b9d8ad30  std::function<>::operator()()
    @     0x5589bab6bd64  util::fb2::FiberQueue::Run()
    @     0x5589ba80a4fb  _ZZN4dfly9TaskQueue5StartESt17basic_string_viewIcSt11char_traitsIcEEENKUlvE_clEv
    @     0x5589ba80b6b1  _ZSt13__invoke_implIvZN4dfly9TaskQueue5StartESt17basic_string_viewIcSt11char_traitsIcEEEUlvE_JEET_St14__invoke_otherOT0_DpOT1_
    @     0x5589ba80b436  _ZSt8__invokeIZN4dfly9TaskQueue5StartESt17basic_string_viewIcSt11char_traitsIcEEEUlvE_JEENSt15__invoke_resultIT_JDpT0_EE4typeEOS8_DpOS9_
    @     0x5589ba80b244  _ZSt12__apply_implIZN4dfly9TaskQueue5StartESt17basic_string_viewIcSt11char_traitsIcEEEUlvE_St5tupleIJEEJEEDcOT_OT0_St16integer_sequenceImJXspT1_EEE
    @     0x5589ba80b27e  _ZSt5applyIZN4dfly9TaskQueue5StartESt17basic_string_viewIcSt11char_traitsIcEEEUlvE_St5tupleIJEEEDcOT_OT0_
    @     0x5589ba80b2fb  _ZN4util3fb26detail15WorkerFiberImplIZN4dfly9TaskQueue5StartESt17basic_string_viewIcSt11char_traitsIcEEEUlvE_JEE4run_EON5boost7context5fiberE
    @     0x5589ba80ae79  _ZZN4util3fb26detail15WorkerFiberImplIZN4dfly9TaskQueue5StartESt17basic_string_viewIcSt11char_traitsIcEEEUlvE_JEEC4INS0_19FixedStackAllocatorEEES8_NS0_13FiberPriorityERKN5boost7context12preallocatedEOT_OS9_ENKUlONSF_5fiberEE_clESN_
    @     0x5589ba80bff2  _ZSt13__invoke_implIN5boost7context5fiberERZN4util3fb26detail15WorkerFiberImplIZN4dfly9TaskQueue5StartESt17basic_string_viewIcSt11char_traitsIcEEEUlvE_JEEC4INS4_19FixedStackAllocatorEEESC_NS4_13FiberPriorityERKNS1_12preallocatedEOT_OSD_EUlOS2_E_JS2_EESL_St14__invoke_otherOT0_DpOT1_
*** SIGABRT received at time=1764861661 on cpu 0 ***
PC: @     0x7fd59abf8b2c  (unknown)  pthread_kill
    @     0x5589bada194b         64  absl::lts_20250512::WriteFailureInfo()
    @     0x5589bada1bcf         96  absl::lts_20250512::AbslFailureSignalHandler()
    @     0x7fd59ab9f330       1808  (unknown)
    @     0x7fd59ab9f27e         32  raise
    @     0x7fd59ab828ff        192  abort
    @     0x5589bad1b376        176  google::DumpStackTraceAndExit()
    @     0x5589bad10629         16  google::LogMessage::Fail()
    @     0x5589bad1056f        160  google::LogMessage::SendToLog()
    @     0x5589bad0fd22         80  google::LogMessage::Flush()
    @     0x5589bad13e1a         32  google::LogMessageFatal::~LogMessageFatal()
    @     0x5589ba2d1be6        192  dfly::RdbSerializer::SaveValue()
    @     0x5589ba2d9bd1        224  dfly::SerializerBase::DumpObject()
    @     0x5589ba2da04f        320  dfly::SerializerBase::DumpObject()
    @     0x5589b9fa20a9        432  dfly::(anonymous namespace)::Renamer::SerializeSrc()
    @     0x5589b9fa18b0        224  dfly::(anonymous namespace)::Renamer::FetchData()::{lambda()#1}::operator()()
    @     0x5589b9fb9761         64  std::__invoke_impl<>()
    @     0x5589b9fb7ccf         64  std::__invoke<>()
    @     0x5589b9fb6938         64  std::invoke<>()
    @     0x5589b9fb50c6         96  absl::lts_20250512::functional_internal::InvokeObject<>()
    @     0x5589ba72619c         64  absl::lts_20250512::FunctionRef<>::operator()()
    @     0x5589ba712b23        240  dfly::Transaction::RunCallback()
    @     0x5589ba711f9e        384  dfly::Transaction::RunInShard()
    @     0x5589b9e4f8d7         64  dfly::EngineShard::PollExecution()::{lambda()#1}::operator()()
    @     0x5589b9e50676        416  dfly::EngineShard::PollExecution()
    @     0x5589ba7150f3        224  dfly::Transaction::DispatchHop()::{lambda()#1}::operator()()
    @     0x5589ba72352e         32  std::__invoke_impl<>()
    @     0x5589ba722f5d         32  std::__invoke_r<>()
    @     0x5589ba722a82         32  std::_Function_handler<>::_M_invoke()
    @     0x5589b9d8ad30         32  std::function<>::operator()()
    @     0x5589bab6bd64        240  util::fb2::FiberQueue::Run()
    @     0x5589ba80a4fb         32  dfly::TaskQueue::Start()::{lambda()#1}::operator()()
    @     0x5589ba80b6b1         32  std::__invoke_impl<>()
    @ ... and at least 11 more frames
[1]    198535 IOT instruction (core dumped)  ./dragonfly --tiered_prefix=/tmp/tiered_test --tiered_offload_threshold=1.0

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions