Skip to content

v2.0.0-rc4: Model loading fails with "perceptron: failed to load perceptron information" (from release package) #156

@meitt8

Description

@meitt8

Environment:

  • Juman++ Version: v2.0.0-rc4 (from official jumanpp-2.0.0-rc4.tar.xz release package)
  • OS: Windows Subsystem for Linux (WSL - Ubuntu 24.04.2 LTS)
  • Compiler/Build Tools: build-essential, cmake

Problem Description:
When attempting to use Juman++ v2.0.0-rc4 built from the official release package, the program fails to load the provided jumandic.jppmdl model. The final error observed after a clean download and build is:
failed to load model from disk: InvalidState: perceptron: failed to load perceptron information

Steps to Reproduce:

  1. Downloaded jumanpp-2.0.0-rc4.tar.xz (verified size ~302MB). Most recent attempt involved wget directly into WSL to ensure file integrity.
  2. Extracted the archive: tar xf jumanpp-2.0.0-rc4.tar.xz
  3. Navigated into jumanpp-2.0.0-rc4.
  4. Made the following modifications to successfully configure CMake:
    • Created version.cmake with content: set(PROJECT_VERSION "2.0.0-rc4")
    • In CMakeLists.txt:
      • Commented out add_subdirectory(test)
      • Commented out add_subdirectory(model) (within its if (IS_DIRECTORY ...) block)
      • Added install(DIRECTORY model/ DESTINATION libexec/jumanpp USE_SOURCE_PERMISSIONS) (e.g., after add_subdirectory(src))
  5. Created a build directory (mkdir bld && cd bld).
  6. Ran CMake: cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr/local
  7. Compiled and installed: sudo make install -j$(nproc)
  8. Attempted to run Juman++: echo "あ" | jumanpp --model=/usr/local/libexec/jumanpp/jumandic.jppmdl

Observed Behaviour:

  • Initially, the model loading error was InvalidState: perceptron: slice size was not equal to model size in header.
  • This was followed by a Segmentation fault (core dumped) when Juman++ attempted to process input.
  • After a completely clean re-download of the tar.xz package (using wget), removal of old build artifacts and installed files, and re-application of the CMake fixes as detailed above, the error now consistently is:
    failed to load model from disk: InvalidState: perceptron: failed to load perceptron information
    backtrace:
        loadModel at /home/acomp/jumanpp-2.0.0-rc4/src/core/env.cc:18
        init at /home/acomp/jumanpp-2.0.0-rc4/src/jumandic/shared/jumandic_env.cc:31
    

GDB Backtrace (from when the Segmentation Fault occurred in an earlier attempt):
The segmentation fault occurred before the "perceptron: failed to load perceptron information" error became the primary stopping point. This GDB backtrace points to an issue in the RNN component, likely due to the model not being loaded correctly.

Program received signal SIGSEGV, Segmentation fault.  
0x000055555580b9b4 in jumanpp::rnn::mikolov::MikolovRnn::applyParallel(jumanpp::rnn::mikolov::ParallelStepData*) const ()  
(gdb) bt  
#0 0x000055555580b9b4 in jumanpp::rnn::mikolov::MikolovRnn::applyParallel(jumanpp::rnn::mikolov::ParallelStepData*) const ()  
#1 0x00005555556423d5 in jumanpp::core::analysis::GbeamRnnState::scoreBoundary(unsigned int) ()  
#2 0x0000555555642c98 in jumanpp::core::analysis::GbeamRnnState::scoreLattice(jumanpp::core::analysis::Lattice*, jumanpp::core::analysis::ExtraNodesContext const*) ()  
#3 0x000055555563eb5c in jumanpp::core::analysis::RnnScorerGbeam::scoreLattice(jumanpp::core::analysis::Lattice*, jumanpp::core::analysis::ExtraNodesContext const*, unsigned int) ()  
#4 0x000055555562ee61 in jumanpp::core::analysis::AnalyzerImpl::computeScoresGbeam(jumanpp::core::analysis::ScorerDef const*) ()  
#5 0x0000555555630781 in jumanpp::core::analysis::AnalyzerImpl::computeScores(jumanpp::core::analysis::ScorerDef const*) ()  
#6 0x000055555562d21d in jumanpp::core::analysis::Analyzer::analyze(jumanpp::StringPiece, jumanpp::core::analysis::ScorePlugin*) ()  
#7 0x00005555556832a7 in jumanpp::core::input::PlainStreamReader::analyzeWith(jumanpp::core::analysis::Analyzer*) ()  
#8 0x00005555555a92d3 in main ()

Expected Behaviour:
Juman++ should successfully load the jumandic.jppmdl model included in the release package and perform morphological analysis.

Suspicion:
This persistent failure related to the "perceptron" component of the model, even after ensuring a pristine download and build environment, suggests a potential issue with the jumandic.jppmdl file as distributed in the jumanpp-2.0.0-rc4.tar.xz package, or a bug in the Juman++ v2.0.0-rc4 code that prevents it from correctly parsing this model.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions