[WIP] refactor: Implement modular candle-binding architecture (#254)

[rootfs]

• Restructure codebase into modular layers (core/, ffi/, model_architectures/, classifiers/)
• Add unified error handling and configuration loading systems
• Implement dual-path architecture for traditional and LoRA models
• Add comprehensive FFI layer with memory safety

Maintains backward compatibility while enabling future model integrations.

refactor: Implement modular candle-binding architecture 

• Restructure codebase into modular layers (core/, ffi/, model_architectures/, classifiers/)
• Add unified error handling and configuration loading systems
• Implement dual-path architecture for traditional and LoRA models
• Add comprehensive FFI layer with memory safety

Maintains backward compatibility while enabling future model integrations.

What type of PR is this?

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #

Release Notes: Yes/No

[netlify]

✅ Deploy Preview for vllm-semantic-router ready!

Name	Link
🔨 Latest commit	a48b0be
🔍 Latest deploy log	https://app.netlify.com/projects/vllm-semantic-router/deploys/68dbcf9933a669000858d5e3
😎 Deploy Preview	https://deploy-preview-266--vllm-semantic-router.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[github-actions]

👥 vLLM Semantic Team Notification

The following members have been identified for the changed files in this PR and have been automatically assigned:

📁 candle-binding

Owners: @rootfs
Files changed:

• candle-binding/src/classifiers/lora/intent_lora.rs
• candle-binding/src/classifiers/lora/intent_lora_test.rs
• candle-binding/src/classifiers/lora/mod.rs
• candle-binding/src/classifiers/lora/parallel_engine.rs
• candle-binding/src/classifiers/lora/pii_lora.rs
• candle-binding/src/classifiers/lora/pii_lora_test.rs
• candle-binding/src/classifiers/lora/security_lora.rs
• candle-binding/src/classifiers/lora/security_lora_test.rs
• candle-binding/src/classifiers/lora/token_lora.rs
• candle-binding/src/classifiers/lora/token_lora_test.rs
• candle-binding/src/classifiers/mod.rs
• candle-binding/src/classifiers/traditional/batch_processor.rs
• candle-binding/src/classifiers/traditional/batch_processor_test.rs
• candle-binding/src/classifiers/traditional/mod.rs
• candle-binding/src/classifiers/traditional/modernbert_classifier.rs
• candle-binding/src/classifiers/traditional/modernbert_classifier_test.rs
• candle-binding/src/classifiers/unified.rs
• candle-binding/src/classifiers/unified_test.rs
• candle-binding/src/core/config_loader.rs
• candle-binding/src/core/config_loader_test.rs
• candle-binding/src/core/mod.rs
• candle-binding/src/core/similarity.rs
• candle-binding/src/core/tokenization.rs
• candle-binding/src/core/unified_error.rs
• candle-binding/src/core/unified_error_test.rs
• candle-binding/src/ffi/classify.rs
• candle-binding/src/ffi/classify_test.rs
• candle-binding/src/ffi/init.rs
• candle-binding/src/ffi/memory.rs
• candle-binding/src/ffi/memory_safety.rs
• candle-binding/src/ffi/memory_safety_test.rs
• candle-binding/src/ffi/mod.rs
• candle-binding/src/ffi/similarity.rs
• candle-binding/src/ffi/state_manager.rs
• candle-binding/src/ffi/tokenization.rs
• candle-binding/src/ffi/types.rs
• candle-binding/src/ffi/validation.rs
• candle-binding/src/model_architectures/config.rs
• candle-binding/src/model_architectures/lora/bert_lora.rs
• candle-binding/src/model_architectures/lora/bert_lora_test.rs
• candle-binding/src/model_architectures/lora/lora_adapter.rs
• candle-binding/src/model_architectures/lora/mod.rs
• candle-binding/src/model_architectures/mod.rs
• candle-binding/src/model_architectures/model_factory.rs
• candle-binding/src/model_architectures/model_factory_test.rs
• candle-binding/src/model_architectures/routing.rs
• candle-binding/src/model_architectures/routing_test.rs
• candle-binding/src/model_architectures/traditional/base_model.rs
• candle-binding/src/model_architectures/traditional/base_model_test.rs
• candle-binding/src/model_architectures/traditional/bert.rs
• candle-binding/src/model_architectures/traditional/bert_test.rs
• candle-binding/src/model_architectures/traditional/mod.rs
• candle-binding/src/model_architectures/traditional/modernbert.rs
• candle-binding/src/model_architectures/traditional/modernbert_test.rs
• candle-binding/src/model_architectures/traits.rs
• candle-binding/src/model_architectures/unified_interface.rs
• candle-binding/src/model_architectures/unified_interface_test.rs
• candle-binding/src/test_fixtures.rs
• candle-binding/src/utils/memory.rs
• candle-binding/src/utils/mod.rs
• candle-binding/Cargo.toml
• candle-binding/src/lib.rs

📁 config

Owners: @rootfs
Files changed:

• config/config.yaml

📁 tools

Owners: @yuluo-yx, @rootfs, @Xunzhuo
Files changed:

• tools/make/build-run-test.mk
• tools/make/common.mk
• tools/make/rust.mk

---

🎉 Thanks for your contributions!

This comment was automatically generated based on the OWNER files in the repository.

[rootfs]

@OneZero-Y @Xunzhuo Let's have the following resolved before merging

• Add more candle unit tests
• Verify API accuracy
• Ensure semantic-router use the right binding API
• Remove legacy comment and code

[rootfs]

@OneZero-Y now since we work on the feature branch, how about you use this branch for both refactoring and new embedding models?