[feature] add functional scheduler to separate critical and functional modules processing

[ipaqsa]

Overview

It adds functional scheduler to separate critical and functional modules processing.

It continues it

What this PR does / why we need it

The scheduler is designed to process functional modules, ensuring they are only run after their dependencies have been processed, and executes their ModuleRun tasks in parallel queues.

During converge functional modules loaded into the scheduler. And the converge task waits until the scheduler finished processing. While waiting the converge task can skip ahead other tasks in the queue if they are waiting it means the functional processing DOES NOT BLOCK the main queue.

How the Scheduler Works

• Dependency Management:
  The scheduler tracks dependencies between functional modules. Each module can declare a list of dependencies, and the scheduler ensures that a module is only scheduled for processing after all its dependencies done(after ModuleRun done it sends signal to the scheduler).

• Parallel Execution:
  Functional modules whose dependencies are satisfied are scheduled to run in parallel queues. The number of parallel queues is configurable (increased to 20 in this PR), and the scheduler distributes tasks across them.

• Critical vs Functional Modules:
  Critical modules are processed separately and are not managed by the functional scheduler. The scheduler only handles functional modules.

Checklist

• The code is covered by unit tests.
• Changes were tested in the Kubernetes cluster manually.

[Copilot]

Pull Request Overview

This PR separates critical and functional modules by introducing a Critical flag, implementing a functional scheduler, and updating task handling and queues to route non-critical modules through the new scheduler.

• Add Critical metadata to tasks and pass it through parallel and converge workflows.
• Implement and integrate a functional.Scheduler for non-critical module execution.
• Expose queue length, adjust parallel queue count, and wire up scheduler dependencies in service and module manager.

Reviewed Changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
pkg/task/tasks/parallel-module-run/task.go	Pass Critical flag to parallel module run tasks
pkg/task/tasks/converge-modules/task.go	Integrate functional scheduler and mark critical tasks
pkg/task/service/service.go	Initialize functional scheduler and handle non-critical tasks
pkg/task/service/converge.go	Lower log level for first converge completion
pkg/task/queue/queue.go	Add method to retrieve queue length
pkg/task/hook_metadata.go	Extend HookMetadata with Critical field
pkg/task/functional/scheduler.go	Implement functional task scheduler
pkg/task/functional/scheduler_test.go	Add tests for functional scheduler
pkg/module_manager/scheduler/scheduler.go	Expose functional dependencies and critical checks
pkg/module_manager/module_manager.go	Add module criticality getters
pkg/app/app.go	Increase number of parallel queues

Comments suppressed due to low confidence (3)

pkg/task/tasks/converge-modules/task.go:383

• [nitpick] The comment here is misleading: this branch actually collects functional modules for later scheduling. Consider updating it to something like // collect functional modules to reflect intent.

		// skip functional modules

pkg/task/functional/scheduler.go:19

• [nitpick] Defining Root as an empty string can be confusing. Consider using a more explicit sentinel or documenting its semantics to avoid ambiguity in calls like Done(Root).

	Root = ""

pkg/task/tasks/converge-modules/task.go:185

• The new functional scheduling logic in Handle (trigger/delay loop) and CreateConvergeModulesTasks is not covered by existing tests. Consider adding unit or integration tests to verify that functional modules are correctly scheduled and completed in the WaitDeleteAndRunModules phase.

		// trigger functional converge