[Bug]: Kafka Consumer Error initializing transform on Hop Server

[petertsneg]

Apache Hop version?

2.16

Java version?

17.0.12.7

Operating system

Windows

What happened?

1. hop-server.bat start project.
2. postman http://localhost:8080/hop/asyncRun/?service=kafka_consumer
3. Log:
   2025/11/24 13:52:58 - consumer - Executing this pipeline using the Local Pipeline Engine with run configuration 'local'
   2025/11/24 13:52:58 - consumer - Execution started for pipeline [consumer]
   2025/11/24 13:52:58 - run - Executing this pipeline using the Local Pipeline Engine with run configuration 'Kafka'
   2025/11/24 13:52:58 - run - Execution started for pipeline [run]
   2025/11/24 13:52:58 - Kafka Consumer1.0 - ERROR: Error initializing sub-transformation
   2025/11/24 13:52:58 - Kafka Consumer1.0 - ERROR: org.apache.hop.core.exception.HopException:
   2025/11/24 13:52:58 - Kafka Consumer1.0 - Unable to load and initialize sub pipeline
   2025/11/24 13:52:58 - Kafka Consumer1.0 - at java.lang.Thread.run (Thread.java:840)
   2025/11/24 13:52:58 - Kafka Consumer1.0 - at org.apache.hop.pipeline.transform.TransformInitThread.run (TransformInitThread.java:66)
   2025/11/24 13:52:58 - Kafka Consumer1.0 - at org.apache.hop.pipeline.transforms.kafka.consumer.KafkaConsumerInput.init (KafkaConsumerInput.java:98)
   2025/11/24 13:52:58 - Kafka Consumer1.0 - at org.apache.hop.pipeline.transforms.kafka.consumer.KafkaConsumerInput.initSubPipeline (KafkaConsumerInput.java:135)
   2025/11/24 13:52:58 - Kafka Consumer1.0 - at org.apache.hop.pipeline.TransformWithMappingMeta.replaceVariableValues (TransformWithMappingMeta.java:316)
   2025/11/24 13:52:58 - Kafka Consumer1.0 - at org.apache.hop.pipeline.TransformWithMappingMeta.replaceVariableValues (TransformWithMappingMeta.java:309)
   2025/11/24 13:52:58 - Kafka Consumer1.0 - at org.apache.hop.pipeline.TransformWithMappingMeta.isInternalVariable (TransformWithMappingMeta.java:326)
   2025/11/24 13:52:58 - Kafka Consumer1.0 - at org.apache.hop.pipeline.TransformWithMappingMeta.isWorkflowInternalVariable (TransformWithMappingMeta.java:335)
   2025/11/24 13:52:58 - Kafka Consumer1.0 - at java.util.ImmutableCollections$SetN.contains (ImmutableCollections.java:937)
   2025/11/24 13:52:58 - Kafka Consumer1.0 - at java.util.Objects.requireNonNull (Objects.java:209)
   2025/11/24 13:52:58 - Kafka Consumer1.0 -
   2025/11/24 13:52:58 - Kafka Consumer1.0 - at org.apache.hop.pipeline.transforms.kafka.consumer.KafkaConsumerInput.initSubPipeline(KafkaConsumerInput.java:204)
   2025/11/24 13:52:58 - Kafka Consumer1.0 - at org.apache.hop.pipeline.transforms.kafka.consumer.KafkaConsumerInput.init(KafkaConsumerInput.java:98)
   2025/11/24 13:52:58 - Kafka Consumer1.0 - at org.apache.hop.pipeline.transform.TransformInitThread.run(TransformInitThread.java:66)
   2025/11/24 13:52:58 - Kafka Consumer1.0 - at java.base/java.lang.Thread.run(Thread.java:840)
   2025/11/24 13:52:58 - Kafka Consumer1.0 - Caused by: java.lang.NullPointerException
   2025/11/24 13:52:58 - Kafka Consumer1.0 - at java.base/java.util.Objects.requireNonNull(Objects.java:209)
   2025/11/24 13:52:58 - Kafka Consumer1.0 - at java.base/java.util.ImmutableCollections$SetN.contains(ImmutableCollections.java:937)
   2025/11/24 13:52:58 - Kafka Consumer1.0 - at org.apache.hop.pipeline.TransformWithMappingMeta.isWorkflowInternalVariable(TransformWithMappingMeta.java:335)
   2025/11/24 13:52:58 - Kafka Consumer1.0 - at org.apache.hop.pipeline.TransformWithMappingMeta.isInternalVariable(TransformWithMappingMeta.java:326)
   2025/11/24 13:52:58 - Kafka Consumer1.0 - at org.apache.hop.pipeline.TransformWithMappingMeta.replaceVariableValues(TransformWithMappingMeta.java:309)
   2025/11/24 13:52:58 - Kafka Consumer1.0 - at org.apache.hop.pipeline.TransformWithMappingMeta.replaceVariableValues(TransformWithMappingMeta.java:316)
   2025/11/24 13:52:58 - Kafka Consumer1.0 - at org.apache.hop.pipeline.transforms.kafka.consumer.KafkaConsumerInput.initSubPipeline(KafkaConsumerInput.java:135)
   2025/11/24 13:52:58 - Kafka Consumer1.0 - ... 3 more
   2025/11/24 13:52:58 - Kafka Consumer1.0 - ERROR: Error initializing transform [Kafka Consumer1]
   2025/11/24 13:52:58 - consumer - ERROR: Transform [Kafka Consumer1.0] failed to initialize!
   2025/11/24 13:52:58 - consumer.hpl - ERROR: Unable to prepare for execution of the pipeline
   2025/11/24 13:52:58 - consumer.hpl - ERROR: org.apache.hop.core.exception.HopException:
   2025/11/24 13:52:58 - consumer.hpl - We failed to initialize at least one transform. Execution can not begin!
   2025/11/24 13:52:58 - consumer.hpl -
   2025/11/24 13:52:58 - consumer.hpl -
   2025/11/24 13:52:58 - consumer.hpl - at org.apache.hop.pipeline.Pipeline.prepareExecution(Pipeline.java:1091)
   2025/11/24 13:52:58 - consumer.hpl - at org.apache.hop.pipeline.engines.local.LocalPipelineEngine.prepareExecution(LocalPipelineEngine.java:232)
   2025/11/24 13:52:58 - consumer.hpl - at org.apache.hop.pipeline.Pipeline.execute(Pipeline.java:529)
   2025/11/24 13:52:58 - consumer.hpl - at org.apache.hop.workflow.actions.pipeline.ActionPipeline.execute(ActionPipeline.java:543)
   2025/11/24 13:52:58 - consumer.hpl - at org.apache.hop.workflow.Workflow.executeFromStart(Workflow.java:767)
   2025/11/24 13:52:58 - consumer.hpl - at org.apache.hop.workflow.Workflow.executeFromStart(Workflow.java:916)
   2025/11/24 13:52:58 - consumer.hpl - at org.apache.hop.workflow.Workflow.executeFromStart(Workflow.java:452)
   2025/11/24 13:52:58 - consumer.hpl - at org.apache.hop.workflow.Workflow.startExecution(Workflow.java:312)
   2025/11/24 13:52:58 - consumer.hpl - at org.apache.hop.workflow.engines.local.LocalWorkflowEngine.startExecution(LocalWorkflowEngine.java:249)
   2025/11/24 13:52:58 - consumer.hpl - at java.base/java.lang.Thread.run(Thread.java:840)

Issue Priority

Priority: 3

Issue Component

Component: Hop Server

[hansva]

Does the pipeline work when it is not being executed via asyncRun ?

[petertsneg]

The program was working correctly in version 2.10, but the error appeared after the upgrade to 2.16. The error persists even after simplifying the pipeline to include only an Injector and an Output.

[sramazzina]

@petertsneg you didn't answered to @hansva question that is very important. Can you give us an answer?

[petertsneg]

WebService test successful, thank you.

[sramazzina]

@petertsneg not clear to me if the issue was solved and we can close the issue or if it still exists. Thank you

[petertsneg]

@sramazzina Using asyncRun fails, but using WebService for the pipeline call succeeds.Thank you

[petertsneg]

The issue was identified in v2.16 Asynchronous Web Service, where the addition of the Request header content variable field without a defined variable name caused an error. Thank you all for your assistance.