Skip to content
This repository was archived by the owner on Jan 30, 2021. It is now read-only.

Dynamic table nameΒ #18

@ravi-oli

Description

@ravi-oli

Reference: https://www.altinity.com/blog/2017/12/18/logstash-with-clickhouse

Hello Andrey

I am trying to make the table name dynamic by passing a value which is combination of JSON fields from data being ingested. For instance:

clickhouse {
                        http_hosts => ["http://172.17.0.5:8123"]
                        user => "default"
                        password => "password"
                        # table => "w46a1d81.srstream" 
                        table => "%{stream}-%{profile_id}"
                        request_tolerance => 1
                        flush_size => 1000
                        pool_max => 1000
                        mutations => {
                                "stream" => "stream"
                                "profile_id" => "profile_id"
                                "user_id" => "user_id"
                               ....
                        }
                }

This set-up when executed throws error. Ingestion URLs are constructed like:

http://172.17.0.5:8123/?query=INSERT%20INTO%20%{stream}-%{profile_id}%20FORMAT%20JSONEachRow

Any ideas how this can be solved? Does this feature exists or we need to build it?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions