Increase size limit when importing large CSV files to Kibana/Elastic pipeline

[trizzosk]

Hello,

first of all - I tried to search but I did not find any suitable solution. I started with SecOnion some weeks ago. Now I need to import a bit larger CSV file (~40MB) to Elastic.

Kibana says the limit is 100MB, but when I use ingest pipeline from CSV, I get a message:

Payload content length greater than maximum allowed

1048576

what is that???

Thgx

[cm-ops]

server.maxPayload
The maximum payload size in bytes for incoming server requests. Default: 1048576

From https://www.elastic.co/guide/en/kibana/current/settings.html#server-maxPayload

Try setting that higher in the kibana.yml with server.maxPayloadBytes: _some_number

[trizzosk]

@cm-ops Thank you very much. That is something I found in the documentation. Howevr - I am not sure where to add this line. My 'kibana.yml' was empty ('/etc/kibana/kibana.yml').

Do I need to adjust this parameter in some saltstack minion? Or any other clue?

I am very sorry for noobie questions, but the ELK stack in general was out of my reach until now.

[cm-ops]

Try adding the following to the minion pillar file for your manager in /opt/so/saltstack/local/pillar/minions/ I used 40MB as a random number.

kibana:
  config:
    server:
      maxPayloadBytes: 40000000

Then restart Kibana and you should see the setting in the kibana.yml in /opt/so/conf/kibana/etc/.

[trizzosk]

Great thx, this worked finally.