#logstash

jegaxd26: It probably should've worked like you expect, but...

jegaxd26: The default index template has some general ideas about fields and datatypes. GeoIP fields have to be explicitly defined.

bjorn_: It works if I specify conversion as you've suggested, thank you

:)

Morning, I found that when you have "Prog: x=1 y=2 z=3" but x,y,z aren't always in the same order or even exist together, it seems best to have many grok's each only extracting the one k/v, can anyone recommend a less duplicatey way of doing this? I tried an array of grok matches but it only seems to match the first

Use the kv filter

bjorn_: It makes so much sense, it just might work. Thanks

np

Note that you can whitelist and blacklist as well.

I have log messages that contain data up to an offset, fields up to the offset aren't defined by their lengths and fields after the offset can be arbitrarily large as well. I managed to get this working by chaining three groks together, but it feels like that won't perform well, can someone suggest an alternative to my approach?

it's hard to provide a better alternative without anything resembling a sample input. however, generally speaking, three groks are not necessarily bad and sometimes even perform better than any alternative

Xylakant: thanks, it seems Ok now, I'm able to run just under 1m loglines via filebeat and it all gets loaded quickly, was mostly asking in case I missed something obvious

Hi Guys.. has anyone succesfully implemented the twitter input plugin with logstash 6.1.2?

As a test, I have a file input plugin which forwards all data correctly into elasticsearch.. (can also be found in kibana)

Though, I cannot seem to find any specific debug messages for my twitter plugin

I'm on a fresh Centos7 with 4GB of Ram and 2 CPU cores. (also tried with ubuntu 17.10)

i am following this tutorial https://www.elastic.co/blog/geoip-in-the-elasti... because im having some problems with geoip. I figured that the problem is that my logstash returns locations as "location" => {"lat" => 151.2,"lon" => -33.9167} instead of "location" => [ [0] 151.2,[1] -33.9167] that i see in the tutorial, so elasticsearch doesn't parse it as geo_point. What might cause this problem?