#logstash

Afternoon all - I'm having issues shipping both nginx access and error logs to Logstash, using Filebeat. Access logs appear to be shipping, and Logstash appears to be receiving error logs (with debug log level, I see a number of "output received" messages for error log entries), but they do not appear in the target index.

My config files are here (for Filebeat and Logstash): https://gist.github.com/g0blinResearch/3eed96ab...

Can anyone shed any light on this please? It's getting very frustrating to see the logs arriving, but not appearing in the ES index.

Hi guys! quick question:

can i match with grok a field on a message and then based on the value of a field, make a specific grok?

something like: host=badum,typefield=type1 interval=10 value=5 and then "if typefield=type1: grok this way. else if typefield=type2: grok that other way"

Ah - if anyone wants to know *why* I wasn't seeing my nginx error logs appearing - access logs have the timezone (+0300), where as the error logs did not. My error logs were appearing three hours ahead. Have added a mutate to mitigate.

Gotxi - that's exactly what I'm going for parsing nginx error / access logs in one endpoint: https://gist.github.com/g0blinResearch/3eed96ab...

Check in nginx.yml, where I'm switching between source.

cool! this should work

thanks man

hmm i see it is not exactly what i need, on filebeat you already specify a source, i dont know what my source is unless i do a initial grok. Then based on the source by that initial grok, i would like to grok again with a specific pattern according to the source

Can I apply a filter on looping on all fields starting with _keyword_ ?

Can I apply a filter looping on all fields starting with _keyword_ ?