#logstash

Tinuviel: I set this on the 'search node' in the elasticsearch config, node.data: false

also, node.master: false so that it can't take over the ES cluster

I see, it looks that I also have no data node

I think it's created automaticaly by Kibana

so maymy my problem is to tune it to cache some more query results

*maybe

Tinuviel: if you're using the embedded elasticsearch from logstash, then yeah this won't work.

Tinuviel: if you're wanting to scale this like you're talking about, I'd recommend installing an elasticsearch instance

Any suggestions for "FacetPhaseExecutionException[Facet [0]: (key) field [@timestamp] not found]"? This is a new elasticsearch/kibana/LS install ...

attractiveape I have Elasticsearch cluster

on 6servers

one of ES node is on logstash server

in logstash config I have:

output { elasticsearch { host => "127.0.0.1" } }

and I think it still, create no data node

I have right now 7nodes

sorry 8 nodes

Tinuviel: that's logstash's config, not elasticsearch's. you need to look at elasticsearch's config

however I started only 7

last one is created by Kibana/logstash mostly probably

logstash will join the ES cluster as a client, but it shows up in the cluster listing, yes. that's a little confusing.

Yes

kibana doesn't connect to the cluster in that manner, just logstash.

I have kibaba-int index

and one extra node, I think it's created by logstash

it's no data node

attributes: { client: true data: false }

So, I think I should start another ES node, with nodata in it's config and point logstash to it. If I get it right, logstash won't create in this case dedicated nodata node but will use this one I just created and pointed logstash to.

question is, I will be steel able to write data to my ES cluster?

Hmm, probably yes, if I can right now.

grok question: how to mutate when there's match? Trying to get timestamps out, mostly it's 100ms , easy, nice.. I want ms. But there are some s* software that logs as 0.100 sec ...

grok works (%{BASE10NUM:ts}\s?se?c?) .. but how to * 100... or just gsub the '.' got

hi all

I am trying to run logstash, kibana and elasticsearch from the jar behind an UFW

but the thing is that if I block 9200 and other ports the kibana sais thatthe elastic search cannot be reached

I have set ufw for 127.0.0.1 to allow 9200 but still does not work

anybody has any idea?

qwebirc944212: kibana3 is using javascript to talk directly to the ES cluster, you need to open that up

what port?

9200

but only internal. I do not wnt to open it from the internet

is this paossible?

qwebirc944212: not with kibana3.

qwebirc944212: you can put nginx in front of elasticsearch, and do IP-based access control maybe?

qwebirc944212: https://github.com/elasticsearch/kibana/blob/ma... might help you with that

<http://tinyurl.com/qd28jft>; (at github.com)

I have appache

but is it safe to have 9200 open for the Internet?

qwebirc944212: I wouldn't advise it. An apache example is here: https://github.com/elasticsearch/kibana/blob/ma...

<http://tinyurl.com/ooksbe7>; (at github.com)

attractiveape: thanks now I see

qwebirc944212: glad I could help

attractiveape: rror Could not contact Elasticsearch at http://xx:9200. Please ensure that Elasticsearch is reachable from your system.

still the same issue

what can I do?

qwebirc944212: test you can connect via curl or your browser to elasticsearch or nginx.

curl -Is http://127.0.0.1:9200 | head -n 1 HTTP/1.1 200 OK

note I run the embedded elastic search with logtsash

qwebirc944212: the embedded logstash only binds to 127.0.0.1 I believe, I don't know if that can be changed

so why would not work then with the firewall enabled

I have also 9200/tcp ALLOW IN 127.0.0.1

in ufw

could need udp?

qwebirc944212: ES is lisening on 127.0.0.1, so only requests on that host to that host can connect to the ES port

Parsing a json give me @field.duration... how can I access that on a mutate , or in the json block itself? add_field => ["ts", "%{@field.duration}"]

qwebirc944212, try 0.0.0.0

nofxx: in the UFW?

UFW? on ES

nofxx: %{field.duration} I think you want

attractiveape, could swear I've tryed... testing again. Thanks

nofxx: oh! sorry I missed something

nofxx: %{field\.duration}

attractiveape, same thing, only the text... it's doesnt get evaluated

'%{field\.duration}'

nofxx: I haven't done mutates in quite some time, can't offer anymore than than :(

attractiveape: I tried different things and is not working unless I open the port 9200 system wide

qwebirc944212: how is apache configured to proxy? Talking to 127.0.0.1:9200 ?

yes

I use localhost

shoild I use 27.0.0.1

that should be right then...check the apache logfiles?

New news from newjiraissues: tongyizuguo created LOGSTASH-1546 - The Example of Grok is don't work <https://logstash.jira.com/browse/LOGSTASH-1546>;

ES delete index api supports wildcards right? curl -XDELETE 'http://localhost:9200/logstash-2006*' doesn't seem to work for me...

deleting specific indexes does though