#logstash

Ok I need to read a file from /proc/fs/lustre/*/totalfiles Where there could be more than one of these, the * is filesystemname-##### Can I grab that name to send more than one event if more than one exists?

brockp: don't completely get what you are trying

sure, i'll try again,

I want to read data out of the directory /proc/fs/lustre/mds/scratch-MDT0000/

but that last directory scratch-MDT0000 could have different names, and there may be more than one of them

ahh okay. you'll need directroy globs then /[proc/fs/lustre/mds/**/file

ie /proc/fs/lustre/mds/scratch-MDT0000/stats and /proc/fs/lustre/mds/filesystemname-MDT0001/stats

When shipping the resuts to graphite though can I somehow grab the directory name and track stats for each?

you should be able to yeah.. there is a field that contains the source. you'll need to process that field

ok, so then if the glob catches more then one entry, it will effectivly loop over all globs that match?

yeah. it will process all the dirs and get all the files

wait this is a file in proc, where the value just changes, don't I have to use the exec {} input type? Thus the source is lost?

how do you mean which value?

right, Im reading cat /proc/fs/lustre/mds/scratch-MDT0000/filesfree where the single value in that file changes as users create files in the parallel filesystem

it isn't a log file that is appended to

ahh okay. hmm

I can hardcode it for now because that filesystem name doesn't change, but we were trying to make a generic config file that would work on all metadata servers

brockp: ah okay if you have more examples of how it can appear as i might be able to help.

well the format is /proc/fs/lustre/mds/<fsname>-MDT#####/filesfree

okay.. then <fsname>-MDT##### can be replaced by ** for directory globbing.

I want to generate a stat like lustre.<fsname>.mdt.filesfree

ah okay

you could use grok to parse the source field and make a grok pattern to match it and extract the fsname from it

ok i'll try it out,

okay :-)

and again if there is more than one matching the glob it is implied to be passed thoguh once each

oh crud the lustre servers don't have java, LAME

brockp: each file is processed separately, so those will be different sources indeed

w00t!

I am getting this error on logstash agent

The error reported is: undefined method `+' for nil:NilClass

any ideas?

does mutate supprt stripping newlines?

I'm getting values like: "message" => "790655031\n",

brockp: it should support it yeah