Exporting From Kibana

For anyone who uses Logstash, chances are you also use Kibana (http://www.elasticsearch.org/overview/kibana/). Kibana is a nice Web UI for querying Elasticsearch. It is Logstash’s official UI. One major gripe with the current state of Kibana is that it does not support exporting search results. I wrote a tool to handle this problem while we wait for the real feature.

As this issue will suggest, many people want Kibana to support exporting- https://github.com/elasticsearch/kibana/issues/205

Our operations team likes to attach logs to tickets as a means of documenting issues. I wrote a tool in ruby that queries elasticsearch and exports results into CSV (or delimiter of your choice). The project is hosted here: https://github.com/robbydyer/stash-query

The tool is CLI-based, so it’s not exactly the nice button-clicking format that many support type people like, but it’s a nice stop-gap measure.

Install It

UPDATE: I’ve published version 0.1.2 to rubygems.org, so you can simply install it with:

gem install stash-query

You can optionally build your own gem from the source. Download the source from github, build it, then install it.

gem build stash-query.gemspec

gem install stash-query-0.1.0.gem

Run It

Usage:

-c, --connect_host [HOST] Logstash host to run query on (defaults to: localhost)
-p, --port [PORT] Logstash port (defaults to: 9200)
-i, --index-prefix [PREFIX] Index name prefix. Defaults to 'logstash-'
-w, --write [FILE] Write output file location (defaults to nil)
-d, --debug Debug mode
-s, --start [DATE] Start date. Format: YYYY-MM-DDThh:mm:ss.SSSZ. Ex: 2013-12-01T12:00:00.000Z
-e, --end [DATE] End date. Format: YYYY-MM-DDThh:mm:ss.SSSZ
-q, --query [QUERY] Query string
-t, --tags [TAGS] Tags to query. Comma delimited
-f, --write-fields [FIELDS] Comma delimited list of Logstash fields to write to output file. Defaults to "message"
-l, --delimiter [DELIMITER] Delimiter to use in output file. Defaults to ','

Example:

stash-query -s 2013-12-01T00:00:00.000Z -e 2013-12-02T00:00:00.000Z -t my_tag -q 'message:hello_world' -w /tmp/my_query.txt