From the Blogosphere
Logentries and Logstash Integration | @DevOpsSummit [#DevOps]
Logstash is an open source tool for managing events and logs
By: Trevor Parsons
Mar. 6, 2015 01:00 PM
Getting Started with the Logentries & Logstash Integration
Logstash is an open source tool for managing events and logs. It is used to collect, search and store logs for later use. If you are using Logstash to collect logs from across your infrastructure already, and you are looking for more sophisticated log analytics tool, you are in the right place.
I will show you how to configure Logstash to forward all your logs to your Logentries account using the plugin and token connection.
The contrib plugins come with a pre-installed Logentries plugin. In order to forward logs from Logstash to your Logentries account you need to create a configuration file in your main Logstash folder. Each plugin has different settings for configuring it. There are three main sections in every configuration file: inputs, filters, outputs.
Let's call our configuration file connection.conf for now and start to fill out these fields one by one.
The user is able to assign additional setting to the input configuration such as:
Parameters listed above vary based on input source and configuration.
Ok, what is actually happening here?
Firstly, we are using grok filter, which is currently the best way in Logstash to parse badly, unstructured log data into something structured and queryable. Grok makes it easy for you to parse logs with regular expressions, by assigning labels to commonly used patterns. One such label is called COMBINEDAPACHELOG.
188.8.131.52 LOG message='184.108.40.206 - kurt [18/May/2011:01:48:10 -0700] "GET /admin HTTP/1.1" 301 566 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:220.127.116.11) Gecko/20100401 Firefox/3.6.3"' @version=1 @timestamp='2015-02-19T17:59:49.834Z' host='Bart-MacBook-Pro.local' path='/var/log/Apache.log'
18.104.22.168 LOG message='22.214.171.124 - kurt [18/May/2011:01:48:10 -0700] "GET /admin HTTP/1.1" 301 566 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:126.96.36.199) Gecko/20100401 Firefox/3.6.3"' @version=1 @timestamp='2015-02-19T18:07:37.437Z' host='Bart-MacBook-Pro.local' path='/var/log/Apache.log' clientip=188.8.131.52 ident='-' auth=kurt timestamp='18/May/2011:01:48:10 -0700' verb=GET request='/admin' httpversion=1.1 response=301 bytes=566 referrer='"-"' agent='"Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:184.108.40.206) Gecko/20100401 Firefox/3.6.3"'
Start Sending Logs
Simply save your configuration file and run bin/logstash -f connection.conf. Your logs will now forward directly into your Logentries account and be easily accessible for tagging, real-time alerting, and data visualizations. Don't have a Logentries account? Get started here in minutes for free!
Latest Cloud Developer Stories
Subscribe to the World's Most Powerful Newsletters
Subscribe to Our Rss Feeds & Get Your SYS-CON News Live!
SYS-CON Featured Whitepapers
Most Read This Week