Comments
yourfanat wrote: I am using another tool for Oracle developers - dbForge Studio for Oracle. This IDE has lots of usefull features, among them: oracle designer, code competion and formatter, query builder, debugger, profiler, erxport/import, reports and many others. The latest version supports Oracle 12C. More information here.
Cloud Expo on Google News
SYS-CON.TV
Cloud Expo & Virtualization 2009 East
PLATINUM SPONSORS:
IBM
Smarter Business Solutions Through Dynamic Infrastructure
IBM
Smarter Insights: How the CIO Becomes a Hero Again
Microsoft
Windows Azure
GOLD SPONSORS:
Appsense
Why VDI?
CA
Maximizing the Business Value of Virtualization in Enterprise and Cloud Computing Environments
ExactTarget
Messaging in the Cloud - Email, SMS and Voice
Freedom OSS
Stairway to the Cloud
Sun
Sun's Incubation Platform: Helping Startups Serve the Enterprise
POWER PANELS:
Cloud Computing & Enterprise IT: Cost & Operational Benefits
How and Why is a Flexible IT Infrastructure the Key To the Future?
Click For 2008 West
Event Webcasts
Logentries and Logstash Integration | @DevOpsSummit [#DevOps]
Logstash is an open source tool for managing events and logs

Getting Started with the Logentries & Logstash Integration
by Bartlomiej Siniarski

Logstash is an open source tool for managing events and logs. It is used to collect, search and store logs for later use. If you are using Logstash to collect logs from across your infrastructure already, and you are looking for more sophisticated log analytics tool, you are in the right place.

I will show you how to configure Logstash to forward all your logs to your Logentries account using the plugin and token connection.

Prerequisites

The contrib plugins come with a pre-installed Logentries plugin. In order to forward logs from Logstash to your Logentries account you need to create a configuration file in your main Logstash folder. Each plugin has different settings for configuring it. There are three main sections in every configuration file: inputs, filters, outputs.

#Configuration file
input {
...
}
filter {
...
}
output {
...
}

Let's call our configuration file connection.conf for now and start to fill out these fields one by one.

Input
The input section can be configured to read from Elasticsearch cluster, local file, syslog, tcp, udp, Heroku and many more. In this post we are going to read from our local access.log file.

input {
file {
path => "/var/log/access.log"
}
}

The user is able to assign additional setting to the input configuration such as:

  • path
  • codec
  • start_position
  • tags
  • host
  • port

Parameters listed above vary based on input source and configuration.

Filter
Filters are used as intermediary processing devices in the Logstash chain. They are often combined with conditionals in order to perform a certain action on an event, if it matches particular criteria. I will present the output with and without active filter.

filter {
grok {
match => { "message" => "%{COMBINEDAPACHELOG}" }
}
}

Ok, what is actually happening here?

Firstly, we are using grok filter, which is currently the best way in Logstash to parse badly, unstructured log data into something structured and queryable. Grok makes it easy for you to parse logs with regular expressions, by assigning labels to commonly used patterns. One such label is called COMBINEDAPACHELOG.

Filter Inactive

46.7.24.63 LOG message='111.141.244.242 - kurt [18/May/2011:01:48:10 -0700] "GET /admin HTTP/1.1" 301 566 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3"' @version=1 @timestamp='2015-02-19T17:59:49.834Z' host='Bart-MacBook-Pro.local' path='/var/log/Apache.log'

Filter Active

46.7.24.63 LOG message='111.141.244.242 - kurt [18/May/2011:01:48:10 -0700] "GET /admin HTTP/1.1" 301 566 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3"' @version=1 @timestamp='2015-02-19T18:07:37.437Z' host='Bart-MacBook-Pro.local' path='/var/log/Apache.log' clientip=111.141.244.242 ident='-' auth=kurt timestamp='18/May/2011:01:48:10 -0700' verb=GET request='/admin' httpversion=1.1 response=301 bytes=566 referrer='"-"' agent='"Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3"'

Output
This section takes advantage of the plugin and configures Logstash to forward all logs from access.log locally stored in our machine to Logentries account using unique token.

output {
logentries{
token => "LOGENTRIES_TOKEN"
}
}

Start Sending Logs
The plugin has to be stored in your logstash-outputs folder:

logstash-x.x.x
├── bin
├── lib
│ └── logstash
│ └── outputs
│ └── logentries.rb
├── LICENSE
├── locales
├── connection.conf
├── patterns
├── README.md
├── spec

Simply save your configuration file and run bin/logstash -f connection.conf. Your logs will now forward directly into your Logentries account and be easily accessible for tagging, real-time alerting, and data visualizations. Don't have a Logentries account? Get started here in minutes for free!

About Trevor Parsons
Trevor Parsons is Chief Scientist and Co-founder of Logentries. Trevor has over 10 years experience in enterprise software and, in particular, has specialized in developing enterprise monitoring and performance tools for distributed systems. He is also a research fellow at the Performance Engineering Lab Research Group and was formerly a Scientist at the IBM Center for Advanced Studies. Trevor holds a PhD from University College Dublin, Ireland.

Latest Cloud Developer Stories
The use of containers by developers -- and now increasingly IT operators -- has grown from infatuation to deep and abiding love. But as with any long-term affair, the honeymoon soon leads to needing to live well together ... and maybe even getting some relationship help along the...
Is advanced scheduling in Kubernetes achievable?Yes, however, how do you properly accommodate every real-life scenario that a Kubernetes user might encounter? How do you leverage advanced scheduling techniques to shape and describe each scenario in easy-to-use rules and configura...
Platform-as-a-Service (PaaS) is a technology designed to make DevOps easier and allow developers to focus on application development. The PaaS takes care of provisioning, scaling, HA, and other cloud management aspects. Apache Stratos is a PaaS codebase developed in Apache and de...
The Japan External Trade Organization (JETRO) is a non-profit organization that provides business support services to companies expanding to Japan. With the support of JETRO's dedicated staff, clients can incorporate their business; receive visa, immigration, and HR support; find...
DevOps is a world surrounded by information, starting from a single commit and ending in roll out to production. In this talk, I'll introduce you to the world of Taboola DevOps data collection, to better understand what goes on under the hood. The system we've developed in-hou...
Subscribe to the World's Most Powerful Newsletters
Subscribe to Our Rss Feeds & Get Your SYS-CON News Live!
Click to Add our RSS Feeds to the Service of Your Choice:
Google Reader or Homepage Add to My Yahoo! Subscribe with Bloglines Subscribe in NewsGator Online
myFeedster Add to My AOL Subscribe in Rojo Add 'Hugg' to Newsburst from CNET News.com Kinja Digest View Additional SYS-CON Feeds
Publish Your Article! Please send it to editorial(at)sys-con.com!

Advertise on this site! Contact advertising(at)sys-con.com! 201 802-3021



SYS-CON Featured Whitepapers
Most Read This Week
ADS BY GOOGLE