Log management isn’t a new concept. As long as there have been computers they have generated logs. But with the advent of modern server and application software, logging has become a staple of the IT management and monitoring process. And, until recently, this is primarily because of the ease of generating log data — not in using it.
Servers and applications can generate log data on a variety of processes, from simply announcing that everything is healthy to detailed information on events and processes running. The trick to effective log management is knowing what needs to be monitored and managed, and having the right tool in place that gives you flexibility to get the deepest, most valuable insights from the mountains of data that logs can contain. To give you a starting point on your own use of log management and monitoring tools we have provided some basic ideas for how to use these log management tools in the most popular areas across the Logentries community.
Monitoring your production systems is the most common use of log management tools. In this use case you have a known state that indicates the optimum range of efficiency and effectiveness for your operating production systems. In this environment you will set a range of operating conditions that represents that optimal operating state. You can then easily configure your log management tool to monitor, track and alert in real-time when events occur to outside the normal range of operation.
The key to rapid resolution of problems with production systems is rapid analysis of problem-related data. Log management is often the best way to implement root cause analysis. Having the data at hand and the tools in place to properly analyze it, using the information that you have derived from comparing systems that are operating properly to those indicating operational issues, makes for rapid resolution of problems. The ability to compare heuristic data in many forms and identifying the point in time that problems began occurring is invaluable in troubleshooting operating systems.
The basics of most log generation by applications, compilers, debuggers, etc., are to alert to exceptions and events. With a variety of logging consoles available from both local and web-based applications it is possible for a log management tool to track events across the operation of an application or process that spans a network and multiple applications. For development purposes this data is invaluable in debugging applications, from problems that cause a hard stop to those that bottleneck the flow of data.
Log data is often an untapped gold mine of business analytic information. From the number of transactions per hour to details on the value of individual transactions, information can be found that is both general and specific. General information, such as the popularity of specific areas of a website, can be combined with information that drills down to a specific product or page. User experience can be evaluated by determining time spent on pages or the response time to load popular pages. Business specific information can be gleaned from the contents of the logs, as well. With alerts configured to meet explicit business criteria that fit your needs.
Monitoring security and compliance issues is a forte of log management. From tracking unauthorized login or access attempts to monitoring changes made to access control lists, events are generated which log data and can therefore be watched by your log management tools. Setting conditions for alerts, such as failed login attempts per minute or the addition of new user accounts can provide a very simple means of meeting regulatory compliance regulations or simply implementing basic security event monitoring.
Good log management tools such as Logentries will allow for all of the use cases we’ve mentioned here to be done in real-time, by centralizing, analyzing and visualizing log data immediately. Information can be presented to the applicable users in the form of generated reports or even real-time graphical dashboards that update to show the manager an at-a-glance look at the data they have determined is most critical for their needs. Multiple dashboards targeted at specific business areas, and shared across the team or organization, enables data to be presented simultaneously in a form most appropriate for each individual user.