When initially setting up logging for your applications and systems, it’s often difficult to know where to start. This page provides some best practices that you can take advantage of to get the most value from your logs and so that you can apply them for different use cases such as user session tracking, performance monitoring, business analytics and more.
Below are some of our top tips that you should think about when thinking about what and how to log.
Use unique identifiers in your logs to allow you to track particular user sessions or system transactions. Unique identifiers can be instrumental in narrowing down the large amount of log data into a manageable subset e.g. to track all activity from a given user. Such unique identifiers could be:
- User ID (email address, login name, social security number, etc.)
- Transaction ID (web session ID, SQL transaction ID, etc.)
- Account ID (company name, company billing code, etc)
Without unique identifiers it can be very difficult to piece together a given system transaction or related system events.
When you’re developing your application and system logging remember that context is key! Imagine the following log line in your data:
05-15-2014 - 11:45:42:000 - wp34056 - login - 200
Now, with context:
05-15-2014 - 11:45:42:000 - USER=wp34056 - action=login - page=blog.logentries.com - referrer=www.google.com - status=200
Much easier to troubleshoot. Although the user, page, and referrer were not necessary to log, this contextual data gives important information when trying to troubleshoot.
A very common question we are asked is: what format should my log entries be in?
The answer depends on what exactly you want to use your log data for. For example if you never need to look at logs and they are purely used for automation then it can make sense for your logs to be designed mainly to be machine readable. However if you would like to be able to easily debug problems by analysing them, then you might want to make sure that they are also human readable. Two common formats that serve both purposes are:
- Key Value Pairs (KVPs)
- JSON logs
By logging in key value pair format you are able to easily read the log entries but also utilize advanced capabilities of our logging platform. So they are essentially human readable and machine readable since Logentries can parse any KVP logs. Key value pair format allows Logentries to create structure out of unstructured data. For example let’s take a look at the following log entry:
<14>MAY 15 13:44:40 Windows2012 Statistics: 75877167 0 516927153 3719513871 17056456704 7456178176 2581681664 3234402816 1812390744 2832606444
You could guess from the log line that these are some kind of statistics but without knowing what was meant by the person who created your log events, you would be remiss to figure out what the numbers are. Now, the same log line in key value pair format:
<14>MAY 15 13:44:40 Windows2012 Statistics: Cpu.user=75877167 Cpu.nice=0 Cpu.system=516927153 Cpu.idle=3719513871 Memory.Total=17056456704 Memory.Active=7456178176 Disk.Read=2581681664 Disk.Write=3234402816 Net.Send=1812390744 Net.Receive=2832606444
You can see from the above line that not only is it much easier to determine what the values are but you can also take advantage of our field level extraction to start working with the values above, such that you can build powerful dashboards from your metrics – e.g. visualise your average CPU.idle values over time and compare them to your Disk.reads for example.
JSON events can also be parsed by Logentries such that you also take advantage of our field level extraction and search functions. However in our opinion JSON events can be a little bit more difficult to read than KVPs from a human readable perspective.
Logging, above all else, requires a well thought out strategy. It’s generally not a good idead to just start logging everything – as this not only greatly increases the amount of data you need to search and retain but it also dilutes the important information with non-essential data. When developing your logging strategy think about what is most important from your perspective and what value would you like to get from your logs.
Some common things that are useful to think about:
- Performance and Resources: logging of memory, CPU, disk, and network over time can show important spikes or point-in-time issues that can be tracked down to high load or low network bandwidth.
- Exceptions and Warnings: logging of errors and warnings for applications, systems, or network devices will help troubleshoot issues and allow greater insight into the historical going ons of your environment.
- User Actions: logging user access and actions within your environment across systems, applications, and devices allows you insight into what your users were doing when an issue occurred and how to resolve.
- Marketing: Campaigns: you can use our pixel tracking to track when emails are opened or ads are clicked on
- Business Metrics: metrics relating to the value of transactions can be useful to analyse how your business is performing on a daily basis and without the need to an expensive and complex BI tool