Logentries allows User different ways of searching their data, either via Regex, String, KeyValue or Keyword search. In this document we will go over the basics of searching operations in Logentries.
Our powerful search language
Logentries Query Langauge (
LEQL) allows you to quickly construct queries that can extract the hidden data within your logs. Whether you are trying to look for the occurrence of a single term or trying to graph your web sites response time over the Last 24 hours LEQL has you covered.
LEQL follows a SQL-style syntax and constructing a query is simple and intuitive. Each search must begin with a
where() statement, you can then insert your query inside of the where statement. If
you want to dive into searching logs and don't want to worry about LEQL then our LEQL Query Bar can help you which you can read more about below.
The LEQL Query Bar has three modes. The first mode is simple mode that will allow you to build queries by selecting the desired functions and keys with your mouse.
The second is the advanced mode which will allow you to enter the full query using the keyboard. Experienced users will likely find this mode faster, but the syntax is strict and you must remember the different analytic functions available.
You can press the down key to see and load sample queries when the text box is empty.
You can quickly switch between the simple and the advanced mode by clicking the mode switcher to the left of the query bar. If you find yourself in the advanced mode with an invalid query, the system will prevent you from returning to the simple mode. To return to the simple mode delete the query or fix the syntax errors.
The third mode is Visual Search, which overlays your raw log data with interactive charts that allow you to quickly filter your log data.
Logentries supports both logical and comparison operactors which allows you to create more complex searches. The below guide will introduce both sets of operators available to use while constructing a query
Logentries supports the following logical operators to create comprehensive search criteria.
Please note that when constructing a Search Query all operators should be typed in UPPERCASE.
expr1 AND expr2
Returns log events that match both criteria
expr1 OR expr2
Returns log events that match one or both criteria
expr1 NOT expr2
Returns log events that match expr1 but not expr2
Returns log events that match the search value – matches numeric and text values
Returns log events that do not match the search value – matches numeric and text values
Returns log events with field values higher than the search value
Returns log events with field values higher than or equal to the search value
Returns log events with field values lower than the search value
Returns log events with field values lower than or equal the search value
NOTE: Numerical values must be formatted as an integer, floating-point value, or in scientific notation to be properly recognized by Logentries. Units are not calculated as part of the comparison. E.g. searching for a value<100bytes would not return a result with value=200bits
Keyword search will work on all logs regardless of their format. Keyword searches are case sensitive by default and will match a full string until it is delimited by a non-letter character. For example, given the log events below.
Apr 13 20:01:01 hostname run-parts(/etc/cron.hourly): starting 0anacron Apr 13 20:01:01 hostname run-parts(/etc/cron.hourly): finished 0anacron
Logentries will match the events by searching for “etc” or “run” because the text is delimited by whitespace and non-letter characters. Logentries would not match “hour” but will match “hourly”.
Keyword search can be combined with logical operators, e.g. “starting AND finished” would return both log events.
Note: When you list a series of keywords Logentries automatically assumes an AND between each keyword. If you want to match an exact string place “Double Quotes” around the search.
Regular Expressions can greatly enhance the power of your keyword searching. Regular expressions must be wrapped with two forward slashes (“/”). The two most common cases for regex search in Logentries are below.
By default, regex search will return partial matches. The search below will match “complete,” “completely,” and “completed”
Using a case-insensitive keyword search means that the search below will match Error, ERROR, error, and any other form of capitalization
Regular expressions use special characters to enable searching for more advanced patterns. These characters are
}.If you need to use special characters as ordinary characters, you will need to escape them with a backward slash (
To learn more about our advanced regular expression search capabilities please read the Regular Expression Log Search Documentation.
Logentries will automatically parse log events that are in a Key Value Pair (KVP) or JSON format for easy use of advanced analytics. The KVP and JSON parsing doc details the specific formats the system will parse. If your logs are not in a standard KVP or JSON format you can utilize the regular expression field extraction to gain access to the same advanced analytics. To understand which KVP and JSON formats
Given the log events below
Searches can be easily written to return important log events
Search for all log events with a response time over 25 to return the first two log events, such as:
You can then add the logical operator OR to include events from containerID 14 to return all three log events:
where(response_time>25 OR containerID=14)
If your logs do not contain any Key Value Pairs (KVP), you can designate a Key Value Pair relationship for a given string using Regular Expression Field Extraction. This will give you access to all of the advanced search, analytic, and visualization capabilities available for KVP and JSON log format.
With our powerful LEQL functions you are able to produce queries that will easily visualise your data without any preprocessing required.
Log search also supports returning a count of matched search results. Append
calculate(COUNT) to your search query or press the
calculate to get the number of search results. An example can be seen below.
You can use the sum function to total the values of your name value pairs. If you had a KVP for sale_value and wanted to know the total sales for a specified time period it would be returned with the following query:
The average function works the same as sum, but it computes the mean of the values matching the search criteria. For instance, to get the average value of your sales, you might invoke a search like:
The Count Unique function returns the number of unique values for a given key. It takes one parameter: the name of the key. For example, if you have the KVP userID in your log file and want to find the number of unique users:
The Min function will return the minimum value of the key for each time period. For example the query below will return the shortest response time for each time period:
The Max function will return the maximum value of the key for each time period. For example the query below will return the longest response time for each time period:
You can perform search functions based on grouping identical datasets. In the example below the logs contain the KVP “status” which represents the HTTP status code for a request from a web server.
where(status) groupby(status) calculate(count)
By default LEQL will automatically sort your result set based a descending order if no
sort keyword is used.
You can increase the number of groups returned by your GroupBy query by using the
Using limit you can specify to return up to 1000 groups in your query. This is done by adding
limit(N) at the end of your query where N is a number between 1 and 1000
where(status) groupby(status) calculate(count) sort(desc) limit(350)
You have the ability to sort the results returned by a group by in ascending or descending order, or alphabetically by the names of the keys that are turned.
This will sort descending based on the count returned by the function
where(status>=300) groupby(status) calculate(count) sort(desc)
This will sort a list of usernames alphabetically:
groupby(username) calculate(count) sort(asc#key)
The keywords to sort descending are
descending. The keywords to sort ascending are
Note If you use
sort in your query, you must place
limit after the
Logentries will automatically calculate 10 equal time intervals when performing a time based query (i..e. when you're not grouping by a key in your logs).
You can manually set the number of time intervals by using the timeslice function. The valid input for timeslice is a number between 1 and 200 (inclusive), or you can specify a unit of time.
The query below used against a 1 hour search period would return the count of 500 errors per minute.
where(status=500) calculate(count) timeslice(60)
This will divide the results into 30 second intervals
where(status=500) calculate(count) timeslice(30s)
This will divide the results into 5 minute intervals
where(status=500) calculate(count) timeslice(5m)
This will divide the results into hourly intervals
where(status=500) calculate(count) timeslice(1h)
The Percentile option now allows you to exclude outliers from your search functions. In simple mode you can quickly select either a 95th or 99th percentile search function based on a key value pair which has a numerical number. Users using advanced mode can specify their own percentile vale by using
pctl(80):key_value_pair in their calculate function.
The Bytes option lets you calculate the size of your logs in byte form. This is useful for users who wish to verify the size of the logs that they have sent to their account. A simple query that would calculate the size of the given log would be
The Standard Deviation option lets you calculate the standard deviation of a given series values. This is useful when trying to establish what values would be considered within normal variance to a given mean e.g response times.
You can also use the keyword
sd as a shortcut e.g
Logentries allows searches to be saved. The
save button allows you to save your current query, and the
saved button allows you to access saved searches.