How to Use 'varnishlog' (with examples)

Varnishlog is a command-line tool used to display logs from Varnish, a popular caching HTTP reverse proxy. Varnishlog provides detailed information about HTTP requests and responses handled by Varnish, enabling administrators to diagnose issues, monitor traffic, and optimize web performance. By using specific options and queries, you can filter and customize the logs displayed to focus on particular aspects of your traffic or address specific problems.

Use case 1: Display logs in real time

Code:

varnishlog

Motivation:

Monitoring Varnish logs in real time is invaluable for administrators and developers who need to keep an eye on live traffic to their websites or applications. This use case is crucial for diagnosing problems as they occur, understanding real-time performance issues, and reacting promptly to unexpected traffic patterns or errors. By observing the logs as requests are made and responses are served, administrators can gain immediate insights into the system’s behavior.

Explanation:

• varnishlog: This command, without additional arguments, initiates a real-time display of all logs generated by Varnish. It captures and displays every HTTP request and response Varnish processes, allowing for immediate feedback.

Example Output:

*   << Request  >> 32768
-   Begin          req 32767 rxreq
-   ReqURL         /index.html
-   ReqMethod      GET
-   ReqHeader      Host: www.example.com
-   End

This output shows a GET request for the URL /index.html, detailing the request and closing with an “End” indicator.

Use case 2: Only display requests to a specific domain

Code:

varnishlog -q 'ReqHeader eq "Host: example.com"'

Motivation:

Filtering logs to display requests only to a specific domain is essential when managing multiple domains on a single Varnish server. It helps administrators focus on the traffic for a particular website, making it easier to troubleshoot issues or analyze performance metrics for that domain without the noise of unrelated requests.

Explanation:

• -q: This option introduces a query for filtering the logs.
• 'ReqHeader eq "Host: example.com"': This query string filters the logs to include only those requests where the ‘Host’ header matches ’example.com’. The ReqHeader field is checked for equality against the specified domain.

Example Output:

*   << Request  >> 32782
-   ReqHeader      Host: example.com
-   ReqURL         /about
-   ReqMethod      GET

The output shows only requests directed to example.com, with details about the method and URL.

Use case 3: Only display POST requests

Code:

varnishlog -q 'ReqMethod eq "POST"'

Motivation:

POST requests typically involve data being sent to the server, often as part of form submissions or API interactions. By filtering for POST requests only, developers and administrators can monitor data submission activities closely, ensuring that forms and APIs are functioning correctly and that no unexpected behavior is occurring within these interactions.

Explanation:

• -q: This option sets up a query to filter logs.
• 'ReqMethod eq "POST"': This query string specifies that only log entries with a method equal to ‘POST’ should be included in the output. The ReqMethod field is checked for this equality.

Example Output:

*   << Request  >> 32796
-   ReqMethod      POST
-   ReqURL         /submit-form
-   ReqHeader      Host: example.com
-   End

The output specifically lists POST requests, detailing the URL and method used.

Use case 4: Only display requests to a specific path

Code:

varnishlog -q 'ReqURL eq "/path"'

Motivation:

When diagnosing or optimizing specific parts of a website, focusing on requests to a specific path can be extremely beneficial. This might be necessary for monitoring the performance or functionality of a critical section of the site, such as a checkout page or a frequently accessed resource, providing insights specifically for that part.

Explanation:

• -q: Engages query mode for log filtering.
• 'ReqURL eq "/path"': Specifies the filter criteria, where only requests to a URL exactly matching ‘/path’ are considered. The ReqURL field is examined for this match.

Example Output:

*   << Request  >> 33201
-   ReqURL         /path
-   ReqHeader      Host: example.com
-   ReqMethod      GET

The logs show only the requests to the exact path ‘/path’.

Use case 5: Only display requests to paths matching a regular expression

Code:

varnishlog -q 'ReqURL ~ "regex"'

Motivation:

Utilizing regular expressions to filter requests allows for highly flexible and sophisticated querying. This is useful for analyzing a specific pattern of URLs, such as all resource files ending with ‘.css’ or URLs containing a certain keyword, thus enabling a broad view of related traffic or data trends across variable paths.

Explanation:

• -q: Initiates a filter query in the varnishlog command.
• 'ReqURL ~ "regex"': This query makes use of a regular expression to match URLs. The tilde ~ in the query denotes regex matching, allowing complex patterns to be specified to tailor the selection of log entries.

Example Output:

*   << Request  >> 33510
-   ReqURL         /blog/2021-nov
-   ReqMethod      GET
-   ReqHeader      Host: example.com

This output captures all the requests whose URLs match the specified regular expression pattern, offering comprehensive insights into related traffic.

Conclusion:

Varnishlog is a versatile tool for monitoring and debugging Varnish Cache by allowing administrators to filter and view logs in various useful ways. By leveraging its querying capabilities, users can focus on particular aspects of web traffic, making it easier to perform targeted diagnostics, ensure proper functionality, and analyze website performance across different dimensions.