Google Cloud Logging is a centralized repository where logs and events from your applications and services are automatically collected, stored, and made available for analysis. As discussed in previous chapters, when you access the logs, you’ll see a vast collection of logs from every service within the project by default. Given the massive scale of logs that could be generated, especially in large projects, filtering becomes an essential tool.
Log filters are powerful tools that help you focus on the specific logs you’re interested in. Think of them as search criteria that can be customized to your needs.
The Google Cloud Logging query language (LQL) is the language you’d use to define these filters. LQL provides a versatile syntax that can be used to write expressions that match specific log entries. The documentation provides more details, and you can dive deeper into the LQL by following the link.
An essential concept when dealing with filters is labels. Labels are like tags, key-value pairs that can be associated with various resources to categorize or mark them. These labels are immensely beneficial when you want to create specific filters. For instance, Cloud Run, the serverless compute platform by Google Cloud, automatically labels each log with the associated service’s name.
So, if you’re trying to focus solely on logs from the skill service, you can utilize this automatic labeling. The filter to accomplish this would look something like:
export FILTER=resource.labels.service_name=skill-service
Set your project back to the project your application is running in:
gcloud config set project $MONITORED_PROJECT_ID
You can then use the filter with the gcloud logging read command to view the logs for the skill service. The limit is set to 10 to limit the number of log entries returned:
gcloud logging read $FILTER –limit=10
This will show just the last 10 logs for the skill service.
Then narrow it down further by matching the log entry that contains the text loaded and tags:
export FILTER=’resource.labels.service_name=skill-service AND jsonPayload.message=~”loaded.*tags”‘
The logs returned are structured logs, so you will see the full log messages as YAML. The actual log messages in this case are in jsonPayload.message. You can use the –format option to extract just the log messages:
gcloud logging read $FILTER –limit=10 –format=’value(jsonPayload.message)’
For example, with the filter above, you will see the following if the service had started twice:
loaded 63653 tags
loaded 63653 tags
Google Cloud Logging is a comprehensive system that automatically consolidates, stores, and allows analysis of logs from all your applications and services. However, given the potentially massive scale of logs produced, particularly in larger projects, filtering these logs becomes paramount.
Creating filters through Google Cloud LQL empowers you to concentrate on specific logs of interest. These filters act as tailored search parameters, taking into account labels—key-value pairs tagged to different resources. This becomes particularly helpful in scenarios like focusing solely on logs from the skill service in a Cloud Run setup.
