Creating custom tags in Google Cloud Logs

In Google Cloud Logging, it can be difficult to easily determine where your logs are coming from, but we have a solution that will make organizing your logs (by service, host, location, etc) incredibly simple!

If we leverage the ability to customize FluentD configurations for a given log source that is deployed, we can include custom fields, or "tags", to help sort through logs after they've been sent to Google Cloud Logging.

In the following example we will be using our MySQL log source:

  1. Read Customize FluentD Config to allow for customizations
  2. Scroll down to the <match> tag and, within that tag, find the <record> tag.
  3. Within the <record> tag, include a custom field, or fields, that resemble the ones in the image below. These custom fields will be included in the jsonPayload of the log entries.
1422

Custom fields within the <record> tag for a MySQL source

  1. Within Google Cloud Logging, using an advanced filter, filter for the logs related to the particular source technology that you've added custom fields to. An example is below:
2274

Advanced filter example filtering using custom fields from jsonPayload

  1. Next, click on Save Search at the top
  2. Ensure that the Custom Fields portion of the form includes the two custom fields that were created. This will include these in the summary lines, highlighted in a light-blue color.
1042

Custom Fields to be displayed in the summary lines of the log table

  1. Congratulations! You've successfully created a saved filter that highlights specific logs from your environment based on custom fields that were added to the FluentD configuration used to collect those logs!
1458

Example of how the logs look based on running the saved search. Includes the custom fields in the summary lines.