Proxy Network Load Balancer logging and monitoring

This page shows you how to configure and use Cloud Logging and Cloud Monitoring for proxy Network Load Balancers.

Monitoring resources

The following table specifies the resource names for the load balancers.

Logging for Proxy Network Load Balancers

Logs provide useful information for troubleshooting and monitoring load balancers. Logs are aggregated for each connection and give you insight into how each connection is routed to the serving backends.

There are no additional charges for using logs. However, based on how you import logs, standard pricing for Cloud Logging, BigQuery, or Pub/Sub applies. Also, enabling logs does not affect the performance of the load balancer.

Logs sampling and collection

The connections that leave and enter load balancer backend virtual machine (VM) instances are sampled. These sampled connections are then processed to generate logs. You control the fraction of the connections that are emitted as log entries according to the logConfig.sampleRate parameter. When logConfig.sampleRate is 1.0 (100%), this means that logs are generated for all of the connections and written to Cloud Logging.

Enable logging on a new backend service

    gcloud compute backend-services create BACKEND_SERVICE \
        --region=REGION \
        --enable-logging \
        --logging-sample-rate=SAMPLE_RATE
    

    gcloud compute backend-services create BACKEND_SERVICE \
        --global \
        --enable-logging \
        --logging-sample-rate=SAMPLE_RATE
    

    {
    "name": "BACKEND_SERVICE",
    "loadBalancingScheme": "INTERNAL_MANAGED",
    "logConfig": {
       "enable": true,
       "sampleRate": SAMPLE_RATE
      }
    }
    

    {
    "name": "BACKEND_SERVICE",
    "loadBalancingScheme": "EXTERNAL_MANAGED",
    "logConfig": {
       "enable": true,
       "sampleRate": SAMPLE_RATE
      }
    }
    

    {
    "name": "BACKEND_SERVICE",
    "loadBalancingScheme": "EXTERNAL_MANAGED",
    "logConfig": {
       "enable": true,
       "sampleRate": SAMPLE_RATE
      }
    }
    

    {
    "name": "BACKEND_SERVICE",
    "loadBalancingScheme": "EXTERNAL",
    "logConfig": {
       "enable": true,
       "sampleRate": SAMPLE_RATE
      }
    }
    

    {
    "name": "BACKEND_SERVICE",
    "loadBalancingScheme": "INTERNAL_MANAGED",
    "logConfig": {
       "enable": true,
       "sampleRate": SAMPLE_RATE
      }
    }
    

Enable logging on an existing backend service

    gcloud compute backend-services update BACKEND_SERVICE \
        --region=REGION \
        --enable-logging \
        --logging-sample-rate=SAMPLE_RATE
    

    gcloud compute backend-services update BACKEND_SERVICE \
        --global \
        --enable-logging \
        --logging-sample-rate=SAMPLE_RATE
    

      PATCH https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/regions/REGION/backendServices/BACKEND_SERVICE
     

    {
    "name": "BACKEND_SERVICE",
    "loadBalancingScheme": "INTERNAL_MANAGED",
    "logConfig": {
       "enable": true,
       "sampleRate": SAMPLE_RATE
      }
    }
    

    {
    "name": "BACKEND_SERVICE",
    "loadBalancingScheme": "EXTERNAL_MANAGED",
    "logConfig": {
       "enable": true,
       "sampleRate": SAMPLE_RATE
      }
    }
    

      PATCH https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/global/backendServices/BACKEND_SERVICE
    {
    "name": "BACKEND_SERVICE",
    "loadBalancingScheme": "EXTERNAL_MANAGED",
    "logConfig": {
       "enable": true,
       "sampleRate": SAMPLE_RATE
      }
    }
    

      PATCH https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/global/backendServices/BACKEND_SERVICE
    {
    "name": "BACKEND_SERVICE",
    "loadBalancingScheme": "EXTERNAL",
    "logConfig": {
       "enable": true,
       "sampleRate": SAMPLE_RATE
      }
    }
    

      PATCH https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/global/backendServices/BACKEND_SERVICE
    {
    "name": "BACKEND_SERVICE",
    "loadBalancingScheme": "INTERNAL_MANAGED",
    "logConfig": {
       "enable": true,
       "sampleRate": SAMPLE_RATE
      }
    }
    

Disable logging on an existing backend service

gcloud compute backend-services update BACKEND_SERVICE \
   --region=REGION \
   --no-enable-logging

gcloud compute backend-services update BACKEND_SERVICE \
   --global \
   --no-enable-logging

 PATCH https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/regions/REGION/backendServices/BACKEND_SERVICE
  {
  "logConfig": {
    "enable": false
   }
  }
 

 PATCH https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/global/backendServices/BACKEND_SERVICE
  {
  "logConfig": {
    "enable": false
   }
  }
 

View logs

When logs are ingested into Cloud Logging and not excluded through a Log Router sink, you can read logs by using the Cloud Logging API and the Google Cloud CLI.

To view all the logs, complete the following steps.

View logs for a specific backend service

To view the logs for a specific backend service, complete the following steps.

View logs for a backend instance group

To view the logs for a specific backend instance group, complete the following steps.

What is logged

Log entries contain information useful for monitoring and debugging your traffic. Log records contain required fields, which are the default fields of every log record.

Log fields

Log records contain required fields, which are the default fields of every log record.

Some log fields contain more than one piece of data in a given field—these log fields are in a multi-field format. For example, the connection field is of the IpConnection format, which contains the source and destination IP address and port, plus the protocol, in a single field. These multi-field log fields are described in the following record format table.

The following table lists all the required log fields for the resource l4_proxy_rule.

IpConnection field format

proxyStatus error field

The proxyStatus field contains a string that specifies why the load balancer returned an error. There are two parts in the proxyStatus field, proxyStatus error and proxyStatus details. This section describes the strings that are supported in the proxyStatus error field.

The proxyStatus error field is applicable to the following load balancers:

• Global external proxy Network Load Balancer
• Regional external proxy Network Load Balancer
• Cross-region internal proxy Network Load Balancer
• Regional internal proxy Network Load Balancer

proxyStatus details field

The proxyStatus field contains a string that specifies why the load balancer returned an error. There are two parts in the proxyStatus field, proxyStatus error and proxyStatus details. The proxyStatus details field is optional and is shown only when additional information is available. This section describes the strings that are supported in the proxyStatus details field.

The proxyStatus details field is applicable to the following load balancers:

• Global external proxy Network Load Balancer
• Regional external proxy Network Load Balancer
• Regional internal proxy Network Load Balancer
• Cross-region internal proxy Network Load Balancer

Failed TLS connection log entries

When the TLS connection between the client and the load balancer fails before any backend is selected, log entries record the errors. You can configure the backend services with different log sample rates. When a TLS connection fails, the failed TLS connection log sample rate is the highest sample rate for any backend service. For example, if you have configured two backend services with logging sample rate as 0.3 and 0.5, the failed TLS connection log sample rate is 0.5.

You can identify failed TLS connections by checking for these log entry details:

• proxyStatus error type is tls_alert_received, tls_certificate_error, tls_protocol_error, or connection_terminated.
• There is no backend information.

The following sample shows a failed TLS log entry with the proxyStatus error field:

   json_payload:    {
   @type: "type.googleapis.com/google.cloud.loadbalancing.type.LoadBalancerLogEntry"
   proxyStatus: "error="tls_alert_received"; details="server_to_client: handshake_failure""
   log_name: "projects/529254013417/logs/mockservice.googleapis.com%20name"
   }
   http_request {
    latency {
      nanos: 12412000
    }
    protocol: "HTTP/1.0"
    remote_ip: "127.0.0.2"
   }
  resource {
    type: "mock_internal_http_lb_rule"
    labels {
      backend_name: ""
      backend_scope: ""
      backend_scope_type: "UNKNOWN"
      backend_target_name: ""
      backend_target_type: "UNKNOWN"
      backend_type: "UNKNOWN"
      forwarding_rule_name: "l7-ilb-https-forwarding-rule-dev"
      matched_url_path_rule: "UNKNOWN"
      network_name: "lb-network"
      region: "REGION"
      target_proxy_name: "l7-ilb-https-proxy-dev"
      url_map_name: ""
    }
  }
  timestamp: "2023-08-15T16:49:30.850785Z"
  

Resource labels

The following table lists the resource labels for resource type l4_proxy_rule.

Monitoring

The proxy Network Load Balancers export monitoring data to Cloud Monitoring.

Monitoring metrics can be used to do the following:

• Evaluate a load balancer's configuration, usage, and performance.
• Troubleshoot problems.
• Improve resource utilization and user experience.

In addition to the predefined dashboards in Monitoring, you can create custom dashboards, set up alerts, and query metrics by using the Cloud Monitoring API.

View Monitoring dashboards

1. In the Google Cloud console, go to the Monitoring page.

   Go to Monitoring

2. If Resources appears in the navigation pane, select Resources, and then select Google Cloud Load Balancers. Otherwise, select Dashboards, and then select the dashboard named Google Cloud Load Balancers.

3. Click the name of your load balancer.

In the left pane, you can see various details for this load balancer. In the right pane, you can see timeseries graphs. To see specific breakdowns, click Breakdowns.

Metric reporting frequency and retention

Metrics for the load balancers are exported to Monitoring in one-minute granularity batches. Monitoring data is retained for six (6) weeks. Metrics are based on sampled traffic (sampling rate is dynamic and cannot be adjusted).

The dashboard provides data analysis in default intervals of 1H (one hour), 6H (six hours), 1D (one day), 1W (one week), and 6W (six weeks). You can manually request analysis in any interval from six weeks to one minute.

Metrics for classic proxy Network Load Balancers

The following metrics for classic proxy Network Load Balancers are reported into Monitoring.

Metrics for other load balancers

The following metrics for regional internal proxy Network Load Balancers, regional external proxy Network Load Balancers, cross-region internal proxy Network Load Balancer, and global external proxy Network Load Balancers are reported into Monitoring.

Filtering dimensions for metrics

Metrics are aggregated for each load balancer. Metrics can be further broken down by the following dimensions.

What's next

• To learn how SSL policies work, see the SSL policies overview.
• To learn how external proxy Network Load Balancers work, see the External proxy Network Load Balancer overview.
• To learn how internal proxy Network Load Balancers work, see the Internal proxy Network Load Balancer overview.