gcloud metastore services update

gcloud metastore services update - update a Dataproc Metastore service

gcloud metastore services update (SERVICE : --location=LOCATION) [--async] [--autoscaling-enabled] [--data-catalog-sync] [--deletion-protection] [--endpoint-protocol=ENDPOINT_PROTOCOL] [--port=PORT] [--update-labels=[KEY=VALUE,…]] [--clear-labels     | --remove-labels=[KEY,…]] [--instance-size=INSTANCE_SIZE     | --scaling-factor=SCALING_FACTOR     | --tier=TIER     | --max-scaling-factor=MAX_SCALING_FACTOR --min-scaling-factor=MIN_SCALING_FACTOR] [--kerberos-principal=KERBEROS_PRINCIPAL --keytab=KEYTAB --krb5-config=KRB5_CONFIG] [--maintenance-window-day=MAINTENANCE_WINDOW_DAY --maintenance-window-hour=MAINTENANCE_WINDOW_HOUR] [--scheduled-backup-configs-from-file=PATH_TO_FILE     | --enable-scheduled-backup --scheduled-backup-cron=SCHEDULED_BACKUP_CRON --scheduled-backup-location=SCHEDULED_BACKUP_LOCATION] [--update-auxiliary-versions-from-file=UPDATE_AUXILIARY_VERSIONS_FROM_FILE     | --add-auxiliary-versions=[ADD_AUXILIARY_VERSIONS,…] --clear-auxiliary-versions] [--update-hive-metastore-configs-from-file=PATH_TO_FILE     | --update-hive-metastore-configs=[KEY=VALUE,…] --clear-hive-metastore-configs     | --remove-hive-metastore-configs=[REMOVE_HIVE_METASTORE_CONFIGS,…]] [GCLOUD_WIDE_FLAG …]

If run asynchronously with --async, exits after printing one operation name that can be used to poll the status of the update via:

gcloud metastore operations describe

Service resource - Arguments and flags that specify the Dataproc Metastore service you want to update. The arguments in this group can be used to specify the attributes of this resource. (NOTE) Some attributes are not given arguments in this group but can be set in other ways.

To set the project attribute:

• provide the argument service on the command line with a fully specified name;
• provide the argument --project on the command line;
• set the property core/project.

This must be specified.

SERVICE

ID of the service or fully qualified identifier for the service.

To set the service attribute:

• provide the argument service on the command line.

This positional argument must be specified if any of the other arguments in this group are specified.

--location=LOCATION

The location of the Dataproc Metastore service.

If not specified, will use default metastore/location. To set the location attribute:

• provide the argument service on the command line with a fully specified name;
• provide the argument --location on the command line;
• set the property metastore/location.

--async

Return immediately, without waiting for the operation in progress to complete.

--autoscaling-enabled

A boolean flag to determine whether Dataproc Metastore autoscaling should be enabled, false if unspecified.

The default minimum and maximum scaling factors are 0.1 and 6.0, respectively.

The minimum and maximum scaling factors can be specified using --min-scaling-factor and --max-scaling-factor.

--data-catalog-sync

Boolean flag to determine whether or not Dataproc Metastore metadata sync to Data Catalog is enabled, false if unspecified. Mutually exclusive with flag --encryption-kms-key. Cannot be updated if the service uses customer-managed encryption keys.

--deletion-protection

Flag that enables delete protection on Dataproc Metastore instance to prevent accidental deletions of the instance. Use --deletion-protection to enable and --no-deletion-protection to disable.

--endpoint-protocol=ENDPOINT_PROTOCOL

The protocol to use for the metastore service endpoint. ENDPOINT_PROTOCOL must be one of:

grpc

The modernized GRPC protocol.

thrift

The legacy Apache THRIFT protocol.

--port=PORT

The TCP port on which the Metastore service will listen.

--update-labels=[KEY=VALUE,…]

List of label KEY=VALUE pairs to update. If a label exists, its value is modified. Otherwise, a new label is created.

Keys must start with a lowercase character and contain only hyphens (-), underscores (_), lowercase characters, and numbers. Values must contain only hyphens (-), underscores (_), lowercase characters, and numbers.

At most one of these can be specified:

--clear-labels

Remove all labels. If --update-labels is also specified then --clear-labels is applied first.

For example, to remove all labels:

gcloud metastore services update --clear-labels

To remove all existing labels and create two new labels, foo and baz:

gcloud metastore services update --clear-labels --update-labels foo=bar,baz=qux

--remove-labels=[KEY,…]

List of label keys to remove. If a label does not exist it is silently ignored. If --update-labels is also specified then --update-labels is applied first.

The instance size of the Dataproc Metastore instance. At most one of these can be specified:

--instance-size=INSTANCE_SIZE

The instance size of the service. INSTANCE_SIZE must be one of:

extra-large

Extra large instance size, maps to a scaling factor of 6.0.

extra-small

Extra small instance size, maps to a scaling factor of 0.1.

large

Large instance size, maps to a scaling factor of 3.0.

medium

Medium instance size, maps to a scaling factor of 1.0.

small

Small instance size, maps to a scaling factor of 0.5.

--scaling-factor=SCALING_FACTOR

The scaling factor of the service. Can be one of [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0].

--tier=TIER

The tier of the service. TIER must be one of:

developer

The developer tier provides limited scalability and no fault tolerance. Good for low-cost proof-of-concept.

enterprise

The enterprise tier provides multi-zone high availability, and sufficient scalability for enterprise-level Dataproc Metastore workloads.

The autoscaling configuration of the Dataproc Metastore instance.

--max-scaling-factor=MAX_SCALING_FACTOR

The maximum scaling factor allowed by the autoscaler. Can be one of [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0]. The default value is 6.0.

--min-scaling-factor=MIN_SCALING_FACTOR

The minimum scaling factor allowed by the autoscaler. Can be one of [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0]. The default value is 0.1.

Information used to configure the Hive metastore service as a service principal in a Kerberos realm.

--kerberos-principal=KERBEROS_PRINCIPAL

A Kerberos principal that exists in the KDC to authenticate as. A typical principal is of the form "primary/instance@REALM", but there is no exact format.

--keytab=KEYTAB

A Kerberos keytab file that can be used to authenticate a service principal with a Kerberos Key Distribution Center. This is a Secret Manager secret version, and can be fully-qualified URL, or relative name in the form projects/{project_id}/secrets/{secret_id}/versions/{version_id}.

--krb5-config=KRB5_CONFIG

A Cloud Storage URI that specifies the path to a krb5.conf file. It is of the form gs://{bucket_name}/path/krb5.conf, although the file does not need to be named krb5.conf explicitly.

The one hour maintenance window that specifies when Dataproc Metastore may perform system maintenance operation to the service, in UTC time.

--maintenance-window-day=MAINTENANCE_WINDOW_DAY

The day of week when the window starts, e.g., sun. MAINTENANCE_WINDOW_DAY must be one of: fri, mon, sat, sun, thu, tue, wed.

This flag argument must be specified if any of the other arguments in this group are specified.

--maintenance-window-hour=MAINTENANCE_WINDOW_HOUR

The hour of day (0-23) when the window starts.

This flag argument must be specified if any of the other arguments in this group are specified.

Scheduled backup configurations to enable a scheduled backup of the metastore service. At most one of these can be specified:

--scheduled-backup-configs-from-file=PATH_TO_FILE

Path to a JSON file containing a mapping of scheduled backup configurations key-value pairs to apply to the scheduled backup.

For example:

backup_configs.json
    {
      "enabled": true,
      "cron_schedule": "0 * * * *",
      "backup_location": "gs://example-bucket",
      "time_zone": "UTC"
    }

Use a full or relative path to a local file containing the value of scheduled_backup_configs.

Scheduled backup configurations to enable a scheduled backup of the metastore service.

--enable-scheduled-backup

Enable or disable scheduled backups for the Metastore service.

--scheduled-backup-cron=SCHEDULED_BACKUP_CRON

Specify the frequency of a scheduled backup of the metastore service.

--scheduled-backup-location=SCHEDULED_BACKUP_LOCATION

Set the Cloud Storage URI to store the scheduled backups of the metastore service.

Auxiliary versions configuration for the Dataproc Metastore service. When specified, a secondary Hive metastore service is created along with the primary service. At most one of these can be specified:

--update-auxiliary-versions-from-file=UPDATE_AUXILIARY_VERSIONS_FROM_FILE

Path to a YAML file containing the auxiliary versions configuration for Dataproc Metastore instance. The file should contain a unique auxiliary service name and auxiliary version that is lower than the primary service version. The primary version's Hive metastore configs are applied to the auxiliary version. Additional Hive metastore configs can be specified for the auxiliary version using "config_overrides". If a specified config property has already been overridden in the primary Hive metastore version's configs, the auxiliary version's override takes precedence. The contents of the file should be structured as follows:

YAML:

  - name: aux-service1
    version: x.y.z
    config_overrides:
      key1: value1
      key2: value2
    …
  - name: aux-service2
    version: x.y.z
    config_overrides:
      key1: value1
      key2: value2
    …
  …

--add-auxiliary-versions=[ADD_AUXILIARY_VERSIONS,…]

Comma-separated list of auxiliary Hive metastore versions to deploy. Auxiliary Hive metastore versions must be less than the primary Hive metastore service's version.

--clear-auxiliary-versions

Clears the existing auxiliary services attached to the primary Hive metastore services. If --add-auxiliary-versions is also specified, --clear-auxiliary-versions is applied first.

Configuration properties specific to running Hive metastore software as the metastore service. At most one of these can be specified:

--update-hive-metastore-configs-from-file=PATH_TO_FILE

Path to a XML file containing a mapping of Hive metastore configuration key-value pairs to apply to the Hive metastore.

For example:

hive-site.xml
    <configuration>
      <property>
        <name>hive.metastore.warehouse.dir</name>
        <value>${test.warehouse.dir}</value>
        <description></description>
      </property>
    </configuration>

Use a full or relative path to a local file containing the value of update_hive_metastore_configs.

Modify the Hive metastore configuration properties by passing key-value pairs in through the flags.

--update-hive-metastore-configs=[KEY=VALUE,…]

Comma-separated list of Hive metastore configurations. Each configuration has the form "NAME=VALUE".

At most one of these can be specified:

--clear-hive-metastore-configs

clear existing Hive metastore configurations. If --update-hive-metastore-configs is also specified, then --clear-hive-metastore-configs is applied first.

--remove-hive-metastore-configs=[REMOVE_HIVE_METASTORE_CONFIGS,…]

Comma-separated list of configuration keys to remove with the form "KEY1, KEY2". If a label does not exist it is silently ignored. If --update-hive-metastore-configs is also specified, then --remove-hive-metastore-configs is applied first.

Run $ gcloud help for details.