Vertex AI for BigQuery users

Use this page to understand the differences between Vertex AI and BigQuery and learn how you can integrate Vertex AI with your existing BigQuery workflows. Vertex AI and BigQuery work together to meet your machine learning and MLOps use cases.

To learn more about model training differences between Vertex AI and BigQuery, see Choose a training method.

Differences between Vertex AI and BigQuery

This section covers the Vertex AI, BigQuery, and BigQuery ML services.

Vertex AI: An end-to-end AI/ML platform

Vertex AI is an AI/ML platform for model development and governance. Common use cases include the following:

  • Machine learning tasks, such as forecasting, prediction, recommendation, and anomaly detection

  • Generative AI tasks, such as:

    • Text generation, classification, summarization, and extraction
    • Code generation and completion
    • Image generation
    • Embedding generation

You can use BigQuery to prepare training data for Vertex AI models, which you can make available as features in Vertex AI Feature Store.

You can train models in Vertex AI in three ways:

  • AutoML: Train models on image, tabular, and video datasets without writing code.
  • Custom Training: Run custom training code catered to your specific use case.
  • Ray on Vertex AI: Use Ray to scale AI and Python applications like machine learning.

You can also import a model trained on another platform like BigQuery ML or XGBoost.

You can register custom-trained models to the Vertex AI Model Registry. You can also import models trained outside of Vertex AI and register them to Vertex AI Model Registry. You don't need to register AutoML models; they are registered automatically at creation time.

From the registry, you can manage model versions, deploy to endpoints for online predictions, perform model evaluations, monitor deployments with Vertex AI Model Monitoring, and use Vertex Explainable AI.

Available languages:

BigQuery: A serverless, multicloud enterprise data warehouse

BigQuery is a fully managed enterprise data warehouse that helps you manage and analyze your data with built-in features like machine learning, geospatial analysis, and business intelligence. BigQuery tables can be queried by SQL, and data scientists who primarily use SQL can run large queries with only a few lines of code.

You can also use BigQuery as a data store that you reference when building tabular and custom models in Vertex AI. To learn more about using BigQuery as a data store, see Overview of BigQuery storage.

Available languages:

To learn more, see BigQuery SQL dialects.

BigQuery ML: Machine learning directly in BigQuery

BigQuery ML lets you develop and invoke models in BigQuery. With BigQuery ML, you can use SQL to train ML models directly in BigQuery without needing to move data or worry about the underlying training infrastructure. You can create batch predictions for BigQuery ML models to gain insights from your BigQuery data.

You can also access Vertex AI models by using BigQuery ML. You can create a BigQuery ML remote model over a Vertex AI built-in model like Gemini, or over a Vertex AI custom model. You interact with the remote model using SQL in BigQuery, just like any other BigQuery ML model, but all training and inference for the remote model is processed in Vertex AI.

Available language:

To learn more about the advantages of using BigQuery ML, see Introduction to AI and ML in BigQuery.

Benefits of managing BigQuery ML models in Vertex AI

You can register your BigQuery ML models to the Model Registry in order to manage the models in Vertex AI. Managing BigQuery ML models in Vertex AI provides two main benefits:

  • Online model serving: BigQuery ML only supports batch predictions for your models. To get online predictions, you can train your models in BigQuery ML and deploy them to Vertex AI endpoints through Vertex AI Model Registry.

  • MLOps capabilities: Models are most beneficial when they are kept up to date through continuous training. Vertex AI offers MLOps tools that automate the monitoring and retraining of models to maintain the accuracy of predictions over time. With Vertex AI Pipelines, you can use BigQuery operators to plug any BigQuery jobs (including BigQuery ML) into an ML pipeline. With Vertex AI Model Monitoring, you can monitor your BigQuery ML predictions over time.

An image of Google Cloud products and where they fit in an MLOps workflow

To learn how to register your BigQuery ML models to the Model Registry, see Manage BigQuery ML models with Vertex AI.

What do you want to do? Resource
Use BigQuery ML to analyze images and text using Gemini on Vertex AI Analyzing movie posters in BigQuery with Gemini 1.5 Flash
Use BigQuery ML to generate text on BigQuery tables or unstructured data with foundation models on Vertex AI Generate text using BigQuery ML and foundation models in Vertex AI
Generate vector embeddings with BigQuery ML over text and images Call a multimodal embedding endpoint in Vertex AI from BigQuery ML to generate embeddings for semantic search
Use two Vertex AI Tabular Workflows pipelines to train an AutoML model using different configurations. Tabular Workflow: AutoML Tabular Pipeline
Use the Vertex AI SDK for Python to train an AutoML model for tabular regression and get batch predictions from the model. Vertex AI SDK for Python: AutoML training tabular regression model for batch prediction using BigQuery
Train and evaluate a propensity model in BigQuery ML to predict user retention on a mobile game. Churn prediction for game developers using Google Analytics 4 and BigQuery ML
Use BigQuery ML to perform pricing optimization on CDM pricing data. Analysis of pricing optimization on CDM pricing data

What's next