
Before you begin
- Labs create a Google Cloud project and resources for a fixed time
- Labs have a time limit and no pause feature. If you end the lab, you'll have to restart from the beginning.
- On the top left of your screen, click Start lab to begin
Create a Vertex AI dataset
/ 20
Create a tag template in Dataplex
/ 20
Add a tag to the Vertex AI dataset
/ 10
Tag other data assets using tag templates
/ 10
Create tag template for Protected Data
/ 20
Apply tag template for Protected Data
/ 20
For many organizations, finding the right datasets and other assets for machine learning workflows with Vertex AI can be tricky. There may already be datasets ready to use for model training, but they may not be easy to find or it is not clear how the data can or should be used.
This is where Dataplex Universal Catalog can help! You can think of Dataplex as a centralized platform that unifies data assets distributed across an organization, and supports data governance without the need to move or transfer data from its original location. Dataplex provides easy-to-use tools for discovering, cataloging, and managing data to power your analytics and machine learning workflows, while also leveraging the security of IAM in Google Cloud.
In this lab, you walk through a typical use case for using Dataplex to catalog and discover data assets that can be used to train Vertex AI models. Full details below.
To get you started, some data assets have been pre-created for you across two Google Cloud projects and include data assets in BigQuery and Cloud Storage, which you use to create a new Vertex AI dataset in the first task.
Project | Project ID | Assigned region | Available Data |
---|---|---|---|
1 | Cloud Storage bucket named |
||
2 | Three BigQuery datasets: damaged_car_ownership (containing a table for owners of the damaged cars); damaged_car_image_info (containing a table with additional image information, such as owner ID and location/timestamp of the image); and damaged_car_image_metadata (an object dataset with metadata derived from the image files in Cloud Storage such as storage path and updated date) |
First, you take on the role of the Data Engineer who is helping their organization get data into usable and accessible formats for machine learning workflows. In Project 1, you create a Vertex AI dataset from images in Cloud Storage, and then you create a custom tag template to tag that Vertex AI dataset (and other potentially useful data in BigQuery), so that it is easy for others in your organization to find using Dataplex.
Then, you transition to the role of the Data Scientist or Machine Learning Engineer who is looking for existing data assets that they can use to train new models. In Project 2, you use the tags applied by the Data Engineer to search for relevant data assets in Dataplex. Then, you create a new custom tag template for personally identifiable information (PII) and tag additional assets that can be useful for future modeling efforts.
User | Goal | Username | Primary project |
---|---|---|---|
1 - Data Engineer | Create and apply tags to help others find available data for a Vertex AI model to be named damaged_car_parts across multiple projects. | ||
2 - Data Scientist or ML Engineer | Use tags to search for Vertex AI assets to train the model to be named damaged_car_parts, and create and apply a new tag for PII to flag protected data. |
In this lab, you learn how to:
While not required, it is helpful to have some previous knowledge about how Dataplex and Vertex AI are commonly used within Google Cloud workflows. For an introduction to these tools before you begin this lab, you can complete the following labs:
Note: To begin this lab, follow the instructions below to log into Project 1:
Read these instructions. Labs are timed and you cannot pause them. The timer, which starts when you click Start Lab, shows how long Google Cloud resources are made available to you.
This hands-on lab lets you do the lab activities in a real cloud environment, not in a simulation or demo environment. It does so by giving you new, temporary credentials you use to sign in and access Google Cloud for the duration of the lab.
To complete this lab, you need:
Click the Start Lab button. If you need to pay for the lab, a dialog opens for you to select your payment method. On the left is the Lab Details pane with the following:
Click Open Google Cloud console (or right-click and select Open Link in Incognito Window if you are running the Chrome browser).
The lab spins up resources, and then opens another tab that shows the Sign in page.
Tip: Arrange the tabs in separate windows, side-by-side.
If necessary, copy the Username below and paste it into the Sign in dialog.
You can also find the Username in the Lab Details pane.
Click Next.
Copy the Password below and paste it into the Welcome dialog.
You can also find the Password in the Lab Details pane.
Click Next.
Click through the subsequent pages:
After a few moments, the Google Cloud console opens in this tab.
For this task, be sure that you have logged into Project 1 (
In a typical workflow to create a new Vertex AI dataset for image classification, you create a CSV input file, in which each row contains a URL image path to a training image and the associated label for that image.
In this lab, this CSV file has been pre-created for you in Cloud Storage as:
gs://
In this task, begin by updating the image paths in data.csv
with your bucket name, and then uploading the revised CSV file back to your Cloud Storage bucket. Then, use this file when creating a new Vertex AI dataset to connect the dataset to the training images in that Cloud Storage bucket.
Remember that you get help by expanding the hint boxes as you need them!
Start by downloading the data.csv
file locally to Cloud Shell, so you can update the image paths and upload the revised file back to Cloud Storage.
Click Activate Cloud Shell at the top of the Google Cloud console.
Create a local copy of data.csv
from your pre-created bucket, so that you can easily update the image paths with your bucket name:
If prompted, click Authorize.
The results display the current path (gs://damaged-car-parts-vertex-dataset/
), which you update in the next step.
data.csv
with your bucket name and subdirectory:The results display the updated paths that begin with gs://{{{project_0.project_id}}}-bucket/damaged-car-images/
.
data.csv
back to your Cloud Storage bucket:Though it is not a required step of this workflow, it is a good idea to check that the data.csv
file has been uploaded to the desired location and successfully updated.
Check that data.csv
was updated in your bucket using any method you already know, or expand the hint below for some helpful steps.
Full solution (Expand to see all of the steps!)
Next, create a new managed dataset in Vertex AI for single label image classification.
Navigate to Vertex AI in the Google Cloud console by clicking on the Navigation menu () > Vertex AI > Dashboard.
Click Enable All Recommended APIs.
In the Vertex AI navigation menu, under Data, click Datasets.
These first three steps got you to the appropriate location in your Google Cloud project.
If the option to select a region is null, you can assume that it will default to the assigned for this lab.
Expand the hints below for some helpful steps!
Option 1: Review the docs
Option 2: Adapt steps from a related lab
Full solution (Expand to see all of the steps!)
When you have successfully created the dataset, the page refreshes and provides new options for selecting an import method, which you use in the next section.
Now, use your updated data.csv
to import the image files of damaged car parts to the Vertex AI dataset.
For Select an import method, click Select import files from Cloud Storage.
In the Select import files from Cloud Storage section, click Browse, and navigate to data.csv
in your Cloud Storage bucket (gs://
Click Select.
Once you've selected an appropriate file, a green checkbox appears to the left of the file path.
Click Continue.
Click Check my progress to verify the objective.
For this task, stay logged into Project 1 (
Now that you have created the Vertex AI dataset, it is time to tag it, so that others can easily find it in Dataplex.
In this task, begin by creating a new tag template for Vertex AI model names. Then, you use that tag template to tag the Vertex AI dataset and associate it with the model name for damaged_car_parts.
Start by creating a custom tag template in Dataplex to tag assets with associated model names.
Navigate to Dataplex Universal Catalog in the Google Cloud console by clicking on the Navigation menu () > View all products > Analytics > Dataplex Universal Catalog.
In the Dataplex menu, under Manage metadata, click Catalog.
Click Create tag template (Deprecated), and then click Proceed.
Property | Value |
---|---|
Template Display Name | VertexAI_Data |
Template ID | Leave the default value. |
Location | |
Visibility | Public |
Property | Value |
---|---|
Field Display Name | Related Models |
Field ID | Leave the default value. |
Field description | Add a text description of your choice for this field. |
Type | String |
Click Done.
Click Create.
Click Check my progress to verify the objective.
Next, find the Vertex AI dataset that you created by using filters for systems and data types in Dataplex.
For Filters > Systems, enable the checkbox for Vertex AI.
For Filters > Data Types, enable the checkbox for Dataset.
Click on the damaged_car_parts dataset.
Now, add a tag to the Vertex AI dataset to associate it with the model name damaged_car_parts.
If this option is not visible, ensure that you are using the Data Catalog version of the search platform (see note in previous section).
For Choose the tag templates, select the template name VertexAI_Data, and click OK.
Under VertexAI_Data, for Related Models, type damaged_car_parts, and click Save.
Remain on this details page for the damaged_car_parts dataset.
Click Check my progress to verify the objective.
Adding an overview for a dataset asset in Dataplex can help others understand the intended purpose of the data and how it can or should be used.
Using the available options on the current page for the damaged_car_parts dataset, add an overview for the dataset that can be useful to others when they discover this dataset.
For example, the text can be something like: This Vertex AI dataset contains images of damaged car parts including bumpers, windshields, etc. Each image has been assigned a single label to categorize the images by these car part types.
Full solution (Expand to see all of the steps!)
For this task, stay logged into Project 1 (
In the previous task, you created a reusable tag template that can be used to tag multiple data assets with associated model names.
You can expand the hints below for some helpful steps!
Option 1: Review Task 2
Option 2: Adapt steps from a related lab
Full solution (Expand to see all of the steps!)
Click Check my progress to verify the objective.
For this task, begin by logging into Project 2 (
Expand the hint below for help with switching to a new project and user.
Full solution (Expand to see all of the steps!)
Now that the Data Engineer (you as Username 1!) has created a custom tag template and used it to tag data assets with associated model names, you can experience being the data scientist or machine learning engineer that benefits from these useful tags to search for relevant data assets and view their lineage.
In Dataplex, you can use tags to search for assets in multiple ways. Here are two useful ways. Both search options result in a list of data assets that have been tagged with VertexAI_Data.
Option 1: Search using text input
In the Dataplex menu, under Discover, click Search.
In Search input box (middle of page), type: tag:vertexai_data
Click Search.
Click on the asset name to see more details (such as the damaged_car_parts dataset).
or
Option 2: Search using checkboxes for tags
In the Dataplex menu, under Discover, click Search.
For Filters > Tags, enable the checkbox for VertexAI_Data.
Click on the asset name to see more details (such as the damaged_car_parts dataset).
Now that you have the steps to search for assets using tags, search for the three data assets that you previously tagged.
Expand to see the answer
Last, review the lineage of some of these assets to learn more about when and how they were created.
In the Dataplex search options, click on damaged_car_parts > Lineage.
Then, click on the asset name for damaged_car_parts on the Graph tab.
In addition to the date and time that the dataset was created, you can also see that it was created in a different project (
Now, return to the search page, and search for damaged_car_image_metadata using its full name.
Click on damaged_car_image_metadata > Entry list.
Then, click on the table name for bumper_images, and then click Lineage.
Last, click on the asset name for bumper_images on the Graph tab.
Similar to damaged_car_parts, you can see the created date, time, project ID, and region in which the asset was created.
If provided more time than this lab duration, Dataplex lineage can also populate more information, in which case you would see the full lineage of the bumper_images table in the object dataset: .jpg files from a Cloud Storage directory named bumper are used in simple data pipeline to create the resulting bumper_images metadata table.
For this task, stay logged into Project 2 (
While working with the data assets tagged by the Data Engineer, you notice that there are additional tags that should be applied. Specifically, the BigQuery dataset with ownership information for the damaged cars contains personally identifiable information (PII) that should be handled carefully.
Challenge yourself to create a simple custom tag template with a PII flag that you can apply to specific columns in the BigQuery table named owner_info in the damaged_car_ownership dataset.
You already created a tag template in Task 2. You can use those steps as a starting point and modify them to the specifications provided below.
Remember that you get help by expanding the hint boxes as you need them!
Begin by creating a new public tag template called Protected Data with one enumerated field called Protected Data Flag with YES and NO values. Be sure to create the template in the assigned region for Project 2 (
Option 1: Review the docs
Option 2: Adapt steps from a related lab
Full solution (Expand to see all of the steps!)
Note: If the progress check is not successful after implementing the full solution, check that you are logged into Project 2
Click Check my progress to verify the objective.
Next, apply the tag template for Protected Data to the BigQuery table named owner_info in the damaged_car_ownership dataset.
You decide not to tag the columns for owner_id, state, and age as PII (as they cannot be used to identify specific individuals), and instead focus on tagging the following columns:
Option 1: Review the docs
Option 2: Adapt steps from a related lab
Full solution (Expand to see all of the steps!)
Click Check my progress to verify the objective.
Last, use the PII tag to search for data flagged as containing PII.
Remember that you searched for assets using tags in Task 4. Repeat those steps to find data assets flagged as containing PII, or expand a hint below for some helpful steps.
Option 1: Ask Gemini
Option 2: Adapt steps from a related lab
Full solution (Expand to see all of the steps!)
In this lab, you learned how to tag Vertex AI assets using custom tag templates, search for Vertex AI assets using filters and tags, and view lineage of Vertex AI assets. Now you have what it takes to start cataloging and discovering models and datasets in your organization's projects using Dataplex!
...helps you make the most of Google Cloud technologies. Our classes include technical skills and best practices to help you get up to speed quickly and continue your learning journey. We offer fundamental to advanced level training, with on-demand, live, and virtual options to suit your busy schedule. Certifications help you validate and prove your skill and expertise in Google Cloud technologies.
Manual Last Updated June 11, 2025
Lab Last Tested June 11, 2025
Copyright 2025 Google LLC. All rights reserved. Google and the Google logo are trademarks of Google LLC. All other company and product names may be trademarks of the respective companies with which they are associated.
This content is not currently available
We will notify you via email when it becomes available
Great!
We will contact you via email if it becomes available
One lab at a time
Confirm to end all existing labs and start this one