Overview

The Cloud Retail service and the Retail API enables customers to build end-to-end personalized recommendation systems without requiring a high level of expertise in machine learning, recommendation system, or Google Cloud.

In this lab, you will configure Retail Search, perform searches using the Retail Search API using the console and the command line, and then modify search behaviour using Retail Search Boost/Bury controls.

This lab uses the Retail product catalog and event data you imported in a previous lab. This catalog data was created using Google Merchant Center data.

Objectives

In this lab, you will learn how to complete the following tasks:

• Configure a Retail Search Serving Config.
• Perform test searches using the Retail Search console.
• Configure and test a Retail Search Boost/Bury control.
• Explore the Retail Search API from the command line.

Setup and requirements

Qwiklabs setup

For each lab, you get a new Google Cloud project and set of resources for a fixed time at no cost.

1. Sign in to Qwiklabs using an incognito window.

2. Note the lab's access time (for example, 1:15:00), and make sure you can finish within that time.
   There is no pause feature. You can restart if needed, but you have to start at the beginning.

3. When ready, click Start lab.

4. Note your lab credentials (Username and Password). You will use them to sign in to the Google Cloud Console.

5. Click Open Google Console.

6. Click Use another account and copy/paste credentials for this lab into the prompts.
   If you use other credentials, you'll receive errors or incur charges.

7. Accept the terms and skip the recovery resource page.

Start Cloud Shell

While in Google Cloud you can operate Google Cloud remotely from your own machine. This lab uses both the Google Cloud Console and the Cloud Shell, a command line environment running in Google Cloud.

1. From the Cloud Console, click Activate Cloud Shell.

   

   Note: If you've never started Cloud Shell before, you are presented with an intermediate screen describing what it is. If that's the case, click Continue and you won't ever see it again.

   Here's what that one-time screen looks like:

   

   It should only take a few moments to provision and connect to Cloud Shell.

   Cloud Shell provides you with terminal access to a virtual machine hosted in the cloud. The virtual machine includes all the development tools that you'll need. It offers a persistent 5GB home directory and runs in Google Cloud, greatly enhancing network performance and authentication. Much, if not all, of your work in this lab can be done through the Cloud Console and Cloud Shell using only a browser.

   Once connected to Cloud Shell, you should see that you are already authenticated and that the project is already set to your project ID.

2. Run the following command in Cloud Shell to confirm that you are authenticated:

   gcloud auth list

   Output:

   Credentialed Accounts ACTIVE: * ACCOUNT: {{{user_0.username| Lab User Name}}}

3. To set the active account, run:

   gcloud config set account {{{user_0.username| Lab User Name}}} Note: The gcloud command-line tool is the powerful and unified command-line tool in Google Cloud. It comes preinstalled in Cloud Shell. Among its features, gcloud offers tab completion in the shell. For more information, refer to the gcloud CLI overview guide.

4. Run the following command to confirm that you are using the correct project for this lab:

   gcloud config list project

   Output:

   [core] project = {{{project_0.project_id | Project ID}}}

5. If the correct project is not listed, you can set it with this command:

   gcloud config set project {{{project_0.project_id| Project ID}}}

   Output:

   Updated property [core/project].

Task 1. Enable the Retail API

Before you can begin using the Retail Recommendations AI or Retail Search APIs, you must enable the Retail API.

1. On the Navigation menu (), click View All Products, then in the Artificial Intelligence click Search for Retail.

2. Click Turn On API.

3. Click Continue and accept the data terms by clicking Accept.

4. Click Continue.

5. Click on Turn On to turn on the Retail Search.

6. Click Get Started.

Task 2. Import product catalog and user event data

In this task, you will import product catalog data from BigQuery and user event data from Cloud Storage.

Import Retail products schema data from BigQuery

In this task, import product data into the catalog from a BigQuery table that uses the Retail products schema.

1. In the Search for Retail navigation menu, click Data to open the Retail Data management page.

2. Make sure the Catalog tab is selected and click Import.

3. Configure the import parameters as follows to import the product catalog:

• For Import type, select Product Catalog
• For Import Branch, select Branch 0
• For Source of data, select BigQuery

4. For Big Query table, click Browse.

5. Enter products in the search box and click Search.

6. Select the radio button for products (Dataset: retail) table.

7. Click Select.

Note: If you click the table name you will open the Data Catalog page and will need to return to the Retail products import page.

8. Click Import.

You need to wait for a pop-up message to appear with a message similar to the following:

Successfully scheduled import operation import-products-6583047802807380211. It may take up to 5 minutes to see your new long running operation in the Integration Activity panel.

When the import task is scheduled you will also see the details of a gcloud scheduler command displayed that you can use to schedule a regular data import task.

9. Click Cancel to close the import page and return to the Retail Data page to check the status of your catalog data import task.

10. Click X to close the popup that appeared to tell you that the import operation was successfully scheduled.

11. In the Search for Retail navigation menu, click Data and then click Activity Status to monitor the progress of the import task.

The import task will take a minute or two for the status of the import task in the Product catalog > Import and purge activity status section to change to Succeeded. A total of 1268 items will have been imported.

Import user event data from Cloud Storage

In this task, import user event data from a BigQuery table.

1. In the Search for Retail navigation menu, click Data to open the Retail Data management page.

2. Make sure the Events tab is selected and click Import.

3. Configure the import parameters as follows to import the product catalog:

• For Import type, select User Events
• For Source of data, select Google Cloud Storage

4. For Google Cloud Storage location, click the Browse button.

5. Navigate to the storage bucket called and select the file recent_retail_events.json.

6. Click the Filename to make sure it is selected.

7. Click Select.

8. Click Import.

You need to wait for a pop-up message to appear with a message similar to the following:

Successfully scheduled import operation import-products-6583047802807380211. It may take up to 5 minutes to see your new long running operation in the Integration Activity panel

When the import task is scheduled you will also see the details of a gcloud scheduler command displayed that you can use to schedule a regular event import task.

9. Wait for the import task to be scheduled with the gcloud scheduler command displayed.

10. Click Cancel to close the import page and return to the Retail Data page to check the status of your event data import task.

11. Click X to close the popup that appeared to tell you that the import operation was successfully scheduled.

12. In the Search for Retail navigation menu, click Data and then click Activity Status to monitor the progress of the import task.

The import task will take a minute or two for the status of the import task in the User events > Import activity status section to change to Succeeded. Approximately 32,000 items will have been imported and 5 items will have failed.

Task 3. Create a Retail Search serving config

You will now create a Retail API serving config for Retail Search.

1. In the Search for Retail navigation menu, click Serving configs.

2. Click Create Serving Config.

3. In the Select product stage, select the Search and Browse product type.

4. For Serving config name, type Test search config.

The Serving config ID field is automatically filled for you with a default value, test-search-config. If you change this default you will need to adjust variables in later steps yourself.

5. Click Continue.

6. Click Create to create the Test search config serving config.

You will now see the new serving config listed below the Recently Viewed (Default) default recommendation serving config.

Task 4. Evaluate a Retail Search serving config

You will now explore the search results and experiment with the search syntax.

1. In the Search for Retail navigation menu, click Evaluate.

2. Navigate to Search tab.

3. Select Test search config from the Select serving config dropdown.

4. Leave the Select Branch selection set to Branch 0 (Default).

5. For Search Query, type Android and click Search Preview.

You will see search results containing items from the catalog that match a search for Android. Note that the search results include wearable items from the Apparel category.

Task 5. Configure and test a Retail Search Boost/Bury control

You will now create a Boost/Bury control that will modify search results.

1. In the Search for Retail navigation menu, click Controls.

2. Click Create Control.

3. For Control name, type Test control ID.

4. For Product selection select Search option.

5. For Control Type, select Boost/bury controls and click Continue.

6. Type android for Partial match query terms and Exact match query terms. Click Continue.

7. In Boost/bury product field select Advanced Editor and copy in the following query string in rule section:

(categories:ANY("Drinkware", "Office"))

8. Drag the Boost value slider across until the Boost/bury value is set to 1.

9. Click Continue.

10. Click Apply to serving configs and select Test search config.

If you used a different name for your Retail Search serving config in a previous step then select that serving config name.

11. Click Continue.

12. Click Submit.

13. In the Search for Retail navigation menu, click Evaluate and navigate to Search tab.

14. Select Test search config from the Select serving config dropdown.

15. Leave the Select Branch selection set to Branch 0 (Default).

16. For Search Query, type Android and click Search Preview.

You will see search results containing items from the catalog that match a search for Android but the list of results now lists items from the Drinkware and Accessories categories ahead of the items in the Apparel categories.

Task 6. Explore the Retail API from the command line

You will now use curl and other command line utilities to make calls to the Retail API to explore how to make requests, get results and then filter and refine the results.

If you have already created the IAM service account you will see an error when you try to recreate the account in the next step. You can ignore the error but complete the rest of the steps as you need to generate the authentication token.

Create an IAM service account to authenticate requests

1. Create environment variables to store the Project ID and Project Number:

export PROJECT_ID=$(gcloud config list --format 'value(core.project)') export PROJECT_NUMBER=$(gcloud projects list --filter="project_id:${PROJECT_ID}" --format='value(project_number)')

2. Create an IAM service account for controlled access to the Retail API:

export SA_NAME="retail-service-account" gcloud iam service-accounts create $SA_NAME --display-name $SA_NAME

3. Bind the service account to the Retail Editor IAM role:

gcloud projects add-iam-policy-binding ${PROJECT_ID} \ --member="serviceAccount:$SA_NAME@${PROJECT_ID}.iam.gserviceaccount.com" \ --role="roles/retail.editor"

Allow the lab user account to use impersonation with the new service account

Creating a role binding on the service account for the lab user with the Service Account Token Creator role allows the lab user to use service account impersonation to safely generate limited duration authentication tokens for the service account. These tokens can then be used to interactively test access to APIs and services.

1. Create a role binding on the Retail API service account for your user account to permit impersonation:

export USER_ACCOUNT=$(gcloud config list --format 'value(core.account)') gcloud iam service-accounts add-iam-policy-binding $SA_NAME@$PROJECT_ID.iam.gserviceaccount.com --member "user:$USER_ACCOUNT" --role roles/iam.serviceAccountTokenCreator

2. Generate a temporary access token for the Retail API:

export ACCESS_TOKEN=$(gcloud auth print-access-token --impersonate-service-account $SA_NAME@$PROJECT_ID.iam.gserviceaccount.com ) Note: This command may fail as it can take up to 10 minutes for the Service Account Token Creator role to propagate. Retry this command after 1 minute if it fails, and retry until it succeeds. You will also see a warning informing you that the command is using impersonation. This is expected.

Perform search queries using the Retail API

When making a search request to the Retail API you need to include some metadata as JSON formatted payload data. To make it easier to modify this JSON object later you will use environment variables to store the parameters.

The VISITOR_ID identifies the user that the results should be tailored for.

The SERVING_CONFIG_ID identifies the specific Retail API Search serving config that should be used to generate the search results. The serving config is referred to as the placements resource in the API and contains the ID for the serving config you created earlier.

The QUERY_STRING variable contains the query string that is passed to the search API and the PAGE_SIZE variable is used to control the pagination of the results.

The full resource name for the placement resource, that includes the serving config ID, must be included in the request.

1. Create environment variables to store the parameters for the call to the Retail API search method:

# Retail API parameters export VISITOR_ID="GA1.3.1260529204.1622654859" export SERVING_CONFIG_ID="test-search-config" export QUERY_STRING="Android" export PAGE_SIZE=5 export PLACEMENT_NAME="projects/${PROJECT_NUMBER}/locations/global/catalogs/default_catalog/placements/${SERVING_CONFIG_ID}"

2. Store the search query payload JSON in an environment variable:

DATA="{ placement: '${PLACEMENT_NAME}', visitor_id: '${VISITOR_ID}', query: '${QUERY_STRING}', page_size: ${PAGE_SIZE} }"

3. Store the Retail Search API URL for search queries in an environment variable:

URL="https://retail.googleapis.com/v2alpha/projects/${PROJECT_NUMBER}/locations/global/catalogs/default_catalog/placements/${SERVING_CONFIG_ID}:search?access_token=${ACCESS_TOKEN}"

Note that the URL includes the desired serving config and again has the access token appended as an inline parameter called access_token.

4. Call the API storing the results in a variable for reuse:

RESPONSE=$(curl -H "Content-Type: application/json; charset=utf-8" -X POST -d "${DATA}" $URL ) echo $RESPONSE | jq

This will generate a list of Product IDs returned by the search model configured for the serving config you specified using the visitor ID, event data, and other parameters to refine the search result.

5. Parse the initial search response using JQ to retrieve the nextPagetoken:

NEXT_PAGE_TOKEN=$(echo $RESPONSE | jq -r ".nextPageToken") echo $NEXT_PAGE_TOKEN

Each page of results includes the nextPageToken property to allow controlled delivery of search results.

6. Construct a new JSON data payload to return the next page of results:

DATA_NEXT="{ placement: '${PLACEMENT_ID}', visitor_id: '${VISITOR_ID}', query: '${QUERY_STRING}', page_size: ${PAGE_SIZE}, page_token:'${NEXT_PAGE_TOKEN}' }"

7. Call the Retail Search API using the payload that includes the nextPageToken property:

RESPONSE_NEXT=$(curl -H "Content-Type: application/json; charset=utf-8" -X POST -d "${DATA_NEXT}" $URL ) echo $RESPONSE_NEXT | jq

8. Compare the products returned in the first query to the second query:

echo "First page of search result product IDs" echo $RESPONSE | jq -r ".results[].id" echo "Next page of search result product IDs" echo $RESPONSE_NEXT | jq -r ".results[].id"

You can see from the output that the results are different for the request with the nextPageToken defined.

Congratulations

Congratulations, you've successfully created a Retail Search Serving Config and used the Retail API to work with Retail Search.