Checkpoints
Assign roles to service account
/ 25
Set up Service Accounts
/ 25
Set up a new ONTAP storage environment
/ 25
Set up a second ONTAP storage environment
/ 25
Leveraging NetApp Cloud Volume ONTAP Remote Caching Capabilities for EDA Workloads
- GSP1066
- Overview
- Setup and requirements
- Task 1. Set up service accounts
- Task 2. Set up two new ONTAP storage environments
- Task 3. Prepping the origin volume
- Task 4. Populating the Origin Volume with Data
- Create the FlexCache target volume on the destination Cloud Volumes ONTAP
- Task 5. Create the FlexCache Target Volume on the Destination Cloud Volumes ONTAP
- Task 6. Mount and use the FlexCache Target Volume
- Congratulations!
- End your lab
- Next Steps / Learn More
This lab was developed with our partner, NetApp. Your personal information may be shared with NetApp, the lab sponsor, if you have opted-in to receive product updates, announcements, and offers in your Account Profile.
GSP1066
Overview
In this lab, you will learn how to take advantage of NetApp’s caching capability, also known as FlexCache. With NetApp FlexCache, only actively-read data is cached, either locally or at geographically dispersed remote sites, rather than entire files or volumes. The capability is built into any ONTAP flavor, and used for cloud bursting, improving productivity, and collaboration across multiple locations and increasing data throughput for read-intensive workloads such as media rendering, EDA, seismic analysis and financial simulations.
This lab is derived from the BlueXP and ONTAP documentation published by NetApp:
https://docs.netapp.com/us-en/occm/task_getting_started_gcp.html
https://docs.netapp.com/us-en/ontap/flexcache/index.html
Objectives
In this lab, you will configure and access FlexCached volumes using Cloud Volume ONTAP for Google Cloud. You learn how to perform the following tasks:
- Create a cluster peer relationship between two ONTAP systems
- Create a Storage Virtual Machine (SVM) peer relationship for FlexCache
- Create a FlexCache volume
- Access the FlexCache volume from a Linux VM using NFS
Prerequisites
This is an introductory lab and no prior knowledge is required. It is recommended to be familiar with using NetApp BlueXP and knowing how to perform foundational tasks in Google Cloud.
Setup and requirements
Before you click the Start Lab button
Read these instructions. Labs are timed and you cannot pause them. The timer, which starts when you click Start Lab, shows how long Google Cloud resources will be made available to you.
This hands-on lab lets you do the lab activities yourself in a real cloud environment, not in a simulation or demo environment. It does so by giving you new, temporary credentials that you use to sign in and access Google Cloud for the duration of the lab.
To complete this lab, you need:
- Access to a standard internet browser (Chrome browser recommended).
- Time to complete the lab---remember, once you start, you cannot pause a lab.
How to start your lab and sign in to the Google Cloud console
-
Click the Start Lab button. If you need to pay for the lab, a pop-up opens for you to select your payment method. On the left is the Lab Details panel with the following:
- The Open Google Cloud console button
- Time remaining
- The temporary credentials that you must use for this lab
- Other information, if needed, to step through this lab
-
Click Open Google Cloud console (or right-click and select Open Link in Incognito Window if you are running the Chrome browser).
The lab spins up resources, and then opens another tab that shows the Sign in page.
Tip: Arrange the tabs in separate windows, side-by-side.
Note: If you see the Choose an account dialog, click Use Another Account. -
If necessary, copy the Username below and paste it into the Sign in dialog.
{{{user_0.username | "Username"}}} You can also find the Username in the Lab Details panel.
-
Click Next.
-
Copy the Password below and paste it into the Welcome dialog.
{{{user_0.password | "Password"}}} You can also find the Password in the Lab Details panel.
-
Click Next.
Important: You must use the credentials the lab provides you. Do not use your Google Cloud account credentials. Note: Using your own Google Cloud account for this lab may incur extra charges. -
Click through the subsequent pages:
- Accept the terms and conditions.
- Do not add recovery options or two-factor authentication (because this is a temporary account).
- Do not sign up for free trials.
After a few moments, the Google Cloud console opens in this tab.
Activate Cloud Shell
Cloud Shell is a virtual machine that is loaded with development tools. It offers a persistent 5GB home directory and runs on the Google Cloud. Cloud Shell provides command-line access to your Google Cloud resources.
- Click Activate Cloud Shell at the top of the Google Cloud console.
When you are connected, you are already authenticated, and the project is set to your Project_ID,
gcloud
is the command-line tool for Google Cloud. It comes pre-installed on Cloud Shell and supports tab-completion.
- (Optional) You can list the active account name with this command:
- Click Authorize.
Output:
- (Optional) You can list the project ID with this command:
Output:
gcloud
, in Google Cloud, refer to the gcloud CLI overview guide.
Network requirements for Cloud Volumes ONTAP in Google Cloud.
Task 1. Set up service accounts
- To start, in your Cloud Shell window, run the following commands to set some environment variables:
- Next, run the following commands to assign roles to the pre-created netapp-cloud-manager service account:
Click Check my progress to verify the objective.
- Lastly, create the netapp-cvo service account and assign it the relevant roles:
Click Check my progress to verify the objective.
Task 2. Set up two new ONTAP storage environments
In this section you will log into NetApp BlueXP, NetApp’s data landscape management platform, and create two new Cloud Volumes ONTAP storage environments on Google Cloud infrastructure. One Cloud Volumes ONTAP instance will host the source volume (FlexCache origin) the other will host the destination volume (FlexCache target).
-
From the Google Cloud Console Navigation Menu, go to Compute Engine > VM instances. You should see the
cloudmanager
VM already deployed for you. -
Click the External IP to connect to the VM in a new window. Use
http://<IP Address>/
.Note: If Chrome gives you a privacy error, continue by clicking Advanced, then Proceed. -
Once you are on the BlueXP SaaS setup page, you can set up your NetApp Account.
First click Sign Up, then enter in your lab credentials. Fill out the rest of the fields to authenticate yourself with NetApp’s Cloud Central and click Sign Up to log in to BlueXP.
-
Next, you will be prompted to create an Connector name and a Account name. For these, just use
user
. Click Let's Start and then click Lets go to BlueXP. You should now be redirected to the BlueXP SaaS page. If prompted click continue. -
Before you add a working environment, navigate to the gear icon on the top-right of the page then click Connector Settings > General.
Note: If Connector Settings is not clickable wait a couple of minutes and try to refresh the page. -
For Automatic Cloud Volumes ONTAP update during deployment, click the box and then uncheck the Automatically update Cloud Volumes ONTAP box to turn the updates off. Click Save. Your configuration settings should resemble the following:
This page may take a few minutes to load after the values are changed.
-
Click BlueXP on the top left to navigate to the home page. Click Add Working Environment to create the first Cloud Volumes ONTAP instance that will serve as your source.
If you happen to run into any issues adding a Working Environment (i.e server fault), this can be due to BlueXP automatically updating. If this happens, wait a few minutes, refresh the page, and try again. -
Add New Working Environment: select Google Cloud Platform.
-
Choose Type: Cloud Volumes ONTAP Single Node by click on Add New next to it.
-
On the Details and Credentials page, specify a Working Environment Name, optionally add labels, and then specify a new Password for the default admin account. Click Continue. Make sure to record the password, since you would need it later on in the lab.
Note: The service account netapp-cvo
(with the Storage Admin role) that you created in the previous section(in Task 1) is automatically associated with your instance. This Service Account allows Cloud Volumes ONTAP to automatically move cold data to Google Cloud Storage.
- On the Services page, disable the Data Sense & Compliance, and Backup to Cloud and click Continue.
- Location & Connectivity:
-
For the Google Cloud Region and Google Cloud Zone, use
-
For the VPC select
default
. -
For the Subnet, and Firewall Policy, leave as default.
-
Select the checkbox to confirm network connectivity to Google Cloud storage for data tiering
-
Click Continue.
- Cloud Volumes ONTAP Charging Methods & NSS Account: Select Freemium (Up to 500 GiB) and click on Continue.
-
Preconfigured Packages: click Change Configuration.
-
On the Licensing section:
-
First click Change version and select
ONTAP-9.7P5
version from the dropdown. -
Click Apply.
-
Next, from the Machine Type dropdown list select
n1-highmem-4
. -
Click Continue.
- Underlying Storage Resources: Choose
Standard
and for the Google Cloud Disk Size select100 GB
. Click Continue.
-
WORM (write once, read many): keep the settings as default and click Continue.
-
Create Volume: Skip as you will later create the necessary storage volumes.
-
Review & Approve: Review and confirm your selections:
- Review details about the configuration.
- Click More information to review details about support and the Google Cloud resources that BlueXP will purchase.
- Select both checkboxes.
- Click Go.
Click Check my progress to verify the objective.
- While the first instance is being deployed,Repeat steps 7-19 to create the second Cloud Volumes ONTAP instance, that will serve as the destination and host the FlexCache volume you will create later on.
Click Check my progress to verify the objective.
Task 3. Prepping the origin volume
In this section you will first create a Cloud Volumes ONTAP volume and then populate it with some data. This volume will serve as the source volume, containing the source data, that later in this lab will be cached to a FlexCache volume located on the destination Cloud Volumes ONTAP system.
-
Back on BlueXP’s Storage Canvas, double-click on the source Cloud Volumes ONTAP working environment (or click it once and then use the Enter Working Environment button that will appear at the bottom right), to enter the Volumes page.
-
On the Volumes page, Add Volume, on the right hand side, and select New Volume.
-
Volume Details, Protection & Tags:
-
Name your volume data1
-
Set size to 10 (size set in GB)
-
Leave Snapshot Policy with default
-
Leave Storage VM (SVM)
-
Click Next
- Volumes Protocol:
-
Select NFS as protocol
-
Verify Access Control is set to Custom export policy
-
Click Next
- Select Disk Type:
-
Keep Disk Type as PD-Standard
-
Click Next
- Usage Profile, & Tiering Policy:
-
Verify the Storage Efficiency is marked as enabled
-
Click Next
- Review:
-
Review and click Add to create the volume. Once the Operational Approval pop-up appears, click the Approve button.
You have just created your source Cloud Volumes ONTAP volume to store your applications data.
Task 4. Populating the Origin Volume with Data
-
Back on the Volumes page, locate the data1 volume and click on its triple-bar.
-
From the list of operations revealed, click on Mount Command, and then click Copy to copy the command to your clipboard.
-
From the Google Cloud Console navigation menu, navigate to Compute Engine in the COMPUTE section and choose VM instances from the menu.
-
On the VM instances page, locate the preconfigured
linuxvm
and connect to it by clicking on SSH. -
Once logged into linuxvm, use the following command to install the nfs-common package, which installs the NFS client needed to mount the volume.
sudo apt install nfs-common -y -
On your linux VM, create a local directory under /mnt named data1 that would serve as a mount point by executing the command:
sudo mkdir /mnt/data1 -
Next, paste the Mount Command, copied in step 2, and replace <dest_dir> with the /mnt/data1 directory, and add the option
-o nfsvers=3
to use NFSv3. Then, click on enter to mount the Cloud Volumes ONTAP volume using NFS.sudo mount -o nfsvers=3 <IP>:/data1 /mnt/data1 Note: In the version of ONTAP used in this lab (9.7P5) while regular ONTAP volumes can be accessed with any NFS version, FlexCache volumes only support NFSv3. So, for a consistent view of files’ ownership, later on in the lab, we will also use NFSv3 to access the source volume. -
To verify that you have mounted the cloud volume correctly, check the output of the
df -h
command for a filesystem mounted on/mnt/data1
: -
Now that the source volume is mounted to your linuxvm, it’s time to put some data in it. Use the following command to create multiple files, in your volume:
sudo dd if=/dev/urandom bs=1024 count=10240 | sudo split -a 4 -b 1024k - /mnt/data1/file. -
Use the following command to verify that your volume was populated:
ls -l /mnt/data1 The output should look like this:
Create the FlexCache target volume on the destination Cloud Volumes ONTAP
Now that your source volume was created and is populated with data, you can proceed to the next section and see how the data is easily cached, on-demand, on a different ONTAP environment that can be located anywhere (on-premises, cloud, and different region).
Task 5. Create the FlexCache Target Volume on the Destination Cloud Volumes ONTAP
In this section you will learn how to enable and use NetApp FlexCache technology built into any ONTAP-based system, including Cloud Volumes ONTAP. Before creating the FlexCache target volume, you will establish a peering relationship that enables secure communication between the Cloud Volumes ONTAP systems. To perform these tasks you will use Cloud Volumes ONTAP’s CLI, accessible via SSH.
Establish Peering Relationships
To establish the peering relationship, you will first locate the necessary IP addresses on each system - the Cluster Management IP that will be used to SSH into Cloud Volumes ONTAP, and the Intercluster IP used for the secure inter-cluster communication channel.
Locating the IP Addresses
-
Back on BlueXP, enter the Volumes page on the source Cloud Volumes ONTAP Working Environment. From the Storage Canvas, double-click on the working environment (or click it once and then use the Enter Working Environment button that will appear at the bottom right).
-
Next, click on the triple-bar menu at the top right hand side and select Information.
-
Under the Access section, locate and record the Cluster Management IP address (will be used to SSH the system) and the intercluster IP address (will be used for secure communication between the systems).
-
Now, go back to BlueXP’s Canvas, and double-click on the destination Cloud Volumes ONTAP Working Environment, to enter its Volumes page.
-
Repeat steps 2-3 on the destination system, to locate and record its cluster management and the intercluster IP addresses.
Creating the Peering Relationship
-
Next, you will need two SSH-in-browser sessions to the linuxvm. Use the following to open an in-browser SSH session:
- From the Google Cloud Console navigation menu, navigate to Compute Engine in the COMPUTE section and choose VM instances from the menu.
- On the VM instances page, locate the preconfigured linux VM and connect to it by clicking on SSH.
Note: before continuing to the next step, make sure you have two SSH-in-browser sessions. -
On one SSH-in-browser session, open an SSH session to the source Cloud Volumes ONTAP using its Cluster Management IP. Use the following command to establish an SSH session with the source:
ssh admin@<source-cluster-manamgemet-ip> (replace
<source-cluster-management-ip>
with the IP you recorded earlier)Use the password you specified when creating the source system to log in.
-
Switch to the other SSH-in-browser session, and open an SSH session to the destination Cloud Volumes ONTAP using its Cluster Management IP. Use the following command to establish an SSH session with the destination:
ssh admin@<destination-cluster-manamgemet-ip> (replace
<destination-cluster-management-ip>
with the IP you recorded earlier) -
Now that you’re connected to both Cloud Volumes ONTAPs’ CLI (aka Clustershell), you’ll initiate a request to create an authenticated peering relationship. Run the following CLI command, on the source Cloud Volumes ONTAP:
cluster peer create -peer-addrs <destination-intercluster-ip> (replace
<destination-intercluster-ip>
with the IP you recorded earlier)Enter a passphrase when prompted, and record it. You will use it in the next step
-
Next, you’ll initiate the same request from the destination system. Run the following CLI command, on the destination Cloud Volumes ONTAP:
cluster peer create -peer-addrs <source-intercluster-ip> (replace
<source-intercluster-ip>
with the IP you recorded earlier)When prompted for a passphrase, enter the passphrase you used in the previous step.
-
Verify that the peering relationship was properly created. Run the following command on either the source or destination system:
cluster peer show The output should look like the following (with your system’s name) where Availability shows ‘Available’ and Authentication shows ‘ok’:
-
The cluster peering was formed and what’s left is to create a Storage Virtual Machine (SVM) peering relationship. Run the following command from the ONTAP CLI session of your destination system, the request a peering relationship:
vserver peer create -vserver svm_<destination-cvo-name> -peer-vserver svm_<source-cvo-name> -peer-cluster <source-cvo-name> -applications flexcache (replace
<destination-cvo-name>
and<source-cvo-name>
with the names you provided to the destination and source systems, respectively).Note: An SVM (aka *vserver*) is an ONTAP entity that contains volumes and IP addresses and you can create multiple SVMs on a single Cloud Volumes ONTAP system for various use cases such as secure multi-tenancy, workload separation and management delegation. -
Back to on the ONTAP CLI session of your source system, run the following command to accept the request for the SVM peering relationship:
vserver peer accept -vserver svm_<source-cvo-name> -peer-vserver svm_<destination-cvo-name> (replace
<source-cvo-name>
and<destination-cvo-name>
with the names you provided to the destination and source systems, respectively). -
Lastly, let’s verify that the peering relationship was properly created. Run the following command on either the source or destination system:
vserver peer show The output should look like the following (with your SVM/vserver and system name) where Peer State shows ’peered’ and Peering Applications shows ‘flexcache’:
Create FlexCache Target Volume On Destination
-
Back on the ONTAP CLI session of the destination system, run the following command to create the FlexCache volume:
volume flexcache create -volume cache -vserver svm_<destination-cvo-name> -origin-volume data1 -origin-vserver svm_<source-cvo-name> -junction-path /cache -aggr-list aggr1 (replace
<destination-cvo-name>
and<source-cvo-name>
with the names you provided to the destination and source systems, respectively). -
Next, verify that the FlexCache volume was properly created. On the ONTAP CLI session of the destination system, run the following command:
volume flexcache show The output should look like the following (with your SVM/vserver and system names):
-
To make your FlexCache volume accessible via NFS, create an export policy with an export rule and associate it with the FlexCache volume. On the ONTAP CLI session of the destination system, run the following commands:
-
Create the export policy:
vserver export-policy create -vserver svm_<destination-cvo-name> -policyname flexcache_policy (Replace
<destination-cvo-name>
with the name you provided to the destination). -
Create a rule within the export policy to allow access to the Linux machine:
vserver export-policy rule create -vserver svm_<destination-cvo-name> -policyname flexcache_policy -ruleindex 1 -protocol nfs -clientmatch 10.128.0.0/20 -rorule any -rwrule any -superuser any (Replace
<destination-cvo-name>
with the name you provided to the destination). -
Assign the Export Policy to the FlexCache volume.
volume modify -vserver svm_<destination-cvo-name> -volume cache -policy flexcache_policy (Replace
<destination-cvo-name>
with the name you provided to the destination).
-
-
To verify that your FlexCache volume was properly assigned with the export policy, run the following command from the ONTAP CLI session of the destination system:
volume show cache -fields policy The output should look like the following (with your SVM/vserver name):
Hurray! Your FlexCache target volume is provisioned and ready to be mounted!
Task 6. Mount and use the FlexCache Target Volume
In this section you will see how the remote caching technology works by accessing and performing read and write operations on the FlexCache target volume, which corresponds with the source volume. In addition, you will see how the source volume is automatically updated when changes or writes are made to the FlexCache target volume.
-
Back on BlueXP, enter the Volumes page on the destination Cloud Volumes ONTAP Working Environment. From the Storage Canvas, double-click on the working environment (or click it once and then use the Enter Working Environment button that will appear at the bottom right).
-
Next, click on the triple-bar menu at the top right hand side and select Information.
-
Under the Access section, locate and record the Data (nfs, cifs) IP address
-
Back on the ONTAP CLI session of the destination system, run the command exit to get back to the CLI of the
linuxvm
. -
On the linuxvm run the following command to create a directory to serve as an NFS mount point:
sudo mkdir /mnt/cache -
Next, mount the FlexCache volume using the following command:
sudo mount -t nfs -o nfsvers=3 <data-ip>:/cache /mnt/cache (Replace
<date-ip>
with the IP address you recorded on step 3). -
Now, let’s confirm that we can see all the file content of the source volume through the FlexCache volume’s mount point. Use the following command to list the content of your target volume:
ls -l /mnt/cache The output should look like this: (The content shown should be identical to the source)
Note: at the moment, only files and directory metadata are cached in the target volume. Once you will start reading files, ONTAP will automatically cache each and every block that was read in the target volume. On subsequent reads operations that performance would dramatically improve since the data is already cached. -
Let’s continue and see what happens when data is added or changed in the FlexCache target volume. Use the following command to create some new files on the target volume you’ve just mounted:
sudo dd if=/dev/urandom bs=1024 count=10240 | sudo split -a 4 -b 1024k - /mnt/cache/newfile -
Now, use the following command to verify the new files are shown on your target volume:
ls -l /mnt/cache The output should look like this:
-
Lastly, use the following command to verify that the new files, you created in step 8, were written to the source volume:
ls -l /mnt/data1 The output should look like this: (The content shown should be identical to the destination)
Hurray! You have created and used a FlexCache volume that reflects the data stored on the source volume located on a different ONTAP environment. As you saw, data created on the source can be read from the FlexCache volume, and when a change is made to existing data, or when new data is added, via the target FlexCache volume, it is immediately available on the source.
Congratulations!
In this lab, you learned the basics of ONTAP’s intelligent remote caching technology called FlexCache. With FlexCache data can be cached, either locally or at geographically dispersed remote sites and used for cloud bursting, improving productivity and collaboration across multiple locations and increasing data throughput for read-intensive workloads such as media rendering, EDA, seismic analysis and financial simulations. By leveraging two Cloud Volumes ONTAP instances (as mentioned previously, that caching relationship can be established between any flavor of ONTAP located anywhere) you configured an active caching relationship between two volumes - a source and its cached destination (FlexCache target), where data from the source is automatically cached on the target volume (based on your applications’ read operations) and when changes are made to cached data (or new data is added) the source volume is immediately and automatically updated.
End your lab
Now that you're finished with the lab, click on the End Lab button at the top of this page, then use the stars to rate your lab experience:
Leave comments about your experiences in the Comment window, Google always appreciates thoughtful feedback.
Ending a lab will remove your access to the Google Cloud project as well as the services and work you've done in it. If you go back to the console, you will see that you've been signed out automatically. You can close that tab now.
Next Steps / Learn More
Be sure to check out the following for more information and to get started:
-
NetApp on the Google Cloud Marketplace!
-
Get started with your 30-days Cloud Volumes ONTAP free trial
Google Cloud training and certification
...helps you make the most of Google Cloud technologies. Our classes include technical skills and best practices to help you get up to speed quickly and continue your learning journey. We offer fundamental to advanced level training, with on-demand, live, and virtual options to suit your busy schedule. Certifications help you validate and prove your skill and expertise in Google Cloud technologies.
Manual Last Updated March 07, 2023
Lab Last Tested March 07, 2023
Copyright 2024 Google LLC All rights reserved. Google and the Google logo are trademarks of Google LLC. All other company and product names may be trademarks of the respective companies with which they are associated.