Overview

In this lab, you will create a private cluster, add an authorized network for API access to it, and then configure a network policy for Pod security.

Note: For this lab, GKE Standard Mode will be used. The lab explores cluster network policies and these are enabled by default in GKE Autopilot.

Objectives

In this lab, you learn how to perform the following tasks:

• Create and test a private cluster.
• Configure a cluster for authorized network control plane access.
• Configure a Cluster network policy.

Setup and requirements

For each lab, you get a new Google Cloud project and set of resources for a fixed time at no cost.

1. Click the Start Lab button. If you need to pay for the lab, a pop-up opens for you to select your payment method. On the left is the Lab Details panel with the following:

   • The Open Google Cloud console button
   • Time remaining
   • The temporary credentials that you must use for this lab
   • Other information, if needed, to step through this lab

2. Click Open Google Cloud console (or right-click and select Open Link in Incognito Window if you are running the Chrome browser).

   The lab spins up resources, and then opens another tab that shows the Sign in page.

   Tip: Arrange the tabs in separate windows, side-by-side.

   Note: If you see the Choose an account dialog, click Use Another Account.

3. If necessary, copy the Username below and paste it into the Sign in dialog.

   {{{user_0.username | "Username"}}}

   You can also find the Username in the Lab Details panel.

4. Click Next.

5. Copy the Password below and paste it into the Welcome dialog.

   {{{user_0.password | "Password"}}}

   You can also find the Password in the Lab Details panel.

6. Click Next.

   Important: You must use the credentials the lab provides you. Do not use your Google Cloud account credentials. Note: Using your own Google Cloud account for this lab may incur extra charges.

7. Click through the subsequent pages:

   • Accept the terms and conditions.
   • Do not add recovery options or two-factor authentication (because this is a temporary account).
   • Do not sign up for free trials.

After a few moments, the Google Cloud console opens in this tab.

Note: To view a menu with a list of Google Cloud products and services, click the Navigation menu at the top-left, or type the service or product name in the Search field.

After you complete the initial sign-in steps, the project dashboard appears.

Activate Google Cloud Shell

Google Cloud Shell is a virtual machine that is loaded with development tools. It offers a persistent 5GB home directory and runs on the Google Cloud.

Google Cloud Shell provides command-line access to your Google Cloud resources.

1. In Cloud console, on the top right toolbar, click the Open Cloud Shell button.

   

2. Click Continue.

It takes a few moments to provision and connect to the environment. When you are connected, you are already authenticated, and the project is set to your PROJECT_ID. For example:

gcloud is the command-line tool for Google Cloud. It comes pre-installed on Cloud Shell and supports tab-completion.

• You can list the active account name with this command:

gcloud auth list

Output:

Credentialed accounts: - @.com (active)

Example output:

Credentialed accounts: - google1623327_student@qwiklabs.net

• You can list the project ID with this command:

gcloud config list project

Output:

[core] project =

Example output:

[core] project = qwiklabs-gcp-44776a13dea667a6 Note: Full documentation of gcloud is available in the gcloud CLI overview guide .

Task 1. Create a private cluster

In this task, you create a private cluster, consider the options for how private to make it, and then compare your private cluster to your original cluster.

In a private cluster, the nodes have internal RFC 1918 IP addresses only, which ensures that their workloads are isolated from the public Internet. The nodes in a non-private cluster have external IP addresses, potentially allowing traffic to and from the internet.

Set up a private cluster

1. In the Navigation menu (), click Kubernetes Engine > Clusters.
2. Click Create and select SWITCH TO STANDARD CLUSTER for the cluster.
3. Name the cluster private-cluster.
4. Select as the Zone.
5. In the left pane, in Node Pools click default-pool.
6. In Number of nodes type 2.

7. In the left pane, in Cluster click on Networking.
8. Ensure Enable authorized networks is not selected.
9. Select the checkbox to Override control plane’s default private endpoint subnet.
10. Under Private endpoint subnet select as default.

11. Select the checkbox to Enable Private nodes.

Note: Google Cloud uses VPC peering to connect the VPC of the cluster with your default VPC network.

You may have to scroll down to see this option.

This setting allows you the range of addresses that can access the cluster externally. When this checkbox is not selected, you can access kubectl only from within the Google Cloud network. In this lab, you will only access kubectl through the Google Cloud network but you will modify this setting later.

12. Click Create.

Note: You need to wait a few minutes for the cluster deployment to complete.

Inspect your cluster

• In the Cloud Shell, enter the following command to review the details of your new cluster:

gcloud container clusters describe private-cluster --zone {{{project_0.default_zone|ZONE}}}

The following values appear only under the private cluster:

• privateEndpoint, an internal IP address. Nodes use this internal IP address to communicate with the cluster control plane.
• publicEndpoint, an external IP address. External services and administrators can use the external IP address to communicate with the cluster control plane.

You have several options to lock down your cluster to varying degrees:

• The whole cluster can have external access.
• The whole cluster can be private.
• The nodes can be private while the cluster control plane is public, and you can limit which external networks are authorized to access the cluster control plane.

Without public IP addresses, code running on the nodes can't access the public internet unless you configure a NAT gateway such as Cloud NAT.

You might use private clusters to provide services such as internal APIs that are meant only to be accessed by resources inside your network. For example, the resources might be private tools that only your company uses. Or they might be backend services accessed by your frontend services, and perhaps only those frontend services are accessed directly by external customers or users. In such cases, private clusters are a good way to reduce the surface area of attack for your application.

Click Check my progress to verify the objective. Create a private cluster

Task 2. Add an authorized network for cluster control plane access

After cluster creation, you might want to issue commands to your cluster from outside Google Cloud. For example, you might decide that only your corporate network should issue commands to your cluster control plane. Unfortunately, you didn't specify the authorized network on cluster creation.

In this task, you add an authorized network for cluster control plane access.

Note: In this task, you make the Kubernetes control plane API accessible to a specific range of network addresses. In a real-world use of GKE, this connection would be used by IT staff and automated processes, not end-users.

1. In the Google Cloud console Navigation menu (), click Kubernetes Engine > Clusters.
2. Click private-cluster to open the Clusters details page.
3. In Details tab, under Control Plane Networking section, click on Edit().
4. Select Enable authorized networks.
5. Click Add an authorized network.
6. For Name, type the name for the network, use Corporate.
7. For Network, type a CIDR range that you want to grant whitelisted access to your cluster control plane. As an example, you can use 192.168.1.0/24.
8. Click Done.
9. Ensure the following parameters are enabled:
   • Access using the control plane's external IP address
   • Access using the control plane's internal IP address from any region
   • Enforce authorized networks on the control plane's internal endpoint
   • Add Google Cloud external IP addresses to authorized networks

Multiple networks can be added here if necessary, but no more than 50 CIDR ranges.

Note: Outside this lab environment, a practical example might be to whitelist only the public, outside address of your corporate firewall. For example, if your corporate firewall's IP address were 8.8.8.14, you could whitelist access to 8.8.8.14/32.

9. Click Save changes at the bottom of the menu.

Click Check my progress to verify the objective. Add an authorized network for cluster control plane access

Task 3. Create a cluster network policy

In this task, you create a cluster network policy to restrict communication between the Pods. A zero-trust zone is important to prevent lateral attacks within the cluster when an intruder compromises one of the Pods.

Create a GKE cluster

1. In Cloud Shell, type the following command to set the environment variable for the zone and cluster name:

export my_zone={{{project_0.default_zone|ZONE}}} export my_cluster=standard-cluster-1

2. Configure kubectl tab completion in Cloud Shell:

source <(kubectl completion bash)

3. In Cloud Shell, type the following command to create a Kubernetes cluster. Note that this command adds the additional flag --enable-network-policy to the parameters you have used in previous labs. This flag allows this cluster to use cluster network policies:

gcloud container clusters create $my_cluster --num-nodes 3 --enable-ip-alias --zone $my_zone --enable-network-policy

4. In Cloud Shell, configure access to your cluster for the kubectl command-line tool, using the following command:

gcloud container clusters get-credentials $my_cluster --zone $my_zone

5. Run a simple web server application with the label app=hello, and expose the web application internally in the cluster:

kubectl run hello-web --labels app=hello \ --image=gcr.io/google-samples/hello-app:1.0 --port 8080 --expose

Restrict incoming traffic to Pods

Let's create a sample NetworkPolicy manifest file called hello-allow-from-foo.yaml . This manifest file defines an ingress policy that allows access to Pods labeled app: hello from Pods labeled app: foo.

1. Create and open a file called hello-allow-from-foo.yaml with nano using the following command:

nano hello-allow-from-foo.yaml

2. Once nano has opened, paste the following into the hello-allow-from-foo.yaml file:

kind: NetworkPolicy apiVersion: networking.k8s.io/v1 metadata: name: hello-allow-from-foo spec: policyTypes: - Ingress podSelector: matchLabels: app: hello ingress: - from: - podSelector: matchLabels: app: foo

3. Press Ctrl+O, and then press Enter to save your edited file.

4. Press Ctrl+X to exit the nano text editor.

5. Create an ingress policy:

kubectl apply -f hello-allow-from-foo.yaml

6. Verify that the policy was created:

kubectl get networkpolicy

Output:

NAME POD-SELECTOR AGE hello-allow-from-foo app=hello 7s

Validate the ingress policy

1. Run a temporary Pod called test-1 with the label app=foo and get a shell in the Pod:

kubectl run test-1 --labels app=foo --image=alpine --restart=Never --rm --stdin --tty Note: The kubectl switches used here in conjunction with the run command are important to note.

--stdin ( alternatively -i ) creates an interactive session attached to STDIN on the container.

--tty ( alternatively -t ) allocates a TTY for each container in the pod.

--rm instructs Kubernetes to treat this as a temporary Pod that will be removed as soon as it completes its startup task. As this is an interactive session it will be removed as soon as the user exits the session.

--label ( alternatively -l ) adds a set of labels to the pod.

--restart defines the restart policy for the Pod.

2. Make a request to the hello-web:8080 endpoint to verify that the incoming traffic is allowed:

wget -qO- --timeout=2 http://hello-web:8080

Output:

If you don't see a command prompt, try pressing enter. / # wget -qO- --timeout=2 http://hello-web:8080 Hello, world! Version: 1.0.0 Hostname: hello-web-8b44b849-k96lh / #

3. Type exit and press ENTER to leave the shell.

4. Now you will run a different Pod using the same Pod name but using a label, app=other, that does not match the podSelector in the active network policy. This Pod should not have the ability to access the hello-web application.

kubectl run test-1 --labels app=other --image=alpine --restart=Never --rm --stdin --tty

5. Make a request to the hello-web:8080 endpoint to verify that the incoming traffic is not allowed:

wget -qO- --timeout=2 http://hello-web:8080

The request times out.

Output:

If you don't see a command prompt, try pressing enter. / # wget -qO- --timeout=2 http://hello-web:8080 wget: download timed out / #

6. Type exit and press ENTER to leave the shell.

Restrict outgoing traffic from the Pods

You can restrict outgoing (egress) traffic as you do incoming traffic. However, in order to query internal hostnames (such as hello-web) or external hostnames (such as www.example.com), you must allow DNS resolution in your egress network policies. DNS traffic occurs on port 53, using TCP and UDP protocols.

Let's create a NetworkPolicy manifest file foo-allow-to-hello.yaml. This file defines a policy that permits Pods with the label app: foo to communicate with Pods labeled app: hello on any port number, and allows the Pods labeled app: foo to communicate to any computer on UDP port 53, which is used for DNS resolution. Without the DNS port open, you will not be able to resolve the hostnames.

1. Create and open a file called foo-allow-to-hello.yaml with nano using the following command:

nano foo-allow-to-hello.yaml

2. Once nano has opened, paste the following into the foo-allow-to-hello.yaml file:

kind: NetworkPolicy apiVersion: networking.k8s.io/v1 metadata: name: foo-allow-to-hello spec: policyTypes: - Egress podSelector: matchLabels: app: foo egress: - to: - podSelector: matchLabels: app: hello - to: ports: - protocol: UDP port: 53

3. Press Ctrl+O, and then press Enter to save your edited file.

4. Press Ctrl+X to exit the nano text editor.

5. Create an egress policy:

kubectl apply -f foo-allow-to-hello.yaml

6. Verify that the policy was created:

kubectl get networkpolicy

Output:

NAME POD-SELECTOR AGE foo-allow-to-hello app=foo 7s hello-allow-from-foo app=hello 5m

Validate the egress policy

1. Deploy a new web application called hello-web-2 and expose it internally in the cluster:

kubectl run hello-web-2 --labels app=hello-2 \ --image=gcr.io/google-samples/hello-app:1.0 --port 8080 --expose

2. Run a temporary Pod with the app=foo label and get a shell prompt inside the container:

kubectl run test-3 --labels app=foo --image=alpine --restart=Never --rm --stdin --tty

3. Verify that the Pod can establish connections to hello-web:8080:

wget -qO- --timeout=2 http://hello-web:8080

Output:

If you don't see a command prompt, try pressing enter. / # wget -qO- --timeout=2 http://hello-web:8080 Hello, world! Version: 1.0.0 Hostname: hello-web / #

4. Verify that the Pod cannot establish connections to hello-web-2:8080:

wget -qO- --timeout=2 http://hello-web-2:8080

This fails because none of the Network policies you have defined allow traffic to Pods labeled app: hello-2.

5. Verify that the Pod cannot establish connections to external websites, such as www.example.com:

wget -qO- --timeout=2 http://www.example.com

This fails because the network policies do not allow external http traffic (tcp port 80).

6. Type exit and press ENTER to leave the shell.

Click Check my progress to verify the objective. Run web server applications

End your lab

When you have completed your lab, click End Lab. Google Cloud Skills Boost removes the resources you’ve used and cleans the account for you.

You will be given an opportunity to rate the lab experience. Select the applicable number of stars, type a comment, and then click Submit.

The number of stars indicates the following:

• 1 star = Very dissatisfied
• 2 stars = Dissatisfied
• 3 stars = Neutral
• 4 stars = Satisfied
• 5 stars = Very satisfied

You can close the dialog box if you don't want to provide feedback.

For feedback, suggestions, or corrections, please use the Support tab.

Copyright 2022 Google LLC All rights reserved. Google and the Google logo are trademarks of Google LLC. All other company and product names may be trademarks of the respective companies with which they are associated.