[{"id":"76321","title":"Introduction","description":"

This module covers the course outline and does a quick refresh on the Apache Beam programming model and Google’s Dataflow managed service.

","steps":[{"id":"527489","prompt":null,"isOptional":true,"activities":[{"id":"520329","href":"/paths/1834/course_templates/218/video/520329","isLocked":false,"duration":111000,"title":"Course Introduction","description":null,"type":"video","isComplete":false,"inProgress":false,"score":null,"disabled":false}],"isComplete":false,"isTestedOut":false,"allActivitiesRequired":false},{"id":"527490","prompt":null,"isOptional":true,"activities":[{"id":"520330","href":"/paths/1834/course_templates/218/video/520330","isLocked":false,"duration":92000,"title":"Beam and Dataflow Refresher","description":null,"type":"video","isComplete":false,"inProgress":false,"score":null,"disabled":false}],"isComplete":false,"isTestedOut":false,"allActivitiesRequired":false}],"expanded":false},{"id":"76322","title":"Beam Portability","description":"

In this module we are going to learn about four sections, Beam Portablity, Runner v2, Container Environments, and Cross-Language Transforms.

","steps":[{"id":"527491","prompt":null,"isOptional":true,"activities":[{"id":"520331","href":"/paths/1834/course_templates/218/video/520331","isLocked":false,"duration":133000,"title":"Beam Portability","description":null,"type":"video","isComplete":false,"inProgress":false,"score":null,"disabled":false}],"isComplete":false,"isTestedOut":false,"allActivitiesRequired":false},{"id":"527492","prompt":null,"isOptional":true,"activities":[{"id":"520332","href":"/paths/1834/course_templates/218/video/520332","isLocked":false,"duration":36000,"title":"Runner v2","description":null,"type":"video","isComplete":false,"inProgress":false,"score":null,"disabled":false}],"isComplete":false,"isTestedOut":false,"allActivitiesRequired":false},{"id":"527493","prompt":null,"isOptional":true,"activities":[{"id":"520333","href":"/paths/1834/course_templates/218/video/520333","isLocked":false,"duration":92000,"title":"Container Environments","description":null,"type":"video","isComplete":false,"inProgress":false,"score":null,"disabled":false}],"isComplete":false,"isTestedOut":false,"allActivitiesRequired":false},{"id":"527494","prompt":null,"isOptional":true,"activities":[{"id":"520334","href":"/paths/1834/course_templates/218/video/520334","isLocked":false,"duration":85000,"title":"Cross-Language Transforms","description":null,"type":"video","isComplete":false,"inProgress":false,"score":null,"disabled":false}],"isComplete":false,"isTestedOut":false,"allActivitiesRequired":false},{"id":"527495","prompt":null,"isOptional":true,"activities":[{"id":"520335","href":"/paths/1834/course_templates/218/quizzes/520335","isLocked":false,"duration":600000,"title":"Quiz","description":null,"type":"quiz","isComplete":false,"inProgress":false,"score":null,"disabled":false}],"isComplete":false,"isTestedOut":false,"allActivitiesRequired":false}],"expanded":false},{"id":"76323","title":"Separating Compute and Storage with Dataflow","description":"

In this module we discuss how to separate compute and storage with Dataflow. This module contains four sections Dataflow, Dataflow Shuffle Service, Dataflow Streaming Engine, Flexible Resource Scheduling.

","steps":[{"id":"527496","prompt":null,"isOptional":true,"activities":[{"id":"520336","href":"/paths/1834/course_templates/218/video/520336","isLocked":false,"duration":85000,"title":"Separating Storage and Compute with Dataflow","description":null,"type":"video","isComplete":false,"inProgress":false,"score":null,"disabled":false}],"isComplete":false,"isTestedOut":false,"allActivitiesRequired":false},{"id":"527497","prompt":null,"isOptional":true,"activities":[{"id":"520337","href":"/paths/1834/course_templates/218/video/520337","isLocked":false,"duration":80000,"title":"Dataflow Shuffle Service","description":null,"type":"video","isComplete":false,"inProgress":false,"score":null,"disabled":false}],"isComplete":false,"isTestedOut":false,"allActivitiesRequired":false},{"id":"527498","prompt":null,"isOptional":true,"activities":[{"id":"520338","href":"/paths/1834/course_templates/218/video/520338","isLocked":false,"duration":69000,"title":"Dataflow Streaming Engine","description":null,"type":"video","isComplete":false,"inProgress":false,"score":null,"disabled":false}],"isComplete":false,"isTestedOut":false,"allActivitiesRequired":false},{"id":"527499","prompt":null,"isOptional":true,"activities":[{"id":"520339","href":"/paths/1834/course_templates/218/video/520339","isLocked":false,"duration":62000,"title":"Flexible Resource Scheduling","description":null,"type":"video","isComplete":false,"inProgress":false,"score":null,"disabled":false}],"isComplete":false,"isTestedOut":false,"allActivitiesRequired":false},{"id":"527500","prompt":null,"isOptional":true,"activities":[{"id":"520340","href":"/paths/1834/course_templates/218/quizzes/520340","isLocked":false,"duration":900000,"title":"Quiz","description":null,"type":"quiz","isComplete":false,"inProgress":false,"score":null,"disabled":false}],"isComplete":false,"isTestedOut":false,"allActivitiesRequired":false}],"expanded":false},{"id":"76324","title":"IAM, Quotas, and Permissions","description":"

In this module, we talk about the different IAM roles, quotas, and permissions required to run Dataflow

","steps":[{"id":"527501","prompt":null,"isOptional":true,"activities":[{"id":"520341","href":"/paths/1834/course_templates/218/video/520341","isLocked":false,"duration":377000,"title":"IAM","description":null,"type":"video","isComplete":false,"inProgress":false,"score":null,"disabled":false}],"isComplete":false,"isTestedOut":false,"allActivitiesRequired":false},{"id":"527502","prompt":null,"isOptional":true,"activities":[{"id":"520342","href":"/paths/1834/course_templates/218/video/520342","isLocked":false,"duration":379000,"title":"Quotas","description":null,"type":"video","isComplete":false,"inProgress":false,"score":null,"disabled":false}],"isComplete":false,"isTestedOut":false,"allActivitiesRequired":false},{"id":"527503","prompt":null,"isOptional":true,"activities":[{"id":"520343","href":"/paths/1834/course_templates/218/quizzes/520343","isLocked":false,"duration":600000,"title":"Quiz","description":null,"type":"quiz","isComplete":false,"inProgress":false,"score":null,"disabled":false}],"isComplete":false,"isTestedOut":false,"allActivitiesRequired":false}],"expanded":false},{"id":"76325","title":"Security","description":"

In this module, we will look at how to implement the right security model for your use case on Dataflow.

","steps":[{"id":"527504","prompt":null,"isOptional":true,"activities":[{"id":"520344","href":"/paths/1834/course_templates/218/video/520344","isLocked":false,"duration":253000,"title":"Data Locality","description":null,"type":"video","isComplete":false,"inProgress":false,"score":null,"disabled":false}],"isComplete":false,"isTestedOut":false,"allActivitiesRequired":false},{"id":"527505","prompt":null,"isOptional":true,"activities":[{"id":"520345","href":"/paths/1834/course_templates/218/video/520345","isLocked":false,"duration":110000,"title":"Shared VPC","description":null,"type":"video","isComplete":false,"inProgress":false,"score":null,"disabled":false}],"isComplete":false,"isTestedOut":false,"allActivitiesRequired":false},{"id":"527506","prompt":null,"isOptional":true,"activities":[{"id":"520346","href":"/paths/1834/course_templates/218/video/520346","isLocked":false,"duration":98000,"title":"Private IPs","description":null,"type":"video","isComplete":false,"inProgress":false,"score":null,"disabled":false}],"isComplete":false,"isTestedOut":false,"allActivitiesRequired":false},{"id":"527507","prompt":null,"isOptional":true,"activities":[{"id":"520347","href":"/paths/1834/course_templates/218/video/520347","isLocked":false,"duration":178000,"title":"CMEK","description":null,"type":"video","isComplete":false,"inProgress":false,"score":null,"disabled":false}],"isComplete":false,"isTestedOut":false,"allActivitiesRequired":false},{"id":"527508","prompt":null,"isOptional":true,"activities":[{"id":"520348","href":"/paths/1834/course_templates/218/labs/520348","isLocked":false,"duration":4500000,"title":"Setup IAM and Networking for your Dataflow Jobs","description":"In this lab, you will learn to setup IAM permissions and use private IPs for your Datafow jobs.","type":"lab","isComplete":false,"inProgress":false,"score":null,"disabled":false}],"isComplete":false,"isTestedOut":false,"allActivitiesRequired":false},{"id":"527509","prompt":null,"isOptional":true,"activities":[{"id":"520349","href":"/paths/1834/course_templates/218/quizzes/520349","isLocked":false,"duration":600000,"title":"Quiz","description":null,"type":"quiz","isComplete":false,"inProgress":false,"score":null,"disabled":false}],"isComplete":false,"isTestedOut":false,"allActivitiesRequired":false}],"expanded":false},{"id":"76326","title":"Summary","description":"

In this course, we started with the refresher of what Apache Beam is, and its relationship with Dataflow.

","steps":[{"id":"527510","prompt":null,"isOptional":true,"activities":[{"id":"520350","href":"/paths/1834/course_templates/218/video/520350","isLocked":false,"duration":234000,"title":"Course Summary","description":null,"type":"video","isComplete":false,"inProgress":false,"score":null,"disabled":false}],"isComplete":false,"isTestedOut":false,"allActivitiesRequired":false},{"id":"527511","prompt":null,"isOptional":true,"activities":[{"id":"520351","href":"/paths/1834/course_templates/218/documents/520351","isLocked":false,"duration":1800000,"title":"Additional Resources","description":null,"type":"document","isComplete":false,"inProgress":false,"score":null,"disabled":false}],"isComplete":false,"isTestedOut":false,"allActivitiesRequired":false}],"expanded":false},{"id":"next-steps","title":"Your Next Steps","description":null,"steps":[{"id":"badge-step","prompt":null,"isOptional":true,"activities":[{"id":"badge","href":null,"isLocked":true,"duration":null,"title":"Course Badge","description":null,"type":"badge","isComplete":false,"inProgress":false,"score":null,"disabled":false}],"isComplete":false,"isTestedOut":false,"allActivitiesRequired":false}],"expanded":true}]

06

Serverless Data Processing with Dataflow: Foundations