Explore
For Enterprise
Join for Free
Log In

예술 & 인문학
비즈니스
컴퓨터 공학
데이터 과학
정보 기술
건강
수학 및 논리
자기개발
물리 과학 및 공학
사회 과학
언어 학습
학위
수료증

Explore all of Coursera

검색
포 엔터프라이즈 (For Enterprise)
로그인
무료 회원 가입

Approach: Move from on-premise to Google Cloud Platform

Loading...

Google Cloud Platform Big Data and Machine Learning Fundamentals

Google 클라우드

4.6(6,591개의 평가) | 69K명의 학생이 등록함

Data Engineering, Big Data, and Machine Learning on GCP 전문 분야에서 5의 강좌 1

This 1-week accelerated on-demand course introduces participants to the Big Data and Machine Learning capabilities of Google Cloud Platform (GCP). It provides a quick overview of the Google Cloud Platform and a deeper dive of the data processing capabilities. At the end of this course, participants will be able to: • Identify the purpose and value of the key Big Data and Machine Learning products in the Google Cloud Platform • Use CloudSQL and Cloud Dataproc to migrate existing MySQL and Hadoop/Pig/Spark/Hive workloads to Google Cloud Platform • Employ BigQuery and Cloud Datalab to carry out interactive data analysis • Choose between Cloud SQL, BigTable and Datastore • Train and use a neural network using TensorFlow • Choose between different data processing products on the Google Cloud Platform Before enrolling in this course, participants should have roughly one (1) year of experience with one or more of the following: • A common query language such as SQL • Extract, transform, load activities • Data modeling • Machine learning and/or statistics • Programming in Python Google Account Notes: • Google services are currently unavailable in China.

강의 계획서 보기

배우게 될 기술

Tensorflow, Bigquery, Google Cloud Platform, Cloud Computing

검토

4.6(6,591개의 평가)

5 stars
4,582 ratings
4 stars
1,661 ratings
3 stars
263 ratings
2 stars
43 ratings
1 star
42 ratings

DK

Apr 08, 2019

This course provides the basics of Google Cloud Platform and makes us understand Google Products and services on Cloud. I would love to explore GCP more and will opt for the remaining GCP courses.

AR

May 27, 2019

This course gives you a pretty good fundamental idea on what you are diving into. Google cloud platform has bucket of data engineering features. This course sets you on the starting line for them.

수업에서

Recommending Products using Cloud SQL and Spark

In this module you will have an existing Apache SparkML recommendation model that is running on-premise. You will learn about recommendation models and how you can run them in the cloud with Cloud Dataproc and Cloud SQL.

Approach: Move from on-premise to Google Cloud Platform9:50

Demo: From zero to an Apache Spark job in 10 minutes or less6:20

Challenge: Utilizing and tuning on-premise clusters6:05

Move storage off-cluster with Google Cloud Storage4:54

강사:

Google Cloud Training

스크립트

So we decided to use

a big data platform like Hadoop and then

RDBMS like MySQL to

solve the housing recommendations problem.

These are both open source technologies.

You probably have Hadoop clusters and

MySQL database running on-premises.

So let's assume that your team

already has this recommendation system working

on-prem and see how to migrate it

from on-premises to Google Cloud Platform.

Of course you want to do

that only if the migration can add value.

So we will look at that as well.

For a housing recommendation model,

let's say our data science team already has a working

on-premise model using

SparkML job on your Hadoop cluster.

They're interested in the scale and flexibility of GCP,

and they want to do a like- for-like migration of

their existing SparkML jobs from on-premises,

and they want to do this as a pilot project.

As explained in the previous lesson,

we can get away with computing

predicted ratings once a day.

We don't need to compute recommendations in real time.

So Hadoop batch processing is enough.

Here, we'll use SparkML,

but instead of doing it on-premises,

we'll run the machine-learning job on Cloud Dataproc

and store the ratings in an RDBMS in Cloud SQL,

because this is a relatively small dataset

of five recommendations for every user.

So this is what you will practice in your first lab.

Incidentally, this is why

the smart reply in Gmail is so amazing.

Unlike our housing recommendations,

Spark replies' recommendations have

to be done in real time,

when a new e-mail pops up in your mailbox.

Needless to say Gmail is doing

something far more sophisticated

than what we're talking about here.

Here's how we decided on using Cloud SQL

among the other big data products

available for storing our ratings.

This is a good reference to

follow based on your storage access pattern.

We'll cover the solutions in

your other scenarios in this course,

but briefly, use Cloud Storage as a global file system.

Use Cloud SQL as an RDBMS as

a relational database management system for

transactional relational data that

you access through SQL.

Use Datastore as

a transactional No-SQL object-oriented database.

Use Bigtable for high-throughput No-SQL append-only data.

So not transactions, append-only data.

So a typical use case for Bigtable is

sensor data for connected devices for example.

Use BigQuery as a SQL data warehouse

to power all your analytics needs.

So here we wanted a transactional database

and we expect to have

data volumes in the gigabytes or less,

and so hence Cloud SQL.

If you're a more visual learner,

here is another good map of

visualizing where to store your data in GCP.

If your data is unstructured,

like images or audio, use Cloud Storage.

If your data is structured and you need transactions,

use Cloud SQL or Cloud Datastore, depending

on whether you want

your access pattern to be SQL or No-SQL,

and by No-SQL we mean a key-value pair.

In other words, you'll be trying to

search for data based on a single key,

use Datastore if you'd be finding

data using SQL use Cloud SQL.

Cloud SQL generally plateaus out at a few gigabytes.

So if you wanted transactional database

that is horizontally

scalable so that you can deal with

data larger than a few gigabytes,

or if you need multiple databases so

you want them spread globally use Cloud Spanner.

So another way to say it as if

one database is enough use Cloud SQL.

If you'll need multiple databases,

either because you have a lot of data or because

your application needs to be transactional

across different continents, use Cloud Spanner.

If your data is structured and you want Analytics,

consider either Bigtable or BigQuery.

Use Bigtable if you

need real-time high-throughput applications.

Use BigQuery if you want

Analytics over petabyte-scale datasets.

For our housing recommendation use case,

we want to store our ratings and predictions somewhere.

This is a structured dataset of user ratings and houses.

It's built for a transactional workload, writes and reads,

and one database is enough for

a small dataset and hence we pick Cloud SQL.

So what's Cloud SQL?

It's a Google-hosted and

managed relational database in the Cloud.

Cloud SQL supports two open-source databases:

MySQL and Postgres, and other database solutions.

In our case, we'll be using MySQL.

One of the advantages of Cloud SQL is that it's familiar.

Cloud SQL supports most MySQL statements

and functions even stored procedures, triggers and views.

It brings the benefits of

Cloud economics in the form of flexible pricing.

You can pay for what you use. What do we mean by that?

GCP manages the MySQL Instance for you.

This means things like backup and replication,

it's on the Cloud so you can connect to it from anywhere.

You can assign it a static IP address,

and you can use typical SQL connector libraries.

Because it's behind the Google firewall it's fast.

You can place your Cloud SQL Instance in the same region

as your App Engine or Compute Engine applications

and get great bandwidth.

Also, you get Google Security.

Cloud SQL resides in secure Google datacenters.

So that covers where

we'll be storing the recommendations.

But how will we compute

these recommendations in the first place?

Where would we be doing the compute?

Let's review where big data computations have

been done historically and even today.

Before 2006, big data meant big databases.

Database design came from a time when storage

was relatively cheap and processing was expensive.

So it made sense to copy the data from

its storage location to

the processor to perform data processing,

and then the result would be copied back to storage.

Around 2006, distributed processing

of big data became practical with Hadoop.

The idea behind Hadoop is to create a cluster of

computers and leveraged distributed processing, HDFS.

The Hadoop Distributed File System,

stored the data on machines in the cluster,

and MapReduce

provided distributed processing of this data.

A whole ecosystem of Hadoop- related software grew up

around Hadoop, including Hive, Pig and Spark.

Around 2010, BigQuery was released

which was the first of

many big data services developed by Google.

Around 2015, Google launched Cloud Dataproc

which provides a managed service

for creating Hadoop and Spark clusters,

and managing data processing workloads.

The other piece of our system is

a software that runs on Hadoop.

The software in this case is to train

a machine learning model

for creating housing recommendations.

In this case we used

SparkML which is part of ApacheSpark.

ApacheSpark is

an open source software project that provides

a high-performance analytics engine

for processing batch and streaming data.

Spark can be up to 100 times faster than

equivalent Hadoop jobs because

it leverages in-memory processing.

Spark also provides a couple of abstractions

for dealing with data including what are called RDDs,

or Resilient Distributed Datasets and DataFrames.

We'll be using our Spark job

for the housing rental recommendations.

We'll be using our Spark job

for the housing rental recommendations,

but we'll be doing the computation on Cloud Dataproc.

Coursera 카탈로그 살펴보기

무료로 참여해 맞춤화된 추천, 업데이트 및 제안을 받아보세요.

Coursera는 세계 최고의 교육에 대한 보편적인 접근 기회를 제공하며, 최고의 대학 및 기관들과 제휴하여 온라인 강좌를 제공합니다.

© 2019 Coursera Inc. All rights reserved.

앱스토어에서 다운로드하기

구글 플레이에서 받기

Coursera
소개
리더십
직업
카탈로그
수료증
MasterTrack™ 수료증
학위
포 엔터프라이즈 (For Enterprise)
정부용

커뮤니티
Learners
Partners
Developers
Beta Testers
Translators

연결
블로그
Facebook
Linkedin
Twitter
Instagram
YouTube
기술 블로그

이상
조건
개인정보 보호
도움말
접근성
보도 자료
문의하기
디렉토리
계열사