Transcript


00:05
[electronic music]
00:17
Sandeep: Hi. My name is Sandeep,
00:19
a developer advocate on the Google Cloud platform.
00:22
Welcome to the Google Data Center
00:23
at the Dalles, Oregon.
00:25
Take a look around.
00:27
Before we go inside, we need to make sure
00:29
that we have the appropriate security clearance.
00:31
Most Google employees can't even get in here.
00:34
So let's go on a special behind-the-scenes tour.
00:39
[keypad beeps, door opens]
00:44
I'm here with Noah
00:46
from the Site Reliability Engineering Team.
00:48
Noah, can you tell us a little bit more
00:49
about the SRE role at Google?
00:51
Noah: Yeah, SREs write and maintain the software systems
00:53
designed to keep our services running.
00:55
Sandeep: So what happens if one of these systems goes down?
00:57
Noah: We've designed our systems from the ground up
01:00
to be able to handle any unexpected failures
01:02
that might occur.
01:03
We have highly redundant power, networking,
01:05
and serving domains so that even if we do lose
01:07
an entire cluster, we're able to re-direct those workloads
01:10
and live migrate data in order to minimize any impact.
01:13
In addition, we have a team of SREs on call 24/7
01:17
that can tackle any problems that might arise.
01:20
Sandeep: Thanks, Noah.
01:21
Now we've learned more about the systems
01:23
that manage our fleet at Google,
01:24
let's take a deeper look
01:25
at the data center infrastructure itself.
01:29
Before we can continue further,
01:31
we need to go through the biometric iris scan
01:33
and circle lock.
01:34
These only allow one person in at a time
01:37
and require dual authentication
01:38
to continue further.
01:40
I'll see you on the other side.
01:44
[control beeps]
01:45
computer voice: Please situate your eyes
01:47
to begin the procedure.
01:52
Please come a little closer to the camera.
01:55
[beep]
01:56
Sandeep: Welcome to the data center floor.
01:59
As you can tell, we have a lot of servers,
02:01
and this is a single cluster in a single floor
02:04
in a single building.
02:06
Managing all of these servers on a global scale
02:08
is quite a challenge.
02:11
To utilize our fleet, we use tools
02:13
such as Borg, Colossus, and Spanner.
02:16
You may be familiar with similar tools,
02:18
such as Kubernetes, Google Cloud storage,
02:20
and BigQuery.
02:22
These tools allow Google engineers
02:24
and Cloud customers
02:25
to more easily manage infrastructure,
02:27
allowing everyone to build
02:28
innovative and scalable applications.
02:36
Here at Google, a lot of our infrastructure is custom-made.
02:39
This gives us the flexibility and performance
02:42
we need to run all of our services at scale.
02:44
Oh, hey, it's Virginia, one of our network engineers.
02:46
Virginia: Hey, Sandeep.
02:47
Sandeep: Virginia, what are you working on today?
02:49
Virginia: Today I'm working with Hardware Ops
02:50
to expand this data center network
02:52
to deploy additional machines in this building.
02:54
Our fleet is constantly growing to support new capacity
02:57
for Google products and our Cloud customers.
03:00
Sandeep: That sounds like a lot of work,
03:01
to be constantly adding capacity around the globe.
03:03
Virginia: Well, we designed our network
03:05
so that this kind of capacity growth isn't very hard.
03:07
Jupiter, our current data center and network technology,
03:10
is a hierarchical design
03:12
using software-defined networking principles.
03:14
So just like with our servers,
03:16
we abstracted away the specific details of our network
03:19
and can manage them like they're software programs and data.
03:22
Sandeep: Abstracting seems to be a common theme here at Google.
03:25
I've also noticed there's a lot of fiber
03:26
running in our data centers. Virginia: That's right.
03:28
A single building can support 75,000 machines,
03:31
and carry over one petabit per second of bandwidth,
03:34
which is actually more than on the entire Internet.
03:37
Sandeep: Wow. Virginia: This allows us
03:38
to reliably access storage and compute resources
03:41
with low latency and high throughput.
03:43
Sandeep: So how is this data center connected to
03:45
all our other data centers around the globe?
03:47
Virginia: Google runs B4,
03:49
our own private, highly efficient backbone network,
03:52
which is actually growing faster
03:53
than our Internet-facing network.
03:55
It connects all our data centers together
03:57
and allows services to efficiently access resources
04:00
in any location. Sandeep: Nice.
04:02
I finally know what all this Google fiber is really used for.
04:04
Thanks, Virginia. Virginia: No problem.
04:07
Sandeep: So now you've seen
04:08
all the compute and networking horsepower
04:10
required to run your workloads in the Cloud,
04:12
let's take a look at where your data is
04:14
safely and securely stored.
04:16
Let's go.
04:19
Whether you're querying terabytes of data on BigQuery
04:22
or storing petabytes in Google Cloud Storage,
04:25
all of your data needs to be stored on a physical device.
04:28
Our data center infrastructure allows us
04:30
to access our storage quickly and securely.
04:33
At our scale, we need to handle hard drive and SSD failure
04:37
on a daily basis.
04:38
While your data is replicated and safe,
04:40
we need to destroy or recycle used hard drives
04:43
so no one can access your data.
04:45
From the time a disc is removed from the server
04:47
to the time it's decommissioned,
04:49
we maintain a very strict chain of custody.
04:52
The discs are completely wiped and then destroyed
04:54
in a huge shredder.
04:56
Let's go shred some hard drives.
05:06
[beeping]
05:17
We've looked at a lot of the hardware
05:19
that runs in our data centers, but it doesn't end there.
05:22
We need to cool and power our infrastructure
05:25
in an environmentally sustainable and reliable way.
05:28
Let's take a look at how we cool our servers.
05:33
Welcome to the mechanical equipment room.
05:35
Looks pretty cool, doesn't it?
05:38
Oh, hey, it's Brian, one of
05:39
our data center facilities technicians!
05:40
Brian: Hey, Sandeep. Sandeep: Hey, Brian.
05:42
Brian, can you tell us a little bit more about this room?
05:44
Brian: Sure. This is a cooling plant
05:46
for one of the data centers that we have on site.
05:49
So a lot of heat is generated on the server floor,
05:51
and it all has to be removed,
05:53
and that starts right here in the cooling plant.
05:55
So it's basically two loops.
05:57
We have the condenser water loop
05:58
and we have the process water loop.
06:00
The process water loop are these blue and red pipes over here.
06:03
So they take the heat off the server floor
06:05
and they transfer it to these heat exchangers here.
06:08
The condenser water loop are
06:09
these green and yellow pipes here.
06:11
They take the cold water from the basin underneath us,
06:14
they transfer it to these heat exchangers here,
06:16
and they send it up to the cooling towers up on the roof.
06:18
Sandeep: I notice our pipes are Google colors.
06:20
It's pretty cool.
06:22
So how efficient is our data center?
06:24
Brian: Well, Google has some of
06:25
the most efficient data centers in the world.
06:27
In fact, when we started reporting our power usage effectiveness
06:30
or P.U.E., in 2008,
06:32
most data centers were around 100% overhead.
06:36
At that point in time, Google was 20% overhead,
06:38
but since then, we've reduced it to just 12%,
06:41
and that even includes our cafeterias.
06:43
Sandeep: Whoa! That is so low!
06:46
Also what's this big green machine for?
06:48
Brian: Oh, well, this is a chiller.
06:50
We very rarely use them,
06:51
but it helps keep the process water temperature
06:53
in the desired temperature range
06:55
when it gets really hot outside,
06:57
basically helping the cooling tower do its job,
06:59
and some of our newer data centers,
07:01
they have no chillers at all.
07:03
Sandeep: I love how our new data centers are even more efficient.
07:06
By the way, can we go up and take a look at a cooling tower?
07:09
Brian: Sure. Let's go.
07:13
Sandeep: Wow, what a view up here!
07:16
Brian: So, Sandeep, this is a cooling tower.
07:18
It uses evaporation to rapidly cool the water
07:20
from the condenser loop and sends it back down to the basin.
07:23
You could say we're making actual clouds with the Cloud.
07:26
Sandeep: Clouds making actual clouds--welcome to Google!
07:29
So, Brian, how do we power the Cloud?
07:31
Brian: Well, that all starts at Google's power substation.
07:33
Let's go take a look.
07:38
So this is the Google-owned power substation.
07:41
This is where the high voltage power enters the site.
07:43
It's reduced and then sent
07:45
to multiple power distribution centers
07:46
such as this one right here.
07:48
Sandeep: What happens if a power distribution center
07:49
loses power?
07:50
Brian: If it loses power, we have
07:52
multiple generator and utility backup sources
07:54
available to maintain power to those servers.
07:56
Sandeep: And where does all the power come from?
07:58
Brian: It actually comes from
07:59
multiple hydroelectric power plants
08:01
that are nearby. Sandeep: I love how Google uses
08:03
reliable green energy whenever possible.
08:06
Brian: We are 100% carbon neutral actually.
08:08
Sandeep: That's pretty cool
08:09
You know, it seems like Google builds reliability
08:11
from the ground up, from the power and cooling
08:13
all the way to the software systems that manage our fleet.
08:16
Thanks for showing me around, Brian.
08:17
Brian: No problem. Have a great day.
08:19
Sandeep: Thank you for joining me on
08:20
this special behind-the-scenes tour.
08:22
Please check out cloud.google.com
08:24
to learn how you can build what's next.