00:01Alright, welcome to this session on Amazon Web Services.
00:06Yeah, last DevSummit, we asked people what we could do better, and they said, "Find some bigger rooms for the presentations."
00:13So here I am. I never thought I'd make it to the big stage.
00:16My name is Sterling Quinn. I'm a product engineer on the ArcGIS Server development team.
00:21Marwa Mabrouk is with me. She is the product manager for ArcGIS Server on Amazon.
00:27Actually, we changed the name to "on Amazon Web Services." You might've noticed that.
00:31That was at Amazon's request. Amazon Web Services is their umbrella for all of the web services that they offer...
00:39...and the Elastic Compute Cloud, or EC2, which we'll be talking about today, is just one part of that.
00:46Now, most of ArcGIS Server is driven by EC2; it runs on EC2, but you may see throughout this presentation...
00:54...where we see or mention other web services types that are offered by Amazon.
01:02Today I'm going to start out with just a brief introduction in case you're new to the idea of running ArcGIS Server on Amazon...
01:08...and then I will do a short demo of how you get started with running it on Amazon, and I will use 10.1...
01:16...including the new Cloud Builder application that Esri made for deploying ArcGIS Server sites on Amazon.
01:25And then I'll turn the time over to Marwa.
01:27She's going to show some case studies of people who have used ArcGIS Server on Amazon...
01:32...and she'll get into a more detailed showcase of that...
01:36...and she'll also talk about business considerations and costs of using the cloud, which is her forte.
01:43And then we'll leave some time at the end for questions...
01:45...so please jot down your question, and we'll be sure to leave time for that.
01:52And if you do have a question, you might want to come up here to ask it. May not be able to hear you.
01:58So just as an introduction, this is a slide that I don't know if you guys need.
02:03Probably most folks at this conference understand the concept of virtualization.
02:09I had this in here at the User Conference sometimes for beginners.
02:14But really, that's the idea of sharing one physical resource to host multiple operating systems...
02:21...or to make it look like there's multiple machines running.
02:24So they share hardware; that's the part you need to understand.
02:28Because Amazon has really procured large amounts of hardware, and they've become very good at making it efficient...
02:35...to the point where they feel that they can rent it out through their web services.
02:40They actually developed some of this hardware to run their own website.
02:43You know, at Christmastime, Amazon gets quite a bit of traffic for the holidays, people buying gifts.
02:49They had increased the power of their site to handle that, and eventually...
02:53...they decided that their infrastructure was something that maybe other people could use as well...
02:57...so they made a business out of renting out hardware infrastructure, and that's what Amazon EC2 has become.
03:07What it is, is you are running virtual machines on Amazon's hardware within their data centers...
03:13...and their data centers are built throughout different geographic regions of the world.
03:17And they make their data centers fail proof or failsafe as much as they can, isolating certain blocks of hardware resources...
03:25...making it power efficient and coming from different power sources.
03:29And they take many types of physical security measures around these data centers...
03:33...that you might not be able to take with your own data center, so there is some advantage to that.
03:38If you think about your layout, your server room might look much like this graphic on the left, where you have a finite number...
03:45...of physical resources inside of an area maybe in your building or somewhere on your organization's campus.
03:54It may or may not be locked up or subject to certain security measures.
03:59It may or may not have adequate air conditioning or other things.
04:03Somebody could trip over the cord.
04:04These are all things to think about that you might - challenges that you might deal with on premises.
04:11When you have your hardware on Amazon EC2, it's in their data centers, and it's optimized for running in their environment...
04:18...and they've been able to set their cost based on their studies of how efficiently...
04:24... they can keep their data centers heated and cooled and power these resources.
04:30So in the end, you're storing your data and you're running your apps in a place that may be more robust than your physical location.
04:39And one of the real advantages is that you can use their hardware if you need to scale out, for example.
04:46If you temporarily need extra machines, you can configure your site so that it adds machines for a certain period of time...
04:53...and then once a certain threshold of inactivity is released, then the machines are deleted or removed from the site.
04:59Then you no longer pay for those machines.
05:02Some of the advantages of this may become obvious to you as you think about it.
05:08We already talked about the infrastructure and elastic deployments, but think about the advantage of having...
05:14...the cloud like this for prototyping.
05:16Suppose you're going to develop an app and you need to have it run on a machine with a certain operating system...
05:23...in a certain size and spec for just a period of time while you prototype.
05:29Amazon cloud can be a way to quickly get that machine available to you just for the amount of time that you need it...
05:36...without having to jump through hoops of, you know, filling out maybe an official purchase order...
05:41...or waiting for the hardware to arrive in the mail.
05:44People find that the cloud is very useful for this.
05:48Also, if you're in an emergency and you need to have server power right away...
05:53...having your app in the cloud makes it a lot easier to quickly add infrastructure.
05:59Some departments that deal with things like natural disasters or weather or news items...
06:06...have found out that the cloud is very helpful for just short periods of time, increasing the capacity of their site.
06:14When we talk about running ArcGIS Server on Amazon, there's two key terms that you need to understand...
06:21...and often we can get these mixed up if we're not careful.
06:25I may even do it in this presentation due to nerves, but I hope to define these two things...
06:30...and help you understand them now so that the things you see throughout will make a little more sense.
06:36An EC2 instance is a virtual machine that runs in the Amazon cloud.
06:42And you can pick out the size and spec of this instance, and I'll show that process a little bit later on.
06:47So EC2 instance is what has ArcGIS Server running on it. An Amazon Machine Image...
06:53...or AMI, or A-M-I, or ay-me or am-ee [phonetic], as I've heard it called this week; pick which one you like.
07:01That defines the operating system and software that gets applied to your instance when you create it.
07:07So it is literally an image that's used to create the instance.
07:12Now what does it mean to say that ArcGIS Server is supported on Amazon EC2?
07:18What it means is that Esri has developed these Amazon Machine Images...
07:23...and is willing to share them with you to use on the Amazon cloud.
07:29When you develop an Amazon Machine Image, you can choose to make it available to everybody - the public...
07:35...or you can choose to share it with certain individuals.
07:38In the case of the Esri AMIs, Esri must share those with your Amazon account in order for you to use them.
07:46So you must talk to customer service or your international distributor and let them know your Amazon account number...
07:52...so that they can share the AMIs with you.
07:54Now, there are three AMIs available at version 10.1 - ArcGIS Server with PostgreSQL running on Ubuntu Linux.
08:04We supported Linux for the first time at 10.1.
08:08Other one is ArcGIS Server with SQL Server Standard running on Windows, or you can go with SQL Server Express.
08:16You can also mix and match these AMIs as need be...
08:20...so if you want to run ArcGIS Server on its own instance and have SQL Server Standard running on a separate one...
08:27...you could launch two instances and then disable ArcGIS Server on one so you just have the database remaining.
08:33And then you'd have two instances - you'd have the server separated from the database.
08:37The disadvantage of that is you'd have to pay for two instances running...
08:40...but the advantage is you wouldn't be competing for resources among those.
08:45You wouldn't be competing for resources like you would if both the server and the database were on the same machine.
08:50So the point of what I'm saying is you can use these AMIs in any combination that works to build your site.
09:00The way that you create an ArcGIS Server site on Amazon is to use the Cloud Builder application...
09:06...that is being developed at 10.1, which I will show.
09:10This is a graphical application that you - just has a wizard and you click through it and it makes a site for you.
09:15You can even use it to set up automatic scaling rules and other parameters for your site.
09:22A more advanced way is to use the AWS Management Console.
09:26That is a web application developed by Amazon for the purpose of administering your resources on Amazon cloud.
09:33Before 10.1, AWS Management Console is the way that you work with ArcGIS Server on Amazon.
09:39So if some of you go and try this out, like, tomorrow with 10.0, you would need to use the AWS Management Console.
09:47I will show that console also briefly during the demo.
09:53Amazon EC2 is driven through an API that you can download and use to script certain actions.
10:00This is very helpful for saving money.
10:04Every time your instance is turned off or stopped, it's not accruing charges that you are...it's not accruing charges for running.
10:12So a lot of people like to script their instance to stop it at the end of the day or at the end of the week...
10:19...and then they run a script on Monday morning to start it back up so that they're not accruing charges over the weekend.
10:25There're other useful things that you can do with scripting, such as setting up detailed autoscaling parameters...
10:31...that you might be interested in.
10:33And then other third parties have developed tools for working with Amazon.
10:36A lot of people like to use ElasticBox. That was around even before Amazon built their own Management Console GUI.
10:43So you might be interested in exploring those.
10:48Let me cut out to show Cloud Builder right now.
10:55This is what Cloud Builder looks like; it's very simple layout.
11:01You will download Cloud Builder from the same place on Esri Customer Care site where you download ArcGIS Server...
11:08...so look for it there among the available downloads.
11:13If you purchased ArcGIS Server, there's no extra cost for purchasing Cloud Builder and setting it up.
11:19Cloud Builder is an Adobe AIR-based application.
11:23When you install it, you'll be prompted to install AIR if you don't already have it.
11:29And then it looks like this.
11:31The first thing you'll see is a login screen where you'll be prompted to enter your Amazon access key and your secret key.
11:39These are credentials associated with your Amazon account, and all applications that use the Amazon APIs...
11:45...require you to submit these keys so that they know which account is working with the application.
11:52I will show you a little bit later on where you can view those keys.
11:55You can view them in the AWS Management Console.
11:59Once you've typed in your keys, you can choose, if you want, to have Cloud Builder just remember those...
12:04...and then you don't have to type them every time you open the app, or you can require a login each time.
12:10Requiring a login is helpful if you're going to use multiple Amazon accounts with the same Cloud Builder application.
12:17So I have already logged in, and I've actually created a site on here already, but I'll show you the process of creating a new site.
12:24You just click this Add Site button up at the top.
12:28I'm going to switch to this mic while I do the demo.
12:39So you could just type a name for your site and - what's today? So this is my site for Tuesday.
12:52And type a description, and then you choose the operating system that you want, so this is the point...
13:00...where if you wanted to go with Linux, you could do that, or Windows, or the third option is to use your own template.
13:06So this is a way that you kind of, you can use a custom image to build your site.
13:12Once you've gone and built your site, you can use Cloud Builder to go back and save that site as a template...
13:17...and then later on, you can launch new sites using the template.
13:20And this is helpful if you want to set up your own services and applications just once and then launch...
13:25...multiple sites that have all those services ready to go, so be sure to note the template option if you use Cloud Builder.
13:34For this demo, I'll just choose Windows.
13:37And then you will be prompted to browse to a license file.
13:39Now, when Cloud Builder makes your site, I just browse to an ECP file...
13:45...which you could get from working with customer service if you needed to.
13:50When you create your site, it's going to license the site and start it up automatically...
13:54...so that's why you have to provide this file right here.
14:00Also, the options that you will see in Cloud Builder are tied to this license file, so actually...
14:05...right now it's looking at that file, and it's going to detect that it's an enterprise license, so it's going to give me...
14:10...more options, such as being able to autoscale and run ArcGIS Server on multiple machines, or EC2 instances.
14:19You must license with Esri licenses any machine or any software that you cause to run inside of ArcGIS.
14:29Sorry. You must license any software that you cause to run inside of Amazon EC2.
14:36At this point, you will choose which region that you want to run in.
14:38So Amazon has set up data centers throughout the world.
14:42In this drop-down, you'll see what they are.
14:47I'm just going to choose the Virginia data center. That was the first one; it's one of their largest data centers.
14:52But they've recently brought new ones online in Oregon and Brazil.
14:57If you're working with 10.0, we do not have images yet for those new regions...
15:01...but they will typically become available at the next service pack.
15:05So when the images come out for service pack 4, we should have those in the Oregon and Brazil regions just a few weeks later.
15:14You can also choose an availability zone if you want.
15:17Within each region, or each data center complex, Amazon has isolated certain areas of hardware...
15:24...and that's what the availability zones are.
15:26So this is just a way of building more failsafe techniques into your site is to spread out the instances among zones.
15:36If you leave No Preference, it will just choose a zone for you.
15:40Now you choose the instance type that you want to launch within your site.
15:44So this is just pulling instance information from Amazon, and this is where you might start to salivate...
15:50...at the large machines that are available to you, all the computing power.
15:54Remember that the choice you make here affects your bill, how much you pay.
16:00Some of these large instance types are much more expensive than the smaller ones.
16:04You should go to Amazon's pricing website to clearly understand the prices and also the specs for these instances.
16:13Amazon Web Services has very good charts on their website for understanding those things.
16:19For my demo, I'll just choose the large instance.
16:21We are working on certifying the medium instance that just became available.
16:25If you're watching Amazon closely, you'll know that they released a lower-priced type of medium instance...
16:31...and we are working on certifying ArcGIS Server on that.
16:36We feel like it's a promising option for perhaps development purposes...
16:41...but the large instance is probably what you would want to go with in a production environment...
16:44...just for the amount of memory that's available and the amount of - and the specs of the machine.
16:52A key pair is a file that you will use if you want to remote desktop into your instance.
16:57To get into an instance, you need to have the administrator password, and Amazon is only going to give that to you...
17:04...if you have a key pair file that you have saved to your machine that's associated with your account.
17:09So you can have Cloud Builder create a key pair for you.
17:12It will place this in the My Documents folder, and you can go take it from there and put it in a more secure location.
17:18But you will need this file if you want to remote desktop into any of the instances that Amazon creates for you.
17:26On the next panel, you will choose whether you want to include an enterprise geodatabase in your site.
17:32So if you check that box, more options become available for SQL Server Express or SQL Server Standard.
17:38And as I mentioned earlier, there is an option to put the database on its own EC2 instance, and if you choose that...
17:46...it will wire it up all for you so that ArcGIS Server is running on one...
17:52...and SQL Server Standard is running on the other and they are connected.
17:58If you choose to have the database on its own instance, you will also get to choose the instance type for the database...
18:03...so you can potentially put the database on a very large machine or a smaller machine.
18:09I see... Is that a quick question?
18:12[Inaudible audience question]
18:19Your licensing at that point...at that point, you're licensing the ArcGIS Server software.
18:26And you know the answer to that? You need a separate license for the database?
18:29Yes, you do. Okay. Thank you. That was an easy answer.
18:36Alright. The number of EC2 instances is how many machines will be launched in your site.
18:42So the default is just one, which is good for just keeping your costs under control.
18:48If you plan on scaling out your site or you're ready to launch a site with lots of computing power...
18:55...you will check this box to enable automatic scaling.
19:01You can actually increase this number without enabling autoscaling. I misspoke. So you could raise this to 2 or 3.
19:08If you want to expand dynamically in response to incoming traffic, you could enable autoscaling...
19:14...and then set a maximum ceiling on the number of machines that would be added into your site.
19:19So I could have, for example, a minimum of one instance and a maximum of three.
19:24Now, what determines whether a machine is added into your site?
19:28You will set triggers down here on the amount of CPU usage coming in to your instances or that are being done by your instances.
19:37For example, when CPU usage reaches 80 percent for five minutes, it will add another instance to your site.
19:45If usage goes below 20 percent for five minutes, it will remove an instance.
19:51And you can adjust these triggers the way that you want.
19:56Remember, again, that whatever instances are created...
20:00...you'll receive a bill from Amazon for that amount of time that it took to run that instance.
20:06Each partial instance hour is charged as a full hour, so you want to be careful that you don't set thresholds...
20:11...that cause your instance to go away and then come back for five minutes and then go away and then come back.
20:16If it's too sensitive, you'll wind up being charged for a lot of those full instance hours over time.
20:24One thing that you do as you proceed is you'll need to set up a primary site administrator account for ArcGIS Server.
20:31Now this is new at 10.1, so I'll take a minute to explain what this is.
20:36When you launch any site in ArcGIS Server 10.1, whether on the cloud or on premises...
20:43...security is enabled and there is one account that initially has access to everything; it's like the superuser...
20:50...and that is the primary site administrator account.
20:53Now, you define this in order to create the site.
20:56Later on, you can choose to disable that account or create other administrators or other users, but in order to launch a site...
21:03...you will need to provide a name and password that you make up that will be the initial, or primary, site administrator.
21:15You can also choose at this time whether an SSL certificate should be applied to your site when you create it.
21:21The SSL certificate needs to already be uploaded and exist on Amazon, and if it detects one associated with your account...
21:29...you will be able to pick it from the drop-down here.
21:32For this demo, I'm not going to do that part.
21:35I've reached the end of the wizard at this point.
21:37I see a summary of everything that's going to be created, and I can just click Finish, and it will start doing a lot of things.
21:44It's using the Amazon APIs in the background to launch this site.
21:50It's setting up all the things that I need with the site, including elastic load balancers, EC2 instances...
21:58...all of the disk drives that are associated with these, and everything else.
22:05If you click this In Progress link, it will actually give you some detailed messages about what's going on...
22:11...if you're curious about what things are being created.
22:14Be aware that, from this point on, you will begin to accrue charges...
22:16...for the things that are created in Amazon EC2 that are charged.
22:23So this one is going to spin for a little while, and I may not even finish creating it during the short time I have for this demo...
22:30...so I've created a site already, and I'm going to show you what it looks like when it's done.
22:35That's this one down here - DevSummit Workshop.
22:38So when you're done creating a site, you'll have the option to back it up...
22:43...which you can restore in case anything happens to your site; you can just restore from backup.
22:48You can stop your site, which is akin to just pushing the power button.
22:52It doesn't delete your site forever; it just stops it from running and stops it from being available to end users.
22:58This is a good way to cut costs if you know that you're not going to be using your server.
23:02You can update some of the properties of your site, or you could delete it...
23:07...and deleting it does make it go away forever, and it does delete all of the items that it created.
23:16It gives you a link here that you can use to open ArcGIS Server Manager for your site...
23:21...so that's how you can go ahead and get started is to click this link and open Manager.
23:34So when you look at this, you'll see the address of my site up in the top...
23:38...and you should see a login page for Manager, if we're fortunate here.
23:44One thing that you'll notice in the URL is that it contains the address of the elastic load balancer.
23:51And a load balancer is an item that you can create inside of Amazon EC2 that allows you to distribute traffic to multiple instances.
24:00And so if you have a GIS server site with three GIS servers in it...
24:04...the load balancer will distribute the traffic to those instances.
24:07So you'll notice that that is one thing that's configured for you.
24:11Now I'm into Manager and I could log in to it at this time if I wanted to.
24:17I will actually come back to this a little bit later.
24:20I also want to make the point that you can connect in Desktop.
24:23So let me copy this URL. I'm going to go into the Catalog window of ArcMap where I have connections to ArcGIS Server.
24:35Now, I have this one, which is my connection to this local laptop.
24:39I also want to add a connection now to that cloud site that I just created, so I'll add an ArcGIS server...
24:47...and notice at 10.1, you get an extra option in here that has publisher-level permissions.
24:52So that's kind of a nice goody in 10.1.
24:54You used to just have these two - use or administer.
24:58I'll choose Publish, but the name and password I am going to type in is that primary site administrator...
25:04...'cause right now, that's the only account that I've made on that site.
25:09The URL is just the same as the manager URL except I'll remove the manager part...
25:15...and then I can type a name and a password of somebody who has publisher privileges.
25:19That would include this admin that I created.
25:23And this should connect me to ArcGIS Server.
25:26Now, this is handy if you don't have physical access to log in to the server...
25:30...or if you don't want to remote desktop into your server and go inside of there.
25:36For example, if I created a Linux site, I could connect to it this way and work with it.
25:41I don't know - I'm not a Linux guru; I don't know how to SSHN and type whatever commands I need to do.
25:48But I could work with it this way and start and stop services and even publish. So I will do that now.
25:54This is a very basic site. It doesn't have any services running on it other than the default services that are launched with 10.1...
26:02...but I have this map here of Spain that I might want to go ahead and publish to the server.
26:08And ArcGIS Server 10.1, you can choose to have data copied to the server at the time that you publish.
26:14The data is actually clipped out of your map, packaged up, and copied up and uploaded to the server...
26:20...and that can be very useful in cloud deployments.
26:22You might remember that, seeing that in the Plenary Session.
26:32I probably made it think too much. Okay.
26:35So you will choose File > Share As > Service, and you choose to publish...
26:41...and you'll see a dialog that allows you to pick the server connection you want to use...
26:45...and so you'll need to be careful to pick your Amazon one.
26:49It's going to show all the connections that you have down here.
27:02So what it's doing now is it's just creating the basic properties of the service that I will need...
27:07...and now it allows me to set some of those detailed parameters.
27:11If I wanted to set up caching and whatnot, I could do that.
27:14I could even set it up so that it starts caching automatically once I publish the service.
27:19That's another extra goody that's available in 10.1.
27:23In my case, I'm just going to choose to go ahead and publish, and at this point, it's going to package up all this data from Spain...
27:31...and upload it to the server and deploy the service.
27:34So we should be able to come back in a little bit and see that available.
27:40Oh, and it, before you copy anything to the server, it gives you fair warning.
27:46You don't want to do this with datasets that are excessively large.
27:49If you have very large datasets that are many gigabytes in size, there's other techniques to get that data up to Amazon...
27:55...but this could be a convenient way if your data is relatively small and contained.
28:00So I'll just accept the warnings about this being copied.
28:03And you'll see the messages that are coming across, that packaging is in progress and so on.
28:09While that's happening, let me show you what's going on behind the scenes in the AWS Management Console.
28:15So if you work with 10.0 or if you want to get a more detailed look at what was created...
28:19...you can log in to this site at aws.amazon.com/console.
28:25You type in your Amazon account credentials and you get to this page.
28:29And if you click EC2, you can see all your stuff that's running inside of the Elastic Compute Cloud.
28:37Earlier on, I mentioned that you have a secret key that you need to type in in order to log in to Cloud Builder.
28:45The way that you can view that is here if you look at your security credentials.
28:49So this'll bring up a separate page where you can view your access key and your secret key.
28:56So you'll just sign in at - it will require you to sign in again, just as an extra security measure...
29:04...and then you can go down and see those.
29:05So you have your access key, and I'm not going to show the secret access key in case we're being recorded...
29:11...but you can display it here and then copy it and paste it into Cloud Builder in order to log in.
29:16I wanted to make sure I showed that part.
29:18Now, if you look at instances, you'll see the instances that I have running on this account...
29:23...and you'll notice that there's an instance that's associated with each site that I showed in Cloud Builder.
29:29Now, if I had chosen to have the geodatabase on a separate instance, you would see an additional one here that had been created.
29:36So I can see my credentials here.
29:40If I needed to log in to this machine through Remote Desktop, I would need to right-click and choose Get Windows Password...
29:48...and then you go browse to that key pair file that was saved when you ran in Cloud Builder.
29:53So if I go to My Documents and this ArcGIS Cloud Builder folder...
30:00...you'll see all the key pairs that have been created as I've run through this wizard over the past few days.
30:04And here's the one I just created.
30:07So when I click Decrypt Password, it will show me the administrator password of that machine...
30:13...and then I could use Windows Remote Desktop to log in.
30:18In the interest of time, I'm not going to do that.
30:21You'll notice that there's a Volumes tab. This also shows the disk drives that are associated with your machine.
30:27You'll get one that's 35 gigabytes that has the C drive, has the software installed.
30:35You also get a D drive attached that's for your data; that's going to be 100 gigabytes.
30:40If you want, you can detach that 100-gigabyte drive and attach a larger one or a smaller one.
30:45In my case, it's 10 gigabytes just because for development purposes, we took the amount down to reduce the cost...
30:51...but when these go final, this will be a hundred-gigabyte drive.
30:58Let's go back and see... It says that my service published successfully...
31:02...and so I want to log in to Manager on that machine and take a look.
31:08So again, this is going to Virginia, and it sees this Spain basemap that I just published...
31:14...and now anybody can interact with this service on there.
31:25So just to recap the ways that you can get data onto your instance.
31:28This might be the trickiest part of working with the cloud, especially if you have large amounts of data.
31:33One way is you can remote desktop in and then copy and paste the data across or use FTP.
31:39Or you can copy the data automatically when publishing a service out of ArcMap...
31:43...which is what I just - the option I just showed.
31:46There's some more advanced cases where you could put data into a storage service like Amazon S3.
31:51I think Marwa may delve into that later on, but she's going to go into some of the use cases now...
31:57...and advanced cases and also some of the business considerations for Amazon.
32:01So I'll hand it over to you, Marwa.
32:03Thank you, Sterling. That was a great demo, wasn't it?
32:06Don't know if you noticed, but he actually didn't log in to the machine at all.
32:11He did all of that without having to log in to the machine on Amazon.
32:14He did all of this by just using Cloud Builder and Desktop. So thank you.
32:25So now that you've kind of got a good overview of how you can quickly deploy Server, run it...
32:31...and publish services to the cloud with Server...
32:35...let's kind of go a little bit deeper into what are the different models people use.
32:38And we're going to kind of take one of them that's more of an advanced case...
32:43...and get into the details of how to implement that.
32:46So starting with cloud deployment examples...
32:56I wanted to use an example, one of our customers, Texas DOT.
33:00Texas had a situation where there were a number of storms that come every now and then...
33:05...and during that time, the road conditions application became very critical for their citizens.
33:10What Texas did is they wanted to make sure during that time the application can scale to meet all the different demands...
33:16...that come in at that time, and then, after that time is over...
33:20...bring back the application to a very small footprint that is suitable for the daily use at that time.
33:26The cloud met that use case explicitly well.
33:31You've seen the model for autoscaling. That works really well when you're trying to publish an application.
33:38Also another example is, typically, if you want to do some testing.
33:43So you are developing a new application, you have a new version, you've made some changes...
33:48...and you want to test that before you push that for somebody else to start using.
33:53The cloud is a very great example for that. Why?
33:56Because you're not affecting the existing systems are running.
33:59You're actually making the modification in a different system that you can keep as many versions of it as you need.
34:05And if you need to roll back, you always can create an Amazon Machine Image and use that image that you've rolled back to.
34:13In addition to that, there's the model for geoprocessing where users would be able to span a number of machines...
34:21...that they need to use for a temporary period of time, and then when they're done, terminate them.
34:26So the type of processing you want to do, the heavy processing, can be done with a bigger number of machines...
34:32...that you would have typically utilized your on-premises resources for.
34:37And similar to that is the concept of cache cooking.
34:40So if you want to cook a cache, typically what most people do today is they allocate a number of machines on premises...
34:46...on their own network, and then they try to be patient to see how long this will take until the end.
34:52Well, another alternative is using the cloud and adding more machines as needed.
34:56So you can span your cache cooking to happen a lot faster, and then maybe, based on that...
35:01...you can plan updates as you have more control of the resources and you can control adding to them as needed.
35:07And when you're done with your cache cooking, you terminate all these machines...
35:10...and you only pay for the number of hours you have run them.
35:15So there's a number of concepts I wanted to reiterate before I start showing you the showcase we've chosen...
35:21...which is the cache cooking, and how to do that in the cloud using ArcGIS Server on Amazon.
35:28An important concept is the concept of custom AMIs, so that's when you've customized your own instance...
35:34...so just what Sterling has done earlier. He's uploaded his service, and now the data is running there along with the service.
35:41He could have customized the instance a little bit further by adding his own application...
35:46...but once you've had all of that, you can create an image, so next time you start a machine from that image...
35:52...you don't have to go and customize it again.
35:54In fact, it's not only going to have ArcGIS Server, but it's also going to have your data, your services, and your applications.
36:01And from a productivity perspective, that machine is going to be around.
36:05Anytime you need to terminate that machine that is running, for any problem, you can launch another one from that custom AMI.
36:14Also, another important concept is security.
36:17So it's important to understand security groups in the cloud...
36:20...because this is a very important definition and configuration that you do whenever you launch an instance.
36:25A security group, in a sense, is an overlay in the network, so you're basically taking a slice of the network...
36:32...and you're isolating it for the machines that are running within this security group.
36:37In addition to having the security group isolation on the network level, each of the machines also has a Windows firewall.
36:45And this Windows firewall gives you another kind of line of defense that is specific to the machine...
36:51...so in fact, you can deny anybody from accessing that one machine except yourself based on specific rules...
36:58...like maybe tying your port to a certain IP address and so on.
37:04So a typical example is setting up an enterprise geodatabase with the server.
37:08A best practice is to associate an elastic IP, which is an IP address that will not change...
37:16...attach that to the enterprise geodatabase, and configure Server to always point to that.
37:21Now, when Server points to that IP, it means the instance underneath the IP could be changed.
37:28So maybe you terminate this instance, start a new one, and replace it under that IP.
37:33Server will always see it, so you can exchange these servers and have no problems.
37:38In addition, you can configure the security groups to have more control on specific tiers of your deployment.
37:45So for the database, you typically don't publish anything on the web...
37:50...so you want to limit the access to just the ports that are utilized for database access...
37:57...while in the case of the web server where your ArcGIS Server is running...
38:01...you probably will publish services and will publish applications, you need something like port 80 open to everyone.
38:08So you can separate both tiers using these security group settings...
38:12...and then you can access each security group by adding it to the other.
38:17So as you can see in this diagram, I have added web to my database security group setting...
38:23...and this allows me to access the database from the server.
38:28And using this security group setting, I have secured my web tier; I've also secured my database...
38:34...and I've made it very specific that only specific ports are open to certain IPs based on my need.
38:43Also, another important concept - and Sterling has gone through that - is the concept of storage.
38:48There are different types of storage in Amazon. One is the S3 and also Elastic Block Storage.
38:56What's different about them is how they're accessed.
38:59The Elastic Block Storage is very much like your local disk. The difference is you can attach it and detach it, you can resize it.
39:07Usually, that disk is similar to an NFS that you're accessing over the network.
39:13And I'm going to show you some tips and tricks later on in my demo...
39:17...on how to get better performance that makes this EBS drive similar to a local disk in performance.
39:25Now, Sterling has gone through a very good way to upload your data and publish your service by using ArcGIS Desktop.
39:32But what if you have a lot of data? What would be the alternative?
39:37One of the good alternatives is a service that's provided by Amazon that's called import/export.
39:42This service is basically, you put your data on a disk and you ship it to Amazon...
39:47...and they receive it and then they put the data on S3 or an EBS drive.
39:52And yes, it's going through the mail.
39:54However, it happens to be a lot faster than if you went over the Internet...
39:58...'cause the Internet bandwidth would take forever to migrate something like a few terabytes.
40:03But in using import/export, you can have that uploaded in a few hours.
40:13Now also Sterling has mentioned the different utilities you can use.
40:17So in the model that I'm going to talk about next, I'm going to show you how to set up a system of ArcGIS Server...
40:25...that includes multiple instances.
40:27This system will all see each other, and it will all collaborate together to build a cache.
40:33Now, the system, each of the instances includes an EBS drive, and the data, I will upload it because it's large in size.
40:42I will upload it and make sure it is available on every instance.
40:47Now, the way you upload it, like I said, you can use import/export, but you can sometimes also upload it directly yourself...
40:54...either using FTP or doing your own upload through S3 as an intermediate step.
41:01And once you have uploaded your data, you can create a snapshot.
41:05So a snapshot is equivalent to the concept of a machine image, but it's only specific to the disk.
41:13So I can create a copy of my disk, and then from that copy, I can create multiple disks.
41:19It's a very handy feature if I am trying to copy my data and make it available across multiple machines.
41:29So that said, to summarize the steps I'm going to go through in the demo, because it's going to take a little bit of time.
41:36And it's a process that's quite an advanced use case...
41:39...but I thought it's really worth sharing with you, since this is our developer community.
41:45First, the data would be uploaded.
41:48The way I uploaded it, I used S3. So I created a bucket in S3, I uploaded the data to that from my local machine...
41:57...and once it was in S3, I created an instance and I copied it to S3.
42:02Now, this instance was just a regular instance; it didn't have any ArcGIS Server.
42:07But once I uploaded the data, I created something called a snapshot, which is that copy of my drive.
42:14So next, that copy is now available; I can go and set up ArcGIS Server and use that copy of my data.
42:20So I'm going to show how I can create my ArcGIS Server site using Cloud Builder...
42:27...and you've seen that already, so I'm not going to go into it in detail.
42:30But then once I have those machines, I'm going to show you how to stripe the disks...
42:35...and that's how you configure it to have better performance; also how to create the snapshot if you want to...
42:42...and from there, how to create a custom template, which is a footprint, or an image, of everything that I have done.
42:50Using that template and that image, I'm going to create the actual site that can cook my cache...
42:57...with some extra features, utilizing the cloud.
43:01So that said, let me get started. Cloud Builder, as you've seen - let me get closer.
43:16Cloud Builder, as you've seen, is an application that runs on my desktop.
43:30If I can print. Okay. Sorry, it looks like I'm having a bit of trouble here with my keyboard.
43:53So I have Cloud Builder installed. I'm going to open it, and like Sterling showed you...
44:00...that application has all the settings I need to start my site in the Amazon cloud.
44:09And I've already configured it with my own credentials, so once I access the application, I will see my already-created sites.
44:17Now, to give you kind of an intro to the demo...
44:20...not because it's cache cooking but because of the time it takes to prepare the systems for every step...
44:26...I kind of took a leaf from cooking shows, and I'm going to show you how things are put together...
44:33...but then show you the end result for the sake of time.
44:36So I'll start by creating a new site, and this new site will allow me to create what I call the template.
44:43I want to build this image that I can use at any time to cook my cache from now on.
44:49So I'll call this the CacheBase.
44:56And I'm going to base this on Windows, I'm going to supply it with my own license, and I'm not going to add any database.
45:08You can add it if you want your data, your source data, for cache cooking to be in a database.
45:13You can utilize the database option, but in this case, my data, even though it's a few gigabytes in size...
45:20...it's all file based, and that's going to be the base I'm going to work on.
45:26So the next step here is going to be based on how my server is authorized.
45:31So now, using 10.1 with Cloud Builder, you don't have to log in to the machine anymore to authorize it.
45:41And I'm going to specify all the information I need to get started.
45:48And like I said, I'm not going to pick a geodatabase. I'm going to start with two instances.
45:54So if you're trying to build a template, you need to start with two instances minimum; you shouldn't start just one.
46:01And I'm going to leave out autoscaling at this stage.
46:04I'm going to provide my administrator's account for the web management tool and say Finish.
46:15Now, while my CacheBase is being created, which I can see here...
46:20...I actually went ahead and created a site that's called CloudCache that had exactly the same settings.
46:27So to show you what's in the site, I'm going to click on the Administrator, and once I get in there...
46:41...I can see in this site what kind of machines do I have - so I can see I actually have two instances already running.
46:48If you recall, I chose two. If you want to make a template, you have to start with two.
46:53And each of them is running; it's started. Everything looks good.
46:58So once I've started these two machines, the next step is to go to the Amazon console, so I'm going to switch back to that.
47:19And over here, I'll refresh my page.
47:28And what I'm going to do here is based on that snapshot that I have created previously...
47:34...I'm going to create the new drives that I would like to be part of my setup.
47:40But before I do that, I just wanted to show you how the striping works really quickly.
47:45So what I'm going to do is I'm going to create my own volumes and attach them to the instance I just started.
47:55Before I do that, the first important step is to know which zone the instance is in. I'm having trouble refreshing my page.
48:08So I'm going to find out which zone. Okay. Let me log in again.
48:30There we go. I do have instances running. And I'm going to say All Instances; and there they are.
48:43Sorry, my resolution changed, and it's changing a lot of things on my machine.
48:48So the machine that I've created is this one; let me make sure this is the one you see.
49:00This is the two machines that I'm looking at, and what I would like to do is understand which region I can see here.
49:08This is USD - us-east-1d, so I'm going to go and create two volumes...
49:18...and I'll give them about 1 gigabyte, but you can make it as big as 1 terabyte.
49:25So if you have a lot of data, you can make this as big as you would like it to be.
49:31And once I have these two volumes created, I'm just going to attach them.
49:44So here are my two. I'm going to attach them to the machine that I have, which I can see it here, and my second one.
49:57I'm doing this twice in case you missed the first.
50:13So now that my instances are attached, the next step is to log in to my instance.
50:18So to log in to my instance, I will use exactly the same approach that Sterling showed you earlier.
50:25I actually had logged in earlier to it for the sake of time, so here is my instance.
50:32And let's go and check my Server Manager to see these volumes that I've just added.
50:37So this is what they look like when you first add them.
50:41What I'm going to do first is bring them online, and then next, I am going to initialize the disk...
50:51...and I'm going to specify it as a GUID partition table.
50:55Now, the reason I'm doing this is that I can maintain whatever striping I have when I do a snapshot of that.
51:03And you can see here that I have both disks identified.
51:09So now that both disks are configured, the next thing is to configure it as a new striped volume...
51:14...so I'm going to say I'm adding both of these disks to be this one volume.
51:20And you could specify if you want to break it down into two volumes so they will look like two disks...
51:25...or if you want it all to look like one big drive.
51:29So I want them together. I'm going to see them together as one drive of 2 gigabytes.
51:35And I'm going to assign this drive the letter M and go with the rest of the settings.
51:42Again, what's important about this is that I will get a boost in performance when I stripe the disks like that.
51:48That'll make it equivalent to what a local disk would give you.
51:52This is a very good way, if you're trying to publish dynamic services that require high I/O performance.
51:58Good example is image services.
52:01So image services rely on very good I/O because of the size of the raster data and how they work, and you can...
52:07...utilize this type of striping of the disk to give you that kind of performance when you're publishing image services.
52:16Now, I've already created a drive that has four striped disks.
52:21The reason I did four, not because I like to repeat it, but because four is kind of the magic number.
52:28That is really how you get the best performance out of striping.
52:31So if you have a certain size like 20 terabytes, try to break it into four and then stripe the four.
52:39Sorry; 20 gigabytes, not terabytes, 'cause you can't go above a terabyte for each.
52:46So now that I've configured my disks, I'm going to do this for every instance.
52:50And I've already done that, but I wanted to make sure you are aware the data needs to be then copied to that disk that I have striped.
52:58So I have done that already on what I call the striped volume, and if I go and explore that...
53:03...you can see that I already have my data.
53:06And if you recall, we created two instances, not just one, and I will copy the data to each one of these instances.
53:15And now that I've done that, I've got nothing else to do except publish the service.
53:22So the way I'm going to publish the service is I'm going to use my local desktop.
53:26So what Sterling showed you was accessing Server by using Desktop from his own local machine...
53:33...but what I'm starting here is the Desktop that is running on the Amazon instance.
53:39And I'm going to use that striped disk that I just used and copied my data to, to use it to access the data.
53:48I am also going to add to my server.
53:55So I'm going to add my server, it's localhost, and I'm also going to add the account...
54:08...so now I can manage my server from ArcGIS Desktop that's on the same instance.
54:14And the first thing I'm going to do is I'm going to set server properties.
54:18So because I'm moving my data and it's a large dataset, I don't want to copy it from my machine; it's already on my machine.
54:25I'm just going to register a folder that points to where my data is.
54:29So here's that striped volume that I created. I'm going to give it the name of the drive.
54:36I'm going to register that with Server.
54:37So what Server is learning here is that it will not copy the data; it will use that location that I have on my local machine.
54:45And that is why I needed to have the data on both of my instances, not just one.
54:51Server will not do any copying of the data, because I know it is too large to copy.
54:56I want it to be efficient; I want to do it myself using the best tools out there...
55:00...and then I'm just configuring Server to point to where I copied it.
55:05And once I've done that, the next step is to just basically go and publish it.
55:10In this case, when I'm publishing it, what's getting published is basically the service definition...
55:17...so that is what's getting added now that both Desktop and Server know the exact location of my service.
55:25I'm going to preview just to make sure I don't have any problems before I publish.
55:31And the next step is publish.
55:34And this is going to take a quick moment because I am creating this service definition file, moving it to the server...
55:40...and the server now is going to publish this as a service by using the location that I have created locally.
55:47And this is the best approach to get the best performance of very large datasets when you have them in the cloud.
55:55Once I've done all of that, I've published my service, this is going to be the base service that I am going to use to create my cache.
56:03I've configured Server the best way it can be to get the best performance of the disk.
56:08There's nothing left; I can go and create my image.
56:11There's one quick step I'd like to advise you to do, which is to go and rename your administrator account.
56:20So by setting the password on your administrator account, you're going to be able to access this as part of your image.
56:27This is important when you're creating imaging, because of the way security works in Amazon.
56:32And with that said, I have finished everything that I need to do.
56:36I am just going to go back to Cloud Builder and, based on this, go and create a template.
56:44So the way I create a template is by using the template manager.
56:48And as you see, I've already been playing around with this. I have a couple of them.
56:52But what I'm going to do is I'm going to create a new one...
56:55...and it's going to be based on the cloud cache setup; that's the one we were looking at.
57:00And I'm going to give it a name, which we said was the CacheBase, and say Finish.
57:11Now, in a few moments, Cloud Builder is going to create that image, so next time I start from this image...
57:17...I do not have to configure all these things ever again. It will do everything on my behalf.
57:22Now, as you see, this happened very fast, but typically, what's happening behind the scenes...
57:27...is we're creating the Amazon Machine Images, we're creating copies of the data...
57:31...we're creating bits and pieces of all sorts of things that you will need to re-create this site as efficiently as possible.
57:39So next time, I want to go and say, I would like to create the site, I will just go and start from that template.
57:52So while all of this is kind of finishing, I will come back and show you how I can use this template...
57:59...to create my final cache cooking established site and how autoscaling will play a part of that.
58:07So while all of this is getting finished, I'm going to go back to my slides...
58:10...and discuss some of the best practices that are important to be aware of as well.
58:28So some of the best practices in Amazon don't just have to do with getting performance but also with pricing.
58:34If you're not aware of the cost of the components you're using...
58:37...it's going to be very hard for you to calculate the cost of the overall setup that you've just done.
58:43And that setup that I've just made is not intended just to give you the best performance of the disk...
58:48...but also to allow you to start machines when you need them and terminate them when you don't need them...
58:53...'cause cache cooking is a resource-intensive type of operation.
58:57So over here, just looking a little bit quickly over the hourly rate that Amazon typically applies to the instance...
59:05...you can see here that I have some examples based on the smaller sized instances.
59:10A large instance for Linux would be $32 per hour and 46 cents per hour for Windows.
59:17And these are the latest and greatest prices.
59:20Amazon has actually been reducing their prices every now and then...
59:24...so it's always good to go and check the price on their website.
59:27Also, there is the reserved model...
59:30...where if you know you're going to run a machine for a certain period of time all the time, you can get a discount.
59:36So this is like a bulk discount type of thing.
59:40And this depends on how much you're going to utilize this machine, so depending on the utilization...
59:44...it can go under light or medium or high, and you can see here, the rates that you pay hourly will vary differently...
59:53...greatly based on that.
59:55And the way to calculate the cost for one machine is by understanding that hourly rate...
1:00:01...in addition to the gigabytes you are attaching to your machine.
1:00:05So as you create a disk and attach it to your instance, you are paying for each gigabyte.
1:00:11The rate you pay is about 10 cents per gigabyte, so it's important to know how many gigabytes you have on each machine.
1:00:17It's important to know, when you're done working with a volume, that it is deleted, so you're not charged for it.
1:00:24And it's important to kind of understand how to utilize them as best as can be.
1:00:27Like I said, try to always use it within the size you need; break it up into four pieces and stripe it to get the best performance.
1:00:35You don't pay any extra fee for striping; what you pay for is per the size of gigabyte.
1:00:41In addition to the hourly rate, which is the computing resources and memory, as well as the hard disk, which is the gigabyte size...
1:00:49...there's also a third part, which is the network cost.
1:00:52And this is usually referred to as the bandwidth - the bandwidth to get you over the network and out to the Internet.
1:00:59Now there are two types of data transfer in Amazon.
1:01:01All data transfer in, that's coming from the Internet, is free.
1:01:05What you actually pay for is what you send out to the Internet, and that is charged also per gigabyte...
1:01:12...and the first gigabyte per month is zero dollars, but after that, you start with 12 cents.
1:01:18And as you get bigger and bigger, the price goes a little bit lower for bulk discount.
1:01:25Also, an important factor to work with when you start your own Amazon web account is the concept of the Amazon reports...
1:01:32...and this is available to you once you create an account.
1:01:35It's very useful, because you can keep track of what you're being charged for what.
1:01:39If you go and check this tool every now and then, you'll see that it's getting up-to-date.
1:01:44And I think Amazon updates this on a daily basis; not too frequently...
1:01:48...but at least every day you get a good feel for what is happening.
1:01:51It's very recommended that you check these reports and compare them back to the cost estimates that you make...
1:01:57...to understand what you're paying and how to keep your costs under control.
1:02:02Also, the Amazon bill typically comes online.
1:02:05You can check that, see what you're paying.
1:02:07So in addition to your usage, the components you started, and how long they ran...
1:02:11...you can also check how many dollars you're paying for that, whether it's in cents or dollars...
1:02:16...and you can keep track of your costs every month and calculate if the costs are coming in accordance with your estimates...
1:02:23...or there's a kind of big disparity, and from there, start coming up with some practices to help you control this type of cost.
1:02:32I put an example here of an Amazon bill. I'm glad you have very big screens; usually this is a hard to look at slide.
1:02:40But you can see here the breakdown between the number of hours I have running and the cost for that...
1:02:46...the EBS drives, how many cents are being paid, the rates, as well as the overall size and the overall cost.
1:02:54And in the end, I get the full bulk amount.
1:02:57And this is an account that I share with other colleagues, and all of us work together starting our instances and so on.
1:03:06Also, a nice feature to keep an eye on, and this is very useful when you're planning the autoscaling.
1:03:12So autoscaling is very easy to set while you're using Cloud Builder...
1:03:16...but it's also a critical thing to know what exactly is the utilization that you're going to set in Cloud Builder.
1:03:23So to keep an eye on this utilization, CloudWatch is free.
1:03:28However, when Cloud Builder sets it up, we create the account that allows you to see a broken down value for CloudWatch...
1:03:35...and this is something you pay for.
1:03:38And you see the time frames your machine is working on...
1:03:43...and also the utilization of different resources across that time frame.
1:03:48As you can see here, these are some stats. You get this through the Amazon console.
1:03:52These stats show the CPU utilization, the network, the storage.
1:03:57You get a full idea of the parameters and how the utilization of the machines is working.
1:04:04Now let's switch quickly back to Cloud Builder before I finalize and see where we're at.
1:04:13So what I wanted to do is to see that all my templates are created and my site is done.
1:04:19So now I can go and create the actual site that I will use for my cache cooking.
1:04:25So to create this site, what I'm going to start with right now is the template that I have created.
1:04:32And I'm giving it a site, CacheCooking. And again, I'm going to use my license...
1:04:51...and again, my instances are going to get licensed without me needing to log in to them and copy my license.
1:04:58It's all happening behind the scenes on my behalf; I don't have to do that.
1:05:03So Cloud Builder's going over the Internet with my license information, all the way to Esri...
1:05:08...to confirm all of this is correct, to bring it back to me and make sure I can license my machines when I move forward.
1:05:15The license I'm using is a generic license; it's specific to Esri, but the license you will get...
1:05:21...you will need to keep track of the number of cores that it works on.
1:05:25If you have a license for four cores, make sure you don't start machines that, in total, have more than four virtual cores.
1:05:34So over here, I could see here are the different versions of the server templates.
1:05:41The one I just created is the CacheBase. I could stick to the size large, and I can select my key pair.
1:05:49Moving on, because I cooked everything without a database, I'm not prompted to select that.
1:05:56And over here, I can start with one EC2 instance, but this is the part where utilizing the cloud capabilities for cache cooking...
1:06:05...to make things a lot more efficient, comes in.
1:06:07And that's the autoscaling. That's where I can use a criterion, which in this case, is the CPU utilization...
1:06:14...to control how many machines I can automatically add to my ArcGIS Server that is running.
1:06:21And by adding more machines automatically, the cache can get cooked much faster.
1:06:26So in this case, I decided to put a limit. Six machines is as much as I can go. I don't want to go beyond six machines.
1:06:34And what I would like to do is that I would like to start shutting off these machines...
1:06:40...when the CPU goes down to 1 percent for a period of 15 minutes.
1:06:45And I'm using this because I know that when it goes down to this for 15 minutes, it has finished the cache.
1:06:52I don't want machines to start terminating themselves when I'm in the middle of cache cooking...
1:06:57...because that will disrupt my operation and possibly corrupt my cache.
1:07:01I want them to only terminate after I'm done.
1:07:04So I'm going to use this criterion for that.
1:07:07I'm going to keep the criterion of 80 percent to start adding more machines, but I'm going to up it from 5 minutes to 10 minutes.
1:07:15So that means if the CPU utilization stays at 80 percent for 10 minutes...
1:07:20...the cloud will automatically add another ArcGIS Server to be added to my cache cooking operation.
1:07:26And through that, I'm going to be able to get more capacity and cook my cache faster.
1:07:32And what's going to happen is, using this criterion, I'm going to keep adding more instances until I reach the maximum six...
1:07:38...and then it's going to hold at that; it's not going to add any more servers beyond that number.
1:07:44And with that setting in place, I'm just going to create a new administrator account for this new setup that I've just created...
1:07:54...and I can get it started.
1:07:57Now while this is cooking, I have actually created this previously, and I have this site that I called CloudCooking...
1:08:05...that was based on the template I created as well from this CloudCache that you've seen before.
1:08:11And to go and look at this quickly, I could show you this site. Going to look into that.
1:08:27And this site, I can see from here, it has only one machine...
1:08:33...'cause that was the one I started with when I started to set my autoscaling.
1:08:38I also can see with my services that my service is already there.
1:08:43I didn't have to copy the data again, I didn't have to stripe my disks again; I didn't have to do any of that.
1:08:49I have everything set up, and what's going to happen is when autoscaling adds these new servers...
1:08:54...they're going to be set up with everything they need.
1:08:57I don't have to go and create anything myself.
1:09:00And now that I have my service, I also have my Desktop locally...
1:09:09...to that machine if I want to get started, so let me do this quickly.
1:09:15I can actually go and log in to that machine here from the console.
1:09:24So here's my CloudCooking site, and I'm going to log in to it by taking the public DNS and logging in to that.
1:09:50And because, if you recall, when I was creating my template, I have hard-coded my password, I know my password already.
1:09:57When I get prompted, I'm going to use that administrator account and that password that I have set in my template.
1:10:08And I hope my network is still working. Looks like I'm having trouble with my network. Let me try this again.
1:10:36Yes. I know why this isn't working, because I missed telling you that in the security group...
1:10:43...when I've created it from the template, I have to go and enable Remote Desktop.
1:10:52So the CloudCooking instance that I have just started doesn't allow me to remote desktop...
1:11:00...because I didn't give access to Remote Desktop here.
1:11:04So what I am going to do is quickly add that, and for the sake of time, I am going with 000...
1:11:11...but please do not ever do that; you should use your own IP.
1:11:16And now I can go and say OK and try that, and I'll be able to log in to my machine.
1:11:29So this is the machine, the end result of all the setup that I've done, the image that I've created...
1:11:36...and now this new site that I have launched based on that image.
1:11:42And within that image, I can go and I can see that I can start Desktop; it's already installed.
1:12:03And from here, I can access the service that's been already cooked in these machines, and I can start my cache cooking operation.
1:12:12So I'm not going to go into details of how to cook your cache.
1:12:15That's going to be the focus of other sessions that we have during the DevSummit...
1:12:19...but I wanted to show you these are all the steps to get to that.
1:12:23And what's the benefit of doing this? The resources; you can get all the resources you need in very quick time...
1:12:29...finish the cache cooking, terminate it, and get your data back, and you didn't have to purchase these resources...
1:12:35...for all the time you're going to have them or own them in your environment for good.
1:12:42So to wrap things up, let's go back to my slides.
1:12:54This is the final slide where we discuss a little bit of best practices.
1:12:59It's always good to plan costs; I can't stress that enough.
1:13:03Inside the Amazon environment, if you start using it like you would with your own local machines...
1:13:08...start them, leave them, create disks and leave them, and then everybody kind of shares an account...
1:13:15...before you know it, you're going to be paying a lot of money, and you're going to wonder...
1:13:17...Why am I using the cloud? Why didn't I just buy a machine?
1:13:21So it's really important to plan your cost and control your resources for what you need to get the benefits you're looking for.
1:13:28Also, creating this type of custom imagery is very important for productivity.
1:13:33As you create these images, they're not just a base that you can start new sites from...
1:13:37...but they're also a very important tool...
1:13:39...if any problem happens, you can terminate the running system and start a new fresh one from the image.
1:13:46Plan security. We've talked a little bit about security groups and how to configure them.
1:13:50Security is a bigger discussion, and there's a lot of security features in the Amazon cloud that you can benefit from.
1:13:55We're going to talk about more of that over the period of the DevSummit.
1:14:00Take advantage of elasticity, which is the autoscaling setup, just like I showed you in Cloud Builder.
1:14:05Have it turn on the machines when they're needed and terminate them when they're not needed.
1:14:10It will do it very efficiently. If you do this manually yourself, you'll just end up paying more.
1:14:15So it's good to use that automation.
1:14:18Plan your data management, how to upload it, where to keep it, and how to move it around.
1:14:22Snapshot is a very good way to move your data around and make multiple copies of that.
1:14:27And in the end, user reporting to keep track of what you're using, what you're paying...
1:14:32...and understand how to make the best use of the cloud.
1:14:36And with that, we move to Q&A, and I'm not sure how we're going to do that with the very big theater...
1:14:43...but we can attempt and try to hear you.
1:14:46If that doesn't work, we're going to be here after the session.
1:14:49If you have a question, please speak very loudly. We'll repeat the question.
1:14:55We have a couple of minutes to do it. Yes?
1:15:00[Inaudible audience question]
1:15:11So the question is, What kind of updates can we do using Cloud Builder?
1:15:15Can we change the OS or do maintenance on the site when it's already created? Want to take that?
1:15:22Yeah, I believe - I don't want to tell you the wrong thing, so I just opened one here.
1:15:26Well, I can answer that quickly.
1:15:27You can't tear into it and change the OS and, you know, you can't get down to those details...
1:15:32...but I believe you can change the description on the site and I want to say the autoscaling settings.
1:15:37Yes. So you can change those, so that's the type of thing that you can change.
1:15:41You can't go in and, like, change the operating system.
1:15:46[Inaudible audience question]
1:15:49Yes, you have to re-create it from a different base.
1:15:51And as Sterling mentioned earlier, we are going to run now on both Windows and Ubuntu...
1:15:56...and new AMIs are going to be available, so as you want to update your site, you could start from a new AMI...
1:16:02...or go directly to your own site and maintain it.
1:16:05Now, once you start a machine, it's yours. You own it, you maintain it, you do in it whatever you want.
1:16:11Nobody else is going to modify it on your behalf and surprise you, so we rely on you to maintain your own machines.
1:16:17So here you can change the autoscaling settings.
1:16:22You can even change, I believe, the size of your instance, because Amazon started allowing that.
1:16:26They allowed you to change the size of an existing instance, which wasn't always the case.
1:16:30So I believe we provide a drop-down, although it says I have to stop my site before I do something like that...
1:16:37...and then I can change, like, the description of my site, the metadata.
1:16:43Okay? Any other questions? Yes.
1:16:47[Inaudible audience question]
1:17:03Yes. So the question is, Can we just add more machines and have them automatically get themselves added to cooking the cache?
1:17:11The demo that I showed you is that.
1:17:13It will automatically add itself to ArcGIS and start contributing to the cache cooking automatically.
1:17:19You don't have to log in to that new machine, set it up, add it to Server, and add it to the service.
1:17:25That's why I created this image, and from that image, I created the autoscaling.
1:17:29Some of this is based on the 10.1 ArcGIS Server architecture, which is brand-new...
1:17:34...so there is some peer-to-peer type of communication between the GIS servers in the site.
1:17:39And the site is able to detect when a new GIS server is added.
1:17:43So when it detects that it's added, it will start handing caching responsibilities to that machine.
1:17:49But you have to kind of understand that, the 10.1 architecture, to get that there's that horizontal...
1:17:54...there's that peer-to-peer piece.
1:17:57Any other questions? Yes.
1:18:01[Inaudible audience question]
1:18:14So the question is, when I started the site that I was going to use to create my template, I said it has to be two instances.
1:18:22This is not a reason because of Amazon; this is a reason because of how we deploy ArcGIS Server 10.1.
1:18:28So that's why it's important to always have two.
1:18:32[Inaudible audience question]
1:18:36Yes. Yeah, and behind the scenes, we take care of all the rest, so we can do this automatic machine addition...
1:18:44...and have them add themselves to the cluster and contribute to the work without you having to do anything else.
1:18:50So that's why it's important to start with two. Other questions? Yes.
1:18:57[Inaudible audience question]
1:19:04Sorry? The last part, what was it?
1:19:06[Inaudible audience question]
1:19:08So there was a question regarding the licensing, how does the licensing work, and the plans for releasing 10.1.
1:19:14The licensing works, the minimum you can buy for Server is four cores, so if you're going to deploy two virtual machines...
1:19:21...and each has two virtual cores, that four-core license will work on the two of them.
1:19:27Now, as you add more cores to your running setup, you will need to purchase a license for each one of those cores.
1:19:34Now, currently, Esri does not offer an hourly rate for ArcGIS Server in the Amazon cloud.
1:19:39What you have is alternatives between negotiating an ELA or purchasing a term license...
1:19:46...which can be for just a period of one month, three months, six months, and one year.
1:19:52And this term license allows you to add those extra cores for limited periods of time, not for the whole time.
1:19:58Regarding the release of 10.1, tentative dates, I think, are June, so the dates there seem to be not...
1:20:09It's within a matter of weeks, very soon, because we're wrapping up.
1:20:15Yep. Any other questions? Yes.
1:20:19[Inaudible audience question]
1:20:40The question is, Can you assign an elastic IP from Cloud Builder?
1:20:44Cloud Builder actually sets up the elastic IPs that you need...
1:20:47...so you don't have to remove or add elastic IPs when you set up your site with Cloud Builder.
1:20:52If you were to set up your site yourself, like if you're using 10.0 and you use the AWS Management console to set up your site...
1:20:59...then yes, you'd need to manually create the elastic IP.
1:21:03Yeah, so I just want to add that the choices you have on the database from Cloud Builder...
1:21:08...you don't have to license these databases yourself; however, you are paying for the license...
1:21:14...because we are using the instances that include the license in the hourly rate for SQL Server Standard.
1:21:21SQL Server Express and Postgres are free.
1:21:24If you're using a different type of database, you can always set it up yourself...
1:21:28...and configure ArcGIS to point to it, and that'll work fine.
1:21:31So if you started ArcGIS through Arc Builder with no database...
1:21:34...and then after that configured it to point to your own database that you set up prior to that, it'll work with no problems.
1:21:41And before I take in any more questions, I just want to make an announcement...
1:21:45...because I'm seeing a lot of people leaving and we're almost done.
1:21:48We've actually received a gift form Amazon where you can get a $50 gift per person, so there's a card here.
1:21:57If you're interested to try it, you have $50 to spend on Amazon Web Services, not Amazon.com.
1:22:03Not the bookstore.
1:22:06So you're very welcome to come over, and we'll give each of you a card of these.
1:22:09And also, we will hand you an evaluation. Please be kind; and we need you to fill these...
1:22:15We also have more by the door if you don't want to stop by and pick them here.
1:22:19We can also take questions down here. I think we may move to that, where you can come up and talk with us.
1:22:24That might be easier than shouting the questions back and forth.
1:22:27Thanks for your attendance, and you can come up here for Marwa and get these.
ArcGIS for Server on Amazon Web Services
Marwa Mabrouk and Sterling Quinn discuss how to leverage the Amazon cloud with ArcGIS for Server.
- Recorded: Mar 27th, 2012
- Runtime: 1:22:33
- Views: 1117
- Published: Apr 20th, 2012
- Night Mode (Off)Automatically dim the web site while the video is playing. A few seconds after you start watching the video and stop moving your mouse, your screen will dim. You can auto save this option if you login.
- HTML5 Video (Off) Play videos using HTML5 Video instead of flash. A modern web browser is required to view videos using HTML5.