00:01Alright, good morning and welcome everybody to this early session on publishing and using maps.
00:07So Tanu put out the survey questionnaires, so please fill them out. I'm supposed to remind you.
00:13So we're, this morning, we're really honored and excited to kind of be able to share with you the hard work...
00:20...that our teams have done over the past year and a half on publishing and map services, and we hope you'll enjoy the session.
00:29We're going to have two separate parts to this session this morning.
00:33The first one is about publishing and using map services, so we kind of take a deep dive into publishing...
00:39...and I'm going to present this together with Ty Fitzpatrick; he was the lead product engineer on publishing.
00:44My name is Stefan Balbo, and I was the lead developer for publishing.
00:49We'll have a little question section after this, and then we're going to the really cool new enhancements of map services...
00:56...which is going to be presented by Tanu Hoque; he's the product engineer for our map server...
01:01...and Craig Williams, our lead product engineer for mapping.
01:05Okay, now let's dive right into publishing, 'cause we have a lot of things to cover.
01:13And we've talked a lot about this DevSummit about publishing, even the last one; it was a really important thing for us.
01:20It's actually one of the driver projects for 10.1.
01:23So why is this so important to us?
01:24Because, you know, we live in a world with blogs and Twitter and Facebook...
01:29...and everybody wants to put their information out to the web or share it with others.
01:34It's the same with your GIS information.
01:36The content you created on your desktop or in your organization, you want to share it with people that you work with...
01:44...with your colleagues within your organization, or even with the public with the open web.
01:50But not only do you want to share your organization, it's really expected from you that your work is available to others.
01:57Information access has to be transparent today.
02:01So that's why we think that, you know, publishing is critical...
02:04...and we spent a lot of effort in the last year and a half developing this functionality.
02:10When we talk about sharing, we actually mean two things. The first one is we can share as packages.
02:16This is kind of...we had that since 9.3.1, where you create map packages, and we enhanced that in 10 and 10.1.
02:22Now you have geoprocessing packages and tile packages, et cetera.
02:27And this is where you, you know, zip up your work into a package and you share it with another professional...
02:32...who also has to have ArcGIS Desktop.
02:36Sharing as a service is the second way to share, and this is publishing.
02:39This is where you really give out your work to everyone.
02:43And we had a vision for 10.1 where we wanted to make it a lot easier for you to share, you know...
02:49...GIS resources like your maps or globes to everyone.
02:54And the first thing we wanted to accomplish was we wanted to have a unified sharing experience.
02:58That means no matter what service you published, it should all kind of work the same way...
03:02...and it should be, you know, a unified experience.
03:06We also wanted to extend the analysis we kind of started with 9.3.1 for map services...
03:11...and expand that to all services, all SOEs, to really make you successful publishers for publishing your services.
03:19And the third thing was we really wanted to, you know, enable sharing to servers in the cloud...
03:24...and as well to ArcGIS Online hosted mapping.
03:28So these were...this was our vision for publishing.
03:33So now let us talk a little bit about these three distinct workflows, how you can publish services from the desktop.
03:40And the first one is the classic ArcGIS Server is running in your enterprise on premises.
03:46This is how you have been publishing since 9.0, I believe, since this came out.
03:51And, you know, it's basically we call this publishing to a server on premises...
03:58...where the server and the publisher machine share the same data.
04:03The second use case or scenario we want to look at today is when you publish to an ArcGIS Server running in the cloud.
04:09You still have an ArcGIS Server, but it's not running on premises; it's running on an EC2 instance in Amazon.
04:16The third case we want to look at, and that's really the easiest way to share; this is the easiest way to publish a service...
04:23...is you directly publish to a service that's running in ArcGIS Online.
04:26So this is a Software as a Service scenario; there's no ArcGIS Server running there. It works very easily.
04:35So we're going to look at this one first...
04:37...and Ty is going to show us a demo now about sharing services as hosted maps on ArcGIS Online. Go ahead, Ty.
04:43Okay. You need to switch over. Oh, sorry. Thanks.
04:47Okay, so in this first scenario, like Stefan said, I'm going to publish to my organization on ArcGIS Online.
04:54And right now, I'm signed in to ArcGIS Online as a special user. I'm an administrator of my organization.
05:00So if I go into ArcGIS Online and I'm signed in here...
05:03...if I go into the My Organization tab, group, or whatever you want to call it...
05:08...you can see all the members of my organization here.
05:12And my name is - where's Ty? It's in here. I'm the administrator, right? No, Paul's the administrator right now.
05:22But I'm the administrator of this group also, and I'm able to publish services as well as administer content of this organization.
05:33So to publish content, from the File menu, go Share As a service, and I'm allowed to publish as a service...
05:42...and I'm going to choose my hosted services.
05:45And here's my service name. Mark it by DS for DevSummit.
05:52And when we get into the service editor, some of you have seen this already in the plenary...
05:56...this is where we can configure all the parameters of our service prior to publishing.
06:00And this allows us to kind of get any best practices that we want to maybe accomplish...
06:04...with our map prior to publishing this service.
06:06So from the Capabilities tab, the two types of services that you can publish as a hosted service...
06:11...are tiled services and feature services.
06:13So in this scenario, I'm going to publish a tiled service.
06:16You go to the caching page, you can see that this is the visible scale range of my map that I currently have right now...
06:24...based on the scale suppression that I've set with the layers in my map.
06:27And I can adjust this if I want.
06:29If I adjust this outside of this range right now, I probably wouldn't get any tiles at that scale range, 'cause the map's not even visible.
06:36I have item information. So all services have item information, or metadata.
06:40As well, I can share this with other groups or with all the other members of my organization.
06:46So let's publish this map. So what it's doing is...
06:50This is a pretty simple map with one layer in it, but if you had data coming from an enterprise geodatabase...
06:55...or a shapefile or other types of data sources, it would package that all up together, with the service parameters...
07:01...that I've defined here, as a single file, transfer that up to ArcGIS Online.
07:04Everything you need to create that tiled service is defined within that package to create that service.
07:10Okay. So while this guy's publishing, let's take another map, and this is going to be a feature service.
07:17So this is all locations of where I want to distribute Hangar 24 beer across Southern California.
07:26And the first map was beer consumption for where you can consume...
07:32...how much consumers are actually drinking beer based on block group.
07:37So we can kind of use these two maps to kind of mash them up.
07:40So, again, from the File > Share As menu, we'll publish this as a service, using my organization Dinoco...
07:53...and in this case, I'm going to publish, instead of a tiled service, I'll publish a feature service.
07:59So from the Capabilities page, I'll change that to a feature service.
08:02Again, I have item description for this, and so forth, share it with the other members of my organization as well.
08:08So here we've got a little issue, and this is - we ran an analyzer; it said "Basemap layers cannot be published."
08:14So we don't allow you to publish a basemap layer with a feature service.
08:18So in this case, I can remove this guy, because I don't need him.
08:26And we want you to be conscious of the things that get removed from the map, so we don't just remove that for you.
08:32We actually kind of make you go through that process when you publish.
08:37So now we're going to publish this feature service.
08:39Let's go back to my tiled service. So that map was successfully published, and if we go to the Catalog window...
08:45...you can see there's my service, under My Hosted Services, and there's my organization.
08:50You can go to View Cache Status, and it'll get an update of how much cache has been built based on that service.
08:57Let's go to the details, and you can see, based on each LOD, the completed level of tiles and the percentage complete...
09:03...and here's the job that's actually being executed.
09:06So at this point, 80 percent of my tiles are created, and so forth, and you can see them starting to be completed.
09:13If you want to get an update, just click the Refresh Status, and you'll see an update of more tiles being generated.
09:21See if this guy's finished.
09:24Okay. So the feature service also got finished.
09:27Now if we go on to ArcGIS Online, and we're going to go to a map, to my organization.
09:32Actually, let's go back here.
09:36You can see under My Content, the two maps that I just published, Hangar24_DS and beer by ZIP code...
09:43...and now we can look at them both in a map.
09:47So let's search based on my content.
09:57Where is it? It's in my organization. There we go. And probably getting too much.
10:10They just haven't gotten into the search yet.
10:14So we could take a look at that service.
10:16Let's look at the Hangar24 one; it hasn't quite completed its process of getting into the organization; I've got a little...
10:26So here are all the beer locations, and again, this is a feature service, so we can edit these points, right?
10:32So if I edit, you can see, based on the unique value renderer, created those templates for that particular feature service.
10:38So I have all the existing ones; I have one new one.
10:41If I wanted to create another one, simply click on the map...
10:44...and you can see that I could put in the attributes and so forth and edit that as a feature service.
10:49So I'm editing features in my hosted feature service on ArcGIS Online.
10:56So this is the two ways that you can host services on ArcGIS Online; one is a tiled service and another as a feature service.
11:06Switch over. Back? No. You on A? Yeah, alright, thank you.
11:25Well, thank you, Ty. And let's recap for a moment what we saw here, what was going on.
11:31So we had our desktop that Ty had with, you know, some local maps and with local data...
11:37...and he wanted to share this out online as a service.
11:40He doesn't have any server and has no...doesn't need to know anything about, you know, IT or infrastructure or proxies.
11:47All he has is an ArcGIS Online account in the cloud.
11:52So what he started off is bringing up that service editor and configuring the service...
11:56...you know, like setting all the parameters for the service, and actually we have pretty good defaults.
12:00So on most cases, you should pretty much be able to just fly through.
12:05The next thing that happened, when he hit Publish, well, he got an analyzer error in the one case.
12:10But the first thing that happens is we're staging.
12:13So this means we're creating the service definition, and that's our new exchange format for publishing a service...
12:19...and I'll talk a little bit more about that in a moment.
12:21But what's happening there, we're basically taking the map, the data, and the service configuration...
12:27...and put that all in one file and zip that up.
12:29And then when we are publishing, this is getting uploaded to ArcGIS Online, where it then, you know...
12:36...to publish, it gets actually, you know, unzipped; unloaded; and if it's a tiled service, the tiles get started to cook...
12:43...and they're in a feature service, the features get loaded into a cloud database.
12:51So what Ty did, he created basically a tiled map service, which was the first one, and then a feature service...
12:58...and then he simply mashed them up in a web map and it was ready to go, and everybody can see it.
13:03He could share it with his organization or publicly, whatever he desired.
13:09We think this is really cool. We think it's a great way for me, as a Desktop user, without, you know...
13:15...having to know Server or having to own a server, to publish my maps.
13:20It's really the easiest way to share.
13:23Like I said, you have no server to buy, to install, or to maintain.
13:27It scales automatically. I mean, your map could be so cool it could be blogged on by CNN...
13:33...or shown on national television, and it would scale. Go ahead.
13:37[Inaudible audience question]
13:41They'll be coming out with 10.1.
13:48And you have no firewall or IT issues.
13:51There's some limitations. Currently, we only support tiled map and feature services, but we're definitely working on more...
13:57...and the data is private to each service.
13:59That means, you know, if you would have two map services who would work on the same data...
14:04...you would actually have to upload the data twice.
14:07So this is great if you are a Desktop user.
14:09You're not, you know, don't have a server and you want to create public-facing services.
14:14Then, you know, hosted mapping is great for you.
14:19Okay, now let's talk a little bit about service definitions.
14:22So this is our new file format for publishing in 10.1.
14:26It essentially replaces the MSD that we had in the past few releases.
14:30But it's kind of expanded; it works essentially for all services.
14:34So whatever service you publish, there's going to be a service definition, your map, geoprocessing, globes.
14:41And it's really just, you know, you can...it's a 7-Zip archive, so you can look inside there...
14:46...and you'll actually, for a map service, find an MSD in there.
14:50You'll also find, you know, the same content of a package if we package data with it, and then some other information.
14:57One cool thing that's new in 10.1 that we've done, we're actually when we're staging, we check and you have a map service...
15:04...we check what fonts your maps use and we check if that font is available on the server...
15:09...and if it isn't and it's legal to embed that font, we'll embed that as well in the service definition.
15:14So your fonts will show correctly up on ArcGIS Online or on any other service you publish to.
15:20Also the service configuration, all the parameters of your service, your caching information...
15:25...that's all in the service definition as well as data if it needs to be put in there.
15:31Again, this gets uploaded to the server when you're publishing.
15:34And in Ty's workflow, he had published right away; we didn't even see the service definition.
15:38It basically got created, gets uploaded, then kind of gets deleted.
15:42But you can also save that out, the service definition, and publish it later either from Server Manager or from Catalog.
15:51Okay, now let's walk on to our next scenario, and that's actually an ArcGIS Server running in the cloud.
16:00We were going to have a demo on that, but we're going to skip that demo 'cause we actually already shown it on the plenary...
16:05...and we are a bit strapped on time, but there is a video for this on videos.arcgis.com...
16:11...where Ty put a video on about publishing map services to an ArcGIS Server in the cloud where the data gets copied.
16:17But I'll talk a little bit about that workflow, but it's very similar to the one we saw just now.
16:22The only difference here is when we start off is that we actually have an ArcGIS Server on an EC2 instance in the cloud.
16:29So we bought an ArcGIS Server, and we installed it on a server in the cloud.
16:34Again, on our desktop we have a map; we have local data, and again, we would create a service configuration.
16:41We'd stage it, it packages up all the data, map, and parameters, and - oops.
16:46And when we publish, it gets uploaded to that machine in the cloud.
16:50In that case, it really runs in ArcGIS Server, so I have the full power of ArcGIS Server there...
16:56...and we could create dynamic map service that we then can use in Desktop or any other client.
17:05So in this workflow, the improvements if you publish to an ArcGIS Server in the cloud...
17:11...I think before, you had to kind of remote desktop in, copy the data manually, and then, you know...
17:16...muddle with tools like MXD Editor to kind of, you know, fix up the paths.
17:21So this would be a very simple workflow, where just the same way as it worked for hosted...
17:25...it just packages everything up, uploads it, and publishes it.
17:28There's absolutely nothing you have to do, so it's very, very easy.
17:32And you also get the benefit of the, you know, comprehensive analysis that we put in 10.1.
17:37When is that great to use?
17:38When you have public-facing services that are outside your firewall but you do need the full functionality of ArcGIS Server.
17:47Alright. Now let's move on to our next scenario, and this is actually kind of the classic publishing scenario...
17:54...that we've all known since 9.0 where our ArcGIS Server sits on premises.
17:59So Ty's going to put a demo now on sharing services to ArcGIS Server in the enterprise...
18:04...and show us all the improvements we made for 10.1 in that area.
18:10Okay, just to kind of wrap up the last demo.
18:12I got a little excited and didn't wait long enough for the two services to be published.
18:17Here's my Hanger24_DS and the beer by ZIP code DS, [proofer: name is BeerByZipSoCalDS] the tiled services...
18:24...and of course, you can click on the attributes and see them and see the field values, and so forth.
18:29So that's just kind of wrapping up that last one, 'cause I didn't quite complete it.
18:35So here's a map, as Stefan said, now we want to publish within the enterprise, within the organization.
18:42We don't want to copy data or anything like that.
18:45So here I have some electric facilities, and I also have a cached map service behind it as a basemap.
18:51So I want to kind of overlay this operational data with my cached map service.
18:58So here I have the basemap, and I can remove that guy, 'cause I don't want to publish that...
19:04...and I just want to publish this as a dynamic map service.
19:10Then I want to reference that as my enterprise geodatabase.
19:15So we're going to publish to my local enterprise - my local ArcGIS Server, and this is the service name that I'm going to use.
19:24Same type of workflow as publishing to a hosted service, except I have more options, obviously...
19:28...when I published to a local server.
19:31So we can analyze for this map, just to make sure we don't have any issues with the map.
19:36I don't have any. We have a couple things that we could take a look at.
19:40Data source is not registered with the server on the database.
19:44The data source is not registered with the server and the data will be copied.
19:48So by default, we would copy the data to the server, and that's kind of the easiest way to publish.
19:53It's a little bit of a reverse from what you're used to, but we've added all these new workflows with publishing.
19:59So to ensure that the server can see your data, we want you to register your data with the server...
20:04...and that allows us to validate and make sure the server can see it prior to publishing.
20:10So we're going to register that server.
20:14[Audience question] I'm sorry, registering the server, or the data?
20:16Sorry, I'm registering the data, so I'm registering the connection of my enterprise geodatabase...
20:22...so that the server can validate that, basically, workspace connection.
20:26[Inaudible audience question]
20:30[Inaudible audience question]
20:37Yeah, yes. Yeah. It will.
20:40[Inaudible audience question]
20:43That's typically the way you would do it.
20:48It would run off of Windows authentication, like if you're logged in to SQL Server using your OS authentication...
20:55...if that's what you mean, it would work that way.
20:56[Inaudible audience question]
21:03That's to run ArcGIS Server as a...yeah.
21:06So your ArcGIS Server would need to be able to see that SQL Server database, right, that user.
21:15So I'm going to register that guy in the data store of the server, so when I analyze now...
21:21...it's not going to want to copy anything to the server.
21:25So if I go back to my ArcGIS Server connection that I've got here, and I go to Server Properties...
21:33...you can go to the data store and you can see, in the data store properties, I've got that registered data store.
21:39So if I look at the properties of it, you can see it's got the name, and as the publisher and the server...
21:45Everything's on the same machine, but assume that this is two different machines...
21:47...where the publisher is on one machine; that's the connection to my data...
21:51...the server's also got a connection to that data.
21:53We're both looking at the same data, and we're going to reference that.
21:56So the publisher and the server are the same.
21:58We're looking at the data the same way, so we're going to reference that data in place.
22:04So now when we publish this, it's going to take the service parameters like I did, similar to the way I did with ArcGIS Online...
22:10...except it's not going to copy any data; it's going to reference all that, the enterprise geodatabase, in place.
22:16So as I make updates, that service will see those updates.
22:19[Audience question] Updates to the geodatabase?
22:21Updates to the geodatabase, yeah.
22:24[Audience question] So you do not have to reregister the service.
22:27You don't have to reregister anything, right.
22:29There's another workflow that Stefan talked about just...
22:32The one that you saw on Tuesday when I did it on stage, you publish with copying data to the server...
22:39...so if you make edits in your enterprise geodatabase, you won't see that.
22:43But the next workflow, I'll show you how you can do that.
22:47But there's publishing with copying data to the server, there's publishing with referencing the data within your enterprise...
22:52...and there's another workflow that we'll go into in a minute.
22:56So that was published successfully. Turn all this off, and you can see my service.
23:03And we'll bring it in, and that's my dynamic map service referencing my enterprise geodatabase.
23:11Switch over. Alright, thank you, Ty. No. You've got to go on A.
23:20Alright, so let's recap this a little bit 'cause there's a few new concepts here that we haven't...
23:26...we kind of introduced with 10.1.
23:28I mean, the workflow is classic, right?
23:30Your server and your desktop, they both kind of have the same access to the data like Ty showed.
23:36So, again, you know, here we start off with a desktop, we have a map...
23:40...and it references data in some enterprise geodatabase here called Ella.
23:45On the server, we have an ArcGIS Server that's running on our premises and has this list of data stores...
23:50...where you can, you know, register data stores with.
23:53So what Ty did, you know, he created, started to publish, created a service configuration...
23:59...and while he did that, he ran into that analyzer warning, and there he could right away, from there, add a new...
24:04...register that data store that his map used with the server.
24:09And so Ella basically gets put onto the server so the server knows about it, and while you register, it gets validated.
24:15So every server in the site will check that it can access that data, and it's actually really valuable...
24:22...and that's the big improvement, 'cause that's how many map services or other services will fail...
24:27...is 'cause the server doesn't have access to the data because of...
24:30...you know, rights issues or, you know, you forgot to put it on one server.
24:37You have many things. That's the very common source of failure.
24:40So once we registered that, it also, our ArcGIS Server references that enterprise geodatabase Ella...
24:47...and then it was a simple matter of creating the service definition, this time without the data...
24:52...because, you know, we know that the service can access the data...
24:56[Inaudible audience question]
25:11You would have to - yeah. Every server has its, or every site has its list of own data stores...
25:16...so if you would take that SD, would publish it to another server, you'd have to register the data store.
25:21[Inaudible audience question]
25:22Yes. But you only have to register it once, so furthermore you use that data store...
25:26...it will of course remember it. You won't get the warning message; it'll just work.
25:30It's basically a onetime thing.
25:34So like I said, the data is not part of this right now because we determined that it's accessible by the server...
25:41...and you can actually mix and match.
25:42You can have data that, you know, in your map that is in a data store that will not get copied...
25:49...and you can have data like a local shapefile that will get copied because, you know, it's not accessible from your server.
25:56So it's actually a really strong workflow.
26:00So we'll publish, and Ty created a dynamic map service that he later then used in Desktop.
26:08So what I kind of did, the cool things about, you know, the improvements we made in 10.1...
26:13...we really wanted to make you successful publishing to shared data by adding this validation step.
26:20We also, you know, have really comprehensive analysis...
26:23...and still just in ArcGIS Server in the enterprise you can get that benefit of really simple publishing...
26:29...where just the data gets copied up.
26:33When do you want to use an ArcGIS Server in enterprise?
26:36Well, you need the full functionality of ArcGIS Server; all the services are available - GP, image service - everything.
26:42You need full control over all your hard- and software...
26:45...and it's kind of the only way if you want to publish services on live, shared data.
26:51Then you use ArcGIS Server in the enterprise.
26:54So let's talk a little bit about analyzers.
26:57So we started that concept in 9.3.1, and we think it was really powerful in improving your services...
27:04...because not every map that you just publish makes a good service.
27:08Actually, it doesn't at all. And that was one of our biggest problems with the server before 9.3.1.
27:14It wasn't that the server was necessarily slow, but we just didn't know if the map was good or not.
27:20So now we've really totally expanded that, and we have a total of 276 analyzers running. It's mind-boggling.
27:27And just 133 for map services. So it checks for everything; it checks for every SOE that everything is correct...
27:35...also GP services have analyzers; feature services, 22 analyzers for feature services.
27:42So our goal and the statement we want to make is if you analyze, if your analysis works, you will get a successful service...
27:51...and that's our goal we have, you know, that you can always publish successfully.
27:58So now next let's talk a little bit about this data stores more, 'cause it's kind of, you know, this new concept...
28:04...and we get quite a bit of questions.
28:07So again, Server has this list of registered data stores, and data stores can be of two types.
28:12It can either be an enterprise geodatabase, where you can register all your SDE databases, your SQL or Oracle...
28:22...or it could be a folder where you can use all your...register all your file-based data, file geodatabase, shapefiles.
28:30I want to mention one tip. Try to register the folder as low as possible to get the maximum benefit of validation.
28:40Then as we saw there, the access to the data store gets validated during registration...
28:45...and the cool thing about it, it actually checks every single machine in the site that it can access the data.
28:51'Cause it was also one, you know, previously a big point of failure that, you know, you had access from one machine...
28:57...of the site, but your other machine couldn't access the data, and you basically...
29:01...it's very hard to debug that and know about that.
29:05There's a really great help topic about it, you know, about registering your data, so I'll point you to that as well.
29:13But let's go a little bit more in detail.
29:14And we have basically two main workflows of registering data, and the first one is basically what Ty just showed us...
29:21...where the publisher and the server have access, have the same access to the data.
29:25And that can be your folder, your file geodatabase, or your enterprise geodatabase.
29:31And the key of that workflow is we want to make sure that it's accessible by the server, and this helps you...
29:37...you know, publishing, with the traditional way of publishing.
29:40So this is the one way, the one reason why you want to register your data store.
29:44There's another workflow, and Ty's going to show us shortly a demo about that, but I'll kind of introduce that here...
29:51...where actually the server and the publisher's data store, they're not the same; they're different.
29:55So I basically, on my publisher's machine, I point to local data in an enterprise geodatabase or in a file geodatabase, et cetera...
30:04...and I have a copy on an ArcGIS Server that has the same data, either used by replication...
30:10...or where I copied the data manually before publishing.
30:16So in that case, when we publish your map up into that ArcGIS Server...
30:20...we actually change the data connections out in the map and point them to that new data.
30:25Basically kind of what MXD Editor used to do, where you kind of had to go by hand and change the paths.
30:31So it does this for you. And this is a really powerful concept...
30:34...and you can get some great workflows going with that, and that's what Ty's going to show us next.
30:40He's going to publish to an ArcGIS Server in the cloud with replicated data.
30:47Okay. Is my map up? Okay.
30:50So I've got a connection to my Amazon, my ArcGIS Server running on Amazon.
30:55You'll look at the properties, you can see the long connection string that I've got up there.
31:01And what I'd like to do is right now I've got a map of some operational points, my citizen request points.
31:07As well, I have a basemap, a tiled service, that I've also got up on that service as well - on that server as well.
31:14So this is the tiled service that I'm looking at on that server, and I want to publish a feature service to that...
31:21...and like Stefan said, I want that to be a feature service that's running with the database up on Amazon...
31:26...and then sync it with my enterprise geodatabase within my organization.
31:30So how could we do that?
31:32So the first thing we're going to do is add a data store, and I'm going to just call this rep, and I'm going to add a data store.
31:43So my local, on my publisher connection - let me go to my... Let me do this without talking here.
31:54I'm going to go to my SQL Server database that's on my laptop, and I'm going to create a replicated data store.
32:00So I'm going to uncheck this, so it's not the same as the publisher, and when I do that...
32:05...it picks up the enterprise geodatabase that's on the ArcGIS Server instance on Amazon.
32:11And that comes with it when you buy CloudBuilder...
32:15...when you create an ArcGIS Server instance on Amazon with CloudBuilder.
32:19So you create that with ArcGIS Server, and you get an enterprise geodatabase, and it picks that up.
32:25And what we're also going to do is create a geodata service at the same time.
32:28This geodata service allows us to create the replica, transfer the data up there to Amazon from my citizen request points...
32:38...and then that's going to allow me to synchronize the changes between the two databases...
32:43...as I make updates to my feature service using the server in the cloud.
32:49So right now it's spinning up that, so we've made that data store, so this is going to allow me to publish to that server.
32:57But first, let's transfer the data up there.
32:59So using the Distributed Geodatabase toolbar, we can create the replica, and I want to make a two-way replica...
33:09...and I want to use that geodata service that I just published or created from a data store.
33:17Let's get connected to that. So I'm going to use that guy; that's my replica name.
33:27Oh, let's get rid of that. I think that's going to...
33:32And I want to use the full extent of the data.
33:39Okay, so now what it's doing is it's taking the citizen request points...
33:42...and it's transferring that up to the enterprise geodatabase using that geodata service.
33:47So that's all done now, so now all that data - it's not that much data, but it transferred that up there.
33:53Let's remove that basemap; I don't need that to publish the service.
33:57And now what I want to do is, using that data store, we're going to publish this as a feature service.
34:05And what it's going to do is it's going to recognize the enterprise geodatabase on my laptop based on that data store...
34:11...and realize I have to switch the paths of the map to the data store, the enterprise geodatabase on my server.
34:27So in my service editor, I'm going to pick Feature Service; it's going to pick that up, right?
34:36So it's still - it's not copying data up there; it's referencing the data.
34:40It knows to reference the enterprise geodatabase that's up there.
34:43So now when I publish, we're going to create a feature service that'll take the map...
34:47...change the paths of the map to the enterprise geodatabase that's on my ArcGIS Server on Amazon...
34:53...and then I can make updates to that guy.
34:58So it's just transferring that file up to the server on Amazon. Give it a second here.
35:17So this is a pretty exciting workflow 'cause before, you'd have to remotely log in to that server...
35:23...try to figure out how to get data up there, and here we've really tried to automate a lot of that workflow for you.
35:29So now, let's go to here. Let me get the connection.
35:41And we're going to create a new map.
35:59So you can see that I've got my service request points, and I want to add that feature service...
36:04...and again, this is looking at that enterprise geodatabase on Amazon...
36:07...and I can also add my parcel access, so that's behind there, and now, let's add a point.
36:18So we added a new citizen request up there, just above the northern, the Naperville northern high school.
36:27And now let's go back here and look at the same map, and we'll sync those changes.
36:35So I added a point to the enterprise geodatabase on Amazon; now let's synchronize those changes...
36:41...and I'm going to use that geodata service on the ArcGIS Server instance right there.
36:56Replica...whoop. Picked the wrong one. Sorry. From a previous test run.
37:09Okay. So we could sync changes both ways if I had made edits locally...
37:17...and now that's going to synchronize those changes between the two databases...
37:22...and bring that change that I made through that feature service that I did on ArcGIS Online to that feature service.
37:29Okay, so there's the point.
37:31So that allowed me to - again, let's recap.
37:33I was able to very easily create the data store, that it gave me that geodata service.
37:38It allowed me to transfer my data from my local enterprise geodatabase up to that ArcGIS Server instance on Amazon...
37:45...without logging in or anything, and then I was able to use that data store to publish a service...
37:51...change the paths from my local enterprise geodatabase to that ArcGIS Server - or that enterprise geodatabase on Amazon...
37:58...and then I was able to sync changes and make edits between those two databases.
38:03So that's, in a nutshell, how that workflow works...
38:06...and we've also got that video up on videos.arcgis.com as well if you want to resee it.
38:11[Audience question] When you join those replicas, can you cross databases?
38:14So if you had SQL Server on Amazon; can you have Oracle on your local?
38:20You can. Generally, yes, you can. But generally, it gets a little trickier in terms of like...
38:28...if you're going to go from like SQL Server and Postgres, there's different requirements for Postgres.
38:32Like generally, you want all the case to be lowercase.
38:36So you have a bit of a different data requirement in some of those scenarios.
38:40So it does get a little bit trickier when you want to do it that way.
38:43This is SQL Server to SQL Server.
38:45And if you wanted to create an ArcGIS Server instance with Linux, you could use that with Postgres in that case.
38:52So you do have different options with cloud, the ArcGIS Server running in the cloud, with different flavors of databases.
38:59[Audience question] What about from an enterprise geodatabase to a file geodatabase? Can you replicate to go that way?
39:06Well, in that case, you would - And sync changes?
39:09Replication doesn't work with file geodatabases...
39:12...so it only works with enterprise geodatabases in this scenario, in terms of syncing.
39:16[Unintelligible audience comment]
39:19[Audience comment] You could one-way replicate to a file geodatabase starting with 9.3.1.
39:24Yes, you can, that's true. In this workflow, though, we're using enterprise geodatabases to do that.
39:32[Inaudible audience question]
39:33In this particular workflow, no. In this particular workflow, no, it won't.
39:40Oh, yeah, yeah. And you can't have an editable feature service with a file GDB anyway.
39:44It has to be an enterprise geodatabase for that particular scenario.
39:49[Audience question] When would you actually use this workflow?
39:52Well, this is the kind of workflow where you have your enterprise geodatabase and you don't want people...
39:56...like external customers, hitting your enterprise geodatabase.
40:00You want them to use the data that's out on an external server, and it doesn't affect your...
40:05You know, say you had 3,000 people editing your enterprise geodatabase.
40:08You don't want, you know, the citizens of whatever county hitting your enterprise geodatabase directly.
40:13So this allows them to hit at a different server and then give you that opportunity to sync those changes...
40:18...when you want to do it and validate the data that's coming and making sure it's accurate, and so forth.
40:26[Audience question] Could you also select not the whole database, just the partial one? [Unintelligible]
40:32Well, it's based on the rules of replication, so in this particular scenario, I used the data for the whole extent of my feature class...
40:39...but you could've picked and choose which feature classes going through that advanced option when you created that replica.
40:44And so all the functionality of creating replicas is there for you to define what you want to push out.
40:54[Audience question] Can you easily apply security in both of those scenarios?
40:59I guess can you clarify that? [Inaudible audience question] I'm sorry?
41:02[Audience question, continued] You say these are all public [unintelligible] servers.
41:05[Audience question, continued] Can you apply security to those [unintelligible]?
41:08You want to answer that?
41:09The question was if you can apply security. Yes, you can.
41:11You have the full power of ArcGIS Server and all the security models that come with it.
41:16If you want more detail, Jay's in the room. He can answer a lot more of those detailed questions. But yes, of course.
41:25So let's move on. We'll have some more time for questions later.
41:30Question here. Okay.
41:32[Audience question] Where do the data store parameters physically get stored?
41:36[Audience question, continued] [Unintelligible] database authentication and a password change?
41:40It gets stored encrypted.
41:42[Audience question, continued] But the data store parameters, where do they physically...
41:45On the server. They live on the server.
41:47[Audience question, continued] So if you...
41:48Only accessible to publishers and administrators, not accessible to general users.
41:53[Audience question, continued] Okay, so if I did a password change...
41:56You would have to change the data store.
41:59[Audience question, continued] Okay, but only one place?
42:00Yes, only one place. Can you switch over, please?
42:08Alright, thanks, Ty, for the demo. Let's recap a little more; hopefully, it kind of sinks in what we saw here.
42:17So the scenario on the desktop kind of stayed the same.
42:19We have our map; we point to some enterprise geodatabase that's local...
42:25...and then we have an ArcGIS Server running in the cloud on an EC2 instance.
42:29There's also a geodatabase in the cloud, also running on an EC2 instance.
42:33And when you, like Ty said, when you use CloudBuilder and install that - and install it, you get both of those for free.
42:41And actually, the data store that you'll see actually gets generated automatically.
42:47Now, then what Ty did, though, the next step he used that EC2 geodatabase and hooked it up...
42:55... with his enterprise geodatabase one, so that means if we - if data has a connection to this enterprise geodatabase...
43:06...we'll swizzle it out when it gets published and it points to that geodatabase on the EC2 instance.
43:12You could do that all before with 10.0; you just had to really do it all kind of manually using Map Path Editor...
43:18...you know, copying up data, remote desk - you know, logging in to your Amazon instance.
43:23The next thing it did automatically was create that geodata service, so it also does that for you...
43:28...so you can actually transfer data and use replication with it.
43:33So once we have that geodata service in place and we created that replica...
43:37...you know, we're kind of in business between our two databases here.
43:41The rest was just simple publishing. No data gets copied up; it simply gets, you know...
43:46...the map and the service configuration get put in the service definition.
43:50It gets, again, uploaded to that server in the cloud, and in that case, a feature service was created, you know, during publishing.
44:00And we can use that in our web map, and Ty edited that feature, and now you saw all the syncing that took place.
44:06We think it's a really powerful workflow, and again, the big improvement was it's easy.
44:12Before you had to, you know, jump through a whole lot of loops and really understand a lot of details.
44:17And when do you want to use it? When you want to synchronize your data between the cloud and the enterprise.
44:21I think it was mentioned before, yeah, I think you kind of want to keep that separate...
44:25...because your cloud server would be public facing...
44:28...but you have your local data in your enterprise that's kind of shielded behind a firewall, then this is a great use case.
44:38There are actually a number of additional publishing scenarios.
44:41We also really improved publishing to Linux, 'cause that was also, in the past, you know, pretty cumbersome.
44:48One of the workflows is you have kind of - what people have, you have many are copies of data on the desktop...
44:55...and the Linux server, but also your Linux server is kind of your public-facing one.
44:59So that workflow, you can also use that replicated - the data stores that are different on the publisher and the server.
45:08Another case could be you have shared data between Linux and Windows using Samba or some other technology like that.
45:17That works really well, too, in that scenario. You just need to set up your data stores once; you publish and it just works.
45:24Another case is publishing feature services when the data gets copied.
45:30When you actually have a file geodatabase locally, it always needs to be copied into a multiuser geodatabase.
45:37There's actually a different type of data store you can register, which is a server-only one...
45:41...and that's where that data gets put.
45:44And a third, you know, major scenario is publishing to a new spatial data server...
45:48...the lightweight server for nongeodatabases as well.
45:55The good news is Ty made a lot of videos...
45:58...so you can see those on videos.arcgis.com for all these scenarios and learn about them.
46:04So I encourage you to look at that, but obviously, we don't have time to go through all these here in this session.
46:12So now one question that's going to come up because, you know, we have all these new service definitions...
46:16...Hey, what am I going to do with my old services?
46:19So in 10.1, we said all services, all map services and actually all other services as well, are based on these service definitions.
46:28So now in order to migrate your services, you need to start - you need to have the existing MXD, the map document.
46:33The MSDs are not going to be compatible.
46:36We deliberately didn't spend the time and effort to kind of, you know, make this work...
46:41...because, you know, you still should have your original MXD.
46:44Now you can two ways - there's two ways you can publish that.
46:47You can do basically what Ty did, you know...
46:49...using the service editor and the kind of nice UI experience to publish, you know, through Desktop...
46:55...but if you have a lot of map services, that might get a little cumbersome and annoying...
47:00...so we actually have a little bit of a publishing API you can use with Python.
47:05So I'm going to introduce that real quick here.
47:08So you want to use that, you know, for batch migration from 10.0 to 10.1 or for scheduled service upgrades.
47:14But it basically starts off with a map document, and then you load it up and prepare the MXD so you can set different things.
47:23And there is one new method in arcpy.mapping, which is called create an SD draft...
47:28...that's kind of a precursor to the service definition; it's an XML file...
47:32...and you can actually then in Python still, you know, modify the XML.
47:37And then you can stage and publish through GP tools.
47:41I'm actually going to show a sample that's also online, CreateMapSDDraft sample...
47:47...but I'm going to just briefly walk you through this.
47:50And so, I said before, first step is you kind of open up the MXD.
47:56You can modify it; like here, he added, you know, summary and tags.
48:00Then the next step is you turn and call the CreateMapSDDraft method...
48:05...so that creates that XML file that contains all the properties of your map, of your map service.
48:11You can then use, you know, parse it in XML and change those properties out, you know, through Python as you need...
48:18...you know, set everything you want that you can in the UI so you have everything available.
48:23And the next step is you analyze. So you just say AnalyzeForSD...
48:28...and then if there were no errors, you execute the Stage Service tool.
48:32So the Stage Service tool will take that SD draft and turn it into a service definition...
48:37...so it will package all the data up as necessary, will do exactly the same thing that Ty did when he had published the first part.
48:44And then once that service definition is there, you call the Upload Service Definition tool...
48:49...and that will take the service definition and you pass in also a connection.
48:54It will upload it to the server and then, you know, publish it from the server and turn it into a service.
49:01So that's an easy way for you to migrate your data. Like I said, that script is online; you can use it.
49:09So now we'll take - how are we on time? We're like 15...
49:14Gotta get going. You want - no, I think we can take about five minutes for questions to publishing.
49:21We haven't done the second section about map services, so if you want to - we were going to wrap up publishing.
49:27Is there any more questions on publishing?
49:29[Inaudible audience question]
49:41There have been a lot of changes in the coding since beta 2.
49:44So the question, he said he was using that code for migration, but it didn't quite work.
49:49[Inaudible audience question, continued]
49:56We still had a lot of changes since beta 2, just bug fixes. I wouldn't...it should work, we use this now.
50:03Actually, all our tests - our internal, our regression tests...
50:07...used these methods, so they should be very reliable at this point. Go ahead.
50:14[Audience question] I assume you can use the Python scripting to do a batch daily process when you post it to ArcGIS Online.
50:23The question was if you can use the Python to publish daily to ArcGIS Online?
50:28[Audience question] Yeah, for a batch.
50:29In batch, yes. That's a Python script; you can put it on a schedule or you can put it in a tool, yes.
50:37That's the kind of called for is for batch updates and migration to use it.
50:46[Inaudible audience question]
50:52The question was if you have to have an organization subscription. Yes, you do.
50:56That's what enables you to publish hosted services. It's not free.
50:59Obviously, you know, there's cost for us involved as well, and we have to pass that on to you guys.
51:05Question in the very back.
51:07[Audience question] Yes, is there a way to have a trigger on a [inaudible] in the cloud...
51:14...so you can synchronize your file geodatabase automatically, or would you have to create a script for that?
51:21The question was if there's a trigger so you can synchronize automatically from your enterprise...
51:27...from your data in the cloud to your enterprise geodatabase.
51:31I'm not sure. This is really a replication question. I know you can put it on schedulers.
51:35I don't know if there's something like a trigger. Any of you guys?
51:39We're not sure. We're not like the absolute replication experts. I would refer that to Gary MacDougall.
51:48[Inaudible audience question]
51:55I'm sorry. Can you repeat that?
51:56[Audience question] How's the new file form for services, the .sd...
51:59...how's that going to affect people that have got [inaudible] server objects?
52:02So the question was how does the new file format, the service definitions, affect people that have SOEs.
52:09It shouldn't really affect them at all.
52:12We will - if you click - if you register a custom SOE in the UI and you click it on as a capability...
52:22...that will get, you know, packaged as the parameters into the SD and it will get published normally, so it shouldn't affect you.
52:30[Inaudible audience question]
52:35Do we still have...
52:36[Inaudible audience question, continued]
52:39I'll talk about that in a few minutes.
52:41The 10.1 map service is an optimized map service only so if you hooked in...
52:46...to fine-grained ArcObjects through an MXD-based map service in the past, that pipeline is no longer available.
52:57[Inaudible audience question]
53:06It's all versioned. Replication is based on versioning.
53:09[Audience question] Do you replicate when you upload from the distributed database...
53:13...do you upload from a version or from the...
53:27I guess the basic question is what's the requirement of doing that in terms of your enterprise geodatabase.
53:31One, it needs to have global IDs, and two, it needs to be a versioned enterprise geodatabase to do that replication workflow.
53:38So those are the two requirements of that.
53:43We'll look into more improvements on this, you know, probably in the future, how we can enhance syncing.
53:48This was kind of our first step into syncing.
53:52We'll look into more stuff later; also, you know, syncing file-based geodatabases.
53:59[Audience question] Can you have multiple connections to the same data store?
54:02The question was if you can have multiple connections to the same data store.
54:08Well, not exactly the same data store. You have to have at least - the user has to be different...
54:13...but it could be through the same database with different users you can, that.
54:17It differentiates with the user at the user level.
54:22Let's take one more. How about in the back there.
54:26[Audience question] In the migration process, can I reuse existing caches from previous...
54:30Yes, you can. The question was if you can reuse existing caches. Yes, you can.
54:35During the publishing process, it will latch on to an existing cache that it finds there.
54:42But there is some disadvantages to that.
54:44I think we just added that in the last week.
54:48I don't know if you've seen the caching workflow, the new one, where you can - it has that status geodatabase...
54:53...where it keeps status. So obviously, you wouldn't have that with an existing cache, but you can re-create that with a tool now.
55:00So that's one thing you would have to do to re-create status so that you can better work with that existing cache.
55:07Okay. I'm going to pass it over now to Craig and map services. Thank you for your attention.
55:13And we wish you a lot of fun with publishing.
55:20I have one. Oh, you have one? He needs one.
55:26Alright. We'll talk about enhancements in map services.
55:28So why do you publish? You publish to get map services really, right?
55:32The workflows they showed actually apply to a variety of different service types...
55:35...but obviously, most of Ty's examples involve map and feature services.
55:41My name's Craig Williams; this is Tanu Hoque.
55:43We're going to go through some of the enhancements to map services, and Tanu will demonstrate some of these.
55:49If you've been going to web API sessions this week...
55:54...hopefully you've seen some of this new functionality as exposed through the web APIs.
55:57Most of it, you actually will interact with through the web APIs, and Tanu's demos will make use of those APIs.
56:06There are some publishing aspects in terms of options to enable when we can enable optional functionality.
56:14So map services at 10.1, we have one unified map service.
56:18So at 10.0 - actually, at 9.3.1 and 10.0, you could publish a traditional map service based on an MXD...
56:26...or you could publish an optimized map service.
56:28And at 10.1, we have one map service. It's based on the optimized map service, which gives you the best performance...
56:35...but we've updated it to support additional capabilities, data types, layers, renderers...
56:40...the things that were missing in the past that may have forced you to use an MXD-based service.
56:45And we've added new extension capabilities, such as network analysis, and geoprocessing is something that...
56:52...you may have a workflow where you want to show geoprocessing results as a map service.
56:57We've changed slightly how that works.
56:59It's actually a more streamlined and successful model that allows people to publish...
57:04...geoprocessing results in a more streamlined fashion, and you should look for that when looking at geoprocessing services.
57:12The map service fits into the new ArcGIS Server paradigm where it's a pure web services-based service.
57:19Most of the interaction is through REST.
57:22There are some SOAP end points that are used, especially when using map services in ArcGIS Desktop.
57:28So added data sources and layers and things like that.
57:32XY events, that was a popular one that optimized map services were missing in the past.
57:38They still have a performance penalty because they're usually a non-spatially indexed data source.
57:44Similar issue with linear referencing events, but if you have linear referencing data...
57:48...you know that that's often updated on the fly.
57:51You want your services to be dynamic because it's purely dynamic data.
57:56For feature renderers for feature layers, we've added dot density charts; geostats, if you've used those...
58:02...and added support for new layer types - dimensions, schematics, network, network analysis...
58:08...and those help power the network analysis workflows; TIN, terrain, and tracking layers.
58:14So that's a large number of layer types that weren't supported at 10.0...
58:17...that are now supported at 10.1 with the optimized map service.
58:23So one big item that we've added to map services as a new concept is called dynamic layers.
58:29And if you've come from an ArcIMS workflow, you may have been used to updating AXL on the fly...
58:34...to update the appearance of your map.
58:36This was something you could not do with ArcGIS services until 10.1...
58:40...so this is new behavior with the map service that allows you to make changes to the appearance of the map...
58:45...the content of the map, on a per-request basis.
58:49It's an optional capability of map services. You can enable it...
58:54...or you can disable it if you don't want people to be monkeying with the appearance of your map on a per-request basis.
59:00It allows for updating renderers and symbols of the map; removing, reordering, adding layers...
59:05...changing data sources; and when you add new layers, you can add them from a list of registered data sources.
59:11So those data sources are registered with the map service.
59:17The use cases we see people doing with dynamic layers - simple updates to the map service.
59:21A lot of people just want to be able to reorder the layers that exist in the map.
59:25Thematic mapping is a big one. You want to change a renderer to show a different attribute...
59:30...maybe join a table to show a different attribute, things like that.
59:34Or adding content to the map service.
59:36If your organization has a large amount of data that's unreasonable to publish in a single map service...
59:41...you may want to make that data available via a dynamic layers workflow...
59:47...where people can add data from the registered data source to the map, on the fly, and define symbology.
59:56One special case that can be done there is you can add query layers to a map.
1:00:01That's quite powerful if you're quite gifted on the SQL side.
1:00:05Imagine the analysis you can do on a single layer adding it to a map with a defined SQL statement...
1:00:10...that will do maybe some analysis to show a result, just in the context of a map service draw.
1:00:18These changes are applied to the map service on a per-request basis.
1:00:22So this means every time you do an export map image request, you need to push this information in.
1:00:28And that's because the map service is stateless.
1:00:31So we're not preserving state of these changes; you have to actually request them each time.
1:00:37And in most cases, the web APIs will handle this for you.
1:00:41So on the web APIs, you make the modifications to those layers in the web API client...
1:00:47...and when the web API goes to make a request to the map service, it will add that information to the request...
1:00:53...so that you get the correct image.
1:00:58So thematic mapping, we really see that as a special primary use case of dynamic layers...
1:01:06...so you want to be able to change the renderer; you want to be able to change data sources.
1:01:10Like I said, a lot of times people want to integrate data from a separate table, join that on the fly to a layer in a map service...
1:01:18...and then set the renderer on a field from that joined table.
1:01:23And one key aspect that's needed to complete this workflow is you need to be able to construct the classes of the renderer...
1:01:31...based on that new data, that new field that you may have joined.
1:01:35And for that we have a generate renderer operation.
1:01:38This supports class break and unique value generation of classes and - drop my mic. Nice.
1:01:46And we have all the classification types that you're used to from ArcGIS Desktop.
1:01:51So any of those classification types that you may have authored maps with in the past in ArcGIS Desktop...
1:01:56...you can generate via this API.
1:02:01So before we talk anymore, we're going to turn over to Tanu...
1:02:04...and he's going to give a demo of dynamic layers with thematic mapping.
1:02:13Thanks, Craig. So here I have a plain vanilla map that shows all the counties of the United States...
1:02:19...and those are just file geodatabase.
1:02:21I have only two layers. One is these counties, and the other is just a shadow to make it look nicer.
1:02:27On the other hand, I have a file - I'm sorry - SDE that contains...
1:02:32...hundreds of feature classes and tables inside and outside feature datasets.
1:02:38So what I would like to do is that I would like to publish a map, a map service...
1:02:44...which will allow the client to modify the renderer of these counties...
1:02:49...as well as add some layers specifically from this particular file geodata - feature dataset, all these feature classes.
1:02:58And then down below, I have an attribute table of that county data that shows all the attributes...
1:03:03...the demographic information available for each county.
1:03:06So I'll just start that publishing process as you have seen many, many times today.
1:03:11I'll just follow the wizard, give it a name, and then click Next. If it will work. Yep.
1:03:21And then I get this service editor.
1:03:24Here I need to point out one quick thing is we have an Advanced button here, collect advanced properties.
1:03:31I know - have any one of you had these issues or nightmare while you're trying to change the properties...
1:03:37...like maximum record counts or maximum image size and height? Anyone?
1:03:42It was a nightmare, right?
1:03:43But this time at 10.1, the publishing team made it simpler so you can do it from UI.
1:03:49You don't need to restart the whole SOC and SOM when editing. I did that in that existing service.
1:03:54So kudos to the publishing team. Can I have a round of applause for them? Yes. Nice.
1:04:02So now, to enable the dynamic layers capability, you need to go to the mapping page of this editing window...
1:04:09...and then click this check box.
1:04:11This will allow you to reorder the layer or change the symbology of a layer.
1:04:17And then I would like to add -- allow my clients to add feature classes or tables on the fly to my map...
1:04:24...so in that case, I will need to register workspaces.
1:04:27So there is a one-to-many relationship between a map service and a workspace, and we support all different types of...
1:04:33...actually four different types of workspaces - a database, which is SDE; then file geodatabase; shape-...
1:04:38...or a folder containing shapefile or folder containing raster.
1:04:42So here I will just create a - point to one of my SDE connection files, and I give it a workspace ID name...
1:04:52...which is very important. I will get to that one in a few minutes.
1:04:56And then we have a little check box here called Lock Version.
1:04:59So if you don't want your client to switch between versions, you can lock it.
1:05:03Otherwise, by default, you can actually switch between different versions.
1:05:07And then I will add another file geodatabase connection. I will call it fgdb_, you know, counties. That's it.
1:05:17I'll go to... Okay. There it is. That's my file geodatabase.
1:05:24Click on it, save it, alright.
1:05:27Now click it, I register my workspace, and as you know that we have to go to the analyzer process...
1:05:32...and as I analyze it, it will find that those two workspaces that I just added are not part of the data store.
1:05:38So I need to register them.
1:05:39So I can right-click and say register it.
1:05:42So this time, registration is much simpler than what Ty showed before.
1:05:45All you have to do is click OK and it'll get registered. Okay.
1:05:49Once it's done, you can just click Publish and it will get for the packaging and it will be published.
1:05:54I will skip that process because it will bring up an awkward silence, and I cannot sing to entertain you guys.
1:06:02Okay, so move on to the nice demo now, the fun part.
1:06:06So this is my application, web application written on Silverlight.
1:06:10So I can see the same map that I just showed in Desktop, and if I want to change the color, I can do that right away.
1:06:18I can zoom in, pan; it looks good.
1:06:20If I want to switch to a different renderer type, like different symbology, I can do that, right?
1:06:26Go back. Looks nice. And I can switch back to default.
1:06:30Default is how you publish it, the symbology that you used with that layer when you published it...
1:06:38...and the way you set it to default is set the renderer to null from the web application.
1:06:44Now the next part is I would like to add some layers, right?
1:06:47So I would add cities. It is coming directly from - you saw that when I published my MXD, I did not have that layer...
1:06:54...so I just added it dynamically to this map service per request.
1:06:59And I will just add the roads here. Looks ugly.
1:07:03If you think that's ugly, you should see my other maps. It's horrible.
1:07:07Okay. Now, since it's covering all my points underneath it, I would like to bring it up...
1:07:12...so that's what Craig mentioned, that we can reorder the layers.
1:07:15All I have to do is move it up and see those points are drawn.
1:07:19So this is, again, it's not client side drawing; it's happening on the server side.
1:07:23Just to prove it, I can show you that here is the request that goes out, and result comes back...
1:07:30...that image that comes back to me with all the layers on it. Okay.
1:07:40Now that we're done with this adding new layers on the fly, now the funnest part is to...
1:07:46...the thematic mapping that Craig just mentioned.
1:07:50So the county layers have lots of demographic information, and this one I can say...
1:07:55...Okay, I would like to render all my county polygons with total population.
1:08:00So this is, again, that thing that Craig mentioned before that we have a utility function called generate renderer.
1:08:05We could call that one to get the renderer, and I'm just applying it to that.
1:08:11Without this one, you would have probably returned like a hundred of lines of code on the client side...
1:08:15...download the data - so there's an overhead also, download the data on the client side - and then postprocessing that one.
1:08:22If I don't like this color schema, I can go and change a different color schema.
1:08:27If I want to go to different, show me to the different attribute values, I can do that.
1:08:31And if I think that, Hey, this method, the classification method, is not good for me, I can change it to different ones...
1:08:36...like natural breaks. Do that, and change the number of class breaks for it, right?
1:08:43It's very powerful, isn't it?
1:08:45Okay, now that we can do it, it's still good...
1:08:48...but the thing that still bafffles me sometimes is that I'm still stuck with the attributes that have been published by that author.
1:08:57What if I have some additional information?
1:08:59Like in my case, I have very simple information; the publishing changed from 2000 to 2010.
1:09:05I save that as a CSV file. I would like to upload it, and I will show you it in a second.
1:09:09So what I do, I can click on it, the Upload button.
1:09:14I'm choosing that CSV file. That's because Excel is not supported in the server right now at 10.1.
1:09:19I click on it. What's happening behind the scenes, I'm calling a geoprocessing function...
1:09:23...which is converting my CSV file to a file geodatabase table.
1:09:26And all the geoprocessing service has an upload operation today at 10.1.
1:09:31I'm calling that upload, which is uploading my CSV file to that server...
1:09:34...converting it to a file geodatabase table, and joining it for my optimum performance.
1:09:39And see how fast it was.
1:09:41So I can see all the population change happened in that last decade, and I see that the green color...
1:09:47...shows where the population declined, and the red is pretty much where population increased.
1:09:51And that's me too. I contributed that one.
1:09:55Okay. So how did it work?
1:09:57So let's take a look at the REST side.
1:10:05We defined a list of the layer, that dynamic layer's options in that export operation.
1:10:13We defined the source.
1:10:14Here I'm saying I want to use one of the existing layers.
1:10:17The layer ID is 0 and the type is map layer ID, followed by the drawing info.
1:10:22That's all I did. I clicked the post, and I can see that got refreshed.
1:10:27If I want to change the color, something else like the first class, the first one, to red...255...sorry about that...
1:10:44...and repost it, I'll see that the first class of the class breaks renderer changed to red.
1:10:49What happens when I do that with the - when I want to add a new layer, right, from that registered workspace?
1:10:59This is how it happens. The web application sends the request with this type of source this time.
1:11:05It says type is a data layer, and this is the workspace ID.
1:11:09Remember the workspace ID that I'd given during the publishing process?
1:11:12And here is my name of the data source, and that's how it knows that it needs to add a new layer from that workspace.
1:11:24Now, that one thing I mentioned before is the geodatabase versioning, right?
1:11:27So sometimes this require, especially [inaudible] companies that will do the parcel editing...
1:11:33...that you want to switch between different versions.
1:11:35In my case, I'm zooming in to my data in Naperville, Illinois, and I see a big junk of parcels...
1:11:41...chunk of parcels there that I would like to see what happened after the subdivision.
1:11:45So I will switch between the versions, and I see that it shows up right away.
1:11:49So it's not two different layers; it's just that we are switching between two versions called base and the subdivision.
1:11:54And the way you do that in there, this is the Silverlight code that I have, it's very simple...
1:12:00...and the logic is similar across all APIs.
1:12:03All you do from the first place, you create a DynamicLayerInfos from the LayerInfo function.
1:12:09Add some collection, then you loop through all the DynamicLayerInfos in that collection, set the version name...
1:12:15...and followed by setting the dynamic layers back to that dynamic map service layer object and call the refresh.
1:12:23That's it, as simple as that.
1:12:26Now, another thing I do want to talk about is the time query layer that Craig mentioned, very powerful one.
1:12:33So here I have a map service that contains - are you familiar with this time series data that you have points in a feature class...
1:12:41...but your time series, they are taken for the same location over the period of time but saved in different tables.
1:12:48And in this particular case, the problem is that the time was not - I'm sorry.
1:12:53The value was not collected at the regular intervals; it's very irregular.
1:12:57So there is a problem that how would you show that?
1:13:02And the weather research people, time series, they would like to know the maximum value...
1:13:07...minimum value for a period of time.
1:13:10Right? Not for one particular instant.
1:13:12So here I'm seeing all the locations here, but I would like to know the two-month average for that site...
1:13:18...the BOD value, which is the biochemical oxygen dissolved value for that.
1:13:23If I want to see the maximum value, I can do that.
1:13:25I can enable the time, then I'll go across the different state, and I could see this changes.
1:13:31So we are taking advantage of this called query layer, and we are sending a query layer request...
1:13:38...and this is how we define the query layer here.
1:13:41Setting the different type, there's a data type again; source is the query layer type; workspace ID; OID field, different type...
1:13:48...and followed by a big query that's a SQL statement.
1:13:54Finally, I have this query stat. That is a new operation; it's nothing to do with dynamic layers.
1:14:01It's a new function that allows you to get some basic stats from a particular layer, existing layer or dynamic layers.
1:14:07So here I'm zooming in one of the places, called Dublin, Ohio, and you can see that if I enable this guy...
1:14:16...as I zoom or pan, I can see my graph's updated.
1:14:19That's because I'm calling that query stat operation to compute the summation of all area that belongs to each land-use group.
1:14:28So I can use that to filter it out as well.
1:14:33Like I can say, Okay, I would like to do only for the residence...
1:14:37...and the land-use type is residential, add those cells, I can do that.
1:14:41If I want to do this, like if I want to group by another field here, like the political units, I can do that as well. Alright.
1:14:56Okay, last thing doesn't work, so I'll hand it back to Craig. I'll see what's going on.
1:15:00So that proves that I was using the real data or the real map, real demo. Thank you.
1:15:08Alright. We're running a little short on time, so we'll just finish up.
1:15:14If you've been coming to this conference for many years and you're quite familiar with the web APIs...
1:15:19..they've been showing thematic mapping with the web APIs for a long time and really pushing that approach.
1:15:24And we just showed server-side thematic mapping; you may be wondering, Which should I use?
1:15:29Well, really it depends on the situation.
1:15:32Many of you have tried pushing thematic mapping into the client side with feature layers in your client-side graphics...
1:15:40...and have not had good performance with it.
1:15:43And that depends on the number of geometries you really have in your map, how complex they are...
1:15:48...and kind of the scale that you're looking at.
1:15:52And if you have a large number of small polygons that are pretty detailed over a map extent, it may bog down in your client...
1:15:59...and that's a case where you may want to use the server-side thematic mapping via dynamic layers.
1:16:06Each approach has trade-offs.
1:16:09Generally, client-side features, the client-side approach, scales better on the server end...
1:16:15...'cause you're just executing queries on the server, whereas what we're showing is actually drawing on the server.
1:16:20But if you're downloading a huge payload of data, that may take time as well...
1:16:24...and generally the images that you get back from a dynamic layer are faster.
1:16:27So you need to play with each trade-off and see which one works.
1:16:31The generate renderer functionality can be used with either approach, so if you're ever trying to roll your own...
1:16:37...classification in the web API, you can now use the generate renderer functionality that Tanu just demonstrated.
1:16:43So if you need more information about dynamic layers as a whole, we urge you to see the ArcGIS Server Help.
1:16:49There's a bit in the ArcGIS Server Help that introduces the concept as we have today...
1:16:53...and then the details into the JSON and all that are in the REST API help.
1:16:59And then finally, Tanu showed the final demo there was some of the query enhancements we've made to ArcGIS map services...
1:17:06...so we've added support for order by, output statistics, which is what he was showing.
1:17:13You can group by with those statistics, and the statistics that we've added are listed here...
1:17:18...count, sum, min, max, average, standard deviation, and variance.
1:17:23There's some additional options with the query method on a map service.
1:17:27You can now optionally return Ms and Zs, if your data supports it.
1:17:31You can query a specific geodatabase version.
1:17:34Tanu showed drawing a specific geodatabase version, but that also applies to queries as well...
1:17:39...if you need to just query features from a specific version.
1:17:43And one of the classic issues with query with map and feature services is you hit the max record count on a query...
1:17:50...and often, you would not know whether or not you hit the max record count.
1:17:55You would just have to guess if the number of features returned equaled what was set as your max record count.
1:18:01We now actually include a flag in the record set when we return it that indicates whether or not you hit the max record count.
1:18:09So at that point, we're done with today's presentation, and since we're running out of time...
1:18:14...I'd ask you to come up to answer questions, 'cause we've got to clear out of this room for the next presentation. Thank you.
Publishing and Using Map Services with ArcGIS 10.1 for Server
Stefan Balbo, Ty Fitzpatrick, Tanu Hoque, and Craig Williams discuss the new service publishing enhancements in ArcGIS 10.1 for Server.
- Recorded: Mar 29th, 2012
- Runtime: 1:18:22
- Views: 2814
- Published: Apr 25th, 2012
- Night Mode (Off)Automatically dim the web site while the video is playing. A few seconds after you start watching the video and stop moving your mouse, your screen will dim. You can auto save this option if you login.
- HTML5 Video (Off) Play videos using HTML5 Video instead of flash. A modern web browser is required to view videos using HTML5.