00:01Alright, good morning and welcome everybody to this early session on publishing and using maps.

00:07So Tanu put out the survey questionnaires, so please fill them out. I'm supposed to remind you.

00:13So we're, this morning, we're really honored and excited to kind of be able to share with you the hard work...

00:20...that our teams have done over the past year and a half on publishing and map services, and we hope you'll enjoy the session.

00:29We're going to have two separate parts to this session this morning.

00:33The first one is about publishing and using map services, so we kind of take a deep dive into publishing...

00:39...and I'm going to present this together with Ty Fitzpatrick; he was the lead product engineer on publishing.

00:44My name is Stefan Balbo, and I was the lead developer for publishing.

00:49We'll have a little question section after this, and then we're going to the really cool new enhancements of map services...

00:56...which is going to be presented by Tanu Hoque; he's the product engineer for our map server...

01:01...and Craig Williams, our lead product engineer for mapping.

01:05Okay, now let's dive right into publishing, 'cause we have a lot of things to cover.

01:13And we've talked a lot about this DevSummit about publishing, even the last one; it was a really important thing for us.

01:20It's actually one of the driver projects for 10.1.

01:23So why is this so important to us?

01:24Because, you know, we live in a world with blogs and Twitter and Facebook...

01:29...and everybody wants to put their information out to the web or share it with others.

01:34It's the same with your GIS information.

01:36The content you created on your desktop or in your organization, you want to share it with people that you work with...

01:44...with your colleagues within your organization, or even with the public with the open web.

01:50But not only do you want to share your organization, it's really expected from you that your work is available to others.

01:57Information access has to be transparent today.

02:01So that's why we think that, you know, publishing is critical...

02:04...and we spent a lot of effort in the last year and a half developing this functionality.

02:10When we talk about sharing, we actually mean two things. The first one is we can share as packages.

02:16This is kind of...we had that since 9.3.1, where you create map packages, and we enhanced that in 10 and 10.1.

02:22Now you have geoprocessing packages and tile packages, et cetera.

02:27And this is where you, you know, zip up your work into a package and you share it with another professional...

02:32...who also has to have ArcGIS Desktop.

02:36Sharing as a service is the second way to share, and this is publishing.

02:39This is where you really give out your work to everyone.

02:43And we had a vision for 10.1 where we wanted to make it a lot easier for you to share, you know...

02:49...GIS resources like your maps or globes to everyone.

02:54And the first thing we wanted to accomplish was we wanted to have a unified sharing experience.

02:58That means no matter what service you published, it should all kind of work the same way...

03:02...and it should be, you know, a unified experience.

03:06We also wanted to extend the analysis we kind of started with 9.3.1 for map services...

03:11...and expand that to all services, all SOEs, to really make you successful publishers for publishing your services.

03:19And the third thing was we really wanted to, you know, enable sharing to servers in the cloud...

03:24...and as well to ArcGIS Online hosted mapping.

03:28So these were...this was our vision for publishing.

03:33So now let us talk a little bit about these three distinct workflows, how you can publish services from the desktop.

03:40And the first one is the classic ArcGIS Server is running in your enterprise on premises.

03:46This is how you have been publishing since 9.0, I believe, since this came out.

03:51And, you know, it's basically we call this publishing to a server on premises...

03:58...where the server and the publisher machine share the same data.

04:03The second use case or scenario we want to look at today is when you publish to an ArcGIS Server running in the cloud.

04:09You still have an ArcGIS Server, but it's not running on premises; it's running on an EC2 instance in Amazon.

04:16The third case we want to look at, and that's really the easiest way to share; this is the easiest way to publish a service...

04:23...is you directly publish to a service that's running in ArcGIS Online.

04:26So this is a Software as a Service scenario; there's no ArcGIS Server running there. It works very easily.

04:35So we're going to look at this one first...

04:37...and Ty is going to show us a demo now about sharing services as hosted maps on ArcGIS Online. Go ahead, Ty.

04:43Okay. You need to switch over. Oh, sorry. Thanks.

04:47Okay, so in this first scenario, like Stefan said, I'm going to publish to my organization on ArcGIS Online.

04:54And right now, I'm signed in to ArcGIS Online as a special user. I'm an administrator of my organization.

05:00So if I go into ArcGIS Online and I'm signed in here...

05:03...if I go into the My Organization tab, group, or whatever you want to call it...

05:08...you can see all the members of my organization here.

05:12And my name is - where's Ty? It's in here. I'm the administrator, right? No, Paul's the administrator right now.

05:22But I'm the administrator of this group also, and I'm able to publish services as well as administer content of this organization.

05:33So to publish content, from the File menu, go Share As a service, and I'm allowed to publish as a service...

05:42...and I'm going to choose my hosted services.

05:45And here's my service name. Mark it by DS for DevSummit.

05:52And when we get into the service editor, some of you have seen this already in the plenary...

05:56...this is where we can configure all the parameters of our service prior to publishing.

06:00And this allows us to kind of get any best practices that we want to maybe accomplish...

06:04...with our map prior to publishing this service.

06:06So from the Capabilities tab, the two types of services that you can publish as a hosted service...

06:11...are tiled services and feature services.

06:13So in this scenario, I'm going to publish a tiled service.

06:16You go to the caching page, you can see that this is the visible scale range of my map that I currently have right now...

06:24...based on the scale suppression that I've set with the layers in my map.

06:27And I can adjust this if I want.

06:29If I adjust this outside of this range right now, I probably wouldn't get any tiles at that scale range, 'cause the map's not even visible.

06:36I have item information. So all services have item information, or metadata.

06:40As well, I can share this with other groups or with all the other members of my organization.

06:46So let's publish this map. So what it's doing is...

06:50This is a pretty simple map with one layer in it, but if you had data coming from an enterprise geodatabase...

06:55...or a shapefile or other types of data sources, it would package that all up together, with the service parameters...

07:01...that I've defined here, as a single file, transfer that up to ArcGIS Online.

07:04Everything you need to create that tiled service is defined within that package to create that service.

07:10Okay. So while this guy's publishing, let's take another map, and this is going to be a feature service.

07:17So this is all locations of where I want to distribute Hangar 24 beer across Southern California.

07:26And the first map was beer consumption for where you can consume...

07:32...how much consumers are actually drinking beer based on block group.

07:37So we can kind of use these two maps to kind of mash them up.

07:40So, again, from the File > Share As menu, we'll publish this as a service, using my organization Dinoco...

07:53...and in this case, I'm going to publish, instead of a tiled service, I'll publish a feature service.

07:59So from the Capabilities page, I'll change that to a feature service.

08:02Again, I have item description for this, and so forth, share it with the other members of my organization as well.

08:08So here we've got a little issue, and this is - we ran an analyzer; it said "Basemap layers cannot be published."

08:14So we don't allow you to publish a basemap layer with a feature service.

08:18So in this case, I can remove this guy, because I don't need him.

08:26And we want you to be conscious of the things that get removed from the map, so we don't just remove that for you.

08:32We actually kind of make you go through that process when you publish.

08:37So now we're going to publish this feature service.

08:39Let's go back to my tiled service. So that map was successfully published, and if we go to the Catalog window...

08:45...you can see there's my service, under My Hosted Services, and there's my organization.

08:50You can go to View Cache Status, and it'll get an update of how much cache has been built based on that service.

08:57Let's go to the details, and you can see, based on each LOD, the completed level of tiles and the percentage complete...

09:03...and here's the job that's actually being executed.

09:06So at this point, 80 percent of my tiles are created, and so forth, and you can see them starting to be completed.

09:13If you want to get an update, just click the Refresh Status, and you'll see an update of more tiles being generated.

09:21See if this guy's finished.

09:24Okay. So the feature service also got finished.

09:27Now if we go on to ArcGIS Online, and we're going to go to a map, to my organization.

09:32Actually, let's go back here.

09:36You can see under My Content, the two maps that I just published, Hangar24_DS and beer by ZIP code...

09:43...and now we can look at them both in a map.

09:47So let's search based on my content.

09:57Where is it? It's in my organization. There we go. And probably getting too much.

10:10They just haven't gotten into the search yet.

10:14So we could take a look at that service.

10:16Let's look at the Hangar24 one; it hasn't quite completed its process of getting into the organization; I've got a little...

10:26So here are all the beer locations, and again, this is a feature service, so we can edit these points, right?

10:32So if I edit, you can see, based on the unique value renderer, created those templates for that particular feature service.

10:38So I have all the existing ones; I have one new one.

10:41If I wanted to create another one, simply click on the map...

10:44...and you can see that I could put in the attributes and so forth and edit that as a feature service.

10:49So I'm editing features in my hosted feature service on ArcGIS Online.

10:56So this is the two ways that you can host services on ArcGIS Online; one is a tiled service and another as a feature service.

11:06Switch over. Back? No. You on A? Yeah, alright, thank you.

11:25Well, thank you, Ty. And let's recap for a moment what we saw here, what was going on.

11:31So we had our desktop that Ty had with, you know, some local maps and with local data...

11:37...and he wanted to share this out online as a service.

11:40He doesn't have any server and has no...doesn't need to know anything about, you know, IT or infrastructure or proxies.

11:47All he has is an ArcGIS Online account in the cloud.

11:52So what he started off is bringing up that service editor and configuring the service...

11:56...you know, like setting all the parameters for the service, and actually we have pretty good defaults.

12:00So on most cases, you should pretty much be able to just fly through.

12:05The next thing that happened, when he hit Publish, well, he got an analyzer error in the one case.

12:10But the first thing that happens is we're staging.

12:13So this means we're creating the service definition, and that's our new exchange format for publishing a service...

12:19...and I'll talk a little bit more about that in a moment.

12:21But what's happening there, we're basically taking the map, the data, and the service configuration...

12:27...and put that all in one file and zip that up.

12:29And then when we are publishing, this is getting uploaded to ArcGIS Online, where it then, you know...

12:36...to publish, it gets actually, you know, unzipped; unloaded; and if it's a tiled service, the tiles get started to cook...

12:43...and they're in a feature service, the features get loaded into a cloud database.

12:51So what Ty did, he created basically a tiled map service, which was the first one, and then a feature service...

12:58...and then he simply mashed them up in a web map and it was ready to go, and everybody can see it.

13:03He could share it with his organization or publicly, whatever he desired.

13:09We think this is really cool. We think it's a great way for me, as a Desktop user, without, you know...

13:15...having to know Server or having to own a server, to publish my maps.

13:20It's really the easiest way to share.

13:23Like I said, you have no server to buy, to install, or to maintain.

13:27It scales automatically. I mean, your map could be so cool it could be blogged on by CNN...

13:33...or shown on national television, and it would scale. Go ahead.

13:37[Inaudible audience question]

13:41They'll be coming out with 10.1.

13:48And you have no firewall or IT issues.

13:51There's some limitations. Currently, we only support tiled map and feature services, but we're definitely working on more...

13:57...and the data is private to each service.

13:59That means, you know, if you would have two map services who would work on the same data...

14:04...you would actually have to upload the data twice.

14:07So this is great if you are a Desktop user.

14:09You're not, you know, don't have a server and you want to create public-facing services.

14:14Then, you know, hosted mapping is great for you.

14:19Okay, now let's talk a little bit about service definitions.

14:22So this is our new file format for publishing in 10.1.

14:26It essentially replaces the MSD that we had in the past few releases.

14:30But it's kind of expanded; it works essentially for all services.

14:34So whatever service you publish, there's going to be a service definition, your map, geoprocessing, globes.

14:41And it's really just, you know, you can...it's a 7-Zip archive, so you can look inside there...

14:46...and you'll actually, for a map service, find an MSD in there.

14:50You'll also find, you know, the same content of a package if we package data with it, and then some other information.

14:57One cool thing that's new in 10.1 that we've done, we're actually when we're staging, we check and you have a map service...

15:04...we check what fonts your maps use and we check if that font is available on the server...

15:09...and if it isn't and it's legal to embed that font, we'll embed that as well in the service definition.

15:14So your fonts will show correctly up on ArcGIS Online or on any other service you publish to.

15:20Also the service configuration, all the parameters of your service, your caching information...

15:25...that's all in the service definition as well as data if it needs to be put in there.

15:31Again, this gets uploaded to the server when you're publishing.

15:34And in Ty's workflow, he had published right away; we didn't even see the service definition.

15:38It basically got created, gets uploaded, then kind of gets deleted.

15:42But you can also save that out, the service definition, and publish it later either from Server Manager or from Catalog.

15:51Okay, now let's walk on to our next scenario, and that's actually an ArcGIS Server running in the cloud.

16:00We were going to have a demo on that, but we're going to skip that demo 'cause we actually already shown it on the plenary...

16:05...and we are a bit strapped on time, but there is a video for this on videos.arcgis.com...

16:11...where Ty put a video on about publishing map services to an ArcGIS Server in the cloud where the data gets copied.

16:17But I'll talk a little bit about that workflow, but it's very similar to the one we saw just now.

16:22The only difference here is when we start off is that we actually have an ArcGIS Server on an EC2 instance in the cloud.

16:29So we bought an ArcGIS Server, and we installed it on a server in the cloud.

16:34Again, on our desktop we have a map; we have local data, and again, we would create a service configuration.

16:41We'd stage it, it packages up all the data, map, and parameters, and - oops.

16:46And when we publish, it gets uploaded to that machine in the cloud.

16:50In that case, it really runs in ArcGIS Server, so I have the full power of ArcGIS Server there...

16:56...and we could create dynamic map service that we then can use in Desktop or any other client.

17:05So in this workflow, the improvements if you publish to an ArcGIS Server in the cloud...

17:11...I think before, you had to kind of remote desktop in, copy the data manually, and then, you know...

17:16...muddle with tools like MXD Editor to kind of, you know, fix up the paths.

17:21So this would be a very simple workflow, where just the same way as it worked for hosted...

17:25...it just packages everything up, uploads it, and publishes it.

17:28There's absolutely nothing you have to do, so it's very, very easy.

17:32And you also get the benefit of the, you know, comprehensive analysis that we put in 10.1.

17:37When is that great to use?

17:38When you have public-facing services that are outside your firewall but you do need the full functionality of ArcGIS Server.

17:47Alright. Now let's move on to our next scenario, and this is actually kind of the classic publishing scenario...

17:54...that we've all known since 9.0 where our ArcGIS Server sits on premises.

17:59So Ty's going to put a demo now on sharing services to ArcGIS Server in the enterprise...

18:04...and show us all the improvements we made for 10.1 in that area.

18:10Okay, just to kind of wrap up the last demo.

18:12I got a little excited and didn't wait long enough for the two services to be published.

18:17Here's my Hanger24_DS and the beer by ZIP code DS, [proofer: name is BeerByZipSoCalDS] the tiled services...

18:24...and of course, you can click on the attributes and see them and see the field values, and so forth.

18:29So that's just kind of wrapping up that last one, 'cause I didn't quite complete it.

18:35So here's a map, as Stefan said, now we want to publish within the enterprise, within the organization.

18:42We don't want to copy data or anything like that.

18:45So here I have some electric facilities, and I also have a cached map service behind it as a basemap.

18:51So I want to kind of overlay this operational data with my cached map service.

18:58So here I have the basemap, and I can remove that guy, 'cause I don't want to publish that...

19:04...and I just want to publish this as a dynamic map service.

19:10Then I want to reference that as my enterprise geodatabase.

19:15So we're going to publish to my local enterprise - my local ArcGIS Server, and this is the service name that I'm going to use.

19:24Same type of workflow as publishing to a hosted service, except I have more options, obviously...

19:28...when I published to a local server.

19:31So we can analyze for this map, just to make sure we don't have any issues with the map.

19:36I don't have any. We have a couple things that we could take a look at.

19:40Data source is not registered with the server on the database.

19:44The data source is not registered with the server and the data will be copied.

19:48So by default, we would copy the data to the server, and that's kind of the easiest way to publish.

19:53It's a little bit of a reverse from what you're used to, but we've added all these new workflows with publishing.

19:59So to ensure that the server can see your data, we want you to register your data with the server...

20:04...and that allows us to validate and make sure the server can see it prior to publishing.

20:10So we're going to register that server.

20:14[Audience question] I'm sorry, registering the server, or the data?

20:16Sorry, I'm registering the data, so I'm registering the connection of my enterprise geodatabase...

20:22...so that the server can validate that, basically, workspace connection.

20:26[Inaudible audience question]

20:29I'm sorry?

20:30[Inaudible audience question]

20:37Yeah, yes. Yeah. It will.

20:40[Inaudible audience question]

20:43That's typically the way you would do it.

20:48It would run off of Windows authentication, like if you're logged in to SQL Server using your OS authentication...

20:55...if that's what you mean, it would work that way.

20:56[Inaudible audience question]

21:03That's to run ArcGIS Server as a...yeah.

21:06So your ArcGIS Server would need to be able to see that SQL Server database, right, that user.

21:15So I'm going to register that guy in the data store of the server, so when I analyze now...

21:21...it's not going to want to copy anything to the server.

21:25So if I go back to my ArcGIS Server connection that I've got here, and I go to Server Properties...

21:33...you can go to the data store and you can see, in the data store properties, I've got that registered data store.

21:39So if I look at the properties of it, you can see it's got the name, and as the publisher and the server...

21:45Everything's on the same machine, but assume that this is two different machines...

21:47...where the publisher is on one machine; that's the connection to my data...

21:51...the server's also got a connection to that data.

21:53We're both looking at the same data, and we're going to reference that.

21:56So the publisher and the server are the same.

21:58We're looking at the data the same way, so we're going to reference that data in place.

22:04So now when we publish this, it's going to take the service parameters like I did, similar to the way I did with ArcGIS Online...

22:10...except it's not going to copy any data; it's going to reference all that, the enterprise geodatabase, in place.

22:16So as I make updates, that service will see those updates.

22:19[Audience question] Updates to the geodatabase?

22:21Updates to the geodatabase, yeah.

22:24[Audience question] So you do not have to reregister the service.

22:27You don't have to reregister anything, right.

22:29There's another workflow that Stefan talked about just...

22:32The one that you saw on Tuesday when I did it on stage, you publish with copying data to the server...

22:39...so if you make edits in your enterprise geodatabase, you won't see that.

22:43But the next workflow, I'll show you how you can do that.

22:47But there's publishing with copying data to the server, there's publishing with referencing the data within your enterprise...

22:52...and there's another workflow that we'll go into in a minute.

22:56So that was published successfully. Turn all this off, and you can see my service.

23:03And we'll bring it in, and that's my dynamic map service referencing my enterprise geodatabase.

23:11Switch over. Alright, thank you, Ty. No. You've got to go on A.

23:20Alright, so let's recap this a little bit 'cause there's a few new concepts here that we haven't...

23:26...we kind of introduced with 10.1.

23:28I mean, the workflow is classic, right?

23:30Your server and your desktop, they both kind of have the same access to the data like Ty showed.

23:36So, again, you know, here we start off with a desktop, we have a map...

23:40...and it references data in some enterprise geodatabase here called Ella.

23:45On the server, we have an ArcGIS Server that's running on our premises and has this list of data stores...

23:50...where you can, you know, register data stores with.

23:53So what Ty did, you know, he created, started to publish, created a service configuration...

23:59...and while he did that, he ran into that analyzer warning, and there he could right away, from there, add a new...

24:04...register that data store that his map used with the server.

24:09And so Ella basically gets put onto the server so the server knows about it, and while you register, it gets validated.

24:15So every server in the site will check that it can access that data, and it's actually really valuable...

24:22...and that's the big improvement, 'cause that's how many map services or other services will fail...

24:27...is 'cause the server doesn't have access to the data because of...

24:30...you know, rights issues or, you know, you forgot to put it on one server.

24:37You have many things. That's the very common source of failure.

24:40So once we registered that, it also, our ArcGIS Server references that enterprise geodatabase Ella...

24:47...and then it was a simple matter of creating the service definition, this time without the data...

24:52...because, you know, we know that the service can access the data...

24:56[Inaudible audience question]

25:11You would have to - yeah. Every server has its, or every site has its list of own data stores...

25:16...so if you would take that SD, would publish it to another server, you'd have to register the data store.

25:21[Inaudible audience question]

25:22Yes. But you only have to register it once, so furthermore you use that data store...

25:26...it will of course remember it. You won't get the warning message; it'll just work.

25:30It's basically a onetime thing.

25:34So like I said, the data is not part of this right now because we determined that it's accessible by the server...

25:41...and you can actually mix and match.

25:42You can have data that, you know, in your map that is in a data store that will not get copied...

25:49...and you can have data like a local shapefile that will get copied because, you know, it's not accessible from your server.

25:56So it's actually a really strong workflow.

26:00So we'll publish, and Ty created a dynamic map service that he later then used in Desktop.

26:08So what I kind of did, the cool things about, you know, the improvements we made in 10.1...

26:13...we really wanted to make you successful publishing to shared data by adding this validation step.

26:20We also, you know, have really comprehensive analysis...

26:23...and still just in ArcGIS Server in the enterprise you can get that benefit of really simple publishing...

26:29...where just the data gets copied up.

26:33When do you want to use an ArcGIS Server in enterprise?

26:36Well, you need the full functionality of ArcGIS Server; all the services are available - GP, image service - everything.

26:42You need full control over all your hard- and software...

26:45...and it's kind of the only way if you want to publish services on live, shared data.

26:51Then you use ArcGIS Server in the enterprise.

26:54So let's talk a little bit about analyzers.

26:57So we started that concept in 9.3.1, and we think it was really powerful in improving your services...

27:04...because not every map that you just publish makes a good service.

27:08Actually, it doesn't at all. And that was one of our biggest problems with the server before 9.3.1.

27:14It wasn't that the server was necessarily slow, but we just didn't know if the map was good or not.

27:20So now we've really totally expanded that, and we have a total of 276 analyzers running. It's mind-boggling.

27:27And just 133 for map services. So it checks for everything; it checks for every SOE that everything is correct...

27:35...also GP services have analyzers; feature services, 22 analyzers for feature services.

27:42So our goal and the statement we want to make is if you analyze, if your analysis works, you will get a successful service...

27:51...and that's our goal we have, you know, that you can always publish successfully.

27:58So now next let's talk a little bit about this data stores more, 'cause it's kind of, you know, this new concept...

28:04...and we get quite a bit of questions.

28:07So again, Server has this list of registered data stores, and data stores can be of two types.

28:12It can either be an enterprise geodatabase, where you can register all your SDE databases, your SQL or Oracle...

28:22...or it could be a folder where you can use all your...register all your file-based data, file geodatabase, shapefiles.

28:30I want to mention one tip. Try to register the folder as low as possible to get the maximum benefit of validation.

28:40Then as we saw there, the access to the data store gets validated during registration...

28:45...and the cool thing about it, it actually checks every single machine in the site that it can access the data.

28:51'Cause it was also one, you know, previously a big point of failure that, you know, you had access from one machine...

28:57...of the site, but your other machine couldn't access the data, and you basically...

29:01...it's very hard to debug that and know about that.

29:05There's a really great help topic about it, you know, about registering your data, so I'll point you to that as well.

29:13But let's go a little bit more in detail.

29:14And we have basically two main workflows of registering data, and the first one is basically what Ty just showed us...

29:21...where the publisher and the server have access, have the same access to the data.

29:25And that can be your folder, your file geodatabase, or your enterprise geodatabase.

29:31And the key of that workflow is we want to make sure that it's accessible by the server, and this helps you...

29:37...you know, publishing, with the traditional way of publishing.

29:40So this is the one way, the one reason why you want to register your data store.

29:44There's another workflow, and Ty's going to show us shortly a demo about that, but I'll kind of introduce that here...

29:51...where actually the server and the publisher's data store, they're not the same; they're different.

29:55So I basically, on my publisher's machine, I point to local data in an enterprise geodatabase or in a file geodatabase, et cetera...

30:04...and I have a copy on an ArcGIS Server that has the same data, either used by replication...

30:10...or where I copied the data manually before publishing.

30:16So in that case, when we publish your map up into that ArcGIS Server...

30:20...we actually change the data connections out in the map and point them to that new data.

30:25Basically kind of what MXD Editor used to do, where you kind of had to go by hand and change the paths.

30:31So it does this for you. And this is a really powerful concept...

30:34...and you can get some great workflows going with that, and that's what Ty's going to show us next.

30:40He's going to publish to an ArcGIS Server in the cloud with replicated data.

30:47Okay. Is my map up? Okay.

30:50So I've got a connection to my Amazon, my ArcGIS Server running on Amazon.

30:55You'll look at the properties, you can see the long connection string that I've got up there.

31:01And what I'd like to do is right now I've got a map of some operational points, my citizen request points.

31:07As well, I have a basemap, a tiled service, that I've also got up on that service as well - on that server as well.

31:14So this is the tiled service that I'm looking at on that server, and I want to publish a feature service to that...

31:21...and like Stefan said, I want that to be a feature service that's running with the database up on Amazon...

31:26...and then sync it with my enterprise geodatabase within my organization.

31:30So how could we do that?

31:32So the first thing we're going to do is add a data store, and I'm going to just call this rep, and I'm going to add a data store.

31:43So my local, on my publisher connection - let me go to my... Let me do this without talking here.

31:54I'm going to go to my SQL Server database that's on my laptop, and I'm going to create a replicated data store.

32:00So I'm going to uncheck this, so it's not the same as the publisher, and when I do that...

32:05...it picks up the enterprise geodatabase that's on the ArcGIS Server instance on Amazon.

32:11And that comes with it when you buy CloudBuilder...

32:15...when you create an ArcGIS Server instance on Amazon with CloudBuilder.

32:19So you create that with ArcGIS Server, and you get an enterprise geodatabase, and it picks that up.

32:25And what we're also going to do is create a geodata service at the same time.

32:28This geodata service allows us to create the replica, transfer the data up there to Amazon from my citizen request points...

32:38...and then that's going to allow me to synchronize the changes between the two databases...

32:43...as I make updates to my feature service using the server in the cloud.

32:49So right now it's spinning up that, so we've made that data store, so this is going to allow me to publish to that server.

32:57But first, let's transfer the data up there.

32:59So using the Distributed Geodatabase toolbar, we can create the replica, and I want to make a two-way replica...

33:09...and I want to use that geodata service that I just published or created from a data store.

33:17Let's get connected to that. So I'm going to use that guy; that's my replica name.

33:27Oh, let's get rid of that. I think that's going to...

33:32And I want to use the full extent of the data.

33:39Okay, so now what it's doing is it's taking the citizen request points...

33:42...and it's transferring that up to the enterprise geodatabase using that geodata service.

33:47So that's all done now, so now all that data - it's not that much data, but it transferred that up there.

33:53Let's remove that basemap; I don't need that to publish the service.

33:57And now what I want to do is, using that data store, we're going to publish this as a feature service.

34:05And what it's going to do is it's going to recognize the enterprise geodatabase on my laptop based on that data store...

34:11...and realize I have to switch the paths of the map to the data store, the enterprise geodatabase on my server.

34:27So in my service editor, I'm going to pick Feature Service; it's going to pick that up, right?

34:36So it's still - it's not copying data up there; it's referencing the data.

34:40It knows to reference the enterprise geodatabase that's up there.

34:43So now when I publish, we're going to create a feature service that'll take the map...

34:47...change the paths of the map to the enterprise geodatabase that's on my ArcGIS Server on Amazon...

34:53...and then I can make updates to that guy.

34:58So it's just transferring that file up to the server on Amazon. Give it a second here.

35:17So this is a pretty exciting workflow 'cause before, you'd have to remotely log in to that server...

35:23...try to figure out how to get data up there, and here we've really tried to automate a lot of that workflow for you.

35:29So now, let's go to here. Let me get the connection.

35:41And we're going to create a new map.

35:59So you can see that I've got my service request points, and I want to add that feature service...

36:04...and again, this is looking at that enterprise geodatabase on Amazon...

36:07...and I can also add my parcel access, so that's behind there, and now, let's add a point.

36:18So we added a new citizen request up there, just above the northern, the Naperville northern high school.

36:27And now let's go back here and look at the same map, and we'll sync those changes.

36:35So I added a point to the enterprise geodatabase on Amazon; now let's synchronize those changes...

36:41...and I'm going to use that geodata service on the ArcGIS Server instance right there.

36:56Replica...whoop. Picked the wrong one. Sorry. From a previous test run.

37:09Okay. So we could sync changes both ways if I had made edits locally...

37:17...and now that's going to synchronize those changes between the two databases...

37:22...and bring that change that I made through that feature service that I did on ArcGIS Online to that feature service.

37:29Okay, so there's the point.

37:31So that allowed me to - again, let's recap.

37:33I was able to very easily create the data store, that it gave me that geodata service.

37:38It allowed me to transfer my data from my local enterprise geodatabase up to that ArcGIS Server instance on Amazon...

37:45...without logging in or anything, and then I was able to use that data store to publish a service...

37:51...change the paths from my local enterprise geodatabase to that ArcGIS Server - or that enterprise geodatabase on Amazon...

37:58...and then I was able to sync changes and make edits between those two databases.

38:03So that's, in a nutshell, how that workflow works...

38:06...and we've also got that video up on videos.arcgis.com as well if you want to resee it.

38:11[Audience question] When you join those replicas, can you cross databases?

38:14So if you had SQL Server on Amazon; can you have Oracle on your local?

38:20You can. Generally, yes, you can. But generally, it gets a little trickier in terms of like...

38:28...if you're going to go from like SQL Server and Postgres, there's different requirements for Postgres.

38:32Like generally, you want all the case to be lowercase.

38:36So you have a bit of a different data requirement in some of those scenarios.

38:40So it does get a little bit trickier when you want to do it that way.

38:43This is SQL Server to SQL Server.

38:45And if you wanted to create an ArcGIS Server instance with Linux, you could use that with Postgres in that case.

38:52So you do have different options with cloud, the ArcGIS Server running in the cloud, with different flavors of databases.

38:59[Audience question] What about from an enterprise geodatabase to a file geodatabase? Can you replicate to go that way?

39:06Well, in that case, you would - And sync changes?

39:09Replication doesn't work with file geodatabases...

39:12...so it only works with enterprise geodatabases in this scenario, in terms of syncing.

39:16[Unintelligible audience comment]

39:18I'm sorry?

39:19[Audience comment] You could one-way replicate to a file geodatabase starting with 9.3.1.

39:24Yes, you can, that's true. In this workflow, though, we're using enterprise geodatabases to do that.

39:32[Inaudible audience question]

39:33In this particular workflow, no. In this particular workflow, no, it won't.

39:38[Inaudible comments]

39:40Oh, yeah, yeah. And you can't have an editable feature service with a file GDB anyway.

39:44It has to be an enterprise geodatabase for that particular scenario.

39:49[Audience question] When would you actually use this workflow?

39:52Well, this is the kind of workflow where you have your enterprise geodatabase and you don't want people...

39:56...like external customers, hitting your enterprise geodatabase.

40:00You want them to use the data that's out on an external server, and it doesn't affect your...

40:05You know, say you had 3,000 people editing your enterprise geodatabase.

40:08You don't want, you know, the citizens of whatever county hitting your enterprise geodatabase directly.

40:13So this allows them to hit at a different server and then give you that opportunity to sync those changes...

40:18...when you want to do it and validate the data that's coming and making sure it's accurate, and so forth.

40:26[Audience question] Could you also select not the whole database, just the partial one? [Unintelligible]

40:32Well, it's based on the rules of replication, so in this particular scenario, I used the data for the whole extent of my feature class...

40:39...but you could've picked and choose which feature classes going through that advanced option when you created that replica.

40:44And so all the functionality of creating replicas is there for you to define what you want to push out.

40:54[Audience question] Can you easily apply security in both of those scenarios?

40:59I guess can you clarify that? [Inaudible audience question] I'm sorry?

41:02[Audience question, continued] You say these are all public [unintelligible] servers.

41:05[Audience question, continued] Can you apply security to those [unintelligible]?

41:08You want to answer that?

41:09The question was if you can apply security. Yes, you can.

41:11You have the full power of ArcGIS Server and all the security models that come with it.

41:16If you want more detail, Jay's in the room. He can answer a lot more of those detailed questions. But yes, of course.

41:25So let's move on. We'll have some more time for questions later.

41:30Question here. Okay.

41:32[Audience question] Where do the data store parameters physically get stored?

41:36[Audience question, continued] [Unintelligible] database authentication and a password change?

41:40It gets stored encrypted.

41:42[Audience question, continued] But the data store parameters, where do they physically...

41:45On the server. They live on the server.

41:47[Audience question, continued] So if you...

41:48Only accessible to publishers and administrators, not accessible to general users.

41:53[Audience question, continued] Okay, so if I did a password change...

41:56You would have to change the data store.

41:59[Audience question, continued] Okay, but only one place?

42:00Yes, only one place. Can you switch over, please?

42:08Alright, thanks, Ty, for the demo. Let's recap a little more; hopefully, it kind of sinks in what we saw here.

42:17So the scenario on the desktop kind of stayed the same.

42:19We have our map; we point to some enterprise geodatabase that's local...

42:25...and then we have an ArcGIS Server running in the cloud on an EC2 instance.

42:29There's also a geodatabase in the cloud, also running on an EC2 instance.

42:33And when you, like Ty said, when you use CloudBuilder and install that - and install it, you get both of those for free.

42:41And actually, the data store that you'll see actually gets generated automatically.

42:47Now, then what Ty did, though, the next step he used that EC2 geodatabase and hooked it up...

42:55... with his enterprise geodatabase one, so that means if we - if data has a connection to this enterprise geodatabase...

43:06...we'll swizzle it out when it gets published and it points to that geodatabase on the EC2 instance.

43:12You could do that all before with 10.0; you just had to really do it all kind of manually using Map Path Editor...

43:18...you know, copying up data, remote desk - you know, logging in to your Amazon instance.

43:23The next thing it did automatically was create that geodata service, so it also does that for you...

43:28...so you can actually transfer data and use replication with it.

43:33So once we have that geodata service in place and we created that replica...

43:37...you know, we're kind of in business between our two databases here.

43:41The rest was just simple publishing. No data gets copied up; it simply gets, you know...

43:46...the map and the service configuration get put in the service definition.

43:50It gets, again, uploaded to that server in the cloud, and in that case, a feature service was created, you know, during publishing.

44:00And we can use that in our web map, and Ty edited that feature, and now you saw all the syncing that took place.

44:06We think it's a really powerful workflow, and again, the big improvement was it's easy.

44:12Before you had to, you know, jump through a whole lot of loops and really understand a lot of details.

44:17And when do you want to use it? When you want to synchronize your data between the cloud and the enterprise.

44:21I think it was mentioned before, yeah, I think you kind of want to keep that separate...

44:25...because your cloud server would be public facing...

44:28...but you have your local data in your enterprise that's kind of shielded behind a firewall, then this is a great use case.

44:38There are actually a number of additional publishing scenarios.

44:41We also really improved publishing to Linux, 'cause that was also, in the past, you know, pretty cumbersome.

44:48One of the workflows is you have kind of - what people have, you have many are copies of data on the desktop...

44:55...and the Linux server, but also your Linux server is kind of your public-facing one.

44:59So that workflow, you can also use that replicated - the data stores that are different on the publisher and the server.

45:08Another case could be you have shared data between Linux and Windows using Samba or some other technology like that.

45:17That works really well, too, in that scenario. You just need to set up your data stores once; you publish and it just works.

45:24Another case is publishing feature services when the data gets copied.

45:30When you actually have a file geodatabase locally, it always needs to be copied into a multiuser geodatabase.

45:37There's actually a different type of data store you can register, which is a server-only one...

45:41...and that's where that data gets put.

45:44And a third, you know, major scenario is publishing to a new spatial data server...

45:48...the lightweight server for nongeodatabases as well.

45:55The good news is Ty made a lot of videos...

45:58...so you can see those on videos.arcgis.com for all these scenarios and learn about them.

46:04So I encourage you to look at that, but obviously, we don't have time to go through all these here in this session.

46:12So now one question that's going to come up because, you know, we have all these new service definitions...

46:16...Hey, what am I going to do with my old services?

46:19So in 10.1, we said all services, all map services and actually all other services as well, are based on these service definitions.

46:28So now in order to migrate your services, you need to start - you need to have the existing MXD, the map document.

46:33The MSDs are not going to be compatible.

46:36We deliberately didn't spend the time and effort to kind of, you know, make this work...

46:41...because, you know, you still should have your original MXD.

46:44Now you can two ways - there's two ways you can publish that.

46:47You can do basically what Ty did, you know...

46:49...using the service editor and the kind of nice UI experience to publish, you know, through Desktop...

46:55...but if you have a lot of map services, that might get a little cumbersome and annoying...

47:00...so we actually have a little bit of a publishing API you can use with Python.

47:05So I'm going to introduce that real quick here.

47:08So you want to use that, you know, for batch migration from 10.0 to 10.1 or for scheduled service upgrades.

47:14But it basically starts off with a map document, and then you load it up and prepare the MXD so you can set different things.

47:23And there is one new method in arcpy.mapping, which is called create an SD draft...

47:28...that's kind of a precursor to the service definition; it's an XML file...

47:32...and you can actually then in Python still, you know, modify the XML.

47:37And then you can stage and publish through GP tools.

47:41I'm actually going to show a sample that's also online, CreateMapSDDraft sample...

47:47...but I'm going to just briefly walk you through this.

47:50And so, I said before, first step is you kind of open up the MXD.

47:56You can modify it; like here, he added, you know, summary and tags.

48:00Then the next step is you turn and call the CreateMapSDDraft method...

48:05...so that creates that XML file that contains all the properties of your map, of your map service.

48:11You can then use, you know, parse it in XML and change those properties out, you know, through Python as you need...

48:18...you know, set everything you want that you can in the UI so you have everything available.

48:23And the next step is you analyze. So you just say AnalyzeForSD...

48:28...and then if there were no errors, you execute the Stage Service tool.

48:32So the Stage Service tool will take that SD draft and turn it into a service definition...

48:37...so it will package all the data up as necessary, will do exactly the same thing that Ty did when he had published the first part.

48:44And then once that service definition is there, you call the Upload Service Definition tool...

48:49...and that will take the service definition and you pass in also a connection.

48:54It will upload it to the server and then, you know, publish it from the server and turn it into a service.

49:01So that's an easy way for you to migrate your data. Like I said, that script is online; you can use it.

49:09So now we'll take - how are we on time? We're like 15...

49:14Gotta get going. You want - no, I think we can take about five minutes for questions to publishing.

49:21We haven't done the second section about map services, so if you want to - we were going to wrap up publishing.

49:27Is there any more questions on publishing?

49:29[Inaudible audience question]

49:41There have been a lot of changes in the coding since beta 2.

49:44So the question, he said he was using that code for migration, but it didn't quite work.

49:49[Inaudible audience question, continued]

49:56We still had a lot of changes since beta 2, just bug fixes. I wouldn't...it should work, we use this now.

50:03Actually, all our tests - our internal, our regression tests...

50:07...used these methods, so they should be very reliable at this point. Go ahead.

50:14[Audience question] I assume you can use the Python scripting to do a batch daily process when you post it to ArcGIS Online.

50:23The question was if you can use the Python to publish daily to ArcGIS Online?

50:28[Audience question] Yeah, for a batch.

50:29In batch, yes. That's a Python script; you can put it on a schedule or you can put it in a tool, yes.

50:37That's the kind of called for is for batch updates and migration to use it.

50:46[Inaudible audience question]

50:52The question was if you have to have an organization subscription. Yes, you do.

50:56That's what enables you to publish hosted services. It's not free.

50:59Obviously, you know, there's cost for us involved as well, and we have to pass that on to you guys.

51:05Question in the very back.

51:07[Audience question] Yes, is there a way to have a trigger on a [inaudible] in the cloud...

51:14...so you can synchronize your file geodatabase automatically, or would you have to create a script for that?

51:21The question was if there's a trigger so you can synchronize automatically from your enterprise...

51:27...from your data in the cloud to your enterprise geodatabase.

51:31I'm not sure. This is really a replication question. I know you can put it on schedulers.

51:35I don't know if there's something like a trigger. Any of you guys?

51:39We're not sure. We're not like the absolute replication experts. I would refer that to Gary MacDougall.

51:48[Inaudible audience question]

51:55I'm sorry. Can you repeat that?

51:56[Audience question] How's the new file form for services, the .sd...

51:59...how's that going to affect people that have got [inaudible] server objects?

52:02So the question was how does the new file format, the service definitions, affect people that have SOEs.

52:09It shouldn't really affect them at all.

52:12We will - if you click - if you register a custom SOE in the UI and you click it on as a capability...

52:22...that will get, you know, packaged as the parameters into the SD and it will get published normally, so it shouldn't affect you.

52:30[Inaudible audience question]

52:35Do we still have...

52:36[Inaudible audience question, continued]

52:39I'll talk about that in a few minutes.

52:41The 10.1 map service is an optimized map service only so if you hooked in...

52:46...to fine-grained ArcObjects through an MXD-based map service in the past, that pipeline is no longer available.

52:54Yeah.

52:57[Inaudible audience question]

53:06It's all versioned. Replication is based on versioning.

53:09[Audience question] Do you replicate when you upload from the distributed database...

53:13...do you upload from a version or from the...

53:17[Simultaneous conversation]

53:27I guess the basic question is what's the requirement of doing that in terms of your enterprise geodatabase.

53:31One, it needs to have global IDs, and two, it needs to be a versioned enterprise geodatabase to do that replication workflow.

53:38So those are the two requirements of that.

53:43We'll look into more improvements on this, you know, probably in the future, how we can enhance syncing.

53:48This was kind of our first step into syncing.

53:52We'll look into more stuff later; also, you know, syncing file-based geodatabases.

53:59[Audience question] Can you have multiple connections to the same data store?

54:02The question was if you can have multiple connections to the same data store.

54:08Well, not exactly the same data store. You have to have at least - the user has to be different...

54:13...but it could be through the same database with different users you can, that.

54:17It differentiates with the user at the user level.

54:22Let's take one more. How about in the back there.

54:26[Audience question] In the migration process, can I reuse existing caches from previous...

54:30Yes, you can. The question was if you can reuse existing caches. Yes, you can.

54:35During the publishing process, it will latch on to an existing cache that it finds there.

54:42But there is some disadvantages to that.

54:44I think we just added that in the last week.

54:48I don't know if you've seen the caching workflow, the new one, where you can - it has that status geodatabase...

54:53...where it keeps status. So obviously, you wouldn't have that with an existing cache, but you can re-create that with a tool now.

55:00So that's one thing you would have to do to re-create status so that you can better work with that existing cache.

55:07Okay. I'm going to pass it over now to Craig and map services. Thank you for your attention.

55:13And we wish you a lot of fun with publishing.

55:20I have one. Oh, you have one? He needs one.

55:26Alright. We'll talk about enhancements in map services.

55:28So why do you publish? You publish to get map services really, right?

55:32The workflows they showed actually apply to a variety of different service types...

55:35...but obviously, most of Ty's examples involve map and feature services.

55:41My name's Craig Williams; this is Tanu Hoque.

55:43We're going to go through some of the enhancements to map services, and Tanu will demonstrate some of these.

55:49If you've been going to web API sessions this week...

55:54...hopefully you've seen some of this new functionality as exposed through the web APIs.

55:57Most of it, you actually will interact with through the web APIs, and Tanu's demos will make use of those APIs.

56:06There are some publishing aspects in terms of options to enable when we can enable optional functionality.

56:14So map services at 10.1, we have one unified map service.

56:18So at 10.0 - actually, at 9.3.1 and 10.0, you could publish a traditional map service based on an MXD...

56:26...or you could publish an optimized map service.

56:28And at 10.1, we have one map service. It's based on the optimized map service, which gives you the best performance...

56:35...but we've updated it to support additional capabilities, data types, layers, renderers...

56:40...the things that were missing in the past that may have forced you to use an MXD-based service.

56:45And we've added new extension capabilities, such as network analysis, and geoprocessing is something that...

56:52...you may have a workflow where you want to show geoprocessing results as a map service.

56:57We've changed slightly how that works.

56:59It's actually a more streamlined and successful model that allows people to publish...

57:04...geoprocessing results in a more streamlined fashion, and you should look for that when looking at geoprocessing services.

57:12The map service fits into the new ArcGIS Server paradigm where it's a pure web services-based service.

57:19Most of the interaction is through REST.

57:22There are some SOAP end points that are used, especially when using map services in ArcGIS Desktop.

57:28So added data sources and layers and things like that.

57:32XY events, that was a popular one that optimized map services were missing in the past.

57:38They still have a performance penalty because they're usually a non-spatially indexed data source.

57:44Similar issue with linear referencing events, but if you have linear referencing data...

57:48...you know that that's often updated on the fly.

57:51You want your services to be dynamic because it's purely dynamic data.

57:56For feature renderers for feature layers, we've added dot density charts; geostats, if you've used those...

58:02...and added support for new layer types - dimensions, schematics, network, network analysis...

58:08...and those help power the network analysis workflows; TIN, terrain, and tracking layers.

58:14So that's a large number of layer types that weren't supported at 10.0...

58:17...that are now supported at 10.1 with the optimized map service.

58:23So one big item that we've added to map services as a new concept is called dynamic layers.

58:29And if you've come from an ArcIMS workflow, you may have been used to updating AXL on the fly...

58:34...to update the appearance of your map.

58:36This was something you could not do with ArcGIS services until 10.1...

58:40...so this is new behavior with the map service that allows you to make changes to the appearance of the map...

58:45...the content of the map, on a per-request basis.

58:49It's an optional capability of map services. You can enable it...

58:54...or you can disable it if you don't want people to be monkeying with the appearance of your map on a per-request basis.

59:00It allows for updating renderers and symbols of the map; removing, reordering, adding layers...

59:05...changing data sources; and when you add new layers, you can add them from a list of registered data sources.

59:11So those data sources are registered with the map service.

59:17The use cases we see people doing with dynamic layers - simple updates to the map service.

59:21A lot of people just want to be able to reorder the layers that exist in the map.

59:25Thematic mapping is a big one. You want to change a renderer to show a different attribute...

59:30...maybe join a table to show a different attribute, things like that.

59:34Or adding content to the map service.

59:36If your organization has a large amount of data that's unreasonable to publish in a single map service...

59:41...you may want to make that data available via a dynamic layers workflow...

59:47...where people can add data from the registered data source to the map, on the fly, and define symbology.

59:56One special case that can be done there is you can add query layers to a map.

1:00:01That's quite powerful if you're quite gifted on the SQL side.

1:00:05Imagine the analysis you can do on a single layer adding it to a map with a defined SQL statement...

1:00:10...that will do maybe some analysis to show a result, just in the context of a map service draw.

1:00:18These changes are applied to the map service on a per-request basis.

1:00:22So this means every time you do an export map image request, you need to push this information in.

1:00:28And that's because the map service is stateless.

1:00:31So we're not preserving state of these changes; you have to actually request them each time.

1:00:37And in most cases, the web APIs will handle this for you.

1:00:41So on the web APIs, you make the modifications to those layers in the web API client...

1:00:47...and when the web API goes to make a request to the map service, it will add that information to the request...

1:00:53...so that you get the correct image.

1:00:58So thematic mapping, we really see that as a special primary use case of dynamic layers...

1:01:06...so you want to be able to change the renderer; you want to be able to change data sources.

1:01:10Like I said, a lot of times people want to integrate data from a separate table, join that on the fly to a layer in a map service...

1:01:18...and then set the renderer on a field from that joined table.

1:01:23And one key aspect that's needed to complete this workflow is you need to be able to construct the classes of the renderer...

1:01:31...based on that new data, that new field that you may have joined.

1:01:35And for that we have a generate renderer operation.

1:01:38This supports class break and unique value generation of classes and - drop my mic. Nice.

1:01:46And we have all the classification types that you're used to from ArcGIS Desktop.

1:01:51So any of those classification types that you may have authored maps with in the past in ArcGIS Desktop...

1:01:56...you can generate via this API.

1:02:01So before we talk anymore, we're going to turn over to Tanu...

1:02:04...and he's going to give a demo of dynamic layers with thematic mapping.

1:02:13Thanks, Craig. So here I have a plain vanilla map that shows all the counties of the United States...

1:02:19...and those are just file geodatabase.

1:02:21I have only two layers. One is these counties, and the other is just a shadow to make it look nicer.

1:02:27On the other hand, I have a file - I'm sorry - SDE that contains...

1:02:32...hundreds of feature classes and tables inside and outside feature datasets.

1:02:38So what I would like to do is that I would like to publish a map, a map service...

1:02:44...which will allow the client to modify the renderer of these counties...

1:02:49...as well as add some layers specifically from this particular file geodata - feature dataset, all these feature classes.

1:02:58And then down below, I have an attribute table of that county data that shows all the attributes...

1:03:03...the demographic information available for each county.

1:03:06So I'll just start that publishing process as you have seen many, many times today.

1:03:11I'll just follow the wizard, give it a name, and then click Next. If it will work. Yep.

1:03:21And then I get this service editor.

1:03:24Here I need to point out one quick thing is we have an Advanced button here, collect advanced properties.

1:03:31I know - have any one of you had these issues or nightmare while you're trying to change the properties...

1:03:37...like maximum record counts or maximum image size and height? Anyone?

1:03:42It was a nightmare, right?

1:03:43But this time at 10.1, the publishing team made it simpler so you can do it from UI.

1:03:49You don't need to restart the whole SOC and SOM when editing. I did that in that existing service.

1:03:54So kudos to the publishing team. Can I have a round of applause for them? Yes. Nice.

1:04:02So now, to enable the dynamic layers capability, you need to go to the mapping page of this editing window...

1:04:09...and then click this check box.

1:04:11This will allow you to reorder the layer or change the symbology of a layer.

1:04:17And then I would like to add -- allow my clients to add feature classes or tables on the fly to my map...

1:04:24...so in that case, I will need to register workspaces.

1:04:27So there is a one-to-many relationship between a map service and a workspace, and we support all different types of...

1:04:33...actually four different types of workspaces - a database, which is SDE; then file geodatabase; shape-...

1:04:38...or a folder containing shapefile or folder containing raster.

1:04:42So here I will just create a - point to one of my SDE connection files, and I give it a workspace ID name...

1:04:52...which is very important. I will get to that one in a few minutes.

1:04:56And then we have a little check box here called Lock Version.

1:04:59So if you don't want your client to switch between versions, you can lock it.

1:05:03Otherwise, by default, you can actually switch between different versions.

1:05:07And then I will add another file geodatabase connection. I will call it fgdb_, you know, counties. That's it.

1:05:17I'll go to... Okay. There it is. That's my file geodatabase.

1:05:24Click on it, save it, alright.

1:05:27Now click it, I register my workspace, and as you know that we have to go to the analyzer process...

1:05:32...and as I analyze it, it will find that those two workspaces that I just added are not part of the data store.

1:05:38So I need to register them.

1:05:39So I can right-click and say register it.

1:05:42So this time, registration is much simpler than what Ty showed before.

1:05:45All you have to do is click OK and it'll get registered. Okay.

1:05:49Once it's done, you can just click Publish and it will get for the packaging and it will be published.

1:05:54I will skip that process because it will bring up an awkward silence, and I cannot sing to entertain you guys.

1:06:02Okay, so move on to the nice demo now, the fun part.

1:06:06So this is my application, web application written on Silverlight.

1:06:10So I can see the same map that I just showed in Desktop, and if I want to change the color, I can do that right away.

1:06:18I can zoom in, pan; it looks good.

1:06:20If I want to switch to a different renderer type, like different symbology, I can do that, right?

1:06:26Go back. Looks nice. And I can switch back to default.

1:06:30Default is how you publish it, the symbology that you used with that layer when you published it...

1:06:38...and the way you set it to default is set the renderer to null from the web application.

1:06:44Now the next part is I would like to add some layers, right?

1:06:47So I would add cities. It is coming directly from - you saw that when I published my MXD, I did not have that layer...

1:06:54...so I just added it dynamically to this map service per request.

1:06:59And I will just add the roads here. Looks ugly.

1:07:03If you think that's ugly, you should see my other maps. It's horrible.

1:07:07Okay. Now, since it's covering all my points underneath it, I would like to bring it up...

1:07:12...so that's what Craig mentioned, that we can reorder the layers.

1:07:15All I have to do is move it up and see those points are drawn.

1:07:19So this is, again, it's not client side drawing; it's happening on the server side.

1:07:23Just to prove it, I can show you that here is the request that goes out, and result comes back...

1:07:30...that image that comes back to me with all the layers on it. Okay.

1:07:40Now that we're done with this adding new layers on the fly, now the funnest part is to...

1:07:46...the thematic mapping that Craig just mentioned.

1:07:50So the county layers have lots of demographic information, and this one I can say...

1:07:55...Okay, I would like to render all my county polygons with total population.

1:08:00So this is, again, that thing that Craig mentioned before that we have a utility function called generate renderer.

1:08:05We could call that one to get the renderer, and I'm just applying it to that.

1:08:11Without this one, you would have probably returned like a hundred of lines of code on the client side...

1:08:15...download the data - so there's an overhead also, download the data on the client side - and then postprocessing that one.

1:08:22If I don't like this color schema, I can go and change a different color schema.

1:08:27If I want to go to different, show me to the different attribute values, I can do that.

1:08:31And if I think that, Hey, this method, the classification method, is not good for me, I can change it to different ones...

1:08:36...like natural breaks. Do that, and change the number of class breaks for it, right?

1:08:43It's very powerful, isn't it?

1:08:45Okay, now that we can do it, it's still good...

1:08:48...but the thing that still bafffles me sometimes is that I'm still stuck with the attributes that have been published by that author.

1:08:57What if I have some additional information?

1:08:59Like in my case, I have very simple information; the publishing changed from 2000 to 2010.

1:09:05I save that as a CSV file. I would like to upload it, and I will show you it in a second.

1:09:09So what I do, I can click on it, the Upload button.

1:09:14I'm choosing that CSV file. That's because Excel is not supported in the server right now at 10.1.

1:09:19I click on it. What's happening behind the scenes, I'm calling a geoprocessing function...

1:09:23...which is converting my CSV file to a file geodatabase table.

1:09:26And all the geoprocessing service has an upload operation today at 10.1.

1:09:31I'm calling that upload, which is uploading my CSV file to that server...

1:09:34...converting it to a file geodatabase table, and joining it for my optimum performance.

1:09:39And see how fast it was.

1:09:41So I can see all the population change happened in that last decade, and I see that the green color...

1:09:47...shows where the population declined, and the red is pretty much where population increased.

1:09:51And that's me too. I contributed that one.

1:09:55Okay. So how did it work?

1:09:57So let's take a look at the REST side.

1:10:05We defined a list of the layer, that dynamic layer's options in that export operation.

1:10:13We defined the source.

1:10:14Here I'm saying I want to use one of the existing layers.

1:10:17The layer ID is 0 and the type is map layer ID, followed by the drawing info.

1:10:22That's all I did. I clicked the post, and I can see that got refreshed.

1:10:27If I want to change the color, something else like the first class, the first one, to red...255...sorry about that...

1:10:44...and repost it, I'll see that the first class of the class breaks renderer changed to red.

1:10:49What happens when I do that with the - when I want to add a new layer, right, from that registered workspace?

1:10:59This is how it happens. The web application sends the request with this type of source this time.

1:11:05It says type is a data layer, and this is the workspace ID.

1:11:09Remember the workspace ID that I'd given during the publishing process?

1:11:12And here is my name of the data source, and that's how it knows that it needs to add a new layer from that workspace.

1:11:24Now, that one thing I mentioned before is the geodatabase versioning, right?

1:11:27So sometimes this require, especially [inaudible] companies that will do the parcel editing...

1:11:33...that you want to switch between different versions.

1:11:35In my case, I'm zooming in to my data in Naperville, Illinois, and I see a big junk of parcels...

1:11:41...chunk of parcels there that I would like to see what happened after the subdivision.

1:11:45So I will switch between the versions, and I see that it shows up right away.

1:11:49So it's not two different layers; it's just that we are switching between two versions called base and the subdivision.

1:11:54And the way you do that in there, this is the Silverlight code that I have, it's very simple...

1:12:00...and the logic is similar across all APIs.

1:12:03All you do from the first place, you create a DynamicLayerInfos from the LayerInfo function.

1:12:09Add some collection, then you loop through all the DynamicLayerInfos in that collection, set the version name...

1:12:15...and followed by setting the dynamic layers back to that dynamic map service layer object and call the refresh.

1:12:23That's it, as simple as that.

1:12:26Now, another thing I do want to talk about is the time query layer that Craig mentioned, very powerful one.

1:12:33So here I have a map service that contains - are you familiar with this time series data that you have points in a feature class...

1:12:41...but your time series, they are taken for the same location over the period of time but saved in different tables.

1:12:48And in this particular case, the problem is that the time was not - I'm sorry.

1:12:53The value was not collected at the regular intervals; it's very irregular.

1:12:57So there is a problem that how would you show that?

1:13:02And the weather research people, time series, they would like to know the maximum value...

1:13:07...minimum value for a period of time.

1:13:10Right? Not for one particular instant.

1:13:12So here I'm seeing all the locations here, but I would like to know the two-month average for that site...

1:13:18...the BOD value, which is the biochemical oxygen dissolved value for that.

1:13:23If I want to see the maximum value, I can do that.

1:13:25I can enable the time, then I'll go across the different state, and I could see this changes.

1:13:31So we are taking advantage of this called query layer, and we are sending a query layer request...

1:13:38...and this is how we define the query layer here.

1:13:41Setting the different type, there's a data type again; source is the query layer type; workspace ID; OID field, different type...

1:13:48...and followed by a big query that's a SQL statement.

1:13:54Finally, I have this query stat. That is a new operation; it's nothing to do with dynamic layers.

1:14:01It's a new function that allows you to get some basic stats from a particular layer, existing layer or dynamic layers.

1:14:07So here I'm zooming in one of the places, called Dublin, Ohio, and you can see that if I enable this guy...

1:14:16...as I zoom or pan, I can see my graph's updated.

1:14:19That's because I'm calling that query stat operation to compute the summation of all area that belongs to each land-use group.

1:14:28So I can use that to filter it out as well.

1:14:33Like I can say, Okay, I would like to do only for the residence...

1:14:37...and the land-use type is residential, add those cells, I can do that.

1:14:41If I want to do this, like if I want to group by another field here, like the political units, I can do that as well. Alright.

1:14:56Okay, last thing doesn't work, so I'll hand it back to Craig. I'll see what's going on.

1:15:00So that proves that I was using the real data or the real map, real demo. Thank you.

1:15:08Alright. We're running a little short on time, so we'll just finish up.

1:15:14If you've been coming to this conference for many years and you're quite familiar with the web APIs...

1:15:19..they've been showing thematic mapping with the web APIs for a long time and really pushing that approach.

1:15:24And we just showed server-side thematic mapping; you may be wondering, Which should I use?

1:15:29Well, really it depends on the situation.

1:15:32Many of you have tried pushing thematic mapping into the client side with feature layers in your client-side graphics...

1:15:40...and have not had good performance with it.

1:15:43And that depends on the number of geometries you really have in your map, how complex they are...

1:15:48...and kind of the scale that you're looking at.

1:15:52And if you have a large number of small polygons that are pretty detailed over a map extent, it may bog down in your client...

1:15:59...and that's a case where you may want to use the server-side thematic mapping via dynamic layers.

1:16:06Each approach has trade-offs.

1:16:09Generally, client-side features, the client-side approach, scales better on the server end...

1:16:15...'cause you're just executing queries on the server, whereas what we're showing is actually drawing on the server.

1:16:20But if you're downloading a huge payload of data, that may take time as well...

1:16:24...and generally the images that you get back from a dynamic layer are faster.

1:16:27So you need to play with each trade-off and see which one works.

1:16:31The generate renderer functionality can be used with either approach, so if you're ever trying to roll your own...

1:16:37...classification in the web API, you can now use the generate renderer functionality that Tanu just demonstrated.

1:16:43So if you need more information about dynamic layers as a whole, we urge you to see the ArcGIS Server Help.

1:16:49There's a bit in the ArcGIS Server Help that introduces the concept as we have today...

1:16:53...and then the details into the JSON and all that are in the REST API help.

1:16:59And then finally, Tanu showed the final demo there was some of the query enhancements we've made to ArcGIS map services...

1:17:06...so we've added support for order by, output statistics, which is what he was showing.

1:17:13You can group by with those statistics, and the statistics that we've added are listed here...

1:17:18...count, sum, min, max, average, standard deviation, and variance.

1:17:23There's some additional options with the query method on a map service.

1:17:27You can now optionally return Ms and Zs, if your data supports it.

1:17:31You can query a specific geodatabase version.

1:17:34Tanu showed drawing a specific geodatabase version, but that also applies to queries as well...

1:17:39...if you need to just query features from a specific version.

1:17:43And one of the classic issues with query with map and feature services is you hit the max record count on a query...

1:17:50...and often, you would not know whether or not you hit the max record count.

1:17:55You would just have to guess if the number of features returned equaled what was set as your max record count.

1:18:01We now actually include a flag in the record set when we return it that indicates whether or not you hit the max record count.

1:18:09So at that point, we're done with today's presentation, and since we're running out of time...

1:18:14...I'd ask you to come up to answer questions, 'cause we've got to clear out of this room for the next presentation. Thank you.