00:01Today's topic is leveraging OGC services in ArcGIS Server.
00:06How many of you here have had a chance to see something similar in a previous conference or at the UC?
00:11Okay, good. Thank you.
00:14Second question. How many of you are familiar with OGC services? Okay, almost everybody. Good.
00:23How many of you use ArcGIS Server or the various clients that we have to work with OGC services today? Most of you.
00:34Are most of you, are you on the 10.0 platform? Okay. Okay, few of us.
00:41Has anybody tried the 10.1 beta prerelease, the OGC parts of it? Oh, you have. That is fantastic. Okay.
00:53With much success? [Inaudible audience comment] Fifty-fifty success.
01:00Okay, then. We should have the conversation shortly after the presentation then.
01:05[Inaudible audience comments]
01:07Okay. No, we just want to make sure. We're here to help.
01:12How many of you use non-Esri products, be it GeoServer, Minnesota MapServer...
01:19...or any type of open source clients to consume these OGC services?
01:23That's excellent too, 'cause one of the old ideas behind this is to, you know...
01:28...create interoperable systems so we can communicate with one another.
01:31Okay. Quick introductions. My name is Satish Sankaran. I work for Esri; I am in product management.
01:38I take care of all things related to interoperability and standards.
01:43I wear multiple hats. I am also the Esri representative to the OGC, so work with OGC; I'm on their...architecture board.
01:54I chair the WMS standards working group and a whole bunch of other things.
01:59So I should be in a position to answer most of your questions, either here or later, not just related to...
02:05...questions related to our product support but also about, you know, some specific standards-related stuff that you might have.
02:12We have Yingqi Tang here. He is part of our software development team back in Esri...
02:18...so he's going to do the second half of our presentation.
02:20I'm here primarily to set the stage and tell you what is it that we should all care about when it comes to interoperability...
02:28...and he's going to dig deep into some of the new things with 10.1...
02:34...and kind of do some nice demonstrations to show why these services could be really useful.
02:41So with that, I think we should start.
02:45Well, what is GIS all about, right? I mean, fundamentally, as I see it at least, there are two things.
02:51It's about creating and managing geoinformation products...
02:55...and then ultimately being able to disseminate those products that you create.
03:01So during the course of working with GIS, you could be working with proprietary formats or protocols...
03:09...or you could be working with some kind of open specifications that have been, you know, published by a specific vendor...
03:18...and that allows you to work with multiple vendors who are able to, you know, work with those specifications.
03:26And sometimes you would be working with some standards-based protocols.
03:32So open specification is when a vendor is opening up his format or his protocol for people to use with no IP restrictions and so on.
03:41Standards is not just one vendor releasing stuff, but usually it happens under a consensus process.
03:47So the OGC, Open Geospatial Consortium, is a geospatial organization where a bunch of members from different vendors...
03:56...from academia, and so on, get together to develop standards that are mutually beneficial to the community.
04:04Today's presentation, we're going to focus on the last aspect of it, which is the standards-based stuff.
04:10So we'll get into the details of some of the OGC standards and how we support them.
04:18On a broader note, though, I mean, since we are at an Esri conference, we need to understand what ArcGIS is...
04:25...and as you have seen this slide many times before, it's a complete system.
04:29So it's not just about, you know, using, creating, and managing your data, but we're living in a very services-oriented world today...
04:37...so you could have your data sitting up there somewhere in a cloud, others could be on your local server...
04:44...or it could be on your enterprise server, if you will.
04:47And you want to provide that information over to a multitude of different clients.
04:55This could be web-based clients, this could be mobile clients, this could be just, you know, desktops.
05:01And when we talk of providing the data, we're talking of not just data being provided through a service...
05:07...but you want to be able to provide different kinds of services that people may want to access.
05:12Discover services, services that allow you to create data...
05:15...services that allow you to just manage data, to visualize data, and so on.
05:24This slide talks about ArcGIS being open and interoperable.
05:31Not everybody uses the same kind of clients, not everybody wants to work with the same programming environments.
05:41Some people want to use .NET, some people want to use Java, so on.
05:45Some people want to use standard-based protocols or formats or interfaces.
05:50Some people don't really care if it's a standard, but they just want to make sure it's open in some way...
05:54...so that they can access it however they want.
05:58So any strategy for interoperability should be kind of comprehensive.
06:06So essentially, we need - as a vendor, our aim is to support a range of strategies for tackling the interoperability problem.
06:16And I also use this slide to emphasize that the bigger problem that we're trying to solve here is the interoperability problem...
06:22...and standards is kind of one solution to it. Right?
06:26But that's what we're focusing on in this presentation, but you have to bear in mind that what we're trying to do...
06:30...is to make sure that our systems are interoperable so that you can, you know, do what you need to do.
06:39Before we go into the OGC services, I've tried to distinguish some of the important aspects that we need to understand...
06:46...like the foundational stuff, before we can get into the services aspect.
06:50There's standards that are used in the creation of geoinformation products...
06:54...and there are some standards that are used in the dissemination of that information.
06:58And this is not an all-inclusive list, but I'd just like to spend a couple of minutes on a couple of these...
07:07...so that we understand some of the important ones that'll help us understand some of the services-based OGC standards better.
07:14To start with, the Simple Features model. When it comes to data standards...
07:19...we'll just talk a little bit about Simple Features model and then a little bit about GML, the Geography Markup Language.
07:25Then I'd like to talk a little bit about metadata standards, and then, as far as dissemination goes...
07:31...we will then talk about the OGC services and how we support them, which is kind of the core of our presentation this morning.
07:38Okay. Simple Features is an OGC standard, also an ISO standard, but it's so transparently been used for so many years...
07:48...people tend to forget its importance.
07:50Almost everything that we do or everything that most vendors work with is all based off of the Simple Features standard.
07:59Right? It provided a data access model; it provided a database schema; it told you how to store vector geometry...
08:05...points, lines, and polygons; how would you store it in a database or otherwise.
08:11And this was a specification that's been in existence for probably now over 10 years.
08:20If you look at the actual specification document, there are two documents there.
08:23There's a Simple Features Specification, the common architecture spec, which talks about a geometry model...
08:29...and how to represent that geometry using, say, well-known text representation or a well-known binary representation...
08:37...and also talks about how to represent spatial reference systems using a textual representation.
08:43The second part of the spec talks about SQL options.
08:47How do you represent all this information in a database?
08:50So it introduces the concept of a SQL geometry type and also introduces standardized routines, SQL routines for, you know...
08:58...constructing and obtaining geometry in these various different formats from the one that's stored in the database.
09:06So that's Simple Features, and for most part, Simple Features is actually a misnomer...
09:11...because what they mean by "simple features" is really simple geometry.
09:15You're talking of points, lines, and polygons. All the lines are basically, you know, just connections of points.
09:22The curves are not - we're not talking of spirals and, you know, NURBS, and various different kinds of curves.
09:29But we're just talking about simple, you know, points that are joined together in a curvilinear fashion.
09:37Okay. So that's how geometry is usually stored and how we work with it in our database environment.
09:42When we talk about a services environment, sometimes we want to be able to provide the features out to a client.
09:49And how we do this normally in the OGC world of things, we do it using GML...
09:55...which is an XML-based encoding for geographic information.
10:00We're not going to go into the details here, but GML allows you to represent geometry, spatial reference...
10:06...topology, you know, 3D objects, and everything under the sun.
10:10So the concept of profiling is normally used so that you can reduce that large GML subset to something simple...
10:20...so that clients can easily work with it.
10:23For example, a point profile reduces the vocabulary use so that all it does is tells you how to represent a point using GML.
10:32We talked a lot about the Simple Features model but just representing points, lines, and polygons...
10:36...so GML, which is an XML encoding, has the ability - has been profiled in such a way to only represent simple features.
10:45So that's called a GML Simple Features profile...
10:47...and that's kind of the one that Esri has been advocating for and using within its product stack.
10:54Then there's the question of application schemas for GML...
10:57...and there are multiple application schemas that are very domain specific.
11:01They use the profiles, but they use also terms and vocabularies that are specific for a domain...
11:07...be it the cadastral people or for transportation networks and so on.
11:11So as you start understanding GML better, you need to understand...
11:14...what the concept of profiles are and what the concept of application schemas are.
11:22Then I would just like to talk a little bit about metadata standards.
11:25We all know what metadata is for.
11:27It provides resource for users, an idea of metadata they should collect, and why it is important and so on.
11:32But more importantly, just two slides here.
11:35One on the fact that the 10.0 release of our software in the desktop supports a whole variety of metadata standards...
11:42...you know, the ISO 19139, along with the North American Profile for that, the INSPIRE standard, and a couple of others.
11:53In 10.1, the metadata support is kind of, it extended over to the server too.
12:02So if you have a 10.1 machine, and if you want to access...
12:08Let's say you're publishing a map service from an MXD, and the MXD has some ISO 19139 metadata.
12:13In 10.1, you're in a position to actually access that metadata via a service end point.
12:23So people that are interested in being able to access service-level metadata at 10.1...
12:29...you know, we could definitely talk about it after this presentation if you have questions.
12:34Okay. So at this point, we're going to delve into the services aspect of it...
12:42...and again, everything that we talk about these days is all services-oriented architecture, if you will...
12:48...so normally you would have a discovery service where you'd have some metadata so you can go in...
12:54...search for a certain kind of functionality or a service, be able to access it.
12:58Then, if it's a view service, you can get to view some maps and so on.
13:01If it's a download service, you're in a position to go and download data and work with that data.
13:07So keeping this in mind, obviously if you are a geo professional and you are thinking in a service-oriented fashion...
13:14...you'll think, okay, this sounds good. What is it that I can do? Right?
13:18So immediately what comes to your mind is the concept of a map service, concept of a geocode service.
13:24In Esri, we have geoprocessing and geometry services that are kind of similar, concept of an image service...
13:30...and a feature service that provides access to feature querying and editing.
13:34Okay. What I will do is I will stop at this point with this and then step into a quick use case scenario...
13:45...something to set up the stage for Yingqi to continue the rest of his presentation.
13:51So imagine a scenario where we're trying to work with services to solve a specific problem.
14:02It's a little bit of a contrived situation, but it helps us build a demo to explain what we can do.
14:07So essentially, Yingqi probably wants to search for some, you know, houses out in San Francisco...
14:14...so one way of doing that is he can get a lot of data that he wants, you know, from some database somewhere...
14:21...which has the cost of houses, and then he can put them on the map and see, you know...
14:26...what kind of localities they are situated in and what kind of grocery stores, et cetera, et cetera, that are right next-door...
14:34...and he can make a decision based off of that information. Right?
14:39Let's assume instead what he gets is a bunch of services; that raw data that's available is provided to him as services. Right?
14:50So in his demonstration, what we're trying to get at is...
14:54...how would you be able to do those kinds of analysis from a services standpoint?
15:00All he has access to is, say, some kind of a map service that provides all kinds of data.
15:06So in this particular one, he sees a whole bunch of homes that are being made available...
15:11...but somehow he wants to be able to only visualize the homes that fall within a particular cost bracket.
15:20Beyond which, he also wants to specify some kind of a polygon that says, hey, I want to have...
15:26...look at homes that are only within this particular area of interest.
15:30And not only that, he wants to, at some point, even want to edit those polygons so that if it's an earthquake-prone zone...
15:36...he wants to remove those parts of it from the polygon, so kind of restrict his filters even further.
15:43And sometimes he may want to do some additional geoprocessing analysis. Could be a drive-time analysis or something.
15:49He wants to do that and he wants it to, you know, interact with the polygons that he's built, filter it further...
15:54...and then be able to take all the data that's within that intersection and send it as an e-mail over to somebody or to himself...
16:03...so that he understands, you know, if this is the house that I pick, what kind of facilities are available right next to the house.
16:09So I think this is a reasonably good use case, which will help us exercise different parts of our software.
16:18All this now is these map services, feature services, and so on, but I'd like Yingqi to do the demonstrations...
16:26...talk about it from an OGC perspective, how would you do this if all you had was an open source client of some sort...
16:34...and you were hitting a map service, a WMS, a WFS, you know, a WPS, and so on.
16:40So the challenge to Yingqi is to do this whole thing in an OGC context, if you will. Yingqi?
16:55Alright, thanks, Satish. So I'm going to follow up, talk about a little bit more on OGC web services in ArcGIS Server.
17:04The business case in San Francisco that Satish just brought up is actually artificially, you know, designed for the purpose...
17:11...that we want these different little requirements so we can touch those different functional areas of ArcGIS Server.
17:21Now, in an Esri ecosystem, these little requirements can be actually done by people publishing different types of services.
17:30For example, map service for the dynamic mapping, the tiled map service for fast tile mapping...
17:36...or the feature service for if you want an editing feature in a geodatabase over the web...
17:41...or finally, a geoprocessing service to do some spatial analysis over the web.
17:47Once those services are up running, there are tons of Esri clients that can consume those services.
17:52We have the desktop solutions and the web APIs, the cloud solutions...
17:58...and the mobile for those common, you know, mobile platforms.
18:03I'm pretty happy with these, and as long as all your clients, potential clients or users, are always in this Esri ecosystem.
18:13But it's not always a perfect world, is it?
18:18So what if there are users coming beyond this ecosystem?
18:23What if a user is coming using gvSIG, uDig, Quantum GIS, Gaia, Google Earth, OpenLayers, GeoCommons, and so on?
18:33How about other commercial GIS vendors like MapInfo or Intergraph?
18:38Currently, none of them can consume any type of service from ArcGIS Server.
18:44Now, what's the solution for that?
18:47The solution is to leverage OGC services in ArcGIS Server.
18:53This actually is a very important slide.
18:56I put a lot of time on it, so if you can only remember one slide when you walk out of the room, please do remember this one.
19:04So this slide carries two important information. The first important information is kind of obvious.
19:10As far as 10.1 goes, we have implemented almost all the major OGC web services here.
19:17WMS has been here for a while for the dynamic mapping.
19:21WMTS, the OGC Web Map Tile Service, is for serving up really faster tile, the cache and tile maps.
19:30WCS is serving out the raw raster data in different formats, like GeoTIFF, JPG2000, HDF, and so on.
19:38WFS, the Web Feature Service, enables the feature accessing over the web, also the feature editing over the web.
19:46WPS, which is new in 10.1, Web Processing Service, is for you to publish...
19:54...your tools and models for the geoprocessing over the web.
19:58And also KML. It's serving out a mixture of data, also the visualization information together.
20:04Now with all those OGC services enabled, what's happened? You can see I put a happy face here.
20:11Now all the clients, non-Esri clients on the left side, are now more or less...
20:16...being able to communicate with ArcGIS Server based on their own scope.
20:23Just to give you a little bit basic feelings of how they work, this is a screen shot of Quantum GIS...
20:32...a very popular desktop GIS consuming the WMS service and WFS service from ArcGIS Server.
20:42Gaia, a very light, free OGC client, although not free.
20:48In this case, it's a screen shot, it's connecting and consuming a WMS from ArcGIS Server...
20:53...and try to identify some features from our server.
20:58Again, Gaia, it has an extension that enables you to edit features in our geodatabase...
21:04...through the WFS-T transaction protocol.
21:09Although it requires a license, this extension's not free, but Gaia's free.
21:15Gaia again, consuming our tiled map service.
21:20gvSIG, which is also a very popular desktop GIS solution from Spain...
21:25...connecting and consuming a WCS, Web Coverage Service, from our server.
21:31uDig, also the same story, consuming a service from our ArcGIS Server.
21:38Now, this is OpenLayers WFS-T.
21:41Actually, this application is built right up on the Autobox sample of OpenLayers, doing the web editing through WFS-T.
21:54GeoCommons, very popular cloud GIS solutions consuming WMS from ArcGIS Server.
22:00GIS Cloud, another cloud solution.
22:04GeoNode, it's the enterprise or cloud solutions from OpenGeo, also consuming the WMS from ArcGIS Server.
22:17Now, the second important information that's conveyed in this slide is not quite obvious.
22:24It's about architecture we implement in our OGC services.
22:27Now, instead of creating those OGC services from scratch...
22:32...our goal is to simply expose the rich functionality from ArcGIS Server services through the OGC interfaces.
22:42And the value, the benefit of that architecture is what I called inheritance.
22:49Now, inheritance is a term we use in object-oriented programming which refers to subclass, you know...
22:55...inheriting all the attributes from its parent class.
23:00What I want to point out here is OGC services inherit a lot of attributes from ArcGIS Server services...
23:07...which can all be summarized here - cartography, security framework, performance and scalability...
23:17...authoring, publishing, and sharing workflow.
23:20And I'm going to dig a little bit into each of them.
23:23Cartography, right. So when you try to publish data as beautiful maps, what usually takes most of the time is what?
23:32It's cartography, right? What kind of symbology you're going to use...
23:36...what labels and fonts you're going to use, and things of that nature.
23:41And Esri is very good, very famous for having good tools to help you to create those beautiful maps.
23:48And all the tools are GUI based, so you don't have to manually work with those XMLs, which are designed for the machines.
23:57Just to give you an example. If you have ever worked on SLD, what do you think...
24:02...how hard do you think it is to create those beautiful maps all manually in SLD? Right?
24:09Is there any tool to help you do that? Yes, there are a few tools, but none of them are even close to what ArcMap can provide.
24:17So, Yingqi, what is SLD?
24:19Oh, yeah. SLD, I'll just clarify what SLD...SLD stands for the Styled Layer Descriptor.
24:25It's also an OGC specification, which helps you to define the symbologies and encodings for a map, for the data.
24:37Now, another example is the GeoCat. Anyone heard of tools called GeoCat Bridge? Right.
24:45So it's a commercial tool that helps you to publish the map that you created in ArcMap to GeoServer.
24:51Now, just think about it, why there is such a commercial tool [that] exists there to help you to publish the map...
24:59...created in ArcMap to some other server products. Because our cartography's good, right?
25:06Security. OGC specs don't define any standard or best practices to secure services...
25:14...but ArcGIS Server does provide a very powerful security framework to secure your services.
25:20And within the same framework, OGC services can all be secured the same way your ArcGIS Server services can be secured.
25:28So we support token-based security, we support HTTP basic and digest.
25:33And all your users and groups can come either from Windows or relational database or LDAP. It's very powerful.
25:45Performance and scalability is always a top priority in our development and QA process.
25:52Just giving an example here, this is a chart of throughput showing one of our benchmark WMS running on a single machine...
26:01...and about 60 percent of CPU usage, and you can see it's stably producing maps, around 40,000 maps per hour.
26:12This is another example.
26:14This is a chart of throughput of another of our benchmark WMS scaling out on a six-node cluster.
26:23Each of the nodes will be running, and they're about 60 percent of the CPU, and you can see it scales it perfectly.
26:30And if you're using full, using the six nodes of the cluster, you're getting close to 800,000 maps per hour.
26:40And just so you know, these charts and numbers, we are actually very carefully monitoring and collecting them...
26:47...on a weekly basis and even daily basis during our development and the QA process...
26:53...to make sure our OGC services have the good performance.
26:58Authoring and publishing. Now you guys must have seen a lot of demos for the 10.1, see how easy it is to author the map...
27:06...package the data, and publish to the local server, remote server, or even the cloud.
27:11OGC services will have the same publishing-authoring workflow - smooth and easy.
27:17Cache management. We added WMTS in 10.1, right? And it's not just the fact that you're having this new service.
27:26You're actually having a set of tools, very powerful tools, to help you to create, to manage, and to update the tiles that you have.
27:35Same idea applies for the web processing service.
27:37It's not just a new service; you have the ModelBuilder, you have the scripting environment in the back end...
27:44...to help you to create the GP functions to aggregate those different single GP functions into powerful models. Right?
27:54Now, let's look at some of the differences between ArcGIS Server services and OGC services, which can be summarized as these.
28:01Interfaces, different; transfer protocols are different; encodings are different; and clients are different.
28:09These are some details of the differences listed in this table.
28:13You can see that OGC uses the OGC specifications...
28:15...and ArcGIS Server services use a SOAP API and our geoservices REST API.
28:20Transport protocol is different. OGC uses the REST-ful, HTTP basic or HTTP CAT or HTTP POST.
28:27And ArcGIS Server services using SOAP and the REST.
28:30Encoding. Most of OGC using XML, different flavors of XML - GML, SLD, feature encoding, and so on.
28:37ArcGIS Server services use a SOAP XML and our geoservices REST JSON.
28:43And of course, the clients are different.
28:45All the OGC-compliant clients are supposed to talk, to communicate with ArcGIS Server through the OGC's...
28:51...and for the Esri products, for the ArcGIS Server services...
28:53...we're using ArcGIS Desktop, web APIs, ArcGIS.com, and so on.
29:00Now, before we go into the demos, I really want you guys to have a look at this little cheat sheet for our OGC services.
29:09I created this, spent a lot of time on it.
29:12It's kind of a good reference for you to look at how our OGC services evolved from 9.2 all the way to 10.1 here.
29:24And I put all the major events, additions in each release.
29:29You're probably not going to see it clearly in these slides...
29:31...but I'm going to package up this PDF file and upload it to ArcGIS.com, so after the conference...
29:37...you guys can go and just search DevSummit 2012 OGC; you'll be able to find this very nice reference.
29:49Now let's go into the demo part. The first little demo - it's not really a demo demo; just a recap of how you publish ...
29:56...OGC services on ArcGIS Server, because the whole publishing workflow has changed from 10.0 to 10.1.
30:04But the publishing workflow for OGC, it kind of remains the same.
30:09You basically follow, publish ArcGIS Server services, go to the service properties and enable the capability of different OGC.
30:18I'm going to show you a little bit on that.
30:23So to save some time, I already have those services published into my server.
30:28You can see there are some map services for different purposes, and there is a GP service here.
30:36So if you right-click a service, okay, go to the service properties...
30:56...it's going to show you a list of available OGC capabilities for that type of services, right?
31:01For the map service, you have WMS, you have WFS, and WCS; for the geodata service, you have WFS and WCS...
31:12...and for the GP, you will have WPS.
31:16My hard drive is still spinning. Hopefully...'cause it stays sleeping probably for a while. Alright, here we go.
31:27So go to your capabilities. You can just enable WMS or WFS, KML's enabled by default.
31:36And if you click all those enable OGC services, you can configure the properties...
31:43...and there are options you can configure for those services.
31:48Same idea for the GP. Website takes time, so I'm going to just go ahead.
31:55Now, another important useful tool is in Manager, ArcGIS Server Manager. Log in.
32:09You will be able to do the same administrating operations - publishing a service, stop, enable OGC, and disable OGC.
32:17One thing very useful here is this OGC Services tab.
32:21If you click on that, you have a drop-down box that lists all available OGC services on this server - WCS down to WPS, right?
32:35And also another useful tool is the site directory. It lists all the services.
32:42If you go to one of the services, you can see that the OGC services that are being enabled on the service are listed here as links.
32:51So if you click on the link, it's going to point you directly to the capability operation of that OGC service.
32:58You're getting the XML back directly, so that's going to be the URL you will copy and paste into the non-Esri client...
33:06...to connect to the URL service.
33:16Okay, so next real demo is dynamic rendering using WMS and SLD.
33:24So WMS basically provides you the dynamic mapping over the web; SLD helps, you know...
33:30...extends WMS for you to get the symbology definition from the server, lets you modify the symbologies...
33:36...and sends back the new symbology to the server so the map can be changed, the styles of the map can be changed on the fly.
33:47This is just what happened under the hood, right?
33:50You have a non-Esri client, you're first initially sending a GetMap request to get a map from our server.
33:57You're also sending GetStyles request to get definition of the styles in XML, SLD XML.
34:03In the client side, you can adjust those styles by changing the XML...
34:07...and finally, you can send back the modified SLD XML to the server.
34:12The server will render a new map based on your change, right? So I'm going to show that.
34:21The client I'm going to use for this demo...
34:31This is a web application based on OpenGeo Styler. Anyone heard of OpenGeo Styler?
34:40It's a component in OpenGeo GIS stack that helps you to change, dynamically change, the styles.
34:46It's not going to work out of the box because it's following their own GeoServer REST API...
34:52...but that can be slightly changed to use the GetStyles and all OGC, you know, standards.
35:00I also give the reference to the source code to that application so you can download it if you want to try it after the conference.
35:07Now I turn on the first layer. It's a WMS layer from our server; it's redfin.
35:14These yellow dots are the real estate listings that I recently downloaded from Redfin.
35:19There are a lot, right? And they're hard to see right now.
35:23So when I click the legend, since I already have the symbology definition on the client side, I can change them, right?
35:32I can change the title, say, the Houses I Want; and change the color to something more visible; the size; the boundary.
35:50Now, since there are too many houses, I want to limit that, right?
35:54I certainly don't want houses that the price is beyond half million, because I can't afford it.
36:00So I can put a list price bigger than - oh, sorry - less than 500,000.
36:14Save. So it narrows down the houses.
36:19Now I realize that I want something, because I have a kid, I want something not too small.
36:28I can put a square feet bigger than 500.
36:36Oh, I must click the - yeah, should use this, "all," instead of "any." So further narrow down.
36:49And I also want something new. I don't want some really old house.
36:54Add another condition, year built, bigger than, bigger or equal than 1990, right? Make sure I select "all" and click Save.
37:13So I further narrowed it down.
37:15Now, I can turn on another layer, it's the parcel layer from my WMS and also see the legend of that.
37:24This layer gives me, the legend gives me the information this is based on, this thematic mapping based on Asian population.
37:31So I'm from Asia, so I want to live somewhere that's close to a place that has more Asian population...
37:37...so I can more easily find a Chinese restaurant or so on.
37:42So again, see that here I can highlight these areas with more Asians with some more visible color. Right?
37:58So, Yingqi, everything is happening on the server, correct?
38:00Right, everything is happening on the server.
38:04I can also turn on the highway. I don't want to live close to a highway, right?
38:09Change the highways' symbology to highlight the highways.
38:19"Road type equals" to - copy that - Major Roads and Highways.
38:35Change the symbology a little bit, solid, to something blue, little wider.
38:46Now, since I adjust those maps and change the styles, it gives me a better idea where should I find my houses, right?
38:55So it's basically this area, far from the highways and close to our Asian population.
39:01Or somewhere here, but it's close to the highways.
39:04Now, let's stop here for a while and I'll pick up in the next demo.
39:14[Inaudible audience question]
39:19Right now it's running local on my machine, but it can definitely on the EC2 on Amazon.
39:27So it was ArcGIS Server WMS services with SLD capabilities, and you used OpenLayers along with the geo style...
39:38Right. This was basically an ArcGIS Server WMS with SLD at the server side and OpenGeo Styler client...
39:45...which is based on OpenLayers and also GeoExt on the client side.
39:54So the next demo I'm going to do is OpenLayers WFS-T editing.
39:59I want to edit some features through the web, right?
40:03This is what happens under the hood.
40:06As a client, I send a GetFeatures request to the ArcGIS Server; I'm getting features back in GML.
40:13I can do whatever I want. I can render them on the client side, I can change the styles...
40:17...and I can modify those features. I can insert new features or I can delete existing features.
40:24And when I'm done with my change, I can send all those changes to the server through transactions.
40:30And these changes will apply to the geodatabase at the back end...
40:35...and ArcGIS Server will send you a transaction confirmation saying you're successful...
40:40...how many features being inserted, how many features being deleted, and how many features being modified.
40:47To demo that, I'm going to use a slightly modified OpenLayers sample.
40:59One thing I want to mention is that earlier than 10.1, like 10 and even earlier...
41:07...ArcGIS Server only supports WFS transaction with locking, which means that you have to lock the feature...
41:14...you have to send the request to lock the feature during your editing.
41:17While you are locking the feature, no other people can touch your feature anymore...
41:22...unless you commit a change and release a lock.
41:26Now, in 10.1, we added support for WFS transaction without locking, which means that you don't have to lock.
41:34Anybody can go and edit the features, and if there is a conflict on editing, the last edits will win.
41:41And OpenLayers WFS-T clients support transaction without locking out of the box...
41:48...so in 10.1, you don't have to have any customized coding; OpenLayers can do editing against our WFS-T service.
41:56Now, this is exactly the sample that I picked from their source code and just changed the service URL...
42:04...changed the type name of the geometries.
42:07So let's pick up where we left from the last demo, right?
42:13I already narrowed down to the houses I wanted, right...
42:18...and I want to create some zones of interest around the house that I'm looking for.
42:25So currently, I have these features. This is a layer, a WFS layer. I'm getting all the polygon features.
42:32This is highlight the areas with more Asian populations.
42:38So you can see these are areas that I'm not interested in, so I can go ahead and delete those first.
42:48Right? Delete that and also let me turn on the console so you'll have a better understanding of what's happened.
43:00Let's delete all of them; let's create some new, okay?
43:04Now, the deletes only happen on the client side, right? Clear the console.
43:09When I click Save, a post request is sent to our server saying that I want to delete those features.
43:20And a response comes back saying totalDeleted 6.
43:26To confirm that, if you launch another application, you won't see any of those polygons that previously existed.
43:43That means the change has already been applied to your geodatabase, right?
43:48So keep going here. Let me create some new areas, zones of interest that I want.
44:01Somewhere here and maybe somewhere here. Okay? And click Save.
44:19Let's check the response. So it says, totalInserted 2, which are the two polygons that I just inserted, right?
44:27Now, if I turn on another WMS layer, this is the earthquake zone.
44:34I found out the zone of interest that I just created has some overlap with that, right?
44:40I don't want that, so I can go ahead and modify the features I just created.
44:53Just a little bit here and something like that. You get the idea, right? So save it.
45:08It's totalUpdated 2.
45:11So that's a demo for the WFS-T using OpenLayers application.
45:16Let's stop for a while here and move on to another demo. You want to add something, Satish?
45:24There are two versions, if you will, of the WFS. One is the read-only and the transactional aspect.
45:31So we support both the read-only as well as the transactions, and what you've seen now is being able to edit...
45:38...you know, geodatabase features via an open standard OGC protocol, using a non-Esri client, right?
45:47So you got OpenLayers being able to edit features.
45:51Yes, we'll take the question. [Inaudible audience question] I'm sorry? [Inaudible audience question]
45:58The WFS-T feature editing via an Esri client? We do not have it at this point in time, but we can discuss that in greater detail.
46:06[Inaudible audience question]
46:11I'm sorry, I have a hard time hearing. Maybe we should... Can we take that question right after the presentation? Thank you.
46:17So let's move on to the third part of the demos, which is more challenging, something that we never did before.
46:25It's geoprocessing in Quantum GIS. This is what happens under the hood.
46:33I have ArcGIS Server. I publish my GP models and tools as a GP service...
46:38...and I enable the Web Processing Service, OGC, you know, for the geoprocessing.
46:45And non-Esri clients like uDig, Quantum GIS, OpenLayers can send capabilities and describe process to the server.
46:55It will get the detail, a list. It will get the list of processes first and also get the details of each process.
47:03The process means the tools or the models that you created, right?
47:06You can have a list of input parameters and the type of input parameters and also the output parameters.
47:14Now, after getting those details, it can send and execute that process to the server to do the analysis actually over the web.
47:22Once those processes are done, ArcGIS Server will send back the result.
47:28The results can be vectors, they can be rasters.
47:31One of the cool features that we have in our WPS is that all the vectors and raster results are automatically published as a WMS.
47:41So you don't have to get all the results at one time; you can actually just look at a certain area of that result.
47:50So I'm going to have to go and demo that.
47:54First, I'm going to launch QGIS. QGIS stands for Quantum GIS. Let's pick up where we left off from last demo, okay?
48:06So I'm going to add a WMS layer first, making sure I'm using the correct projection.
48:28This is my background layer, and I'm also going to add a WFS service.
48:40So just one extra thing here. Quantum GIS has the ability to add multiple plug-ins to it...
48:46...and you need to add the WPS plug-in to Quantum GIS to be able to do any web processing service-related items.
48:57Alright. Okay, I need one more layer to quite explain.
49:15And you also notice that Yingqi has been using multiple clients to show various parts of it. That's primarily for one reason.
49:23I mean, you've got these wonderful, quote, unquote, open source clients, but they don't do everything that needs to be done.
49:30So Quantum GIS does not do some WFS transactional aspects.
49:33OpenLayers does not do a whole bunch of other things.
49:36So, I mean, that's why we use these multiple clients to make our point here as to how all this can be done...
49:43...but that's the state of art today in terms of, you know, where support is...
49:47...when it comes to many of these open source products and their support for OGC.
50:07Okay, so this is basically what we have at the last demo, right?
50:10We have our basemap, we have picked out our candidate, house candidate, right? And we also created a zone of interest.
50:24Now, the first thing that I'm going to do is I'm going to do analysis.
50:29I'm going to draw a polygon with the center of the house that I wanted...
50:37...and the area will cover the three minutes' drive around that house.
50:44So I'm using the QGIS WPS client, connecting to the GP service that I published in ArcGIS Server through the WPS.
50:56So the traditional ArcGIS Server drive-time analysis polygon's being drawn here for the three minutes.
51:02Hold on. Let me just select the houses that I wanted. Select this one, select that one.
51:17Now connect again. Use this process called CreateDriveTimePolygons. This is a WPS process.
51:28Input will be the Redfin candidates.
51:33I want to create a polygon for the three minutes' drive, and I want the output to be in my projection, the Mercator...
51:42...and I want the results to be added back to the QGIS as a vector layer.
51:49So I can click and run that. When it's running, it's actually sending the request to the server.
51:54The server will go ahead and calculate that drive-time polygon.
52:00So there is no additional code that you've written for this one, right? It's just...
52:04There's no code I've written for that.
52:07It's just the regular WPS plug-in for QGIS, and all the feature data is being handled using GML.
52:13GML for WFS. You can see I'm adding all the WFS layers and WMS layers.
52:19So when the result comes back, I choose the projection I wanted - oh, I must select all the features.
52:31Let's delete it and do it again. I missed that option here, sorry.
52:38Still choose that Redfin, three minutes.
52:42I should click that, so only the selected feature will be the input instead of all the houses.
52:48Still type in a projection and let's run it again.
52:58Okay, this time it's quicker. Comes back; it's all these are the polygons for the three minutes' drive.
53:05Let's change the style a little bit to make it hollow, change the color of the boundaries, [unintelligible], quick.
53:20Now, the next thing I'm going to do is, 'cause I have the zone of interest that I've just created and these drive-time polygons...
53:28...I want to do another analysis, which is to calculate an intersection between those two polygons...
53:33...to create the final zone of interest that later I want to extract more data from.
53:40I can connect - let's turn off this - to the same WPS and this time pick Calculate zone of interest.
53:52The input will be - oh, should turn around and go back. Connect.
54:09One of the inputs is the drive-time polygon I just created; the other input is the zone of interest I created in the previous demo.
54:19And also put the projection as my...just click Run.
54:34I did a little bit of trick at the service output.
54:36That will not exactly match these two polygons, but you get the idea that trying to calculate the intersection of those two polygons.
54:47So these are the final polygons that I want. Change the color of it, change that, solid, wider.
55:09Now, the final steps I want to do is, as Satish showed you in the very beginning, the business case...
55:16...we have a lot of data, like bars and schools and bike routes, in San Francisco.
55:22Now I want to extract all those data within those zones of interest and zip them...
55:27...and, finally, ship them to one of the e-mail addresses.
55:35Using this process called ClipZipAndShip; choosing that output, polygon layers would be the one we just created...
55:47...that's the area I'm interested in. I want the vector data to be shapefile. I want the raster data to be TIFF.
55:55And I'm sending to myself, email@example.com. And click Run.
55:59Right, exactly. Exactly, yes.
56:10So that's like a traditional clip, zip, and ship kind of operation where it takes all the data that's within the polygons...
56:16...puts it all together, and sends it out as a ZIP file to an e-mail address.
56:21And obviously, you can specify what format you want with it...
56:24...standard, GML- based, or a file geodatabase or a shapefile, and so on.
56:29So once finished, I'll get an e-mail with a ZIP attachment having all those data.
56:37[Inaudible audience question]
56:42Yeah, we can talk later on, and we can...
56:45Okay. Shall we switch or... Maybe a few more seconds?
56:53[Inaudible audience question]
57:07Looks like it's running into some errors, but let's try it again.
57:12So we'll wait for you to complete the process, and I'm going...
57:14Yeah, you can switch. Let me try it again.
57:16And then we'll have enough time for questions and so on. Right.
57:18I just have two more slides, and what we have up here is, you know, when we talk about OGC support, you know...
57:25...within ArcGIS 10, 10.1, whatever, unless you understand the OGC space really well...
57:32I mean, most people understand them as, okay, we got WMS and WFS.
57:36But there are a whole bunch of what you call additional specifications...
57:39...that work in conjunction, if you will, with WMS and so on.
57:44Some of that Yingqi mentioned. He talked about the SLD, which is a Styled Layer Descriptor...
57:49...allowing users to submit styles and filters along with their requests.
57:53Then there's filter encoding. There's time support. There was WFS as read-only and there's transactions.
57:59With WCS, it's a question of whether you can support these multiple different formats.
58:03With WMTS, do you or do you not support REST-ful interfaces, as well as the KVP encoding?
58:09With WPS, it's a question of do you support synchronous/asynchronous and so on?
58:13And with the WPS services, data could be provided either as GML, or it could be actually provided as a WFS service...
58:20...that's bringing in GML, and if there's any rasters that are being written out, they can just be written out as files...
58:25...or maybe they can be automatically published as a WMS, a kind of service chaining, if you will.
58:30And then we also have support for the Catalog Services Web.
58:34We have traditionally had, you know, a great amount of difficulty trying to do this particular type of presentation every year...
58:42...because there is so much to tell.
58:45We're kind of doing the whole GIS spectrum within this one hour, 15 minutes or whatever we have for this.
58:52And at this point, he has not been able to show WCS, we haven't had time to show OGC KML.
59:00We have not even been able to show, you know...
59:02...the fact that we have got multiple server products that support some of this stuff.
59:07We've got the Geoportal extension, we've got the ArcGIS Server.
59:10When it comes to clients, you know, there could be a few holes here and there...
59:14...but the point is we do support many of these with our desktop applications.
59:19We support many of these using our various APIs, and by "APIs," we're talking of Silverlight supporting WMS, WMTS...
59:31Then you've got the Flex environment supporting WMS and WMTS and so on.
59:37So, to me at least, I think our strategy for OGC support is very comprehensive. And it does take time.
59:46Sometimes we have to catch up on a few things, because remember, OGC specifications have multiple versions.
59:52If you have WMS, there is WMS 1.0, 1.1, 1.1.1, and so on.
59:56And then there are the SLDs, which have their own versions, on top of which we have multiple servers...
1:00:02...and multiple products, clients, and each of our solutions have their own versions, right?
1:00:08So we're dealing with a pretty complex space out here, in spite of which, I think with great confidence...
1:00:15...I think we should be in a position to tell you that, hey, in 10.1, especially with WPS, with the web processing service...
1:00:22...which is kind of bleeding edge as far as I'm concerned because there's not too much support for that in the outside world.
1:00:29I think we've done a fairly bang-up job with our 10.1 support for standards.
1:00:33So I'd like to kind of finish it off with this and then take some questions.
1:00:39If you're aware of the OGC space, I mean, going back to my first slide, we talked of proprietary standards...
1:00:46...and open specs when a vendor releases its specification, and then there's standards-related stuff.
1:00:53Our shapefiles are an open specification.
1:00:55We recently released the File Geodatabase API as an open specification, right, which meant anybody would be...
1:01:03...in a position to read or write a file geodatabase using some openly provided libraries that we have.
1:01:10Currently, the Esri REST API, for the last year and a half or so, has been an open specification.
1:01:20September of 2010 was when we released our REST APIs, an open spec...
1:01:23...which meant anybody could go ahead and implement the same function signatures, if you will...
1:01:29...on top of their server platform or have a client that can consume the REST API, right?
1:01:35So if you, for example, had your own home-built server and you could somehow mimic the same thing...
1:01:41...that our map service did or our feature service did, you could go ahead and do that with no IP restrictions...
1:01:46...and you could participate in this wonderful large ecosystem that we've built around it with all our client APIs and so on.
1:01:52But what has happened in the last eight months or so and continuing to happen is we're taking this open specification...
1:01:58...and we're working it through the OGC process.
1:02:02So all the members of the OGC are sitting together, announcing, okay, Esri has brought this REST specification...
1:02:08...because OGC, with their KVP encoding and SOAP focus, has not really spent time on the REST stuff.
1:02:15So with all the JSON encoding and the REST way of thinking, people see a lot of value in our particular offering.
1:02:22So right now, it's working its way through the OGC process, and fingers crossed, you know.
1:02:27There'll be questions that are being raised, and we're trying to, you know...
1:02:29...fix whatever things that other members have an objection to.
1:02:34So over time I think this will all smoothen out, and if everything goes well...
1:02:39...there's a good possibility that the geoservices REST...
1:02:42...or the Esri REST [unintelligible] release of the geoservices REST specification, would also become an OGC standard...
1:02:48...helping, you know, GeoServers and the Minnesota MapServers and whatever other open source stuff out there...
1:02:54...to participate more closely with some of what Esri is doing.
1:02:58So with that, I'd like to say thank you, and I'll take questions. And since Yingqi's process did complete okay?
1:03:05Well, what happened is actually - the process actually executed, but it turns out the connection...
1:03:09...network connection here fails to actually connect to the Esri mail server.
1:03:14So there's some network restriction, so I'm getting this...
1:03:20Yeah, so... I need to try it outside; somehow didn't work.
1:03:23Okay, so we have just a few minutes, I think, but we're willing to take questions.
1:03:27And for those of you who have more specific questions and want to grill us more, I mean, I'm definitely available after this presentation.
1:03:34So with that, thank you very much. Yes, sir.
1:03:36[Inaudible audience question]
1:03:41Okay. That was something that we have not touched upon today, on the sensor web stuff.
1:03:46So what we have done is we've got some business partners and...
1:03:51...we're part of this relationship with an organization called 52North out in Germany, which is an open source organization.
1:03:58We've worked very closely with those guys, and for 10.1 platform - it's not final for release yet, this particular component...
1:04:06...what we have, what we will release is an open source server object extension for ArcGIS Server...
1:04:14...that conforms to the SOS, or the Sensor Observation Service specification.
1:04:19So we do have a solution which is ready, it's just that it's not been put out there, which we should be doing in a couple of weeks.
1:04:28So that's a totally open source extension to ArcGIS Server, meaning you can take it, modify it, do more stuff with it.
1:04:35Okay, more questions? Yes, sir.
1:04:37[Inaudible audience question]
1:04:45To do...to do, yes, yes. [Inaudible question, continued] Right. The geodata...
1:04:50[Inaudible question, continued]
1:04:5310.0 SDE, for sure, 'cause I use 10.0 ArcSDE so...
1:04:58So he'd be able to use 10.0 SDE with the 10.1 server.
1:05:00No problem. Yeah.
1:05:01I mean, the rules will be the same for what you use with a map server and other stuff.
1:05:07Yeah, 10.0 should be fine. 9.3, I wasn't sure if there was any migration needs to do to migrate your 9.3 SDE to 10.0...
1:05:17...before you can have your 10.1 server to connect with it.
1:05:20But the more important point is...
1:05:21But 10.0 is okay.
1:05:23More important point is the OGC part does not introduce anything different from what you would need for the rest of the stuff.
1:05:29Right? Yeah. More questions, please? Yes.
1:05:35[Inaudible audience question]
1:05:43With replication? There has been an attempt.
1:05:46It is, I think, available as some kind of, what they call, best practices or discussion paper...
1:05:52...and you would go in and search for geosynchronization. That is the name of that spec that they're working on.
1:05:59It's not a spec; it's just some ideas how GML-based geosynchronization could happen between multiple databases.
1:06:06[Inaudible audience question]
1:06:10I haven't followed that particular part very closely, but it's GML based, works on top of WFS...
1:06:17...and it brings its own host of problems, I guess.
1:06:21Yes. And we can continue that conversation on the WFS-T.
1:06:25What is important is traditionally, we've been very server-centric, or server focused...
1:06:32...so most of our effort on OGC development has been on the server.
1:06:35We do have client support, but like I said, we have multiple clients, so there may be small areas where we are falling short...
1:06:41...as in we don't have a WFS transaction client for the desktop.
1:06:47But traditionally, we've rarely run into good servers out there that are actually publishing WFS-T services.
1:06:55So our focus has primarily been whatever resources Esri folks have, you know, they should be in a position to expose them...
1:07:03...using standardized interfaces so others can consume it.
1:07:07But I must say that our story both in server and client with 10.1 is quite complete.
1:07:14Just another note that all the materials, the slides and the data and the GP tools, that I used to create those demos...
1:07:22...I will package them in a package, and I'm going to upload it to ArcGIS.com pretty much after the conference.
1:07:30So just after the conference, just go to ArcGIS.com and type search DevSummit 2012 and OGC...
1:07:36...should be able to find that download link.
1:07:39So you should be able to get it all...
1:07:40If you're interested in reproducing something at home with 10.1 Server, so welcome to do so.
1:07:47Right. Question right there. Yeah. Is there more questions?
1:07:56Well, if there's no more questions, thank you very much for attending this.
Leveraging OGC Services in ArcGIS for Server
Satish Sankaran and Yingqi Tang demonstrate how to publish ArcGIS for Server services as OGC web services.
- Recorded: Mar 28th, 2012
- Runtime: 1:08:00
- Views: 942
- Published: Apr 23rd, 2012
- Night Mode (Off)Automatically dim the web site while the video is playing. A few seconds after you start watching the video and stop moving your mouse, your screen will dim. You can auto save this option if you login.
- HTML5 Video (Off) Play videos using HTML5 Video instead of flash. A modern web browser is required to view videos using HTML5.