00:01Today's topic is leveraging OGC services in ArcGIS Server.

00:06How many of you here have had a chance to see something similar in a previous conference or at the UC?

00:11Okay, good. Thank you.

00:14Second question. How many of you are familiar with OGC services? Okay, almost everybody. Good.

00:23How many of you use ArcGIS Server or the various clients that we have to work with OGC services today? Most of you.

00:34Are most of you, are you on the 10.0 platform? Okay. Okay, few of us.

00:41Has anybody tried the 10.1 beta prerelease, the OGC parts of it? Oh, you have. That is fantastic. Okay.

00:53With much success? [Inaudible audience comment] Fifty-fifty success.

01:00Okay, then. We should have the conversation shortly after the presentation then.

01:05[Inaudible audience comments]

01:07Okay. No, we just want to make sure. We're here to help.

01:12How many of you use non-Esri products, be it GeoServer, Minnesota MapServer...

01:19...or any type of open source clients to consume these OGC services?

01:23That's excellent too, 'cause one of the old ideas behind this is to, you know...

01:28...create interoperable systems so we can communicate with one another.

01:31Okay. Quick introductions. My name is Satish Sankaran. I work for Esri; I am in product management.

01:38I take care of all things related to interoperability and standards.

01:43I wear multiple hats. I am also the Esri representative to the OGC, so work with OGC; I'm on their...architecture board.

01:54I chair the WMS standards working group and a whole bunch of other things.

01:59So I should be in a position to answer most of your questions, either here or later, not just related to...

02:05...questions related to our product support but also about, you know, some specific standards-related stuff that you might have.

02:12We have Yingqi Tang here. He is part of our software development team back in Esri...

02:18...so he's going to do the second half of our presentation.

02:20I'm here primarily to set the stage and tell you what is it that we should all care about when it comes to interoperability...

02:28...and he's going to dig deep into some of the new things with 10.1...

02:34...and kind of do some nice demonstrations to show why these services could be really useful.

02:41So with that, I think we should start.

02:45Well, what is GIS all about, right? I mean, fundamentally, as I see it at least, there are two things.

02:51It's about creating and managing geoinformation products...

02:55...and then ultimately being able to disseminate those products that you create.

03:01So during the course of working with GIS, you could be working with proprietary formats or protocols...

03:09...or you could be working with some kind of open specifications that have been, you know, published by a specific vendor...

03:18...and that allows you to work with multiple vendors who are able to, you know, work with those specifications.

03:26And sometimes you would be working with some standards-based protocols.

03:32So open specification is when a vendor is opening up his format or his protocol for people to use with no IP restrictions and so on.

03:41Standards is not just one vendor releasing stuff, but usually it happens under a consensus process.

03:47So the OGC, Open Geospatial Consortium, is a geospatial organization where a bunch of members from different vendors...

03:56...from academia, and so on, get together to develop standards that are mutually beneficial to the community.

04:04Today's presentation, we're going to focus on the last aspect of it, which is the standards-based stuff.

04:10So we'll get into the details of some of the OGC standards and how we support them.

04:18On a broader note, though, I mean, since we are at an Esri conference, we need to understand what ArcGIS is...

04:25...and as you have seen this slide many times before, it's a complete system.

04:29So it's not just about, you know, using, creating, and managing your data, but we're living in a very services-oriented world today...

04:37...so you could have your data sitting up there somewhere in a cloud, others could be on your local server...

04:44...or it could be on your enterprise server, if you will.

04:47And you want to provide that information over to a multitude of different clients.

04:55This could be web-based clients, this could be mobile clients, this could be just, you know, desktops.

05:01And when we talk of providing the data, we're talking of not just data being provided through a service...

05:07...but you want to be able to provide different kinds of services that people may want to access.

05:12Discover services, services that allow you to create data...

05:15...services that allow you to just manage data, to visualize data, and so on.

05:24This slide talks about ArcGIS being open and interoperable.

05:31Not everybody uses the same kind of clients, not everybody wants to work with the same programming environments.

05:41Some people want to use .NET, some people want to use Java, so on.

05:45Some people want to use standard-based protocols or formats or interfaces.

05:50Some people don't really care if it's a standard, but they just want to make sure it's open in some way...

05:54...so that they can access it however they want.

05:58So any strategy for interoperability should be kind of comprehensive.

06:06So essentially, we need - as a vendor, our aim is to support a range of strategies for tackling the interoperability problem.

06:16And I also use this slide to emphasize that the bigger problem that we're trying to solve here is the interoperability problem...

06:22...and standards is kind of one solution to it. Right?

06:26But that's what we're focusing on in this presentation, but you have to bear in mind that what we're trying to do...

06:30...is to make sure that our systems are interoperable so that you can, you know, do what you need to do.

06:39Before we go into the OGC services, I've tried to distinguish some of the important aspects that we need to understand...

06:46...like the foundational stuff, before we can get into the services aspect.

06:50There's standards that are used in the creation of geoinformation products...

06:54...and there are some standards that are used in the dissemination of that information.

06:58And this is not an all-inclusive list, but I'd just like to spend a couple of minutes on a couple of these...

07:07...so that we understand some of the important ones that'll help us understand some of the services-based OGC standards better.

07:14To start with, the Simple Features model. When it comes to data standards...

07:19...we'll just talk a little bit about Simple Features model and then a little bit about GML, the Geography Markup Language.

07:25Then I'd like to talk a little bit about metadata standards, and then, as far as dissemination goes...

07:31...we will then talk about the OGC services and how we support them, which is kind of the core of our presentation this morning.

07:38Okay. Simple Features is an OGC standard, also an ISO standard, but it's so transparently been used for so many years...

07:48...people tend to forget its importance.

07:50Almost everything that we do or everything that most vendors work with is all based off of the Simple Features standard.

07:59Right? It provided a data access model; it provided a database schema; it told you how to store vector geometry...

08:05...points, lines, and polygons; how would you store it in a database or otherwise.

08:11And this was a specification that's been in existence for probably now over 10 years.

08:20If you look at the actual specification document, there are two documents there.

08:23There's a Simple Features Specification, the common architecture spec, which talks about a geometry model...

08:29...and how to represent that geometry using, say, well-known text representation or a well-known binary representation...

08:37...and also talks about how to represent spatial reference systems using a textual representation.

08:43The second part of the spec talks about SQL options.

08:47How do you represent all this information in a database?

08:50So it introduces the concept of a SQL geometry type and also introduces standardized routines, SQL routines for, you know...

08:58...constructing and obtaining geometry in these various different formats from the one that's stored in the database.

09:06So that's Simple Features, and for most part, Simple Features is actually a misnomer...

09:11...because what they mean by "simple features" is really simple geometry.

09:15You're talking of points, lines, and polygons. All the lines are basically, you know, just connections of points.

09:22The curves are not - we're not talking of spirals and, you know, NURBS, and various different kinds of curves.

09:29But we're just talking about simple, you know, points that are joined together in a curvilinear fashion.

09:37Okay. So that's how geometry is usually stored and how we work with it in our database environment.

09:42When we talk about a services environment, sometimes we want to be able to provide the features out to a client.

09:49And how we do this normally in the OGC world of things, we do it using GML...

09:55...which is an XML-based encoding for geographic information.

10:00We're not going to go into the details here, but GML allows you to represent geometry, spatial reference...

10:06...topology, you know, 3D objects, and everything under the sun.

10:10So the concept of profiling is normally used so that you can reduce that large GML subset to something simple...

10:20...so that clients can easily work with it.

10:23For example, a point profile reduces the vocabulary use so that all it does is tells you how to represent a point using GML.

10:32We talked a lot about the Simple Features model but just representing points, lines, and polygons...

10:36...so GML, which is an XML encoding, has the ability - has been profiled in such a way to only represent simple features.

10:45So that's called a GML Simple Features profile...

10:47...and that's kind of the one that Esri has been advocating for and using within its product stack.

10:54Then there's the question of application schemas for GML...

10:57...and there are multiple application schemas that are very domain specific.

11:01They use the profiles, but they use also terms and vocabularies that are specific for a domain...

11:07...be it the cadastral people or for transportation networks and so on.

11:11So as you start understanding GML better, you need to understand...

11:14...what the concept of profiles are and what the concept of application schemas are.

11:22Then I would just like to talk a little bit about metadata standards.

11:25We all know what metadata is for.

11:27It provides resource for users, an idea of metadata they should collect, and why it is important and so on.

11:32But more importantly, just two slides here.

11:35One on the fact that the 10.0 release of our software in the desktop supports a whole variety of metadata standards...

11:42...you know, the ISO 19139, along with the North American Profile for that, the INSPIRE standard, and a couple of others.

11:53In 10.1, the metadata support is kind of, it extended over to the server too.

12:02So if you have a 10.1 machine, and if you want to access...

12:08Let's say you're publishing a map service from an MXD, and the MXD has some ISO 19139 metadata.

12:13In 10.1, you're in a position to actually access that metadata via a service end point.

12:23So people that are interested in being able to access service-level metadata at 10.1...

12:29...you know, we could definitely talk about it after this presentation if you have questions.

12:34Okay. So at this point, we're going to delve into the services aspect of it...

12:42...and again, everything that we talk about these days is all services-oriented architecture, if you will...

12:48...so normally you would have a discovery service where you'd have some metadata so you can go in...

12:54...search for a certain kind of functionality or a service, be able to access it.

12:58Then, if it's a view service, you can get to view some maps and so on.

13:01If it's a download service, you're in a position to go and download data and work with that data.

13:07So keeping this in mind, obviously if you are a geo professional and you are thinking in a service-oriented fashion...

13:14...you'll think, okay, this sounds good. What is it that I can do? Right?

13:18So immediately what comes to your mind is the concept of a map service, concept of a geocode service.

13:24In Esri, we have geoprocessing and geometry services that are kind of similar, concept of an image service...

13:30...and a feature service that provides access to feature querying and editing.

13:34Okay. What I will do is I will stop at this point with this and then step into a quick use case scenario...

13:45...something to set up the stage for Yingqi to continue the rest of his presentation.

13:51So imagine a scenario where we're trying to work with services to solve a specific problem.

14:02It's a little bit of a contrived situation, but it helps us build a demo to explain what we can do.

14:07So essentially, Yingqi probably wants to search for some, you know, houses out in San Francisco...

14:14...so one way of doing that is he can get a lot of data that he wants, you know, from some database somewhere...

14:21...which has the cost of houses, and then he can put them on the map and see, you know...

14:26...what kind of localities they are situated in and what kind of grocery stores, et cetera, et cetera, that are right next-door...

14:34...and he can make a decision based off of that information. Right?

14:39Let's assume instead what he gets is a bunch of services; that raw data that's available is provided to him as services. Right?

14:50So in his demonstration, what we're trying to get at is...

14:54...how would you be able to do those kinds of analysis from a services standpoint?

15:00All he has access to is, say, some kind of a map service that provides all kinds of data.

15:06So in this particular one, he sees a whole bunch of homes that are being made available...

15:11...but somehow he wants to be able to only visualize the homes that fall within a particular cost bracket.

15:20Beyond which, he also wants to specify some kind of a polygon that says, hey, I want to have...

15:26...look at homes that are only within this particular area of interest.

15:30And not only that, he wants to, at some point, even want to edit those polygons so that if it's an earthquake-prone zone...

15:36...he wants to remove those parts of it from the polygon, so kind of restrict his filters even further.

15:43And sometimes he may want to do some additional geoprocessing analysis. Could be a drive-time analysis or something.

15:49He wants to do that and he wants it to, you know, interact with the polygons that he's built, filter it further...

15:54...and then be able to take all the data that's within that intersection and send it as an e-mail over to somebody or to himself...

16:03...so that he understands, you know, if this is the house that I pick, what kind of facilities are available right next to the house.

16:09So I think this is a reasonably good use case, which will help us exercise different parts of our software.

16:18All this now is these map services, feature services, and so on, but I'd like Yingqi to do the demonstrations...

16:26...talk about it from an OGC perspective, how would you do this if all you had was an open source client of some sort...

16:34...and you were hitting a map service, a WMS, a WFS, you know, a WPS, and so on.

16:40So the challenge to Yingqi is to do this whole thing in an OGC context, if you will. Yingqi?

16:55Alright, thanks, Satish. So I'm going to follow up, talk about a little bit more on OGC web services in ArcGIS Server.

17:04The business case in San Francisco that Satish just brought up is actually artificially, you know, designed for the purpose...

17:11...that we want these different little requirements so we can touch those different functional areas of ArcGIS Server.

17:21Now, in an Esri ecosystem, these little requirements can be actually done by people publishing different types of services.

17:30For example, map service for the dynamic mapping, the tiled map service for fast tile mapping...

17:36...or the feature service for if you want an editing feature in a geodatabase over the web...

17:41...or finally, a geoprocessing service to do some spatial analysis over the web.

17:47Once those services are up running, there are tons of Esri clients that can consume those services.

17:52We have the desktop solutions and the web APIs, the cloud solutions...

17:58...and the mobile for those common, you know, mobile platforms.

18:03I'm pretty happy with these, and as long as all your clients, potential clients or users, are always in this Esri ecosystem.

18:13But it's not always a perfect world, is it?

18:18So what if there are users coming beyond this ecosystem?

18:23What if a user is coming using gvSIG, uDig, Quantum GIS, Gaia, Google Earth, OpenLayers, GeoCommons, and so on?

18:33How about other commercial GIS vendors like MapInfo or Intergraph?

18:38Currently, none of them can consume any type of service from ArcGIS Server.

18:44Now, what's the solution for that?

18:47The solution is to leverage OGC services in ArcGIS Server.

18:53This actually is a very important slide.

18:56I put a lot of time on it, so if you can only remember one slide when you walk out of the room, please do remember this one.

19:04So this slide carries two important information. The first important information is kind of obvious.

19:10As far as 10.1 goes, we have implemented almost all the major OGC web services here.

19:17WMS has been here for a while for the dynamic mapping.

19:21WMTS, the OGC Web Map Tile Service, is for serving up really faster tile, the cache and tile maps.

19:30WCS is serving out the raw raster data in different formats, like GeoTIFF, JPG2000, HDF, and so on.

19:38WFS, the Web Feature Service, enables the feature accessing over the web, also the feature editing over the web.

19:46WPS, which is new in 10.1, Web Processing Service, is for you to publish...

19:54...your tools and models for the geoprocessing over the web.

19:58And also KML. It's serving out a mixture of data, also the visualization information together.

20:04Now with all those OGC services enabled, what's happened? You can see I put a happy face here.

20:11Now all the clients, non-Esri clients on the left side, are now more or less...

20:16...being able to communicate with ArcGIS Server based on their own scope.

20:23Just to give you a little bit basic feelings of how they work, this is a screen shot of Quantum GIS...

20:32...a very popular desktop GIS consuming the WMS service and WFS service from ArcGIS Server.

20:42Gaia, a very light, free OGC client, although not free.

20:48In this case, it's a screen shot, it's connecting and consuming a WMS from ArcGIS Server...

20:53...and try to identify some features from our server.

20:58Again, Gaia, it has an extension that enables you to edit features in our geodatabase...

21:04...through the WFS-T transaction protocol.

21:09Although it requires a license, this extension's not free, but Gaia's free.

21:15Gaia again, consuming our tiled map service.

21:20gvSIG, which is also a very popular desktop GIS solution from Spain...

21:25...connecting and consuming a WCS, Web Coverage Service, from our server.

21:31uDig, also the same story, consuming a service from our ArcGIS Server.

21:38Now, this is OpenLayers WFS-T.

21:41Actually, this application is built right up on the Autobox sample of OpenLayers, doing the web editing through WFS-T.

21:54GeoCommons, very popular cloud GIS solutions consuming WMS from ArcGIS Server.

22:00GIS Cloud, another cloud solution.

22:04GeoNode, it's the enterprise or cloud solutions from OpenGeo, also consuming the WMS from ArcGIS Server.

22:17Now, the second important information that's conveyed in this slide is not quite obvious.

22:24It's about architecture we implement in our OGC services.

22:27Now, instead of creating those OGC services from scratch...

22:32...our goal is to simply expose the rich functionality from ArcGIS Server services through the OGC interfaces.

22:42And the value, the benefit of that architecture is what I called inheritance.

22:49Now, inheritance is a term we use in object-oriented programming which refers to subclass, you know...

22:55...inheriting all the attributes from its parent class.

23:00What I want to point out here is OGC services inherit a lot of attributes from ArcGIS Server services...

23:07...which can all be summarized here - cartography, security framework, performance and scalability...

23:17...authoring, publishing, and sharing workflow.

23:20And I'm going to dig a little bit into each of them.

23:23Cartography, right. So when you try to publish data as beautiful maps, what usually takes most of the time is what?

23:32It's cartography, right? What kind of symbology you're going to use...

23:36...what labels and fonts you're going to use, and things of that nature.

23:41And Esri is very good, very famous for having good tools to help you to create those beautiful maps.

23:48And all the tools are GUI based, so you don't have to manually work with those XMLs, which are designed for the machines.

23:57Just to give you an example. If you have ever worked on SLD, what do you think...

24:02...how hard do you think it is to create those beautiful maps all manually in SLD? Right?

24:09Is there any tool to help you do that? Yes, there are a few tools, but none of them are even close to what ArcMap can provide.

24:17So, Yingqi, what is SLD?

24:19Oh, yeah. SLD, I'll just clarify what SLD...SLD stands for the Styled Layer Descriptor.

24:25It's also an OGC specification, which helps you to define the symbologies and encodings for a map, for the data.

24:37Now, another example is the GeoCat. Anyone heard of tools called GeoCat Bridge? Right.

24:45So it's a commercial tool that helps you to publish the map that you created in ArcMap to GeoServer.

24:51Now, just think about it, why there is such a commercial tool [that] exists there to help you to publish the map...

24:59...created in ArcMap to some other server products. Because our cartography's good, right?

25:06Security. OGC specs don't define any standard or best practices to secure services...

25:14...but ArcGIS Server does provide a very powerful security framework to secure your services.

25:20And within the same framework, OGC services can all be secured the same way your ArcGIS Server services can be secured.

25:28So we support token-based security, we support HTTP basic and digest.

25:33And all your users and groups can come either from Windows or relational database or LDAP. It's very powerful.

25:45Performance and scalability is always a top priority in our development and QA process.

25:52Just giving an example here, this is a chart of throughput showing one of our benchmark WMS running on a single machine...

26:01...and about 60 percent of CPU usage, and you can see it's stably producing maps, around 40,000 maps per hour.

26:12This is another example.

26:14This is a chart of throughput of another of our benchmark WMS scaling out on a six-node cluster.

26:23Each of the nodes will be running, and they're about 60 percent of the CPU, and you can see it scales it perfectly.

26:30And if you're using full, using the six nodes of the cluster, you're getting close to 800,000 maps per hour.

26:40And just so you know, these charts and numbers, we are actually very carefully monitoring and collecting them...

26:47...on a weekly basis and even daily basis during our development and the QA process...

26:53...to make sure our OGC services have the good performance.

26:58Authoring and publishing. Now you guys must have seen a lot of demos for the 10.1, see how easy it is to author the map...

27:06...package the data, and publish to the local server, remote server, or even the cloud.

27:11OGC services will have the same publishing-authoring workflow - smooth and easy.

27:17Cache management. We added WMTS in 10.1, right? And it's not just the fact that you're having this new service.

27:26You're actually having a set of tools, very powerful tools, to help you to create, to manage, and to update the tiles that you have.

27:35Same idea applies for the web processing service.

27:37It's not just a new service; you have the ModelBuilder, you have the scripting environment in the back end...

27:44...to help you to create the GP functions to aggregate those different single GP functions into powerful models. Right?

27:54Now, let's look at some of the differences between ArcGIS Server services and OGC services, which can be summarized as these.

28:01Interfaces, different; transfer protocols are different; encodings are different; and clients are different.

28:09These are some details of the differences listed in this table.

28:13You can see that OGC uses the OGC specifications...

28:15...and ArcGIS Server services use a SOAP API and our geoservices REST API.

28:20Transport protocol is different. OGC uses the REST-ful, HTTP basic or HTTP CAT or HTTP POST.

28:27And ArcGIS Server services using SOAP and the REST.

28:30Encoding. Most of OGC using XML, different flavors of XML - GML, SLD, feature encoding, and so on.

28:37ArcGIS Server services use a SOAP XML and our geoservices REST JSON.

28:43And of course, the clients are different.

28:45All the OGC-compliant clients are supposed to talk, to communicate with ArcGIS Server through the OGC's...

28:51...and for the Esri products, for the ArcGIS Server services...

28:53...we're using ArcGIS Desktop, web APIs, ArcGIS.com, and so on.

29:00Now, before we go into the demos, I really want you guys to have a look at this little cheat sheet for our OGC services.

29:09I created this, spent a lot of time on it.

29:12It's kind of a good reference for you to look at how our OGC services evolved from 9.2 all the way to 10.1 here.

29:24And I put all the major events, additions in each release.

29:29You're probably not going to see it clearly in these slides...

29:31...but I'm going to package up this PDF file and upload it to ArcGIS.com, so after the conference...

29:37...you guys can go and just search DevSummit 2012 OGC; you'll be able to find this very nice reference.

29:49Now let's go into the demo part. The first little demo - it's not really a demo demo; just a recap of how you publish ...

29:56...OGC services on ArcGIS Server, because the whole publishing workflow has changed from 10.0 to 10.1.

30:04But the publishing workflow for OGC, it kind of remains the same.

30:09You basically follow, publish ArcGIS Server services, go to the service properties and enable the capability of different OGC.

30:18I'm going to show you a little bit on that.

30:23So to save some time, I already have those services published into my server.

30:28You can see there are some map services for different purposes, and there is a GP service here.

30:36So if you right-click a service, okay, go to the service properties...

30:56...it's going to show you a list of available OGC capabilities for that type of services, right?

31:01For the map service, you have WMS, you have WFS, and WCS; for the geodata service, you have WFS and WCS...

31:12...and for the GP, you will have WPS.

31:16My hard drive is still spinning. Hopefully...'cause it stays sleeping probably for a while. Alright, here we go.

31:27So go to your capabilities. You can just enable WMS or WFS, KML's enabled by default.

31:36And if you click all those enable OGC services, you can configure the properties...

31:43...and there are options you can configure for those services.

31:48Same idea for the GP. Website takes time, so I'm going to just go ahead.

31:55Now, another important useful tool is in Manager, ArcGIS Server Manager. Log in.

32:09You will be able to do the same administrating operations - publishing a service, stop, enable OGC, and disable OGC.

32:17One thing very useful here is this OGC Services tab.

32:21If you click on that, you have a drop-down box that lists all available OGC services on this server - WCS down to WPS, right?

32:35And also another useful tool is the site directory. It lists all the services.

32:42If you go to one of the services, you can see that the OGC services that are being enabled on the service are listed here as links.

32:51So if you click on the link, it's going to point you directly to the capability operation of that OGC service.

32:58You're getting the XML back directly, so that's going to be the URL you will copy and paste into the non-Esri client...

33:06...to connect to the URL service.

33:16Okay, so next real demo is dynamic rendering using WMS and SLD.

33:24So WMS basically provides you the dynamic mapping over the web; SLD helps, you know...

33:30...extends WMS for you to get the symbology definition from the server, lets you modify the symbologies...

33:36...and sends back the new symbology to the server so the map can be changed, the styles of the map can be changed on the fly.

33:47This is just what happened under the hood, right?

33:50You have a non-Esri client, you're first initially sending a GetMap request to get a map from our server.

33:57You're also sending GetStyles request to get definition of the styles in XML, SLD XML.

34:03In the client side, you can adjust those styles by changing the XML...

34:07...and finally, you can send back the modified SLD XML to the server.

34:12The server will render a new map based on your change, right? So I'm going to show that.

34:21The client I'm going to use for this demo...

34:31This is a web application based on OpenGeo Styler. Anyone heard of OpenGeo Styler?

34:40It's a component in OpenGeo GIS stack that helps you to change, dynamically change, the styles.

34:46It's not going to work out of the box because it's following their own GeoServer REST API...

34:52...but that can be slightly changed to use the GetStyles and all OGC, you know, standards.

35:00I also give the reference to the source code to that application so you can download it if you want to try it after the conference.

35:07Now I turn on the first layer. It's a WMS layer from our server; it's redfin.

35:14These yellow dots are the real estate listings that I recently downloaded from Redfin.

35:19There are a lot, right? And they're hard to see right now.

35:23So when I click the legend, since I already have the symbology definition on the client side, I can change them, right?

35:32I can change the title, say, the Houses I Want; and change the color to something more visible; the size; the boundary.

35:50Now, since there are too many houses, I want to limit that, right?

35:54I certainly don't want houses that the price is beyond half million, because I can't afford it.

36:00So I can put a list price bigger than - oh, sorry - less than 500,000.

36:14Save. So it narrows down the houses.

36:19Now I realize that I want something, because I have a kid, I want something not too small.

36:28I can put a square feet bigger than 500.

36:36Oh, I must click the - yeah, should use this, "all," instead of "any." So further narrow down.

36:49And I also want something new. I don't want some really old house.

36:54Add another condition, year built, bigger than, bigger or equal than 1990, right? Make sure I select "all" and click Save.

37:13So I further narrowed it down.

37:15Now, I can turn on another layer, it's the parcel layer from my WMS and also see the legend of that.

37:24This layer gives me, the legend gives me the information this is based on, this thematic mapping based on Asian population.

37:31So I'm from Asia, so I want to live somewhere that's close to a place that has more Asian population...

37:37...so I can more easily find a Chinese restaurant or so on.

37:42So again, see that here I can highlight these areas with more Asians with some more visible color. Right?

37:58So, Yingqi, everything is happening on the server, correct?

38:00Right, everything is happening on the server.

38:04I can also turn on the highway. I don't want to live close to a highway, right?

38:09Change the highways' symbology to highlight the highways.

38:19"Road type equals" to - copy that - Major Roads and Highways.

38:35Change the symbology a little bit, solid, to something blue, little wider.

38:46Now, since I adjust those maps and change the styles, it gives me a better idea where should I find my houses, right?

38:55So it's basically this area, far from the highways and close to our Asian population.

39:01Or somewhere here, but it's close to the highways.

39:04Now, let's stop here for a while and I'll pick up in the next demo.

39:14[Inaudible audience question]

39:19Right now it's running local on my machine, but it can definitely on the EC2 on Amazon.

39:27So it was ArcGIS Server WMS services with SLD capabilities, and you used OpenLayers along with the geo style...

39:38Right. This was basically an ArcGIS Server WMS with SLD at the server side and OpenGeo Styler client...

39:45...which is based on OpenLayers and also GeoExt on the client side.

39:54So the next demo I'm going to do is OpenLayers WFS-T editing.

39:59I want to edit some features through the web, right?

40:03This is what happens under the hood.

40:06As a client, I send a GetFeatures request to the ArcGIS Server; I'm getting features back in GML.

40:13I can do whatever I want. I can render them on the client side, I can change the styles...

40:17...and I can modify those features. I can insert new features or I can delete existing features.

40:24And when I'm done with my change, I can send all those changes to the server through transactions.

40:30And these changes will apply to the geodatabase at the back end...

40:35...and ArcGIS Server will send you a transaction confirmation saying you're successful...

40:40...how many features being inserted, how many features being deleted, and how many features being modified.

40:47To demo that, I'm going to use a slightly modified OpenLayers sample.

40:59One thing I want to mention is that earlier than 10.1, like 10 and even earlier...

41:07...ArcGIS Server only supports WFS transaction with locking, which means that you have to lock the feature...

41:14...you have to send the request to lock the feature during your editing.

41:17While you are locking the feature, no other people can touch your feature anymore...

41:22...unless you commit a change and release a lock.

41:26Now, in 10.1, we added support for WFS transaction without locking, which means that you don't have to lock.

41:34Anybody can go and edit the features, and if there is a conflict on editing, the last edits will win.

41:41And OpenLayers WFS-T clients support transaction without locking out of the box...

41:48...so in 10.1, you don't have to have any customized coding; OpenLayers can do editing against our WFS-T service.

41:56Now, this is exactly the sample that I picked from their source code and just changed the service URL...

42:04...changed the type name of the geometries.

42:07So let's pick up where we left from the last demo, right?

42:13I already narrowed down to the houses I wanted, right...

42:18...and I want to create some zones of interest around the house that I'm looking for.

42:25So currently, I have these features. This is a layer, a WFS layer. I'm getting all the polygon features.

42:32This is highlight the areas with more Asian populations.

42:38So you can see these are areas that I'm not interested in, so I can go ahead and delete those first.

42:48Right? Delete that and also let me turn on the console so you'll have a better understanding of what's happened.

43:00Let's delete all of them; let's create some new, okay?

43:04Now, the deletes only happen on the client side, right? Clear the console.

43:09When I click Save, a post request is sent to our server saying that I want to delete those features.

43:20And a response comes back saying totalDeleted 6.

43:26To confirm that, if you launch another application, you won't see any of those polygons that previously existed.

43:43That means the change has already been applied to your geodatabase, right?

43:48So keep going here. Let me create some new areas, zones of interest that I want.

44:01Somewhere here and maybe somewhere here. Okay? And click Save.

44:19Let's check the response. So it says, totalInserted 2, which are the two polygons that I just inserted, right?

44:27Now, if I turn on another WMS layer, this is the earthquake zone.

44:34I found out the zone of interest that I just created has some overlap with that, right?

44:40I don't want that, so I can go ahead and modify the features I just created.

44:53Just a little bit here and something like that. You get the idea, right? So save it.

45:08It's totalUpdated 2.

45:11So that's a demo for the WFS-T using OpenLayers application.

45:16Let's stop for a while here and move on to another demo. You want to add something, Satish?

45:24There are two versions, if you will, of the WFS. One is the read-only and the transactional aspect.

45:31So we support both the read-only as well as the transactions, and what you've seen now is being able to edit...

45:38...you know, geodatabase features via an open standard OGC protocol, using a non-Esri client, right?

45:47So you got OpenLayers being able to edit features.

45:51Yes, we'll take the question. [Inaudible audience question] I'm sorry? [Inaudible audience question]

45:58The WFS-T feature editing via an Esri client? We do not have it at this point in time, but we can discuss that in greater detail.

46:06[Inaudible audience question]

46:11I'm sorry, I have a hard time hearing. Maybe we should... Can we take that question right after the presentation? Thank you.

46:17So let's move on to the third part of the demos, which is more challenging, something that we never did before.

46:25It's geoprocessing in Quantum GIS. This is what happens under the hood.

46:33I have ArcGIS Server. I publish my GP models and tools as a GP service...

46:38...and I enable the Web Processing Service, OGC, you know, for the geoprocessing.

46:45And non-Esri clients like uDig, Quantum GIS, OpenLayers can send capabilities and describe process to the server.

46:55It will get the detail, a list. It will get the list of processes first and also get the details of each process.

47:03The process means the tools or the models that you created, right?

47:06You can have a list of input parameters and the type of input parameters and also the output parameters.

47:14Now, after getting those details, it can send and execute that process to the server to do the analysis actually over the web.

47:22Once those processes are done, ArcGIS Server will send back the result.

47:28The results can be vectors, they can be rasters.

47:31One of the cool features that we have in our WPS is that all the vectors and raster results are automatically published as a WMS.

47:41So you don't have to get all the results at one time; you can actually just look at a certain area of that result.

47:50So I'm going to have to go and demo that.

47:54First, I'm going to launch QGIS. QGIS stands for Quantum GIS. Let's pick up where we left off from last demo, okay?

48:06So I'm going to add a WMS layer first, making sure I'm using the correct projection.

48:28This is my background layer, and I'm also going to add a WFS service.

48:40So just one extra thing here. Quantum GIS has the ability to add multiple plug-ins to it...

48:46...and you need to add the WPS plug-in to Quantum GIS to be able to do any web processing service-related items.

48:57Alright. Okay, I need one more layer to quite explain.

49:15And you also notice that Yingqi has been using multiple clients to show various parts of it. That's primarily for one reason.

49:23I mean, you've got these wonderful, quote, unquote, open source clients, but they don't do everything that needs to be done.

49:30So Quantum GIS does not do some WFS transactional aspects.

49:33OpenLayers does not do a whole bunch of other things.

49:36So, I mean, that's why we use these multiple clients to make our point here as to how all this can be done...

49:43...but that's the state of art today in terms of, you know, where support is...

49:47...when it comes to many of these open source products and their support for OGC.

50:07Okay, so this is basically what we have at the last demo, right?

50:10We have our basemap, we have picked out our candidate, house candidate, right? And we also created a zone of interest.

50:24Now, the first thing that I'm going to do is I'm going to do analysis.

50:29I'm going to draw a polygon with the center of the house that I wanted...

50:37...and the area will cover the three minutes' drive around that house.

50:44So I'm using the QGIS WPS client, connecting to the GP service that I published in ArcGIS Server through the WPS.

50:56So the traditional ArcGIS Server drive-time analysis polygon's being drawn here for the three minutes.

51:02Hold on. Let me just select the houses that I wanted. Select this one, select that one.

51:17Now connect again. Use this process called CreateDriveTimePolygons. This is a WPS process.

51:28Input will be the Redfin candidates.

51:33I want to create a polygon for the three minutes' drive, and I want the output to be in my projection, the Mercator...

51:42...and I want the results to be added back to the QGIS as a vector layer.

51:49So I can click and run that. When it's running, it's actually sending the request to the server.

51:54The server will go ahead and calculate that drive-time polygon.

52:00So there is no additional code that you've written for this one, right? It's just...

52:04There's no code I've written for that.

52:07It's just the regular WPS plug-in for QGIS, and all the feature data is being handled using GML.

52:13GML for WFS. You can see I'm adding all the WFS layers and WMS layers.

52:19So when the result comes back, I choose the projection I wanted - oh, I must select all the features.

52:31Let's delete it and do it again. I missed that option here, sorry.

52:38Still choose that Redfin, three minutes.

52:42I should click that, so only the selected feature will be the input instead of all the houses.

52:48Still type in a projection and let's run it again.

52:58Okay, this time it's quicker. Comes back; it's all these are the polygons for the three minutes' drive.

53:05Let's change the style a little bit to make it hollow, change the color of the boundaries, [unintelligible], quick.

53:20Now, the next thing I'm going to do is, 'cause I have the zone of interest that I've just created and these drive-time polygons...

53:28...I want to do another analysis, which is to calculate an intersection between those two polygons...

53:33...to create the final zone of interest that later I want to extract more data from.

53:40I can connect - let's turn off this - to the same WPS and this time pick Calculate zone of interest.

53:52The input will be - oh, should turn around and go back. Connect.

54:09One of the inputs is the drive-time polygon I just created; the other input is the zone of interest I created in the previous demo.

54:19And also put the projection as my...just click Run.

54:34I did a little bit of trick at the service output.

54:36That will not exactly match these two polygons, but you get the idea that trying to calculate the intersection of those two polygons.

54:47So these are the final polygons that I want. Change the color of it, change that, solid, wider.

55:09Now, the final steps I want to do is, as Satish showed you in the very beginning, the business case...

55:16...we have a lot of data, like bars and schools and bike routes, in San Francisco.

55:22Now I want to extract all those data within those zones of interest and zip them...

55:27...and, finally, ship them to one of the e-mail addresses.

55:35Using this process called ClipZipAndShip; choosing that output, polygon layers would be the one we just created...

55:47...that's the area I'm interested in. I want the vector data to be shapefile. I want the raster data to be TIFF.

55:55And I'm sending to myself, ytang@esri.com. And click Run.

55:59Right, exactly. Exactly, yes.

56:10So that's like a traditional clip, zip, and ship kind of operation where it takes all the data that's within the polygons...

56:16...puts it all together, and sends it out as a ZIP file to an e-mail address.

56:21And obviously, you can specify what format you want with it...

56:24...standard, GML- based, or a file geodatabase or a shapefile, and so on.

56:29So once finished, I'll get an e-mail with a ZIP attachment having all those data.

56:37[Inaudible audience question]

56:42Yeah, we can talk later on, and we can...

56:45Okay. Shall we switch or... Maybe a few more seconds?

56:53[Inaudible audience question]

57:07Looks like it's running into some errors, but let's try it again.

57:12So we'll wait for you to complete the process, and I'm going...

57:14Yeah, you can switch. Let me try it again.

57:16And then we'll have enough time for questions and so on. Right.

57:18I just have two more slides, and what we have up here is, you know, when we talk about OGC support, you know...

57:25...within ArcGIS 10, 10.1, whatever, unless you understand the OGC space really well...

57:32I mean, most people understand them as, okay, we got WMS and WFS.

57:36But there are a whole bunch of what you call additional specifications...

57:39...that work in conjunction, if you will, with WMS and so on.

57:44Some of that Yingqi mentioned. He talked about the SLD, which is a Styled Layer Descriptor...

57:49...allowing users to submit styles and filters along with their requests.

57:53Then there's filter encoding. There's time support. There was WFS as read-only and there's transactions.

57:59With WCS, it's a question of whether you can support these multiple different formats.

58:03With WMTS, do you or do you not support REST-ful interfaces, as well as the KVP encoding?

58:09With WPS, it's a question of do you support synchronous/asynchronous and so on?

58:13And with the WPS services, data could be provided either as GML, or it could be actually provided as a WFS service...

58:20...that's bringing in GML, and if there's any rasters that are being written out, they can just be written out as files...

58:25...or maybe they can be automatically published as a WMS, a kind of service chaining, if you will.

58:30And then we also have support for the Catalog Services Web.

58:34We have traditionally had, you know, a great amount of difficulty trying to do this particular type of presentation every year...

58:42...because there is so much to tell.

58:45We're kind of doing the whole GIS spectrum within this one hour, 15 minutes or whatever we have for this.

58:52And at this point, he has not been able to show WCS, we haven't had time to show OGC KML.

59:00We have not even been able to show, you know...

59:02...the fact that we have got multiple server products that support some of this stuff.

59:07We've got the Geoportal extension, we've got the ArcGIS Server.

59:10When it comes to clients, you know, there could be a few holes here and there...

59:14...but the point is we do support many of these with our desktop applications.

59:19We support many of these using our various APIs, and by "APIs," we're talking of Silverlight supporting WMS, WMTS...

59:26...JavaScript supporting it, ArcGIS.com viewer supporting WMS and KML.

59:31Then you've got the Flex environment supporting WMS and WMTS and so on.

59:37So, to me at least, I think our strategy for OGC support is very comprehensive. And it does take time.

59:46Sometimes we have to catch up on a few things, because remember, OGC specifications have multiple versions.

59:52If you have WMS, there is WMS 1.0, 1.1, 1.1.1, and so on.

59:56And then there are the SLDs, which have their own versions, on top of which we have multiple servers...

1:00:02...and multiple products, clients, and each of our solutions have their own versions, right?

1:00:08So we're dealing with a pretty complex space out here, in spite of which, I think with great confidence...

1:00:15...I think we should be in a position to tell you that, hey, in 10.1, especially with WPS, with the web processing service...

1:00:22...which is kind of bleeding edge as far as I'm concerned because there's not too much support for that in the outside world.

1:00:29I think we've done a fairly bang-up job with our 10.1 support for standards.

1:00:33So I'd like to kind of finish it off with this and then take some questions.

1:00:39If you're aware of the OGC space, I mean, going back to my first slide, we talked of proprietary standards...

1:00:46...and open specs when a vendor releases its specification, and then there's standards-related stuff.

1:00:53Our shapefiles are an open specification.

1:00:55We recently released the File Geodatabase API as an open specification, right, which meant anybody would be...

1:01:03...in a position to read or write a file geodatabase using some openly provided libraries that we have.

1:01:10Currently, the Esri REST API, for the last year and a half or so, has been an open specification.

1:01:20September of 2010 was when we released our REST APIs, an open spec...

1:01:23...which meant anybody could go ahead and implement the same function signatures, if you will...

1:01:29...on top of their server platform or have a client that can consume the REST API, right?

1:01:35So if you, for example, had your own home-built server and you could somehow mimic the same thing...

1:01:41...that our map service did or our feature service did, you could go ahead and do that with no IP restrictions...

1:01:46...and you could participate in this wonderful large ecosystem that we've built around it with all our client APIs and so on.

1:01:52But what has happened in the last eight months or so and continuing to happen is we're taking this open specification...

1:01:58...and we're working it through the OGC process.

1:02:02So all the members of the OGC are sitting together, announcing, okay, Esri has brought this REST specification...

1:02:08...because OGC, with their KVP encoding and SOAP focus, has not really spent time on the REST stuff.

1:02:15So with all the JSON encoding and the REST way of thinking, people see a lot of value in our particular offering.

1:02:22So right now, it's working its way through the OGC process, and fingers crossed, you know.

1:02:27There'll be questions that are being raised, and we're trying to, you know...

1:02:29...fix whatever things that other members have an objection to.

1:02:34So over time I think this will all smoothen out, and if everything goes well...

1:02:39...there's a good possibility that the geoservices REST...

1:02:42...or the Esri REST [unintelligible] release of the geoservices REST specification, would also become an OGC standard...

1:02:48...helping, you know, GeoServers and the Minnesota MapServers and whatever other open source stuff out there...

1:02:54...to participate more closely with some of what Esri is doing.

1:02:58So with that, I'd like to say thank you, and I'll take questions. And since Yingqi's process did complete okay?

1:03:05Well, what happened is actually - the process actually executed, but it turns out the connection...

1:03:09...network connection here fails to actually connect to the Esri mail server.

1:03:14So there's some network restriction, so I'm getting this...

1:03:19That's okay.

1:03:20Yeah, so... I need to try it outside; somehow didn't work.

1:03:23Okay, so we have just a few minutes, I think, but we're willing to take questions.

1:03:27And for those of you who have more specific questions and want to grill us more, I mean, I'm definitely available after this presentation.

1:03:34So with that, thank you very much. Yes, sir.

1:03:36[Inaudible audience question]

1:03:41Okay. That was something that we have not touched upon today, on the sensor web stuff.

1:03:46So what we have done is we've got some business partners and...

1:03:51...we're part of this relationship with an organization called 52North out in Germany, which is an open source organization.

1:03:58We've worked very closely with those guys, and for 10.1 platform - it's not final for release yet, this particular component...

1:04:06...what we have, what we will release is an open source server object extension for ArcGIS Server...

1:04:14...that conforms to the SOS, or the Sensor Observation Service specification.

1:04:19So we do have a solution which is ready, it's just that it's not been put out there, which we should be doing in a couple of weeks.

1:04:28So that's a totally open source extension to ArcGIS Server, meaning you can take it, modify it, do more stuff with it.

1:04:35Okay, more questions? Yes, sir.

1:04:37[Inaudible audience question]

1:04:45To do...to do, yes, yes. [Inaudible question, continued] Right. The geodata...

1:04:50[Inaudible question, continued]

1:04:5310.0 SDE, for sure, 'cause I use 10.0 ArcSDE so...

1:04:58So he'd be able to use 10.0 SDE with the 10.1 server.

1:05:00No problem. Yeah.

1:05:01I mean, the rules will be the same for what you use with a map server and other stuff.

1:05:07Yeah, 10.0 should be fine. 9.3, I wasn't sure if there was any migration needs to do to migrate your 9.3 SDE to 10.0...

1:05:17...before you can have your 10.1 server to connect with it.

1:05:20But the more important point is...

1:05:21But 10.0 is okay.

1:05:23More important point is the OGC part does not introduce anything different from what you would need for the rest of the stuff.

1:05:29Right? Yeah. More questions, please? Yes.

1:05:35[Inaudible audience question]

1:05:43With replication? There has been an attempt.

1:05:46It is, I think, available as some kind of, what they call, best practices or discussion paper...

1:05:52...and you would go in and search for geosynchronization. That is the name of that spec that they're working on.

1:05:59It's not a spec; it's just some ideas how GML-based geosynchronization could happen between multiple databases.

1:06:06[Inaudible audience question]

1:06:10I haven't followed that particular part very closely, but it's GML based, works on top of WFS...

1:06:17...and it brings its own host of problems, I guess.

1:06:21Yes. And we can continue that conversation on the WFS-T.

1:06:25What is important is traditionally, we've been very server-centric, or server focused...

1:06:32...so most of our effort on OGC development has been on the server.

1:06:35We do have client support, but like I said, we have multiple clients, so there may be small areas where we are falling short...

1:06:41...as in we don't have a WFS transaction client for the desktop.

1:06:47But traditionally, we've rarely run into good servers out there that are actually publishing WFS-T services.

1:06:55So our focus has primarily been whatever resources Esri folks have, you know, they should be in a position to expose them...

1:07:03...using standardized interfaces so others can consume it.

1:07:07But I must say that our story both in server and client with 10.1 is quite complete.

1:07:14Just another note that all the materials, the slides and the data and the GP tools, that I used to create those demos...

1:07:22...I will package them in a package, and I'm going to upload it to ArcGIS.com pretty much after the conference.

1:07:30So just after the conference, just go to ArcGIS.com and type search DevSummit 2012 and OGC...

1:07:36...should be able to find that download link.

1:07:39So you should be able to get it all...

1:07:40If you're interested in reproducing something at home with 10.1 Server, so welcome to do so.

1:07:47Right. Question right there. Yeah. Is there more questions?

1:07:56Well, if there's no more questions, thank you very much for attending this.