00:01We're going to have a little bit of, you know, a mix of both.

00:03We're going to try to keep the presentations as light on the code...

00:08...but still show some code and show the capabilities of the software.

00:13My name is Tope Bello, I work at Esri Redlands here.

00:17And with me today is Jay Cary and Javier Gutierrez.

00:23Jay is going to be presenting the ArcGIS Data Reviewer piece...Javier is going to be showing us...

00:30...a demo of how you actually integrate ArcGIS Workflow Manager and Data Reviewer Server.

00:40So here is a quick road map of what we're going to be doing today.

00:44We want to give you a quick overview, for those of you that have never really seen these products.

00:48We want to give you a quick overview of what Workflow Manager is...

00:53...talk a little bit about the system architecture...

00:55...for both Workflow Manager and Data Reviewer, as well.

00:58We'll focus a little bit more on the server side of the technology...

01:03...talk a little bit about the capabilities available to you on the desktop side...

01:07...and then we'll have a demonstration of how you can enable workflows...

01:11...that integrate, basically, data quality focus.

01:16At the end of this presentation, we're going to leave you with some resources...

01:20...that you can take away for more information when you go home or back to your offices...

01:25...and we'll open it up for questions at the end.

01:32So let's dive in here real quick.

01:33So for Workflow Manager, one of the few things we want to talk about...

01:37...here is the business problems, alright?

01:39So a lot of you may wonder, why exactly was this software created?

01:44Why would I need this product to manage my work, for example?

01:49We're going to talk a little bit about that.

01:50We're going to talk about some of the key features within Workflow Manager.

01:55These features actually are both desktop- and server-centric.

01:59I guess they go across both platforms.

02:02We're also going to talk about the system architecture...

02:04...how it's built and what you need to actually use it.

02:08Lastly, I'll talk about your options for extending ArcGIS Workflow Manager...

02:12...and I'll pass the buck up to Jay.

02:17So why exactly is managing work important?

02:21Here are a couple of reasons why we feel that you need to manage your work.

02:26These are basically feedback that we've received from customers...

02:29...or people that basically have these problems on a day-to-day.

02:33Optimizing data production and time, optimizing or improving data quality...

02:38...reducing costs of data production, automating processes, streamlining processes...

02:48...and also allocating resources appropriately.

02:51These are problems that exist within most organizations that have a lot of work or a large team...

02:58...and Workflow Manager is Esri's solution to some of these problems, and I'll tell you how.

03:05So what, exactly is Workflow Manager?

03:07Well, it is Esri's workflow management application.

03:11It allows you to streamline your business processes...

03:14...allows you to basically draft out your workflows, standardize these processes...

03:19...into repeatable steps or repeatable actions.

03:23It allows you to basically serve out these workflows as copies.

03:27Basically, you're making copies of these workflows into what we call jobs...

03:31...so that the people actually doing your work can grab this and perform their work more effectively.

03:38Lastly, it allows you to improve some of your processes.

03:41So you may say, Oh, what if I define a workflow today that has a series of steps...

03:48...that maybe it is not the most efficient way of doing things?

03:51Well, you can go back in and modify those.

03:55Workflow Manager highlights key areas within your workflows...

03:59...where things need to be improved.

04:01And again, I'll talk a little bit about that later on.

04:06So let's take a quick look at some of the key features...

04:09...and capabilities within Workflow Manager.

04:12I mentioned real quick that you have the workflow, and you create your workflow...

04:17...and you create jobs and you sign into individuals within your organization.

04:21It's just a quick highlight of the things that actually make up a job.

04:25The key takeway from this slide is the main thing in the middle which is the workflow itself.

04:30That's the heart and soul of the job.

04:32That is the operational aspect of your job.

04:34And we'll talk about the details later.

04:36In addition to the workflow, we've got what we refer to as the resource.

04:41The resource is the person that is actually performing the work.

04:44The resource is the person that grabs the workflow and executes the steps...

04:47...that has been defined by the business specialist who understands the business process.

04:53We also provide tools for managing geospatial data.

04:56So version management could be a little bit daunting for some users.

05:01Workflow Manager actually provides you tools that allow you to basically integrate that...

05:06...into some of the workflows and automate some of these processes...

05:08...version creation, deletion, assigning versions to certain workflows, and so on and so forth.

05:14Again, we have the concept of what we call the geographic area of interest.

05:19This is one of the key points within Workflow Manager, allowing you to narrow down the area...

05:23...in which your work is going to take place.

05:25Rather than having someone go into a map that contains the entire US, you want to drill it down...

05:30...to perhaps the convention center in Palm Springs where the work needs to happen.

05:35Lastly, when all these activities are happening, we store it in a history table.

05:39Every job has activities logged and you can go back and basically look at it as an audit trail...

05:45...see what happened on a job, who did what, and when it happened.

05:52So a good example of a workflow here, like I said it's the main component of a job.

05:58In this particular workflow, the workflow starts other processes.

06:04So you can spin off other processes that maybe are...you know, a different department...

06:09...a different group of people within your organization needs to take care of.

06:13You could have steps that actually run at the same time, simultaneously.

06:18So that way, user A is not waiting for user B to finish his work...

06:23...before they can actually grab the work and start performing their duties.

06:27Or you can have workflows that actually loop back and forth.

06:30These looping workflows, again, kind of get modeled into what we call decision points...

06:34...so you can have some sort of intelligent behavior modeled into your workflow so that...

06:39...until a certain criteria is met, the workflow would not proceed to another stage.

06:48You can send e-mails during these workflow executions.

06:50Again, this is key for those of you that typically, you know, you would have to hand off a file...

06:55...or go knock on someone's office to tell them they have work waiting for them to be done.

07:00When things happen within Workflow Manager...

07:02...e-mails get fired off automatically.

07:05Again, all of this is fully configurable, and you can model this to suit your business need.

07:10Here is a good example of how complex workflows could be in a real-world environment.

07:16What you see here is a graphic of an overall workflow.

07:19And the blowout there is just a subset of that workflow.

07:24You can start to see how complex your workflows can look...

07:27...depending on how complex your business processes are.

07:33Like I said, within Workflow Manager, you can send e-mails...

07:37...or notifications, as we call them, when certain things happen within your jobs.

07:42For example, you may want to send an e-mail when a job is assigned to a certain resource.

07:47You want them to know that there's work waiting in the pool for them to grab and do work on, right?

07:53You may also want to send an e-mail when a certain thing happens.

07:56Like, say for example I execute a step that requires me to load data.

08:00You want maybe the DBA to know that, you know, something's going on with the database.

08:05Or you want people that actually are in the field...Sorry, people that are actually back in the office...

08:10...to know that the field crew is out there doing work.

08:12You really don't want to, for example, turn on the electricity switch...

08:16...when people are working on an electric pole, for example.

08:22You may also want to send e-mails when edits happen to your geospatial data.

08:28This is what we refer to as spatial notifications within Workflow Manager.

08:32You can build your rules so that when a certain feature is modified...

08:36...or a certain feature class is modified, or a certain feature...

08:40...within a specific geographic area of interest is modified...

08:44...an e-mail is sent to interested parties.

08:48Like I say, all of this is fully configurable.

08:52So I've talked about some of the key components.

08:55In the beginning, I also mentioned process improvements.

08:57How do you know when to improve some of your processes?

09:00How do you know when you have problems with your processes?

09:03Well, reports allow you to actually get that information and act upon them, right?

09:10Within Workflow Manager you can use some of the reporting capabilities...

09:13...to figure out when things are late, when work is completed...

09:19...when work is delayed because of certain things.

09:22These types of questions, or answers I guess...

09:27...allow you to make better decisions with your workflow.

09:29Maybe remodel your workflow, redesign it, move a step out...

09:33...add a couple more steps, add more notifications, and so on and so forth.

09:40The reporting capability is also available to you in a graphical way...

09:44...so you can display this as a dashboard to your stakeholders.

09:47I talked about the audit trail.

09:49The audit trail allows you to basically see what's going on, on a particular job.

09:54So every activity that a user does gets stored in a table.

09:58And this information is displayed and you can go back and see who did what...

10:02...what they did, and when it happened.

10:04Again, ensuring quality within your data production or within the work you do overall.

10:12Once these bottlenecks, I guess, are found...

10:14...you can start to reautomate some of these other business processes.

10:19So, for example, you may decide to add version deletion; a step to delete versions...

10:26...because you notice that after every job is done, the versions stay behind, right?

10:31So you end up with a database that has 100 versions that are not being used anymore.

10:36It could become 1,000 versions all the time.

10:38You can model your workflow to allow you to perform these activities more effectively.

10:44You can improve your QA process.

10:46Maybe you need to add a step in there that routes the work to the QA team...

10:49...because you've noticed inefficiencies in the data.

10:53The demo today will actually show you how you can use both Data Reviewer...

10:57...and Workflow Manager to basically make these kind of decisions.

11:02So we've talked about some of the key features within Workflow Manager.

11:06So what, exactly, is this product?

11:08What, exactly, is it made of?

11:10To just kind of give you a quick overview of the way the product is...

11:14...there is a desktop piece to Workflow Manager and there is a server piece to Workflow Manager.

11:19But to both of them, there's a common denominator I guess.

11:23And that is the Workflow Manager geodatabase.

11:26This is where all those tables and all that information...

11:29...that I said we pull, display, store, and so on, get stored in, okay?

11:35ArcGIS Workflow Manager Desktop is built on top of ArcObjects...

11:38...and it does require you to have an SDE database...

11:41...that contains your Workflow Manager system tables.

11:44You do require a standard edition or higher, I guess.

11:47At 10.1 that's kind of the new license level for those of you that are not familiar.

11:52Moving on.

11:54We have an application that allows you to interact with the work...

11:57...so that's where the end user is going to go and actually execute their workflows...

12:02...potentially if you're using a desktop, obviously.

12:04And the administrator is used to design this workflow.

12:07So you've got like a drag-and-drop canvas that looks like the ModelBuilder...

12:10...for you to define these workflows.

12:13We also have an administrator application, like I said...

12:16...that kind of looks like any other administration tool like...

12:21...You know, for those of you that are familiar with database administration...

12:24...it kind of looks like SQL Server, SQL Server Management Studio.

12:28For those of you that are familiar with Windows...

12:30...it overall looks like your component management console in Windows overall.

12:38So for the server piece of things, again...

12:41...interaction is done again through the Workflow Manager database.

12:46That contains all the job information.

12:49However, to interact with those tables, the server extension is required...

12:54...so that you can go through our SOAP or REST API.

12:58Through that, you can start using web applications that consume these services...

13:04...to display or send back information back and forth to the Workflow Manager service...

13:09...to the Workflow Manager database.

13:12The key thing there, I guess, is this is a Windows-only product.

13:17We do not support Linux or non-Windows platforms overall as of yet.

13:23So, why would I use Desktop versus Server?

13:26Well, this question comes out a lot.

13:29It's not really a why would I use one over the other...

13:32...it's more of a when do I need to get Server to complement my Desktop capabilities, or vice versa?

13:39The Desktop piece is used...should be used mostly for your GIS...

13:43...every GIS editors...like people that need to do things on the desktop.

13:48So for example, if part of my workflow is to open up ArcMap to drive me into an area...

13:54...to make specific edits using the ArcGIS Desktop tools to make my edits...

13:59...then I need the Workflow Manager Desktop.

14:02If all I do is enter in a request through something that sends it back to the GIS department...

14:09...maybe all I really need is a form that accesses the Workflow Manager server endpoint...

14:14...to create this job for the GIS department to go work on.

14:17So the use case here is basically for your heavy GIS users...

14:22...you do require ArcGIS Workflow Manager Desktop.

14:25But if you want to take advantage this across your organization, you want to take...

14:29...you want to open this up to maybe your customer service representative.

14:34They get a call in, they want to log something into the system...

14:37...and then pass it on to the GIS Department.

14:39Maybe all you want to do is expose a simple web app that allows them to do this.

14:45Lastly, you can also interact with other systems.

14:47So you may already have systems for your accounting department in place that...

14:52...you know, maybe you track your work orders, or track your budgets...

14:56...and you want to link projects, GIS projects back to that.

15:00Through the REST API, you can do that.

15:02Through the REST API, you can communicate, you can send information to other systems...

15:07...that obviously support that kind of protocol, right?

15:11Or, get information back from those kinds of systems as well and display it in Workflow Manager.

15:19So, you may be wondering, so what exactly is Workflow Manager Server?

15:23Workflow Manager Server is basically an ArcGIS service.

15:30A lot of you are probably familiar with what a map service is, right?

15:34A lot of you are familiar with what a geoprocessing service is.

15:37A Workflow Manager service is exactly the same concept...

15:40...as a map service or a geoprocessing service.

15:44What it really is, is, it takes your Workflow Manager database, serves out the connection...

15:49...over to the server, and using our API you can start to interact with certain objects...

15:54...directly within the Workflow Manager system.

15:57At 10.1, for those of you that...you know, you've been here all week...

16:01...you've probably seen the ArcGIS Server Manager...

16:04...Workflow Manager Server at 10.1 is fully integrated with the Server Manager.

16:08You can start to manage this more effectively, and this is kind of what it looks like.

16:13I have a screen shot over there.

16:15Publishing the service happens from within the Workflow Manager administrator.

16:20Yes, if you buy the Server product, we do require you to buy the Desktop as well...

16:25...because you can only author your workflows through the desktop.

16:29So you need the Desktop to author your workflows and serve it out to the server, similar to how you need ArcMap...

16:35...to author your maps and serve it out to the server.

16:41Once you serve out your Workflow Manager service, there's two main things...

16:44...that are available to you right off the bat with ArcGIS Server.

16:49One of them is the SOAP API, the other one is the REST API.

16:54You can start to interact with the system through this interface.

17:00So you can build your heavy...maybe ASP.NET applications hitting SOAP directly...

17:07...or you can build your lightweight theme clients, rich clients rather, using the REST endpoints.

17:12And I'll talk about some other capabilities with some of our APIs.

17:18So the REST and the SOAP are available to you.

17:21However, at 10.1 we are also releasing some of our web APIs.

17:25So we're going to have a Flex API and a Silverlight API and a JavaScript API...

17:29...that you can build applications against.

17:31So you can start to communicate with the Workflow Manager.

17:34Which...some of you are more familiar with Silverlight...

17:37...you want to write your application in Silverlight.

17:39Yes, you can do that at 10.1.

17:41You want to write it in Flex, you can do the same thing at 10.1.

17:44What Javier is going to be showing us today is just a demo on Flex.

17:47And again, you can relate or take this away...

17:50...as, you can write the same thing in your preferred language.

17:54JavaScript, Flex, or Silverlight.

18:00So how can I extend Workflow Manager?

18:03So how can I take what comes out of the box and you know...

18:08...maybe tweak it a little bit farther?

18:11Within Workflow Manager, there's a limit to the things you can do...

18:15...on both platforms, and then there are some capabilities that actually go across both platforms.

18:19And by platforms, I mean the desktop extension and the server extension.

18:24From the Desktop standpoint, just to go over that real quick...

18:27...you could create your own custom commands, toolbars, views, tabs, and views...

18:31...within stand-alone applications using the desktop SDK.

18:35Again, this is built on top of ArcObjects.

18:39For the server piece, you can build your stand-alone web applications.

18:43The example you're going to see today is a simple web app that looks at...

18:46...that takes advantage of our Flex API.

18:50But across these two platforms, you can actually do a little bit more.

18:54You can start to write workflow steps that are cross-platform...

18:59...things that you can execute both on the server or on the desktop.

19:03You can also start to interact with other systems.

19:05I talked about interacting with the work order system; for example, through the REST API.

19:10You can send information over there or you can go fetch information and ingest that...

19:14...and display to your users in a centralized dashboard.

19:18Just something to keep in mind.

19:20For those of you that are going to be writing code, a custom step development...

19:24...which, again, I talked about the Workflow Manager extendability for the server.

19:33Through the workflows, you're going to need to write some custom steps...

19:36...depending on what your business process is, right?

19:39You can write those in any language that you prefer.

19:43We support Java, we support C++ and any .NET language for writing custom steps.

19:51On the web side of things, I've talked about the API.

19:53So we have a Flex, JavaScript, and a Silverlight API.

19:58That's all I have.

19:59I'm going to pass it on to Jay.

20:28Can everybody hear me okay? Okay.

20:35Hi, my name is Jay Cary; I'm the product manager for ArcGIS Data Reviewer.

20:40And so what I'm going to do first is kind of go over sort of to set everyone's attention...

20:46...to the importance of data quality.

20:48It's not something folks like to really dwell on, especially folks who are busy...

20:52...doing other development tasks and analysis work.

20:54But you know, we want to talk a little bit about why it's important...

20:56...and how it might actually impact the work that you do.

21:00While I want to talk, like Tope did, a little bit about what ArcGIS Data Reviewer is...

21:04...that folks may not be familiar with that extension...

21:06...we'll provide a little bit of background about what it does.

21:09There we go.

21:11We'll talk more specifically about ArcGIS Data Reviewer for Server...

21:15...which is a new offering coming out at 10.1, and we'll talk specifically about...

21:19...some of its capabilities and what it would bring to the 10.1 platform.

21:24And finally, at the end, we'll talk about some of the integration that Data Reviewer has...

21:28...done both on Desktop as well as some ideas we have...

21:31...on how integration would work on the server.

21:35Then we'll follow up with Javier's demo on how that works out in kind of a loosely coupled manner.

21:42So data quality, it's kind of amazing when I go out and talk to folks...

21:46...about data quality, everyone sort of gets it.

21:48They understand bad data quality can produce all sorts of undesirable things...

21:53...like bad analysis results, cartographic products that don't actually work right...

21:57...or communicate information effectively.

22:00But it's surprising to me the number of people that actually don't do much about it.

22:04They don't really think about coming up...or investigate the options...

22:08...that are out there for doing things in an automated way.

22:13As system developers, you've probably been bitten by this.

22:15You've been building applications, assuming the data is good.

22:19Only to come to find when you actually try to deploy those applications...

22:22...it doesn't work out very well.

22:24And usually it gets into a big finger-pointing match...

22:25...in terms of well, the application works great, your data is at fault.

22:29So it has lots of different ramifications to both the organization, as well as developers...

22:33...who are trying to implement automated methods to use the GIS data.

22:40So ArcGIS Data Reviewer is an extension.

22:43At prior to 10.1, it was an extension to ArcGIS Desktop.

22:47What it does provide is a repeatable, automated methodology for doing data quality control.

22:54It includes rule-based workflows that are in the form of configurable data validation checks...

23:00...as well as tools to help you do interactive visual quality control...

23:04...that sometimes has to be done when you can't detect errors with the automated checks.

23:09We also provide the mechanism for tracking errors through what we call the error life cycle.

23:14And we'll talk a little bit about what we mean by an error life cycle in the next slide.

23:19We've found that the people who gain benefit from this most heavily...

23:24...actually are kind of a mixed bag.

23:26We've got large organizations who maybe have distributed data production environments...

23:30...where you have maybe outsourced data production to an outside contractor or team.

23:35They benefit by having automated processes in place...

23:38...because they can do repeatable data control and data validation of that data coming in.

23:43But also on the flip side, small teams actually can use that as well...

23:46...when you're short-staffed or you have interns or maybe summer hires or...

23:50...someone you're bringing in who isn't really great at using GIS yet.

23:55Using Data Reviewer enables you to kind of validate...

23:57...and make sure that they're producing good quality data.

24:03And as I mentioned, Data Reviewer has been around for awhile.

24:05It's been a Desktop extension since version 8.

24:08At 10.1, we'll be offering a server extension...

24:10...which brings many of the capabilities to the server environment.

24:16So I talked a little bit about the framework that we use for managing error.

24:20And a little bit of detail there is that Data Reviewer provides you...

24:24...a whole host of tools to both do automated review of your data using a number of configurable...

24:30...data validation checks, or you can leverage interactive tools for doing visual QC review of data.

24:37Once those errors are detected, we log the error to an archive...

24:41...to what we call the Reviewer workspace...

24:44...which is kind of similar to the Workflow Manager workspace...

24:47...in that we have a series of tables where we manage those error results.

24:53Whether it's you or someone inside your team or outside contractors...

24:59...using Desktop tools to correct these errors, using Data Reviewer, the error is logged...

25:06...so they can quickly get to the error itself, understand a little...

25:09...kind of what the problem or the error was...what was actually going on...

25:12...what detected this error, and how should I fix it?

25:18There are occasions when some errors are actually false positives.

25:22So in those cases, we can...the technicians can use Data Reviewer...

25:24...to note those as exceptions and then move on.

25:29The third step in the life cycle is really the verification phase...

25:32...which for a lot of organizations is sort of an optional step.

25:35At this step, at this part of the workflow...

25:37...you may actually have folks who wear a QC hat, who double-check...

25:41...that they actually...that the error was actually fixed, that it was fixed properly...

25:45...and also, more importantly, maybe to make sure that no new errors were introduced...

25:48...as a by-product of that correction.

25:51So oftentimes in that phase, you'll rerun your data validation...

25:55...just to make sure that nothing new has come into the system.

26:00So when we approached the 10.1 development for Data Reviewer Server for extension...

26:04...we had some high-level goals in mind.

26:07The first was we wanted to provide customers with more flexible ways...

26:11...of running these automated validations, especially for organizations...

26:15...that have really large business rule sets that have to validate lots of data.

26:20We needed to figure out, how do we enable those customers to validate data in different ways?

26:25We also wanted to focus on, how do we involve more people in the data quality control process?

26:31How do we bring in other subject matter experts, or non-GIS users into this conversation...

26:37...and use their input in a productive way?

26:41The last thing we focused on was, okay, since we've detected all this...

26:44...we have all this information about the errors and anomalies in our database...

26:47...how do we leverage that information, communicate either internally or externally...

26:52...to stakeholders who maybe need to be made aware of the impacts...

26:56...of data quality on the organization's mission?

26:59And how do we engage with that audience in a more effective way?

27:05So the workflows at 10 are obviously very heavily based.

27:09For automated validation, you have users who are using ArcGIS Desktop extension...

27:14...Data Reviewer extension on Desktop; they're authoring their business rules using the extension...

27:20...they're modifying them, and they're executing them from Desktop...

27:23...validating data in their data workspaces, logging the errors into the Reviewer workspace.

27:29They may be leveraging manual QC tools to look for missing features...

27:34...or features that are positionally out of place when they're doing their visual QC review.

27:41On the reporting side, at version 10, Data Reviewer introduced a whole host...

27:45...of automated reports that very quickly can give you an understanding...

27:49...of where the error is happening, how many features are in error?

27:53What's my accuracy calculations based on the run of my data?

27:57How accurate is my data based on a percentage?

28:02At 10.1, really we are expanding that in some significant ways.

28:07How can we use Server to actually run those automated validations?

28:11A lot of criticism we get from customers is...

28:13...I can't afford to tie up my desktop for hours on end validating millions of features.

28:18How can we leverage those servers that are typically run 24-7, 365?

28:22How can we move that into that infrastructure and take advantage of it?

28:27On the visual QC side is, there's a lot of eyeballs looking at our data...

28:31...as you publish it out as a service, right?

28:33But you only have a few GI technicians.

28:36How can we leverage those data consumers...

28:39...but in a managed way to get their feedback so that we can make our data better.

28:43And finally, on the data quality reporting side, how do we engage that maybe senior leadership...

28:48...who aren't technical folks, but who should be aware of the impacts of the data quality...

28:52...before they make decision-making or before they making funding decisions.

28:59So some of the capabilities of Data Reviewer for Server, one of the big things...

29:02...that sells Data Reviewer are the over 40 different automated checks that can validate your data.

29:08These include things like just basic feature integrity validation...

29:12...looking for null or empty geometries, they could be attribution validation...

29:16...to include things like execution of SQL queries against your data...

29:20...to compare multiple attribute value combinations, things like regular expressions...

29:26...to validate attribute match of pattern you're looking for, to also include...

29:31...things like validating spatial relationships between features.

29:35One of the big advantages of Data Reviewer is that it also supports multiple file types.

29:40So you can validate shapefiles against features in your geodatabase.

29:45You can validate features in two different geodatabases against one another.

29:50And also, you can also include things like metadata, which we've introduced at 10.

29:56The ability to automate the validation of feature metadata as well.

30:00They're configurable; and by that I mean you don't need to be a developer...

30:04...to configure the checks to validate your business rules.

30:07They can be configured once and run over and over and over, in what we term a batch job.

30:12A batch job is nothing more than a little XML file that's stored on disk...

30:15...or on a network share that is your configured business rules.

30:19And they can be distributed across the team...

30:21...so that everybody's doing quality control the same way.

30:25They're also additive and scriptable, in that if one of those 42 checks...

30:29...doesn't do what you need it to do, you can create your own custom code...

30:33...to run custom logic and validate your data with.

30:36The results of which, though, are still logged to the Reviewer workspace...

30:40...just like any of our off-the-shelf checks.

30:46So what does this mean on Server?

30:48So this means that we can take what used to happen on the desktop and actually run it on Server...

30:52...either as a scheduled event...that is to say, I need to have my data validated...

30:57...every night at midnight during the workweek so that when my technicians...

31:02...or my staff come in in the morning, they have a punch list of things they need to fix for that day.

31:07Or, it could be an ad hoc, on-demand validation, which is what Javier is going to show today...

31:11...which is there's step and a workflow that I need to validate some data.

31:16It's not going to be repeated, it's just one time.

31:18But I still want to log those results to an archive so that I have a record of those errors.

31:23The benefits, of course, are that we get the data editors and the Desktop users...

31:27...out of the business of having to run these validations on their own.

31:30They can go about doing things like data...more data extraction...

31:34...cartographic production, or analysis.

31:36It also puts those servers to work.

31:39And, because of the 10.1 architecture, we're able to take advantage of the clustering.

31:43So as your validation needs scale, you can scale your cluster as well to take advantage of that.

31:53In terms of the feedback from the web, you know, we've kind of talked to a lot of customers.

31:58And typically, data quality is always seen as a GIS technician's problem.

32:04And it's really not. It's actually an organizational problem.

32:07So what we're trying to do is provide a platform for which you can harness...

32:11...all of the data consumers, the people...the subject matter experts, the biologists...

32:14...the planners who aren't really GIS people, but boy, they know a lot about the data, don't they?

32:19How do I harness their input in a managed way so that we can improve data together?

32:25So some of the interesting workflows that might emerge from this is the traditional...

32:30...there's an SME in my organization that I've...will not use GIS...

32:33...I always print out maps or plots for him...Which is normally a desktop process and not very effective.

32:41With 10.1, you'll have the ability to engage with them in a different way...

32:44...in maybe a lightweight application where you can get their feedback.

32:48The benefit being that it gets logged into the Reviewer workspace...

32:51...just like any other data quality result comes in...

32:54...and then it enters it into that Data Reviewer workflow...

32:57...where it can be validated, fixed if necessary, and then verified and tracked.

33:02What's kind of more interesting, though, is this idea that...

33:04...I can also now engage with external customers.

33:07These might be stakeholders or other cooperators to my organization, and maybe data consumers...

33:14...or volunteers who also want to help out, but who need to be channeled in a way...

33:20...that I can manage the feedback that we get in a way that's a little more efficient.

33:25So this last bit is kind of an interesting way of harnessing a lot of eyeballs...

33:29...who potentially know more about my data than I do.

33:31Maybe they live in my community and they know that something's in the wrong spot...

33:35...or a new development happened, it's just down there, I can let people know.

33:40So that's kind of an interesting angle that we're going to kind of track as we go forward.

33:46The final bit is dashboard reporting.

33:48As I mentioned earlier, at 10 we released a whole bunch of really technical reports...

33:52...that give you a very granular breakdown...

33:54...of how our data is based on the business rules that were run against it.

33:58We wanted to expand that into more of a summary dashboard service, but the goal here...

34:03...was to engage an audience that typically may not be super technical...

34:07...but should be involved in this process and made aware of...

34:10...data quality impacts to the organization.

34:13In the demo today, Javier will show you ways of pushing out info graphics...

34:18...that are an engaging way to get non-GIS users invested...

34:23...in understanding data quality impacts.

34:28So the system architecture, the Data Reviewer Server is implemented...

34:32...as a server object extension at 10.1.

34:35We also use geoprocessing services to do a lot of the heavy lifting...

34:39...when it comes to validating our data.

34:44Because we're based on 10.1, you can take advantage of the clustering that's going on...

34:48...and if you need to scale the services to do more data validation, you certainly can.

34:55At initial release, we'll be putting out a Flex as well as a JavaScript API...

34:59...that'll help you kind of get up and running as a developer very quickly.

35:03We're also providing a series of what we call starter applications and widgets...

35:07...that will be provided in OpenSource that you can use as-is...

35:11...or you can take and customize or modify as you see fit.

35:16Now that also doesn't preclude you from building your own stand-alone Flex or JavaScript application...

35:22...using the API to work with the Data Reviewer services or to go through...

35:27...and actually create your own custom application to work directly against the REST API.

35:36Alright, so let's talk a little bit about what we've been doing...

35:39...to integrate the two products, Workflow Manager and Data Reviewer.

35:44On the Desktop side at release 10, we created a couple different custom step types.

35:50These custom step types included the ability to create a Reviewer session.

35:55And a Reviewer Session is just a method for us to manage our error results...

35:58...within the Reviewer workspace.

36:00It enables us to group error results.

36:03In the case of Workflow Manager, maybe by job.

36:06So I want to validate my data, but I want all error results for that job...

36:10...to be stored in a way that I can retrieve them very quickly.

36:14We also deployed a custom step type for executing batch jobs.

36:18So as a step in a workflow, a user could actually run data validation...

36:23...without ever having gone into ArcGIS Desktop.

36:27The configured rules are run and, as far as the user knows, something's happening...

36:32...but this is work being done behind the scenes for me.

36:36We leveraged, in this step type, the different configured checks that are stored...

36:40...in the Reviewer batch job, we leveraged the validation area...

36:43...as being the job area of interest that Tope talked about.

36:47Where is the work being done?

36:50Results, or the data workspace parameter from Workflow Manager is also an argument to it.

36:55So I don't have to tell it what data to check.

36:58It's already a property of my job, so the batch job runs against it for me...

37:02...and where the results are written to are also a property...

37:06...that could be inherited through the system.

37:07So as an end user, it just does the validation for me.

37:12I don't have to think about where the data resides or where the results should be written to.

37:17Finally, we also added a token so that when the Launch ArcMap step happens...

37:21...which is an automated step which Workflow Manager does, which is to launch ArcMap...

37:25...for you from the job, drill you down to the area of interest, potentially load in...

37:29...a task-assisted managed workflow, and with this custom token will also initiate...

37:34...and start a Reviewer session and open up the results for you right away.

37:38So your technicians get right to the issue, like, What's the problem with my data?

37:47For the server side of things, we're really...it's really an emerging kind of...

37:52...requirements are still kind of emerging.

37:54We don't yet have this product on the street.

37:57So we're looking for impact...you know, feedback from customers...

38:00...as they deploy this to really form up what would be the true integration.

38:04This integration could be like we did on Desktop, which are custom, server-side, custom step types...

38:11... it could include using geoprocessing scripts and models as a step and a workflow...

38:16...or it could be actually using Workflow Manager's execute URL.

38:19We're still sort of in the early stages of figuring out what would be the best approach to this.

38:25But as I said, it is a new product, we had some other things to get done first to get it on the street.

38:32Alright, so we'll turn this over to Javier, and he's going to work through a demonstration of the...

38:38...well, I'm going to call it what we'll call loosely coupled integration at this point.

38:42So we've been doing a lot of scrum on the team...

38:44...so I thought we'd phrase this in a traditional user story.

38:48So the scenario to this is that as a contracting officer, right?

38:52This is the person doing the work, we're going to be assessing the quality of data...

38:56...that I need to provide to contractors who are going to actually bid on a job.

39:00So I'm not a GIS guy, I'm a contract officer.

39:03But I don't want to send data out to a contractor for bid if it's in bad shape.

39:08So the goals here are, how do we get a non-GIS user using data quality tools without them knowing it?

39:16 How do we also...this also takes the form of an independent kind of data quality review, right?

39:22This isn't the GIS folks reviewing the data, this is somebody else in the organization.

39:28And finally, how do we report those errors out so this non-GIS user knows what's going on...

39:33...and knows whether he can have a go or no-go decision in the workflow.

39:37Okay? Alright. Let's turn it over.

39:45Alright. Can you hear me okay? Wait, I'm here.

39:52I'll have to speak a little weird. So...okay.

39:59Thanks for introducing the presentation.

40:02As Jay mentioned, we're going to be looking at kind of a sort of loose integration...

40:08...between Data Reviewer Server and Workflow Manager Server.

40:10We don't have the story on Server that integrates...

40:16...Workflow and Data Reviewer [inaudible] step...

40:19...however, these two products have web APIs available both in Flex and JavaScript API at 10.1...

40:26...and those two APIs can be leveraged together to construct web applications...

40:32...that can help users create...utilize workflow management and data validation...

40:39...in a web environment easily.

40:42So that's what I'm going to be talking about.

40:44First of all, how many of you have worked on ArcGIS Server 10.1, yet?

40:51Few hands? Okay.

40:54ArcGIS Server at all? Okay, good, that's good.

40:59So 10.1 ArcGIS Server has a lot of changes in it, and it's a really improved product.

41:05It leverages the cloud a lot and cluster.

41:08I'm sure you've heard about these presentations during the week.

41:11Here's the ArcGIS Server Manager website.

41:14The ArcGIS Server Manager website at 10.1 is very useful...

41:18...very user friendly, and, you know, quick and fast.

41:25I'm not saying that it wasn't, but it is faster now.

41:29So here we see a list of the services that I have in my system.

41:33This is all you need to run Data Reviewer Server and Workflow Manager Server.

41:37You need a Workflow Manager service.

41:39Tope mentioned you have geoprocessing services, map servers.

41:44In this case, it's a Workflow Manager service.

41:46It's starting and running, all you need to do is configure this service...

41:49...to point to a specific Workflow Manager database.

41:53But it's running in SQL Express in my laptop.

41:57The Reviewer service, it's not a specific service like this.

42:00We're leveraging a map service.

42:02And this is because behind the scenes, the Reviewer service...

42:07...leverages the Reviewer workspace to store validation results.

42:10Those validation results contain error geometries.

42:14And using this map service, we can visualize those error geometries on the web...

42:19...so that you can have a spatial view, overview, of your quality of your data.

42:25So if you have an error with a lot of red dots, that's bad, right?

42:28You want to be able to target that area for your validation.

42:32That map service, if you click on it, you can see the capabilities that has enabled.

42:38Mapping is always enabled.

42:40This is the default ArcGIS capability.

42:43And here's the Data Reviewer Server capability.

42:47That basically is a server object extension.

42:49That's what the Data Reviewer Server provides, that server object extension.

42:55It's configured with the Reviewer workspace that's pointing to a SQL Express instance...

43:00...and it's pointing to the GB service that is also running on this server right here.

43:07And this is it for the server configuration.

43:11Oh, one more thing before I forget. Oh, hold on...the instances are not running.

43:21Well see, I might have to restart our GIS server. It seems like it's not running.

43:24But I'll see that in a minute.

43:28So when you go to the capability stuff again, you can see this URL here...

43:32...and this will take you to the service endpoint, to the services directory directly.

43:37So this little shortcut is really helpful when you're debugging...

43:41...and building against your REST endpoint.

43:44At the bottom of the map server REST endpoint...

43:47...you can see the Data Reviewer server extension.

43:50Here you can get a quick look, again, about the workspace that's configured...

43:54...the spatial reference that the workspace is configured, as well.

43:58And you can go through the different child resources...

44:01...and these REST endpoints.

44:03Easily navigate through it, and then make sure that it's actually running.

44:08One quick way to do that is go to the dashboard endpoint...

44:11...and, for example, get a severity count.

44:16Jay mentioned that we're providing this dashboard capability.

44:18So all we're doing on the server is summarizing those Reviewer results in such a way...

44:23...that you get those numbers on the web right way.

44:27This is real-time, looking on the database, summarizing the data, and providing it for you.

44:31It can be filtered, as we'll see in a minute.

44:35So that's the ArcGIS Server overview.

44:38Looks like it's working.

44:41We'll get into the code.

44:44It's Flex code, I hope you guys are familiar with it.

44:48But if not, it's hopefully easy to follow.

44:51I'm not going to get into all the details of the application, I just want to focus...

44:56...on the both Data Reviewer Server API and Workflow Manager API bits.

45:04Let's see how we're doing on time...we're good. Okay.

45:08So, let's look at the design of this application.

45:12Let's go back to what Jay was saying about, what is this [inaudible] going to do?

45:15What is this application going to do?

45:18What we're trying to accomplish is, this Naperville town in Illinois needs to create requests...

45:25...for proposals for a contractor to do construction on the city.

45:31Contractors that will take water utilities infrastructure and improve it or fix it, whatnot.

45:37And what the city hall wants to make sure of is that the RFP...

45:41...contains good data for the contractors.

45:44Well, if you put out a service option, you want to make sure that the contractors...

45:50...have the best data that you can provide them so that the whole operation can go smoothly.

45:56And to do that, the manager doesn't want to get the GIS department too busy.

46:01So what he's going to do is do a quick validation of the data.

46:04If the data passes the validation, then that's good.

46:07If the data doesn't pass that validation, then he'll create a Workflow Manager job...

46:12...and assign it to the data editors to fix that data.

46:16And that's what this application is going to be doing.

46:19Here is the design, how it's going to look.

46:23It's a little...I designed it myself, so it doesn't look very good.

46:26But it's not the purpose of this demo...sorry about that.

46:31We'll see, we have a map area, this is a map component from the Esri API.

46:37And then we have a tab area here on the left.

46:41There's two tabs...the Jobs list will contain the information currently stored on the system...

46:47...and then another tab that will create new jobs if necessary.

46:50This is where the...how did you call it, contracting officer, sorry.

46:55The contracting officer will work really hard to analyze the data.

47:03Alright. So if we run this application right now, it's not going to do anything.

47:08Because I removed chunks of the code that are necessary to run it.

47:14We see empty application, and we see the map and some dots here.

47:18So that's actually pointing to the Reviewer workspace that I was talking about before.

47:24Those green dots are error geometries that a previous validation found on the data.

47:35We'll create more of those error results in a minute.

47:39Just to show you, that's actually pointing to the server. Firebug will help us with that.

47:49Here, you can see these requests here are going to just the map server...

47:58...and just calling the export operation, and getting the results from the map server.

48:04We'll get to the Firebug later...more.

48:07So let's start with the Workflow Manager piece.

48:10Here we see Workflow Manager, this is a list, and then we want to get the jobs.

48:14So let's...let's work on this code.

48:22This is the tab number one, and then the Workflow Manager section.

48:27I created a little component to modularize the code.

48:29Modularizing code is always very helpful, because you don't want to have a very long class.

48:36In this case, I created this jobs list component...

48:39...I'm passing it the map, and Workflow Manager Server's URL.

48:45I have these hard-coded up here.

48:48The server URL for Workflow Manager...

48:51...and the map server URL for Data Reviewer Server.

48:54These URLs, we saw them before in the ArcGIS Server Manager website.

48:57You can copy them straight from there.

48:59They're the same as the services directory.

49:03So when you go into the Workflow Manager's list component, all it is, is a list.

49:09So a Flex list.

49:11However, it doesn't show anything, it doesn't display anything.

49:14Because that Jobs button is not working. Get Jobs button.

49:18The Get Jobs button is right here.

49:20This is the Get Jobs button.

49:22And it has the click event, and it says, Job List, Query Jobs.

49:27Okay. So I'm going to go and write that operation...Query Jobs.

49:31And that's what I have to do here.

49:40This is how you query jobs in Workflow Manager.

49:48Oh, sorry, I stepped over the task part.

49:53What am I doing here? Workflow Manager job task.

49:55And I'm querying jobs.

49:58The Workflow Manager task is the same pattern as the whole Flex API in core.

50:03The task class is take care of the communication between the client and server.

50:09Workflow Manager provides a few tasks and Data Reviewer provides a few tasks.

50:14In this case, I want to use the job task as defined up here to query the jobs.

50:22The query...the job task can do more things than query the jobs...a lot of things.

50:29So I'll refer you to the documentation in this case.

50:34A query ad hoc job takes job query parameters, and that's what we're going to define.

50:40In this case, I want to get all the jobs in the system; but you could, in theory, filter...

50:49...you could easily filter this query to return just a few jobs.

50:54Specifically, we have coded here, if you just want to get the jobs that are assigned to one person...

50:59...or another person on the system, you can do that easily...

51:03...by adding by adding a where clause to the query.

51:06When the Query Job Parameters is complete, you add the fields, the aliases...

51:11...the tables, and then you execute the query.

51:18The third parameter that we see here in this query is the asynchronous responder.

51:21This will take care of the asynchronous execution of this task.

51:24We pass [unintelligible] function, which is right here.

51:31And then the [unintelligible] result function will do something really simple.

51:36It will assign the result to the jobs list that we saw before.

51:43This is the jobs list.

51:46It will assign the result to that data provider.

51:49So let's see if this works.

51:57If you have any questions, please stop me.

52:00I'm trying to go a little fast because we are going to run out of time, I think.

52:04So if you have any questions, let me know.

52:13It's not working.

52:18Let's see what's happening.

52:27Little debugger frame...Sorry about that.

52:53Hmmm...doesn't make any sense.

53:02Oh, sorry, it's just still querying it.

53:05You know, I think...I'm afraid the server needs to be restarted...yeah, still querying it.

53:22Sorry about that, it's going to take a minute.

53:24I'm going to have to restart my RGS server.

53:26I think I had it on sleep or something, and it's going to take just a couple of minutes to restart.

53:41If you have any questions for me...

53:42...or for Jay and Tope at this point while I restart the RGS server, go ahead.

53:49Yeah, question in the back there?

53:50[Inaudible audience question or response]

54:05The question is, Do you need a separate workspace...

54:08...to store the error results from the data validation?

54:11Yes.

54:12Right? So the answer is yes.

54:14Well, no, not...well, it depends on your scenario.

54:16You need a geodatabase.

54:18So that could be either a personal or a file, or it could be an enterprise geodatabase.

54:23Reviewer creates schema in that geodatabase to store the error results.

54:28Now, some organizations actually store their error results in their production workspace.

54:32So they have their data and they also have their Reviewer results stored together in a single workspace.

54:39Does that answer your question?

54:40Yes.

54:41Okay. Cool. Any other questions? Yeah, right here?

54:44[Inaudible audience question or response]

55:07So the question was, What level of license do I need for ArcGIS Desktop...

55:12...to use the Data Reviewer extension?

55:14And then the follow-on is, Do I need SDE to implement Data Reviewer?

55:21So for Data Reviewer, you can run it with ArcView and higher...

55:25...or sorry, Basic and higher licensing level, right?

55:31Though some of our checks, though, used in leveraged topologies, so we have a number...

55:35...of topology checks that someone would have had to create a topology to do that work, okay?

55:42When it comes to integration with Workflow Manager, yes, you do need an enterprise geodatabase...

55:47...for storing the Workflow Manager workflows and configurations.

55:53Data Reviewer Desktop doesn't require that.

55:56But if you...you know, it's going to be the most conservative, right, of the two.

55:59You'll need...if you want to integrate the two and work with them together...

56:01...you'll need the enterprise geodatabase.

56:04On the Data Reviewer for Server side of things...

56:09...we recommend strongly that you use enterprise geodatabases...

56:14...for storing your results if you use the Server product because of the potential locks...

56:20...that could happen with file geodatabases and things like that.

56:24That would be an enterprise deployment, because there's so many things...

56:26...that potentially could go on in terms of reading and writing results.

56:30That make sense?

56:31Yes.

56:32Okay. Any other questions? Yes?

56:34[Inaudible audience question or response]

56:50So the question is, How do I enable ArcGIS Server to utilize the extension...

56:55...and how do I deploy that SOE? Is that right? Okay.

56:58So Data Reviewer Server is an extension for the server extension, so you'll have a keycode...

57:05...that'll enable the extension to work.

57:08As you configure the system, you will need to deploy the SOE to the machine or the cluster.

57:17But the one thing different here is that there is an install of the DLLs on the server that needs to...

57:23If you have a cluster, you do need to install the Data Reviewer extension...

57:27...on each machine, though the SOE can be deployed throughout.

57:32That make sense? So we're kind of...we're not leveraging the SOE as much as we could...

57:36...but we do have a lot of DLLs that have to get loaded onto the machine itself.

57:40Alright? Okay. Any other questions? Okay. Right here?

57:44[Inaudible audience question or response]

57:53So the question is, How do I extend Data Reviewer's validation capabilities?

57:58Or Workflow Manager.

58:01[Inaudible audience question or response]

58:09Okay.

58:10So we have a couple different ways of...if the checks...

58:14...if one of those 42 checks don't work for you.

58:16You can develop a custom check, and I believe it's C Sharp for development.

58:23And then that DLL gets registered on the machine...

58:25...and then it's configured as you would any other check.

58:28You register the DLL, identify the parameters...

58:31...and then store it inside of a batch job and then it runs along with other checks.

58:36At 10, and expanded on at 10.1 on Desktop, we have a series of GP tools.

58:42So potentially you could develop a script or a GP model that uses our right to Reviewer ...

58:47...GP tool to the results of that analysis, or that model...

58:52...could be written into the Reviewer workspace.

58:54So it's kind of a middle area where you don't have to be a developer, you just need...

58:58...to be able to write a Python script or develop a model, the results of which...

59:02...would be read into or written into a Reviewer workspace.

59:05So that's sort of the scriptable side of things. Does that make sense?

59:08Yeah.

59:09Okay. Are you ready to go?

59:10Yeah.

59:11Okay. We'll take some more questions at the end, but I think Javier's got his machine up.

59:14Yeah, sorry for the little glitch. Hopefully...I mean, thankfully it was just a glitch.

59:19This seems to be working again.

59:21So the Click button that I coded before will get the list of jobs and will render...

59:27I have a little item rendered here for the list that will know how to display these jobs correctly.

59:32These are the job IDs. This is job type, and then job number.

59:37This is the job creator, Tope, thank you. And then myself. I created one before.

59:42This is the one that I actually generated the results that we saw before.

59:47We saw these Reviewer errors here. And how would we know that?

59:53We would click on this and it would take us to the AOI.

59:56But that's still not in the code, so I'm going to work on that right now.

1:00:00How do we get the AOI of that job?

1:00:05So one thing that I didn't mention, I don't know if you're...knew that...

1:00:09...but Workflow Manager jobs can have an area of interest that...an area of interest...

1:00:15...or an analysis area assigned to them, so that the person that has that job...

1:00:20...assigned can, or knows where to work spatially...

1:00:25...in which area of the data to work.

1:00:27In this case, we're doing that...we're using that AOI...

1:00:31...to link both Data Reviewer Server and Workflow Manager.

1:00:38So here's how we're going to get the AOI.

1:00:40I mentioned when you click on the list, right?

1:00:42So what I did was add a change event handler, which is when...

1:00:49...the selection changes on the list, the list will throw an event.

1:00:53That's the change event.

1:00:55And this handler will get the AOI of the job, it will be this function here.

1:01:04This function...really simple, again.

1:01:10This is a bunch of projection code that I'm not going to show, but just to show you...

1:01:14...how to get the AOI from a Workflow Manager, we go back to the task...

1:01:19...that we were talking about before and we get the actual job.

1:01:29That will give us all the details about that job.

1:01:32Who created it, when was it created, what was the due date.

1:01:35In this case, all we're going to do is get the job based on the job ID...

1:01:39...that we had before and then do on Result Event, we'll get the AOI.

1:01:44That's all we're interested in right now.

1:01:46So with this little piece of code here, now we can see, hopefully, the AOI on the web.

1:01:59So we get the jobs, and now these two jobs don't have AOI.

1:02:03It's not that it doesn't work. This job has an AOI.

1:02:06So the AOI now shows up on the map and displays. That was my validation area before.

1:02:13And we have an outlier here. I don't know what happened with that one.

1:02:19Anyways. The next thing we're going to work on, then, is the Data Reviewer piece.

1:02:23We talked about how to get a list of jobs and how to navigate to that AOI...

1:02:27...but I also want to see more details about those errors that we found during validation.

1:02:32So to do that, we provide the dashboard capabilities that will summarize the data...

1:02:38...and will show in a graphical way, What is the status of your data?

1:02:43 How is your error trend going...

1:02:48...Do you have more errors in this particular check title or this particular feature class?

1:02:52In this case, we're going to show our distribution by severity...

1:02:55...meaning you can have errors in multiple severities.

1:02:57Some errors might be very critical to the infrastructure, some errors might be not so critical.

1:03:03So we'll add a pie chart here that will show the distribution of severity of these errors.

1:03:09So we'll work on that. Back to the index page.

1:03:14We looked at the Workflow Manager section; I'm going to close it.

1:03:17And now we go into the Data Reviewer section.

1:03:21We just have some decorations here, and another component.

1:03:25I created another component [unintelligible] modularize the code again...the dashboard component.

1:03:31What do we pass to this component?

1:03:32We pass the URL as before, import the URL for contacting the task.

1:03:38And then we also pass AOI, the AOI that we just used.

1:03:42I clicked on that job, and then we got the AOI from the server, and now I'm showed it on the web.

1:03:48And that, thanks to the Flex binding, will automatically get injected into this component.

1:03:56This component...this is how it looks like. It's very basic.

1:04:00Dashboard tasks, like I talked before about the task, these will communicate...

1:04:03...with the server, and then a pie chart, Flex pie chart standard, and then a legend.

1:04:09It doesn't have anything else, really. A couple of labels. That's all it is.

1:04:15In the script, we see a setter for the URL...

1:04:20...we see a setter for the AOI, and then a refresh method here.

1:04:24So when the AOI is set into the dashboard, we're going to refresh and get the data.

1:04:31I was going to split it in two steps, but I'm just going to do it...

1:04:33...right away, because we're running out of time.

1:04:46So this refresh method, all it's going to do is check...

1:04:49...to see if we have an actual AOI in this component, because we might not.

1:04:54If we don't, we'll get results by severity.

1:04:58What this function is going to do is go to this REST endpoint...

1:05:00...that I showed at the beginning and get the counts.

1:05:02It's going to get a summarized count of the amount of errors...

1:05:05...that you have in your Reviewer workspace divided by severity.

1:05:09So for severity 1 it's going to give you 10 errors or whatever.

1:05:12For severity 2, 20 and so forth.

1:05:15If we have an AOI, what we're going to do is create a special filter.

1:05:20The special filter will apply to that request on the dashboard...

1:05:24...and if we're only looking at a specific area of the database...

1:05:28...the summaries will update only with that information upon that area.

1:05:32Then we call this different operation, we pass the same field name...

1:05:39...but then we pass a list of filters.

1:05:41We're showing a special filter.

1:05:42You can filter by many other fields as well. We're running out of time.

1:05:48So let me show you that working.

1:06:06So right now, the pie chart is loading the total errors on the workspace.

1:06:13Not sure why it's [unintelligible]. Another little problem.

1:06:31Oh, yeah. Of course, the dashboard task has two pieces.

1:06:37One is get the results and another one is handle the results.

1:06:40So this is an asynchronous call, right?

1:06:43And what I need to do is, when the request completes is...

1:06:50...I need to handle the result and I didn't do that.

1:06:52So I'm sorry. That's why the pie chart wasn't loading.

1:06:55What I'm going to do is add this dashboard event and handle the results.

1:07:01We did a little, again, decoration code.

1:07:04And to handle the results, all I'm going to do again...

1:07:07...as we saw before with the Workflow Manager is assign the results...

1:07:13...to the data provider of the chart.

1:07:15In this case it's not a list, it's a chart.

1:07:17With that little line of code, we'll see the chart populate with the data.

1:07:26There you go, that's the chart.

1:07:29So now we're looking at a total count of the workspace.

1:07:32If we wanted to look at a specific area, we'll select...

1:07:35...the job, Workflow Manager job, and it will filter.

1:07:39I don't know if you noticed, there's one out here, there's a difference of one between 38 and 39.

1:07:46Can you see that? 39, 38...that's the difference. Alright.

1:07:53That's the display of the current system.

1:07:54Let's look at what the contracting officer would do.

1:08:00Am I running out of time? Five minutes? Okay.

1:08:04A contracting officer would do.

1:08:05I'm going to plug in all the code and just show you how it works before we run out of time.

1:08:10Sorry about that.

1:08:25So what the contracting officer will do is go to this page and say...

1:08:30...Well, I have this polygon here, but I want to validate area in another section of the town.

1:08:37For example, down here. So I'm going to draw a new polygon and I want to know...

1:08:45...if this area contains enough errors for my data editors team to start working on it or not.

1:08:52Alright, so I select that polygon and then, what do I need to do to validate it?

1:08:57I need to find a rule set. What rules do I want to validate this data against?

1:09:02Those rule sets are in Data Reviewer, are configured as a batch job...

1:09:06...which is an XML file that you'll probably have in your hard drive.

1:09:10You need to select the one that you want.

1:09:12In this case, I'm going to look for water utilities and upload it.

1:09:18Then click on Execute and that's going to work and then the validation will run.

1:09:22So we'll add that code really quick.

1:09:30This is a lot of code for uploading.

1:09:32Flex uploads our [unintelligible].

1:09:35It's not complicated, it's just you need to write a lot of code to make it work.

1:09:42But this is our code for executing a batch validation.

1:09:45We create a batch validation job, we assign a title, and we assign the analysis area...

1:09:49...that will be the same as the job AOI, and then we assign it a session.

1:09:55Session IDs are defined through Data Reviewer Desktop.

1:09:59And the last thing we do is just execute it.

1:10:01And again, we have to go to the batch validation task and look at the event...

1:10:05...that will be thrown when this execution is successful, right?

1:10:10So that execution, what it's going to do, since it's...it could be a lengthy execution...

1:10:16...we're going to do it asynchronously.

1:10:19This handler...we'll start a timer and we'll check the server until it's actually completed.

1:10:33And this is how we know if it's actually completed.

1:10:36We've got the execution details.

1:10:40And with the execution details, it will tell us the status.

1:10:42If the status is executing, we continue waiting on that loop.

1:10:45If the status is not executing anymore, that means we're done.

1:10:50I mean, we could add more error checking here because the validation could fail also.

1:10:54But in this case, we assume it's going to work. So let's see that. Actually...sorry.

1:11:08Before we see that, let me add the rest of the code so we can just run everything at once.

1:11:16I'll talk to this a little bit.

1:11:42Alright, back to the same page, same AOI, and we're going to create a new one.

1:11:59We picked a bad job, it will contain several checks...

1:12:03...and it will run against the data that we have it configured.

1:12:06So the execute batch job, we saw how we create the settings object...

1:12:09...and we pass it to the server, and now it's executing.

1:12:14This is checking with the server every 15 seconds to see if the execution succeeded.

1:12:22This will take a couple seconds here. And there's a bug in my code. It's probably a bug.

1:12:46So I won't be able to show you that.

1:12:50But basically, when the execution succeeds, we'll be able to see the results here.

1:12:55There's a connection problem here between the execution and the results...

1:12:59...but we'll see that updating anyways when it's done.

1:13:03We just don't see the [unintelligible] message running anymore.

1:13:10When that actually executes, the user will be able to decide, based on how many results...

1:13:19...that we find here and what is the distribution of the severity, we'll be able to define...

1:13:24...well, this job needs to be...or this area needs to be edited.

1:13:27So that will mean yeah, I'll create a new job.

1:13:30Create a new job, I'll assign it to my data editors, and that will go on the Desktop...

1:13:33...and run the workflow through the desktop steps.

1:13:38I think we're running out of time, so I'm just going to create a job...

1:13:44...the job is created, and it shows here on the list.

1:13:49And the data editors have more work now to do.

1:13:52The validation is still running, sorry. I guess my server is a little slow today.

1:13:58But when it's done, it will show the results here.

1:14:03So that's it, we're almost out of time, so if you have any questions...

1:14:13Yes?

1:14:14[Inaudible audience question or response]

1:14:25Yeah, we don't currently have plans for a Silverlight API, but we'll monitor that.

1:14:29As a new release, we kind of had to put our resources into certain areas...

1:14:34...so we're going to monitor that to see how things go with Silverlight going forward.

1:14:37If we get a lot of requests, we'll consider it.

1:14:41Any other questions? In the back?

1:14:44[Inaudible audience question or response]

1:14:46So the question was, Is the Flex source code available? We can make it available.

1:14:52We haven't released Data Reviewer Server yet...

1:14:54...so you won't be able to execute it unless you have access to prerelease.

1:14:59But we can put it on the web. We'll work on that as soon as we can.

1:15:04Another question? Yes?

1:15:08[Inaudible audience question or response]

1:15:12So the question was, Is Data Reviewer Server available for prerelease?

1:15:17So we did a very small prerelease with only distributors.

1:15:21If you're interested in evaluating it...

1:15:24...as soon as the 10.1 site goes live, you'll be able to get evals of it beforehand.

1:15:29Release is not too far off, though, at this point. Any other questions?

1:15:35Alright, just real quick, just a couple seconds left here.

1:15:41Resources, resources...the beta resources center...

1:15:45...is up for both Workflow Manager and Data Reviewer.

1:15:49You can find things like the help, the documentation...

1:15:52...this is all going to be prerelease documentation at this point.

1:15:55The gallery includes samples, tutorials, templates that you can take and run with.

1:16:02This will also be where we publish our web client APIs for Flex and Silverlight and JavaScript.

1:16:09Forums as well as blogs on that site as well.

1:16:12Additionally, training is available in both products, either as instructor-led training classes...

1:16:17...or virtual instructor-led classes.

1:16:19There are a series of web courses that you can take online...

1:16:21...as well as some free training seminars for both Workflow Manager and Data Reviewer.

1:16:30Any last questions? If not, I thank you very much. Please take an opportunity to give us your feedback. There's some eval forms in the back. Let us know how we did or how badly we did, that's fine too. And thank you very much for your participation. Have a good one.