00:01Hi. Well, welcome! Welcome to the ModelBuilder Advanced Techniques session.

00:04 I had a weird thought the other day, sort of like, the User Conference, it's like...

00:11...the source is pure here, like all these users; you guys are the source...

00:15You're the source for me, I'll tell you that.

00:17But I got...you might remember that, that's what they said in Tron, so I was thinking...

00:21...I didn't know if anybody would remember Tron.

00:23But I'm riding on the bus this morning...

00:26And they got this graphic arts convention and they got Tron signs all over San Diego.

00:34I couldn't believe it.

00:35So, I'm happy to be here and we're going to talk a little bit about ModelBuilder and basically...

00:42...we call this Advanced Techniques in ModelBuilder.

00:45 The idea here is, say, well, what do you need to do to become proficient in building models?

00:51All right, think about...

00:52So primarily ModelBuilder, the main thing ModelBuilder is, it's a way to put tools together.

00:58So, the most important thing is to know how to use geoprocessing tools.

01:03I'm not going to talk about that in this session, but that's the most important thing.

01:07Know the tools for the problem that you're working on, and how to combine them.

01:13So, there's some pretty nice help topics in the geoprocessing help about introduction to commonly used tools...

01:22...and then there's the geoprocessing model tool and script gallery that you can go to, to see samples of the way...

01:28...that other people have put things together.

01:31So, really make sure you go through the geoprocessing with ModelBuilder help, make sure you check out...

01:42...ArcGIS.com/geoprocessing and all the samples there, and also the forms there to help each other building models...

01:50...and how to solve the problem that you're dealing with.

01:51Okay, so now, specifically, so let's say, I’ve got that figured out.

01:55I'm building...I've become proficient in the basics of models, putting tools together...

02:00...basically making models more flexible.

02:01...and the things I want to do.

02:02What else can I do?

02:03Well, there's some more advanced techniques, some things like what we call iteration or looping of models.

02:10There's a technique we call inline variables; we can make parameters a little more flexible.

02:17There's some special utilities that you really only use in ModelBuilder, they have...

02:24They don't really make sense as tools and scripting.

02:27They basically have the counterparts of utilities that you get in scripting languages.

02:33And the others control the flow.

02:35So those are the things we're going to talk about in this session; so how would you take your models and do...

02:40...a couple more advanced structures with them in processing.

02:45 Just make sure everybody knows where the help is; and we have the professional library.

02:51Within that we have the geoprocessing book, and within that we have Geoprocessing with ModelBuilder.

02:57All right; go through all the topics here, there's actually a section in there called Advanced Methods within Geoprocessing...

03:08...within ModelBuilder, and those are the things that we'll be talking about.

03:13So the first thing I want to talk about is model iteration.

03:17So the idea here is, hey, I just want to repeat the same set of operations, over and over, on either different datasets.

03:26Different datasets, or different collection of features.

03:30So, for example, I just want to process all the feature classes in the workspace. 00:03:35

03:41And you see this little hexagon. 00:03:42

03:48You give it a workspace, essentially, so that model will run for every feature class in that workspace. 00:03:57

04:04They each have different utilities, or different cases in which you would use them.

04:11So, For is just like a fixed number of times.

04:13So you just say, hey, I want to run this model five times.

04:17Okay, you put in For, it's just like a For loop in scripting.

04:23While gives you the ability to run a model until a condition is true.

04:30So those are pretty classic, For and While.

04:33So the next group is feature selection, road selection, and field value.

04:37They basically read values out of feature classes and tables. 00:04:40

04:45And then the other one is what I think about are the dataset types, so the for each dataset in a workspace...

04:51...so for each feature class in a workspace.

04:54For each raster in a workspace.

04:57So basically you're doing the same type of processing.

05:00What you want to do is a throw a workspace at the model and say, hey, every raster in here, process.

05:05I want to do the exact same thing to everything in that workspace.

05:11So those are the types of iterators we have, and I'm just going to go through a couple of the examples here...

05:17...hopefully to illustrate what they look like.

05:28Actually, just to start out here, I'm going to just create a new toolbox, and inside the toolbox, create a new model.

05:42And within that, come down here to Iterators, and I pick Feature Classes, and then I just want to go down...

05:55...and the main property in here is to pick the workspace that I want to use.

06:01In this case I want to use my Exercise 1.

06:11And then for each of the feature classes in Exercise 1, I want to do some kind of processing.

06:16So I'm going to open up another model, and I've got what I want to do in there already.

06:23It's just...a real simple one called Calculated Areas.

06:28So for every feature class in this workspace...actually I'm just going to copy it, I'm going to add a field to it...

06:36...in this case I'm going to add a field called Acres, and I'm going to do a calc field.

06:47I just want to point out something if you're not aware of, that calc field has a really nice little feature in here.

06:51You can do shape-dot-area; you can do shape-dot-length.

06:54In fact, in here you've got shape-dot-area at acres.

06:58So if you have a projection, we can convert it for you.

07:00So that's a nice little feature in Calculate Field.

07:04Now the other thing is, you say hey, I'm calculating acres, I want to make sure that I'm only doing polygon feature classes.

07:09So, I've got my workspace, but I can also come in here and I can specify that I only want to do polygons, in this case.

07:18I can also put in wild cards; I could put in "only start with feature classes that start with A", or something like that.

07:25So the idea is, I can cycle through there.

07:27And also, with all these dataset iterators, there's recursive.

07:32So the thing about recursive is, it'll start that folder.

07:36It will start at... well, it really only has an impact if you deal with folders, it'll have impact with workspace...

07:41...and feature datasets, also.

07:44For example, you pick a folder, it will find all the feature classes in that folder, and all the feature classes...

07:49...in the subfolders.

07:50So, that could be a nice feature.

07:52It could also be very dangerous.

07:54Depending on what you're going to scan.

07:57It actually is possible to pick the C drive and scan your entire disk.

08:00It takes a while.

08:01It's not highly recommended.

08:04But, we actually did some testing that way, just to make sure it wouldn't blow up.

08:09So, just be careful.

08:12That's what really what I wanted to say was, be careful with that.

08:17So, I'm just going to take this and run it now.

08:20And what it will do is cycle through for all the polygon feature classes that are in that.

08:26It will copy them and output the values.

08:29I just happened to put them into this scratch workspace.

08:33For example, I had vegetation; I just copied it, and all I want to show here is that in the table now...

08:42...I've got a new field called Acres with the value in it.

08:47The processing was simple.

08:48It's a simple example, in this case, whatever your model does, it will repeat it...

08:53...for every feature class that is selected by, by the iterator there.

09:07That's one example of setting up... and I'm going to clean up my directory here a little bit.

09:20We also get a lot of requests for...what I want to do is, I want to read a value out of the table...

09:26...and for every value in the table, I want to do something with the value.

09:34So, in this case, we have "iterate field values" and what it does is, you give it a table, you give it a field.

09:41In this case I set it, and I set it to be a unique value, so I'm going to use vegtype.

09:46So, all the features that have, you know, and I'm going to grab that value out and I'm going to use it.

09:51So, if you look at the output now, now I have this thing called value, and that's going to be vegetation type...

09:56...each time the model runs.

09:58What am I going to do with value?

10:01Well, in this case, I'm just going to use it in a Select expression.

10:05So I got in here vegtype equals value.

10:09And you might notice in here, I'm going to go a little bit...

10:13I just want to point out, you see this percent value percent?

10:18So that, this is an expression, a select expression.

10:21But I want to use, if you think about it, what I want to do is I want to take whatever's in this variable...

10:28...and I want to make it part of this expression.

10:30And I don't want to rewrite the entire expression.

10:33I don't want = vegtype equals will always be the same.

10:37So, we have a convention; we call it inline variables.

10:40And what you do is you say percent, the variable name, percent.

10:44And at runtime that would be substituted.

10:46So, what's happening here in this model, is each time it runs it extracts value.

10:52It does a select statement based on the value.

10:54And, in fact I'm going to collect those values up.

10:57So let's just run that.

11:02So I really actually think of this, it's almost like split by attribute.

11:05So now what I've, if I go look at this, I've generated these, in this case these six feature classes.

11:15I can add them to the map real fast if I wanted to, but basically what it did is just split by attribute.

11:20And in this case, I did a Select statement, and I generated a different feature class for each of the outputs in that case.

11:29But the main thing to realize is, hey, you can use a table to draw a model.

11:34It grabs the values out.

11:36But one thing I actually point out here too is that, one of the things we have on here is data type.

11:42So, the data type is actually not the data type of the field; it's the data type of what you're going to use it for.

11:50Now in this case I set it to string because I was just going to use it as a string in this Select expression.

11:56It could be anything.

11:58If you look under data type, it's any of the geoprocessing data types.

12:02So, we take that string and convert it into whatever you say to convert it to.

12:08So, we can take that string.

12:10Imagine if it was a table of path names to feature classes.

12:16We could do that.

12:18If it was a linear unit.

12:21So it doesn't have to be just strings and numbers.

12:24It can be...that text can be converted into any kind of value that you want to convert it into.

12:35So that was...I sort of used that as a segue; I said it was going to split by attribute...

12:40...so we actually started here.

12:44That was really the number one request we got, which was, I want to be able to run a model based on values in a table.

12:50I said, great. Well, what are you going to do with those values once you use them?

12:54Oh, I'm going to do a Select expression to select features.

12:57So we basically said, well, why don't we just give you an iterator to select over features?

13:01We don't want to give you an iterator...grabbing values over tables is nice, but let's just go...

13:07...straight to the feature and be able to do that also.

13:10So in this case, this one's called Feature Iterator.

13:16It inputs the feature class or the layer feature class that you have available to it.

13:23Again, you pick a field...you can leave Field blank.

13:26If you leave Field blank, it will do every row one by one.

13:28Essentially, that's like clicking the OID field.

13:31That's the Object_ID field.

13:33Or you can pick a class field to group things by, so everything that has the same value will become a group.

13:42Essentially, it's like calling Make feature layer with a Where clause.

13:45In fact, that's exactly what it does under the hood.

13:47It creates a layer of those features, now you do whatever you want with those features.

13:53You can enter multiple fields here.

13:56So if you want to classify by multiple fields you can do that.

14:00Also note, with all the ones that with field values, you've got to figure out what to do with Null; these damn null values.

14:09Do you skip them or do you use them when you pull values out?

14:13Sometimes you might want to say, skip the null values.

14:17Other times we'll just convert them to zeroes or blanks, if we run into, if you don't skip them.

14:24So, this one, if I run this, it cycles through each of the features.

14:31It copies them out.

14:33I really did the same thing that the other model did, but in this case I didn't have to do the...

14:37...Select statement, I just output all the values, and now I have each of the individual ones separated out...

14:44...if that's what you want to do in those cases.

14:54I'm going to switch over to another example.

15:04So this is a case there's actually, I pulled it off of, a question off the forum.

15:09And the question was, well, how do I make a viewshed for every point?

15:15Now, viewshed will take multiple points.

15:17So I don't want to say, viewshed will take multiple points and for a given area it will tell you how many points can see it.

15:23But it doesn't tell you which point can see it.

15:26Now there is another tool, and I forget the name of it, I'm sorry, I'd have to go look it up, that will take points...

15:31...and tell you which point sees it, but it has a limit of the number of points.

15:35If I was only doing three points that's what I would use, but for the purposes of this demo I only wanted to do three points.

15:42So the idea is, I want to be able to take a point feature class, and then for every point in that point...

15:47...feature class I want to make a viewshed, okay?

15:51So, if you look at Generate Viewsheds, it comes in here and...let me make this a little bit larger so you see it better...

16:07...so I just start with Iterate Feature Selection, I start with a feature class, and I'm going to...

16:13...in fact, in this case I leave everything blank.

16:16I'm using ID because I want to put the ID value after each one.

16:20And I run viewshed based on that.

16:23And I do the basic things I do with viewshed; I run the majority filter and I collect the values out.

16:28Now one thing I do want to show is that when I do the majority filter, if you look at the path name here...

16:33...again, hey look at that, percent value percent.

16:37So, the output of Iterate Feature Class has the feature class itself.

16:44So anything that takes a features class as input, you can use that as input to any other tool.

16:48What we've found was real useful is a lot of times, in particular, I wanted to use that value field as the name of...

16:55...to append it to the name of an output.

16:58So that's why we have this little appendage on there, a value.

17:01So, mostly you do the processing on the feature class, but this value variable becomes useful to use for naming things.

17:11For things that you name as output.

17:15So this model cycles through each of the features, and it creates a...and it does a majority filter on there.

17:23Now what I want to do is I want to take all those, and I want to merge them into one.

17:28So I have each of the individual ones.

17:32So I have the Generate Merge set, and I have the combined one, so that's actually a separate model.

17:40So one thing to think about, one of the key points about iterators is, one iterator per a model, the entire model runs.

17:49Okay, not just what's connected to the iterator; it basically treats the model as a Do loop.

17:55So you have to partition things.

17:57So if I'm looking at this, well, I'm going to generate the viewsheds, I want that to run, so everything...

18:04...in this is going to run through each feature.

18:07And then when I'm done, I want to call Combine.

18:11So I needed a method to take the results of this, more than one output, and send them to the Combine tool.

18:19So there's a...one of the model-only tools that we have here, look at this, called Collect Values.

18:27And the way you find that is you go to Model Only Tools, Collect Values, and you connect this output to it...

18:35...and what that does is, as the model is running, it will collect all the, in this case, all the rasters that were...

18:40...generated by that.

18:42So I'm going to take this now, and I'm going to run the main model.

18:50So now it runs the submodel that has the iterator; it does three viewsheds, and it combines them together...

18:56...and now I'm just going to pop them up on the screen, and then you get those three together.

19:02And if I open the attribute on this, what Combine does, it gives me Mview1, Mview2, and Mview3...

19:08...so I get 0,0,0, so in each of the case I can say, oh, that was seen by point 2 and point 3.

19:14I get all the different combinations of viewing in that case.

19:22So the main point there is, hey, I've got a main model that does some processing that only runs once.

19:27I've got a submodel that has an iterator in it that does processing, and I collect those values together and pass it on out.

19:37And you can do the exact same thing with features.

19:43So, in this case I did Combine; here I have the outputs are, it's basically the same model, the only difference is...

19:52...I do a little polygon to raster to polygon here to convert the values, and then take those values and merge them all together.

20:04So now I'm actually going to just run this as a tool here, it does the processing, merges them all together...

20:14...in that case it just happens to be a feature class of the output that you want.

20:30Okay. So I want to stop there, and open it for questions.

20:34So I just want to make sure we got through the basics there of, you can add the iterator to model...

20:40...it runs the entire model a number of times.

20:43And there's a couple of different ones that basically, on what type of data you have...

20:48...you have different kinds of iterators.

20:50So, are there any particular questions right now about that, right here?

20:55[audience participation - inaudible]

21:01Can you iterate by excluding a feature?

21:11No, the iterators themselves don't have that built in.

21:14You could do an additional Select by Attribute that would build that in to do that.

21:21But, no we don't have an exclusion list, I'm sorry. Good question.

21:25[audience participation - inaudible]

21:38The question is those values, so those values that come out of the iterators, can I use them as extent?

21:44Well, if it's an extent, yes you can do that.

21:46So, those outputs, if you go, whether there's any of these...yeah, any of these variables now are available...

22:01...to be used as inputs to other tools.

22:04There are data-type issues there.

22:06So, you can always use a geodataset for extent.

22:11[audience participation - inaudible]

22:23Absolutely. If you were doing feature selection, that's something you could use as a mask.

22:28If you were doing iterate raster, that's something you could use as a mask, exactly.

22:33[audience participation - inaudible]

22:38No, you just connect it to the tool.

22:40So, let's say for example, I wanted to, so in Generate Viewsheds, if I connect this to the viewshed tool...

22:51...one of the environment settings is extent or mask.

22:54Okay? So boom. It's connected and I'm using it for extent, in that case

23:02So you don't, I'll talk a little bit more about the inline variables, is that, while you bring it up, is it's...

23:09...oh, I can use percent value percent everywhere.

23:12There's only a handful of places where you really need to use it.

23:16It's really useful in expressions, because it's not the entire value.

23:21If it's just the entire value, connect it.

23:24So use that.

23:26Question here?

23:27[audience participation - inaudible]

23:35All right, so the question is, to get iterators it runs through; is there a way to nest them, you nest them with submodels.

23:41So, we don't allow multiple iterators per a model, so if I come in here and I go to iterators...

23:46...you'll see it's disabled.

23:48The rules about how they combine get really complicated.

23:51But, what I can do, basically I can make another model, and I put an iterator in there, and then what I would do is...

23:59...I would pass, say pass this as a parameter to the submodel, right, to do that.

24:09[audience question – inaudible]

24:17Okay. So the question is, how do you get a model into a submodel?

24:21Let's say that if I...So I'm going to create a new model here, and I can just say, hey, I got this one Generate Viewsheds...

24:32...I just drag that in there, like that, and in case, what I want to do is I want to combine it with the Combine tool.

24:42And I'm still kind of old school, I still go find it in the tree view.

24:52And then you find the Combine tool and you put it in there.

24:55So just any model can be included in another model.

24:58We actually do, but there are checks, if I went to Generate Viewsheds, and I try to put...

25:06...Generate Viewsheds in it, it says you can't put a model in itself.

25:10Hopefully, I'll get the right.... Yeah, there we go.

25:13So, it doesn't call itself all the time when it does that.

25:17That's really important.

25:18You know, models can call other models, you've got to...I'm going to talk about that later on about how you set up models...

25:24...to be tools, what parameters you expose to do that.

25:27Same thing with scripts.

25:29Any other model, any other script tool, all the system tools can be included in your models.

25:42So a couple of key points.

25:43The questions, I think, covered most of them, but one of the model runs for every iteration...

25:51...only include things you want to run.

25:54Every time you use submodels to separate things out; a lot of times I have a main model, it does some preprocessesing...

26:00...it runs the submodel that has the iterator, and then does things with the results.

26:06A model may only have one iterator, so you've got to use submodels to combine them.

26:11Collect Values is a real interesting tool.

26:13It will run the model, and then for everything that gets generated, it will keep a list of those values...

26:19...and then you can use that to pass, to other tools or to other models.

26:28Use the name and value variables, inline variables; I'll show some more examples of that a little bit later on.

26:35So the dataset areas are recursive; remember, be careful about that; it's pretty cool, but ...

26:40...it will just keep running unless you press Cancel.

26:46It does, actually. Pretty nice, huh?

26:51It checks at every workspace.

26:53So, it goes out, makes a giant list so, at least you know if you press it halfway through it will stop.

27:00It really aggravated me when it didn't, so I put it in there.

27:04I figured it'd aggravate you, too.

27:08So, feature, row, and value, goes by each, so you can do each row, and you can also group them by fields.

27:18So there's another tool called Iterate Multivalue. And most of the iterators are data...

27:24...what I would call data driven.

27:26You point them at a dataset, you point them at a workspace, and they just read it.

27:30That's really what you mostly want to do, but sometimes you want to pick things.

27:34I want to pick a list of things.

27:36We have this Iterate Multivalue, you create a variable and the data type you want, and you use that.

27:41And I also make a comment, basically this replaces, if you use... we've had iteration in models...

27:48...since 9.2 and 9.3, and basically there was a simple tab, it was For and While, you could give it the...

27:55...number of runs, and you had these things called List and Series, and we basically found that the use of...

28:00...these was cumbersome, let's just say that.

28:04And so we tried it.

28:06We learned some things and we came back and we worked on iterators, hopefully to make these things easier to do.

28:12So really, Iterate Multivalue replaces List and Series.

28:15List and Series will still work if you have models that have that in there.

28:19But probably if you're building new ones, I really recommend that....

28:22you go to using the iterators, they have an easier structure to deal with.

28:30So we talked about inline variable substitution.

28:31A really good example is inside of an expression, so it's really a way...

28:38 If I look at a parameter, and I see that the entire parameter is the inline variable, well...

28:44...there should be no reason for me to use inline variable.

28:47Inline variable should only part of the parameter, part of the test.

28:51In this case we look at it; all it is, is the value that is there.

28:56So, primarily use a string or path parameters are what you use it in.

29:02It's keywords, so what are the keywords that you can use?

29:06You can use any variable name that's in the model.

29:09You can use environment setting, so you can use scratch workspace, which is actually a real common one...

29:17...to use. You could use cell size.

29:18You could use extent, if it was appropriate to. In that case...

29:23...there's a built-in keyword, which is percent n percent n, which is the iteration number.

29:27One, two, three, four, that you can always use inside expressions.

29:34So it's Select expression.

29:35So, as I put a big capital...

29:40Make sure you include quotes!

29:43So, look at this expression.

29:44This guy, it says, "airport ID equals," you know, whatever expression.

29:49So, what I recommend is that you build the expression first and see what it looks like.

29:54And then the piece you want as substitute, type that in there.

29:58In this case, airport ID.

30:01The substitution doesn't know when or how to quote or not quote things.

30:06You actually have to interject that.

30:08So if airport ID was a number, I wouldn't quote it.

30:11If airport ID is a string, then I do need to quote it.

30:14So you have to have some knowledge about how it's being used for the expressions you're building.

30:21Calculate Field is a good place to use it.

30:23Path names to output datasets are a great place to make, this is in particular scratch workspace...

30:31...name of a feature class, is a great way to make things portable.

30:35So whatever the crash workspace is, this model will use it.

30:42So one of the things that we saw a few in some of the examples I did...

30:47...I didn't just do iterators, I've got some of these other tools that we were working with, so...

30:52...for example, Collect values to do that.

30:55So, there are these things we call model only tools; sort of the best name we came up from; basically they are tools...

31:02...that only make sense to use in models.

31:03I don't need to do things like, I don't need to do Calculate value in scripting...

31:09...because I can do just do simple equation in scripting.

31:11So the idea was, hey there's some things I can do in scripting, let's work out the corollary that...

31:16...I can make it real easy to do inside of a model.

31:22So one of them is Calculate Value.

31:23It's just a simple calculator, so if I want to do a simple calc...it's really like Calculate field but just for one expression.

31:32Not for every feature in a table or every row in a table.

31:35So you could put any Python expression in here.

31:39You can also put a code block in there, so it's actually kind of a tricky little way to embed some Python in your model.

31:46So, for simple things that are constant, this is a good way to do that.

31:50This is actually one from an example I'll show later on, but what it does, is it actually uses ArcPy to get the install directory...

32:02...and from the install directory it gets a style sheet that pulls the value, and notice the data type on the bottom.

32:08So mostly you're just doing simple expressions, numbers and Booleans but sometimes...

32:11...you do more complex things and you can do that within Calculate value.

32:19There's one called select data, and the reason why...the primary use of it is, well, the only use of it is...

32:24...to pull value out of things, so...

32:27...if you're familiar with Network Analyst, Network Analyst has a tool called Solve.

32:32So you have your network, and you enter various locations in it and you call Solve.

32:37And Solve itself is a Network Analysis layer.

32:40It's a layer that has lots of sublayers.

32:42Well, Intersect doesn't intersect Network Analysis layer, Intersect intersects feature classes.

32:47One of the sublayers is a feature class, which happens to be the route or the service area that you use.

32:53So in order to get to that we have this thing called Select Data.

32:56So, really just think about, it grabs a subdataset out of something that's an intermediate data...

33:02...inside of your model.

33:07Collect values, we saw that, we use it mainly with iterators, we run the iterator, you make it as the output, it collects all of it...

33:15...it will make a multivalue of all the values that gets generated when that model is run.

33:21It's also kind of a handy little deal that, you can take two multivalues and combine them together into one with it.

33:28So, the idea is that, basically, like if you're connecting to merge and I've got two lists, two multivalues...

33:36...and I want to send it, well merge will only let you connect one.

33:38It says, hey, I let you connect multivalue, I let you connect lots of single things, I don't let you mix and match.

33:45That was the rules that we made for that.

33:55Some of you might have used lists in 9.3, and a lot of times you generate lists and that was nice...

34:00...and then you want that list to go into merge, and merge says well, it's going to run more than once...

34:05...and this is actually a way to turn that list into a multivalue for merge.

34:10Get field value just grabs the value out of a table.

34:13So, this is real interesting. You run sum of statistics, you get the sum of something and you want to use that number.

34:19Hey, just grab it. Give you the table and the field.

34:22It grabs it out of the first row.

34:24It doesn't walk all the rows.

34:26We have the iterator, Iterate Field Value that does all the rows.

34:29If you just want to grab one real fast out of there, use that tool.

34:33And merge branch; I'll talk about that later on about how you can use that with preconditions...

34:39...do some of the same processing in a model.

34:42There's another one called parse path; you can give it any string and it will do the classic get the directory from it...

34:49...get the name from it, get the file from it, get the extension from it, if you ever wanted to do those things.

34:54A lot of times it's really useful.

34:57You get an input path, you grab the name off of it, and you use it to name outputs in another directory to do that.

35:04And then there's a tool called Stop. And really Stop, it's like While, and I'll do a demo of While a little bit later on...

35:12... the model will run.

35:14But the idea is, iterators, you can't combine them, so I can't do Iterate features and put a While in it.

35:21Those were our rules; we can't put more than one iterator in it.

35:23But, what I might want to do is, I want to iterate the first 10 features...

35:26...or I want to iterate the first 10 feature classes.

35:29How could I stop the model?

35:31So we added this tool called Stop so you can go in there.

35:34Basically, it runs the model until...the iterator will keep running until Stop becomes false.

35:40That you can build that into the model.

35:50So let's go to some other processing examples here.

36:00Okay, so I just want to do something a little more intensive...

36:05...than the ones before.

36:06We actually had this problem come to us.

36:10And the idea was, if you look at this, for each one of those, I want to do the buffer of the center point of polygons.

36:16In this case, I just grab the ZIP Codes of California.

36:20And you think okay, but I want the buffers to stop at the boundary of the polygons.

36:26And, how would you do that?

36:29So I made a model that does that, called a center buffer.

36:38Now just to real quickly look through what this does.

36:41What it does is it takes the polygons and it gets the points out of it, gets the centroid...

36:50...or, actually in this case, somewhere in, as close to the centroid as possible of it.

36:57And then we just do three buffers.

36:58Okay, great, I've got three buffers, I could run multiring buffer on that.

37:05But what I really want is, I want the annaluses.

37:07If I run multiring buffer and I dissolve, well, then, the buffers all dissolve into each other...

37:12...and that's not what you want in this case.

37:15Because they could overlap in the boundary.

37:18So, the way this works is, you buffer each of them, and then you identity back to the original ones.

37:27And without showing the individual pieces, what that does is that gives you...

37:31...so now I know for each of the buffers I get, I know it's SourceID, and I know what it overlaps...

37:36...and I need to make sure that the ring, basically the idea here is that my FID, the FID of the ZIP Codes are equal to...

37:46...the original FID, and I also put in here buffer distance equal to zero.

37:50So that gets rid of a lot of chaff that you don't want after you do the identities, because you're going to get...

37:56...a lot more polygons than you want after that.

38:01You can dissolve those, and then there's symmetric difference, how do you take rings and make...

38:06...annuluses when you do symmetric difference of.

38:09Like you take the two outer rings, and you run it, and you get annuluses out of that.

38:14And then you merge them all, all together like that.

38:16Okay. So that was actually kind of a little, an interesting model and it gives you this result.

38:23And if you run it on about a couple hundred features, it works.

38:28But if you try to run it on the state of California, it really bogs down.

38:36And what happens is that it bogs down in these dense areas.

38:40And this is actually one of the classic problems of overlay of identity, is, are these really dense overlapping circles.

38:48They generate lots and lots...if you've ever run into the problem is your bane of overlay...

38:53...it's just one of the things that causes really intensive overlays.

38:58So, the idea here is to say, well, how do we scale this up?

39:00How can we scale this model to be able to run for all the features in California, or all the features in the United States?

39:13So one way to actually...let me back up.

39:18Go back to this area here.

39:21So I actually made a little model, so you can think about...

39:26Well, let’s do it for every feature.

39:29For each feature, I'm going to...you could think of it just like if I was writing a cursor.

39:35If you're a programmer, you relate to that.

39:38Just basically think about this. For every feature I would get the point.

39:44I would buffer the point three times.

39:47I would use symmetric difference to get the annuleses.

39:51And then I would put them to the outer area and merge them back together and append it.

39:59So the idea is hey, I could take this model and I'm going to run it for every feature.

40:05And I've got this outer model, that what it does is, it creates the feature class, it adds the fields I want onto it...

40:16...and then it passes it into this inner model, and if you look at the inner model, it has a parameter on it...

40:25...which is the target, down here, which is what things will be appended to.

40:31So, for each of the features it will create the three annuluses and will clip them to the area and append it to that.

40:38The other thing I want to point out to this is to make this fast, or as fast as it can be, I use what's called...

40:47...in-memory feature, in-memory workspace.

40:51So it's a built-in workspace that we built into geoprocessing, so that will go to in-memory feature class.

40:59So, in this case it's good because I know it's small and I'm going to overwrite it a bunch of times.

41:04You've got to be a little careful about in-memory workspaces, but if I know that my data is going to be manageable...

41:09...I can use that, and in this case I'm going to be running it over and over again.

41:13It's really nice; hey, it's in memory, and it's faster.

41:16And I don't want to save it every time.

41:17It's just intermediate, I make it, I use it, and then it goes away to do that.

41:24In fact, I did that for all of the outputs here.

41:29So, really, all of these, it's just going to be a one-feature feature class, each of these are a little in-memory...

41:35...basically a one-memory record set, and then we append those values back to the original one.

41:44And then in fact actually what I did on the Create feature class I also did to in-memory, when you do that.

41:51So, I can see the value that's there.

41:54So I created a feature class, at the very end I just do Copy features to a shapefile to save it.

42:01So now, for the purposes of the demo, I'm not going to pick all of California because it takes too long.

42:08But I do want to show that you can take this, you can run it, and then for each of the features, it cycles through...

42:17...and it takes about, this one's actually taking about 1/8 of a second.

42:22Not an 1/8 of a second; about 8/10 of a second per feature.

42:26Which if you expand it out to California, it would take about 20 minutes.

42:31It would take longer than I want it to take.

42:33But it will cycle through, so you can just give it the features, give it complex processing, cycle through...

42:40...and append the values together.

42:48I always get worried that I didn't use the select set. But...

42:57Because it is running a long time.

42:58I might have to press, oh, no cancel button needed.

43:00All right, great.

43:02So it only did it for the selected features, and output those values.

43:07But I've already determined that, well I was thinking about, like, ok, well, so I can do that...

43:15...and it will work for everything.

43:19And if you look at the original model works if the features don't, aren't close to each other.

43:26Because, if I don't have lots of overlaps with the identities and the buffers, so how could I partition my data in such a way...

43:33...that it spreads the features out?

43:39Now, we've got a tool, so actually if you look at, here, there's a tool called Sort.

43:47You can sort by shape.

43:50So it takes all the features and it sorts them from, in this case, upper left to lower right.

43:56Okay, so that's what I want.

44:02So in this case I made 10 groups.

44:04And the way I did that, I did a sort, and then I grouped features in calc field, I just did mod of the FID.

44:14So basically I'm coming here, I'm going to do 10, so I'm going to take the FIDs, I'm going to make them into groups of 10.

44:19You could also do a random number.

44:21Random numbers are working in this case.

44:23You don't have to sort it if you do that.

44:26So now, I've got an attribute on my features with a group, and for each of those groups I want to run my model.

44:33So if I look at, I've got a model here called Main; it does the same thing.

44:39What it does it creates, it does Create feature class, adds a couple fields that I want, and then it does Iterate groups...

44:50...and then for each of the groups, in this case, it was iterate features, I'm using my group field.

44:59And then for each of those groups I'm going to call Center Buffer.

45:02It'll output the value and it will append it to the target.

45:09So if I go back to, in this case actually I can select a few more features.

45:18Go to my main...I'm going to double-check that I'm using the layer I actually...and run that.

45:35So in this case what it's doing is, I figured out 10 groups was the best number for all of the state of California.

45:43If I was using a smaller number, I'd probably use a smaller number of groups.

45:47Or actually might even succeed with all of those features.

45:50So the idea there is, I wanted to basically think about, how can I take... I've got a problem that ...

45:56...even if my original model didn't have to tile and create...in order to process all the different overlaps that are created...

46:08...for those results...let me turn those off...how can I...if I break them up it will actually run faster to do that.

46:17And in fact that will actually execute in about three minutes for the entire state of California, and actually I think...

46:24...it's going to scale out.

46:25I haven't done it for the United States yet, but I see no reason why it's not going to scale out.

46:29Think about it, I just wanted to share some, it's a little complicated iteration, and it's also ways, well...

46:38...one of the reasons to do iteration is to partition out your data.

46:42So, it's to do some, to break it up into pieces so that you have units that you want to do some processing on to do that.

46:56All right.

46:58Is there any particular questions on that one, that relate to that particularly in...before I move on.

47:04[inaudible audience question]

47:10The question is, will that particular one work for millions of...

47:12Actually, I think it will work for millions.

47:15You got to figure out what other groups are, and how do you want to group them up.

47:18It's going to take a while to run.

47:20Is it going to do it...

47:24You'd have to experiment with the pieces, and what makes sense of groups of features to run on.

47:31Right here?

47:32(inaudible audience participation)

47:41Right. Okay, so the question about the memory is, yeah actually it's the PC... yeah it's just a PC memory...

47:50...but you know, well, what's happening, the DLLs of all the programs you are loading there; ArcMap has a bunch of...

47:56If I go in, look at Task Manager and look for ArcMap in here it's going to tell me what the memory usage is.

48:00You have to make sure that you don't exceed...

48:08Well, if you do in-memory workspace, it's going to go into that memory, okay?

48:11So that's what it's using.

48:15We don't have checks in there that prevent the crash in the system.

48:22You put these things in there.

48:24It only works with feature classes and for tables.

48:27We don't have in-memory raster at the moment.

48:32But it works for small things.

48:33It's a really good way to make small things go fast.

48:35[inaudible audience question]

48:38Yeah, so the question is, is it available in 9.3.1? Yes, it is.

48:41[inaudible audience participation]

48:46The question is, if you export out to a Python script, it will still work.

48:49In-memory work, no, we do not export iterators to Python.

48:54What we do is we export one iteration.

48:59So, it will export as if it were a single model, of one runtime, and then you'd have to go to the model and use...

49:07...the various For loops and listing to do that yourself.

49:11We sort of drew the line there.

49:13We didn't sort of. We drew the line there of, ModelBuilder being a script tool generator.

49:19Okay, So. ModelBuilder is for people who don't do scripting to do work, ...

49:24...and we want to help people generate scripts.

49:26But for more complex things like that, you've got to learn scripting to do that.

49:35[inaudible audience participation]

49:42The question is, how do you automate your model?

49:45You actually don't have to convert it to a script to automate it.

49:49But you can write a Python script that calls your model.

49:52All right, so it's a one-line script.

49:54So, a lot of times I see that. That's actually a good question.

49:57Well, to automate it, number one, is you do need to make it into Python executable.

50:01Actually, if you go to the help and you search for Schedule Tool, you'll find it.

50:07We show you how to take a Python script and schedule it.

50:12And the best way to do that for, and you could just make a one-line Python script that calls your tool.

50:18In fact, that's what I recommend that you do.

50:21Okay, so I'm going to move on to the next presentation...the next sample...

50:30In this case, I want to do While, so I want to do a conditional, a conditional model.

50:36And I've got this one...here the idea is, given...so I'm just going to come in here and I'm going to select one feature...

50:46...and I want to run this...what this model does is, let's just run it real fast.

50:53What it does is it selects neighbors until the number of the population exceeds some maximum value.

51:01So, I got the input feature class, then I do...select by...I do Select By Location.

51:11If you look at Select By Location, you say input features and you say "share a line with," that's basically...

51:18...select on itself.

51:20So it starts with the polygons, and it selects all the polygons that touch them.

51:26And I want that to grow out, in this case.

51:28And I'm also, I'm adding to the selection each time.

51:32So the next thing is I want to know, what is the population?

51:35So I just do Summary Statistics in that case, and I'm summing up the population value.

51:42So now I've got a number in a table.

51:43Remember that Model Only tool that I told you about, called Get Field Value.

51:49I just give it the table.

51:51I give it the field I want to get; and now I've got a simple variable that has that number in it in the model.

51:56And since I ran it the last time, it was, it was 26,000.

52:01So I want this to run until this number is greater than that number.

52:06Well, how do I do that?

52:08Right, so in here I have 10,000, and I can use Calculate Value.

52:12And that Model Only Tool.

52:14It's a little utility tool...I call it calculate value...it's like the Swiss Army knife of ModelBuilder.

52:21You can do lots of interesting things with it.

52:23In this case, all I'm doing is "sum is less than maximum."

52:28A real simple equation, and I want the output to be a Boolean.

52:31True or False.

52:34So if sum is less, it's true, you keep going; if it's greater than, it's false, you stop.

52:40And so now here's the iterator While, it takes the input, this output value...

52:47...I probably should have given it a better name, but it takes as input what gets generated there.

52:51So as long as this is true, the model will keep running.

52:54Once it's false the model will stop. That's the While in there.

52:59And then I have calc value.

53:00I just put in there n+1, so I just give you the number of iterations that that model ran.

53:08So then for each of...if I select a polygon and I run that, it expands it out.

53:20So actually I'm going to come back to this a little bit later and do some things.

53:23I'm going to make this into a tool and say, hey, well, what if I want to give it a point and expand that out?

53:29But I'm going to stop here and talk about a couple of concepts first.

53:43Okay. So now I want to sort of jump to, so now I've got... I've made my model and I want to be able to use it...

53:51...in another model, or I want to call it from a script.

53:55I want to basically make that [unintelligible] encapsulate some interesting processing.

53:59I don't want to copy/paste it every time.

54:00I don't want to copy/paste 10, 20, 30 things into another model when I do the exact same thing.

54:07So how can I make that a unit of processing that I can use in other tools.

54:13So the idea is, you can take var...what you do is you take variables, and in fact, you noticed...

54:20...that there's this little p on here next to that variable block group.

54:26It says, is it a model variable, or is it not a model variable?

54:30It's that little p that comes up.

54:32And what will happen is that, if I double-click this and use it as a tool, basically everything that's a...

54:36...model parameter will show up in the dialog.

54:40So you control what are the inputs and what are the outputs that are exposed.

54:44In this case, I'm exposing the block groups.

54:49And I'm outputting the block groups, but that doesn't show up, and I'm also making a parameter of...

54:54...maximum to do that.

54:58The variable equals the data type, the data type of the parameter equals the data type of a variable.

55:03You can change from optional to required, you could specify a filter, and you could specify output symbology.

55:10So, in model properties, there's this property page called for parameters, it gives you the name and the data type.

55:18For things that are optional you can switch it to required, and then there's also a filter here that you can go in there...

55:25...depending if it's a feature class, or it will let you do a feature class filter; if it's a string, it'll let you do a string filter...

55:30...if it's a number, it'll let you do a number filter.

55:32So only certain data types let you put a filter value in there.

55:36But if it's appropriate, it will let you edit the filter for that.

55:41So, if I have a model that inputs a feature class, I only want it to put in points...

55:47...then I can modify the filter to do that.

55:54The other thing that control is when I want the model to run, at the very end I want it to have it to have nice symbology.

56:01And if you go to, so in this case my output, well, in this case my output are theselected features.

56:17I don't have it saved.

56:18But you go to Properties, you go to Layer Symbology, and you pick a layer file.

56:23And a layer file becomes a template that gets used when the layer is made for that feature class.

56:29We have, it can be static, where it just always uses the same symbology.

56:37It can also support dynamic.

56:38You have dynamic, unique value and dynamic range to do that.

56:43And go look at the, there's a section in the Help that talks about setting up those layers and saving them out...

56:51...and what properties on the renderer make it dynamic or not.

57:03So it's classified; if it's not manual, it will redo the classification.

57:07Unique values, if all other values are false, it will recompute unique values each time a layer is generated...

57:14...for the output there.

57:18So let's go back to this example.

57:22Well, that was kind of neat, but everytime I had to come in here and I gotta select another one...

57:27...and run it, and select another one, and run it...

57:30So what I want to do is I want to kind of make this so that I can enter a point and it runs it for that area.

57:38So, I've got Main here; what I did was say that I take, well, let's just start from scratch here.

57:51And inside here, a new model.

57:55And what I can do is, I can do Select Number of Customers.

58:01And it's in there, and it takes as the input is blocked groups.

58:04And before, it starts out with a selected set of the blocks, and then what I want to do is, well before I run it...

58:11...I want to select the area on that.

58:14So, what I did was, so I have this, the finished model here.

58:19If you look at it, it does Select by Location with a Point.

58:27Let me make that bigger.

58:30Okay, it does Select by Location with a Point.

58:32So then that's the feature that will start, and then this will run until some maximum customer value that we put in there.

58:41So now, I can run this, if I run Main.

58:45And in fact I'm going to just pick one point, and I want to up this number a little bit.

58:53And I want it to run to 50,000, and then I can...oops, nope, sorry.

59:06And I can zoom to selected.

59:10Oh!! You idiot you! Okay.

59:13Sorry about that.

59:14All right. Well, you know what I want to do?

59:16I want to write a model that does zoom to selected for me. Forget this.

59:24So, actually, there's a couple of things I want to do here.

59:27Actually, here's what I want to do.

59:29I want to run this model; I want to pick a point.

59:33And then, it will expand on that area and then zoom to selected on it.

59:38All right, well how do you do that?

59:39Okay. Did you like that? I kind of liked that.

59:47I liked that so much I'm going to do it again.

59:53All right. Uh-oh. Now that's where you cancel. Yeah, don't hit those islands.

1:00:02See the Cancel works.

1:00:08Okay. So, the idea here is, so, how did I do that.

1:00:12So, remember before that the, this input here, the point was a point feature class.

1:00:20But if I look at this, I can do...if I look at the data type on this...and I go to data type you'll see that it's a feature set.

1:00:29And so, what I do, I do Create Variable, pick feature set, okay, click on the properties of that...

1:00:39...and what I do is I pick a layer file or feature class that is the template for that.

1:00:48So now if I open this up you'll see, well, I get this interactive input thing.

1:00:53And in ModelBuilder it's blocked, you can't use it inside of ModelBuilder, but what you do is...

1:00:58...you make it a parameter, and now that's the parameter that we use.

1:01:02So, basically the thing is for interactive input - point, line, or poly, it doesn't matter what type - you can use feature set for that.

1:01:09You draw it and you do that.

1:01:11So now I enter the point and I do Select By Location, I run the submodel that grows until it's done...

1:01:19...and then actually there's a...and then I added on to this little script called Zoom To Selected.

1:01:25So, now if you're not, you may have heard, one of the new things in version 10 for geoprocessing...

1:01:34...is what we call ArcPy mapping.

1:01:35Anyway, the mapping module of ArcPy.

1:01:38So, a lot of people have asked for, hey I want to open a map document, I want to turn layers on and off...

1:01:43...I want to add layers, remove layers, change extent, and I want to print it.

1:01:46Okay, I want to do things to maps.

1:01:49And so there's a little scripting language for doing that; in this case, you open up the current map document...

1:01:58...you get the data frame, and in fact you get the layer and off of the layer, you get the selected extent.

1:02:05And, at the end of the class I've got a listing about when this class is, if you're interest specifically in ArcPy mapping.

1:02:15So what that does is zoom to selected.

1:02:17So, in my main model I select one point, I feed it to my submodel that does iteration.

1:02:26And then when I'm done, I just do Zoom To Selected on that.

1:02:35All right. Any questions about that?

1:02:38It's kind of a lot to throw at you at once.

1:02:43All right.

1:02:50How do you make things interactive? You can use Feature Set.

1:02:54Okay. So the last thing I want to talk about...how are we doing on time here?

1:03:01Are we, we're 11:30. Okay.

1:03:06So a little flow control.

1:03:08So we have these things, we call them preconditions.

1:03:11The idea say, there's two problems that it solves.

1:03:16One is that sometimes I just want one tool to run before another tool.

1:03:20And ModelBuilder has no ordering built into it.

1:03:23There's no way to say...you know, basically ordering is built in by the chain.

1:03:29If one thing is subordinate to another, it will run first.

1:03:32But if two things are in separate chains, it's not random what is...basically, the one that's entered first will run.

1:03:40You can't look at the model and know which one was entered first.

1:03:43You can't look at it and change the order that things are running.

1:03:46So the idea is, oh, basically I made it a pseudo input.

1:03:49We have these things called Preconditions, I can connect, in this case I connect the output of Create Feature Dataset...

1:03:55...as a precondition to Clip, and now basically it becomes a...well, it becomes an input.

1:04:03It's not a parameter input, it's just a processing input.

1:04:06Basically it says, this needs to run before that.

1:04:09So if you need to ever enforce that, you can do that with Precondition.

1:04:13So preconditions can also be used for if-then branches in a model.

1:04:18So this is, the idea here is a tool...it's the upgrade metadata tool.

1:04:25And depending on what metadata type you pick, it does this processing, and if you do the other type...

1:04:31...it does the lower processing.

1:04:32So, if you can see then there's essentially two branches that are going on, so there's some condition checking here...

1:04:39...it does the processing, and then it brings it back together.

1:04:43The way it does that is, basically you do upgrade type; you do upgrade type Esri ISO, or you do upgrade types FGDC.

1:04:51Just a simple string.

1:04:53So if that string says Esri ISO, this chain runs, so this is a true/false value...

1:04:58...that's used as a precondition here, and here's a true-false value that's used as a precondition there.

1:05:03So that's how I start the branch, you break it out, and then at the very end we have merge branch.

1:05:09So let's actually go take a look at that live and see what that looks like.

1:05:16So in the conversion tools, inside a metadata, there's upgrade metadata...

1:05:24There's actually lots of real interesting things in here.

1:05:27A lot of these techniques we've been talking about are part of this model here.

1:05:32So if I go look at model properties, go to Parameter, go to Update Type, there's a filter on there.

1:05:40So that controls what you input. So now I get a choice list of what are the inputs for that.

1:05:45You don't just type in whatever you want.

1:05:50Calculate upgrade.

1:05:51In this case, this is that one that has the, it goes and gets the style sheet based on the type that you input there.

1:06:01And then one of the calculate value here, if you look at that, basically it says upgrade type equals Esri ISO.

1:06:09The two choices that you put in the choice first, and it outputs a Boolean.

1:06:14And the way that works, so if we come in here, and if I look at the properties on here, and I go to precondition...

1:06:20...you'll see that Esri ISO is a precondition of this running.

1:06:25So, if Esri ISO true, it will run; if Esri ISO is false, it will not run.

1:06:31You could put multiple things in here.

1:06:32If any one of them is false, that will not run.

1:06:36So that's the way you break it up.

1:06:42So that's how I set up; in this case, ISO versus FGDC, it does whatever the appropriate processing is.

1:06:49Now at the very end, I want this model, I only want a single output, right, so I've got these two chains, right.

1:06:56I've got the ISO result and the FGDC result, but I only want one output of this, and we have a tool called Merge Branch.

1:07:03It takes those two inputs, and what it does is it looks at which one's been validated, which one has actually been run.

1:07:09If this has been run, it passes it through; if this has been run, it passes it through.

1:07:13So, if they've been run it passes the first one through.

1:07:16So, you know, the idea is set your model up so that only one of them succeeds.

1:07:21Only one of the branches will run when it does that.

1:07:35Okay. So, yeah, I mentioned for the Python, the unlisted session.

1:07:40It's in the online session but it's not in your little printed thing, so we're at Python Scripting for Map Automation...

1:07:46...that's Tuesday at 1:30 and Friday at 9:00.

1:07:49If you want to see it, please fill out the forms.

1:07:53And, any other questions you have?

1:07:54Open to questions here right now.

1:07:59Way back in the corner there.

1:08:04I'm sorry I can't hear you.

1:08:05[inaudible audience participation]