00:01We'd like to welcome Brendan. I think this is his third year of being selected to the user presentation track.

00:06If you don't know what that is, these Mesquite rooms all week are mostly filled with presentations from the user community.

00:13You can submit them, and we don't pick them, you do.

00:16So users submit, users vote, and then you build your track with the content that you want to see.

00:21We've been doing this for the past few years, and it's been very successful.

00:24At least Brendan thinks it's successful, because he keeps gets voted in.

00:28So without further ado, I'll turn it over to Brendan, and thanks.

00:32Hi, everybody. Can people hear me okay? It's not too big of a room.

00:37Thank you guys for coming to this presentation, and thanks to Jim and Esri for kind of giving me the opportunity to talk.

00:44I'm going to be talking about developing online/offline applications, specifically taking an existing Flex application...

00:52...and converting it to work completely offline.

00:55It's a hot topic right now.

00:56We saw in the plenary continually talking about tile packages, about being able to function in a disconnected environment...

01:03...and this is some of the work that we've done over the past couple of months working with the same topic.

01:11So the work was done for the World Resource Institute, and we helped them tell their story about forest management...

01:19...in Central Africa.

01:20So we developed a Flex application on the web, which was highly configurable and let us choose different countries...

01:26...to highlight their forest management situations.

01:30So this is the Republic of Congo, and this was our first candidate country for bringing offline.

01:37I'm going to break out of the PowerPoint here for a second and just show you that was the web version.

01:43And so this I pinned to my taskbar.

01:46I have the offline version, and so let me just make sure I'm completely off.

01:52I have no plugs. My radio is off.

01:57So this is a very similar user interface to what we had.

02:02It's actually the exact same code.

02:04We have our tiles functioning.

02:07We're able to identify different areas.

02:11Some of this data is...the type of data we're looking at...we're looking at different forest concessions for logging...

02:18...data pertaining to mining permits, the nationality of the primary investors for a lot of these logging areas...

02:26...logging roads, sawmills.

02:29You can imagine that a lot of this data is pertinent to people on the ground in the Republic of Congo.

02:34Sitting on the web, we've extended the reach of this data, right, from off someone's hard drive...

02:39...but it's not getting to a lot of people in country, and so that's obviously where the offline part comes into play.

02:46And so, just continuing here, here's just some screen shots.

02:52But the real tool that got us there, that got this functioning was AIR, right, because we were already committed to Flex.

02:58We had a great code base for a configurable app, and simply changing the, instead of application, to windowed application...

03:06...suddenly we were able to go offline.

03:08Now, we still had a lot of dependencies on the web, right.

03:13Just changing your tag doesn't mean that you don't need the Internet anymore.

03:17So in kind of considering how we were going to bring this offline, we decided which features to support.

03:24There's some features you might not want to bring offline.

03:27Also, figuring out where all the dependencies we had were and how to capture them and store them...

03:33...and then how to extend the ArcGIS Server Flex API to look locally to the machine.

03:40So I use the word "remote dependency."

03:43I kind of define that as any asset that's not locally accessible and that is required for an application feature...

03:51...and just by this kind of definition, it gives you a hint of how to deal with remote dependencies.

03:57Either you make them local, or you just disable the feature altogether, which is another valid way to bring something offline.

04:04Simply...example, Twitter and sharing in Facebook. You could set up a queue of shares, and so when they go back online...

04:14...they're sent out, but in this case, we decided just to disable those sorts of features to get something out quickly.

04:21In identifying where these remote dependencies were, an HTTP proxy, like Charles or Firebug is awesome, right.

04:28You can just be using your online app and see all the calls that it's making...

04:33...and then understand that you're going to have to replicate all those calls locally.

04:37And it's very important to understand the nature of the dependency, whether it is dynamically generated at runtime...

04:46...or if it's sitting statically on disk like a tile.

04:50Dynamic map requests--pretty difficult to start doing offline.

04:55You probably just basically have to write ArcIMS in ActionScript, if you want to do that offline, [inaudible].

05:01So we focus on tiles and geometry, right.

05:06Those are easy things to bring offline.

05:10So what was our strategy for storing these assets?

05:13Well, we wanted a really simple data model.

05:16This is all implemented in a real project that had a real deadline, and the deadline was short.

05:20And so keeping track of the asset URLs was the key part.

05:27We'd take a URL that was going on the web, and we'd look up that asset locally on the machine.

05:33That was our mechanism for finding things that were usually on the web.

05:39We did not normalize any of the response data that would come back.

05:42So, for instance, you'd get a JSON response from a query.

05:46We didn't then break that out into fields of data.

05:49We just simply stored that response as if it were coming, you know, just raw from the web.

05:56And so in first looking at how to store these things, we looked at the local file system, right.

06:03AIR has an awesome file API.

06:05It's easy to see your documents directory, whether a file exists or not, and that was our first approach.

06:11Very easy to implement.

06:14If you start storing map tiles just as flat files in your AIR app...

06:18...they'll quickly become a little unmanageable as the number increases.

06:23But, as I mentioned before, storing the URL to where this asset came from was vitally important to us.

06:30And so our solution, right, how do you store the URL as just a flat file, was to name the file the URL. Right.

06:38We'd take a file offline, and as we were saving it down, the name of the file became the URL.

06:43But, you're going to run into a problem, right, with doing that is that you have a lot of slashes and periods...

06:48...and you don't want to name something.

06:50MD5, the cryptographic hash function, is a great way to encrypt the URL and name the file the hash.

07:00So that's what our approach was for dealing with flat files.

07:04Now, it's always a one-way ticket with MD5. You can't come back.

07:08So the application itself would simply hash any URL it was looking for.

07:12Take that hash, look on disk, and if it can find it, it would use it. Right.

07:15That was the approach for the local file system.

07:18But we grew out of the local file system into SQLite pretty quickly because of scaling...

07:24...and also packaging up these assets with the AIR installer.

07:28We were using 4.5 at that point.

07:30We didn't have the captive runtime.

07:33SQLite was real easy to use.

07:35And here's a kind of another tip that we used.

07:38We stored geometry in AMF format in a text field.

07:42If you're using AIR and ActionScript, there's no reason to have to parse JSON if you're storing it offline. Right.

07:47So you take you, AMF response coming back from ArcGIS Server, say a query.

07:51You encode it as Base64 text and you stick it in a text field.

07:55When it comes back out, you rehydrate, you decode from Base64, and you have your object just there.

08:00No For loop. Right. When that geometry comes out, it's just ready to go.

08:08Tile packages for our basemaps is definitely a great next step.

08:10This is a really exciting kind of new feature that will save a lot of time instead of having to go in SQLite and roll your own.

08:19Mansour released this TPKLayer on his blog recently. That was awesome, awesome layer and great code...

08:26...and I recommend that, you know, when you're getting into this that you check that out.

08:32That's going to come up a little later in a slide.

08:36But, tangent, Python--how's Python fit into this?

08:39Well, we didn't have a lot of time to write the data models for SQLite.

08:45So we decided to use my little pony, Django.

08:49I put that up there to show you how intimidating it is to get into Django.

08:54But, you know, the mantra of Python is that you have batteries included.

08:58You have all these libraries. You have MD5. You have the Base64 library.

09:01You have CSVs and parsers--really easy to set up your data model in using something outside of AIR.

09:10And so I found that somewhat liberating, step outside of Flex and Air for a second, get my data prepared, and then go back.

09:17So we were using these Django management commands for different scraping routines to go and...

09:24...get data from ArcGIS Server and then the object relational mapper inside of Django to create our tables...

09:31...and write our SQL for us.

09:32So that was really helpful not to write SQL by hand.

09:37Okay, so I'm going to change a little bit to just some task-oriented slides...

09:42...talking about some things you're going to encounter as you develop offline and roll your own offline solution.

09:49So the first...are you still connected to the Internet?

09:52We want a seamless application.

09:54When our application goes off, we don't want errors, we just want it to work.

09:58When it comes back on, we want it to get new data and download that data and be able to use that data offline.

10:06The key class is the URL monitor class in AIR, right.

10:09This is the bread and butter of monitoring.

10:14And it's real easy to set up.

10:15Now these functions are wrapped in a larger class that's a connectivity util, right...

10:19...that's abstracted from the rest of the application, and you can simply bind to this is connected property...

10:26...and know whether you're online or offline.

10:29Some tips kind of in doing this is you can use...it takes a URL request and it will continually ping that URL request...

10:36...and know whether it's online or offline.

10:39When doing that, you don't actually need to use a get request or a post.

10:44You can simply use a HEAD request.

10:46Are people familiar here with a HEAD request?

10:49It's basically bringing back only the header of the call instead of the body.

10:54You don't need the body.

10:55If you get the header back then, you know, you're good to go.

10:59And then when you get your status change event, you'll know to handle it accordingly.

11:06So kind of the next situation that we were dealing with is accessing our data from SQLite.

11:15The API is not made to access SQLite.

11:18And so if you're finding graphics, for instance, are a pretty simple thing to get out.

11:25Tiles can be different. That's harder.

11:29When thinking about databases, a great...knowing whether you're going to connect asynchronously or synchronously...

11:37...is very important. Right.

11:40Most of the time you want to be connecting asynchronously.

11:43If you're doing it synchronously, then you should probably have a reason why in your head that you're connecting synchronously.

11:49And the asynchronous process won't use the main UI thread, and so your application is not going to freeze...

11:56...while you're getting data or you're running a long query.

11:59And that's important in Flex. That will really help the user experience.

12:02You can see it's real straightforward, and you can tell it's synchronous, right...

12:06But, there are some scenarios where you do want to be connecting synchronously to your data.

12:12One of them is fetching tiles, and we're going to take a look at that in a sec.

12:16But, the UI does freeze while you're grabbing data if you use a synchronous connection.

12:26...because, look at, I already have my data in this same function.

12:29So I have this...I'm parsing the get result of the statement...

12:34...but it's just right there instead of having to actually add a listener and then handle that.

12:39And so in the actual result, it's doing the exact same thing as the synchronous is just doing right in line.

12:49Okay. So this is one of the beasts was retrofitting the tile map service layer to run off local tiles instead of off of tiles...

13:02...say in ArcGIS Online or tiles that we made ourselves.

13:06So the function that you override, how many people have seen this getTileURL function?

13:13One. Got Steve in the back there.

13:18This is the best function of the tile map service layer, right.

13:22This is the key that Esri has given us to be able to customize our tile layers.

13:28A common scenario where you'd override this would be when you put tiles on S3, right...

13:33...and you want to manipulate the URL path that comes back to an individual tile.

13:39So you see the arguments are the, you know, the x, y, and z of the tile, the level row and column...

13:48...and then it returns URL requests.

13:50So when we start storing tiles just simply in the file system, this was no problem.

13:56We would hash, do the MD5 hash, look up the tile, put the URL of that hash right in the URL request and return that request.

14:07Worked fabulously.

14:08When we moved to SQLite, we ran into an issue which was that a URL request cannot make a direct connection to SQLite. Right.

14:22We were needing to get the data out of SQLite and get it into that request some way.

14:28And the only way that I came up with before Mansour came to my rescue, was to actually manually copy synchronously...

14:36...the tile onto disk each time and then give the path to the tile on disk back to the URL request.

14:45Now that worked. It worked, and it was not bad while running on the local machine on USB drives.

14:53We really wanted to just deploy this off of USB drives.

14:56It was slow, and you could feel it.

14:58You could feel the write happening for each tile, and then that synchronous grab and the UI freezing.

15:05And so this is the solution, and this is marvelous solution that the Flex team came up with, and it's in there...

15:17...the TPKLayer, the tile package layer, that Mansour put out.

15:20Let's look at this just a little bit.

15:24First, so I've abstracted the active going to SQLite and getting the actual ByteArray of that tile.

15:31But what comes back is this tile bytes, and we were simply putting the tile bytes directly into the URL request data.

15:41There's a little snippet just to tell, I guess, the next function that's going to hit this that we don't, you know...

15:48...are not privy to what's going on the level, row, and column so it can place the tile.

15:53But this was something that I would have never dreamed up as something that would be possible...

16:00...which works just fine now with the 2.5 API.

16:03It will not work in 2.4, but 2.5 and further.

16:07And so, thank you, Mansour, and the Esri team for providing that and seeing that issue of accessing tiles.

16:15They're using it for the tile package, for accessing the tile package, but you can use it for SQLite, too. It's awesome.

16:28And we've already...we're favoring the offline content.

16:32See, we're returning the request once we find any content offline.

16:38But if we haven't found anything and we're connected, we're going to go out and we're going to get it...

16:41...and we're going to store it. We're going to have it for next time.

16:43So this is a great scenario for when, you know, the team is in country in Congo.

16:50They have an Internet connection. They're going to one specific place.

16:52They want to drill down to it to way, you know, level 17 at one small location.

16:59They're able to grab the content while they have...while they are online and use it while they're offline.

17:04So that was a good thing.

17:09So just quickly, I'm going to wrap up and kind of have some time just to have a discussion about this theme and Flex in general.

17:17But your deployment options.

17:20At the Flex SDK 4.6 and AIR has captive runtimes, which are amazing, which are really, really great...

17:29...because it means that the end user doesn't have to install AIR on their machine.

17:34It also does not have the AIR installer. There is no install process.

17:39Just like a program...how many people have used like PuTTY or something that's just simply an .ext...

17:46That's exactly what this is.

17:48You can put it on a disk or a thumb drive and run it directly off that thumb drive, which is amazing.

17:54So on Windows you get an .exe. On Mac, you know, you get the .dmg.

17:59You are limited to either one of these, depending on your environment.

18:03If you're running Flash Builder on Windows then you're not going to be able to compile the .dmg.

18:09I was kind of bummed when I saw that, but, you know, that's okay, too.

18:15There's also the .air file. Right. Still great.

18:19It works across a lot of platforms, and if you don't know what platform your users are going to be on...

18:24...then that might be a good option.

18:26But, it also means that they have to install the AIR Runtime.

18:30I did include the little Linux penguin down there, but I believe at 2.7 they're losing support for Linux.

18:38But, earlier, if that is a demand, you can use, you know, an earlier SDK so you can support Linux users also.

18:48We've been talking about desktop, but this has a lot of bearing on mobile also.

18:52I've reused the, you know, the URL monitoring, the offline tile layer, all that stuff, for both iOS and Android applications.

19:01And it works just fine.

19:03With these devices getting faster, the Flex API is performing really well on the newer devices.

19:09I've seen it running on the iPad 2 and 3, and it's awesome. I really can't speak, speak [inaudible].

19:19But that is my presentation. I've been doing...my last presentation was a night talk...

19:24...and definitely the time frame of those can mess you up on your next one.

19:30Does anyone have any questions or comments?