Real-world example of the HTML5 FileSystem API

A quick note about the code in this article before we begin: the code in this article is not meant to be typed in piece by piece. Instead, download the zip and extract it to a local web server. You cannot run it by double-clicking on the file. The code must run via a working web server, although that server can certainly be your local development web server.

One of the more powerful features coming soon to a browser near you is the ability to read and write to the local file system. You can find the File API: Directories and System working draft on the W3C website. Dubbed the FileSystem API, it provides the browser a safe little sandbox where data (both textual and binary) can be read and written. Many excellent articles out there detail the API. (I'd start with the excellent one from HTML5Rocks.com: Exploring the FileSystem APIs.) This article assumes you have at least a familiarity with the basics. With that assumption, we're going to look at a real world scenario where browser file system access can be useful – the local caching of media resources.

Imagine for a moment you are creating a hip, new web property. You want to make use of several high quality images or sound files. Imagine if you could package up all these resources into one zip file, send it to the browser, and have it store a copy locally? This is not necessarily for offline access, but as a way to minimize remote network calls and simply offload some of the "weight" to the client. To be even more efficient, you also want to track the date the zip file was last updated. You can then use a lightweight network request to see if it has been updated before you go through the work of processing it again. Let's get started!

As I described above, this article isn't meant to introduce you to the FileSystem API. My hope is that you read the related resources I mentioned. But if you're anything like me, you probably skipped that step. We all do. One of the first things the demo code does is determine if it can even use the FileSystem API feature. Right now the API is fraught with vendor prefixes. In the future, the API needs to be generalized, but here, we start off caring only about Chrome. Notice a variety of vendor prefixes in the code, like the following example:

(In case you're curious, the init function is being run via a body/onload call. The full template is below.) The initialization routine begins by checking for the existence of webkitStorageInfo. Because this demo is only concerned with demonstrating File System API stuff, we can immediately quit if it isn't supported.

The File System API differentiates between a temporary and persistent file system. Their very names indicate when you would use one over the other. For this application, choose a persistent file system so you can store your resources until they have to be updated. To work with the persistent file system, you request access from the user. This is done via a JavaScript function, but the actual prompt is handled by the browser, much like geolocation. The following code block demonstrates how to request the storage and what will be done after it has been approved (or denied):

Note that you request a size, but it's possible the size given may be smaller. Don't even worry about what you're given for now. In the future, you may want to record this ( localStorage ) and ensure you stay within your quota. But the important thing to note here is that once you've been approved a bucket of space, you can request the actual file system.

Here's an example of what the user sees using the latest Chrome. This UI may change in the future.

The call there, webkitRequestFileSystem , essentially returns a pointer for all future read/write file and directory options. It's success handler, in this case onInitFs , is run once you're good to go. Finally, our errorHandler is run if anything goes wrong. Take a quick look at that before going on:

The above code was taken (and slightly modified) from the HTML5 Rocks article. It's just for testing and doesn't actually present any nice response to the user. It only uses the console to report errors. Be sure you do your testing with the console open. (Don't all good JavaScript developers do that?).

Working with the File System

So at this point, you've established that your browser supports a file system. You've requested storage from the user. And you've asked for a pointer to the file system. After all of that, onInitFs is finally run.

It's probably a good idea to refresh to clarify the goal at this point. The goal is to download a zip file, extract the contents, and store it on the local file system. To enable that, begin by defining a folder where your files are stored. Call this variable resourceDIRLOC :

There isn't anything special about this name, but you want a subdirectory to add more stuff in the future, and not have to worry about organization. Even though this is a sandbox separated from the rest of the file system, it's important to think of this as any other file system. You don't want to make mess. Both for your user's sake and for your own sanity.

First, open up this directory. The API allows you to open a directory that doesn't exist. We do this by passing a create flag. You can only do this for one level of directories at a time. So for example, you can't try to open /resources/images/highres and have the API simply create all those nested folders. In a case like that, you need to create each subdirectory one at a time. Luckily, this example has a simplier target:

Copy the file system handle, fs , to a globally-scoped variable. You need fs later so it's best to copy it right away. Next, call to get the directory. Notice the path is based on one of the properties of the file system handle – root.fullPath . The root object is a directory pointer to the path of our sandbox. The fullPath is simply that – the actual directory path. Combining that with a separator (and note – you can use / whether or not you are on Windows) and the resource directory name, you then have a complete path to the folder to use. The create flag handles the first-time creation. All calls to the file system API are asynchronous, so begin a callback function in the last argument. Finally, the very first thing we do in the callback is cache a pointer to the new directory. resourceDIR is a global variable to use again later.

Now for the interesting part: the zip file you download is pretty large. You only want to download it the first time, and after that, only if it's been modified. To remember the modification date, use localStorage to cache it. Consider the next block:

The first portion of the code block above executes if you have a value for when the zip was last modified. (Soon you will see where to set that.) If resourceLastModified exists, you create a HEAD -only Ajax request. This is a light-weight network call that just returns the headers of the remote resource. We check the Last-Modified header. If it is different in anyway, we need to re-get our zip file. That's done in the fetchResource() call. Finally, you see the else block simply runs fetchResource() .

Getting and Processing the Zip File

Let's take a look at the fetchResource() method. It's responsible for getting the remote zip file, unzipping it, and saving it to the file system. JavaScript doesn't have native support for working with zip files. I used the simple, yet powerful, zip.js library written by Gildas Lormeau. You can find the zip.js library on GitHub. Note that you only need the files zip.js and deflate.js .

Let's begin looking at fetchResource :

The code above shows the portions of the function that handle the Ajax request. For now, the onload is empty, because it's a bit complex. Note a few things. First, the response type is arraybuffer . You need this to process the binary data from the zip. Secondly, the resourceURL is simply a static url defined earlier in our code:

Now dig into the code run when the request is done:

The very first thing you do is cache the date the zip was modified. LocalStorage makes this incredibly easy to use. Make note of that key there, resourceLastModified . You can test the code multiple times. Either build new zips and update their last modified value via the command line, or simply use your browser's console to delete the value.

Next, prepare the binary data before handing it off to the zip library. This is a multi-step process that involves a "Builder" sourced by the raw response and then the actual Blob object created by specifying our particular MIME type for our data. The end result, though, is zip binary data. Now, parse the zip:

The code above is probably a bit confusing as you have callbacks calling callbacks. In a nut shell, you begin by creating an instance of a zip reader. This is based on the zip.js API. One of the many ways to initialize the zip reader instance is by passing in our blob object. You then provide a callback to handle the reader. Within that, you call getEntries on the reader. This allows you to enumerate over each item in the zip file.

This is the point where you begin writing data to the file system. Remember resourceDIR ? It's just a pointer to our directory. Use it to create files within it by calling getFile . Pass in a name, based on the zip file entry name. So, if the first entry in our zip is foo.jpg , entry.filename is foo.jpg . getFile opens the file on the file system. Within the success handler, you then use entry , which is the file in the zip, and suck the data out with getData . That was probably even more confusing. Essentially, you open a file on the file system and siphon out the bits from the zip file entry into the file you opened. The first argument to getData is a file writer. That handles the actual bits. Two empty callbacks in there could optionally monitor the progress. But since this is a relatively simply process (again, sucking the bits from one thing to another), you can leave them alone for now.

And that – as they say – is it. In order to test, I first made use of an excellent Chrome plugin called Peephole. Peephole is an extension that lets you browse the file system associated with a website.

The files listed in the figure, above, are all images from the zip file. I also built a simple function that renders a few of these images:

After the user clicks on a button, I loop over an array of file names and see if they exist in the file system. If so, I simply add an image to the DOM. Note the use of file.toURL() . I use this call to get a reference to the image that I can then reference from HTML.

I hope this article gives you an idea of what could be done with the file system. While support is still somewhat limited, the benefits of being able to store resources locally make it more than worthwhile even if the API is a work-in-progress. Keep your eye on the File API W3C working draft for progress.