JavaScript is the next C

I've been coding in JavaScript for almost a year now. I find it a decent language to write code in. The code written in JavaScript doesn't look as pretty as python, but it doesn't look as bloated as Java either. But that's not why I chose the title of this post. It's everything else that's happening around JavaScript that makes me think that it's going to play as significant a role in computing as C has played.

Before C, there was assembly. (I know there were many other languages before programming evolved from assembly to C, but I am talking about only the mainstream candidates.) The abstraction that C provided was perfect enough that programmers didn't have to learn about the Assembly details. That made C the perfect candidate to build a solid layer on top of Assembly. Programmers could write accounting software, graphic programs, games without bothering about what CPU architecture and memory bus size they were running on. It all worked and C became an integral part of computing stack. Today we don't use C to write new programs, but all operating systems, their device drivers, native libraries are written in C. C has become so ubiquitous that it's now invisible.

We have spent a long time searching for a candidate language to build the next layer, on top of the one built by C. C made our code hardware-agnostic. But over time we developed a variety of operating system platforms that created their own stacks of libraries. With the advent of Internet we needed to transfer code over the wire and be sure that it can be run on all the machines irrespective of which platform they were running. We needed a write-once-run-everywhere solution. Java emerged as the solution to specifically fill that need. For a period of time, it seemed it would indeed be the one. But for several reasons it failed. Today when we are deciding a platform independent solution to write a GUI program, what number does Swing score in our preferred list? Java did a great job of freeing the programmers from worries of memory management. But the main reason for its failure is probably its awkward ownership by a single commercial entity Sun (and now Oracle). The lawsuit Google is facing over Android is enough to corroborate this. If someone can sue you for using their language, then how can such language be adopted by entire industry.

The latest candidate to build the next abstraction layer in the computing stack is JavaScript. It's hardly a new language. It's hardly a perfect language. But there are two technologies that will make JavaScript the next C - HTML5 and Node.js.

HTML5 (and whatever goes under that umbrella name) is a new web framework whose primary programming language is JavaScript. It is bound to succeed, in fact if you consider it as just a new fancy name for HTML then it has already succeeded. It's not developed by a single company, but many big guns are simultaneously promoting it  - Google, Mozilla, Apple, Microsoft (by accepting most of the standard for IE9), are writing virtual machines that keep improving the speed of JavaScript. The new age behemoths - Twitter and Facebook - have strengthened HTML5 merely by adopting it.

On the other hand, Node.js has become a great success. It has a teeming community around it. The success of a platform depends on the libraries its provides to do various tasks. Just look at this wiki page that lists different Node.js modules to accomplish various tasks. I spent last week writing a factory server for 3DTin using Node.js. Everything I needed - from binary encoding library to canvas rendering libraries - I got it from this page. And there are more than one option for each job. Most of the libraries are nascent and will mature over coming few months. But they are a strong sign of a solid platform taking shape.

Yet another sign of JavaScript's growing might is, its choice as a target language for compilers of other languages. There are projects compiling Ruby, Python, Java right into JavaScript. The shortcomings of JavaScript as a language are being fixed by many frameworks successfully, think of jQuery. Libraries like Underscore.js or projects like CoffeeScript are making programming in JavaScript more fun.

Between HTML5 and Node.js you can now be sure that if you write your next application in JavaScript it will run on any server, desktop or mobile. JavaScript will build that next layer in computing stack that we are waiting for and that's why it will be the next C - ubiquitous and then invisible.

HTML5 offline cache - use swapCache carefully

HTML5 Offline caching is an incredibly useful feature that allows your users to load your website even when they are not connected to the Internet. When they visit your website first, the browser checks if your webpage mentions a cache manifest file in its html tag. If so, the browser downloads the manifest file and then downloads all the resources mentioned in that file. You are supposed to add all the resources that your webpage needs in the manifest file, so that when user is offline the browser can load the webpage from the cached version of those resources. You will find many articles that will help you implement offline caching for your website- like this one. This post isn't about it.

If you are using offline caching, you may want to use advanced features like handling the events that are fired when the browser is updating the cache. If you frequently change your webapp, this is a good idea. I handle these events in 3DTin to let the user know that the browser has found a newer version of the app on server and is now downloading it in background. When the download is complete, I show the user a refresh button which they can use to reload the page so that they can start using the new version.

It was while coding this functionality that I came across the swapCache() API call. It sounded very cool and from some examples that I read I got to believe that when the cache download is finished (i.e. in the onupdateready handler), I can simply call swapCache() and user will start using the new version of my app. He/she won't have to reload the page. If you don't read the documentation of swapCache carefully you will get the same impression from simple examples around the net. It may not be a big issue, if your cache contains passive resources like image files. But my erroneous interpretation of swapCache led to a nasty bug in 3DTin few days ago, because I am caching javascript files too.

The documentation clearly says:

[swapCache] does not cause previously-loaded resources to be reloaded; for example, images do not suddenly get reloaded and style sheets and scripts do not get reparsed or reevaluated. The only change is that subsequent requests for cached resources will obtain the newer copies.

This has huge impact if you are loading javascript or css files from cache. In my case, I use WebWorkers which load a separate javascript file after they are started. When 3DTin cache was updated by the browser and swapCache was called, the javascript file that WebWorker used to load was at a different version than all the other files that were loaded when webpage was opened. This led to failures that left me scratching my head for some time. Eventually when I realized this is what was happeneing, I stopped calling swapCache() and instead showed user a button to reload the app manually.

I hope this post will save you from doing the same mistake that I did.

How to save HTML5 canvas image to Google App Engine

This is a very specific problem and not many would have it. But I had to implement it and came up with a solution that works for me. This is how I did it for 3DTin.

First why it's specific. In a typical case a web app wants to save user submitted images to Google app engine data store. This can be simply done by presenting user with a form and adding an 'input' tag with type 'file'. The submitted image can be extracted in GAE using request.get() and packaged into db.Blob to eventually store into BlobProperty. I am not going to explain this typical case, because it's covered nicely in the GAE's documentation. However, the process is not as straightforward when the image you want to store is not an image file on user's computer, but a dump of HTML5 canvas element in the web app.

An HTML5 canvas element lets you export its content as jpeg or png image with the toDataURL API. This function returns a data URL that contains base64 encoded jpeg/png image. So how do we convert this image data into GAE's db.Blob object?

The solution is to send the base64 encoded data as POST param to GAE app, do some regex matching to extract the exact portion of the data URL that is the encoded image and pass it through the base64 decoder (which is part of standard python library).

Here is client side:


Here is server side code:


Code is also accessible as gist here, in case you can't see the embedded version above.

This code works in 3DTin where a thumbnail of user's canvas is sent to GAE app for storage. It works without problem.

Ads:

Portable apps for iPhone, Android, Pre, n900 - Can Processing.js help?

I don't know. But if you want to find out, then I have something for you.

Processing.alt

Visit this website from your desktop browser. You will see an IDE. Write some processing code to create a sketch and save it when you are done.

Visit the same website from your iPhone/Android/n900/Pre browser. You will get a simple viewer interface. Find your sketch under My Sketchbook and run it.

Processing is a well known language to create sketches, animations, games. It is a simplified dialect of java. The resulting sketches are in the form of bytecode and can be run on desktop or in java applets. In Summer of 2008, John Resig (jQuery fame) ported Processing to javascript. Processing.js uses the HTML5 canvas element to render the sketches.

The modern browsers that ship with today's smart phones support HTML5 element. So it is very much plausible to get your Processing sketch/game working on all the modern smart phones without any knowledge of native app development for each platform. Processing.js has a potential to become an uber-portable library for mobile development.

That was the idea I started out with about a month ago. After several iterations, I ended up with a website that you see now. It is an IDE when opened from desktop browser. You can write Processing code, run it, save it, share it for others to see in the Gallery. There are also some samples you can find in Gallery right now. Try them out.

The same website acts as a viewer for your processing sketch when opened from a mobile phone. You can view the shared sketches from the Gallery or you can login and find your saved sketches in My Sketchbook.

One great advantage of Processing.js is, it can intermix javascript with Processing code. So in the samples you will find how javascript is used to automatically find  the kind of browser and adjust the dimensions of the sketch accordingly.

I have tested the site on Android HTC Magic and Nokia n900. A friend told me it worked as expected on his iPhone too. Also I am treating Palm Pre's user agent as mobile platform. So on these four platforms you should see the mobile interface of the website. But in case if you don't, let me know and I'll see what is wrong.

As you will see that running Processing code on smartphones is indeed possible, but is fairly sluggish. However I believe the future is not far when the phones will become more powerful, browsers will become more optimized and Processing.js can also be tuned for speed. We already have two gadgets with 1GHz processors - Nexus One and iPad.

The IDE also has a lot of scope for innovation. But before jumping ahead of myself, I decided to open it as beta. I would like to hear your thoughts and suggestions on this idea.

So try it out and leave your opinion in the comments.

P.S. For logging in to save your sketches, use your GMail account. You don't need to login to browse through the Gallery.


Ads:
* Processing: A Programming Handbook for Visual Designers and Artists
* Learning Processing: A Beginner's Guide to Programming Images, Animation, and Interaction (Morgan Kaufmann Series in Computer Graphics)
* Processing: Creative Coding and Computational Art (Foundation)