How to debug WebWorker threads?

WebWorkers offer your Web app a way to do multithreading. If your advanced web app does some number crunching keeping the CPU busy for a while, then it makes sense to do it in a separate thread instead of doing it in the main thread, which may lead to blocking the browser tab (Chrome) or the entire browser (Firefox and others). One of the troubles of writing programs in WebWorkers is, they are hard to debug. Two key mechanisms required for debugging a program are not available in WebWorkers - print statements (console.log is not available) and breakpoints (even if you manage to place breakpoints in the code running in WebWorker, they won't be hit).

During my work on 3DTin, I have learnt couple of techniques that help me debug my WebWorker code.

1. Use postMessage as replacement for console.log

postMessage is how you send some data from the WebWorker thread to the main thread. This is the primary way of returning the result of the computation you perform in WebWorker. But we can use it for other purposes too. Here is how you can use the postMessage mechanism for communicating both the successful response as well as debug messages.

2. Catch exceptions and send their stack trace over postMessage

When an exception occurs in the code running in WebWorker, the debugger will print the line at which exception occurred, but not the entire stack trace. In most cases the whole stack trace can help you find the root cause quickly. To achieve that, place most of your code that runs in the WebWorker inside a try-catch statement. Then in the catch clause extract the stack trace of the exception, format it nicely and send it to the main thread using postMessage, as discussed in the first technique above. Here is the exception handling code that helps you extract stack trace at least on Google Chrome and other Webkit browsers. I have cherry-picked it from the stacktrace.js library. If you need a browser agnostic solution, paste the stacktrace.js library at the beginning of your WebWorker code.

I hope this makes your life easy while debugging WebWorker code.

Elon Musk quote

Some original thoughts I've heard about running a company (from the latest Elon Musk interview)

I run both [electric car company] Tesla (TSLA) and SpaceX myself. It's a heavy workload, and I've never really wanted to run companies. Unfortunately, I came to the conclusion I was better than the CEOs we hired. If I'm not CEO, I can't make the inventions happen in the way they need to happen. Professional managers—MBA CEOs—are not very creative or adaptable, and their skills don't suit a startup. Business is like a multidimensional probabilistic chessboard. The rules aren't set, and the same moves don't always make you win. A lot of people can be really good in a set-piece battle; my biggest differentiating skill is I can invent new pieces.

When you think you are better than others, admit it.

JavaScript is the next C

I've been coding in JavaScript for almost a year now. I find it a decent language to write code in. The code written in JavaScript doesn't look as pretty as python, but it doesn't look as bloated as Java either. But that's not why I chose the title of this post. It's everything else that's happening around JavaScript that makes me think that it's going to play as significant a role in computing as C has played.

Before C, there was assembly. (I know there were many other languages before programming evolved from assembly to C, but I am talking about only the mainstream candidates.) The abstraction that C provided was perfect enough that programmers didn't have to learn about the Assembly details. That made C the perfect candidate to build a solid layer on top of Assembly. Programmers could write accounting software, graphic programs, games without bothering about what CPU architecture and memory bus size they were running on. It all worked and C became an integral part of computing stack. Today we don't use C to write new programs, but all operating systems, their device drivers, native libraries are written in C. C has become so ubiquitous that it's now invisible.

We have spent a long time searching for a candidate language to build the next layer, on top of the one built by C. C made our code hardware-agnostic. But over time we developed a variety of operating system platforms that created their own stacks of libraries. With the advent of Internet we needed to transfer code over the wire and be sure that it can be run on all the machines irrespective of which platform they were running. We needed a write-once-run-everywhere solution. Java emerged as the solution to specifically fill that need. For a period of time, it seemed it would indeed be the one. But for several reasons it failed. Today when we are deciding a platform independent solution to write a GUI program, what number does Swing score in our preferred list? Java did a great job of freeing the programmers from worries of memory management. But the main reason for its failure is probably its awkward ownership by a single commercial entity Sun (and now Oracle). The lawsuit Google is facing over Android is enough to corroborate this. If someone can sue you for using their language, then how can such language be adopted by entire industry.

The latest candidate to build the next abstraction layer in the computing stack is JavaScript. It's hardly a new language. It's hardly a perfect language. But there are two technologies that will make JavaScript the next C - HTML5 and Node.js.

HTML5 (and whatever goes under that umbrella name) is a new web framework whose primary programming language is JavaScript. It is bound to succeed, in fact if you consider it as just a new fancy name for HTML then it has already succeeded. It's not developed by a single company, but many big guns are simultaneously promoting it  - Google, Mozilla, Apple, Microsoft (by accepting most of the standard for IE9), are writing virtual machines that keep improving the speed of JavaScript. The new age behemoths - Twitter and Facebook - have strengthened HTML5 merely by adopting it.

On the other hand, Node.js has become a great success. It has a teeming community around it. The success of a platform depends on the libraries its provides to do various tasks. Just look at this wiki page that lists different Node.js modules to accomplish various tasks. I spent last week writing a factory server for 3DTin using Node.js. Everything I needed - from binary encoding library to canvas rendering libraries - I got it from this page. And there are more than one option for each job. Most of the libraries are nascent and will mature over coming few months. But they are a strong sign of a solid platform taking shape.

Yet another sign of JavaScript's growing might is, its choice as a target language for compilers of other languages. There are projects compiling Ruby, Python, Java right into JavaScript. The shortcomings of JavaScript as a language are being fixed by many frameworks successfully, think of jQuery. Libraries like Underscore.js or projects like CoffeeScript are making programming in JavaScript more fun.

Between HTML5 and Node.js you can now be sure that if you write your next application in JavaScript it will run on any server, desktop or mobile. JavaScript will build that next layer in computing stack that we are waiting for and that's why it will be the next C - ubiquitous and then invisible.

HTML5 offline cache - use swapCache carefully

HTML5 Offline caching is an incredibly useful feature that allows your users to load your website even when they are not connected to the Internet. When they visit your website first, the browser checks if your webpage mentions a cache manifest file in its html tag. If so, the browser downloads the manifest file and then downloads all the resources mentioned in that file. You are supposed to add all the resources that your webpage needs in the manifest file, so that when user is offline the browser can load the webpage from the cached version of those resources. You will find many articles that will help you implement offline caching for your website- like this one. This post isn't about it.

If you are using offline caching, you may want to use advanced features like handling the events that are fired when the browser is updating the cache. If you frequently change your webapp, this is a good idea. I handle these events in 3DTin to let the user know that the browser has found a newer version of the app on server and is now downloading it in background. When the download is complete, I show the user a refresh button which they can use to reload the page so that they can start using the new version.

It was while coding this functionality that I came across the swapCache() API call. It sounded very cool and from some examples that I read I got to believe that when the cache download is finished (i.e. in the onupdateready handler), I can simply call swapCache() and user will start using the new version of my app. He/she won't have to reload the page. If you don't read the documentation of swapCache carefully you will get the same impression from simple examples around the net. It may not be a big issue, if your cache contains passive resources like image files. But my erroneous interpretation of swapCache led to a nasty bug in 3DTin few days ago, because I am caching javascript files too.

The documentation clearly says:

[swapCache] does not cause previously-loaded resources to be reloaded; for example, images do not suddenly get reloaded and style sheets and scripts do not get reparsed or reevaluated. The only change is that subsequent requests for cached resources will obtain the newer copies.

This has huge impact if you are loading javascript or css files from cache. In my case, I use WebWorkers which load a separate javascript file after they are started. When 3DTin cache was updated by the browser and swapCache was called, the javascript file that WebWorker used to load was at a different version than all the other files that were loaded when webpage was opened. This led to failures that left me scratching my head for some time. Eventually when I realized this is what was happeneing, I stopped calling swapCache() and instead showed user a button to reload the app manually.

I hope this post will save you from doing the same mistake that I did.

How Wikipedia and Flattr can together change the World

Wikipedia is one of the best things that could have happened to humanity. That's why it feels awkward to see Wikipedia beg for funds every year, due to their decision to not generate revenue via advertisements. A debate between whether contextual ads are better or worse than page wide banners of Jimmy Wales can go to any length, without result. On the other hand, there is however a middle ground. Why doesn't Wikipedia accept micropayments?

Flattr is a micropayment service, that launched last year. If you have a blog, photo gallery or any "Thing" that has its webpage, you can place a Flattr button on it. Visitors of your page will click on that button, like they drop a dime in a tip jar. This action - called flattring - will gain you some monetary compensation. The way it works is, your visitor has a Flattr account that he/she fills with some money each month. During that month while browsing the web they encounter the Flattr buttons on various sites (like yours). They click on those buttons if they like what they see. At the end of the month Flattr divides their monthly payment by the number of Flattr-clicks they did in that month and pays that amount to the owners of those webpages.

Although Flattr has a very compelling solution for micropayments, I have seen some problems with the model from my own experience. The biggest problem is that, your visitor has to have a Flattr account. Whenever I come across a flattr button while browsing, I wish I had a flattr account so that I could tip this guy. But creating a Flattr account costs at least €2. It is very small amount, but it's not as attractive as €0 (to see how big that difference is, read some books on behavioral economics listed at the end) and it's a monthly expense. It doesn't help that I do not see Flattr buttons all that often while browsing. If I were to see them as frequently as Facebook Like buttons, I would definitely go register with Flattr. In my opinion, this is going to be the biggest hurdle in adoption of Flattr or similar micropayment solutions.

My guess is, the people who register a Flattr account are the ones who also have a creative "Thing" on which they want to place the Flattr button. Take my example. I had Flattr account since it was invite-only, but I never saw any point activating it by putting money into it. I hardly come across 2-4 pages each month that have Flattr button. Only yesterday when it struck me that I can put a Flattr button on 3DTin, did I take the pain of adding 3 euros and activating my Flattr account. So it seems that content creators have a reason to register with Flattr. But they are the minority in the population of the internet. The majority of netizens are consumers. Unless the consumers adopt micropayment, the creators cannot hope to make meaningful living out of Flattr like schemes.

Now imagine: What if Wikipedia places a Flattr button on each wikipedia page?

I read at least 5 Wikipedia articles every day, I am sure if you are reading this post, you have similar appetite for Wikipedia. When we read those articles we thank that Wikipedia exists. But most of us didn't pay $25 when Jimmy Wales asks for it once a year. We get the benefit of wikipedia in micro packets everyday, so when it comes to compare the its benefit with a big monetary sum, we tend to value money more. Therefore micro payments is the ideal model for us to express our gratutide to wikipedia.

If wikipedia adopts Flattr as their micropayment solution, overnight tons of people will open a Flattr account. And that is the exact impetus a service like Flattr needs to get over the critical mass. Once that limit is crossed, Flattr will be everywhere and will become a standard for micropayments. Then every content creator will be able to put Flattr button on their page with a confidence that their visitors will very likely be Flattr users too.

Wikipedia is in a unique position to bring a revolution in online payment. Any micropayment solution they choose will become a standard for the web and it will benefit Wikipedia too, without allowing advertisements on their pages.

So what's stopping this from happening?