Saturday, November 24, 2012

Coffeescript: To switch or not to switch

For a long time during the development of 3DTin I was toying with the idea of switching to Coffeescript ("CS" henceforth) from Javascript ("JS" henceforth). I was looking for a comprehensive guide about what are the pros and cons for such move. I didn't find any such document at the time. Now I have been using CS on-and-off for 3DTin in past 6 months. I have learnt a lot about the potential benefits and pitfalls of CS. This blog post is a collection of what I've learnt. Hopefully it will benefit those who are wondering whether to jump the boat. But note that, my experiences are not conclusive. I am still not sure that complete switch to CS is a good thing for every JS project.

Let's get started...

Coffeescript is not a language

A better characterization of CS is a set of preprocessor macros (think of C preprocessor). It's just that Coffeescript is such a complete set of macros that it can pass as a language. It's more like a shorthand for JS. Why is it important to think it like this? For two reasons:

1. You cannot get away from learning (or better mastering) Javascript. One of CS's design goals is to stay as close to JS as possible and not implement language features that JS cannot natively support. That means even if you use CS everywhere in your project, you should have good understanding of JS.

2. You can use CS alongside JS. Since CS is merely a shorthand for JS, some files in your project can be written in CS, while other are written in plain JS. That helps in adopting CS in already big Javascript codebase, like ours.

If you call Coffeescript a language written on top of Javascript, many people right away assume it's bound to be slower than corresponding Javascript (which is never true). (Do C preprocessor macros inherently make the generated C code slower? Not unless they are defined so)

The iterations

Iterating is something we do so many times. Therefore writing "for(int i=0, l=arr.length; i<l; i++) { ... }" every time is very painful, especially if you are used to Python's way of iteration "for obj in arr: ...".

In pure JS you have to use the long form for incantation. There is `forEach` in modern JS implementations, but who knows in which browser it breaks in what way. Our solution for this problem is underscore.js's _.each function. It is everywhere in our code.

Only recently however I realised, that liberal use of _.each can lead to significant slow downs. If the loop count is very large compared to the time spent in the body of the loop, you will see clear improvement in speed by using old-fashing for loop instead of _.each. CS gives you a compact way of iterating over an array and it automatically converts it to the most efficient JS code for doing it.

This iteration in CS
translates to this in JS

Best of both the worlds.

Named arguments

Naming the arguments is the best way to implicitly document the source code.

Moreover, in an ever-evolving environment the function signatures keep changing very often. If you rely on positional arguments, every time you add a new argument to the function, you have to check all the places from where it's already being invoked and check that the new argument doesn't break them. In JS, most of the times such new arguments are optional for the logic of the function, i.e. they can be undefined. Therefore they can normally added at the end of the argument list, so that the existing invocations won't need any change. For example, your current function signature is function foo(arg0, arg1). You decide to accept another optional argument newArg. You can change definition of foo to function foo(arg0, arg1, newArg). All the existing invocations of foo won't have to be changed, provided the body of foo handles newArg being undefined properly. But what if it makes sense to add newArg before arg0?

In such circumstances, it helps to have named arguments. JS doesn't support them.

For that matter CS also doesn't support them seamlessly, but it's easy enough to do. You can also add a line of code to gracefully define default value for an argument.
I learnt this trick from this Stackoverflow question.

Built-in classes

There are many ways to define Classes in JS, because they aren't natively supported. We use Base.js in 3DTin. With CS, classes are supported as part of language definition. You can define your classes in a more familiar OO syntax without worrying about all the black-magic with prototypes.


Avoids variable leaking

In JS it's very easy to forget to add var before a variable, resulting in that variable getting defined in global namespace. Do not think that creating scopes with (function() { ... })() blocks is going to help. It helps only if you define the variable with var, then it won't be visible outside that scope. But if you forget to add var, it will leak and will get defined as member of the window object. Here's a proof.

In CS you don't have to worry about explicitly declaring any variables. The CS compiler automatically generates the necessary var declarations for all the variables you use. IMO this is one of the biggest advantages of CS.

this. becomes @

In object oriented JS code, you are always going to access members of the class from inside it using this. prefix. If there was a more compact way to do this, it could save a lot of keystrokes and source code real-estate. CS designers realized that and have provided a shorthand for this usage.

The Fat arrow

There are lots of callback functions in typical JS code. Classes are also functions in JS. That leads to a problem with the use of this keyword sometimes. Inside the body of a function, this refers to the context of that function. Inside the body of a function that is part of a prototype, this context is the instance of the object. Therefore if you have written this, inside a callback function body, that is defined inside a prototype member function, which is it going to point to? It points to the callback function's context. But many times you assume it's referring to the instance of the object. This is a common mistake a  developer makes while starting with JS. If you want to refer to the instance of the object from inside the callback function, then this is the solution for it.



CS Fat arrow notation lets you write this in a more compact manner.

The use of fat arrow => automatically tells CS compiler to define a reference to outer context, so that the code inside the closure body can use simple @ notation and still resolve to the right context. We have such pattern all over the code. It's easy to imagine then how cleaner our CS code looks than the original implementation.

return is not necessary

CS automatically adds a return wherever it deems it necessary (and also unnecessary). This is a huge benefit while writing functional code.

You can write
instead of


String formatting

Are you tired of writing
CS makes it easy for you.


Parenthesis during function calls - can be skipped (almost)

CS lets you skip Parenthesis during function invocations. This can be a good thing or a bad thing. It can make your code look more beautiful, but its overuse can make it confusing. Also the skipping rules are not consistent. Take a look.


Braces and commas for dictionaries - can be skipped

Similar to parenthesis, you don't have to use braces while defining dictionaries / maps / objects. CS will automatically infer their structure based on context or indentation. This contributes a lot towards cleaning up redundant tokens from your source. If you are going to put each name-value pair on new line, you can get rid of commas too.

Translates to


Spanning source lines

Since CS infers a lot of things from indentation, when you try to break a long line of source into multiple lines, you can trigger compilation failures or at worse mis-interpretation.



In closing
In past 6 months I have used CS for all new source code and rewritten some JS code in CS if it made sense. I clearly saw certain benefits, but also some pitfalls. Not sure which path I will continue on. But I hope this post will help you make your own decisions.

It's of enormous help to have a continuously running watch script that compiles coffeescript files as you save them. I've written such script that watches all coffee, jade and less.css files in a directory tree and compiles them as soon as they change. You can find it here.

Thursday, November 22, 2012

The single most reason that keeps me using Vim

I've wanted to use all new and cool Code editors that have come around in past few years. Many times I have switched to them determinedly, only to switch back to Vim after a few days. All the fancy features that I see in the screencasts of these code editors are irresistibly inviting. Yet I can't let go of Vim. After thinking a lot about it, I think it boils down to one single reason.

The h-j-k-l-b-e and Ctrl+w, Ctrl+u, Ctrl+d key sequences.

These are key sequences for moving around inside a code window and then switching between multiple code windows. They are what I use the most in Vim (except for typing actual text). Over years my muscles have deeply memorized these keyboard movements. Every other code editor forces me to use arrow keys or mouse to do these navigations. I cannot get used to it even if I try it for a month. Therefore I switch back to Vim.

Of course I diligently install the Vim emulation plugins that come with the respective new code editor I am trying. But the code navigation key sequences are so frequently used that even a slightest departure from the original Vim behavior creates a lot of cognitive friction.

I'm not saying that h-j-k-l is the most intuitive way to do code navigation, but once your fingers memorize those movements it seems impossible to find anything else that can feel as satisfactory.

Water from air

It's fascinating how a really simple idea can change life as we know it. Today I read this news via slashdot about a new startup NBD nano, that is building a device that can extract water from air. The most interesting part is its implications and applications.

we can somehow have a way to tap into one of the largest reservoirs of water, and that being the air
...
And we see this being applicable to anything from marathon runners to people in third world countries, because we realize that water is such a large issue in the world today
...
We’re actually investigating a type of device that can be used for drip irrigation. And we are looking to incorporate this in greenhouses or green roofs in the immediate future
...
We actually see the maritime environment as really a very large market for us because humidity is actually constantly regenerated over a large body of water and then we can pull that humidity from the air to support people who possibly take long trips on yachts, or provide a sort of potable water source that can be run off a solar panel while at sea.
And the underlying technology seems convincing. Amazing stuff!

Sunday, November 18, 2012

Time travel fiction

This year I read two very gripping thriller novels based on Time Travel.

First one was back in January. 11/22/63 By Stephen King


and recently the twin tomes Black Out and All Clear by Connie Willis.

  

They belong more appropriately to thriller genre than Science fiction, despite the Time travel theme.

Building a huge plot where the timelines and characters are interwoven brilliantly is quite a challenging task. Adding time travel to it makes it even more interesting. Not to mention the awkward paradoxes that arise in the Universe where time travel is possible.

Both (actually all three) novels, are highly recommended if you like reading stories that keep you at the edge of your seat most of the time. Last sentence of almost every chapter is bound to surprise you. (My personal observation: Such novels are really good to improve your reading speed)

In 11/22/63 Stephen King tells the story from a man's point of view who has found a portal through time that opens in the past - more accurately in the year 1958. The entire novel is based on this man's attempt to exploit this portal in order to stop the Kennedy Assassination in 1963. King's description of the America in 60's is very inviting (if only we knew of such portal). The ups and downs in the whole novel keep you entertained all of the time. Here is one of my favorite quotes:

"I saw that most of the passengers were smoking. The atmosphere in there must have been roughly akin to the atmosphere on Saturn" [in reference to popularity of smoking in the 60's]

Black Out and All Clear are two novels continuing the same storyline. The story is about a group of students of history from the year 2060, who travel back to the years of World War II, to study the history first hand (as part of their research assignments) and get trapped in the past. The narration of London during the entire period of war is quite fascinating. The characters of the story live through the London bombings because their portals back to their time stop working for some reason.

Although in 11/22/63, Stephen King has conveniently dumbed down the paradoxical implications of time travels; Connie Willis bravely weaves her story on top of them. Of course no one can give foolproof resolution for the impossible issues that arise with the concept of time travel. However Connie has been fairly successful in building a maze of mysteries and then using her flavor of Time travel theory to explain those mysteries satisfactorily. You have to admire how much work she must have put into these two novels.

Highly recommended.

Thursday, October 11, 2012

When your git branch diverges from your remote

This is a tricky state and in past when I occasionally got myself into it I could never cleanly recover from it. Until now. I just found a clean way.

What's this tricky problem?

You know that git commit --amend is a very useful way to make changes to your last commit. You can add a couple of diffs to it or just change the commit message. It works great as long as you haven't pushed that commit to any remote. If the remote already has your last commit and now you make changes to it locally, your local branch and the remote branch are going to diverge. You will get a message like this

# On branch master
# Your branch and 'origin/master' have diverged,
# and have 1 and 1 different commit(s) each, respectively.

If you try to push to remote, git won't let you (if I recall correctly). Therefore I almost always check whether I've pushed the commits to origin/master before I try to invoke --amend. But sometimes accidentally I forget to check and end up in this tight situation. Well today I got a way to resolve it.

The state of your git repository is like this

Remote: A --> B
         \
Local:  A --> C

After commit A, you created commit B and pushed it to the remote. Then you accidentally did --amend on commit B and changed it to commit C. Here are the steps to recover.
  1. Rewind C. In the past I described different ways of undoing a commit. One of them helps us here. We run git reset --soft HEAD^ to rewind commit C. That is, the changes in commit C come back to the staged state. Now these changes are sum of the diffs that went in commit B and the changes you added with --amend. 
  2. Note the changes you did in --amend (copy them somewhere, hopefully they weren't many). 
  3. Now either erase these changes with git checkout or stash them away with git stash save
  4. Now do a git pull. This will bring your local repo in sync with remote. i.e. your repo's HEAD will now be at commit B. 
  5. Now do the changes that you were trying to commit with --amend (that you saved somewhere earlier). 
  6. Now commit them as a new separate commit D. 
  7. If you had stashed the old mixed changes, do a git stash clear to loose them.
That's it. Now your local and remote branches will look like this after you push

Remote: A --> B --> D
         
Local:  A --> B --> D


Thursday, October 04, 2012

Something like Olivia - John Mayer

One of the best lyrics in John Mayer's latest album - "Born and Raised".

You find a deeper meaning when you ask why he says "Something" and not "Someone".



(and yes, I would like to believe he means Olivia Wilde :)

Monday, July 30, 2012

Kelsey Grammer is spectacular

After watching 11 seasons of Frasier, I couldn't imagine Kelsey Grammer could be anyone but a comedian. When I watched Boss I was glad to be wrong. The character of Tom Kane is as diametrically opposite of Frasier as possible and Kelsey has packed it with genuine terror of the scale I have rarely seen.

Monday, June 18, 2012

Why Patriotism doesn't make sense

Imagine you are one in a group of young students who are going on a field trip across the country in a plane. This is first time on a plane for most of you and everyone is eager to get the window seat to see how the world looks from high up in the sky. As the seating arrangement in a typical plane goes, some seats are in the middle which obviously can't offer the beautiful view that the window seats can. The teacher is going to make seat assignments and her decision is perfectly arbitrary. It is clear to everyone that, if you get the window seat you will get a magnificent view and if you get to sit in the middle row all you will see is the back of the seat before you.

Being unconditionally proud of the country of your birth means getting the seat in the middle row and claiming that it has the most spectacular view that anyone in the entire plane has.

Which country you are born in is as arbitrary as an unbiased teacher allocating seats on the plane. In that case, if you are born in a country that is not good at something, but you decide to stand up for it just because you were born there then you are fooling no one but yourself.

One should glorify or villify one's country based upon the logical/ethical/moral reasoning and not because one was born or not born in that country.



Here are some great quotes on the subject:

Patriotism is your conviction that this country is superior to all other countries because you were born in itGeorge Bernard Shaw

A nation is a society united by a delusion about its ancestry and by common hatred of its neighbours - William Ralph Inge

It is not easy to see how the more extreme forms of nationalism can long survive when men have seen the Earth in its true perspective as a single small globe against the stars - Arthur C Clarke

Can anything be stupider than that a man has the right to kill me because he lives on the other side of a river and his ruler has a quarrel with mine, though I have not quarrelled with him? - Blaise Pascal

Sunday, June 10, 2012

Prometheus

Prometheus in IMAX 3D was a great experience. Some of the scenes were mind blowing. Only regret is there weren't more of them.

The IMAX 3D experience is worth for some initial scenes - ones right after the opening titles - the sprawling and calm lake, roaring waterfalls of murky water. The most breath taking scene is watching the Prometheus ship hung in the middle of space and some time later watching it in the backdrop of the planet and its rings. I would like to watch it again and again. Too bad it only lasts 5-10 seconds in the movie. Also the scenes where the humans and human-made objects are shown besides the gigantic alien structures are also awe-inspiring.

These things are worth a trip to the theatre. But besides that, the story is lame. Characters and actors are good, but their motives and actions could have used more thought. A work of fiction is more appealing if it can be easily extrapolated from the reality we know.

Saturday, June 09, 2012

Why sea steading is important to pursue?


For decades now, people have feared that the explosion of global population is going to harm the planet because we won't have enough space for all of them to live and not enough food to feed them. The population of the planet is now about 7 billion (A number that some scifi authors foresaw to happen when we inhabited entire galaxy). Yet there is enough food and enough space for everyone to live - on land itself. The only problem is that, the food and space is not distributed evenly. The problem is not about producing enough food, but to empower all individuals of the planet to earn their fair share of it. The problem of growing population has less to do with technology, and more to do with social sciences.

We do not need sea-steading because we are running out of space on land. We need sea-steading because there is no space left on land to do experiments that will build a better society.

In any scientific field, when we want to test our theories about new inventions, we do experiments. If you have an idea for a radical drug that will cure cancer then you won't be able to test it on real patients right away. Because there are laws that restrain you from doing tests of unproven drugs on humans. Instead, you first try it on rats and if things prove promising then after several years of research the refined drug is sanctioned for human testing. If you have an idea for self driving car, then you don't let your prototype loose on crowded streets. You take your prototypes in deserts for experiments. Only after years of testing in deserts, can someone drive a semi-autonomous Prius on streets of Mountain View under manual supervision. For testing radical world-changing ideas we need a test environment. This is as true for socio-economic experiments as it is for technological ones.

For successfully tackling the exploding population of the world, we need to do radical social and economic experiments. At present we try to find the solutions only in the framework that is allowed by the laws of Governments. Governments are democratically elected entities. They enforce the policies that only masses can approve. Radical and world-changing ideas are by definition not agreeable by consensus. Therefore if you have some social policy - like free health care to everyone, social security based on carbon footprint, or a variant of capitalist economy that won't lead to Wall Street - you cannot try it.

In 1947, India became an independent nation. At the time, it didn't have a clue what kind of economic policy it wanted to execute. The world had two strong economic philosophies - Socialism and Capitalism. Both of them were unproven at the time. India implemented a mixed policy - allowing private sector industries, but keeping the Government stake in major infrastructure projects. In 1980s, with the fall of Soviet Union, it became clear a Socialist economic policy with total Government control is doomed. Therefore in the 90s, India (among other countries) opened its policies gradually. Letting private sector and multinational investments in major infrastructure projects. Countries learn from each other for creating their social and economic policies. If one country can execute a social experiment and prove that it works, other Governments will be able to convince their population that it works and eventually adopt the policies themselves. Better solutions can only be found if one can experiment.

Now that populations are growing and are connecting more closely, they enforce the consensus more effectively. In many obvious cases of social welfare, it works greatly. But due to massive public opinion, even the greatest leaders in the Office think twice before implementing a radical change in nation's policies.

If we want to develop better policies, we have to be able to start from scratch. We need petri dishes to test our experiments. There remains no land to create these petri dishes, so we need to go on water. We need to build cities and towns on water where we can implement new laws and rules. Not all of them will be successful, but they will give the terrestrial nations a data point to correct their policies.

Thursday, May 17, 2012

Rolling back latest git commit

It's surprising how difficult it's to undo the latest commit you accidentally made in git repository. It's not difficult, but the commands are obscure.

Depending upon the situation you can do one thing or the other.

1. You have already pushed the commit to remote. Then only safe way I know is to do "git revert ". It will create a new commit that essentially negates the effect of the given commit. This will leave two self-cancelling commits in your git history, but it's a safe choice in case you are in doubt.

2. You haven't pushed the commit to remote and you only want to change the message of the last commit or only want to add some more files to the last commit. Then you can run "git commit --amend".

3. You haven't pushed the commit to remote. You can still use "git revert ", but you have another option. You can run "git reset --soft HEAD^". This will undo your last commit and leave the files from that commit in "staged" status. Now if you didn't want some of those files in that commit, you can unstage them.

Before you use any of the above commands however, here's an advice. Git commands are abstruse and coding scenarios vary enormously. My situation may have been different when I used above commands from yours. Some git commands can cause loss of work if not performed correctly (git reset flavors especially). Here is the best way to go about.

Create a temporary directory, do "git init", create some mock files and mock commits. Then try the git commands you are doubtful about. Once assured about what they do is what you want, apply them to your working repository.

Friday, April 20, 2012

Google Lunar X Prize

Came across this long article about how the teams in Google Lunar X Prize are faring so far. It looks like many teams are finding it hard to secure funding for the project and may not make it to the 2015 deadline. Many are dropping out.

After I learnt about the things that these teams will have to achieve in order to claim the prize, I thought the whole thing was unnecessarily over-ambitious.
The $30 million Google Lunar X PRIZE will be awarded to the first privately funded teams to build robots that successfully land on the lunar surface, explore the Moon by moving at least 500 meters (~1/3 of a mile), and return high definition video and imagery
It means the team has to accomplish a number of difficult things like
  1. launching successfully from earth
  2. landing safely on moon 
  3. building a rover that can at least move 1/2 km, autonomously or remotely
  4. communication system with robot for control and video transmission 
  5. robust packaging of the rover during the flight and landing
If the goal of the competition is to further the Space exploration technologies, the prize should have focused only on first two things - launch from earth and land safely on moon. There are other competitions to promote advancement in robotics. Why waste limited funding resources on technologies that are not directly related to Space. They should have limited the scope of the competition to launching a piece of rock from earth and safely landing it on moon. That still would have been a massive achievement in itself.

If a team successfully puts a rover on Moon and fails to drive it, who cares? Driving a robot and recording a video is trivial, compared to landing on moon.

The most important obstacle to space exploration is the exorbitant costs involved in just escaping the gravity well. We need to focus solely on that single portion of the problem first.

Saturday, March 31, 2012

iTunes Download stopped (err =9006)

It's pretty frustrating to see this error in iTunes when you have spent a long time downloading some stuff over a slow network. You hit that re-download button, hoping that it will just immediately mark it alright after it has found the already downloaded content, but it starts the download all over again.



Don't click that re-download icon.

In my case when I saw the above error after the download was complete, I found the files hidden in the iTunes folder. I could copy them elsewhere and open the content in VLC.

The folder you will find it in is
$HOME/Music/iTunes/iTunes Media/*

Sunday, March 18, 2012

Good essay on user centric software design

Came across this long post "#46 – Why software sucks" by Scott Berkun via Hacker news.

It discusses how software developers don't think about software design from their users' point of view and eventually create a product that sucks. There is nothing new in the writeup that I haven't read elsewhere before, but it's well written and it seems to explain the problem quite clearly. Here is a quote that I especially found appealing - it addresses the difference between good code and good product.
One illustration of the philosophical differences between love of construction and love of good things is the belief that code should be beautiful. I’m a fan of beautiful things: the world can use more of them, including more lines of beautiful code. But the trap is that code is an artifact of constructing software. When code is well written (beautiful or homely), it compiles. And it’s the output of the compiler (or browser) that people see. It’s the complied code that changes the world. The beauty of the code is relevant only to people that look at code. To worry about code aesthetics more than the aesthetics of the product itself is akin to a song writer worrying about the aesthetics of the sheet music instead of the quality of the sounds people hear when the band actually plays.
...
Computer science is taught with a construction mentality. Even the theory and philosophy that are covered support construction, not higher level design, or how to make good software in the sense I’ve described. Aspects of design are covered, but at an internal level, the design of object models, data structures and networks, not at the level of what happens when the technology meets the world or the people in it