search for notability

Time to get technical. Not that the ongoing rumpus over CRU, Steve McIntyre's pioneering work in the GNU-blessed stats language R (with its underlying Fortran bringing back less-than-fond memories of undergraduate days at Cambridge - perhaps the only week of programming in my life I didn't thoroughly enjoy) or the redoubtable, anonymous statistician VS using MatLab and the Akaike Information Criterion to try to model basic climate time series isn't all very much so in its own way. And there's a connection back into all that at the end here. But first, the (other) news.

I've greatly enjoyed reconnecting with the London Ruby User Group a couple of times this year, meeting old friends like Ben Griffiths and new ones like Paul Ardeleanu and Alex McCaw, who have interesting open source projects in play to make Objective-C development for iPhone/iPad more Rails-like and Ruby/JavaScript/HTML5/CSS something more like a full GUI, respectively. There's a whole lotta innovation goin' on, even in one's own back yard. It's inspiring to pick up what the Ruby neck of the open source world is up to. And be assured that I'm not forgetting the Python contribution - I'm sure Google isn't going to let me do that!

But this Wednesday I had a problem, in that LRUG clashed with Open Source Show and Tell, which I'd not been to since BT Osmosoft kicked it off a couple of years ago. (I think OSSAT began as monthly and has since settled in to being semi-annual but, as this was my first in 20-30 months, any frequency better than that will do fine, thanks guys.)

There were a number of reasons for prefering the event just south of Drake's old boat at The Team this time. Free food and drink is never to be discounted in my case. I was keen to see Jeremy Ruston, inventor of TiddlyWiki, and Martin Budden, a stalwart ex-Symbian member of his team, again after some gap. I was very interested to meet key Ruston associates Chris Dent and Premasagar Rose for the first time - and learn much more about the dark state of the art in writing fail-safe widgets for the web from Prem. But top of the list for sure was hearing from Phil Hawksworth on node.js, as I told Phil when I wandered in early, evidently building the pressure for the guy as he carried out last minute fixes to his Keynote slides and live demo - something I always love to do.

I'd felt I'd had another key choice in the last week: whether to do some exploration myself with this very lightweight, single-threaded but non-blocking event-driven library, about which there's been substantial buzz on GitHub and in the JavaScript community generally, or to get into Johnson. I chose the latter and that turned out really well, as I'll come to. But the other reason for waiting on node.js was to pick Phil's brains first, who's been tinkering with it on and off for a number of months. As I expected he said it was awesome - that was in the title - and I quickly fell into line. JavaScript has always been a powerful language. I remember thinking so when it was still called LiveScript, reading the manual from Netscape in 1995, reflecting even then that this is closer to what I've believed in for over ten years - dynamically typed, prototype-based, with functions/closures as first class objects - than Java will ever be. More recently good people like Aptana have been trying to push JavaScript on the server in new ways. But it's needed something to act as a catalyst. And (based especially on the session in the pub on the edge of Borough Market afterwards) Phil and I tend to agree that node.js and the many things already being built on it and around it is sufficient for that. JavaScript, aided and abetted by the V8 engine from Google, is certainly going to be a force on the server as well as the client. And where does that leave Ruby, Python, Perl and PHP, to say nothing of C# and Java, I hear you ask?

Well, I don't entirely know. So I'm going to concentrate on a couple of simpler questions. Why am I, Richard Drake, still likely to be mucking around with Ruby in the coming months and years? And what's the relationship between Google's commitment to JavaScript, exemplified by V8, its undoubted fondness for Python and its sponsorship of Go, discussed in November?

Second question first. Is Google simply saying "We don't care, let the most popular man win"? No, of course they care. They prefer Python to Perl or PHP and rightly so. They still use C++ very heavily in the core areas of search, MapReduce, BigTable and all that, and their experience of that - by no means all positive - has led among other things to the slick dependency detection and blazing compilation speed of Go. But JavaScript is the most pervasive software language in the world, because it was built into the browser by Netscape and then Microsoft, as well as chosen by Adobe for programming Flash (as ActionScript). There may be no stopping it, on client and server. Google is quite right to want the best possible implementations of it to win out. And whatever one's doubts about the ethics of Big G in other areas, to open source V8 from the word go is a tremendous gift to the world. That - plus Linus Torvalds' open source efforts in C/C++ on the Git distributed version control system and the closed-source Ruby/JavaScript running GitHub, now the premier place on the planet for open source collaboration (don't miss the usual open-closed irony in passing) - all this has enabled the node.js flowering to happen. It really is an awesome thing, both technically and socially. Expect a blog post with more about that in the next few weeks.

There were issues left unaddressed there. Did the Go announcement signal that Google had run into the limits of Python, as some people suggested at the time? As the originators put it:

Programmers who could were choosing ease over safety and efficiency by moving to dynamically typed languages such as Python and JavaScript rather than C++ or, to a lesser extent, Java. Go is an attempt to combine the ease of programming of an interpreted, dynamically typed language with the efficiency and safety of a statically typed, compiled language.

I'm not sure I buy all of that and I don't how it will all pan out, for Google or the rest of us. The concise inbuilt concurrency based on CSP is another key aspect to Go, as discussed before. But what's the purpose of blog entries with dates on them if it isn't to say more later, maybe even change one's mind in the light of emerging evidence? Enough on all that for now.

Another subject I won't get to the bottom of is why I don't want to give up Ruby for JavaScript everywhere, not until certain things change. Top of those would be better, more consistent handling of collections (as Smalltalk once called groups, lists, bags, sets and dictionaries) and very clean iteration over them. The Prototype library made a valiant effort to put this right but has in many ways been superseded by jQuery, which has other strengths in the browser/DOM environment. A broader point is the lack of a common library - CommonJSbeing another brave attempt to attend to this.

It's not just the libraries, there's modules out of the box, thought through from the beginning, and some syntactic sugar in Ruby that makes it, for me, a good deal more readable and less easy to make silly mistakes in. There are also symbols, distinct from strings, as a first class concept built in (following Smalltalk) with very easy :syntax (just like that). And without exhausting the subject by any means, that takes me to Johnson and the other reason I'll be using Ruby for a while.

And that's because it's there. Not in as many places as JavaScript, due to the ubiquity of the browser. But largely due to initial Ruby on Rails hype (much of it deserved, given the state of the art at the time), Ruby has now emerged and proved itself as a viable real-world language for all kinds of things - and one of those things is a set of scripts by Osmosoft to chop up and put back together TiddlyWikis of all shapes and sizes. I have no idea who originally chose Ruby for this humble task - certainly these days Osmosoft are prefering Python on the server, notably in Chris Dent's very interesting efforts on TiddlyWeb. But I'm a big TiddlyWiki user. Size doesn't always matter but when I boasted to folks like Martin Budden and Phil Hawksworth on Wednesday night of getting on for 20 MB in a single wiki for my own personal information and research (correction, it's around 16MB, but you know how it goes with men and size) there was as I hoped some intake of breath. Yes, I know the dark side of the Tiddly experience. But mostly it remains an invaluable tool for me. And in the process of writing my own plugins to tailor the experience (under the brand name of Whiteword) and sharing stuff with other folk I've got pretty familiar with ginsu.rb and cook.rb, the Ruby files that chop up and reassemble TiddlyWikis.

It was only last weekend that it came to me that as there's been another version of TW I needed to check conflicts between my stuff and whatever functions and globals have been changed in 2.6.0 from 2.5.3 - and that there were surely by now some Ruby libraries that would allow me to parse JavaScript to assist with this task. Indeed there are. I was soon favouring Johnson, the cheekily-named Ruby-JavaScript bridge that I'd only vaguely heard of before, because of the many interesting things John Battelle and Aaron Patterson set out to achieve with it, as indicated in their hilarious joint presentation at RubyConf in late 2008.

Johnson came about before V8 was out and relies on the Mozilla SpiderMonkey JavaScript implementation used in Firefox (briefly called TraceMonkey but that moniker is no longer needed, the Johnson guys tell me). I'd been thinking on the weekend that someone smart might have taken this open source C++ and embedded it as an extension to Ruby in MRI, Matz's original C-based implementation of Ruby, now known to its friends as version 1.8.7. And that's exactly what Battelle and Patterson got working, more or less, in 2008, overcoming the pain of the memory management issues and other interesting boundary problems between the two languages.

This is a wonder of the open source world. You suddenly think something might be possible and you find someone smarter than you got there first and you can at once benefit, even if there are things not yet perfect (as is true with Johnson, for sure).

And benefit I did. Once I'd got over a very silly oversight that after recent playing with RVM I'd left my Mac OS X system with a rather old version of Ruby 1.8.7, it took me just a few hours to knock up a script in Ruby that analysed all the TiddlyWiki code, my own JavaScript and the plugins of others I've been using and told me where there might be conflicts. I took the parse tree provided by Johnson and converted to S-expressions, as I found this easiest to read, simply symbols and strings in arrays. I know my life is sad in many ways but it was truly exciting to see one's own JavaScript code expressed this way in irb:

ruby-1.8.7-p249 > sexp = Johnson::Parser.parse(IO.read('../tickle/parse_cases.js')).to_sexp
ruby-1.8.7-p249 > sexp[3] 
=> [:var, [[:assign, [:name, "basicMorpheme"], [:new, [[:name, "RegExp"], [:name, "wikiChunk"], [:str, "g"]]]]]]  
...
$ parse.rb ../tickle/parse_cases.js 
...
basicMorpheme            %p No time to explain - more later - except to say that basicMorpheme is a global defined by my JavaScript, and is here being picked out as such by my little parser in Ruby. It was truly satisfying to see this happen, then make the transition to TiddlyWiki 2.6.0 painlessly on the back of what I learned.

I was going to finish with an account of my chat outside the pub after OSSAT with Jordan Hatcher of Open Data Commons about the Open Climate Initiative. Of course I mentioned my promised lunch with Stephen Fry next month, where OCI is squarely on the menu. But that's got to be a separate post now. Yes I know, I need to break things up a little more, which I will do on this blog in future, I promise. But here's a neat connection I hadn't spotted until writing this: I went off to GNU because R is one of their languages (heavily used by the best climate auditors like Steve McIntyre and Willis Eschenbach) and, lo and behold, there was Fry's avuncular visage on yet another online video, this time celebrating the birthday of Richard Stallman's crucial baby. ALMOST AS UBIQUITOUS AS JAVASCRIPT? (The caps lock came on by mistake there but it seemed to fit.) But, joking apart, what a good man, a genuine modern-day polymath. More for sure in the next post and I hope beyond.