Shiny Things

Identity Recall

2011-06-04T08:11:00.002-04:00

io.com is going away. It's funny how upsetting this is to me. I've been jimm@io.com since 1994 or so—maybe a year or two earlier than that. You know what I'm worried about most? All those open source projects, emails, and other digital resources that point to jimm@io.com are going to be pointing nowhere in a month. It feels like my online identity is being stolen. Except it's not being stolen, of course—merely recalled.

A Slashdot commenter also pointed out the potential security risk: "Think about it, among other things whoever owns that domain now will be able to intercept all mail to io.com accounts, and with the quickness and suddenness of the transfer not everyone's who uses those addresses is going to be able to completely transition off them before the transfer happens"

io.com was the ISP run by Steve Jackson Games—the company raided by the Secret Service because they were writing a role playing game called "Hacker". SJ Games fought the Secret Service and won. I think the EFF grew out of that case.

io.com was bought by prismnet.com years ago. PrismNet changed hands a few times. The last guy who sold it to the current owner (for $20) didn't sell the io.com domain. He kept it but let them use it until July 1, 2011. My guess is he wants to sell it to I/O Digital for a skillion dollars.

From now on, I'm jim@jimmenard.com. Also jim.menard@gmail.com, or jimm@prismnet.com. Time to start wrapping my head around being jim@jimmenard.com instead of jimm@io.com. Bye, jimm@io.com.

io.com is dead. Long live io.com.

Play Framework Configuration

2010-07-25T17:39:00.006-04:00

I've been using the Play framework for a big project at work. It's fantastic. I almost hesitate to say it, but it's like Ruby on Rails for Java. Watch the video that's on the home page; it's a good quick introduction.

The Tech Ops team at work wants control over configuration of the app. This makes sense for a few reasons. First, they're responsible for running it so they should be able to tweak things like memory settings. Second, they need to tell the app where the database lives, what the password is, etc. Tech Ops also wants the app to run as a WAR inside an app server. That's fine: Play comes with a war command that bundles your app into either an exploded WAR or a WAR file.

The problem is, Play doesn't know how to read any configuration file other than the one that's in its conf directory. At least, that's what I thought. It turns out there is an undocumented feature of Play configuration files: If any configuration key is named@include.foo, then the value is used as a path and that file is read, too.

A few caveats.

The path is relative to the application directory within the WAR. So, for example, to find a file in Tomcat's conf directory, the path will have to be something like ../../../../../conf/play.conf.
The file that gets included isn't treated exactly like the normal configuration file. Magic values like ${play.path} and ${application.path} are not interpreted,.
The current Play framework id (similar to Rails' environment) is ignored. However, since the include code is run after everything else is loaded, this isn't really a problem since the external configuration file you're loading is presumably for the environment you're running.

There is also a plugin architecture, and a hook that runs after the configuration file is read. This means that, even if @include.foo didn't exist, you can write a plugin to do anything you want to alter the app's configuration.

Update: Core Play developer Guillaume Bort wrote on the email list, "This feature is not yet documented because it is not ready: as you observed it the placeholders and the framework id stuff are not handled properly." Caveat emptor.

Up, Up, and Away

2009-09-12T12:31:00.003-04:00

Let's say you are in a subdirectory, and there's a Makefile in a parent directory. How do you run make? You climb back up to the parent directory and type "make", or use "make -C parent_dir". In either case, you have to know what the parent dir is. Here are a pair of scripts that do that for you.

The findup script's job is to find a file in a parent directory. It prints the directory name if found, else it exits with status 1.


#! /bin/sh
#
# usage: findup file [file...]
#
# Finds any of the files in the current directory or any directory above and
# prints the directory name. If no such directory is found (we hit the root
# directory), exit 1. Note: will not find files in the '/' directory.

start_dir=`pwd`

while [ `pwd` != '/' ] ; do
    for f in $* ; do
        if [ -f $f ] ; then
            echo `pwd`
            exit 0
        fi
    done
    # Keep swimming...keep swimming...swimming, swimming, swimming...
    cd ..
done

# No parent directory
exit 1

The makeup script uses findup.


#! /bin/sh
#
# usage: makeup [args...]
#
# Finds the first makefile in the current directory or any directory above and
# then runs make, passing on any args given to this script.
#
# Relies on the "findup" command, which is found in this directory.

mfdir=`findup makefile Makefile`
if [ -z "$mfdir" ] ; then
    echo no makefile found
    exit 1
fi

# We've found a makefile. Use the -C flag to tell make to run from the
# directory where we found the makefile. We do this so that error messages
# produced during the make process are relative to the current directory. I
# think.
make -C $mfdir $*

ChucK Hack Fork

2009-08-03T15:47:00.005-04:00

I've created a GitHub fork of ChucK, the "strongly-timed, concurrent, and on-the-fly audio programming language." In my fork I've started to add missing features like string methods, and hope to add many more such as raw file I/O and perhaps some form of MIDI file I/O.

So far, I've added string.ch which is a getter and setter for string characters (really one-character length substrings) and string.substr. This allowed me to write a function in ChucK that converts strings like "c#4" to MIDI note numbers.

It looks like ChucK's code was last updated in 2006, so I don't feel to bad about forking it.

Update: I just read in the chuck email list archive that the team is planning to start active development again, and they will be adding file I/O. If they get it done before I do, that would be very nice.

A Weak Man Begs Forgiveness

2009-06-22T03:52:00.004-04:00

Dearest Colleagues,

I write this electronically-delivered letter to you in the hopes that you will forgive me. A terrible fate has befallen my lovely daughter V_______, and I am afrid that I now share in her misfortune. To protect you all from similarly horrible destinies, I must eschew your company this day.

It all started innocently enough, with a persistent cough. My eldest daughter would delicately hack to "clear her throat". The coughs became more frequent and urgent. She developed a fever which burned her pure, innocent forehead. As time passed, she begain to writhe in agony, delirious with dreams of God knows not what Heavenly retributions and Hellish tortures.

My dearest wife E____ conveyed her to the pediatrician's office on Sunday. Her doctor (a woman, by God! Ah, well. I know not what muse guides Science, but she must hold in favor the fairer sex) diagnosed V_______ with Swine Flu. The woman practiced her Art with confidence, perscribing the most modern treatments available.

We repaired to our home, powders and poultices in hand. The cure had barely taken hold of my dear daughter when I myself began to cough. At first, I denied the obvious. What strong man would not? Having fought in The War, my thoughts were not for my own safety or bodily condition but for that of my child.

I tell you now: Mr. Poe never wrote of horrors so small. The tiny demons that ravage my body even now defy all description, and to burden you with reports of the agony they cause does not merit further thought.

The submarine-like conditions of the Juicyorange office being the breeding grounds for animalcules it is, it is best that I remain far, far away. This is why I have decided to remain in an entirely different State of the Union. My sorrow at not joining you is only mitigated by the knowledge that you will not be exposed to the fiends that wrack my body.

In the fervent hope that this missive is not rerouted to your Spam folder,

With Deepest Respect,
James J. Menard
Babbage Engine Servitor

Using will_paginate in Rails to Submit an AJAX Form

2009-04-18T09:58:00.002-04:00

I'm using will_paginate for a Rails project. It works great, but I wanted to use it for a sidebar that performs searches and displays the results. It's not too hard to write a custom link renderer that will submit the will_paginate link using Ajax, but my situation was a bit different: I wanted will_paginate to inject the page number into the search form and submit the form using Ajax.

Here's what I ended up doing: writing a link renderer that calls a JavaScript function that injects the page number into the form and calls the form's onsubmit function (which then submits the form using Ajax).

The code. First the Haml, which is in a partial that gets rendered on a number of pages:

%h1 Asset Search
- form_remote_tag(:html => {:id => 'ssform', :name => 'ssform'}, ...) do
  %input{:type => 'hidden', :id => 'sspage', :name => 'sspage'}
  / ...
  %input{:type => 'submit', :value => 'Search'}

  - paginated_section(sidebar_assets, :class => 'rhs_pagination', :renderer => SidebarSearchLinkRenderer) do
    %table.list
    / ...

Next, the will_paginate renderer, in the file app/helpers/sidebar_search_link_renderer.rb. The "#" is the href value, and the onclick attribute calls a JavaScript funtion.

class SidebarSearchLinkRenderer < WillPaginate::LinkRenderer

  def page_link(page, text, attributes = {})
    @template.link_to text, '#', attributes.merge(:onclick => "return sidebar_search(#{page});")
  end

end

Finally, the JavaScript function which I put in public/javascripts/application.js:

function sidebar_search(page) {
  $('sspage').value = page;
  document.forms['ssform'].onsubmit();
  return false;
}

Time Keeps On Slipping, So Freeze It

2009-02-25T12:42:00.007-05:00

A friend of a friend has trouble testing timestamps. I tried replying there, but don't think my answer got through so here it is.

The short answer: all time operations should not use the wall clock, but a system time object that you create. In production, the system time object returns wall clock time. In testing, you replace/mock/modify it so that it uses the time you give it. In other words, it becomes a clock that you can freeze and re-set whenever you need to.

Here is an example SystemClock class in Ruby. All of your classes should use this class to get the current date and time instead of using the Time or Date classes directly.

# Keeper of the current system time. This class is sometimes overridden during
# testing to return different times.
class SystemClock

  def self.date
    Date.today
  end

  def self.time
    Time.new
  end

end

Here is the mock used during testing. You can set the time using this mock object and it will not change until it is set again.

# This mock lets you set the system date and time during testing. For example,
# to set the date to tomorrow,
#
# A few examples of use:
#
#   SystemClock.date = Date.today + 1 # Time also set: to tomorrow 9:10:11 am
#   SystemClock.date = nil            # Go back to using system date and time
#   SystemClock.date = Date.civil(2006, 4, 15)
#   SystemClock.time = Time.local(2006, 3, 2, 8, 42, 42) # Date changed, too
#
# When you set the date, the time is set to 9:10:11 am of the same day, local
# time. When you set the time, the date is set to the same day.

require 'models/system_clock'

class SystemClock

  @mock_date = nil
  @mock_time = nil

  def self.reset
    @mock_date = nil
    @mock_time = nil
  end

  def self.date=(date)
    @mock_date = date
    @mock_time = date == nil ? nil :
      Time.local(date.year, date.month, date.day, 9, 10, 11)
  end

  def self.time=(time)
    @mock_time = time
    @mock_date = time == nil ? nil :
      Date.civil(time.year, time.month, time.day)
  end

  def self.date
    @mock_date || Date.today
  end

  def self.time
    @mock_time || Time.new
  end

end

So in your normal production code you get the current time by calling SystemClock.time instead of Time.now. In your test code, you'd do something like this:

# Make sure to include the mock class
SystemClock.time = Time.now # Or any arbitrary time
thing = Thing.new           # Uses SystemClock to set created_at attribute
assert_equal SystemClock.time, thing.created_at

text2midi App Uses midilib

2009-01-28T08:58:00.004-05:00

Gabriel G has released text2midi, a Rails app that converts—wait for it—text that you paste into the Web page to MIDI, and plays it for you and lets you download the MIDI.

text2midi uses midilib, my Ruby MIDI manipulation library.

Go, Gabriel!

10gen Shifting Emphasis to the Database

2009-01-15T08:26:00.005-05:00

10gen has announced that it is Shifting Emphasis to the Database. They decided to focus on the Mongo database after finding that there was much more interest in it than in their Babble cloud computing app server.

Babble is still alive, though. It has a new web site and has shifted to the Apache license.

This may all be good news for 10gen, but it's bad news for me. Since I was working on bringing Ruby and Rails to Babble, I and a number of other fine engineers at 10gen have been let go. It hurts, but I've had a blast and learned a lot at 10gen.

For the next few weeks, I'm going to be finishing up work on the Ruby Mongo driver, MongoRecord (an ActiveRecord-like framework for Mongo that is independent of Rails), and Rails ActiveRecord support for Mongo.

Oh, yeah: here's my resume.

Doing Homework With a Wii

2008-12-26T11:24:00.003-05:00

My younger daughter just said to me, "Dad, I know how to cheat at homework with the Wii." I asked what whe meant. She said that if you had to alphabetize a list of words (which she sometimes does for homework), you could create a Mii (an in-game avatar on the Wii) for each word, and give it that word as the name. Then use the game's "sort by name" feature to display all the Miis in alphabetical order. Voila!

I'm so proud of her.

Rails Without ActiveRecord

2008-12-23T11:35:00.004-05:00

When we announced Rails support for the 10gen cloud computing platform, we said that ActiveRecord support was not yet included. This led naturally to the question, "what the heck good is that?"

Actually, you can run Rails without ActiveRecord just fine. Rails has been designed that way. For an example app that uses the Mongo database, see my 10gen-rorob Rails app. You might want to use the branch that uses Rails 2.2.2 instead, because the master branch uses Rails 1.2.6 and the 2.2.2 branch also has a few more features.

On the other hand, internally we have a subset of ActiveRecord working already and are working on supporting as much of it as makes sense with a non-relational database. I already have the Rails app from "Agile Web Development With Rails" running unchanged.

On the gripping hand, you can run the 10gen cloud yourself, in which case you can run it with MySQL or any other relational database. In that case, you can use ActiveRecord unchanged.

How To Freeze Gems In Rails

2008-12-23T11:29:00.004-05:00

Here's how to freeze any gem into your Rails app. This is a simplified version of the larger discussion Vendor Everything at err.the_blog.

Copy the gems you want into the directory vendor/gems.

$ mkdir vendor/gems
$ cd vendor/gems
$ gem unpack some_gem_name # repeat for each gem

Edit config/environment.rb and add the following inside the Rails::Initializer.run block:

Rails::Initializer.run do |config|
  # add the next three lines
  config.load_paths += Dir["#{RAILS_ROOT}/vendor/gems/**"].map do |dir|
    File.directory?(lib = "#{dir}/lib") ? lib : dir
 end
end

Now you can require those gems in your code, even if the system you are running on doesn't have that gem installed.

10gen Officially Includes Ruby and Rails

2008-12-17T23:00:00.000-05:00

For the last four months I've been sequestered deep in the heart of the 10gen labs among the test tubes full of bubbling primary-colored liquids, glass jars filled with mysterious biological samples, and mysteriously glowing rectangular pixilated displays. The whole time, I've been pressing small plastic keys in arcane sequences and muttering to myself.

Finally—after going through three lab coats, two lab assistants, and more brains than I care to remember—the lab doors are creaking open to reveal the secrets within: Ruby and Rails are now officially part of the 10gen cloud computing platform.

It's alive! ... ahem... It's alive! ... AHEM ... IT'S ALIIIIVVVVEEEE!

...

Damn. There's never a good dramatic thunderclap around when you need one.

Actually, this announcement isn't worth all the theatrics. Initial Ruby and Rails support has been in the SDK for a while now. This is more of a soft launch. Ruby support should be complete. Rails support does not yet include ActiveRecord. We do have an XGen::Mongo::Base class which is Mongo-specific with an awfully ActiveRecord-like API.

This launch is more of an official milestone where we are asking the community to take a look at what we've done and give us feedback. Please go grab the 10gen SDK and give it a try. Let me know if you have any questions or suggestions, either here or preferably on the 10gen mailing list. If you are in the NYC area, come to one of our hackathons.

See the Ruby Language Center page for links to documentation.

Clojure Mystery Solved

2008-12-15T17:01:00.003-05:00

I figured out what was wrong with the Clojure code in the Mongo Java Driver Clojure sample: we weren't turning the value returned by the database into a seq. You can see the fixed version here.

Ruby Mongo Driver Progress

2008-12-09T15:41:00.003-05:00

Over the last few days I've added a few features to the Ruby Mongo driver:

Sorting
Read/write support for arrays and regular expressions
An ObjectID class and read/write support
More tests

Database commands are JSON objects (sort of) which are almost like Ruby hashes, but not quite. Their keys are ordered, so I whipped up an OrderedHash class to use when the driver sends commands to Mongo.

Adrian Madrid (aemadrid@gmail.com) has been helping out a lot by providing invaluable feedback, tests, suggestions, and fixes.

Clojure

2008-12-07T14:10:00.004-05:00

I've been playing a bit with Clojure recently. Playing only: I haven't written anything serious yet. One thing I did write, though, was the Clojure example in Geir Magnusson Jr's Java Mongo driver.

There's a bug somewhere, though: the sample only prints records 1 and 3 out of 3. The Java sample works correctly, as does the JRuby sample. I can't figure out why, because the Clojure code that iterates over the records that are inserted is pretty simple. Here's the code:

(def mongo (org.mongodb.driver.impl.Mongo.))
(def db (.getDB mongo "clojure"))
(def coll (.getCollection db "test"))

(. coll clear)                          ; erase all records in the collection

; insert three records
(dorun (map #(do (.insert coll {"a" (+ % 1)})) (range 0 3)))

; print the number of records in the collection.
(println "There are" (.getCount coll (org.mongodb.driver.MongoSelector.))
  "records in the collection 'test'")

; one way to do a query
(loop [i (.find coll)]
  (when i
    (do (println (first i))
        (recur (rest i)))))

; and another
(dorun (map println (.find coll)))

If you can see what's wrong, please let me know.

Ruby Mongo driver

2008-12-07T13:56:00.003-05:00

I've been working on a pure-Ruby driver for Mongo, 10gen's document database. If you are of the Java persuasion, then check out Geir Magnusson Jr's Java driver for Mongo.

One thing that we are discussing internally at 10gen is which layer of the system should be responsible for automatically creating primary key fields for records. Mongo does not require them, but Babble (the 10gen cloud computing app server) does. I think what will end up happening is that the driver will not add them unless the caller specifically asks for it.

Ruby in the Clouds Video

2008-12-07T13:53:00.004-05:00

The videos of most of the RubyConf 2008 talks are now available online at the Confreaks RubyConf 2008 web site, including my talk Ruby in the Clouds.

Ruby In the Cloud

2008-09-15T12:05:00.003-04:00

I'll be giving the talk "Ruby In the Cloud" at RubyConf 2008 in November. For a poorly-formatted outline, see http://rubyconf.org/talks/20.

Geir Magnusson Jr and I gave a preliminary version of this talk last week to the nycruby, the NYC Ruby Users' Group. Thanks to everyone who attended for all their questions and suggestions.

Downpour

2008-07-21T12:15:00.008-04:00

Here's a bunch of big news from cloud computing application framework company 10gen: their SDK is now available for download, they are making the entire platform Open Source, and they've received Series A financing. Congratulations to 10gen for reaching these big milestones, and for the decision to make their code Open Source.

In related news, I'm joining 10gen as of August 11th. I can't wait to start. I'm hoping to be able to blog about what I'll be working on: improving their Ruby implementation and making Ruby on Rails work on their cloud computing platform with Mongo, the 10gen distributed OO database.

Magpies and Shiny Things

2008-01-07T09:38:00.000-05:00

Over at Coding Horror Jeff Atwood just posted The Magpie Developer. As somebody with a blog named "Shiny Things", I'm well aware of my penchant for new technologies. Recently, it's been Scala and Erlang.

The wise magpie doesn't throw away his old shiny things. He takes what he learns from new technologies and tries to apply them to what he's using now.

LilyPond

2007-11-16T08:46:00.002-05:00

I've always wanted to learn to use LilyPond. My first two attempts are Mosquito Byte Rag [link fixed] (PDF, 148K) and a piano version of the Equal Rites Main Theme (PDF, 117K). There are no dynamic or note markings yet.

I wrote Mosquito Bite Rag thirty years ago. Thirty. Years. Ago. That makes me feel old, but what's worse is that I haven't written anything seriously in over ten years. Even the Equal Rites theme is around fifteen years old.

JavaScript Crashing Mac OS X Firefox 2.0.0.9

2007-11-16T08:41:00.001-05:00

Recently I upgraded to Firefox 2.0.0.9 on Mac OS X. Ever since then, Firefox has been crashing extremely frequently. Most often, it's when I'm closing a tab---usually GMail.

When the Apple problem reporter app fires up, I get to see the stack. Every time I've checked, the crash has been in the JavaScript engine somewhere (the times I've remembered to check, it's always crashed in libmozjs.dylib).

I just reproduced the problem frighteningly easily: after a crash I restarted Firefox. I then went to GMail. After the page loaded, I closed the single tab containing GMail. Crash. Ouch.

What is keeping me from switching to Safari or Opera? At this point, familiarity and Google's bookmark synchronizer. The synchronizer has its own problems, though.

Gmail Adds "Delete" Keyboard Shortcut

2007-11-02T08:37:00.001-04:00

Today my GMail account was upgraded to the new interface. The contacts management interface has been improved, but that's not what I'm excited about.

On a whim, I decided to review the keyboard shortcuts found at http://mail.google.com/support/bin/answer.py?answer=6594. Guess what? Google has added "#" as a keyboard shortcut for "delete". YAY! They've also added a bunch of other keyboard shortcuts I haven't seen before: mark as (un)read, archive and previous/next, and undo.

As a die-hard keyboarder, the lack of a "delete" shortcut has been most annoying. I've been waiting for this ever since GMail launched.

Thanks, Google.

Halloween Emacs Tips

2007-10-08T14:53:00.001-04:00

Here in time for Halloween: a new section on my Emacs tips page about skeletons.

Yaws and Mnesia

2007-08-07T10:27:00.001-04:00

I'm playing with translating a small Web application from Rails to the Erlang web app framework Yaws. When I tried to access a Mnesia database from a page, it failed because Mnesia wasn't running. Oops. I tried running it from another erl shell, but that didn't work.

After asking on erlang-questions, here's what I've learned from the responses: You can tell Yaws about Mnesia by using the command line switch '--mnesiadir full-path-to-mnesia-directory' (but that only works when yaws is running as a daemon).

Or, you can connect to the yaws (erl) runtime and start Mnesia from there. If you started Yaws with -sname and cookie you can connect to it like any other Erlang runtime.

Finally, the simplest way to do this is to run yaws from the command line and start mnesia from within the yaws session. That's what I've done.

Thanks to the members of erlang-questions for their help.

Erlang Notes

2007-07-18T09:31:00.001-04:00

Two Erlang-related notes, neither about the language per se. First, take a look at "Why do you like Erlang?". I love the Magic card. The article's good, too.

Second, my Erlang blog entries for June were selected as that month's contest winners by Pay Eyler of On Erlang, PragDave Thomas, and Joe Armstrong. Thanks to all of them for the honor. There has been no official announcement (and there might not be), but my copy of Programming Erlang is on its way now that the paper edition is shipping. I've had my PDF version for quite a while, and I highly recommend the book.

CSI:Munich - ZFS Saves the World

2007-06-28T08:33:00.001-04:00

CSI:Munich - ZFS Saves the World. Warning: for geeks only.

Star Ship Dave

2007-06-27T09:03:00.001-04:00

Occasionally movies, TV shows, and ads are filmed around where I work in NYC. I've never seen a "star", but I often see the food trailers and office trailers and once or twice I've seen locations with cameras and lights—no action yet, though.

For a few weeks last fall, an extras tent was set up on the street where I work. The costumes on the extras were really cool: modern, colorful versions of '30s suits and dresses.

Today I saw a catering setup. Nothing odd about that. What was odd was the sign hand-written on a whiteboard attached to the food truck: "Welcome, Star Ship Dave". I can't help chuckling every time I think of that title. Whether or not it's a working title, it's got me wondering. Who is Dave? Is it the ship?

Perhaps it's a Dave we know already: David Hasselhoff, Dave Chapelle, David Letterman, Dave Barry, or David Bowie. (Nah, not Bowie—some Davids are always "David", never "Dave".)

Perhaps I'll brave the 90-degree heat today and venture outside in a while. I want to see what the costumes look like.

Erlang Boids Simulation Design

2007-06-23T11:01:00.003-04:00

As one of my Erlang programming exercises, I decided to write the non-graphical part of a Boids simulation. I've written the same thing in a few different languages (Java, Ruby, Lisp, C++) before. Using Erlang would be interesting because of its support for message-based parallelism and concurrency.

I figured at first that each bird should run in its own process. In each of my other implementations there is also a flock object. Not necessarily a big-O Object, but at least some central place where the birds are collected that can answer questions about them such as the average velocity of the flock and its center of mass. Even with a multi-process Erlang implementation, it makes sense to have a flock process. Flocking behavior rules depend upon each bird's knowledge of the position and velocity of all other birds in the flock (more realistically, all other birds it can see). There are two choices: make each bird know about all other birds or have a flock object that can answer questions about all birds. Making each boid (bird) aware of all the others seems to be a bad design decision: it requires O(n²) communications overhead for every calculation. Each boid has to ask all the others its position and velocity all the time.

With a flock object, the same communications are O(n) because the flock can ask each boid for its position and velocity, and then respond with the aggregate information (average positions, average velocities) when asked.The number of calculations is still O(n²) because each boid wants to know the average position and velocity excluding itself. It's the number of messages and the number of connections between processes that is O(n). To reduce the number of calculations to O(n), you could ignore the "excluding itself" qualification.

My first Erlang implementation resulted in a nasty deadlock. The flock periodically messaged each boid, telling it to move. When the boid received the move message, it updated its position and velocity. In order to do that, it had to message the flock, asking for information about the other boids. When that happened, the program hung because the flock not only wasn't listening for messages, it was still waiting for a response from that boid. Deadlock!

The next version had the flock asking each boid for its location information first, and then telling each boid to move, pasing in the information it needed. This worked, but it was synchronous and definitely not elegant.

Finally, I hit upon a better design: The flock would request position information from each boid periodically, and separately each boid would peroidically tell itself to move, asking the flock for the other boid's information when it needed it. There is a chance that a boid could have moved while other boids were relying upon it's old position and velocity cached by the flock, but that's not really important. In real life birds using old, slightly out-of-date information all the time: they blink, or are distracted by a bug, or are subject to the speed of light's limitations on information gathering.

Firefox: Of Thee I Sing, Thunderbird: Of Thee I Hum

2007-06-23T11:01:00.001-04:00

or: Don't Fence Me In
or: Freedom, by George Michaels

I've tried the new Safari Beta 3. It's really fast and refreshingly standards-compliant. I don't care for the brushed aluminum look, though supposedly that's going away.

There's no way I'll switch from Firefox, though. There are two main reasons: I rely on many useful Firefox extensions, and Firefox is still more cross-platform than Safari. Firefox also supports more search engines in its search text field. To be fair and balanced, I must point out that Firefox 2.X crashes or hangs on Mac OS X all the time. I put up with that because of all the other benefits.

That first reason is the biggest though. Firefox is more hackable, which makes it more useful. Here is the list of Firefox extensions I use. The Firefox Add-ons page is also a great resource.

"It's All Text!" lets you edit any textarea's text using your favorite editor. Mine's set to use Emacs, of course. You can set the editor in the preference dialog box, but it might be easier for you to edit the Firefox's user prefs Javascript file and add

user_pref("extensions.itsalltext.editor",
          "/Applications/Emacs.app/Contents/MacOS/Emacs");

I should probably use emacsclient instead so the file is open in the currently running Emacs instead of in a new instance.

Switching to another browser like Safari would force me to give up too many extensions that I have come to rely on. Google Browser Synch is probably the most useful to me, with Firebug and LiveHttpHeaders coming in a close second and third.

Safari does have some features that Firefox doesn't, of course. Sharing your bookmarks with those around you via Bonjour and integration with Apple's AddressBook are the two I can think of. I don't want either. How often have I wanted to share all my bookmarks with others or see all of theirs? Never. Hell, I password-protect my Firefox so that if other people use it they won't see the form field values that Firefox saves for me.

There are some little things that bug me about Safari, too. For example, the keyboard shortcut to open/close the bookmarks sidebar is Option-Command-B and the command to switch to the Google search text field is Option-Command-F. I hate the Option-Command combo; it forces me to switch my whole left hand position away from the home row. In Firefox it's Command-B and Command-K for Mac OS X, Control-B and Control-K for Windows and Linux.

For similar reasons, I use Thunderbird instead of Apple's Mail. Not because of the extensions (I only use one: Enigmail), but because it is cross-platform, uses a standard format for email message storage, and uses an open format for its address book. I'm using Mac OS X now, but it's possible that I might want to switch to Linux (Ubuntu is quite nice) or be forced to switch to Windows at some point. I can use Thunderbird on those platforms, too.

When I must open Word documents, Excel spreadsheets, and PowerPoint presentations I use NeoOffice. On Windows, I use OpenOffice.org.

It's most important to me that my data is portable not only between operating systems but also between computers and applications. I refuse to get locked down into any proprietary format or have my data locked down on only one computer. I'm letting Google store my (encrypted) browser information, but I have my own copies of the data on more than one computer, inside Firefox itself.

Feeling Left Behind

2007-06-09T08:29:00.000-04:00

A few months ago, a study surfaced on Reddit that stated that the ability to digest milk in human adults is a relatively recent adaptation. It further posited that this ability was convincing proof of the theory of evolution: humans have evolved to be able to digest lactose later in life.

Years and years ago, I remember reading one of those fun kids' science articles that talked about genetic traits like the ability to roll your tongue or wiggle your ears. The one that stuck with me was that some people have a small, solitary hair between the last two knuckles of the ring fingers on either hand. I always imagined that having that hair proved that you were a genetic throwback.

I have that hair. I'm also slightly lactose intolerant. I feel like a caveman. Maybe I could get a job acting in a Geiko commercial.

At least I have some compensations: I can roll my tongue and wiggle my ears. So there, all you evolved monkeys!

Speedier Erlang

2007-06-05T12:30:00.000-04:00

In the comments to Erlang Fractal Benchmark, Ulf Wiger noted in the comments that adding is_float guard clauses speeds up the code.

When I asked why on the Erlang email list, Ulf and others explained: when the compiler sees is_float(X) it knows that X must be a float. Instead of worrying about type checks, casting, and other inefficiencies, it can optimize the code that uses X.

Erlang Fractal Benchmark

2007-06-04T08:21:00.000-04:00

While looking at a simple fractal benchmark that showed up on the programming Reddit, I noticed that there wasn't an Erlang version. Below is one I wrote last night. Erlang fares rather well. One thing mildly surprised me: it runs slightly faster in an Erlang shell within Emacs than in both Apple's Terminal and iTerm on Mac OS X. Within Emacs it runs in 1.09000 (runtime) 1.14100 (wall clock) seconds. In both Terminal and iTerm it runs in around 1.11000 (runtime) 1.16600 (wall clock) seconds. Perhaps screen I/O isn't as fast in the terminal programs.

Two caveats: first, these numbers were generated on my 2.33 GHz Intel MacBook Pro; I don't know what the original benchmarks used. Also, I only ran the code a handful of times and picked a "typical" time to report. A better test would have been to run the code hundreds or thousands of times and average the values.

This post also says a bit about intuition vs. measuring. I discuss some code modifications and their expected and actual effects below.

Another thing to note: the author of the fractal benchmark page says that he hasn't bothered to optimize the code for each language he tested. I don't know if using lists:map/2 or extracting iter_value/5 and using guard clauses would disqualify this version in his opinion.

-module(fractal_benchmark).
-author("Jim Menard, jimm@io.com").
-export([run/0]).

-define(BAILOUT, 16).
-define(MAX_ITERATIONS, 1000).

%% Idea from http://www.timestretch.com/FractalBenchmark.html

run() ->
    io:format("Rendering~n"),
    statistics(runtime),
    statistics(wall_clock),
    lists:map(fun(Y) ->
                      io:format("~n"),
                      lists:map(fun(X) -> plot(X, Y) end, lists:seq(-39, 39))
              end,
              lists:seq(-39, 39)),
    io:format("~n"),
    {_, Time1} = statistics(runtime),
    {_, Time2} = statistics(wall_clock),
    Sec1 = Time1 / 1000.0,
    Sec2 = Time2 / 1000.0,
    io:format("Erlang Elapsed ~p (runtime) ~p (wall clock) seconds~n",
              [Sec1, Sec2]).

plot(X, Y) ->
    case iterate(X/40.0, Y/40.0) of
        0 ->
            io:format("*");
        _ ->
            io:format(" ")
    end.

iterate(X, Y) ->
    CR = Y - 0.5,
    CI = X,
    iter_value(CR, CI, 0.0, 0.0, 0).

iter_value(_, _, _, _, I) when I > ?MAX_ITERATIONS ->
    0;
iter_value(_, _, ZI, ZR, I) when ZI * ZI + ZR * ZR > ?BAILOUT ->
    I;
iter_value(CR, CI, ZI, ZR, I) ->
    Temp = ZR * ZI,
    ZR2 = ZR * ZR,
    ZI2 = ZI * ZI,
    ZRnew = ZR2 - ZI2 + CR,
    ZInew = Temp + Temp + CI,
    iter_value(CR, CI, ZInew, ZRnew, I + 1).

You might have noticed that ZI * ZI and ZR * ZR are calculated twice: once in the body of the last clause and once in the second guard clause. The guard clause has to be executed every time, which means that running the last, most frequently executed clause executes the multiplications twice. I tried pre-calculating those values and adding them as parameters, so the arg list is

iter_value(_CR, _CI, _ZI, _ZI2, _ZR, _ZR2, I)

Did it help? It did indeed. Execution time in Emacs went down to 0.890000 (runtime) 0.919000 (wall clock) seconds. Here are the modified versions of iterate and iter_value:



iterate(X, Y) ->
    CR = Y - 0.5,
    CI = X,
    iter_value(CR, CI, 0.0, 0.0, 0.0, 0.0, 0).

iter_value(_, _, _, _, _, _, I) when I > ?MAX_ITERATIONS ->
    0;
iter_value(_, _, _, ZI2, _, ZR2, I) when ZI2 + ZR2 > ?BAILOUT ->
    I;
iter_value(CR, CI, ZI, ZI2, ZR, ZR2, I) ->
    Temp = ZR * ZI,
    ZRnew = ZR2 - ZI2 + CR,
    ZInew = Temp + Temp + CI,
    iter_value(CR, CI, ZInew, ZInew * ZInew, ZRnew, ZRnew * ZRnew, I + 1).


One final thing I tried was commenting out the calls to io:format. In my experience, screen I/O usually slows things down quite a bit. (In the case statement in plot/2, I had to replace them with void statements instead of simply commenting them out.) The result: execution time went down to 0.830000 (runtime) 0.839000 (wall clock) seconds within Emacs. In iTerm, execution time was only slightly slower than that. So the time decreased, but not nearly as much as it did when I removed the extra multiplications. I'm surprised. Is multiplication that expensive in Erlang, or is I/O well optimized, or was my instinct wrong? Come to think of it, io:format is only called once per coordinate; the multiplications happen thousands of times for each.

A distributed version of this algorithm is certainly possible (say, one process for every X,Y coordinate). My gut tells me that it would run slower because the calculation for an individual coordinate is relatively small and the message passing overhead and gathering and coordination of the results would outweigh the benefits. My intuition was wrong about the effects of I/O, though. I'd have to try it to make sure.

Additions: In the comments, Ulf Wiger suggested that I add is_float guards for all the function parameters that are floats. Doing this to iter_value/7 reduced execution time by almost half to 0.450000 (runtime) 0.455000 (wall clock) seconds, a huge savings. (Does anybody know why adding these guard clauses speeds up execution? I would imaging that extra checking would slow it down.) Here's the new code for iter_value/7:

iter_value(_CR, _, _ZI, _ZI2, _ZR, _ZR2, I)
  when I > ?MAX_ITERATIONS,
  is_float(_CR), is_float(_ZI), is_float(_ZI2), is_float(_ZR), is_float(_ZR2) ->
    0;
iter_value(_CR, _, _ZI, _ZI2, _ZR, _ZR2, I)
  when _ZI2 + _ZR2 > ?BAILOUT,
  is_float(_CR), is_float(_ZI), is_float(_ZI2), is_float(_ZR), is_float(_ZR2) ->
    I;
iter_value(CR, CI, ZI, ZI2, ZR, ZR2, I)
  when
  is_float(CR), is_float(ZI), is_float(ZI2), is_float(ZR), is_float(ZR2) ->
    Temp = ZR * ZI,
    ZRnew = ZR2 - ZI2 + CR,
    ZInew = Temp + Temp + CI,
    iter_value(CR, CI, ZInew, ZInew * ZInew, ZRnew, ZRnew * ZRnew, I + 1).


Ulf and others also suggested that I avoid printing each character
separately. Doing so did not seem to change execution time at all. Here's what I did:

% changed the inner map call to plot
run() ->
    % ...
    Seq = lists:seq(-39, 39),
    lists:map(fun(Y) ->
                      CharList = lists:map(fun(X) -> plot(X, Y) end, Seq),
                      io:format("~s~n", [CharList])
              end,
              Seq),
    % ...

% changed plot to return single-character strings
plot(X, Y) ->
    case iterate(X/40.0, Y/40.0) of
        0 ->
            "*";
        _ ->
            " "
    end.


I also tried commenting out the io:format/2 call above. Execution time went up about 1/100 of a second.



An Erlang MIDI File Reader/Writer
2007-05-22T22:11:00.000-04:00
I've been learning Erlang. (See Learning a New Programming Language.) As part of that process, I've written three programs so far: a Boids flock simulation, a simple Web application using Erlyweb, and a MIDI file reader/writer.

One point that I might not have made clearly in my previous post is the reasons that I learn a new language: to learn new ways to think about and solve coding problems, to see what's out there that's better than what I'm using (I try not to be a Blub programmer), to keep my brain sharp, and because it's fun.

Coding the MIDI file library helped me learn about Erlang's file I/O, the binary data type, and a bit more about pattern matching. (The Boids simulation has much more pattern matching goodness; more about that in another post.) The code isn't distributed, nor does it really take advantage of many of Erlang's strengths such as distributed process (Boids does). It does make use of the binary data type and pattern matching quite heavily.

I was also forced to think about data representation in Erlang: how would I represent a sequence, a track, and a single MIDI event? I'm not convinced that I have come up with the best representations, because I have not yet had the time to use this library for any MIDI file manipulation. When I get to that point, I fully expect that the data representation will change.

Having written this code in a few different languages before, I knew where to start: with the easy stuff! I first defined the constants. midi_consts.hrl contains all of the define statements for all the constants I might need. It was while creating this file that I learned how to write hex numbers in Erlang. (That sounds like a small, silly thing but I was quite annoyed for a few minutes before I figure out how to do that.)

% Channel messages
-define(STATUS_NIBBLE_OFF, 16#8).
-define(STATUS_NIBBLE_ON, 16#9).
% ...
% System common messages
-define(STATUS_SYSEX, 16#F0).
-define(STATUS_SONG_POINTER, 16#F2).
% ...


Next, I dove into reading a MIDI file. Since I only deal with MIDI type 1 files which contain multiple tracks in a single file, that's all I bothered writing. I knew I'd have to read the header and one or more tracks. Erlang code tends to use atoms, tuples, lists, and records to represent many types of data, so I came up with a proposed data format for my sequence data. From the comment for midifile:read/1, which takes the path to a MIDI file and returns a seq tuple:
%% Returns
%%   {seq, {header...}, ListOfTracks}
%% header is {header, Format, Division}
%% each track is
%%   {track, ListOfEvents}
%% each event is
%%   {event_name, DeltaTime, [values...]}
%% where values after DeltaTime are specific to each event type.
%% If the value is a string, then the string appears instead of
## [values...].


After writing some code that read one byte at a time, I went back to the "Programming with Files" chapter in Programming Erlang by Joe Armstrong and realized that I could slurp all of the MIDI data into memory and randomly access it. This made my code cleaner and faster. It also made pattern matching much easier, because I could write midifile:read_event with many different pattern-matched arguments but the same arity. Here's the code that reads an event list within a track:

event_list(_F, _FilePos, 0) ->
    [];
event_list(F, FilePos, BytesToRead) ->
    [DeltaTime, VarLenBytesUsed] = read_var_len(file:pread(F, FilePos, 4)),
    {ok, ThreeBytes} = file:pread(F, FilePos+VarLenBytesUsed, 3),
    ?DPRINT("reading event, FilePos = ~p, BytesToRead = ~p, ThreeBytes = ~p~n",
     [FilePos, BytesToRead, ThreeBytes]),
    [Event, EventBytesRead] =
 read_event(F, FilePos+VarLenBytesUsed, DeltaTime, ThreeBytes),
    BytesRead = VarLenBytesUsed + EventBytesRead,
    [Event | event_list(F, FilePos + BytesRead, BytesToRead - BytesRead)].


Let's start at the end: event_list/3 is recursive. It builds the list of events by reading a single event, then calling itself with the remaining bytes to read in the track. If the remaning number of bytes is 0, the first clause (the first two lines above) matches and an empty list is returned. The second, longer clause starts by reading a variable length integer (more about that below). It then reads the next three bytes of the file (again, randomly accessing an in-memory copy of the file). Why three bytes? Because most MIDI events are two or three bytes long. If I need fewer bytes, no harm done. If I need more, I read more. Those three bytes are passed on to read_event/4, which is a function that is made up of a really long series of clauses, each of which matches a different MIDI event.

Here are the first few read_event/4 clauses. The first three arguments are the file, current file position, and delta time of the event. These clauses all return an array consisting of the event tuple and the number of bytes used by the event (excepting the length of the delta time).

read_event(_F, _FilePos, DeltaTime,
    <<?STATUS_NIBBLE_OFF:4, Chan:4, Note:8, Vel:8>>) ->
    ?DPRINT("off~n", []),
    put(status, ?STATUS_NIBBLE_OFF),
    put(chan, Chan),
    [{off, DeltaTime, [Chan, Note, Vel]}, 3];
% note on, velocity 0 is a note off
read_event(_F, _FilePos, DeltaTime,
    <<?STATUS_NIBBLE_ON:4, Chan:4, Note:8, 0:8>>) ->
    ?DPRINT("off (using on vel 0)~n", []),
    put(status, ?STATUS_NIBBLE_ON),
    put(chan, Chan),
    [{off, DeltaTime, [Chan, Note, 64]}, 3];
read_event(_F, _FilePos, DeltaTime,
    <<?STATUS_NIBBLE_ON:4, Chan:4, Note:8, Vel:8>>) ->
    ?DPRINT("on~n", []),
    put(status, ?STATUS_NIBBLE_ON),
    put(chan, Chan),
    [{on, DeltaTime, [Chan, Note, Vel]}, 3];
read_event(_F, _FilePos, DeltaTime,
    <<?STATUS_NIBBLE_POLY_PRESS:4, Chan:4, Note:8, Amount:8>>) ->
    ?DPRINT("poly press~n", []),
    put(status, ?STATUS_NIBBLE_POLY_PRESS),
    put(chan, Chan),
    [{poly_press, DeltaTime, [Chan, Note, Amount]}, 3];
read_event(_F, _FilePos, DeltaTime,
    <<?STATUS_NIBBLE_CONTROLLER:4, Chan:4, Controller:8, Value:8>>) ->
    ?DPRINT("controller ch ~p, ctrl ~p, val ~p~n", [Chan, Controller, Value]),
    put(status, ?STATUS_NIBBLE_CONTROLLER),
    put(chan, Chan),
    [{controller, DeltaTime, [Chan, Controller, Value]}, 3];


The put calls store the status and channel in the process dictionary, one of the few places in Erlang that you can modify values. To handle running status bytes, we need to remember the status and channel. I picked the process dictionary as the place to store that.

Added 2007-05-23: "Running status bytes" are a way to reduce the size of MIDI files. If the status byte is the same as the previous status byte, you can omit it. Also, as a special case a note-on value with a velocity of zero is considered to be a note-off message. I needed code that would recognize that the next byte was not a status byte, and use the previous status byte. I made that an additional read_event/4 clause, like this:

% Handle running status bytes
read_event(F, FilePos, DeltaTime, <<B0:8, B1:8, _:8>>) when B0 < 128 ->
    Status = get(status),
    Chan = get(chan),
    ?DPRINT("running status byte, status = ~p, chan = ~p~n", [Status, Chan]),
    [Event, NumBytes] =
 read_event(F, FilePos, DeltaTime, <<Status:4, Chan:4, B0:8, B1:8>>),
    [Event, NumBytes - 1];


We read the status and channel from the process dictionary and pass them back to read_event/4, which will use Erlang's pattern matching to go back and find the proper code for that status byte. End of additions.

?DPRINT is a macro that I defined to output when DEBUG is defined and do nothing when it isn't:

% -define(DEBUG, true).
-ifdef(DEBUG).
-define(DPRINT(X, Y), io:format(X, Y)).
-else.
-define(DPRINT(X, Y), void).
-endif.


MIDI encodes certain multi-byte integer values by "variable length encoding" it: splitting it into seven bit chunks and outputting them with the highest-order chunk first. The last, lowest-bit chunk has the high bit set to zero, the earlier, higher chunks have the high bit set to one. Thus zero is encoded as 00000000 and 129 is encoded as 10000001 00000001. Notice that the code in event_list/1 always reads the next four bytes; that's because the MIDI spec guarantees that var length numbers are at most four bytes long. Here's the code for read_var_len/1, which returns a list containing the value and the number of bytes it uses:

read_var_len({ok, <<0:1, B0:7, _:24>>}) ->
    [B0, 1];
read_var_len({ok, <<1:1, B0:7, 0:1, B1:7, _:16>>}) ->
    [(B0 bsl 7) + B1, 2];
read_var_len({ok, <<1:1, B0:7, 1:1, B1:7, 0:1, B2:7, _:8>>}) ->
    [(B0 bsl 14) + (B1 bsl 7) + B2, 3];
read_var_len({ok, <<1:1, B0:7, 1:1, B1:7, 1:1, B2:7, 0:1, B3:7>>}) ->
    [(B0 bsl 21) + (B1 bsl 14) + (B2 bsl 7) + B3, 4];
read_var_len({ok, <<1:1, B0:7, 1:1, B1:7, 1:1, B2:7, 1:1, B3:7>>}) ->
    ?DPRINT("WARNING: bad var len format; all 4 bytes have high bit set~n", []),
    [(B0 bsl 21) + (B1 bsl 14) + (B2 bsl 7) + B3, 4].


bsl means "bit-shift left".

Only after finishing the MIDI file reader code did I attack the MIDI file writer. For that, I created an in-memory "IO list". An IO list is a list that consts of other IO lists, binaries, or bytes (integers between 0 and 155). The Erlang library method file:write_file/2 writes an IO list with one call.

Here is the code that writes a MIDI var len value:

var_len(I) when I < (1 bsl 7) ->
    <<0:1, I:7>>;
var_len(I) when I < (1 bsl 14) ->
    <<1:1, (I bsr 7):7, 0:1, I:7>>;
var_len(I) when I < (1 bsl 21) ->
    <<1:1, (I bsr 14):7, 1:1, (I bsr 7):7, 0:1, I:7>>;
var_len(I) when I < (1 bsl 28) ->
    <<1:1, (I bsr 21):7, 1:1, (I bsr 14):7, 1:1, (I bsr 7):7, 0:1, I:7>>;
var_len(I) ->
    exit("Value " ++ I ++ " is too big for a variable length number").


This code makes use of guard clauses: expressions that determine whether or not to execute a function clause. Note that the final clause exits. This matches Erlang's philosophy that you should code for the normal case and let errors happen. It might have been more in the Erlang style to leave off the last case entirely, and let the program fail with a "bad match". Perhaps I'll remove it.

After finishing the writer, I read in a file with the reader and output the Erlang representation using the writer. The result: a few differences, but all due to decisions about running status byte values. The code isn't perfect; it has a long way to go. I've learned a lot, though.

Next: a Boids simulation in Erlang.



Learning a New Programming Language
2007-05-22T08:46:00.000-04:00
I'm a self-professed language maven. I love learning new programming languages. Recently, I've been learning Erlang, thanks to Programming Erlang by Joe Armstrong, the online documentation, and the Erlang mailing list.

I'm a hands-on learner (see Just Try It). For me, there are a few phases to learning a new language:


Overview
Installation and Documentation
Syntax
Standard libraries
Philosophy
Frameworks


These phases don't occur sequentially; they blur together. For example, when trying to absorb what I call the philosophy of a language, I'll be writing code that forces me to look for how to use the standard libraries and look for existing frameworks.

The overview phase is when I initially hear about a language. It might be through posts in mailing lists or discussions groups, the various RSS feeds to which I subscribe, or on Reddit. During this phase I tend to form a few first impressions about a language: is it really different? How? Why does it exist? Will it be useful to me in my day-to-day work? Is it worth learning to exercise my brain, or because I will learn new concepts?

Next comes installation and documentation. I grab a copy of the language and whatever documentation I can. This is a key step: if I can't install the language or get it running, or if the documentation doesn't give me enough information to get started, then I'm done.

Learning the syntax gets me as far as being able to follow along with a book or the documentation. This is also another place for me to be turned off by a language. For example, when I started playing with Python, I decided it wasn't for me for a few reasons: significant whitespace, the __funky__ naming convention for magic OO methods, and having to refer to "self" all the time. Please remember, Pythonistas (all one of you that might read this :-), that I'm talking about my impressions and my decisions, not yours or anybody else's.

I'm not sure what made me temporarily abandon Scala (see Scala Baby-Steps) for Erlang. Partly, I was seeing references to Erlang when reading about Scala. Partly, it felt a bit less developed and complete then Erlang. Mostly, I think I'm learning more new things with Erlang.

Becoming familiar with a language's standard libraries or classes lets me know what comes "out of the box", as opposed to what I'll have to go look for or build for myself. It also gives me a good sense of the language's approach to solving various kinds of problems, large and small, and at different levels (from one-liners like iterating through a list through larger libraries like parsers or network communications). Again, the libraries can be make-or-break for me: while playing with Python, I found that simple string and regular expression operations were too difficult to find. (Actually, this is a problem with Erlang, too: string operations are all over the place in different libraries. One place for making a string upper case, another for making it lower case.)

By philosophy I mean a few different things: what the language is best at (and worst at), what it was designed for, what its strengths and weaknesses are, and what good code in the language looks like. Philosophy often, but not always, goes hand-in-hand with learning the syntax and the standard libraries.

During this phase, I usually pick for myself a small number of decent-sized tasks. By this time, I've been writing code in the new language for a while, but mostly toy exercises. It's time to write a larger program. Usually I take a project I've already written in a few different languages and port it to the new language. The code starts as a straight port, but as I progress I try to rewrite it using the idioms and techniques offered by the new language.

By this time, I've usually look for and found a few different language frameworks on the order of as a Web application server or app development framework. This gives me an idea not only of what's available, but what the community is like.

Speaking of community, by this time I've probably joined the language's primary mailing list.

I plan to blog about three different projects I've been working on in Erlang: a MIDI file reader/writer, a Boids simulation, and a simple Web application using Erlyweb.



Scala Baby-Steps
2007-01-09T10:28:00.000-05:00
I'm starting to learn Scala. It's a
  statically- and strongly-typed functional language that compiles to Java
  byte code and integrates well with Java.

One of the things I'm struggling with is the language's strong static typing. Scala is nice in
  that it tries very hard to infer types. That means the programmer doesn't
  spend much time typing "wasteful" type declarations. If Scala didn't have
  that, I would have run away screaming already.

I've figured out how to connect to a MySQL database. There are a few
  database access methods that are being developed in Scala. The one that ships
  with the language is called "dbc". It comes with a PostgreSQL connection
  interface but no MySQL version. Here's my MySQL version along with some
  simple code that performs a select. (I'd move the connection information
  into a properties file if I were to really use this code.)

import scala.dbc._
import scala.dbc.Syntax._
import scala.dbc.syntax.Statement._
import java.net.URI

object MysqlVendor extends Vendor {
  val uri = new URI("jdbc:mysql://localhost:3306/my_database_name")
  val user = "my_database_user"
  val pass = "my_database_password"
  
  val retainedConnections = 5
  val nativeDriverClass = Class.forName("com.mysql.jdbc.Driver")
  val urlProtocolString = "jdbc:mysql:"
}

object BulkUploadRunner extends Application {
  val db = new Database(MysqlVendor)

  val rows = db.executeStatement {
    select fields ("email_id" of characterVarying(32)) from ("bulk_uploads")
  }
  for (val r <- rows;
       val f <- r.fields) {
    Console.println(f.content.nativeValue) // or .sqlValue
  }

  db.close
}



   Link



Firebug
2006-12-08T10:14:00.000-05:00
Firebug is an incredibly useful Firefox extension for developers. You can debug JavaScript, edit HTML and CSS, inspect the DOM, determine the download times of all of the elements on a page, inspect headers, and much more.

Firebug 1.0 beta has just been released. I can't say enough good things about this extension. If you are a Web developer, it really should be part of your tool set.



Java's Enhanced For Loop Mystery
2006-12-05T08:51:00.000-05:00
I like Java 5's new enhanced for loop. It makes the code that iterates
  over collections and arrays less verbose. Anything that reduces Java's
  verbosity is a good thing in my book.

Yesterday I wondered if the Java 5 enhanced for loop calls the collection
  expression once or every time through the loop. I searched for the answer
  online, and couldn't find one. Every example I found assigned the
  collection or array to a variable first and then used that variable in the
  expression, like this:

  String[] list = {"a", "b", "c"};
  for (String item : list) {
    // ...
  }


What I was looking for was a statement like, "the list expression is only
  evaluated once". After a few minutes, I realized that it would be faster to
  write a small program to determine the answer.
  (See Just
  Try It.)


import java.util.*;

public class Test {

  public Collection<String> stringCollection() {
    System.out.println("I'm creating a new list now.");
    // Don't get me started on the verbosity of the next line...
    ArrayList<String> list = new ArrayList<String>();
    list.add("a");
    list.add("b");
    list.add("c");
    return list;
  }

  public static void main(String[] args) {
    Test t = new Test();
    for (String s : t.stringCollection())
      System.out.println(s);
  }
}


With a simple javac Test.java && java Test I had the answer:
  stringCollection() is only called once.



Just Try It
2006-12-05T08:44:00.000-05:00
I love answering programming questions. It lets me feel like a grizzled
veteran sharing my arcane knowledge with those eager to learn. (Yes, I'm
delusional.) If I don't know the answer, I enjoy finding out.

Most people don't know all the arcane language rules of every language they
use. As I get older, I'm less inclined to memorize a language spec or
an API. For
example, yesterday I wondered if the Java 5 enhanced for loop calls the
collection expression once or every time through the loop. I searched for the
answer online, and couldn't find one. After a few minutes, I realized that it
would be faster to write a small program to determine the answer. Within two
minutes, I knew. (I'm saving the answer and the code for another blog
post.)

If you have a question about a language rule or the use of a library, there
is more than one way to find the answer: read the documentation, ask somebody
else, or try it. If at all possible, Just Try It™—write a small
program that answers your question. Even better, if your language has
a REPL or even accepts input
from stdin, then type a few lines of code interactively.

Ruby comes with irb, the Interactive Ruby Browser. Every Lisp
has a REPL. Smalltalk has the Workspace. Shell languages have the shell.

When you Just Try It, you gain the immediate satisfaction of knowing the
answer. You also use a different part of your brain then when you read the
answer online; I claim you absorb the answer more thoroughly if you write a
small code snippet. You might want to keep a collection of these code snippets
for yourself. I have a personal Wiki (an Emacs Wiki, of course) where I store
these sorts of things if they're useful enough. There are also a few snippet
collection Web sites out there, such
as Code Snippets.



Settling In
2006-12-05T08:28:00.000-05:00
Welcome to the new home of Shiny Things, a blog about programming, programming languages, tools, frameworks, programming productivity tips, and whatever else catches my eye.

The old home of Shiny Things will remain. One reason I moved Shiny Things here is to allow comments. I haven't decided if I will copy some or all of the posts.