Migrating VexFlow to SCons

I love tools. Tools keep my projects predictable and smooth. Tools help me code fast and release fast.

VexFlow has gone through many iterations of design, implementation, and deployment, and I couldn't have brought it this far so quickly without my tools. (Well, automated testing had a lot to do with it too, but that's for another post.)

Yesterday, I added a new tool to my toolbox - SCons. I migrated all my building, packaging, test driving, and deployment code to SCons. Compared to the ugly shell scripts I previously used, SCons is a lot cleaner, a lot faster, and significantly easier to manage.

I chose SCons for two reasons:

1. Simplicity. It's Python-based and super-easy to work with.
2. Familiarity. I've used it before, so already know my way around it.

Since I use the Google Closure Compiler to build and minimize my JavaScript code, I had to write a new builder for SCons. That turned out to be pretty straightforward to implement.

def js_builder(target, source, env):
  """ A JavaScript builder using Google Closure Compiler. """

  cmd = env.subst(
      "$JAVA -jar $JS_COMPILER --compilation_level $JS_COMPILATION_LEVEL");

  # Add defines to the command
  for define in env['JS_DEFINES'].keys():
    cmd += " --define=\"%s=%s\"" % (define, env['JS_DEFINES'][define])

  # Add the source files
  for file in source:
    cmd += " --js " + str(file)

  # Add the output file
  cmd += " --js_output_file " + str(target[0])

  # Log the command and run
  print env.subst(cmd)
  os.system(env.subst(cmd))

I also needed a new builder to stamp my output with the relevant build information. So, I created a Stamper, which is just a builder that runs some string substitution on files with sed. The stamper looks like this:

def vexflow_stamper(target, source, env):
  """ A Build Stamper for VexFlow """

  cmd =  "sed "
  cmd += " -e s/__VEX_BUILD_PREFIX__/$VEX_BUILD_PREFIX/"
  cmd += " -e s/__VEX_VERSION__/$VEX_VERSION/"
  cmd += ' -e "s/__VEX_BUILD_DATE__/${VEX_BUILD_DATE}/"'
  cmd += " -e s/__VEX_GIT_SHA1__/`git rev-list --max-count=1 HEAD`/ "
  cmd += ("%s > %s" % (source[0], target[0]))

  print env.subst(cmd)
  os.system(env.subst(cmd))

Before you can use these builders, you need to add them to your environment:

env.Append(BUILDERS = {'JavaScript': Builder(action = js_builder),
                       'VexFlowStamp': Builder(action = vexflow_stamper)})

Once this is done, you can add build JavaScript targets with the JavaScript command.

env['JAVA'] = "/usr/bin/java"
env['JS_COMPILER'] = "support/compiler.jar"
env['JS_DEFINES' ] = {
  "Vex.Debug": "true",
  "Vex.LogLevel": "4"
}

sources = ["src1.js", "src2.js", "src3.js"]

env.JavaScript("src.min.js", sources)

This really is just scratching the surface. There's a lot more you can do with SCons to automate and streamline your builds. To learn more, take a look at the user guide.

I added support for testing, packaging, and deployment (of the web pages and demos) to my SCons scripts in a matter of hours, and finally purged all my nasty shell scripts from the VexFlow codebase.

Give it a try. I guarantee you'll be happier.