Customizing Buildbot Forms

I’ve been using Buildbot for several months to do both immediate and nightly builds of hardware RTL and software projects in svn.

Buildbot is fairly useful for what it is: a free, Open Source, standalone continuous integration (CI) tool that supports several revision control systems and has several display methods.

CI is very useful for doing automated building and testing of projects with more than one developer or department, or testing on multiple platforms. Being alerted to fix freshly introduced bugs is much easier than way down the road.

Buildbot is written in Python using the Twisted framework. The configuration files themselves are written in Python.

Most Buildbot administrators will also need to modify the Buildbot source code at some time to get the precise behavior they need.

Based on irc traffic on freenode#buildbot, I would say that most installations are customized, but that most of those changes are not contributed back.

Some of the reasons for that are uniquely local requirements, divergence from the trunk, and that usually the changes would be made by non-professional programmers – release or test staff – who modify just enough to get it running and don’t have any authority to release work products.

It’s generally recommended that programmers new to Buildbot start with the relatively simple status classes (irc, waterfall, grid) first to build up a reasonable comfort level before tackling the build classes.

(I started by enhancing the words.py irc class.)

My complaint about the build classes code is that the instance and class names basically all look the same – BuildXXX – so reading and tracing code is a challenge.

Recently I had to update the “force build” form to add a “Fresh checkout:” checkbox option.

Here’s the changes I had to make to update the form, submit the value and pass it to a script:

web/base.py:

+ make_row("Force fresh checkout:",
"<input type='checkbox' name='fresh' />")


web/builder.py:

from buildbot.process.properties import Properties

req = BuildRequest(r, s, builderName=self.builder_status.getName(), properties=Properties(fresh=fresh))


my_build.py:

from buildbot.process.properties import WithProperties

factories.addStep(ShellCommand, description="checkout",
command=['/shared/common/checkout', "/shared/common/work", WithProperties("%(fresh:-off)s")],
haltOnFailure=True)


An example of a common builder property is buildnumber, which you can pass to a script with:


... WithProperties("%(buildnumber)d")


To customize notifier emails, update status/mail.py like this:

for log in build.getLogs():
s = "%s" % log.getStep().getName()

m['Date'] = formatdate(localtime=True)
m['Subject'] = self.subject % { 'result': res,
'projectName': projectName,
'builder': name,
'step': s,
'build': num,
}

Then the step and build parameters can be used in the config files:

notifier = mail.MailNotifier(builders=["myproject"],
fromaddr="buildbot@example.com",
extraRecipients=["SW@example.com"],
subject="buildbot %(result)s in %(projectName)s on %(builder)s for build %(build)d - failed on %(step)s",
sendToInterestedUsers=True)


wikipedia: Buildbot
Google Code: buildwatch for Mac