Continuous Integration for Perl Applications with Jenkins

Jenkins LogoAt first glance, it might seem odd to setup a build environment for applications written in Perl, since Perl scripts aren’t compiled to a binary.

However, a build environment can be used for many other things besides making binaries:

  • checking out source code
  • syntax checking with perl -c
  • style checking with Perl::Critic
  • test coverage checking
  • spinning up VMs or cloud instances
  • running test harnesses on multiple test servers
  • doing performance testing
  • doing localization tasks and coverage
  • building XS or SWIG modules
  • generating documentation
  • archiving build artifacts
  • deploying successful builds.

Jenkins (formerly Hudson) is an Open Source (MIT License) project that provides a user-friendly, standard UI, API and framework for automating the above. It runs as an HTTP server on port 8080 by default.

Jenkins brings transparency to the build process, which helps in the communication between dev, qa and mgmt. It also improves continuity of software projects as employees join and leave a project.

It takes about an hour to install Jenkins, and less than a day to get it doing useful work.

If you need to install git support but repo updates for Perl modules are suppressed (common in cpanel or some commercial systems), then try this:

yum install git --disableexcludes=main

You can integrate existing build scripts into Jenkins using the Execute Shell job feature. (Just return zero for success and one for failure.)

Resources

dk-hostmaster: Perl and Continuous Integration with Jenkins

http://blog.cloudbees.com/2012/03/using-git-with-jenkins.html

http://blog.cloudbees.com/2012/01/better-integration-between-jenkins-and.html

Installing Datastax Cassandra and Python Driver on CentOS 5

Featured


Cassandra Logo

Cassandra can run on CentOS 5.x, but there is no yum repo support.

If you can’t upgrade linux distros, here’s how to install Datastax Cassandra Community Edition and the python cassandra driver on CentOS 5.x.

It’s not difficult, but there’s several steps, including updating java.

(The following steps would make a complete chef or puppet recipe for a non-SSL install with vnodes.)


# setup environment
groupadd -g 602 cassandra
useradd -u 602 -g cassandra -m -s /sbin/nologin cassandra
mkdir /var/lib/cassandra /var/log/cassandra /var/run/cassandra
touch /var/log/cassandra/system.log
chown -R cassandra:cassandra /var/lib/cassandra /var/log/cassandra /var/run/cassandra
mkdir -p /opt && cd /opt


cat >> /etc/security/limits.conf <<EOD
cassandra soft memlock unlimited
cassandra hard memlock unlimited
cassandra soft nofile 8192
cassandra hard nofile 10240
EOD


# upgrade java
yum remove java
# download, then install JDK 7.x from oracle.com
rpm -Uvh jdk-7u67-linux-x64.rpm
# download, then install recent jna.jar from https://github.com/twall/jna
mv jna.jar /usr/share/java
ln -s /usr/share/java/jna.jar /opt/cassandra/lib/
# update envariables
cat >> /etc/profile <<"EOD"
export JAVA_HOME=/usr/java/default
export JRE_HOME=/usr/java/default/jre
export CASSANDRA_HOME=/opt/cassandra
export PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME/bin:$CASSANDRA_HOME/bin
EOD


# get Datastax DCE
curl -L http://downloads.datastax.com/community/dsc.tar.gz >dsc-cassandra-2.0.9.tar.gz
tar zxvf - < dsc-cassandra-2.0.9.tar.gz
ln -s /opt/dsc-cassandra-2.0.9 /opt/cassandra
chown -R root:root /opt/cassandra/
bash cassandra/switch_snappy 1.0.4

# open cassandra firewall ports if necessary (not needed if using internal interface on most servers)
vi /etc/sysconfig/iptables
-A INPUT -i eth0 -m state --state NEW -m multiport -p tcp --dport 7000,7199,9042,9160 -j ACCEPT
service iptables restart
# configure /opt/cassandra/conf/cassandra.yaml (at least listen_address, rpc_address, seeds and tokens before starting server. If you need a do-over, clean the cassandra data with # rm -fr /var/lib/cassandra/*)

# download startup script:
wget http://jebriggs.com/php/start_cassandra.txt -O /etc/init.d/cassandra
chown root:root /etc/init.d/cassandra
chmod 755 /etc/init.d/cassandra
chkconfig --add cassandra

# start cassandra server (if it is standalone, or a seed server. otherwise start after the seed servers):
service cassandra start

# cat /etc/redhat-release 
CentOS release 5.10 (Final)

[root@www1 conf]# nodetool status
Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address   Load       Tokens  Owns   Host ID                               Rack
UN  10.0.1.2  71.87 KB   256     66.8%  8302c6d5-4c88-4695-bbf4-762bc7f24544  rack1
UN  10.0.1.3  136.63 KB  256     69.9%  eddb03b2-98d3-46ff-be63-95435414a883  rack1
UN  10.0.1.4  100.08 KB  256     63.3%  2a8dde5e-29b0-4a67-8204-40769376c44a  rack1

If you only see the node on localhost, then you have a problem:

  • read and fix any errors in /var/log/cassandra/system.log until there are zero errors. snappy-related errors are from /tmp being noexec or not running the switch_snappy 1.0.4 command above.
  • disable iptables firewall, test and reenable later
  • in log4j-server.properties, increase log4j.rootLogger to DEBUG
  • if you have multiple NICs, JMX (ie. nodetool) can bind to the wrong interface. You likely need to configure the-Djava.rmi.server.hostname=[address] option in cassandra-env.sh - to the address you want to listen on
  • public/private IP address problems in AWS EC2. You may need to set broadcast_address: [public_ec2_address]
  • normally rmiregistry is not needed unless you have some atypical firewalling or routing (NAT.)

Datastax Opscenter 5.0

You can install the binary from yum or tarball, but the important things to know are:

  • the monitoring agent will be installed on each cassandra node and uses port 61621. The init script is called datastax-agent.
  • the UI only needs to be installed once, but needs ports 61620, and 8888 for HTTP.
  • to allow Opscenter to remotely manage nodes with ssh, remove old ssh entries from .ssh/known_hosts first, connect manually to each node, then Opscenter should be happy
  • by default, Opscenter listens for agents on 0.0.0.0, phones home to Datastax.com each day, and does not require web authentication, so you likely want to change those.

Python also needs to be upgraded if you want to use cqlsh or the python client cassandra driver.


# install python 2.6 and dependencies
yum install gcc python26 python26-devel libev libev-devel


# install python's pip module
curl --silent --show-error --retry 5 https://bootstrap.pypa.io/get-pip.py | python26


# install cassandra driver for python
pip install cassandra-driver


# install blist.py
tar zxvf - < blist-1.3.6.tar.gz
cd blist-1.3.6
python26 setup.py install
cd ..

# cluster.py - test installation

from cassandra.cluster import Cluster

cluster = Cluster(['127.0.0.1'])

def dump(obj):
   for attr in dir(obj):
       if hasattr( obj, attr ):
           print( "obj.%s = %s" % (attr, getattr(obj, attr)))

dump(cluster);
# python26 cluster.py

obj.__class__ = <class 'cassandra.cluster.Cluster'>
[...]

Troubleshooting connection problems in JConsole
datastax.com: Storing OpsCenter Data in a Separate Cluster

Napa Earthquake 6.0

I was awake in Sunnyvale around 3 am during the 6.0 “Napa Earthquake” (officially the “American Canyon Earthquake”.)

I knew immediately it was over 5.0 because the walls were moving and groaning, for about 30 seconds.

Napa suffered almost 200 injuries and several historic brick buildings were heavily damaged due to the shallow quake only a few miles away. 911 was briefly overwhelmed, 64,000 were left without power and fire broke out.

Thousands of bottles and barrels of wine fell down.

USA Today: 120 injured, 3 critical after Napa, California quake

Bay Area Light Sport Aircraft (LSA) Rentals

Light Sport Aircraft (LSA) have two advantages:

  1. lower rental cost and lower fuel burn, typically around 4 gallons/hour
  2. can fly with “drivers licence medical” instead of FAA airman medical.
Airport FBO Airplane Tail Price (Block)/hr Verified Notes
RHV AeroDynamic (formerly Amelia’s) 2005 Zlin Savage Cub     2014-09-15  
RHV AeroDynamic (formerly Amelia’s) Evektor SportStar N712MF $104 ($94.55) 2014-08-01 544 useful load, 17.2 fuel
RHV Tradewinds Remos GX N113GX $108 2014-08-01 650 useful load, 22 fuel
RHV Tradewinds Remos G-3 N188WA $108 2014-08-01 620 useful load, 21 fuel
PAO WVFC Cessna 162/G N5207R $109 2014-08-01 456 useful load, 24 fuel, G300
PAO WVFC Cessna 162/G N5207Z $99 2014-08-01 461 useful load, 24 fuel, G300
SQL San Carlos Flight Center Cessna 162 N5210H $125 2014-08-01 449 useful load, 24 fuel, G300
KWVI AeroDynamic (formerly Amelia’s) Aeronca Champ 7AC N2791E $81 ($73.64) 2014-08-01 459 useful load, 18 fuel

wikipedia: Light-sport aircraft

Other Resources


San Carlos Aviation & Supply Inc. (San Carlos Pilot Shoppe) – Open weekends
Crosswind Training Simulator at SQL

If you have any comments or corrections, please leave a comment.

Garmin GPSMap 96C GPS Updates

Garmin 96C
Originally, the Garmin 96C shipped with a 2006 GPS database.

The Feb. 2011 obstacle database size increased enough to require a new version of the software, 3.0.0.

The 96C software update steps are:

  1. connect USB cable (regular to mini) from 96C to Windows computer. The 96C USB port is located underneath the rubber circular cover, while the narrow oval cover is for the external antenna connector. Both covers can be easily pried up without removing the screws. Almost all Garmin GPSes use the same USB cable. Turn GPS on.
  2. you will be alerted if you need to install a Windows USB driver, if not download USB driver (free download) from Garmin and install first
  3. Backup custom waypoints, flightlog, etc.
  4. Install 96C Version 3.0.0 software (free download from Garmin)
  5. verify successful software installation (choose Menu … System … Software Version). It should say 3.0.0.
  6. Purchase new map database from Garmin ($50) or buy a coupon from ebay ($40.)

Garmin Aviation databases are updated every 28 days.