Bash Function to Convert Error String to Vim Arguments

Vim Logo on FireI’ve always wanted a a bash function or alias to convert the beginning of an error string like “file:10″ into a vim editor command that would jump to the error line like “vim file +10″.

Finally, I wrote it. :)

To install on Linux and Mac OS X:

  1. save in ~/vil.sh
  2. choose the function name to use (like an alias) or just use the default, vil
  3. Mac OS X: in .bash_profile add source vil.sh, Linux: in .bashrc add source vil.sh
  4. and open a new login to start using it!
# Program: vil.sh
# Author: James Briggs, California, USA. 2014.
# Usage: vil vil.sh:6 or vil vil.sh:/James
# Version: 1.0
# Note: to verify that IFS is reset correctly: cat -etv <<<"$IFS"

vil () {
# bash function to translate "vil.sh:6" => "vim vil.sh +6"
oifs=$IFS
IFS=':' x=($1) # expand input string using : delimiter into array
IFS=$oifs
vim "${x[0]}" +"${x[1]}"
unset oifs
}

For those learning bash programming, the above function illustrates the most direct solution to parsing the arguments.

Using ( $var ) expansion is simpler than using a helper function call, read or for, as recommended on StackOverflow.

To learn more about advanced bash programming, read my cassandra top program.

keywords: bash, vim, vi, alias, function.

Posted in Linux, Open Source, Tech | Leave a comment

Avweb Articles on Aircraft Electrical Systems

Avweb has two very good articles on small aircraft electrical systems:

  1. When It All Goes Dark (Note that Swissair had unsafe entertainment system wiring and bundles were wrapped in metalized mylar (ie. flammable plastic), which aren’t small aircraft problems)
  2. Batteries — Dying Young

I once had a dead battery during daylight in a Cessna 152 in Kahoʻolawe, Hawaii, a fairly deserted airport with no regular maintenance 30 minutes across the strait from Honolulu.

I decided to hand-prop the plane. It’s a relatively dangerous thing to do for one person, especially a low-time pilot, but I took my time setting up the plane – throttle twisted tight just above idle, brakes on, and chocked on pilot side – and made sure my shoes were tied in case I needed to outrun the plane!

I phoned in my ETA to HNL Tower and used my handheld radio to listen to the tower instructions. Tower gave me a couple 90 degree turns upon radar contact to verify my compliance, and then gave me a runway to land on from Kokohead, about 5 miles.

That’s right – NORDO into a Class Bravo airport. :)

Everything turned out ok.

Without realizing it, I was the test pilot for my FBO, as I had the most hours there.

After I found out, I got serious real quick. I started reviewing the squawks and maintenance sign-offs before going up, and wearing a life-vest and carrying a raft anytime over the ocean.

I didn’t mind, as I was single and had the experience to evaluate the risks involved moreso than solo students. The planes were so run-out that I had no illusions about bothering to “save the airplane.”

The biggest surprises I got were with a plane that had new control cables and was “re-trimmed” without a new trim marking – almost stalled that one on takeoff, so look carefully and deliberately behind you on preflight – and another with a missing sparkplug, that made the panel unreadable due to vibration like a jackhammer.

Until this day, I usually make better Mx evaluations than instructors.

For example, at controlled airports, brakes on both sides are a must for hold-short instructions. If either pilot is “feeling soft”, I insist on a shutdown to find a mechanic.

Hand Propping a Cessna 150 (good demo – never forget that the propeller can be live at any time and stay out of the arc, so stand stable enough to avoid falling face-first into it.)
cnn.com: Passengers get out and push frozen plane in Siberia
CAP-21: Flat Spin and Prop Comes Off at Airshow

Posted in Flying, Tech | Leave a comment

SC14 Notes

Meanderful has some nice notes from SC14:

“Some of my old friends at Metamako were at SC14 next to the Xilinx booth with their compelling platform. If you want the fastest way to get two packets from two wires onto the one wire at 10G, say for facing an exchange, risk gateway or other financial choke point, such as a mandated firewall, then their new MetaMux 32 reigns supreme. It is the fastest way at ~100ns to get two packets onto the same wire at 1G / 10G speeds. This is in additional to their fancy-smancy layer 1 reconfig tricks. I think the biggest use case for me for Metamako is simply using their MetaConnect platform just for timestamping though. You could save a bunch of money using one of their MetaConnects, or perhaps even the MetaMux, to timestamp and collect packet information rather than using expensive timestamping cards…”

Posted in Business, Tech | Leave a comment

How Many Facebook Engineers Does it Take to Fix a Health Check?

Q: How many Facebook engineers does it take to fix a health check?
A: 35, apparently. :)

I’ve seen a similar problem when choosing which volume to write on in a global storage network. If you choose the healthiest *single* one, it will get crushed until it is marked as unhealthy by the next health check. Ad nauseum.

Using an algorithm like “pick a random volume from all the healthy ones” distributes the load (scales) much better. Only took 3 months to convince the original developers.

So the lessons are:

  1. beware of custom application-level load balancers because they’re often overly simplistic and wreak havoc
  2. it will take persistence to fix the problem, proportional to how short the health check code is because, “what could possibly go wrong?”

code.facebook.com: Solving the Mystery of Link Imbalance: A Metastable Failure State at Scale
Hightail buddies up with Cleversafe

Posted in Cloud, Linux, MySQL, Open Source, Tech | Leave a comment

Database Permute Indexes Utility

Perl LogoI wrote a perl program called permute_index.pl to autogenerate the list of index statement permutations for a MySQL, Oracle, Postgres or SQL Server query.

Typical query performance improvements of 100x to 1,000x are seen with reporting queries that use subselects or more than 2 tables. Not bad for 5 minutes work.

This is an example of the Monte Carlo method, applied to database applications. An interesting twist is that the samples are not random (they’re permutations), but the database optimizer behavior is treated as random – a reverse Monte Carlo method. :)

Some use cases:

  • accelerate existing Open Source (WordPress, Joomla) and commercial apps
  • test database optimizer results
  • stress-test databases with hundreds of index objects
  • verify DBA understanding of index behavior
  • as part of a continuous integration pipeline by diffing EXPLAIN output before and after
  • and the most interesting to me, improve performance of infrastructure tools that use metadata stores (Open Stack, CloudStack, DynamoDB, Bacula, etc.)

Summary

Autogenerate all valid index statements for EXPLAIN to optimize complex queries automatically.

Example

$ vi permute_index.pl (update user settings)

$ permute_index.pl | tee permute_index.sql
   alter table t1 add index idx_jb_001 (c1,c3);
   alter table t1 add index idx_jb_002 (c1,c2,c3);
   alter table t1 add index idx_jb_003 (c1,c3,c2);
   [...]
   alter table t2 add index idx_jb_007 (c4,c5);
   [...]

# make a backup of your production database and restore in dev or qa

$ mysql -h dev -u root -p test <permute_index.sql

$ mysql -h dev -u root -p test
  mysql> explain select * from t1, t2 where c1=c4 and c1=? and c2=? 
            and c3=? and c5=?;
  Table | Key
  ------------------
  t1    | idx_jb_002
  t2    | idx_jb_007

(Finally, drop the indexes except idx_jb_002 and idx_jb_007.)

Now anybody can be a Peter Zaitsev! :)

Posted in Linux, MySQL, Open Source, Oracle, Perl | Leave a comment