Archive for the ‘Perl’ Category

Parallel SSH Command Execution Utilities

Sunday, November 2nd, 2008

I usually work on clusters of linux servers, so it’s handy to submit remote commands in parallel to the whole farm.

I have my own map.pl script for doing that that has different options for serial and parallel execution, immediate and sorted output, and timeouts.

Logging of commands and performance information might be a good idea.

I find it handy to maintain a small script that can be customized for different scenarios, especially if you maintain the server host and status information in a database.

Several Open Source projects have also sprung up.

Minimalist ssh-like command line tools:

Less minimal Distributed Virtual Terminal utilities:

SourceForge: search for “parallel ssh”
Parallel SSH execution and a single shell to control them all
pssh: Run Command On Multiple SSH Servers
Bitmover BitCluster

OSCON 2008, Portland

Friday, July 25th, 2008

I attended the O’Reilly Open Source Conference, once again in Portland, Oregon.

Overall my impression was that the talks and vibe were oriented towards Web 2.0 primarily.

I would say that the talks were not as strong as previous years, but it’s easy to compensate for that with the “hallway track” and access to the original Open Source authors.

Several attendees used the EEE sub-notebook computer, and were happy with it as a email/browser tool.

Wednesday

PHP Taint Tool: It Ain’t a Parser

- CS’y effort at PHP parser for code analysis, reminds me of early days of Perl’s B tools
- not suitable for end-users

Write Beautiful Code (in PHP), Laura Thomson, Mozilla

- good general background on good programming practises
- not a lot of specifics about PHP, but available for questions

Hypertable, Doug Judd, Zevents

- HyperTable is a clone of Google’s BigTable, from public paper
- room was packed, some turned away
- still alpha, maybe beta in August
- preferred distributed filesystem is HDFS, works with others
- I recommend reading web site and then looking at the curt slides
- plans to do benchmarks with same hardware as Google has published.

Open Source Virtualization for People Who Feel Guilty About Using VMware So Much, andy michelle, EDA

- cute talk about VirtualBox, Xen and VMware
- Xen has weird nomenclature compared to other tools
- VMware wins on tools and polish
- showed screenshots of unreleased and alpha mgmt. tools.

Barely Legal XXX Perl, Jos Boumans, RIPE

- stunning and twisted example of overloading, short-circuiting, import-faking, whatever it takes to make a loaded module do something other than intended
- illustrates great flexibility of perl, for good or ill
- could be useful for things like testing harnesses, etc.
- motivated to win bet of $100 or 1 vertical meter of beer
- said it took 3 or 4 hours to complete.

I walked around the exhibits area.

Got a demo of Atlassian’s continuous integration (CI) tool, Bamboo. They’re also the vendors of JIRA issue tracker and Confluence wiki, which I’ve used before.

One company had a public Wii game happening.

Thursday

Scaling Databases with DBIx::Router, Perrin Harkins

Ultimate Perl Code Profiling, Tim Bunce (Shopzilla)

- talk and screenshots about NYT perl profiler


The New York Times Perl Profiler

Top 10 Scalability Mistakes, John Coggeshall (Automotive Computer Services)

- good overview of writing high-performance, maintainable Internet systems
- interesting opinion that scalability is not just about increasing performance. scalability can be about scaling up or down, performance or maintainability, etc.
- recommended php.ini settings list

Perl Lightning Talks

- popular with audience, attendees seemed to like all the talks
- Mail::ESMTP looks very interesting for testing and production

Code is Easy, People are Hard: Developing Meebo’s Interview Process, Elaine Wherry (meebo)

- struggled to find time, right approach to interview new candidates in 1996, likely at behest of VCs
- external recruiters hit-and-miss, conferences and jobs email link useless
- phase where non-founder employees doing interviews wanted a founder involved in interview process
- trying to preserve culture (finger rockets, social networking, 2 female founders, etc.)
- came up with process involving reading resumes, phone screens, and office “sim” that adds a new candidate within 3-6 weeks
- “sim” has 3 versions: office manager (plan to erect a meebo office sign), front-end engineer (write a JavaScript app), and back-end engineer (write a server) in 4 hours
- current goal is to keep interview time down to 8 hours per candidate over 10 days
- now up to about 40 employees
- my feeling was that their hiring process started off clueless due to inexperienced mgmt. and is still oriented towards junior engineers. Silicon Valley is full of expert engineers and it doesn’t take 8 hours to interview them.

BOF

mysql-sandbox

Giuseppe Maxia discussed and demoed his very useful mysql-sandbox utility for managing several versions and instances of MySQL on the same machine.

He wrote it for his testing work at MySQL AB. Very well received by attendees. This is a great example of what I call “anti-virtualization” - using ports instead of resource-intensive VMs.

MySQL Conference 2008 Presentation

State of the Onion Address, Larry Wall

- talk about Perl6, random anecdotes, etc.

Friday

Open Voices, Jim Zemlin (The Linux Foundation), Keith Bergelt (Open Invention Network), Karen Sandler (Software Freedom Law Center), Phil Robb (Hewlett Packard)

- panel discussion of various free software efforts, some little-known

An Illustrated History of Failure, Paul Fenwick (Perl Training Australia)

Paul gave an interesting talk on notable Software Failures and estimated a price tag for each. I had heard news reports of many of them, but it was interesting to hear an updated analysis of what really happened behind the scenes.

Thanks to Google for sponsoring the fairly good almost-gourmet lunches. Sure beats the O’Reilly lunchbags from the dot bomb days. (Everybody I know bailed and found a subway shop back then.)

Notes

- Burgerville popular with attendees, can upgrade combos to a shake.
- Red Lion hotel has a small cardio gym with 1 universal machine, no free weights, open til 11 pm
- WiFi password changed weekly, in middle of remodel, lobby just finished.
- There is a 24-Hour Fitness that is actually open 24 hours near downtown Portland. Has basketball court and 2-lane pool. $15 for non-member visitors.

OSCON 2008 Presentations

YAPC 2008 Chicago

Friday, June 20th, 2008

Once again I attended the Yet Another Perl Conference (YAPC), and again it was at IIT in Chicago (same as in 2006.) Josh McAdams and his wife did a great job organizing the conference.

YAPC is an affordable ($100 conference fee) organized by volunteers for The Perl Foundation (TPF).

I’m already an experienced perl programmer, but perl is a vast programming environment and one can always learn more about techniques or available modules.

After the 3-day YAPC, I went to the 2-day Perl Catalyst framework class.

Overall, I would say that the talks were not as technical as previous years, but with 3 tracks there was always something interesting.

Many people make up their own “hallway track” anyway, since most of the perl heavyweights come each year and are very accessible.

The IIT dorm was only $60/night, but even that was over-priced. Some investment is needed in maintenance, and the attendants need to actually hand out linens and control the AC next time.

Although there was supposed to be an online form to add cash to the access card, one has to go to 201 Hermann Hall while they get organized.

Here’s my notes on some of the events that were memorable:

Monday

Tiny Modules, Adam Kennedy

- no dependencies on other modules
- fast to load
- fast to run (near real-time)

Config::Tiny (popular module)
XML::Tiny
Object::Tiny
Date::Tiny

Moving to mod_perl2, Jim Brandt

- Apache2::Compat can be used for backward compatibility
- some methods have different arguments now though
- loads everything, so uses a lot of memory
- slower because some code is now Perl instead of C
- content_language, write_client, send_http_header, get_remote_addr, etc.

Porting Tools

- Apache2::Reload
- Apache2::porting

Also read your error log and the Migration manual.

Apache::Registry is now Mod_perl::Registry

Photo Processing for the Web, Kent Cowgill

kentcowgill.net

- bunch of stuff for managing cell phone photos
- speaker talked about various image processing and mgmt. problems with his old nokia cameraphone
- embed iso in a pdf
- bought a real camera, problems went away

PAR+FUSE+PDF, Chris Dolan

Tuesday

HTML::App Framework, Jim Krajewski

Catalyst, Matt Trout

- a profane overview of handlers
- 490 CPAN Catalyst modules

Catalyst Downsides

- need packager for catalyst apps
- attribute syntax
- unaccelerated CGI not great (lack of persistence, slow to start)?

Dinner and Auction

- quite a variety of food: mediterranean, italian, indian, american
- dozens of books and t-shirts to bid on
- Wii games

Wednesday

Perl Lightning Talks

swish-e

- command line search tool
- now has perl interface, solid

cons

- no utf8
- not pi
- no index updates
- swish3 should fix that

joshr.com/src/docs
linux journal

where2getit.com
- AJAX maps with mod_perl
- openlayers, prototype, scriptaculous
- rewrote 100kloc old perl app into 22klog perl plus JS

Chemchains Sandbox

- boolean logic to understand and visualize myriad possible chemical reaction pathways

Math::Combinatorics

- works at bookfinder.com
- generate test data on authors using perl, then test clustering techniques

Devel::Cover::TT

Ingy strip show

Do You Believe in the Users?, Brian Fitzpatrick and Ben Collins-Sussman

- slide deck suggesting that developers focus on end user experience
- interesting graphical line added to most graphs accounting for programmer pain/cost

The Perl Foundation (TPF) Keynote, Richard Dice

Nokia 810

I talked to a fellow field-testing a Nokia 810 and keyboard as a notebook replacement before his next trip. He seemed pretty happy overall. He said he had to do a couple days of setup to get it working to his liking.

Thursday and Friday

Catalyst Class by Jonathan Rockway in association with Stonehenge

- Jon wrote a book on Catalyst and is a core catalyst programmer, less active at committing now.
- class actually a busy 2-day lab, not a lecture
- install Catalyst from CPAN (65 minutes!)
- also went over DBIx and sqlite
- modify various sample programs, like a small wiki and address book.

Thanks to the many corporate sponsors.

Twiki Meetup in Santa Clara

Thursday, November 29th, 2007

There was a meetup tonite for Twiki users from 5:30 pm to 8:00 pm at the Plug and Play Tech Center, 440 North Wolfe Road, Sunnyvale, CA 94085.

I attended most of it, though 5:30 pm is pretty early for most people to leave work and drive there. Nonetheless, turnout was good, with over 30 audience members plus staff from twiki.net, the company providing support for Twiki.

The format was a slide show, followed by a very energetic community evangelist who got the audience involved.

Several members of other local Bay User perl and linux user groups dropped in.

Twiki is notable in offering many plugins that combine to create a very feature-rich wiki. For example, it’s possible to embed twiki spreadsheets, forms or do programming in twiki pages.

I’ve used twiki, confluence, mediawiki and trac. I’d say twiki is my favorite for complex wikis.

The pizza was not great, although it was nice of them to serve both soft drinks and wine.


Plug and Play Tech Center, Sunnyvale

Defcon 15, Las Vegas

Sunday, August 5th, 2007

I gave up on Defcon after Defcon 11 because of the venue overcrowding at the Alexis Hotel. But since I had no other plans this weekend and they moved to the larger Riviera hotel last year, I decided to give them another shot this year.

What an improvement! 7,000 attendees and 5 talk tracks, yet now fairly well-organized. I registered Thursday at 9:30 am for $100, but they had already run out of the awesome white animated SMT LED badges with a “HUMAN” cut-out conference badges, and programs.

They should have done a photocopy run of the schedules, but didn’t, which is bad since the schedules are not posted outside the lecture rooms.

Security staff in red shirts called “goons” vigilantly enforced access points and fire codes. I was told that lecture room exits had to be clear in case of sudden emergencies like a smoke bomb attack.

The hotel was big enough that the attendees didn’t swamp the hotel, and the rooms for Tracks 2, 3 and 4 were usually big enough. The Track 5 room and the closing ballroom were too small. Hotel security seemed happy and stayed in the background.

The hotel coffee shop was very busy but served ok food ($10 for a hamburger and fries.) The upstairs dinner buffet was a good deal at $16. Good variety of fresh food and desserts, including prime rib, mexican and asian food.

I didn’t bring a notebook computer this year because it would just get hacked if I turned it on, and I also left my Blackberry off. I understand that some people buy a computer at Fry’s and return it after the weekend to get re-imaged, or use it as a honeypot. Some people did use their work Blackberrys with bluetooth disabled. One guy had a Nokia 770 wifi PDA that he planned to re-image after going home. Nice screen with scalable fonts.

The talks that I attended on Thursday and Friday were very strong, usually presented by the original researchers or somebody deeply involved in the topic. Defcon talks are unusual in that most audience questions are held until after the talk in a separate Q&A room. Not my preference, since expert attendees often get more out of the questions that the talk.

Thursday

Thomas Holt: The Market for Malware

Insight into mainly Russian malware industry:

- pincher programs for intercepting username and password data
- joiner programs to bind pincher program payloads with images or downloads
- like to be paid with e-gold, don’t like Western Union
- like ICQ, irc
- tools cheaper for other Russians to purchase than foreigners
- forums for promoting and rating developers and programs
- good authors provide good customer support, upgrades ($10), manuals and customization ($30)
- admin UI programs very polished and professional looking - some are even skinnable.

Pilgrim: How to be a WiFi Ninja

Pilgrim is the real deal - he knows how wifi works, owns a wifi accessories shop in Florida and is a perennial show vendor.

He gave tips on improving wifi transmission and reception:

- thinner cable is lossier, so keep under 10′
- cable is optimized for Channel 6
- wifi signals transmit better in drier air
- most omni AP transmitters can be made more directional by using a tin-foil reflector behind them, preferably parabolic shape
- used Dish satellite receiver antenna could be very useful
- made a wok dish antenna and recommends it
- recommends USB receivers over PC Cards because of external antenna
- recommends USB cable run to smart antenna instead of long runs of cable
- transmit power isn’t everything. try to balance transmitter, receiver, cable and geometry
- re-orient AP antennae to get better vertical or horizontal reception, especially in 2-storey buildings. same when war-driving.

Broward Horne: Click Fraud Detection with Practical Memetics

Broward gave a great talk.

He has the website RealMeme.com and does experiments in web site promotion and Internet mindshare. He left some blog comments on Casey Serins’ IAmFacingForeclosure.com website but received no traffic to his site initially. He posted a comment inquiring about that, got a bunch of traffic, and upon log analysis realized that it was bot traffic, implying that Casey was involved with bots for AdSense click fraud.

He showed some graphs of discussion activity before and after the Pope’s death, which expanded the bandwidth of discussion, and the SARS outbreak, which barely registered.

D.J.Capelis: Virtualization: Enough holes to work Vegas

Awesome talk on how pathetic x86 virtualization is from a security perspective.

He talked mainly about VMware Server and Xen, but problems generally applicable are:

- vulnerable to physical attacks and DoS at PCI level for shared hardware like video, network and drive controllers
- vulnerable to IP and MAC address changes
- vulnerable to practically undetectable covert channels between VMs
- vulnerable to timing attacks similar to the Intel HT ones
- all the image migration tools use plaintext, possibly across ethernet
- any rogue partition can violate all other partitions, subverting your firewall and network security
- bad default configurations, as documented.

The expensive VMware ESX product fixes a few but not all of the above problems.

He’s hoping IBM can leverage their 30 year virtualization experience on LPARs to do a good job.

He released a script to somewhat improve the default security configuration of VMware Server.

Dave Josephsen: Homeless Vikings, (short-lived BGP prefix hijacking and the spamwars)

He presented a history of spam and countermeasures timeline with commentary (he likes content filtering and thinks anything else is just a pointless technical arms race that can’t be won by the good guys).

Then he talked about how BGP can be used by spammers to spoof address blocks or commandeer unassigned IP space, likely the same techniques used by intelligence agencies now.

Gadi Evron: Webserver Botnets

Peter Gutmann: The Commercial Malware Industry

Some repetition of the talk at 10 am, but with more detail.

Daniel Peck & Ben Feinstein: CaffeineMonkey: Automated Collection, Detection and Analysis of Malicious JavaScript

They demonstrated some utilities for de-obfuscating javascript malware and presented some graphs that illustrated how malware and legitimate javascript profile very differently.

Also, they talked about spidering and analyzing some web sites and being surprised at how clean myspace is for example - no JavaScript malware found, probably a credit to their staff.

atlas: Remedial Heap Overflows: dlmalloc style

atlas did a Linux Buffer Overflow 101 class.

He used python to inject the shell code.

All the hotels surrounding the Rivera were full, so I stayed in the Hilton Vacation Getaway Hotel, a moderate walk from the Riviera. My $169 room was a very nice and new suite with a 30″ HDMI plasma TV, jacuzzi, shower, bedroom TV and laundry. The downstairs deli tuck shop is very complete and you can order custom sandwiches there. There is also an outside grill with $5 hamburgers and $4 hotdogs that’s open for lunch. The basement business center is 24 hours and has computer rentals and printing for $1/page. The hallway vending machines have $1 sodas.

Friday

Brendan O’Connor: Greater than 1: Defeating “strong” Authentication in Web Applications

Excellent talk reviewing US online banking so-called strong authentication, then attacking it.

- in-person banking is 2-factor authentication (something you have (card) and something you know (PIN)
- online banking is not 2-factor (you know a PIN but normally you don’t provide card, token or biometrics)
- browser fingerprinting is pointless because everybody buys the same configurations from Dell or HP
- browser fingerprinting is pointless because the implementations are bungled (commented source, little effort)
- banks should display all recent logins, not just the last one
- bank should not star out account numbers, then display the full check thumbnail!
- bolt-on auth systems from 3rd-party vendors weaken overall security and increase the attack surface
- SiteKey is worthless, since they have a limited image catalog indexed by alt tag
- knowledge base questions based on public databases as implemented now are worthless, but could be improved by displaying the same question until correctly answered and not randomizing choices

He finished by demonstrating a MITM attack by writing a newbie-level Perl program to relay the browser fingerprint, setting up Defcon Bank and doing a MITM attack on his personal bank which uses Sitekey.

David Byrne: Intranet Invasion With Anti-DNS Pinning

He discussed DNS pinning issues with IE and Firefox, pinning in Java and also how LiveConnect in Firefox and Opera reduce pinning. Also he showed how to use an exploited browser as a web or socks proxy and talked about using the socket capabilities in Flash 7 and above.

In his demo, he owned somebody’s browser, ran Nessus 3, and started a shell.

Billy Rios & Nathan McFeters: Biting tha Hand that Feeds You - Storing and Serving Malicious Content From Well Known Web Servers

Billy and Nathan are the reason for the recent Firefox 2.0.0.5 and 2.0.0.6 updates.

They talked about:

- XSRF
- serving warez from webmail hosts, in particular Yahoo! and gmail, because they’re free anonymous accounts, have a large storage capacity, good network bandwidth, high-reputation domain names, and plausibly deniable.
- domain substitution
- what can you trust on the Internet? only the domain name in your location bar
- Flash settings XML config file
- browser scheme and %00%00 filetype handlers
- IE 7 and Firefox URI hand-off exploits
- possibly KDE registry might also be vulnerable to filetype handler issues.

The award ceremonies went on 2 hours. It was interesting to learn about the whole Defcon community: security, logistics, press, events, speaker coordination, etc.

40 hardware kits were handed out to people wanting to hack the badge, but only 7 submissions resulted. The 2 winners built a graphical, gray-scale multimeter and a pong game. DT suggested it would be hard to top the badge next year, unless it was converted to a fibrillator or laser beams were added.

A 17 year-old won one of the lock-picking categories. The overall winner mentioned preferring home-made tools.

Some of the award winners received a Black Badge - good for free life-time show admittance. Some got a used notebook or Dish receiver.

The trivia show winning team was booed for their poor result - sometimes needed a dozen clues and still getting the wrong answer - but still got black badges. Next year there will be a pre-qualifying test.

I took one of the airport shuttle buses back to the airport. Depending on how you look at it, either I got a free ground tour of Las Vegas, or they wasted a half hour of my time trying to find a passenger who booked 24 hours in advance but didn’t show up on time. Eventually they found him … back at the Riviera.

theinquirer.ne: How to break forensics software
GData: An Online MD5 Hash Database

OSCON July, 2007 - Portland

Friday, July 27th, 2007

I attended the O’Reilly Open Source convention again, making it 10 years in a row. Once again it was held at the Convention Center near downtown Portland, a convenient light rail ride from the airport.

Like many experienced developers, I spent a lot of time in the “hallway track” talking to other developers and users, as well as in one of the 15 simultaneous talks.

The general consensus was that the talks were not as strong as in previous years (not even compared to the MySQL conference this year), but it’s worthwhile to me if I can get even one juicy nugget from each talk, or gain an understanding of a developing trend in programming or system administration.

Many of the presenters griped about there not being enough time to look at source code in a 40 minute talk.

For those who want a conference summary in a nutshell:

  • OpenID is popular
  • lucene and its REST interface have more mindshare than projects like Kinosearch, language-specific bindings, etc.
  • Yahoo! released the yslow browser plug-in for front-end performance evaluation
  • Perl: no ORM appears to be gaining the upper hand, though DBIx is respected. Tim Bunce would like to see a wrapper around JDBC for each scripting language. Alison Randal is updating the Perl license.
  • PHP: no good way to do vector reporting graphics, especially since IE doesn’t support SVG and Adobe is killing the Macromedia plug-in in December. PHP4 is being EOL’ed 8/8/8 so that the PHP developers can focus on 5 and 6 only.

Google was heavily recruiting at the conference. I ran into 3 recruiters, and there were even more in the Google booth.

Pretty good food for lunch, usually chicken or fish in some kind of red sauce with steamed veggies. Better than the wilted sandwich boxes from previous years that mainly got tossed out.

Tuesday nite

I arrived at the Convention Center in time for the evening Google Open Source awards. Happened to sit next to Zak and the 20 year-old OpenID guy, David Recordon, who won $5,000 and a colored, transparent, angular plastic trophy and base that we had fun stabbing each other with.

The OpenID Foundation is offering a $5,000 bounty to the first 10 OSI-approved projects that add OpenID support. Many programmers were busy adding it, including SocialText and others. (David works at Verisign.)

I walked over to the Doug Fir Lounge with a few guys, 2 of them Austrian. I had the halibut fish and chips and lemonade for $20 including tip. It was ok. They have a log cabin motif happening with a restaurant, patio and bar upstairs, and dance club downstairs, so ID is required to enter. Open from 7 am to 2:30 am every day, 1 503 231 WOOD.

Wednesday

Nagios

- general overview of features
- Event Broker most powerful, least used

Bigger and Faster
Rasmus Lerdorf

Rasmus did his usual “PHP is as secure as any other language”, and “pick on a PHP app and make it go faster” talk.

He said he’s still not a Y! Paranoid, but his work does often touch on PHP and web security.

He used to use httpload, but now prefers siege for load testing because it has support for cookies.

http://developer.yahoo.com/yslow/
Live HTTP Headers
APC

sla.ckers.org/forum/list.php?3
php.net/filter
xdebug.org/docs/profiler
talks.php.net/show/oscon07

xdebug
jeremiah san diego xss console author
scanmus.corp.yahoo.com

PHP and Ruby Envy
- NZ programmer on Silverlight CMS (BSD licensed)
- own object system in PHP5
- Ruby less available on web servers, less mindshare
- rolled his own PHP OO frameword apparently

Exhibits

- talked to Mark Finkle of mozilla.org
- said hi to Larry. He had the whole family there.

Trac
Vivek Khera

- he uses RT for public tickets, Trac internally
- doesn’t require much resources since only a few developers
- Trac is used on many Ruby/PHP projects
- gives you wiki/tickets/etc.
- modified BSD license

Afterwards mentioned:

- uses Trac in a BSD jail
- an alternative to Trac would be basecamp (or I guess Sourceforge software). See slashdot.org threads for more ideas.
- likes pfSense firewall as an appliance
- nagios alerts too much, and no good rule builder for multiple hosts
- own web framework called Rowdy (RWDE)
- software as complicated to install as RT should be treated as an appliance
- he submitted 6 related talks on software development environment, only 1 accepted.

Steve Souders
Chief Performance Yahoo!
souders@yahoo-inc.com
Exceptional Performance Group

http://developer.yahoo.com/performance

- IBM Page Detailer Pro
- yslow (crawls the DOM, not a packet sniffer)
- firebug
- jslint - The JavaScript Verifier

80-90% of end user response time is spent on the front-end. so optimize there.

14 Rules for a Better User Experience

1. make fewer HTTP requests
2. use a CDN
3. add an Expires header
4. gzip components - even JS and CSS
5. CSS at top
6. JavaScripts to bottom
7. avoid CSS expressions
8. make JS and CSS external
9. reduce DNS lookups
10. minify Javascript
11. avoid redirects
12. remove duplicate scripts
13. configure Etags - disable in most cases if load-balanced or multiple web servers
14. make AJAX cacheable

move JS to onload
remove bottom tabs
avoid redirects
images sprites
expires

Thursday

PHP Graphics
Luke Welling, OmniTI

Luke presented an overview of raster and vector graphics modules for PHP.

He prefers vector graphics, but there’s not many free options for doing that.

He feels that Yahoo! Finance and Google analytics sites are state of the art in presentation graphics with anti-aliasing, interactivity, esthetics, text and maps. I’d say that’s aiming a little low, but it’s a start I guess.

Perl Lightning Talks

  • Vani Raja did a talk on Yahoo! JS
  • Ask did his talk on qsmtpd again
  • talk on Test::More 3?
  • talk on task lists for hit and run volunteers
  • Schwern did one talk on making tea for 5 minutes, and one on “Blame Schwern” - just do it instead of waiting for permission
  • Tim Potter did a talk on a messaging standards effort for his employer, saying that the ANSI process was too slow and looking for an alternative
  • Andy Lester did a talk on ack
  • a talk on SVN::Notify
  • http://angerwhale.org/
  • Tim Bunce talked about DBD::Gofer Proxy and next-gen cross-scripting language DB API based on JDBC API
  • guitar song about #perl

YouTube: Perl Lightning Talks on Handycam by Schtonk

Perl Auction

Larry’s talk on comparative languages and Perl6. Sounds like we’ll be able to do something like foreach (1..infinity).

Full Text Search BOF
Peter Zaitsev
- based in London, England but often in SV
- uses Sphinx on several servers
- http://boardreader.com/ one TB of searchable data
- geneology is big on full-text search

- after insert, mysql full text gets slow, run optimize.
also, doing it at insert time causes index update per keyword
- gin or gyst for Postgresql 8.4?
- Michael Kimsal, SOLR
- hard disk space is free (enough for whatever indexing is required)
- mostly news search involves last 5 minutes of feed
- MessageOne stores email for lawyers to mine. They like to search, archive and expire. Mostly Exchange lusers, rarely Unix admins.
- Lucene and REST interface
- Monty says MySQL AB hired a programmer to work on search, but he’s working on another project now. They need somebody with a burning desire to make progress in an area like that, but they recognize the importance of search.
- Monty poured out free Finnish chocolate rum from a Pepsi bottle that was so powerful it scared most people. He said it was banned for 2 years in Finland because it was so addictive.

Sun BOF

- audience talked to senior Sun staff about Java and Solaris a little.
- free beer, cheese and crackers.

Friday

A bunch of Postgres people went to the Portland wine tasting on the river event in the afternoon.

Call for Software Whiteboard

OSCON07 Call for Software Whiteboard
flickr.com: Jeff Kubina’s OSCON 2007 Whiteboard set of tiles

FrontBase RDBMS

Friday, July 6th, 2007

I came across the Frontbase RDBMS recently. Their 288-page users manual was so nice and clear I decided to read through it.

The FrontBase project was started in 1996 for Copenhagen’s 911 service (MySQL has similar practical origins, written to support retail data warehousing contracts). They claim it’s a “database designed for the Internet” and requires zero maintenance. Currently they’re on version 4.2.

It’s a proprietary, closed-source database written in ANSI C, but free for most users. They currently charge for support and consulting, and sell licenses for embedded users. The FBAccess client library source is available on request for programmers to add support to their oses and devices.

It seems to have traction in the Mac community, likely because of the nice Mac client program and FileMaker and RealBasic support.

What’s nice is that it supports transactions and full-text search at the same time, unlike MySQL.

FrontBase supports:

- servers: Mac OS X, Linux, Windows, Solaris, FreeBSD
- clients: FrontBaseManager Aqua client for Mac OS X allows mgmt. and monitoring, also web, common scripting languages (Perl, PHP, Tcl), JDBC, ODBC, and the sql92 console client
- SQL92
- Unicode
- statement-based replication from transaction logs
- full SQL logging options, allowing replay (SET WRITE SQL TRUE GLOBAL;)
- full-text indexing
- stored procedures
- clusters
- table spaces
- raw device driver
- live (hot) backup due to versioning (WRITE DATA;)
- data and communications encryption
- whitelisting and blacklisting of IP addresses
- row-level locking
- row-level privileges
- read-only databases (note that replication slaves are read-only)
- import utilities for Access and FileMaker
- migration tools for FileMaker (tables only) and MySQL (no enum or set column support)

They have been changing their licensing, so contact gclem at frontbase.com for the latest info.

FrontBase sure looks like a clean, modern RDBMS implementation.

I plan to learn more about its reliability and performance.

I have noticed that it is prone to crashing on Mac OS X PPC. Even a simple query like SELECT DISTINCT(OID) FROM TABLE; on 500,000 records with mostly nulls can crash the database, but hard to tell if it’s Frontbase, the OS, or what.

YAPC::NA 2007 Perl Conference, Houston

Saturday, June 30th, 2007

I was at the YAPC::NA 2007 Perl Conference this week at the University of Houston in Texas.

Once again, it was a great conference, with about 200 people attending.

It was my first time in Houston, and boy was it humid. My eyeglasses fogged up every time I went outside. It might be the most humid place on earth, even moreso that Bali (on the equator.)

YAPC is a great value. The conference, 2 days of tutorials plus book and 5 nights of dorm accomodations was $384.

Sunday

- got up an hour before my flight
- hot and humid outside airport in Texas
- took bus from the airport, 73 and 30, 88 doesn’t run on Sunday
- city is not a war zone like Detroit or other inner cities, but rough. one drunk on the bus
- staying in Taub dorm in “the quadrangle”, has ok common
basement with TV lounge, weight room and laundry room
- spent evening at The Cadillac Mexican food restaurant
- very off-campus $25 cab ride
- $30 fajitas and cheesecake

Monday

- meeting rooms are in the 3-storey University Center
- scattered around vertically
- coffee
- number of fast-food restaurants: Wendy’s, Subway, Chili’s Too
- Cougar shop
- TV lounge with wide-screen flat-scree TV
- Game Center with FPS, pool, air hockey and bowling

Larry gave 2 talks in morning

- briliant comparative language analysis
- Perl borrows from other languages heavily
- Perl hits the sweet spot for programmer freedom
- Prolly too many object models in Perl5. one is enough.

Lifetime of Many Hats

- born near Pasadena
- almost drowned with styrofoam surfboard
- liked to specialize in everything
- dropped out of original uni program, worked, went
to missionary school and studied linguistics, back to uni

The Perl Foundation

- case study of dev shop that couldn’t find enough perl programmers,
tried Java, ended up with 25 of each
- creating materials for the public in MBA-speak
- Forester called them and asked for scripting language survey participation
- the analyst groups rank language by current capability
and long-term strategy equal weighed, perhaps problematic for perl
which has a mature perl5 but fuzzy perl6
- somebody has written a Perl module for the Facebook API,
overlooked in their initial API release

Dave Rolsky

- error, validation, logging
- good talk

Abigail

- sudoku solver with Perl regex
- regex is NP complete … so let’s do something insane
- 250 Kilobyte regex to solve 9×9 sudo, 90 minutes to run
- he can solve it by hand in less time

Tuesday

Resume talk was funny. Basically, don’t do things
like send a picture of yourself in a disco, show up
late, or be difficult. Based on real-world experience.

Perrin Harkin

- always thoughtful and practical
- but lolcats galore in slides
- tried VMware for snapshotting environment, 4x slow
disk io on their hw, moved to lvm
- uses version control, indifferent, cvs good enough
for a long time anybody who said otherwise was a crackpot
- use 2 branches (dev on main plus maintenance branch)
- test is important, Test::More is not enough because
of namespace pollution?
- tried to store db operations and roll back in perl,
prolly better to just truncate

Casey West

- working at SocialText, SocalCalc
- started project with MochiKit, web 2.0 ish framework
- roundelement for rounded corners
- cross-browser keyboard modifiers
- animation lib
- functional
- did one lolcat to mock Perrin
- switched to MooTools for JS, smaller memory footprint

Evan Carroll, Houston.pm, Dealermade

- catalyst with Dojo, template toolkit
- ruby with scriptaculous JS, integral

Nice dinner in Hilton ballroom.

Auction hosted by Uri. Quite humorous, raised about $4,000.

Dorm party afterwards, lot of alcohol. Talked to Julian Cash
a little. He is very artistic.

Talked to Beth and Adrian, 2 bioinformatics perl chicks.
Beth runs Ubuntu on her Powerbook G4 and Adrian Debian on her Dell
notebook.

Taub dorm party.

Wednesday

Beth from Ithaca did a talk on database programming with postgresql.
She uses it for bioinformatics work.

Some type of skit about the power of the yapc perl t-shirt.

Town Hall meeting rehashing usual Perl adoption issues:

- lack of corporate funding
- lack of PR
- lack of mindshare vs other languages
- poor state of Perl distros, but in LSB

Richard Dice mentioned some feedback from Forrester Research.
Perl was rated ok overall, with a hole in WSDL support.

This YAPC finished about $6,000 in black.

Talked to Perrin a little about file uploads and memory leaks
on mod_perl. He recommended either using Apache api,
reading by row, or using exit in your Apache::Registry script
to force child to die after the upload.

Talked to a Larry a little about parallel programming
with Perl6. Seems like comprehensive support for features
found in other languages like Haskell.

Thursday

Damion was unable to come to YAPC this year, so Randal Schwartz filled in for him for 2 days in the Advanced Perl training tutorial.

Randal did a good talk on his page-by-page views on Damion Conway’s Perl Best Practises book from 9 am to 2:30 pm.

I tend to identify more strongly with Randal’s programming style than Damion’s in some cases. For example, I’m not a big fan of English.pm.

Randal delved in detail into some areas of Perl upon prompting by the audience. He spent some extra time talking about Perl control structures.