James’ World

I attended the O’Reilly Open Source Conference, once again in Portland, Oregon.

Overall my impression was that the talks and vibe were oriented towards Web 2.0 primarily.

I would say that the talks were not as strong as previous years, but it’s easy to compensate for that with the “hallway track” and access to the original Open Source authors.

Several attendees used the EEE sub-notebook computer, and were happy with it as a email/browser tool.

Wednesday

PHP Taint Tool: It Ain’t a Parser

- CS’y effort at PHP parser for code analysis, reminds me of early days of Perl’s B tools
- not suitable for end-users

Write Beautiful Code (in PHP), Laura Thomson, Mozilla

- good general background on good programming practises
- not a lot of specifics about PHP, but available for questions

Hypertable, Doug Judd, Zevents

- HyperTable is a clone of Google’s BigTable, from public paper
- room was packed, some turned away
- still alpha, maybe beta in August
- preferred distributed filesystem is HDFS, works with others
- I recommend reading web site and then looking at the curt slides
- plans to do benchmarks with same hardware as Google has published.

Open Source Virtualization for People Who Feel Guilty About Using VMware So Much, andy michelle, EDA

- cute talk about VirtualBox, Xen and VMware
- Xen has weird nomenclature compared to other tools
- VMware wins on tools and polish
- showed screenshots of unreleased and alpha mgmt. tools.

Barely Legal XXX Perl, Jos Boumans, RIPE

- stunning and twisted example of overloading, short-circuiting, import-faking, whatever it takes to make a loaded module do something other than intended
- illustrates great flexibility of perl, for good or ill
- could be useful for things like testing harnesses, etc.
- motivated to win bet of $100 or 1 vertical meter of beer
- said it took 3 or 4 hours to complete.

I walked around the exhibits area.

Got a demo of Atlassian’s continuous integration (CI) tool, Bamboo. They’re also the vendors of JIRA issue tracker and Confluence wiki, which I’ve used before.

One company had a public Wii game happening.

Thursday

Scaling Databases with DBIx::Router, Perrin Harkins

Ultimate Perl Code Profiling, Tim Bunce (Shopzilla)

- talk and screenshots about NYT perl profiler

The New York Times Perl Profiler

Top 10 Scalability Mistakes, John Coggeshall (Automotive Computer Services)

- good overview of writing high-performance, maintainable Internet systems
- interesting opinion that scalability is not just about increasing performance. scalability can be about scaling up or down, performance or maintainability, etc.
- recommended php.ini settings list

Perl Lightning Talks

- popular with audience, attendees seemed to like all the talks
- Mail::ESMTP looks very interesting for testing and production

Code is Easy, People are Hard: Developing Meebo’s Interview Process, Elaine Wherry (meebo)

- struggled to find time, right approach to interview new candidates in 1996, likely at behest of VCs
- external recruiters hit-and-miss, conferences and jobs email link useless
- phase where non-founder employees doing interviews wanted a founder involved in interview process
- trying to preserve culture (finger rockets, social networking, 2 female founders, etc.)
- came up with process involving reading resumes, phone screens, and office “sim” that adds a new candidate within 3-6 weeks
- “sim” has 3 versions: office manager (plan to erect a meebo office sign), front-end engineer (write a JavaScript app), and back-end engineer (write a server) in 4 hours
- current goal is to keep interview time down to 8 hours per candidate over 10 days
- now up to about 40 employees
- my feeling was that their hiring process started off clueless due to inexperienced mgmt. and is still oriented towards junior engineers. Silicon Valley is full of expert engineers and it doesn’t take 8 hours to interview them.

BOF

mysql-sandbox

Giuseppe Maxia discussed and demoed his very useful mysql-sandbox utility for managing several versions and instances of MySQL on the same machine.

He wrote it for his testing work at MySQL AB. Very well received by attendees. This is a great example of what I call “anti-virtualization” - using ports instead of resource-intensive VMs.

MySQL Conference 2008 Presentation

State of the Onion Address, Larry Wall

- talk about Perl6, random anecdotes, etc.

Friday

Open Voices, Jim Zemlin (The Linux Foundation), Keith Bergelt (Open Invention Network), Karen Sandler (Software Freedom Law Center), Phil Robb (Hewlett Packard)

- panel discussion of various free software efforts, some little-known

An Illustrated History of Failure, Paul Fenwick (Perl Training Australia)

Paul gave an interesting talk on notable Software Failures and estimated a price tag for each. I had heard news reports of many of them, but it was interesting to hear an updated analysis of what really happened behind the scenes.

Thanks to Google for sponsoring the fairly good almost-gourmet lunches. Sure beats the O’Reilly lunchbags from the dot bomb days. (Everybody I know bailed and found a subway shop back then.)

Notes

- Burgerville popular with attendees, can upgrade combos to a shake.
- Red Lion hotel has a small cardio gym with 1 universal machine, no free weights, open til 11 pm
- WiFi password changed weekly, in middle of remodel, lobby just finished.
- There is a 24-Hour Fitness that is actually open 24 hours near downtown Portland. Has basketball court and 2-lane pool. $15 for non-member visitors.

OSCON 2008 Presentations

I attended the O’Reilly Open Source convention again, making it 10 years in a row. Once again it was held at the Convention Center near downtown Portland, a convenient light rail ride from the airport.

Like many experienced developers, I spent a lot of time in the “hallway track” talking to other developers and users, as well as in one of the 15 simultaneous talks.

The general consensus was that the talks were not as strong as in previous years (not even compared to the MySQL conference this year), but it’s worthwhile to me if I can get even one juicy nugget from each talk, or gain an understanding of a developing trend in programming or system administration.

Many of the presenters griped about there not being enough time to look at source code in a 40 minute talk.

For those who want a conference summary in a nutshell:

• OpenID is popular
• lucene and its REST interface have more mindshare than projects like Kinosearch, language-specific bindings, etc.
• Yahoo! released the yslow browser plug-in for front-end performance evaluation
• Perl: no ORM appears to be gaining the upper hand, though DBIx is respected. Tim Bunce would like to see a wrapper around JDBC for each scripting language. Alison Randal is updating the Perl license.
• PHP: no good way to do vector reporting graphics, especially since IE doesn’t support SVG and Adobe is killing the Macromedia plug-in in December. PHP4 is being EOL’ed 8/8/8 so that the PHP developers can focus on 5 and 6 only.

Google was heavily recruiting at the conference. I ran into 3 recruiters, and there were even more in the Google booth.

Pretty good food for lunch, usually chicken or fish in some kind of red sauce with steamed veggies. Better than the wilted sandwich boxes from previous years that mainly got tossed out.

Tuesday nite

I arrived at the Convention Center in time for the evening Google Open Source awards. Happened to sit next to Zak and the 20 year-old OpenID guy, David Recordon, who won $5,000 and a colored, transparent, angular plastic trophy and base that we had fun stabbing each other with.

The OpenID Foundation is offering a $5,000 bounty to the first 10 OSI-approved projects that add OpenID support. Many programmers were busy adding it, including SocialText and others. (David works at Verisign.)

I walked over to the Doug Fir Lounge with a few guys, 2 of them Austrian. I had the halibut fish and chips and lemonade for $20 including tip. It was ok. They have a log cabin motif happening with a restaurant, patio and bar upstairs, and dance club downstairs, so ID is required to enter. Open from 7 am to 2:30 am every day, 1 503 231 WOOD.

Wednesday

Nagios

- general overview of features
- Event Broker most powerful, least used

Bigger and Faster
Rasmus Lerdorf

Rasmus did his usual “PHP is as secure as any other language”, and “pick on a PHP app and make it go faster” talk.

He said he’s still not a Y! Paranoid, but his work does often touch on PHP and web security.

He used to use httpload, but now prefers siege for load testing because it has support for cookies.

http://developer.yahoo.com/yslow/
Live HTTP Headers
APC

sla.ckers.org/forum/list.php?3
php.net/filter
xdebug.org/docs/profiler
talks.php.net/show/oscon07

xdebug
jeremiah san diego xss console author
scanmus.corp.yahoo.com

PHP and Ruby Envy
- NZ programmer on Silverlight CMS (BSD licensed)
- own object system in PHP5
- Ruby less available on web servers, less mindshare
- rolled his own PHP OO frameword apparently

Exhibits

- talked to Mark Finkle of mozilla.org
- said hi to Larry. He had the whole family there.

Trac
Vivek Khera

- he uses RT for public tickets, Trac internally
- doesn’t require much resources since only a few developers
- Trac is used on many Ruby/PHP projects
- gives you wiki/tickets/etc.
- modified BSD license

Afterwards mentioned:

- uses Trac in a BSD jail
- an alternative to Trac would be basecamp (or I guess Sourceforge software). See slashdot.org threads for more ideas.
- likes pfSense firewall as an appliance
- nagios alerts too much, and no good rule builder for multiple hosts
- own web framework called Rowdy (RWDE)
- software as complicated to install as RT should be treated as an appliance
- he submitted 6 related talks on software development environment, only 1 accepted.

Steve Souders
Chief Performance Yahoo!
souders@yahoo-inc.com
Exceptional Performance Group

http://developer.yahoo.com/performance

- IBM Page Detailer Pro
- yslow (crawls the DOM, not a packet sniffer)
- firebug
- jslint - The JavaScript Verifier

80-90% of end user response time is spent on the front-end. so optimize there.

14 Rules for a Better User Experience

1. make fewer HTTP requests
2. use a CDN
3. add an Expires header
4. gzip components - even JS and CSS
5. CSS at top
6. JavaScripts to bottom
7. avoid CSS expressions
8. make JS and CSS external
9. reduce DNS lookups
10. minify Javascript
11. avoid redirects
12. remove duplicate scripts
13. configure Etags - disable in most cases if load-balanced or multiple web servers
14. make AJAX cacheable

move JS to onload
remove bottom tabs
avoid redirects
images sprites
expires

Thursday

PHP Graphics
Luke Welling, OmniTI

Luke presented an overview of raster and vector graphics modules for PHP.

He prefers vector graphics, but there’s not many free options for doing that.

He feels that Yahoo! Finance and Google analytics sites are state of the art in presentation graphics with anti-aliasing, interactivity, esthetics, text and maps. I’d say that’s aiming a little low, but it’s a start I guess.

• JpGraph (GD-based, QPL/Commercial E85.+)
• raw Adobe PDF
• Dojo charting (BSD/Academic)
• FusionCharts (Commercial $499+)
• DIY Flash
• Flex
• Ming Flash
• ez graph (GD and swf-based,
• another is zxChart (swf-based, shareware E15.)
• another is Open Flash Chart (swf-based, GPL)

Perl Lightning Talks

• Vani Raja did a talk on Yahoo! JS
• Ask did his talk on qsmtpd again
• talk on Test::More 3?
• talk on task lists for hit and run volunteers
• Schwern did one talk on making tea for 5 minutes, and one on “Blame Schwern” - just do it instead of waiting for permission
• Tim Potter did a talk on a messaging standards effort for his employer, saying that the ANSI process was too slow and looking for an alternative
• Andy Lester did a talk on ack
• a talk on SVN::Notify
• http://angerwhale.org/
• Tim Bunce talked about DBD::Gofer Proxy and next-gen cross-scripting language DB API based on JDBC API
• guitar song about #perl

YouTube: Perl Lightning Talks on Handycam by Schtonk

Perl Auction

Larry’s talk on comparative languages and Perl6. Sounds like we’ll be able to do something like foreach (1..infinity).

Full Text Search BOF
Peter Zaitsev
- based in London, England but often in SV
- uses Sphinx on several servers
- http://boardreader.com/ one TB of searchable data
- geneology is big on full-text search

- after insert, mysql full text gets slow, run optimize.
also, doing it at insert time causes index update per keyword
- gin or gyst for Postgresql 8.4?
- Michael Kimsal, SOLR
- hard disk space is free (enough for whatever indexing is required)
- mostly news search involves last 5 minutes of feed
- MessageOne stores email for lawyers to mine. They like to search, archive and expire. Mostly Exchange lusers, rarely Unix admins.
- Lucene and REST interface
- Monty says MySQL AB hired a programmer to work on search, but he’s working on another project now. They need somebody with a burning desire to make progress in an area like that, but they recognize the importance of search.
- Monty poured out free Finnish chocolate rum from a Pepsi bottle that was so powerful it scared most people. He said it was banned for 2 years in Finland because it was so addictive.

Sun BOF

- audience talked to senior Sun staff about Java and Solaris a little.
- free beer, cheese and crackers.

Friday

A bunch of Postgres people went to the Portland wine tasting on the river event in the afternoon.

Call for Software Whiteboard

OSCON07 Call for Software Whiteboard
flickr.com: Jeff Kubina’s OSCON 2007 Whiteboard set of tiles

Several years ago I attended a talk by Chip Salzenberg, a former Perl Pumpkin (lead maintainer), at OSCON.

One of his slides had these three rules for writing high-performance code:

1. Don’t do it.
2. Do it later.
3. Let somebody else do it.

Simple rules, but they get powerful results.

I especially like “Don’t do it.”

• Don’t walk that array - use a hash data structure. (Perl and JavaScript have hash built-in, C has it in Boost.)
• Don’t do locking if serialization is not needed.
• Don’t add more columns or surrogate keys to your database if not needed.
• Don’t do that join.

CPUs have gotten faster over the years (disks haven’t), but server code still needs to be tight so that you can provide a better user experience (under 300 ms page creation time) with fewer boxes (save money, space, manpower and energy.)

Smile and “Just say No.” on your project.

Your users and DBA will thank you later.

Virtues of a Perl Programmer: Laziness, Impatience, Hubris

About once a year I need to migrate a medium-sized web application using MySQL to Oracle.

Usually it’s to prepare a successful intranet application for a move to a formally supported production environment - and in Silicon Valley, that usually means Oracle.

I was apologizing to Monty at OSCON this year about my latest migration project away from MySQL.

His surprising response was, “It’s not a bad thing. I don’t mind hearing about conversions because successful migrations demonstrate that people don’t have to worry about database lock-in.”

Here’s some notes on doing migrations.

Converting MySQL 4.x Apps to Oracle 10g

Conceptual Issues

• Skills: Oracle has a steep learning curve for programmers unfamiliar with database transaction programming, so you should have at least one programmer who has worked with Oracle in the past and can write stored procedures. You will also need an experienced Oracle DBA for non-trivial projects.
• Portability: decide if you want to maintain dual database support or not. It’s not difficult to do since 90% of SQL queries work the same, and it’s unlikely you’ve done anything tricky like stored procedures or views in MySQL since those are new features in 5.0. It’s nice to have dual-database support if your developers use notebook computers (MySQL is fairly light-weight), or if you want to market the software later (for example, most ISP hosting plans support MySQL only.)
• Autocommit: decide if you want to use autocommit or not with Oracle. Usually not.
• Performance: MySQL has a very limited query optimizer, resulting often in slow table scans. Oracle however has a great query optimizer and can optimize queries even with multiple subselects and OR clauses.

  If you do mass updates, consider checkpointing them (loop and commit) to avoid filling the Oracle redo log. A MyISAM UPDATE statement that touches all records can fill up the log in Oracle if not sized correctly. For the sames reason, Oracle TRUNCATE is much more efficient than doing a DELETE * FROM TABLE.

  MySQL has table types that have specialized features and performance, such as MyISAM/FullText, Innodb/Transactions, Heap, Blackhole, etc.

  Oracle has 3 storage engines (heap/parallel query, index-organized, external) with features on top of that. Heap is the default storage engine, index-organized is a B-tree optimized for compactness and quick access that can be several times faster for index and range queries than heap, often used for data warehousing.

• MySQL supports database names, but Oracle is quite different … it uses one database but optionally supports multiple schemas based on userid
• Schema: converting the database schema and data migration will likely be more difficult than doing the application source code changes. There are tools to help. Spectral Core sells Full Convert. Same with regression testing. MySQL can be accomodating about blank vs. null. vs 0000-00-00. Oracle is not, so loading data can be touchy.
• Timezone: MySQL-based applications often use whatever timezone the MySQL server uses. Oracle recommends for performance reasons setting the database to UTC (+0000). Beginning with MySQL 4.1.3 and Oracle 9i, both databases have similar tz features, as both are based on the Olson timezone database. To do timezone conversion, MySQL uses CONVERT_TZ and Oracle uses FROM_TZ. Note that in MySQL 5.0, NOW() is a session timestamp (computed once so replication-safe), while SYSDATE() is a real-time timestamp and would be a different value on the slave.

  
mysql> SELECT @@global.time_zone, @@session.time_zone; # show MySQL tz settings
SYSTEM SYSTEM

  mysql> select * from time_zone_name; # see if tz database is loaded yet
  Name | Time_zone_id
  Africa/Abidjan | 1
  Africa/Accra | 2
  Africa/Addis_Ababa | 3

  sqlplus> SELECT * FROM v_$timezone_names;

• Character Set: MySQL-based applications often use whatever character set that MySQL defaults to. When moving to Oracle, you likely want to add explicit support because production Oracle instances are usually set to AL32UTF8 these days (Oracle’s “UTF8″ character set is actually Unicode 2.0 from 8i days). You can see which character set a database or column is set to with:

  
select * from v$nls_parameters where parameter in ('NLS_LANGUAGE',
'NLS_TERRITORY', 'NLS_CHARACTERSET');
select dump(mycolumn,1017) from mytable where rownum=1;


• Case-sensitivity: MySQL generally does case-insensitive string comparisons if you don’t use the BINARY keyword, but Oracle is case-sensitive. MySQL database and table names are case-sensitive on Unix (but not Windows or Mac OS X HPFS+) because databases and tables are actually directories and files, but Oracle silently upper-cases them and appears case-insensitive.
• Sequence Numbers: Oracle sequence numbers are not guaranteed to be sequential. Values are “lost” in a rollback, and will most likely be “lost” if cached sequence numbers are specified and there is a shutdown or panic or library age-out. Non-cached sequence numbers can cause a noticeable performance impact (a disk access), which is why the default is to cache 20 values per allocation. 1000 is commonly used for bulk loading.

  Some Oracle sequence factoids: sequences never rollback after being incremented, they can be non-numeric, and you can use multiple sequences per table in Oracle. (Oracle sequences are actually separate objects from tables.)

  To use sequences in Oracle, you can either specify NAME_OF_SEQ.NEXTVAL followed on the same $dbh with NAME_OF_SEQ.CURRVAL in 2 statements, or combine both with an INSERT … RETURNING … INTO statement.

  You can emulate MySQL’s autoincrement feature with an Oracle sequence and a trigger. This is documented in the blog posting How to Create Auto Increment Columns in Oracle.
  When migrating from MySQL to Oracle, you may want to consider dropping useless surrogate keys altogether, reducing the need for application code changes or creating sequences.

• Trailing spaces: MySQL and Oracle handle trailing spaces in columns differently when doing string comparisons. Oracle’s NCHAR preserves trailing spaces, and NVARCHAR2 does not.
• Performance: MySQL is a lightweight database that usually performs well with little planning. Updating a couple of rows and doing a select from a MyISAM table may take milliseconds in MySQL, but one second calendar time in Oracle without prior thought to using batch inserts in a single transaction, or batch sequence numbers.
• NULL: In MySQL, the empty string may be inserted into a column and is not a NULL value. Oracle converts the empty string to NULL.
• MySQL silently truncates input data when too wide for a column, but Oracle considers the column width to be a constraint and fails the insert or update.
• It is smart to quickly migrate a representative sample of your migrated code with the converted Oracle schema to notice any surprise problems.

SQL Syntax Issues

• the ANSI join syntax works in 10g, so that makes porting much easier than in the past. (The Oracle DBAs I have talked to said that the new features in 9i were not ready for prime-time.)
• MySQL allows GROUP BY on any column. Oracle only allows GROUP BY on all of the columns in your query result set.
• MySQL unix_timestamp() can be converted to an Oracle stored procedure
• if you were using a MySQL database for scratch tables, you can do a similar thing in Oracle by declaring tables to be in a scratch tablespace, but in the same schema
• should be able to use Oracle transactions and ACID, and remove MySQL LOCK TABLE and UNLOCK TABLE statements.
• in MySQL, a database is a combination of a hostname and database name. In Oracle it’s a SID, and is defined in the tnsnames.ora configuration file.
• MySQL LIMIT can be replaced with a subquery using ROWNUM in Oracle, for example SELECT * FROM (SELECT ROWNUM limit, … ORDER BY …) WHERE limit BETWEEN ? and ?. Note that MySQL LIMIT is 0-based but Oracle ROWNUM is 1-based.
• Oracle object names (column, table, sequence) are 32 characters and by default case-insensitive. In MySQL, database and table names are just files, so case-sensitive on case-sensitive file systems like Unix
• MySQL autoincrement columns will instead need a sequence in Oracle.
• the optional MySQL AS alias statement keyword is not recognized in Oracle after a FROM clause table name… just delete it. (AS is valid after a column name.)
• MySQL CONCAT can be rewritten as || in Oracle
• MySQL syntax INSERT INTO table SET is not supported in Oracle.
• MySQL syntax for batch INSERT (mutiple VALUE lists) is not supported in Oracle.
• MySQL NOW() can be replaced with Oracle SYSDATE, or CURRENT_TIMESTAMP which works in both MySQL and Oracle. CURRENT_DATE is also portable.
• MySQL SELECT on a datetime field for display without explicit formatting can be emulated with select to_char(SYSDATE,’YYYY-MM-DD HH24:MI:SS’) from dual;
• MySQL EXPLAIN can be done 2 ways in Oracle: EXPLAIN PLAN FOR …; @$ORACLE_HOME/rdbms/admin/utlxpls.sql, or the DBA can create a plan_table so that in SQLPLUS you can type SET AUTOTRACE ON
• MySQL CREATE TABLE … LIKE would need a stored procedure in Oracle to copy metadata. Oracle’s CREATE TABLE … AS SELECT does not copy indexes, triggers, constraints, tablespaces or sequences. Oh, and in Oracle, non-primary indexes must have unique names across tables because they are schema objects that do not actually belong to a table.
• SELECT COUNT(*) FROM table_name is very fast with MySQL MyISAM tables, because the total row count is stored in the index. In Oracle, expect a much slower result as an index scan is required. An estimate is stored in NUM_ROWS which is updated after most non-INSERT table changes, but it can be wildly inaccurate.
• To read blobs in Oracle, you will likely need to allocate memory for the result: $db->{LongReadLen}=500000; # Make sure buffer is big enough for BLOB
• MySQL always sorts NULLs last, but Oracle ORDER … DESC sorts NULLs first, unless you specify NULLS LAST
• queries returning the “top n” results are usually implemented in MySQL with ORDER … LIMIT, but in Oracle there are a number of ways of doing that with subselects, ROWNUM and RANK keywords.
• The MySQL client program (mysql) is a fairly usable text-mode application. Oracle’s version, sqlplus, is inadequate for programmers. Consider using Oracle SQL Developer (Java, so kind of slow sometimes but does work on MacIntel machines), Squirrel, or Qwest TOAD instead.
• MySQL’s LOAD DATA INFILE and SELECT INTO OUTFILE statements can be emulated in Oracle with SQL Loader and BCP external programs. Perl programmers can use the CPAN module Oracle::SQLLoader, although it is simplistic and needs more testing for customized control files.
• Perl DBI’s $sth->rows() returns the row count from a SELECT result set in MySQL, but in Oracle does not. At best it will indicate -1 for failure and 0E0 for success, so do a COUNT(*) or loop over the result set with while and fetch for a row count. In Oracle, $sth->rows() is incremented as you do the fetch, often too late for your program logic.
• Jeremy’s MyTop for MySQL has an analogous display in Oracle’s SQL Developer Reports .. DD Reports .. DB .. Top SQL.
• MySQL’s best-effort statement-based replication and Oracle’s replication are very different. Oracle replication is done in a transaction across master and slave, so they stay in sync.

A sequence hack, that I would seldom recommend, to mimic MySQL autoinc in Oracle is this. Define one sequence in Oracle called APP_ID, have the app call APP_ID.NEXTVAL everywhere an autoinc is needed, regardless of table, and create an Oracle stored procedure called LAST_INSERT_ID that calls APP_ID.CURRVAL. That way you can minimize source code changes to your MySQL app.

The downside is that a busy app will soon be using very large numbers as IDs, perhaps needing wider columns, and making it difficult for humans to write or verbalize them, or even predict what the next value will be for a given table.

Please post your migration tips!

Oracle Globalization Whitepaper
Oracle Documentation
MySQL Documentation
Oracle: Welcome to the “2 Day DBA”
Planet MySQL (High Quality Blog Aggregator)
Oracle-Base: EXPLAIN PLAN Usage
Oracle SQL*Loader FAQ
SOLUTIONS TO COMMON SQL*LOADER QUESTIONS
BULLETIN: CACHING ORACLE SEQUENCES
OraFAQ: BLOBs
MySQL Manual: MySQL Server Time Zone Support
MySQL Manual: MySQL Date and Time Functions
Oracle Date Functions
Oracle Resumable Transactions
Write Time Zone Aware Code in Oracle
Speeding Up Index Access with Index-Organized Tables
Contact James if you need database conversion or Perl consulting.

OSCON 2006 was in Portland, Oregon again this year, in the cavernous Oregon Convention Center. I stayed in the Inn at the Convention Center as usual: it’s both the cheapest and closest hotel to the Convention Center, and the Red Lion-side gets free wireless Internet in the rooms.

Apparently some local programmers created a FOSSCON, a parallel free conference, that many of the Ruby speakers contributed to.

Here’s my notes from some of the sessions and BOFs that I attended:

Painless Web Proxying with Apache mod_proxy
Justin Erenkrantz, UCB and Google.
erenkrantz.com/oscon/

- Apache reverse proxying mainly was discussed
- more new and helpful features than I expected
- Apache 2.2 and even better 2.3 has awesome features
- new version 3 rewrite of mod_proxy that can actually work
- load balancing
- ProxyPass with min max connection parameters
- mod_jk now included for Tomcat
- binary protocols available for ajp and fastcgi, 2% or 3% faster
- new fastcgi module included that is a good example of modern Apache module writing. great for ruby.
- mod_cache can fetch data and cache it locally,
shared cache with NFS
- mod_proxy understands http, https, ftp get, ajp, fastcgi
- squid is more intended for forward caching, mod_proxy more intended for reverse proxy and those already familiar with apache configuration
- mod_proxy is efficient (just sits around waiting for requests)
- HA with multiple reverse proxies
- ajp does not support healthcheck currently

New Query Interface
Randy Newsome

- show cardinality of query (how many permutations or results may be selected)
- freeform (text field) vs. pulldown menus (under say 50 items) for numerous returns
- show total matches
- db and memory is free these days
- for paging, cache all ints for ids (ints are small)
- get by fine without Ajax
- some members of the audience concurred with his ideas
- not a very technical talk, but a good UI is key these days.

Exhibits/Recruiting Area

- Rasmus (I work as a PHP architect, not a PHP programmer) and Jeremy 1 were recruiting in the Yahoo! booth
- ActiveState was there on their own after the spin-out from Sophos. Looking for new tools to write.
- talked to Matthew Olander, ex-BSDI, now CTO of iXsystems. maybe a BSDcon in future?

PHP 6 and Unicode
Andrei Zmievski, Yahoo!

- good update on ICU programming effort
- ICU features did not make it into 5.2 or 5.3 because it was “intrusive”

memcached BOF
- chaired by Brian Moon of DealNews
- he replaced NFS with memcached, 64 MB per server
- uses Omniture for tracking - expensive but worth it
- PHP C client is 4x faster than Perl
- think about slab size
- many discussed namespaces in a future memcached
- new Perl XS lib
- cache db, page components, sessions
- Brian Aker says look at MySQL cluster NDB

Building Search Applications With KinoSearch
Marvin Humphrey

- stemmer snowball
- stopwords snowball
- put stemmer last
- use same analyzer at index-time and search time
- google prefers non-stemmed now
- plucene is “profligate wastrel” oo code
- stemmer is not i18n, one language at a time obviously
- kino,xapian,swish,plucene, look at CPAN
- utf-8 safe - tested

PHP Data Objects (PDO)
Wez Furlong

- PDO ships with 5.1, 5.0+ needed
- very fast
- looks nice (convenient and complete)
- Detailed talk by PDO author, though he said a half day would be needed to do it justice.
- PDO Wiki

The Damian Conway Channel 2006

Damian presented 2 modules that he wrote.

List::Maker

- source filter module that DWIMs list syntax for you
- my @list = < 10 .. 1 >;

Contextual::Return

- replacement for wantarray in your code that DWIMs.

Damian: “I’m the high plains drifter of perl. I come down from the hills, shoot people, give them weapons to shoot themselves, then back to the hills for 10 months.”

Sun zones and zfs BOF
Channing Lovely
Brian Utterback
a dozen other Sun employees!

- impressive demo of zones and zfs (23 MB for new zone) via shell script
- 10 zones have noticeable overhead
- global zone (top can see all processes and run dtrace)
- non-global zone (restricted privileges like cannot see other network traffic)
- solaris 10 update 2 206 needed for latest updates
- had the balls to ask the audience, “what’s wrong with Sun?”
- open bar (nice!) with cheese and crackers

subversion
svnadmin hotcopy
autoversioning for users by dragging into SMB share - save to a fs and it versions it

dervish

A young Ruby programmer mentioned these Ruby and vim links:

vim talk

vimdiff
vimmerge

MVC testing suite for Ruby

mongrel ruby web server has good or adequate performance

MySQL BOF

Mostly talked about improvements that MySQL needs for GIS, data warehousing (materialized views, stable indexes), query optimization improvements (i18n may have introduced bugs). Got a free book, Pro MySQL. Monty, Kai, Jay Pipes and Arjen were there.

Greant said he liked the closing keynotes, including Damian’s CXAP (pronounced CRAP) rant about Web 2.0.

The wireless Internet (sponsored by IBM) worked great in the Convention Center. Plus free wireless Internet at PDX. Nice.

Registration and Schedule Area

Convenient Starbucks next to Registration Area

Larry Wall and Damian Conway at book signing

Lloyd Center Mall’s Atrium

View from Convention Center of DoubleTree Hotel

Convention Center Skylight Detail

Convention Center Skylight

OSCON 2005 was in Portland, Oregon again this year. This time it was held in the cavernous Oregon Convention Center instead of the cramped hotel conference center as in the previous 2 years. That allowed for a proper area for the exhibitors for a change, but the building was too large to maintain a convenient and intimate conference. Wireless worked great for a change.

I lucked out with my hotel this year. I just showed up in town and rented a room at the closest hotel, the Inn At The Convention Center, directly across the street from the OCC. It turned out to be the most convenient and also the cheapest (under $100/nite) in the area. It had wifi thanks to its sister hotel, the Red Lion Hotel Portland Convention Center, across the street, broadcasting a channel. (The official conference hotel, the DoubleTree Hotel & Executive Meeting Center Portland - Lloyd Center, was actually a few blocks from the Center for some reason.) The MySQL guys also picked my hotel. For some reason hotel cable TV had 2 reality shows about tattoo parlors.

There was good attendance with all the usuals there. My favorite talks were PHP and Unicode: A Love At Fifth Sight with ICU (sponsored by Yahoo!), to be in the next point release, and the one on 45 Things to Do with RSS and Atom, which inspired me to add an RSS 2.0 feed to a product I work on before my flight out. I attended a few BOFs, including the one on mod_perl.

In the exhibits area, I asked some postgresql questions at their booth. Apparently the favored replication method is sloany. Cardomain.com was recruiting Perl programmers. Occasionally I saw Jeremy manning the Yahoo! booth. Some company was promoting a graphics application for linux with only a SQL query interface - no programming API. Too bad.

Miguel de Icaza gave a good closing talk on cool things at Novell/Suse. He demoed Mac-like graphical window manager capabilities on X. The winning photo contest entries were chosen from a flickr tag.

The inflatable furniture in the halls was fun to look at and sit in.

Apparently there’s only one major Perl employer in town, Rentrak, an online real estate company. I saw their building near the airport on my way out.

Attended OSCON 2004.

This one was cramped indeed, but the wireless worked better than the previous year.

Some of the talks I attended (they were all short but excellent) were:

• Building Your Own Storage Engine for MySQL, Brian Aker
• MySQL Innodb Tuning Experience, Peter Zaitsev
• Practical i18n with PHP and MySQL, Jim Winstead
• How DBD::mysql enhancements can benefit the Perl Developer, Patrick Galbraith
• HTTP Caching and Cache-busting, Michael Radwin
• Planning Ahead, A Survey of Caching Strategies, Marcel Levy
• MySQL High-Availability Options, Jeremy Zawodny
• Database Abstraction with Class::DBI, Casey West
• Building A High-Speed Website (using Persistent Perl instead of mod_perl), Vivek Khera
• PostgreSQL Replication Panel
• Build Your Own Spam Firewall, Zach Levow
• Building Scalable Websites with Perl, Perrin Harkins

Most of the slides are here.

Thursday, July 17, 2003 10:15 PM

I attended at the O’Reilly Perl Conference in Portland, Oregon.

Once again everybody had a great time. It’s great rekindling friendships that spark up annually at the conference with people from different states and countries.

The hotel conference rooms were a little small for some talks, and the wireless AP was flaky, but the talks were interesting as always. And Portland is beautiful, kind of rustic and situated next to a large river, a nice place to eat lunch.

I went to the Bricolage and Postgres BOFs. Like usual, people were impressed by the power of Bricolage. Postgres people outlined coming support for 2 important features, Windows support (required by Bruce Momjian’s sponsor in Japan) and replication. Much whining over mindshare of MySQL, mostly underestimating its feature progress.

Some of the talks I went to included Casey West’s really good anti-spam talk about Perl and Spam Assassin, the “MySQL And Progress Toward The SQL Standard(s)” talk, the Perl lightning talks, Brian Aker’s “Making MySQL Do More” with embedded perl and customized full-text search and Jeremy’s MySQL benchmarking talk.

Many of the presentation slides are here.

And you can’t fault the evening parties - free mixed drinks by Stonehenge and DynDNS.

Joe S. told me how great yum is (like apt-rpm), and Allison Randal and friends were getting a consulting company called OnyxNeon Inc. off the ground (Open Source consultants, many of whom have written O’Reilly books). Hammered Monty with some MySQL performance questions and got good answers, as you would expect.

Had dinner with the O’Reilly site programmers. They’re looking for somebody to move to Sebastopol and work on their sites, which use MySQL/Perl/Linux/custom CMS app:
http://jobs.oreilly.com/

Also made it to Powell’s Technical Books, a fairly large collection of new and used books. Nice collection of science books on Nikola Tesla.