OSCON 2012, Portland

Back to Portland for the 2012 O’Reilly Open Source Conference (OSCON) @oscon.

The main themes this year were:

  1. looks like 2012 is the year that the Redhat GlusterFS distributed filesystem really takes off. Gluster is the first DFS ready for general consumption, with Ceph following behind by 2 years or so.
  2. Cloud computing keeps expanding and improving – vendors, tools, acceptance. See Brian Aker’s keynote “Scaling OpenStack Technology. Lessons From The Field.” It sounds like HP talked Brian into tuning OpenStack’s Software Defined Networking (SDN) Controller. 🙂
  3. vagrant, a virtual machine management tool for developers, was featured in at least 3 different talks. A devops tool called Juju (formerly Ubuntu Ensemble) is also good to be aware of for cloud/VM deployment.

It was another very well-organized, valuable OSCON conference. Some ideas for improvement are:

  1. I found the exhibitors’ area was too crowded to actually talk to anybody in the booths, so maybe more spacing is needed
  2. 90% of the Ignite Talks were lame (either amateur comedy skits or not informative), so better quality control there. It’s a technical conference, after all.
  3. the WiFi availability wasn’t good. Consider more bandwidth, setting up an anonymous FTP server with large files (browsers, ISOs, vagrant boxes, etc.), and reminding sponsors to download ISOs at home instead the conference (I’m looking at you, Redhat.)
  4. the lunches were good, maybe a little rich over 4 days. Consider offering both conference entrees and also a simpler choice, like burgers or fresh sandwiches. Also, get some more feedback from vegans for next time, being Portland and all.

Here’s my notes on some of the tutorials, talks and BOFs that I went to.

Monday Tutorials

Building Applications with MongoDB: an Introduction
Steve Francia, 10gen

– “the schema is in your code”
– the mongo client shell program uses JavaScript, not the server
– indexes are optional. The index types are: multi-value, geospatial, soon full-text
– _id is ObjectId, created on client-side
– ObjectId is 12 bytes from “ts MAC PID inc”
– supports atomic select for update unlike most other NoSQL servers
– find is lazily evaluated except in the JavaScript shell
– paging: supported with .sort .skip .limit
– “Oh my gosh! That’s the highest percentage of windows users I’ve seen in a class like this. ”
– mongod usually uses port 27017, but can run multiple daemons: no need for VMs
– curl -L http://j.mp/OSCONvenues | mongoimport -d milieu -c venues
– “show tables” is shorthand for “show collections”, most-often used at 10gen
– db.venues.count()
– db.venues.findOne() is a way to show a sample record
– explain() command
– recommend not use Mac ports, and Mac OSX Leopard no longer supported
– Foursquare-type location web app lab using Ruby/Sinatra called Milieu
– “Any Rubyists in the room who think I’ve done an injustice to your language by trying to teach it in 5 slides?”
slides
docs.mongodb.org

Monday Lunch

– lunch with some university and hospital programmers/sysadmins
– “R looks like APL” when doing long-term support. But then so do all of the stats tools, regardless of price.
– python assistant expert system for electronic medical charts, especially valuable with preemies.

Moose is Perl: A Guide to the New Revolution
Ricardo Signes, Perl5 Maintainer

Moose is a postmodern object system for Perl 5 that takes the tedium out of writing object-oriented Perl. It borrows all the best features from Perl 6, CLOS (Lisp), Smalltalk, Java, BETA, OCaml, Ruby and more, while still keeping true to its Perl 5 roots.”

Slides
Moose Manual
– lectured at length on employee and network programming with Moose objects
– Moose adds .25 second to initial compile time or so, more or less depending on what features are used
– here’s a sample program fragment

use namespace::autoclean;

package Point;
use Moose; # automatically turns on strict and warnings

has 'x' => (is => 'rw', isa => 'Int');
has 'y' => (is => 'rw', isa => 'Int');

sub clear {
   my $self = shift;
   $self->x(0);
   $self->y(0);
}

package Point3D;
use Moose;
extends 'Point';

has 'z' => (is => 'rw', isa => 'Int');

after 'clear' => sub {
   my $self = shift;
   $self->z(0);
};

1;

Ignite Talks

Ross Turk, Inktank (the Ceph company)
Ceph

Matt Sussman
Replace Yourself with a Bot
– amusing twitter feed @common_squirrel
– created bot to answer common svn questions. nobody could tell the difference after 2 weeks.
– you’re not as important as you think you are

Anthony Bargnesi, Boston
git your linux
– use git to manage your linux configuration files and home directory dot files ie. /etc, /home, …

Ed Snajder, Jive Software DBA @edinor
The Raspberry Pi and eMaker
– stories and samples
– Sanguino is Arduino on steroids

@dbently, Google
Metric is like Programming Languages (humor)
– scientists don’t use metric (GeV, Farad, Mole, etc.)
– how arrogant to not use metric for laypersons
– csh is not a good C and not a good shell
– 0K is dead and 100K is dead

Hotpepper.com
– compare biz leadership to Spanish Conquistadores

Cooking Eggs
Justin Martinson, Ignite Seattle
– hmm. dude with an egg fetish

Tony McCormick
Open Sourcing the Diamond Autocad Library
– http://www.DiamondCADBlocks.org/
– developed on arch projects worth $2 billion, no lawsuits yet

Dave Gruber
code.ohloh.net
– indexing code in SF, github, ruby gems, other repos
– add your own project

Some guy in a toque with sunglasses (humor)
– premise: cute animals are ruining CS lectures to their search
– lolcats, including 1 in cardboard tank

Ceph BOF

– OSD Performance and Tuning – hastexo
– OSD = Object Storage Devices
– RADOS gateway emulates most of S3/Swift Other sims: Parkplace, S3mockup, Fake S3
crushtool
Ceph Wiki
Ceph and RBD benchmarks
– linux 2.6.37 should have working rbd RADOS
– “resilvering” can mean “rebuilding”, or it can mean pre-populating a new brick
– “graceful decommissioning”

Ceph extends storage with open scalability

Tuesday Tutorials

GlusterFS
John Mark Walker, Community Sales and Support
Dustin Black, Redhat Support

Coming Attractions

– RHEL pronounced “RELL”
– FUSE has improved over recent years, O_DIRECT possible
– improving HDFS API (replace HDFS with GlusterFS)
– geo is currently master-slave, future is master-master. So 747 is handled, not follow the sun useful
– Pie in the sky: snapshots, versioning, GeoRep Sparse Replicas, File compression and de-dupe
Gluster Planning34 wiki
– thinks gluster is good (complete, performant, geo-features) for NAS, ceph is good for block storage (Florian agrees, plus use-of-use, but thinks ceph technology is far ahead on block storage, resilvering, etc.)
– buy Redhat Storage Appliance if you want support (no apps on nodes are supported)
– most common problems:
– performance with copying VMs
– fixed problem with doing a checksum on a file froze the copy first
– PHP programmers not configuring for reasonable performance
– possible master-master geo-distributed Open Source optons: Galera, Cassandra, AFS
– Florian on clustered filesystems deployment: “it’s only hard for the first 10 projects”
Lance Albertson from OSL uses Google’s Ganeti on Gluster and has a web-based mgmt. console

Tuesday Lunch

– the Redhat Storage/Gluster support team sat at my table (same as last year pre-acquisition), so we talked more Gluster 🙂

Tuesday Afternoon

Redhat Storage Tutorial
Kevin Black, Redhat Storage Support (East Coast)

Private Cloud
– x86_64
RHEL HCL

Public Cloud
– AWS
– EBS+XFS

– if you use hardware RAID, then RAID6 is required for support
– scale up and scale out: just add a brick!
– Chapter 1. Introducing Red Hat Storage Software Appliance
– http://www.redhat.com/summit/2012/presentations/files/gluster_for_sysadmins-summit-201206-1_dk.odp
– volume top command for performance, but might be doing dd in background
– can connect via ssh
– CDB, DR, /home, HPC, IAAS, replace HDFS
Some customer testimonials:
– CIC Electronic Signature Solutions
– Pandora (songs in 12 formats), 1 administrator
– Brightcove (media serving) 1 PB before, adding 4 PB
– Pattern Energy – 32 HP nodes with 20 TB+ storage
– Fedora 17 yum install gluster 3.2
– RHEL 6.2+ for XFS and gluster 3.3
– slow with 100,000 files per directory?
– master-master coming soon
– take a look at Open Source amahi and greyhole file system for home use

GlusterFS as a Development Platform (Tips for Writing a GlusterFS Translator)
Anand Avati, Gluster Architect (Employee #3)

– everything (every feature) in gluster is implemented as a translator
– FOPS interface (File Operations)
– HekaFS (Redhat’s Jeff Darcy clone effort of glusterfs) UIDmap translator mangles uid/gid fields for multi-tenancy, etc. in cloud
– gluster works on Linux and NetBSD, has worked on Solaris in the past
– Jason Brook (jebpages.com) has a good Gluster tutorial (ex-eweek Labs tech tutorials author)
– SE Linux works a lot better these days. Try enabling it.
Stop Disabling SE Linux!

Drupal Conversation with Rudy Grigar (ex-Acquia, ex-Tagone)

Some free tips:

– use memcached, Pressflow
turn on the CSS file aggregation option to combine JavaScript files
– some Drupal7 performance features have been backported to Drupal6, so also take a look at that
Tagone Consulting does Drupal performance tuning. Principal is Jeremy Andrews (East coast)

I gave a demo of vagrant to Kevin Black from Redhat. He said he uses libvirt on his desktop at home to manage gluster VMs while doing customer support.

Tuesday Exhibits

– talked to Monty about MariaDB connection pooling
– talked to Alfresco about plugins like slideshare
– talked to Charlie Bell from Oracle outside DRBD booth
– talked to Ross Turk about Ceph.

Wednesday Morning

Keynotes

Brian Aker, “Scaling OpenStack Technology. Lessons From The Field”
Tim O’Reilly, “The Clothesline Paradox and the Sharing Economy”

Unicode Snowman Test

PHP 5.4 Features You Will Actually Use
Lorna Jane

“European audiences may appear dead, but not American ones. I usually have to remind them that it’s my talk.”

Useful new PHP 5.4 features:

– Upload progress status
– Built-in webserver
– JsonSerializable

“If you can’t grep a function name in the source, you will have a support problem.”

– Much faster
– Half memory as 5.2

http://lornajane.net
@lornajane

Wednesday Lunch

Personal Postgres replication tutorial at lunch from Bruce Momjian 🙂

– logshipping with binary data
– surprise – just works

Wednesday Afternoon Talks

Running MongoDB for High Availability
Greg Brockman, Stripe

– blackhole a new host if you don’t want it to pull from primary
– consider using IP addresses if names will be reused
– global write lock, yielded several times per sec
– use object versioning
– creates indexes in background
– test cluster upgrades first, read logs
– read bugs, wait for others to test
– upgrades are ez since always backward-compatible

Developing Easily Deployable PHP Applications
John Mertic, SugarCRM

– wrote 2 books on SugarCRM, for Apress and O’Reilly
– PHP Windows has Wincache, APC has a data cache, two memcache libraries
– SugarCache:: is GPL3
– use Factory class instead of include/call
– jenkins and selenium are nice, but manual testing is also necessary
– walked away with microphone. 8 minutes in the life of John. 🙂

One-man Ops: Deploying at Scale in EC2 While Letting You Sleep Through the Night
Jos Boumans, Krux

– DynamoDB is charged per IOP, so not affordable for busy applications
– S3 is the most reliable storage
– Elastic Load Balancers rock
– Global Load Balancers also awesome see dyndns
– statsd, collectd
– Yahoo! Boomerang
– iptables to create shadow traffic against old and new servers
– predict performance rather than wait for an alert
– Slideshare/jiboumans

Programming with the OSS Cloud Stack
Mike Amundsen, Layer 7 Technologies

– actually used Chromebook only for web programming 6 months
– Cloud9 with git, Couchdb
– Saucelabs Selenium
– C9 can deploy to Heroku
– Ctrl+alt+del on chromebook gives u a shell, can ssh host and rdp
– any Perl cloud IDEs?

Wednesday Nite BOFs

GlusterFS

– Redhat needs to work with Debian Gluster maintainer for fresher packages
– Another tutorial set in San Diego in August
– I explained about the use case of google-type nodes and need for locality of disk reference

Cloud Camp

– “Why License is Still Important!” by Krish Subramanian – need for Open Source cloud software and APIs
Patrick Chanezon from VMware did a talk on their CloudFoundry
– companies want no lock-in and source, as well as private-cloud hybrid option
– “ActiveState Update” by Diane Mueller. She did a talk on their cloud product, Stackato
– en-something did a talk on their stuff

Thursday Afternoon

HTML5 and Geolocation
Andy Gup, ESRI

– your privacy policy needs to state whether opt-out means your data is also deleted or not
– cold and warm location fixes
– cold immediately after device starts
– warm enough if error is less than 100 m, and trend decreasing

http://blog.andygup.net/
http://esriurl.com/javascript
http://caniuse.com/
http://mobilehtml5.org/
@agup or agup@esri.com

Web Application Accessibility
Alice Boxhall, Google Chrome Accessibility Engineer

– ChromeVox is a screen reader plugin for Chrome
– “icon” becomes “earcon” 🙂
– ensure form widgets are in order for easy tabbing
– browser can do the accessibility work if:
– clean HTML5
– provide text alternatives like alt
– manage focus
– add key handlers
– add aria for screen readers
– use HTML label or aria-label
– focus is important, especially on popups
– HTML tabindex
– ARIA/WAI
– aria roles (checkbox, button, etc.)
– aria-live (screen area subject to updating)
– Accessibility Developer Tools + ChromeVox
– http://goo.gl/8irxZ
– http://accessibility-developer-tools.googlecode.com
– http://chromevox.com
– http://google-axs-chrome.code.google.com
– often when she reports bugs, they apply to Webkit generally, so Apple also gets involved in fixes.
– there’s a group with 4 disabled people, one for each major OS, who use their choices of screenreader.

Automating Cloud Deployments with and Puppet and OpenStack
Dan Bode, PuppetLabs

– OpenStack is big and complicated, so use a configuration mgmt. system like Chef or Puppet
– Puppet has OpenStack:: recipes

Develop and Test Configuration Management Scripts with Vagrant
Mitchell Hashimoto, Kiip

– 90 minute tutorial from Velocity Conference 2012 compressed into 40 minutes
– working a couple hours per day on vagrant now, but planning to work full-time on it
– supports Virtualbox now, wants to add support for VMware
– but Virtualbox does seem to have some bugs on Mac OS X resulting in occasional panics

Four ways to onboard new developers:

  1. setup script – that’s always out of date
  2. uber-readme – that can never be explicit enough
  3. “you’re on your own” – can be a 2 week IQ test for some people
  4. vagrant up – even HTML/CSS designers can handle one command.

– managing VMs is nice, but the real power is in adding configuration mgmt. with Puppet or Chef. Wants to add CFEngine
– after, he said that the complexity of Virtualbox network scripting wasn’t a problem since he just incrementally adds support for what he needs

John Bender – Snap setup from scratch the Vagrant way

Thursday BOFs

MySQL Replication BOF
Peter Zaitsev, Percona

– Monty described the history of MySQL replication (original requirements from Yahoo! contract)
– Oracle said that 5.6.5 magically solves all problems (now has Global Transaction IDs, checksums)
– some users asked about scaling a busy master (hard), zero downtime (harder)

MySQL 5.6 Replication: FAQ

MariaDB BOF
Monty, MariaDB

– talked about using Kickstart for funding
– Mark did some free Like SEO for MariaDB (I’m Like #1500.)
– talked about optimizer improvements
– extent of MariaDB being a fork vs. Oracle and Percona

Poker Night Sponsored by Citrix

– cold beer and bad pizza
– caught up with Randal

Friday

Social Commerce With Magento and Open Graph
John Jawed, X.Commerce/Paypal/eBay

– explanation of how eBay/Paypal has created an API (branded as X.commerce) that can be called to show
who bought what on your merchant site, in conjunction with Facebook
– X.commerce maintains counts and aggregates on their site of your sales, but not Facebook friends
– moderately complex to understand, configure and install
– plans to add to Shopify and other sites and carts
– X.commerce has plenty of item databases for matching ISBN/SKU/etc.

O’Reilly Open Source Awards

This year’s recipients were Elizabeth Krumbach, Bradley Kuhn, Christie Koehler, Jim Jagielski and Massimo Banzi.

The Paul Fenwick Show

Once again this year, Paul closes OSCON with a humorous and thought-provoking show. This time it’s called “Mindware Upgrades For Fun And Profit.”

Fruitstand Break

Nice selection of fresh fruit, cheeses and crackers for attendees to snack on while milling around.

I talked to two Redhat Openshift developers/evangelists about RHEL6. They mentioned that Redhat is planning to do incremental updates of some related packages with “stacks” in RHEL6 to avoid the staleness problem in RHEL5 (PHP in particular.) There should be a programming languages stack, web tools stack, etc.

OSCON 2012 Speaker Slides & Video
OSCON 2012 Videos

Vagrant: Home Wikipedia Ruby, MIT License
Juju: Home Wikipedia Python, GPL3 License, Sponsored by Ubuntu

Facebook gives devs easy access to Heroku Cloud
Shuttleworth drops one million cluster bucks on Ceph upstart

This entry was posted in BSD, Business, Cassandra, Cloud, Conferences, Hadoop, Linux, MySQL, Open Source, Oracle, OSCON, Perl, Photography, Psychology, Storage, Tech, Toys, User Groups. Bookmark the permalink.

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.