The name “Gluster” was derived from the words “GNU” and “Cluster.” No relation to the Luster filesystem, and actually they have opposite overall architectures.

GlusterFS is a GPL3 distributed network filesystem that runs as a service in user mode on Linux on a network of servers (conceptually like Google GFS.) Redhat bought Gluster, Inc. in 2011 and calls it “Redhat Storage.” By running in user mode and reusing existing linux features and modules, GlusterFS gained reliability in months instead of the usual 10 years for other filesystems.

gluster.org is the community website for GlusterFS.

Previously, AB was CTO at California Digital Corporation (CDC). One of his projects was the 1024-node linux “Thunder” HPC supercomputer for LLNL, the most powerful in the USA at the time.

- after that, he still wanted to do Open Source projects, preferably without bureaucratic encumbrances
- got some angel funding, but also a seismic data company paid $500,000 to adapt HPC code in 3 months to replace IBM Regatta system, then a follow-on storage contract for 1.2 PB in a 6 months project
- audience member: “In 10 years 1 PB will fit on a SD card.”
- GlusterFS is in some ways architecturally the opposite of VMware. GlusterFS is userland code.
- in 2006 large companies like Lehman, etc. started appearing on mailing list, to some surprise and awe
- originally the company was called Z Research, renamed to Gluster, Inc. to have clearer brand name
- no in-house test storage hardware, so developed on customer hardware!
- was still doing other paying work such as embedded kernel stuff, web dev, etc to reduce burn rate initially. Too distracting.

- Google mixes app and GFS, app generates 64 MB chunks, GFS manages metadata. Too complicated for general use.
- GlusterFS is a distributed storage OS in user-space
- create container without knowledge of filesystem (Posix, ACL, etc. ) because no known common user pattern for storage
- lots of general C programmers available to recruit, but no filesystem developers without baggage about kernel
- in 2008-2009 added too many features to actually test
- VCs contacted them and invested A and B series total of $15 million, despite their storage “experts” saying it was crazy, users said it was awesome
- lowest layer is native filesystem like EXT3 or XFS, thus idiot-proof
- use extended attributes for metadata
- block, replication, striping, elastic hashing algorithm
- striping support by cleverly using sparse files with different offset on each server
- read server choice based on fastest response
- every directory has its own hash space
- good default behavior when adding servers (no thundering herd)
- striping is good for hotspot files or files too big for 1 volume, like saving HPC results
- will be unified file and object protocol for object storage
- there is a pathinfo command can query extended attribute, could be used with ssh for a fake MapReduce
- GET and PUT at command line
- GlusterFS is most heavily used for lots of files containing unstructured data
- 3.3 will have faster healing operations, better granularity for 100 VMs, KVM support, etc.
- currently shared-nothing, but with a little sharing healing can be faster
- will be HDFS clone mode

Data Storage models:

• objects
• file
• block
• structured data
• NoSQL
• semi-structured data.

- Redhat bought Gluster, Inc. for about $136 million in October, 2011. It was about 60 employees at the time. Now there’s about 40 engineers working on GlusterFS at Redhat. AB chose Redhat over other suitors because of its commitment to Open Source and linux.

A dozen people attended dinner afterwards:

- it was tough hiring people for Gluster Inc. since the concept of doing file systems in userland confused a lot of developers and managers.
- AB’s philosophy is that the Open Core model doesn’t serve end-users well, as all users need “extras” like user-friendly mgmt. programs, not just licensees
- companies seem to be happy to pay for GlusterFS support, one of the reasons being lack of in-house storage engineers
- lots of discussion about Illumos (OpenSolaris fork), ZFS and containers
- take a look at Nexenta
- An efficient WAN replication method with GlusterFS is to use the marker framework / queue using extended attributes to feed rsync a list of changed files, scales better than inotify
- no storage tiering yet for incoming/outgoing hotspot files
- Redhat is pushing xfs heavily internally, and has hired the available ex-SGI xfs developers
- some checksumming is done in GlusterFS, but no end-end checksumming. Need to look at performance and demand.
- historicaly, linux has had slower adoption and community interest in India than Western countries due to lagging Internet performance (often more practical to install linux from a magazine CD-ROM than attempting large downloads) and relatively higher cost of computers compared to local salaries
- AB got started in programming on a Spectrum microcomputer, and progressed over time to fixing minor bugs in the linux network drivers, culminating in GlusterFS.

Thanks again to Symantec for hosting this event.

GlusterFS Downloads
Chip’s Rant: Why GlusterFS is Glusterfsck’d Too
wikipedia: Extended file attributes

There was a good turnout, with about 30 outside attendees and about the same of ISC staff. 3 BIND developers from the Czech and Chinese registrars also attended, as well as Cricket Liu.

Here’s my notes.

DNS

- can learn what you need to know (50%) in a day, lifetime for the next 50%

BIND9

- monolithic program
- 10 years old
- different syntax than BIND8, 30% slower

User Story – Quick Refresh at AOL

- VMs spark up in 9.5 seconds, but DNS takes 15 minutes or more to propagate in their system
- goal is 5 minute updates across globe

BIND10

- sponsored by 10 TLDs, unlike BIND9 which was sponsored by Big Iron vendors
- TLDs represent registrars, users or citizens
- started April 1, 2009
- 5 year plan
- now on year3 – features and performance
- release every 6 weeks
- will be no slower than BIND9
- postfix processes model, not threads
- scalable across cores
- modular
- logging
- reporting
- REST API
- save cache object to memory or network, could be 10s to 100s of MB
- no query or response logging, use tcpdump
- look at DLZ

DNS Provider Issues

- Dealing with large zones
- Dealing with lots of zones
- resource estimates (for sizing new servers)
- high-availability installations and technologies
– anycasting
– cooperate better with firewalls and load balancers
– auto-clustering?
– quagga support?
- multi-master
- support for fully mastered slave
– akin to powerdns “super masters”

- some debate over how much routing is appropriate inside dns
- do it all or just be fast
- does it make sense to send acl info over port 53? needed for cluster setup

DNSSEC Issues

- Deployment and Maintenance procedures
- in-line signing
– aka “bump on the wire” 9.9.0
– hidden master sends signed zones to slaves
– aka “signer in the middle”
- registry vs. registrar
- look at DomainKeys optionality
- DLV keys (don’t need a signed parent)
- zone monitoring is crucial
- 4 HSM known, $80k for qty 1
- some firefox plugins check ad bit
- dnssec slowest on negative answer, nsec3 worst

The event photos are available here.

Thanks to Facebook for hosting this event.

The unit that arrived was a model BNRV200 with Software version 1.2.0 installed.

In short, the Nook Color is a remarkably capable and polished 7″ tablet.

I use tablets mainly for final software testing of websites, and it will be handy to have a 7″ Android for that.

The Nook team at B&N should all get a bonus for their great work and attention to detail.

Getting Started

After recharging the battery and agreeing to a 100 page Terms and Conditions form (!), you must register online via WiFi with a free B&N account to “activate” it. (No credit card is required until you want to buy a book or app. No PC is needed.)

Hardware

• 7″ “VividView” glossy color display is 16 million colors, 1024×600 and bright. Great screen.
• built-in speaker is adequate for home use, but not loud enough if there’s ambient noise, so you will need earphones for use in public
• has microSD slot that supports up to 32 GB storage
• has custom miniUSB port (2-level contacts) and custom cable needed for charging from wall at 1.9 amps. (Will only charge slowly when connected to a PC. My MacBook Pro sometimes displays a warning dialog that says there’s overcurrent and disables that USB port.)
• RAM is 512 MB, and the 8 GB total internal flash memory is partitioned into 2 GB system, 5 GB content and 1 GB sideloading (end-user)
• WiFi b/g/n works with any access point that you can login to, plus B&N access points that provide free extra services like book previewing. Some forum posts report that older access points may not be compatible.)
• TI OMAP 3621 CPU is a single-core ARM Cortex 800 MHz
• long battery life using internal Li-ion Polymer battery (over 8 hours)
• solid-feeling, but kind of heavy at 15.1 oz.
• useful corner cutout in case for locking to a desk or anchoring a strap on the unit.

Software

• Android-based (originally Android 2.1, but has been updated) with customized UI
• complete all-around eReader and Internet-capable tablet.
• some games are included, including chess, sudoku and crossword puzzles, along with Pandora music and email apps. (Angry Birds is a $2.99 downloadable purchase.) Notably missing are free solitaire and minesweeper games, and there’s no free calculator app.
• the included software is fine. If B&N drops support in the future, there is third-party Android software for this device.
• file formats supported for reading include: ePub, BBeB, PDF, XLS, DOC, PPT, PPS, TXT, DOCM, XLSM, PPTM, PPSX, PPSM, DOCX, XLSX, PPTX, JPG, GIF, PNG, BMP, MP3, AAC, MP4. Quickoffice® Pro app can be used to edit many Microsoft formats.
• free in-store WiFi access, free 1-hour eReading per day, free use of Angry Birds Golden Eagle feature to clear a level.

Usage Notes

• it’s a good idea to examine the settings page and customize them. I disabled location reporting and forms memory.
• international users (billing address outside the USA) can’t use the store.
• there’s a bug in Software Version 1.4.1 when entering your credit card information: the form subtracts 1 from your month, so choose the next higher month, or your card will be declined with a “processing error” dialog
• battery life can be extended by setting WiFi to “Off” (airplane mode) when using it as an eReader, or putting it into sleep mode.
• the power button is multifunction: 2 seconds is on or sleep, 5 seconds is power off, 20 seconds is software reset.
• I use a netbook case to store it. The least expensive official accessories are the Aalto Cover ($14.95) and the Silicone Frame ($19.95)
• Software version 1.4.1 does not have third-party support at this time.

B&N “Guru” Notes

I talked to a store employee and he had this advice:

• the straps they sell are “for decorative purposes only”, so don’t rely on them to be strong enough for carrying your Nook Color
• if you don’t want to use your existing credit cards for purchases from the app store, you can buy a prepaid VISA or MasterCard debit card and use that alone, or in combination with a gift card. I checked on prepaid cards at Safeway and found that there are steep fees and require your name, address and SSN to purchase. Some credit cards also allow a temporary number to be generated, sometimes known by the brand names SafeShop or NetSafe. Due to a forms bug, you need to increase the month by one.

Nook Color Tablet (BNTV250) – Next Generation

Similar screen as older Nook Color, but double the RAM (1 GB) and flash (16 GB), and still only 1 GB partition for end-users. Dual core 1 Ghz CPU. Netflix app and better video capability. 50% more battery life. Third-party software support unknown due to lockdown, prolly requested by Netflix’s content providers.

Official Nook Color FAQ
androidtablets.net: Noook Forums
Barnes & Noble is the first Angry Birds Magic Place for your Nook Color
wikipedia: Nook, Nook Color, Nook Tablet, microSD
EReader Notes: Kindle Fire vs Nook Tablet
flightcrewlife.com: Using a Nook Color as an Android Tablet
That Triangle-Shaped Tablet From “The Office” – One Guy Wants to Build It
wsj: Barnes & Noble Offers Nook Discounts

Here’s some interesting links, mostly related to installing and maintaining alternate energy technologies for heating and cooling:

youtube.com: HOW I MADE A SOLAR POP CAN FURNACE / HEATER

ocregister.com: Electric cars can require home rewiring

This Old House: Quieting Noisy Baseboards at 15:10
This Old House: How to Quiet a Noisy Forced-Air System

Solar Powered Exhaust Fans: rewci.com

Ask TOH: Panel Upgrade to 200 Amps and Grounding

All I can say is that I’m glad my town council didn’t fall for that one.

Santa Clara, 49ers line up $850M stadium loan
Santa Clara, 49ers announce deal to pay for stadium
wikipedia: City of Santa Clara
wsj.com: SEC Is Investigating Marlins Stadium Deal
NFL owners approve $200 million loan for 49ers stadium


Please note that the price of Seagate 1TB Barracuda 7200.11 Bulk/OEM Hard Drive ST31000340AS has increased from $160.00 to $225.00 since you placed it in your Shopping Cart. Items in your cart will always reflect the most recent price displayed on their product detail pages.


theinquirer.net: Hard drive prices double after Thai floods
storagebits: How will the disk shortage affect you?

I worked at Netscape, and all databases were stored in the USA, even for international sites. Only partial caching was allowed offshore.

It was a simpler era in that RDBMS’ were centralized, with occasionally proxies or caches or LDAP caches configured remotely.

Generally that worked well pre-9/11 as a default, as the USA had some of the most permissive laws in the world, especially California, and followed the rule of law.

The reasons had to do with legal problems in other countries:

• Legal Jurisdiction
• taxation nexus
• local laws thorny to user-generated content (ie. French and Austrian anti-Nazi rulings, Turkish Article 301)
• privacy – regimes demanding complete copies of user databases
• corruption of court officials allowing competitors to access your databases locally
• overreaching trial discovery requests.

Recently I had a conversation with some IT Managers and a corporate lawyer at a Web 2.0 company, and was surprised to find them completely ignorant or dismissive of these issues.

Although Feds generally don’t follow the rule of law post-9/11, an American’s company’s best choice for database jurisdiction is still the USA, along with a real-time slave hosted in Canada.

As we start to build globally distributed databases with tools like Cassandra, it’s worth considering the legal and privacy issues of multi-jurisdictional databases.

Facebook’s Swedish data centre will be subject to Snoop Law
Stoel.com: E-Commerce and Internet Law Topics
wsj.com: Feds Can Get Twitter Users’ Data Without Warrant, Judge Says
PATRIOT Act clouds picture for tech

One would think that privatization might be the answer, but there’s 2 points to consider:

1. Caltrain service was private until 1980, when Southern Pacific gave up due to on-going losses and it was “nationalized”
2. the Caltrain right-of-way is planned to be used for the eventual high-speed rail system between San Jose and LA via SF. Privatizing anything before the new system is built would enormously complicate negotiations.

I suppose contracting out operations would save money, but outright privatizing of Caltrain may not be a good idea at this point.

An example of the disadvantage of privatizing an essential transportation asset is the Ambassador Bridge between Windsor and Detroit, owned by billionaire “Matty” Moroun. Any suggestion of affecting the existing traffic using that bridge results in an automatic lawsuit.

Caltrain will probably always find enough money to continue operating. Dumping 40,000 cars onto local highways is not feasible, and Stanford University commuters rely on Caltrain.

greencaltrain.org: Upcoming Funding Choices for Caltrain
wikipedia: Caltrain, Ambassador Bridge

Tomer did a great job of answering a variety of technical questions.

In order to improve High Availability (HA), MapR has:

• rewritten Hadoop Distributed File System (HDFS) in C++ as MapR-FS. 3 compressed copies are stored. It is snapshottable and exportable as NFS.
• distributed the namenode across all storage nodes, resulting in 1000x namenode performance and avoiding Java GC
• made the jobtracker restartable by checkpointing
• granular permissions allow multi-tenancy
• added a nice mgmt. User Interface (UI) with heatmaps.

So MapR has replaced HDFS, and modified Apache Hadoop to change how namenode and jobtracker work. Hadoop-ecology layers above that are not affected, like Pig or Hive. Nobody has requested HCatalog support yet.

MapR benchmarks appear to be 2x to 4x faster than Apache Hadoop, esp. on small files, so a cluster would need half as many nodes. (See Slideshare presentations for benchmark graphs.)

There’s 2 MapR product versions:

1. MapR M3 Free Edition is non-HA and forum-supported.
2. MapR M5 includes HA and phone support and is $4,000/node/year.

EMC is reselling MapR. They are planning to build a 1,000 node cluster for demo purposes.

Currently, the deployed MapR clusters are small (in the range of 10 to 100 nodes), but MapR is designed to handle clusters of 10,000 nodes. Apache Hadoop maxes out at almost 4,000 nodes currently.

Thanks once again to Symantec for hosting the event.

MapR on YouTube
Ted Dunning’s MapR Talks on Slideshare YouTube: Localhost mounting of MapR NFS
Apache Hadoop FAQ
Tomer Shiran on Quora.com
Ryan’s Review of Hadoop Summit 2011
gigaom.com: Investors make $20M bet on MapR to win Hadoop war
gigaom.com: Why MapR Is Right to Give Back to Apache Hadoop
wikipedia: MapR

1. Python At Google (A Google IO talk)
2. YouTube for Your Business

Python At Google (A Google IO talk)

Python 2.6 is an officially-supported production language, along with C++ and Java.

The cluster and meta-cluster management software was about 70% Python/30% C++, may have changed.

For applications that are to be deployed on a million servers, saving even 5% memory is important, so C++ is used there.

Around a dozen well-known Python developers and authors work at Google around the world.

Even internal programmers are encouraged to use Google AppEngine.

YouTube for Your Business, presented by JJ Behrens

JJ Behrens is a developer advocate at Google specializing in YouTube APIs.

Some year-old YouTube numbers:

• 3 billion videos viewed per day
• every minute, 48 hours more video is uploaded
• usage seemed to be about 10:1 of PC vs mobile viewing
• new video formats and resolutions are added over time in a reasonably aggressive manner

One promotion strategy that has been found to work well for many businesses is to create instructional and demonstration videos and upload them to YouTube and serve them from there.

You can also embed the video on your site in an iframe, or for more advanced users use the API.

Read TOS, Monetization, Branding and API TOS guides. Spirit is that if the page only contains a YT video, then you can’t place your own ads.

Although it’s easy to broadcast videos to promote your business, reporting on who views them is not available.

You can do live streaming while saving your video via YouTube upload and download, though it requires some organization to do it well.

Many games allow directly uploading videos from in-game to YouTube now, including PC and PS3 and games.

Youtube Direct on AppEngine
Shortform.com has manually curated content
Gdata apis
Storify.com
Memolane.com – curation app that helps you create a lane (channel) of all your friends’ social media items.

YT/create
Squrl.com
Lvzzk

JJ mentioned that Python was the likely reason that the original YouTube team was able to add features more quickly than the Google Videos team. YouTube used Python, while Google Videos used C++. He felt that Python was 5x more productive.

Thanks once again to Symantec for hosting the event.

Google I/O 2011: Python@Google
wsj.com: YouTube Tees Up Big Talent
Your own movie database in 5 minutes with IMDb API and Perl