The name “Gluster” was derived from the words “GNU” and “Cluster.” No relation to the Luster filesystem, and actually they have opposite overall architectures.

GlusterFS is a GPL3 distributed network filesystem that runs as a service in user mode on Linux on a network of servers (conceptually like Google GFS.) Redhat bought Gluster, Inc. in 2011 and calls it “Redhat Storage.” By running in user mode and reusing existing linux features and modules, GlusterFS gained reliability in months instead of the usual 10 years for other filesystems.

gluster.org is the community website for GlusterFS.

Previously, AB was CTO at California Digital Corporation (CDC). One of his projects was the 1024-node linux “Thunder” HPC supercomputer for LLNL, the most powerful in the USA at the time.

- after that, he still wanted to do Open Source projects, preferably without bureaucratic encumbrances
- got some angel funding, but also a seismic data company paid $500,000 to adapt HPC code in 3 months to replace IBM Regatta system, then a follow-on storage contract for 1.2 PB in a 6 months project
- audience member: “In 10 years 1 PB will fit on a SD card.”
- GlusterFS is in some ways architecturally the opposite of VMware. GlusterFS is userland code.
- in 2006 large companies like Lehman, etc. started appearing on mailing list, to some surprise and awe
- originally the company was called Z Research, renamed to Gluster, Inc. to have clearer brand name
- no in-house test storage hardware, so developed on customer hardware!
- was still doing other paying work such as embedded kernel stuff, web dev, etc to reduce burn rate initially. Too distracting.

- Google mixes app and GFS, app generates 64 MB chunks, GFS manages metadata. Too complicated for general use.
- GlusterFS is a distributed storage OS in user-space
- create container without knowledge of filesystem (Posix, ACL, etc. ) because no known common user pattern for storage
- lots of general C programmers available to recruit, but no filesystem developers without baggage about kernel
- in 2008-2009 added too many features to actually test
- VCs contacted them and invested A and B series total of $15 million, despite their storage “experts” saying it was crazy, users said it was awesome
- lowest layer is native filesystem like EXT3 or XFS, thus idiot-proof
- use extended attributes for metadata
- block, replication, striping, elastic hashing algorithm
- striping support by cleverly using sparse files with different offset on each server
- read server choice based on fastest response
- every directory has its own hash space
- good default behavior when adding servers (no thundering herd)
- striping is good for hotspot files or files too big for 1 volume, like saving HPC results
- will be unified file and object protocol for object storage
- there is a pathinfo command can query extended attribute, could be used with ssh for a fake MapReduce
- GET and PUT at command line
- GlusterFS is most heavily used for lots of files containing unstructured data
- 3.3 will have faster healing operations, better granularity for 100 VMs, KVM support, etc.
- currently shared-nothing, but with a little sharing healing can be faster
- will be HDFS clone mode

Data Storage models:

• objects
• file
• block
• structured data
• NoSQL
• semi-structured data.

- Redhat bought Gluster, Inc. for about $136 million in October, 2011. It was about 60 employees at the time. Now there’s about 40 engineers working on GlusterFS at Redhat. AB chose Redhat over other suitors because of its commitment to Open Source and linux.

A dozen people attended dinner afterwards:

- it was tough hiring people for Gluster Inc. since the concept of doing file systems in userland confused a lot of developers and managers.
- AB’s philosophy is that the Open Core model doesn’t serve end-users well, as all users need “extras” like user-friendly mgmt. programs, not just licensees
- companies seem to be happy to pay for GlusterFS support, one of the reasons being lack of in-house storage engineers
- lots of discussion about Illumos (OpenSolaris fork), ZFS and containers
- take a look at Nexenta
- An efficient WAN replication method with GlusterFS is to use the marker framework / queue using extended attributes to feed rsync a list of changed files, scales better than inotify
- no storage tiering yet for incoming/outgoing hotspot files
- Redhat is pushing xfs heavily internally, and has hired the available ex-SGI xfs developers
- some checksumming is done in GlusterFS, but no end-end checksumming. Need to look at performance and demand.
- historicaly, linux has had slower adoption and community interest in India than Western countries due to lagging Internet performance (often more practical to install linux from a magazine CD-ROM than attempting large downloads) and relatively higher cost of computers compared to local salaries
- AB got started in programming on a Spectrum microcomputer, and progressed over time to fixing minor bugs in the linux network drivers, culminating in GlusterFS.

Thanks again to Symantec for hosting this event.

GlusterFS Downloads
Chip’s Rant: Why GlusterFS is Glusterfsck’d Too
wikipedia: Extended file attributes

There was a good turnout, with about 30 outside attendees and about the same of ISC staff. 3 BIND developers from the Czech and Chinese registrars also attended, as well as Cricket Liu.

Here’s my notes.

DNS

- can learn what you need to know (50%) in a day, lifetime for the next 50%

BIND9

- monolithic program
- 10 years old
- different syntax than BIND8, 30% slower

User Story – Quick Refresh at AOL

- VMs spark up in 9.5 seconds, but DNS takes 15 minutes or more to propagate in their system
- goal is 5 minute updates across globe

BIND10

- sponsored by 10 TLDs, unlike BIND9 which was sponsored by Big Iron vendors
- TLDs represent registrars, users or citizens
- started April 1, 2009
- 5 year plan
- now on year3 – features and performance
- release every 6 weeks
- will be no slower than BIND9
- postfix processes model, not threads
- scalable across cores
- modular
- logging
- reporting
- REST API
- save cache object to memory or network, could be 10s to 100s of MB
- no query or response logging, use tcpdump
- look at DLZ

DNS Provider Issues

- Dealing with large zones
- Dealing with lots of zones
- resource estimates (for sizing new servers)
- high-availability installations and technologies
– anycasting
– cooperate better with firewalls and load balancers
– auto-clustering?
– quagga support?
- multi-master
- support for fully mastered slave
– akin to powerdns “super masters”

- some debate over how much routing is appropriate inside dns
- do it all or just be fast
- does it make sense to send acl info over port 53? needed for cluster setup

DNSSEC Issues

- Deployment and Maintenance procedures
- in-line signing
– aka “bump on the wire” 9.9.0
– hidden master sends signed zones to slaves
– aka “signer in the middle”
- registry vs. registrar
- look at DomainKeys optionality
- DLV keys (don’t need a signed parent)
- zone monitoring is crucial
- 4 HSM known, $80k for qty 1
- some firefox plugins check ad bit
- dnssec slowest on negative answer, nsec3 worst

The event photos are available here.

Thanks to Facebook for hosting this event.

The unit that arrived was a model BNRV200 with Software version 1.2.0 installed.

In short, the Nook Color is a remarkably capable and polished 7″ tablet.

I use tablets mainly for final software testing of websites, and it will be handy to have a 7″ Android for that.

The Nook team at B&N should all get a bonus for their great work and attention to detail.

Getting Started

After recharging the battery and agreeing to a 100 page Terms and Conditions form (!), you must register online via WiFi with a free B&N account to “activate” it. (No credit card is required until you want to buy a book or app. No PC is needed.)

Hardware

• 7″ “VividView” glossy color display is 16 million colors, 1024×600 and bright. Great screen.
• built-in speaker is adequate for home use, but not loud enough if there’s ambient noise, so you will need earphones for use in public
• has microSD slot that supports up to 32 GB storage
• has custom miniUSB port (2-level contacts) and custom cable needed for charging from wall at 1.9 amps. (Will only charge slowly when connected to a PC. My MacBook Pro sometimes displays a warning dialog that says there’s overcurrent and disables that USB port.)
• RAM is 512 MB, and the 8 GB total internal flash memory is partitioned into 2 GB system, 5 GB content and 1 GB sideloading (end-user)
• WiFi b/g/n works with any access point that you can login to, plus B&N access points that provide free extra services like book previewing. Some forum posts report that older access points may not be compatible.)
• TI OMAP 3621 CPU is a single-core ARM Cortex 800 MHz
• long battery life using internal Li-ion Polymer battery (over 8 hours)
• solid-feeling, but kind of heavy at 15.1 oz.
• useful corner cutout in case for locking to a desk or anchoring a strap on the unit.

Software

• Android-based (originally Android 2.1, but has been updated) with customized UI
• complete all-around eReader and Internet-capable tablet.
• some games are included, including chess, sudoku and crossword puzzles, along with Pandora music and email apps. (Angry Birds is a $2.99 downloadable purchase.) Notably missing are free solitaire and minesweeper games, and there’s no free calculator app.
• the included software is fine. If B&N drops support in the future, there is third-party Android software for this device.
• file formats supported for reading include: ePub, BBeB, PDF, XLS, DOC, PPT, PPS, TXT, DOCM, XLSM, PPTM, PPSX, PPSM, DOCX, XLSX, PPTX, JPG, GIF, PNG, BMP, MP3, AAC, MP4. Quickoffice® Pro app can be used to edit many Microsoft formats.
• free in-store WiFi access, free 1-hour eReading per day, free use of Angry Birds Golden Eagle feature to clear a level.

Usage Notes

• it’s a good idea to examine the settings page and customize them. I disabled location reporting and forms memory.
• international users (billing address outside the USA) can’t use the store.
• there’s a bug in Software Version 1.4.1 when entering your credit card information: the form subtracts 1 from your month, so choose the next higher month, or your card will be declined with a “processing error” dialog
• battery life can be extended by setting WiFi to “Off” (airplane mode) when using it as an eReader, or putting it into sleep mode.
• the power button is multifunction: 2 seconds is on or sleep, 5 seconds is power off, 20 seconds is software reset.
• I use a netbook case to store it. The least expensive official accessories are the Aalto Cover ($14.95) and the Silicone Frame ($19.95)
• Software version 1.4.1 does not have third-party support at this time.

B&N “Guru” Notes

I talked to a store employee and he had this advice:

• the straps they sell are “for decorative purposes only”, so don’t rely on them to be strong enough for carrying your Nook Color
• if you don’t want to use your existing credit cards for purchases from the app store, you can buy a prepaid VISA or MasterCard debit card and use that alone, or in combination with a gift card. I checked on prepaid cards at Safeway and found that there are steep fees and require your name, address and SSN to purchase. Some credit cards also allow a temporary number to be generated, sometimes known by the brand names SafeShop or NetSafe. Due to a forms bug, you need to increase the month by one.

Nook Color Tablet (BNTV250) – Next Generation

Similar screen as older Nook Color, but double the RAM (1 GB) and flash (16 GB), and still only 1 GB partition for end-users. Dual core 1 Ghz CPU. Netflix app and better video capability. 50% more battery life. Third-party software support unknown due to lockdown, prolly requested by Netflix’s content providers.

Official Nook Color FAQ
androidtablets.net: Noook Forums
Barnes & Noble is the first Angry Birds Magic Place for your Nook Color
wikipedia: Nook, Nook Color, Nook Tablet, microSD
EReader Notes: Kindle Fire vs Nook Tablet
flightcrewlife.com: Using a Nook Color as an Android Tablet
That Triangle-Shaped Tablet From “The Office” – One Guy Wants to Build It
wsj: Barnes & Noble Offers Nook Discounts

Here’s some interesting links, mostly related to installing and maintaining alternate energy technologies for heating and cooling:

youtube.com: HOW I MADE A SOLAR POP CAN FURNACE / HEATER

ocregister.com: Electric cars can require home rewiring

This Old House: Quieting Noisy Baseboards at 15:10
This Old House: How to Quiet a Noisy Forced-Air System

Solar Powered Exhaust Fans: rewci.com

Ask TOH: Panel Upgrade to 200 Amps and Grounding

DARPA Grand Challenge

- Tully participated in the DARPA Grand Challenge on a team that made a self-driving car called Bob that went 3 miles, CMU went 7 miles. (Bob got confused and ended up on the wrong side of a barbed-wire fence.)
– Alice went farther with rackmount servers running Gentoo
– Little Ben used Mac minis, easier to unload nightly
– 90% of effort repeated by each team because code not designed to be reusable frameworks (Message passing, etc. ), thus ROS
– the Grand Challenge result was just a race, without a technology work product despite the massive effort invested, but did make people think about how to do robotics more efficiently

Robot Types

Robots are classified into 3 application groups:

1. Industrial – currently over 1 million robots used in factories. Foxconn plans to add 1 million. (I would argue that CNC machines are also robots.)
2. Service – also over 1 million robots used in cleaning, etc., largely from iRobot
3. Education/Hobby/Other – mostly custom-built at universities.

Robot Operating System (ROS)

– ROS protocol with listeners and clients in many languages, no project preferred language
– ROS is BSD-licensed
– goal is really good documentation so that researchers don’t feel the need to keep rewriting basic robotics software
– also maintain OpenCV and Pointclouds
– ROS uses Jenkins for Continuous Integration (CI), emails author of a build failure quickly
– Japanese industrial robot manufacturer Motoman is currently the only industrial mfg. that supports ROS. Most industrial mfgs. use the excuse that Open Source developers would pose a quality problem, but in reality want to maintain proprietary code as a trade secret.

Willow Garage PR2 Robot

– intended for general purpose combined robotics/visual processing research and education
– PR2 weighs 400 pounds, 4.5′ to 5.5′ tall, uses 4 lb. motors, and arms stay where set with series elastic actuators (springs)
– 4 lb. motors are unlikely to injure anybody, and no exclusion zone is needed between robots
– PR2 is $400k, SE is $285k, 30% off for approved Open Source projects
– 1200 watts max power consumption (500 watts per i7 computer plus 200 watts for all motors and sensors)
– can fetch beer, fold laundry in 5 or 6 min, or plug itself in
– has software simulator Gazebo of all motors and sensors, runs about 1/3 of actual PR2
– about 11 have been distributed to leading universities
– PR2 is designed to be heavy-duty, and universities can use it 3 lab shifts per day
– used industrial robots start at $4,000, so the PR2 is an expensive way to get into robotics unless you’re a vision researcher.

PR2 ($400k) Playing Pool

TurtleBot

– TurtleBot is an entry-level, affordable robot
– built from iRobot Create, Kinect and Asus EeePC 1215N (Atom), ROS
– about $1,400, assembled is $1,500
– also available is an accessory arm kit for $750, and a 6-pack holder
– Create is a Roomba minus vacuum plus serial port
– working on robot swarming, need better reconnect
– YouTube: Create Fridgemate Explained, in action

Above: TurtleBot ($1,500.) Black object on top is Kinect, black object in middle is Asus eeePc.



Above: I-Heart Create Arm ($750) plus 12 VDC Power Assembly

General Robotics Comments

- main limitation on current personal robots is battery power, followed by computing power
- the wireless network connection between robot and computing cloud is a severely limiting bottleneck, better to do on-robot
- bright lighting really helps robotic vision
- Japanese robots seem to be more interesting from a video promo standpoint than a technical one

Audience Comments

- one person expected robots to have legs and be able to walk like humans for some reason
- some were hopeful that Willow Garage was hiring, but they’re not a fast-growth company

BiliBot

- BiliBot ($1,200) is another affordable educational robot, designed at MIT. A basic arm is included that can lift 3 lbs.
- YouTube: BiliBot picking up a bunch of grapes

Above: BiliBot ($1,200.)

Toys

- HexBug makes inexpensive (under $30) but fragile toy robots with legs for ages 8+
- WowWee Roboquad (CES 2007) is an awesome $400 toy for robot enthusiasts
- USB Rocket Launcher, handy to cannibalize for PC-controlled motors ($25) with 360 degree horizontal rotation and 45 degree vertical rotation

Thanks to Symantec for hosting the meeting.

With these autonomous cars, who needs you to drive?
wikipedia: ROS
iRobot SPARK
darpagrandchallenge.com
Stanford Point Clouds FAQ
Roomba game encourages you to attack dust bunnies
IEEE Spectrum DIY: Willow Garage’s TurtleBot Brings Mobile 3D Mapping and ROS to Your Budget
Bilibot Is the Affordable ROS Platform You’ve Been Looking For
everything-robotic.com: Where are the robots? Where’s that $100 billion industry we’ve seen in the forecasts?
Open source DARwIn-OP bot can be yours for just $12,000
Used-Robots.com
robotsltd.co.uk: Industrial Robot Applications
7 Jobs That Can Be Done By A Robot

I couldn’t make it there tonite, but here’s the slideshare and PDF. It’s an excellent presentation.

Olve is a software developer at Cisco Systems. He has a blog and twitter.

Even if you’re not a full-time C/C++ programmer, it pays to stay current. I had to drop into C twice this year to solve problems, in addition to reading the usual strace/oprofile dumps.

My recommendation is to use lint and build with 2 different compilers, then try to fix all warnings. (Brian Aker goes further and says that all warnings should be fixed, and enforces that on the Drizzle project.)

Thanks again to Symantec for hosting the meeting.

cnn.com: 23 of the coolest QR codes
cnn.com: Does anybody actually use QR codes?
QRcodify.com
Perl Module GD::Barcode::QRcode
Successful QR Code Campaigns
msn.com: How QR codes hide privacy, security risks
theregister.co.uk: QR Codes and Spam

This is called domain name canonicalization.

There’s 2 things to consider:

1. what’s convenient for the webmaster
2. what’s best for SEO

Fortunately, using HTTPD2 name-based virtual hosts accomplishes both.

Here is an actual example where www.jebriggs.com is the main site domain name, and Redirect permanent is used to send a 301 redirect response of the alternate domain names to www.jebriggs.com:

<VirtualHost *:80>
    ServerName www.jebriggs.com
    DocumentRoot /home/james/public_html
    [...]
</VirtualHost>

<VirtualHost *:80>
    ServerName jbriggs.com
    ServerAlias www.jbriggs.com jbriggs.com jebriggs.com
    Redirect permanent / http://www.jebriggs.com/
</VirtualHost>

How To Test

You can use lynx with the –noredir option to verify that the redirection is working correctly:

lynx --noredir http://www.jebriggs.com/index.html (should show homepage)

lynx --noredir http://jebriggs.com/index.html (301 Moved Permanently)
lynx --noredir http://jbriggs.com/index.html (301 Moved Permanently)
lynx --noredir http://www.jbriggs.com/index.html (301 Moved Permanently)

Google Webmaster Guidelines
theseoace.com: What’s in a name? In SEO, potentially a lot

Tomer did a great job of answering a variety of technical questions.

In order to improve High Availability (HA), MapR has:

• rewritten Hadoop Distributed File System (HDFS) in C++ as MapR-FS. 3 compressed copies are stored. It is snapshottable and exportable as NFS.
• distributed the namenode across all storage nodes, resulting in 1000x namenode performance and avoiding Java GC
• made the jobtracker restartable by checkpointing
• granular permissions allow multi-tenancy
• added a nice mgmt. User Interface (UI) with heatmaps.

So MapR has replaced HDFS, and modified Apache Hadoop to change how namenode and jobtracker work. Hadoop-ecology layers above that are not affected, like Pig or Hive. Nobody has requested HCatalog support yet.

MapR benchmarks appear to be 2x to 4x faster than Apache Hadoop, esp. on small files, so a cluster would need half as many nodes. (See Slideshare presentations for benchmark graphs.)

There’s 2 MapR product versions:

1. MapR M3 Free Edition is non-HA and forum-supported.
2. MapR M5 includes HA and phone support and is $4,000/node/year.

EMC is reselling MapR. They are planning to build a 1,000 node cluster for demo purposes.

Currently, the deployed MapR clusters are small (in the range of 10 to 100 nodes), but MapR is designed to handle clusters of 10,000 nodes. Apache Hadoop maxes out at almost 4,000 nodes currently.

Thanks once again to Symantec for hosting the event.

MapR on YouTube
Ted Dunning’s MapR Talks on Slideshare YouTube: Localhost mounting of MapR NFS
Apache Hadoop FAQ
Tomer Shiran on Quora.com
Ryan’s Review of Hadoop Summit 2011
gigaom.com: Investors make $20M bet on MapR to win Hadoop war
gigaom.com: Why MapR Is Right to Give Back to Apache Hadoop
wikipedia: MapR