Velocity Conference Santa Clara 2014 Tips Game Cards

Featured

The O’Reilly Velocity Web Operations & Performance Conference is June 24-26 in Santa Clara.

Next to the messages/jobs board was a Web Ops & Performance Tips board:

- use source maps to debug compressed JS and CSS
- use ::before to optimize font rendering
- use local storage to persist markup and templates to reduce requests and payload
- avoid CSS block rendering in chrome by not using screen media type until after. Then put screen back to element
- use gatling stress tool for load generation/perf testing (Apache Licence 2.0)
- learn curl
- learn POSIX before recreating another tool that already exists. Bill Joy (?)
- “if you do it more than twice a week, automate”
- it takes no skills to do NoOps! :)

Yahoo Mail Basic is Broken, but Full Featured Works

If you’re still using Yahoo Mail Basic version and have frequent connection timeout problems since mid-2013, then I recommend clicking on the upgrade link.

Full Featured version (free) does not appear to have those problems. The main drawback of this version is that Inbox paging was replaced with infinite scrolling.

Yahoos: something is misconfigured in handling Yahoo Mail Basic web browser requests …

It usually takes 3 Send requests to send an email after composing it, and often the same for refreshing Inbox. The browser status bar says, “Waiting for us-mg6.mail.yahoo.com…” for a long time on connection attempts.

Full Featured appears to work fine on the same computer (Macbook Pro Leopard, Firefox browser and Comcast Internet connection.

Oh, and please add a Settings option to disable infinite inbox scrolling. :)

AWS Pop-up Loft, San Francisco

Featured



Amazon Web Services pop-up loft (Ask an Architect area, lecture hall, kitchen/lounge)
Photo credit: Amazon.com.

I happened to be in SF today, so I went to the Amazon Web Services pop-up loft on Market St.

Amazon rented an empty storefront for 4 weeks for lecture sessions upstairs, and a computer lab and an ‘Ask an Architect’ bar downstairs.

One of the hosts said the loft was a shell in May, and they had to build out everything: the kitchen area, 2 bathrooms and various partitions.

I asked the experts about new EBS and RDS features, and they had answers as well as a $100 AWS credit.

The weather was sunny and warm in SF.

Lots of street performers and hustlers, including a very smooth male R&B singer. A young rapper named Rap2K15 was selling hand-made CDs.

Update 2014 06 23: Apparently a drawing was held, and I was one of 3 winners of a free general pass to the AWS:Reinvent Conference :)

Update 2014 06 24:

AWS Bootcamp

Full-day AWS overview, including EC2, S3, RDS, VPC and IAM, with 2 labs.

“Provisioning and Managing AWS Infrastructure with Chef” with special guest George Miranda, Chef Technical Consultant, Chef

George talked about using Chef tools like chef metal, knife and chef zero and a minimal amount of ruby to make an AMI and provision a MySQL server and 5 Nginx web servers.

Slides

@gmiranda23, chef-ami-factory

Update 2014 06 26:

Dealing With Obstacles at Scale, Bob Hagemann, Twilio

To reduce pain:

- UTC timezone
- UTF8
- use thin AMI and chef/puppet instead of thick AMI
- wrote boxconfig a few years ago (like netflix asgard)
- remote admin mainly
- small teams 3-8
- services should run in 3 AZs
- monitoring with nagios, cron, pingdom
- haproxy on each host as proxy
- MySQL, MHA, LVM. Manual failover.
- SQS DLQ
- global low latency with route53
- http://github.com/twilio
- @bobzilla42
- Uses freeswitch plus own telcom sw
- billing system 100s QPS
- Ops team is about 8 people
- VPNs to HQ and carrier-approved colo
- three founders, one came from Amazon.

925 Market Street, SF
June 4 – 27, 2014 (likely closed on the 27th for dismantling)
Free registration, tshirts and lunch. Closes 5:30 pm, 6:00 pm or 8:00 pm daily.
Muni 30 and 45 return from Market St. and 5th to Caltrain.

@AWSstartups #AWSloft

Restaurant Review: Arka Indian Restaurant, Sunnyvale

I had dinner at Arka Indian Restaurant and Bar in Sunnyvale tonite.

It is a large restaurant on El Camino, perfect for office get-togethers of 20 or more diners. Very nice ambience with Indian decor inside and an outdoor veranda/patio area. Lots of parking. (They also have a Fremont location.)

The menu is a la carte Indian, though a few of the items could serve as a complete meal.

I ordered the Malai Murgh Kebab ($11), which is chicken kebabs on a plate with a tiny salad of raw onions and lettuce, preceded by the complementary puffed rice. The chicken medallions were boneless, tasted good, very similar to tandoori.

A friend says the appetizers are delicious, especially the chick peas.

Dinner averages $20/person plus drinks.

Arka “Festive Indian Flavors” Indian Restaurant, Bar, Lounge, Banquet
725 S. Fair Oaks Ave (at El Camino)
Sunnyvale, CA 94086
408 RED ARKA (733 2752)
FAX 408 608 6666

Advanced Liquibase Techniques

Featured

Liquibase LogoI recently did some work with liquibase. Here’s some techniques for advanced users to workaround limitations to calculate query cost.

Liquibase Introduction

Liquibase is an Open Source (Apache 2.0 License) Java utility and API for specifying and versioning schema changes (DDL) for several popular databases. It is commonly introduced to projects by programmers, rather than DBAs.

What liquibase can do:

  • allow “refactoring” of SQL schema changes to target multiple databases using XML by using a database-independent syntax, or raw SQL, depending on your preference
  • allow conditional execution and rollback of SQL based on database type or environment.

What liquibase can’t do:

  • has no built-in provisions for operational concerns, like conditionally executing SQL based on time/cost. There’s an assumption that schema changes are online, often true on Oracle and SQL Server, less so on MySQL, especially prior to 5.6
  • does not do intelligent merges to the same object across changesets, like adding multiple columns to the same table in one statement.

How liquibase works:

  • the programmer specifies schema changes in Java, XML or JSON and runs the liquibase command
  • liquibase creates 2 tables in your database to store version, user and patch name information and to lock out other simultaneous liquibase runs.

How to Make Liquibase Consider Cost for MySQL

After some experimentation, there’s a couple liquibase features you can use to do more advanced things:

  1. create a savepoint using the tag and rollback options:
    • liquibase tag rel0; liquibase update …; liquibase rollback rel0
  2. prepend and append logic to each changeset to use information_schema on the SQL DDL statement. on failure, exit with 1 (See XML example below)

changeset.xml:

<?xml version="1.0" encoding="UTF-8"?>

<databaseChangeLog
  xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog
       http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.1.xsd">

    <changeSet id="1" author="james">
     <sql>
       create table if not exists `profiling` ( `connection_id` int(11) not null default 0, `query_id` int(11) not null default '0', `state` varchar(40) default '', KEY (query_id));
       truncate table profiling;
       set profiling=1;

       alter table department add column test2 int default null;
       insert into profiling (connection_id, query_id, state) select connection_id(), query_id, state from information_schema.profiling where query_id=2;
     </sql>
     <rollback>
        <sql>alter table department drop column test2</sql>
    </rollback>
    </changeSet>

    <changeSet id="1-post" author="james">
      <preConditions onFail="HALT">
        <sqlCheck expectedResult="0">SELECT count(*) from profiling where state='copy to tmp table'</sqlCheck>
      </preConditions>
    </changeSet>
</databaseChangeLog>

Notes:

  1. the changeset DDL statement will still have run, even if the precondition HALTs – they’re separate changesets, after all
  2. the rollback in “1″ will not be executed, even if “1-post” HALTs.

The workaround for those 2 issues is to combine the two techniques in a shell script:

#!/bin/bash

liquibase tag rel0

liquibase update changeset.xml || {
    # fail the build pipeline to not propagate changeset to next stage
    # (ie. don't run in production)
    liquibase rollback rel0
    mysql -e 'alter table test.department drop column test2' 
    exit 1
}

The above looks a little kludgy, but provides a stepping stone for the reader to customize in their particular environment. (The preConditions and bash script can be easily autogenerated with a Perl or Python script.)

An alternative to XML is using the Java API to set everything up.

Please leave a comment if you have any suggestions or a Java API program.