Archive
Goodbye to Google+
Google+ is officially shutting down for consumers on 4/2/2019 and I have surprising feelings about it as a product and a codebase, having worked to re-write it in my time as an engineer at Google in 2014.
“Looks like you’ve reached the end” – Indeed we have, G+
I’d never been an active G+ user and found the Circles concept confusing/clunky and the feature set too large. I’ve always viewed Twitter as the ideal, time-based news feed. When Polar as acquired by Google and we joined G+, we began an exciting project to overhaul G+ with product focus and a delightful newsfeed experience, mobile first.
Internally, we re-wrote a Java monolith using a new isomorphic (client/server-side) web component framework and micro-services. We designed and built mobile-first creating fast, responsive experiences that worked on both small phones and desktop displays. We built on the success of Google Photos to embed, analyze, and render content in the news feed beautifully.
Re-writing an application from scratch is generally not the right solution to your problems and often fraught with peril. Our thesis was to make a faster, mobile first implementation using a new internal framework and simplify the product by cutting features. Both of these were essential pre-requisites to executing a successful re-write. We also had the infrastructure to build/validate our prototype and then migrate the application URL path-by-path from the legacy monolith into the new system. We let early-adopters beta test the new system. This was a technical success, though not without toil and setbacks.
I’ll never forget the full, public launch in the small hours of the night when we ramped production traffic to 100% from the legacy to new version of G+ – only to have our backends overloaded and latency spike. After 24 hours of urgent debugging, we found a browser toolbar sending high volumes of traffic from all over the world to load a formerly minimal page – just to scrape the unread post count for a user. In the re-write, this page contained an expensive fully rendered news-feed. With the query cost for that page grown dramatically, the traffic overloaded our newsfeed backends. We blocked the traffic and re-launched successfully.
Another time, we experienced a production outage when users were served up a blank, white page. The feature flag/experiment system had a bug resulting in users being shown the HTML markup for one variant and the CSS for another – causing no visible content on the page. Yet everything was working as far as our automation was concerned and it took manual bug reports to realize what had happened.
I will always remember the teams building G+ and the expertise, idealism, and excitement of the staff in the Social product area fondly.
I won’t miss Google+ in my life, but I am proud of what we built and how we built it.
Mapping Rails 3.0 Commands
Rails 3.0 introduces ./script/rails
When Rails 3.0 was released, all of the individual command utilities (the Rails console, Rails generator, etc) were all consolidated into a single script:
./script/rails
That’s Annoying
Being used to typing “./script/console” for the last 4 years of my life, this is annoying. It’s a lot easier to tab complete within the ./script directory to the exact command you want and then fill in the arguments.
My Answer: rails_command_stubs
To solve this, I built a wrapper script you can drop in your Rails ./script directory. You can then symlink in commonly used commands and reference them directly and they pass through to the ./script/rails equivalent.
./script/console production # Calls ./script/rails console production
Guerilla Scrum: Minimum Viable Process
TL;DR
You can start with the minimum viable process right now (Standups, a Backlog, and Demo/Retro). You can use this foundation to build a customized process framework that works for your team.
Frustrations with Heavy Process
The Software world seems split between process dogmatists and pragmatists. Dogmatists believe that the entire canon of Scrum practices must be enacted as a whole, or “you’re doing it wrong.”
I couldn’t disagree more – every process was designed to solve a problem in a particular context. This attitude runs rampant, and many teams chafe under the weight of heavy process. This post aims to enumerate the minimum viable scrum process.
Think Critically: A Process Should Solve a Problem
Any process or rule should solve a specific, measurable problem. Processes should exist to make your life better. You should know the problem(s) each process is intended to solve and answer:
- Is it working?
- Do you still need it?
Guerilla Scrum: Core Processes
I call these core Scrum processes Guerilla Scrum, the minimum viable process to get started and generate what you need:
- Daily Standups
- Single, Prioritized Backlog
- Fixed Iterations
- Demo & Retrospective
Passenger Resource Collision
Passenger and Smart Spawn Mode
Phusion Passenger has an intelligent application server worker pool based on Apache for hosting Rails apps. It uses a clever forking process spawner with Ruby Enterprise Edition to Pre-load your Rails application and environment in a parent process before forking off Child workers.
This allows you to load your application once for N workers, speeding up startup time and reducing memory usage dramatically with REE copy-on-write support.
Preventing Resource Collision
However, there is a hidden risk in forking children this way. Each child inherits an initialized Rails environment from the parent, which can cause resource duplication or collision. Passenger resets the database connection out of the box – each Rails worker has it’s own ActiveRecord connection pool.
However, other resources like Redis or Memcache are NOT reset or protected this way. When deploying a production scale application it’s essential to implement this. Otherwise, you could have each of 8+ workers on one server trying to re-use the same connection or file-handle.
PatientsLikeMe solution – Rails Initializer to Reset Resources
require 'passenger_safe_resource'
# Initialize Redis
PassengerSafeResource.load('redis') do
env = ENV["RACK_ENV"] || ENV["RAILS_ENV"] || "development"
redis_conf = YAML.load_file("config/redis.yml")[env]
fail "No configuration for #{env}" unless redis_conf
host, port, db = redis_conf.split(':')
REDIS = Redis.new(:host => host, :port => port, :thread_safe => true, :db => db)
end
# Reset Vanity Redis Connection
PassengerSafeResource.load('vanity') do
Vanity.playground.reconnect!
end
# Reset Memcached Connection
PassengerSafeResource.load('memcache') do
PassengerSafeResource.reset_memcache(Rails.cache)
end
Helper to Manage Passenger events
module PassengerSafeResource
# Helper to reset memcache connection
def self.reset_memcache(cache)
return unless cache.kind_of?(ActiveSupport::Cache::MemCacheStore)
cache.instance_variable_get(:@data).reset
end
# Helper to load/reset a resource with Passenger
def self.load(resource_name, &block)
if defined?(PhusionPassenger)
PhusionPassenger.on_event(:starting_worker_process) do |forked|
if forked
Rails.logger.info "PassengerSafeResource(#{resource_name}): Forking Child, loading in child"
yield
else
Rails.logger.info "PassengerSafeResource(#{resource_name}): Non-Forking Spawn, NOOP"
end
end
else
Rails.logger.info "PassengerSafeResource(#{resource_name}): Non-Passenger mode, loading resource"
yield
end
end
end
Best of 2010 – Top 5 Books/Albums/Films
Books
- William Gibson – Zero History
This book is the culmination of the trilogy of present-day futurist fiction from Gibson, a great adventure through the streets of London and Paris in search of underground fashion and conspiracy. - Freedom – Jonathan Franzen
This is the first novel to live up to The Corrections, beautiful writing, horribly true to life characters, and a gift for capturing the modern American situation. Deeply human and flawed characters grappling with building a life, being parents, and finding meaning despite the unintended choices that lead us to our lives. - Although, of Course, You End Up Becoming Yourself – David Lipsky
Less a book than a fairly raw presentation of a 48 hour road-trip/interview with David Foster Wallace, a solid meal for anyone interested in the world of writers, the life of DFW, or interesting conversations about making a life for yourself.
Protecting Yourself from Firesheep Using an SSH Tunnel
What is Firesheep?
Firesheep is a recently released packet sniffer with built in side-jacking, that monitors insecure networks (usually open WiFi) for web application traffic, steals session information, and automatically impersonates your logged in session to many sites (Google, Facebook, Yahoo, etc).
SSH Tunneling/Proxy in OSX
The simplest way to protect yourself is to establish a secure VPN/tunnel for all of your web browsing to prevent sniffing of that traffic on the network. This moves the insecure traffic between the server and the web application and off of the local network your browsing.
If you have access to a Linux server with SSH, you can build a local SSH tunnel from a port on your machine out through the server to the internet. For those of you at PatientsLikeMe, dev2 is a great server to use for this. Below is an example SSH command to load a persistent SSH tunnel with a SOCKS proxy locally forwarding traffic over it.
ssh -D 8080 -f -C -q -N wpeterson@dev2.plm
MacBook Pro SSD Upgrade
I’ve been suffering with abysmal disk performance on my work laptop for some time, so I decided to pick up an OCZ Vertex2 SSD on sale over Black Friday and 8gigs of RAM. Total cost for both was under $300 (though the same from Apple would have been about $1200).
I wanted to get OSX re-installed, data migrated, and my development environment setup on my own time without wasting productive work time, so I did the following:
- Installed SSD in USB enclosure
- Installed OSX on SSD Drive as secondary
- Migrated Data/Apps from old internal drive
- Boot off of USB/SSD and finish config
- Install SSD as primary drive
Everything went smoothly. Here’s the disk portion of a system XBench benchmark before/after:
Overall, about an 8x performance increase split between 4x for Sequential Read/Write and 28x for Random Access. The system boots up in about 15 seconds now. Applications load very quickly, Chrome with 10-12 saved tabs opens in about 1 second.
