How to Fix GMail Unread Messages Favicon

04/12/2013 Leave a comment

Living Without GMail’s Unread Message Icon feature

GMail provides an amazing Google Labs feature to show the unread message count in the Favicon of the GMail web page. If you use features like Chrome’s “Pin Tab” to manage your gmail tabs, this is essential to seeing when you have new mail. Unfortunately, this feature broke in public GMail sometime in October/November of 2012 based on public reports.

How to Fix Unread Favicon in GMail for Chrome

Thanks to Bertrand Schneider for the code snippet and fix.

Why did the favicon break?

It looks like there was a regression that causes two favicon link elements to be generated in the GMail markup, causing only the first to take precedence for the page (the default favicon) and the smart favicon with the unread count not to be shown.

Work-around for old GMail (on Google Apps domain)

Change the match string in the JavaScript only to match your public gmail account, for example https://mail.google.com/mail/u/0/* if your public GMail is your first logged in google account.

Categories: Uncategorized

Leaving Brightcove for Input Factory

03/10/2013 1 comment

Joining Input Factory

Brightcove vs. Input Factory
I’m leaving the App Cloud team at Brightcove for a new startup. Starting Monday, I’ll be the first employee at Input Factory – building a stable of mobile/social applications and taking ownership of the web and API components driving mobile apps.

Why Input Factory?

LukeWPolarJeff Cole
Input Factory was founded by Luke Wroblewski, an entrepreneur/world-class designer and Jeff Cole, co-founder/CTO at PatientsLikeMe. Polar, the first iOS project, lets you create and vote on simple A/B style polls.

I’m excited to build up the infrastructure for a new engineering team and launch some products.

Categories: Career, Startups

ActiveModel Type Coercion and API Validation

02/03/2013 Leave a comment

ActiveModel Type Coercion

ActiveModel is a powerful ORM that looks up the data types for model values and applies automatic type coercion and casting for user data. Data coming from Web requests as JSON or query parameters are usually cast from Strings into other datatypes, implicitly, during assignment. This is a helpful feature 99% of the time, but can be frustrating when invalid values are silently discarded. Especially if you’re trying to use model level validations for API validation.

Optional API Value Validation

As we built the API for App Cloud, we put together rules for handling JSON input and validating data. For optional user input values, we decided that if a user provided invalid data, we should return a validation error vs. silently discarding invalid values. For example, if there was a “duration” value, that was an Integer number of days a value of “foo” should produce an error. However, the default behavior is to silently discard a value like “foo” assigned to a numeric field.

We know this has happened when the coerced value is nil but there is some user data present before the coercion. There’s an example Validator below implementing this logic.

A Coerced Value Validator

Digging into the source code, ActiveModel first assigns the value for a field ‘duration’ into an attribute ‘duration_before_type_case’, which preserves the raw value before coercing the input based on the type of the attribute. Using this, we can mix-in validations for models in our API that will mark the record invalid if data is silently discarded during type coercion. Before this, providing ‘foo’ as an integer value results in nil but no errors. Now, we’ll capture the error on this input.

class CoercedValueValidator < ActiveModel::Validator
  def validate(record)
    coercible_attrs = record.keys.select do |k,v|
      [Integer, Float, Date, Time].include?(v.type)
    end.map(&:first)

    coercible_attrs.each do |attr|
      if record.send("#{attr}_before_type_cast").present? && record.send(attr).nil?
        record.errors.add(attr, 'is invalid')
      end
    end
  end
end
Categories: Database, Rails

Cookies in Hybrid Android Apps

01/17/2013 4 comments

Sharing Cookies between WebViews and native code

Hybrid mobile applications require a close collaboration between native code and WebViews. Unlike iOS, Android doesn’t provide any implicit bridge between HTTP cookies in WebViews and native HTTP requests made through HttpConnection or Apache’s HttpClient interfaces. As a developer, it’s up to you to manually synchronize these systems if you want to share state or authentication context between them.

This is often a useful feature for integrating third party web services via their login page in your app’s WebView. Authentication credentials stored as HTTP cookies should be shared for subsequent HTTP calls, whether they’re native or a WebView.

Android Cookie Management and Persistence

Add CookieManager and CookieSyncManager hooks to properly setup your application to accept and save cookies, while writing them to disk periodically or when the application pauses or shuts down.

Application

public class YourApplication extends Application {
  public void onCreate() {
    super.onCreate();

    //Setup Cookie Manager and Persistence to disk
    CookieSyncManager.createInstance(this);
    CookieManager.getInstance().setAcceptCookie(true);
  }
}

Main Activity

public class MainActivity extends BaseActivity {
  public void onResume() {
    CookieSyncManager.getInstance().stopSync();
  }

   public void onPause() {
     CookieSyncManager.getInstance().sync();
   }
}

Synchronizing Android CookieManager to Apache HttpClient

Now that cookies have an application wide cookie store that will save WebView cookies and persist the cookie store to disk, we need to sync cookies from this store for use by native code.

Synchronize native HttpClient with App’s Cookie Store
This example shows populating an Apache CookieStore configured in an HttpClient that is pre-populated with any HTTP Cookies that apply to this URL. Then, after executing the request, writing all of the cookies saves in the CookieStore into the App’s Android CookieManager.

public static String getURLContentsAsString(String url) {

  DefaultHttpClient httpClient =  new DefaultHttpClient();

  //Injects App's CookieManager cookies for this URL into HttpClient CookieStore
  syncCookiesFromAppCookieManager(url, httpClient);

  HttpGet hp = new HttpGet(url);
  HttpResponse response = hc.execute(hp);

  //Save's cookies from HttpClient in App's CookieStore
  syncCookiesToAppCookieManager(url, httpClient);

  // Process Response here
  //...
}

syncCookiesFromAppCookieManager()
This helper function fetches all cookies for this URL from the Android CookieManager, which gives us a raw Cookie HTTP header value. The Apache CookieStore expects Cookie objects, which we will generate using the RFC2109Spec that matches the format of the Android CookieManager.

public static void syncCookiesFromAppCookieManager(String url, DefaultHttpClient httpClient) {

  BasicCookieStore cookieStore = new BasicCookieStore();
  httpClient.setCookieStore(cookieStore);

  CookieManager cookieManager = CookieManager.getInstance();
  if (cookieManager == null) return;

  RFC2109Spec cookieSpec = new RFC2109Spec();
  String rawCookieHeader = null;
  try {
    URL parsedURL = new URL(url);

    //Extract Set-Cookie header value from Android app CookieManager for this URL
    rawCookieHeader = cookieManager.getCookie(parsedURL.getHost());
    if (rawCookieHeader == null) return;

    //Convert Set-Cookie header value into Cookies w/in HttpClient CookieManager
    int port = parsedURL.getPort() == -1 ?
      parsedURL.getDefaultPort() : parsedURL.getPort();

    CookieOrigin cookieOrigin = new CookieOrigin( parsedURL.getHost(),
                                                  port,
                                                  "/",
                                                  false);
    List<Cookie> appCookies = cookieSpec.parse(
      new BasicHeader("set-cookie", rawCookieHeader),
      cookieOrigin);

    cookieStore.addCookies(appCookies.toArray(new Cookie[appCookies.size()]));
  } catch (MalformedURLException e) {
    // Handle Error
  } catch (MalformedCookieException e) {
    // Handle Error
  }
}

syncCookiesToAppCookieManager()
This helper extracts Cookie objects from the Apache CookieStore and converts them into Set-Cookie header strings for the Android CookieManager.

public static void syncCookiesToAppCookieManager(String url, DefaultHttpClient httpClient) {

  CookieStore clientCookieStore = httpClient.getCookieStore();
  List<Cookie> cookies  = clientCookieStore.getCookies();
  if (cookies.size() < 1) return;

  CookieSyncManager syncManager = CookieSyncManager.getInstance();
  CookieManager appCookieManager = CookieManager.getInstance();
  if (appCookieManager == null) return;

  //Extract any stored cookies for HttpClient CookieStore
  // Store this cookie header in Android app CookieManager
  for (Cookie cookie:cookies) {
    //HACK: Work around weird version-only cookies from cookie formatter.
    if (cookie.getName() == "$Version") break;

    String setCookieHeader = cookie.getName()+"="+cookie.getValue()+
      "; Domain="+cookie.getDomain();
    appCookieManager.setCookie(url, setCookieHeader);
  }

  //Sync CookieManager to disk if we added any cookies
  syncManager.sync();
}

Monitoring Redis Replication in Nagios

12/13/2012 Leave a comment

I’ve put together the following Nagios plugin for monitoring Redis slave server replication to ensure replication is successful and the lag is within a reasonable time limit:

#!/usr/bin/env ruby
require 'optparse'
options  = {}
required = [:warning, :critical, :host]

parser   = OptionParser.new do |opts|
  opts.banner = "Usage: check_redis_replication [options]"
  opts.on("-h", "--host redishost", "The hostname of the redis slave") do |h|
    options[:host] = h
  end
  opts.on("-w", "--warning percentage", "Warning threshold") do |w|
    options[:warning] = w
  end
  opts.on("-c", "--critical critical", "Critical threshold") do |c|
    options[:critical] = c
  end
end
parser.parse!
abort parser.to_s if !required.all? { |k| options.has_key?(k) }

master_last_io_seconds_ago = `redis-cli info | grep master_last_io_seconds_ago`.split(':').last.to_i rescue -1

status = :o k
if master_last_io_seconds_ago < 0 || master_last_io_seconds_ago >= options[:critical].to_i
  status = :critical
elsif master_last_io_seconds_ago >= options[:warning].to_i
  status = :warning
end

status_detail = master_last_io_seconds_ago == -1 ? 'ERROR' : "#{ master_last_io_seconds_ago.to_s }s"
puts "#{status.to_s.upcase} - replication lag: #{status_detail}"

if status == :critical
  exit(2)
elsif status == :warning
  exit(1)
end

This can be wired up in Nagios with an NRPE remote execution check:

define service {
    name                            redis_replication
    register                        1
    check_command                   check_nrpe!check_redis_replication!$HOSTNAME$ 100 250
    service_description             Redis Replication
    hostgroup_name                  redis_slave
}
Categories: Operations

MongoDB Indexing, count(), and unique validations

11/10/2012 Leave a comment

Slow Queries in MongoDB

I rebuilt the database tier powering App Cloud earlier this week and uncovered some performance problems caused by slow queries. As usual, two or three were caused by missing indexes and were easily fixed by adding index coverage. MongoDB has decent index functionality for most use cases.

Investigating Slow count() queries

Unfortunately, I noticed a large variety of slow queries issuing complex count() queries like:

{ 
  count: "users", 
  query: { 
    email: "bob@company.com", 
    _id: { $ne: ObjectId('509e83e132a5752f5f000001') }
  }, 
  fields: null 
}

Investigating our users collection, I saw a proper index on _id and email. Unfortunately, MongoDB can’t use indexes properly for count() operations. That’s a serious drawback, but not one I can change.

Where were these odd looking queries coming from? Why would we be looking for a user with a given email but NOT a given id?

The uniqueness validation on the email key of the User document and many other models was the culprit. Whenever a User is created/updated, ActiveModel is verifying there are no other Users with the given email:

class User
  include MongoMapper::Document

  key :email, String, unique: true
end

Use the Source!

Why is a unique validation triggering this type of count() query? Within Rails 3.x, this functionality is handled by the UniquenessValidator#validate_each implementation, which checks for records using the model’s exists?() query:

  finder_class.unscoped.where(relation).exists?

The exists?() method is a convention in both ActiveRecord and MongoMapper, checking for any records within the given scope. MongoMapper delegates it’s querying capability to the Plucky gem, where we can find the exists?() implementation using count():

  def exists?(query_options={})
    !count(query_options).zero?
  end

Root Cause and a Patch to work-around MongoMapper/Plucky

In SQL, using count() is a nice way to check for the existence of records. Unfortunately, since MongoDB won’t use indices properly for count(), this incurs a big performance hit on large collections.

I added a MongoMapper patch to work-around the issue. We can patch the exists?() method to use find_one() without any fields instead of the expensive count() path:

module MongoMapper
  module Plugins
    module Querying
      module ClassMethods
        # Performance Hack: count() operations can't use indexes properly.
        # Use find() instead of count() for faster queries via indexes.
        def exists?(query_options={})
          !!only(:_id).find_one(query_options)
        end
      end
    end
  end 
end

Password Salting isn’t Just For Servers

06/07/2012 Leave a comment

Current Events: LinkedIn Password Breach

Everyone has been talking about the massive LinkedIn password breach. Apparently, unknown actors have compromised some subset of the LinkedIn user base and a password hash database has been released into the wild.

What Does it Mean?

The good news is that the password database isn’t associated with your login credentials and isn’t immediately exploitable by everyone in the world. The bad news is that this is certainly proof a group of bad actors have the full credentials for most of LinkedIn.

Weak Password Storage

It appears this breach was possible due to poor algorithm choices by LinkedIn, using a simple one-way hash for each password that can be cracked more easily than proper password storage. There are several great posts about how to do this right as an application developer, but this post is focused on the user.

What Does it Mean For You?

The real threat in breaches like this is that people share the same password on multiple sites. It’s easy enough to recognize when LinkedIn or one site has a breach and change your password, but what else could someone with your email/login gain access to if you re-use passwords like most people do?

What’s the Best Plan to Protect Yourself?

The best way to manage this is to use a password manager with random passwords for every site you authenticate to. However, this is too difficult and complex for the majority of people. There’s an easier way to use a few h3 secrets to create unique, user salted passwords.

What’s the Realistic, Everyday Plan?

If you can’t use truly random passwords and a password manager, you should generate a random 8 letter password, and then add a word or 4-6 letters to it to get a unique password just for that site. For example, if you’re strong password was “Nafra6he” you could make your password “linkedinNafra6he”. It’s just as easy to remember and only a little harder to type. If the password database for LinkedIn is compromised and you have a custom prefix on all your other sites, you won’t be at risk. For example, if your google password with this schema was “googleNafra6he”.

Categories: LifeHacks, Security
Follow

Get every new post delivered to your Inbox.