A Resume-Based WSGI Load Balancer


PyCon 2012
March 9, 2012

Jim Fulton
jim@zope.com

Problem

Cache Architecture

caching.png

ZODB Object Cache

Client Cache

Application memory usage

Shared cache / Memcached

With a naive approach, each worker works on the same data.

Squid Caches

Spread corpus across workers

If workers could specialize, then they could work on smaller part of corpus.

Resume-based load balancer

Request classification

Load-balancing algorithm

Secondary benefit of algorithm

No request class uses more than ~ 1/variance fraction of users.

Worker resumes

Adding Workers

Removing workers

Network Architecture

network.png

gevent

ZooKeeper

Tree Example

/ee-resumelb
  variance = 4
  /ee-pool
    main -> /ghm/databases/main
    sessions -> /ghm/databases/sessions
    ugc -> /ghm/databases/ugc
  /providers
    /10.1.1.50:8080
      backdoor = u'127.0.0.1:55253'
      pid = 11375
  /workers
    history = 9999
    /providers
      /10.1.1.50:30894
        pid = 11064
      /10.1.1.50:35284
        pid = 11001
      ...

See http://pypi.python.org/pypi/zc.zk

Evaluation

Log Playback

Playback results

Production test

Other applications

Issues

Future: Response Caching

Status

Questions?

http://jimfulton.info/talks/resumelb.html

resumelbqr.png