Friday, May 24, 2013

Announcing Crochet 0.7: Easily use Twisted from threaded applications

Crochet is an MIT-licensed library that makes it easier for threaded applications like Flask or Django to use the Twisted networking framework. Features include:
  • Runs Twisted's reactor in a thread it manages.
  • Hooks up Twisted's log system to the Python standard library logging framework. Unlike Twisted's built-in logging bridge, this includes support for blocking logging.Handler instances.
  • Provides a blocking API to eventual results (i.e. Deferred instances).
This release includes better documentation and API improvements, as well as better error reporting.
You can see some examples, read the documentation, and download the package at:

https://pypi.python.org/pypi/crochet

For those of you who have seen Crochet before, I'd like to feature a new example. In the following code you can see how Twisted and Crochet allow you download information in the background every few seconds and then cache it, so that the request handler for your web application is not slowed down retrieving the information:

"""
An example of scheduling time-based events in the background.

Download the latest EUR/USD exchange rate from Yahoo every 30
seconds in the background; the rendered Flask web page can use
the latest value without having to do the request itself.

Note this is example is for demonstration purposes only, and
is not actually used in the real world. You should not do this
in a real application without reading Yahoo's terms-of-service
and following them.
"""

from flask import Flask

from twisted.internet.task import LoopingCall
from twisted.web.client import getPage
from twisted.python import log

from crochet import run_in_reactor, setup
setup()


class ExchangeRate(object):
    """
    Download an exchange rate from Yahoo Finance using Twisted.
    """

    def __init__(self, name):
        self._value = None
        self._name = name

    # External API:
    def latest_value(self):
        """
        Return the latest exchange rate value.

        May be None if no value is available.
        """
        return self._value

    @run_in_reactor
    def start(self):
        """
        Start the background process.
        """
        self._lc = LoopingCall(self._download)
        # Run immediately, and then every 30 seconds:
        self._lc.start(30, now=True)

    def _download(self):
        """
        Do an actual download, runs in Twisted thread.
        """
        print "Downloading!"
        def parse(result):
            print("Got %r back from Yahoo." % (result,))
            values = result.strip().split(",")
            self._value = float(values[1])
        d = getPage(
            "http://download.finance.yahoo.com/d/quotes.csv?e=.csv&f=c4l1&s=%s=X"
            % (self._name,))
        d.addCallback(parse)
        d.addErrback(log.err)
        return d


# Start background download:
EURUSD = ExchangeRate("EURUSD")
EURUSD.start()


# Flask application:
app = Flask(__name__)

@app.route('/')
def index():
    rate = EURUSD.latest_value()
    if rate is None:
        rate = "unavailable, please refresh the page"
    return "Current EUR/USD exchange rate is %s." % (rate,)


if __name__ == '__main__':
    import sys, logging
    logging.basicConfig(stream=sys.stderr, level=logging.DEBUG)
    app.run()