Wednesday, April 10, 2013

Crochet: Background Operations for Threaded Applications

In my previous post I showed Crochet doing a blocking call against a Twisted API. In this example, you can see how Twisted and Crochet allow you to run an operation in the background. An HTTP request for a new user starts a download in the background, and a reference is stored in the user's session. Every time the user reloads the page, a check is made to see if the download is finished, and if it is done it is display. You can also see the stash()/retrieve_result() API in use, which allows temporarily storing results under a key suitable for serialization in a session object.

import logging
from flask import Flask, session, escape
from crochet import setup, in_reactor, retrieve_result, TimeoutError
setup()

app = Flask(__name__)


@in_reactor
def download_page(reactor, url):
    """
    Download a page.
    """
    from twisted.web.client import getPage
    return getPage(url)


@app.route('/')
def index():
    if 'download' not in session:
        # @in_reactor functions return a DeferredResult:
        result = download_page('http://google.com')
        session['download'] = result.stash()
        return "Starting download, refresh to track progress."

    # retrieval is a one-time operation:
    result = retrieve_result(session.pop('download'))
    try:
        download = result.wait(timeout=0.1)
        return "Downloaded: " + escape(download)
    except TimeoutError:
        session['download'] = result.stash()
        return "Download in progress..."


if __name__ == '__main__':
    import os, sys
    logging.basicConfig(stream=sys.stderr, level=logging.DEBUG)
    app.secret_key = os.urandom(24)
    app.run()

No comments:

Post a Comment