Swift middleware vs. multithreading

asked 2016-02-04 10:54:10 -0600

anonymous user

Anonymous

I would like to launch some async process in a swift proxy server middleware. At the moment, the 'process' does nothing (it simply waits for 5 seconds time.sleep(5)).

I tried two ways:

  • First try: my process is defined in a class extending multithreading.Thread class, and starts in each __call__ of my middleware
  • Second try: use of swift.common.utils.GreenAsyncPile, initialized in the __init__ function of my middleware, then use of spawn in middleware __call__ function.

The first try works with swift 2.5.0 (swift all in one, ubuntu server 14.04). With swift 2.6.0, as soon as the process starts, every request sent to swift during the process execution remains pending up to the end of the process, even if my process is well started in an async way (I can see a log immediately after a thread start call).

The second try behaves in same way in swift 2.5.0 & 2.6.0 (blocking call).

I've tried with multiple values for swift proxy server number of workers and size of the thread pool, without success.

I wonder :

  • Did I do it the wrong way ? I did not find doc or sample for doing this.
  • Is there any other way to define async behavior in a swift middleware ?

My middleware looks like:

import DummyTask
import DummyThread
from swift.common.utils import GreenAsyncPile

class DummyMiddleware(object):

    def __init__(self, app, conf=None):
        self.app = app
        self.pool = GreenAsyncPile(10)

    def __call__(self, env, start_response):
        t = DummyThread()
        t.daemon = True
        t.start()
        # or blocking call : self.pool.spawn(DummyTask().run)
        return self.app(env, start_response)
def filter_factory(global_config, **local_config):
    conf = global_config.copy()
    conf.update(local_config)

    def factory(app):
        return DummyMiddleware(app, conf)
    return factory

My task looks like:

import time
class DummyTask(object):

def run(self):
    time.sleep(5)

My thread looks like:

import time
from threading import Thread
class DummyThread(Thread):

    def __init__(self):
        Thread.__init__(self)

    def run(self):
       time.sleep(5)
edit retag flag offensive close merge delete