Revision history [back]

I am writing a Contentsmanager for a Jupyter project, where we will be storing data in a SwiftStore.

I'm mostly getting there, and can upload data to the store:

# horribly paraphrased
def write(self, path, content):
    _opts = {'object_uu_threads' : 20}
    with SwiftService( options = _opts ) as swift, OutputManager() as out_manager:
        output = io.StringIO(content)
        response = swift.upload(self.container, [output])

(you will notice the contents are in a variable)

What I need to do is go the other way: download into a variable

I suspect it's something to do with the options passed into the download function.... perhaps the out_file parameter.... but

1. Is that actually what that parameter is for, and
2. How to I (pythonically) set that up to be a variable in my code?

I am writing a Contentsmanager for a Jupyter project, where we will be storing data in a SwiftStore.

I'm mostly getting there, and can upload data to the store:

# horribly paraphrased
def write(self, path, content):
    _opts = {'object_uu_threads' : 20}
    with SwiftService( options = _opts ) as swift, OutputManager() as out_manager:
        output = io.StringIO(content)
        response = swift.upload(self.container, [output])

(you will notice the contents are in a variable)

What I need to do is go the other way: download into a variable

I suspect it's something to do with the options passed into the download function.... perhaps the out_file parameter.... but

1. Is that actually what that parameter is for, and
2. How to I (pythonically) set that up to be a variable in my code?

## This works by downloading the file to disk then reading the contents of
## that file into memory, before deleting the file
## NOTE this is reading text files!
## NOTE this really only works with files in the local direcotry, but given local filestore will disappear when the docker ends, I'm not too bothered.
def read(self, path):
    content = ''
    with SwiftService() as swift:
      response = swift.download(container=self.container, objects=[path])
      for r in response:
        if r['success']:
          filename = open( r['path'] )
          content = filename.read()
          os.remove( r['path'] )
    return content

I am writing a Contentsmanager for a Jupyter project, where we will be storing data in a SwiftStore.

I'm mostly getting there, and can upload data to the store:

# horribly paraphrased
def write(self, path, content):
    _opts = {'object_uu_threads' : 20}
    with SwiftService( options = _opts ) as swift, OutputManager() as out_manager:
        output = io.StringIO(content)
        response = swift.upload(self.container, [output])

(you will notice the contents are in a variable)

What I need to do is go the other way: download into a variable

I suspect it's something to do with the options passed into the download function.... perhaps the out_file parameter.... but

1. Is that actually what that parameter is for, and
2. How to I (pythonically) set that up to be a variable in my code?

post-acceptance edit:

This is my current _read_read

## This works by downloading the file to disk then reading the contents of
## that file into memory, before deleting the file
## NOTE this is reading text files!
## NOTE this really only works with files in the local direcotry, but given local filestore will disappear when the docker ends, I'm not too bothered.
def read(self, path):
    content = ''
    with SwiftService() as swift:
      response = swift.download(container=self.container, objects=[path])
      for r in response:
        if r['success']:
          filename = open( r['path'] )
          content = filename.read()
          os.remove( r['path'] )
    return content

It is not elegant.