22. How to stream uploaded files directly to disk

By default, CherryPy handles in the same way all data posted to the server through a form. In all cases, CherryPy converts that data to a string or a list of strings. This data can be a short string or a big file that's being uploaded.

In most cases, this is very convenient and it works very well. However, if your application requires users to upload very big files, then converting them to a string and having this string in memory can be a problem ...

In that case, you may want to write these files directly to a file instead of having them in memory.

Here is how this can be done:

Here is an example code that does this:

import cgi

def initRequestBeforeParse():
    if request.path == 'postFile':
        request.parsePostData = 0

CherryClass Root:
mask:
    def index(self):
        <html><body>
        <form method=post action=postFile enctype="multipart/form-data">
            Upload a file: <input type=file name=myFile><br>
            <input type=submit>
        </form>
        </body></html>

view:
    def postFile(self):
        # Use cgi.FieldStorage to parse the POST data
        dataDict = cgi.FieldStorage(fp=request.rfile, headers=request.headerMap, environ={'REQUEST_METHOD':'POST'}, keep_blank_values=1)

        value = dataDict['myFile']

        # Value has 2 attributes:
        #    - filename contains the name of the uploaded file
        #    - file is an input stream opened for reading

        f = open('/tmp/myFile', 'wb')
        while 1:
            data = value.file.read(1024 * 8) # Read blocks of 8KB at a time
            if not data: break
            f.write(data)
        f.close()

        return "<html><body>The file has been saved in /tmp/myFile</body></html>"

Note that the file is not streamed directly from the browser to /tmp/myFile. Instead, it is saved in a tempfile by the cgi module and then streamed from this tempfile to /tmp/myFile.

See About this document... for information on suggesting changes.