Before you declare me an insane old man, I have to say that I needed it for an embedded board with only 64MiBs of RAM. And (mostly due to the specifics of the tasks), I ended up using bash hooked to lighttpd via cgi. Bringing on heavy artillery (php or python) really complicated the task, since I mostly needed the outputs of different shell utils.
Anyway, I quickly got to the part where I needed to handle POST requests to upload binary files. First, I googled for a solution, and one of them eventually did what I needed. But alas, it had a drawback.
If you have a look at the source from the link
if [ "$REQUEST_METHOD" = "POST" ]; then TMPOUT=/tmp/fwupdate cat >$TMPOUT # Get the line count LINES=$(wc -l $TMPOUT | cut -d ' ' -f 1) # Remove the first four lines tail -$((LINES - 4)) $TMPOUT >$TMPOUT.1 # Remove the last line head -$((LINES - 5)) $TMPOUT.1 >$TMPOUT # Copy everything but the new last line to a temporary file head -$((LINES - 6)) $TMPOUT >$TMPOUT.1 # Copy the new last line but remove trailing \r\n tail -1 $TMPOUT | perl -p -i -e 's/\r\n$//' >>$TMPOUT.1 fi
You will see that a temporary file is used. And that pretty much sucks, since my flash media on that board was horribly slow, I didn’t want to waste erase cycles, nor did I have enough RAM to handle all of that in tmpfs. Ideas? Here goes my implementation of that… thing.
#!/bin/bash
OIFS="$IFS"
read boundary
read disposition
read ctype
read junk
#Holy fuck, this sucks!
#Due to \n\r line breaks we have 2 extra bytes per line read,
#6 + 2 newlines == 10 junk bytes
a=${#boundary}
b=${#disposition}
c=${#ctype}
a=$((a*2+b+c+d+10))
IFS="${IFS}&:"
set $QUERY_STRING
dir=$1
file=$2
SIZE=$((HTTP_CONTENT_LENGTH-a))
echo "Content-Type: text/html"
echo ""
echo "Upload complete, $SIZE bytes stored
"
echo "Written to $dir, filename $2 ($dir/$file)
"
dd ibs=1 obs=512 count=$SIZE of=$dir/$file
That’s it! No temporary files, no need to buffer the whole file in memory, like PHP does. Just one plain dd and some utter voodoo.
No POST variables can be handled, so instead I pump two variables via GET, separated by a ‘:’.
In this raw way it’s pretty much a hell of a security hole, but since the thing I’m working on will be unlikely to ever be exposed to the internets, and online only when there’s a user nearby – why not?
Hi, thanks for the tips!
Here’s how I do it, with a POST form upload (slight variation of your script) :
echo "Content-type: text/html"
echo ""
echo ""
cat header.html
maintenance
debugmsg "HTTPS=$HTTPS"
if [[ "$REQUEST_METHOD" == "POST" ]]; then
debugmsg "CONTENT_LENGTH=$CONTENT_LENGTH"
if [[ $CONTENT_LENGTH -gt 0 && $CONTENT_LENGTH -le $MAX_FILE_UPLOAD_SIZE ]]; then
echo "Uploading file, please wait ..."
# single file upload
# we must strip the first 4 lines (http headers) of received data, and the last 2 at the end
read hline1 </dev/stdin && debugmsg "$hline1"
read hline2 </dev/stdin && debugmsg "$hline2"
read hline3 </dev/stdin && debugmsg "$hline3"
read hline4 </dev/stdin && debugmsg "$hline4"
# calculate remaining extra bytes to strip at the end of data (boundary string + extra cr/lf)
size=$(( ${#hline1} + 5 ))
outfile="uploads/tmpfile"
# read remaining data from stdin, except the last $size bytes (http headers)
head -c -$size "$outfile"
if [[ $? -eq 0 ]];then
echo "File has been uploaded successfully"
else
echo "Error : Unable to write file!"
fi
else
echo "File is too big for uploading, maximum allowed file size is $MAX_FILE_UPLOAD_SIZE bytes"
fi
else
echo "Error: Invalid method"
fi
@Fredo: Nice, thanks, looks like a cleaner solution.
For info, I noticed that some http clients do not work well with this method : the native web browser on Android 2.3.6 on my smartphone, for exemple (badly formatted http headers, although I’m not sure if they are really breaking the rules, I would need to read the RFC’s!).
But Firefox Mobile and Opera on the same platform work OK 🙂