Login Main site Create account

17.06.2005 09:07

CGI Post method handling in bash scripts

I just wanted to post my way of handling CGI POST method Request Strings using bash here because I didn't find anything useful in the web straightaway. The following code reads all Data from stdin (which is POST's way to send you the data) and parses it according to the CGI protocol rules.
The main problem is, that all special charactes such as exclamation marks, white spaces, german umlauts, etc. are being encoded as %xx, where xx is the hexadecimal code of this ascii character. So one has to transform these occurences to their representatives in the ascii-table.
Please keep in mind, that this example is written for use with busybox so I had to include some tricks that might not be necessary on normal installations (e.g. busybox's sed is not able to display ascii characters with the notation \xnn (where nn is any hexadecimal number)).

Update 2006-04-21:
There was an error in this script that caused it to behave incorrectly if a percent sign (%25) was found in $POST_STRING. Here's the updated version:
if [ "$REQUEST_METHOD" = "POST" ]; then

  while true; do
    HEX="$(echo "$POST_STRING" | sed 's/^.*[^%]%\([0-9a-fA-F][0-9a-fA-F]\).*/\1/')"
    if [ "$HEX" = "$POST_STRING" ]; then
    REP=$(echo -e \\x$HEX)
    # to avoid mishandling of %25 (== '%') replace all occurences of the
    # percent sign itself with a double percent sign (kind of escaping)
    [ "$REP" = "%" ] && REP="%%"

    POST_STRING="$(echo "$POST_STRING" | sed 's/^\(.*[^%]\)%\([0-9a-fA-F][0-9a-fA-F]\)/\1'$REP'/')"

  # replace all escaped percent signs with a single percent sign
  POST_STRING=$(echo $POST_STRING | sed 's/%%/%/g')

  # replace all ampersands with spaces for easier handling later
  POST_STRING=$(echo $POST_STRING | sed 's/&/ /g')

POST_STRING then contains the decoded Parameter-List which could be fed into a bash-array (of course, that's also not possible with the busybox builtin, because there's no declare plugin (not at the moment, at least)).
In my application I did also use the following line afterwards to replace all ampersands by spaces because I used " for token in $POST_STRING" afterwards (you may also set the IFS environment variable to "&" if you like).
  # replace all ampersands with spaces for easier handling later
  POST_STRING=$(echo $POST_STRING | sed 's/&/ /g')
If you have any idea of how this could be done better, don't hesitate to contact me.
Comments added earlier to http://tuxx-home.at/archives/2005/06/17/T09_07_39/index.html:
Eric Janz on 2005-10-28 15:17:00 wrote:

there is a little problem with this script if we encounter a % in the posted data. % will appear as %25 and will be substituted with % by the script. But this causes an infinite loop!!

I'm working to find a solution....
Alexander Griesser on 2006-04-21 09:06:56 wrote:
OK, today I had a need for this script again and looked
over the code and reproduced the issue.

See the article again, there's an update that fixes this
Guest on 2007-11-17 18:49:22 wrote:
POST_STRING=$(echo $POST_STRING | sed 's/&/ /g')
may also be written as:
thereby avoiding a subshell, pipe and invocation of `sed`.

Thanks for the script! :)
Regards: Johannes ( http://johannes.truschnigg.info/ )
Alexander Griesser on 2007-11-17 18:59:36 wrote:
Of course, feel free to do that :)
At the time I wrote the script, I was using it in combination
with a small busybox ash that wasn't capable of such
variable substitutions and therefore I didn't make use of them, but thanks for the suggestion ;)
Guest on 2007-12-19 07:09:30 wrote:
The while loop unnecessarily complicates matters:

# substitute URL encoded (% escaped) characters with
# octal (\ escaped) characters and finally interpret them as
# ASCII characters
POST_STRING=$(echo -e ${POST_STRING//\\%/\\x0})

or the corresponding command with sed if variable substitutions are missing. Much better, no?

Naveed Hasan
Alexander Griesser on 2007-12-19 08:58:13 wrote:
You may need to do a `make modules_prepare` in your kernel source directory.

Your comment (HTML tags will be stripped !!):

To verify You are not a bot, type down text from this image.

Your try: