Saturday, September 13, 2008 
Cheetah Tips Cheetah Tips

At Cozi, we're writing our new web services in Python (a story for another day). I wrote up a few hard-won tips on using the Cheetah Template library at the Cozi Tech Blog.

posted on Sunday, September 14, 2008 2:34:49 AM (Pacific Daylight Time, UTC-07:00) 
#    Comments [0]
Friday, January 12, 2007 

http://www.georgevreilly.com/blog/content/binary/PythonBatch.jpg

Batchfile Wrapper

I've made some significant changes to my Python Batchfile Wrapper. The main virtue of this wrapper is that it finds python.exe and invokes it on the associated Python script, ensuring that input redirection works.

I've also adapted py2bat to work with my wrapper. I'm calling my version py2cmd.

Here's my latest batch file, which is shorter than its predecessor.

To use it, place it in the same directory as the Python script you want to run and give it the same basename; i.e., d:\some\path or other\example.cmd will run d:\some\path or other\example.py.

 @echo off
 setlocal
 set PythonExe=
 set PythonExeFlags=-u

for %%i in (cmd bat exe) do (
for %%j in (python.%%i) do (
call :SetPythonExe "%%~$PATH:j" ) ) for /f "tokens=2 delims==" %%i in ('assoc .py') do (
for /f "tokens=2 delims==" %%j in ('ftype %%i') do (
for /f "tokens=1" %%k in ("%%j") do (
call :SetPythonExe %%k ) ) ) "%PythonExe%" %PythonExeFlags% "%~dpn0.py" %* goto :EOF :SetPythonExe if not [%1]==[""] (
if ["%PythonExe%"]==[""] (
set PythonExe=%~1
)
)
goto :EOF

This is sufficiently cryptic that it merits some explanation.

The first set of nested loops attempts to find python.cmd, python.bat, and python.exe, respectively, along your PATH:

 for %%i in (cmd bat exe) do (
for %%j in (python.%%i) do (
call :SetPythonExe "%%~$PATH:j"
)
)

The %%~$PATH:j expression searches the PATH for %%j (i.e., python.cmd, etc). If it's found, the expression evaluates to the full path to %%j. Otherwise, it evaluates to the empty string. I've bracketed the expression with double quotes in order to handle spaces in directory names.

The SetPythonExe subroutine simply sets %PythonExe% to %1 if and only if %PythonExe% doesn't already have a value and %1 is not empty:

We can't set %PythonExe% directly in the loop. As explained at for loops and variable expansion, environment variables in the body of the loop are evaluated once before the loop starts and won't change until after the loop terminates:

 :SetPythonExe
if not [%1]==[""] (
if ["%PythonExe%"]==[""] (
set PythonExe=%~1
)
)
goto :EOF

Note: the %~1 notation strips off any surrounding double quotes. (ss64.com has details on parameter syntax.)

The square brackets and double quotes are necessary to make it all work if either %PythonExe% or %1 contains spaces. Getting this right was one of the hardest parts of the whole exercise.

The second set of nested loops are scarier:

 for /f "tokens=2 delims==" %%i in ('assoc .py') do (
for /f "tokens=2 delims==" %%j in ('ftype %%i') do (
for /f "tokens=1" %%k in ("%%j") do (
call :SetPythonExe %%k
)
)
)

The outer loop runs once: assoc .py yields .py=Python.File and %%i is set to Python.File. Running ftype Python.File yields Python.File="C:\Python24\python.exe" "%1" %* (on my machine).

The second loop also runs once: %%j is set to everything on the right-hand side of the =.

The third loop also runs once: %%k is set to the first token in %%j, "C:\Python24\python.exe", which is passed in to SetPythonExe.

At this point, %PythonExe% will have a value if python.cmd (or python.bat or python.exe) existed on your path, or the .py extension was registered.

If it doesn't have a value, then the invocation of "%PythonExe%" will fail, setting %errorlevel% to 9009:

 "%PythonExe%" %PythonExeFlags% "%~dpn0.py" %*
goto :EOF

%PythonExeFlags% was set to -u at the beginning of the script. As explained in my Python Batchfile Wrapper post, this treats stdin, stdout, and stderr as raw streams, instead of transliterating \r\n into \n. If you want cooked input, simply remove the -u.

The "%~dpn0.py" notation yields the absolute path to the Python script with the .py extension sitting beside this batch file: another example of parameter syntax.

Finally, goto :EOF ends execution of the batchfile, skipping the :SetPythonExe subroutine.

Whew!

py2cmd

You can have a batchfile sitting alongside a Python script as above, or you can have a self-contained batchfile cum Python script.

py2bat has been kicking around for years. It takes a Python script and turns it into a batchfile, by relying on a couple of tricks.

I've adapted py2bat into a new script, py2cmd. In essence, the generated batchfile looks like this:

 @echo off
REM="""
... set PythonExe as above ...
"%PythonExe%" -x %0
goto :EOF
"""

# python code starts here
# ...

When this file is executed by cmd.exe, the control flow should be obvious. Disable echoing to the screen, a funny-looking REM, set %PythonExe% as before (not shown), invoke python.exe with the -x flag on the current batchfile, and finally skip past the rest of the file.

When Python is invoked with the -x flag, it skips the first line of the script (@echo off). The second line sets the variable REM to the multiline string which continues down to the closing """ below the goto :EOF. Everything after that is the original Python script. All the batchfile nonsense is wrapped up inside the REM variable.

Download py2cmd.

Other Wrappers

Fredrik Lundh's ExeMaker generates a stub executable to launch a Python script with the same basename. It requires that Python already be installed on the target machine. I couldn't get ExeMaker to work properly. The stub executable leaves me at the Python interpreter's interactive prompt.

py2exe takes a Python script and bundles up all the Python support files to make it run on a machine that doesn't have Python installed. Works fine for me, but you get 4MB+ of associated runtime. Massive overkill if the target machine is known to have Python installed.

posted on Saturday, January 13, 2007 2:49:31 AM (Pacific Standard Time, UTC-08:00) 
#    Comments [0]
Thursday, December 28, 2006 

content/binary/PythonBatch.jpg

I've been getting into Python lately. One problem that I've encountered under Windows, is that input redirection doesn't work if you use the .py file association to run the script; e.g.:

 C:\> foo.py < input.txt

There's a well-known input redirection bug. The fix is to explicitly use python.exe to run the script.

A related problem for me was that stdin was opened as a text file, not a binary file, so \r bytes were being discarded from binary input files. The fix is to run python.exe -u (unbuffered binary input and output).

I didn't want to hardcode the path to python.exe in a batch file, so I came up with the following wrapper, which parses the output from assoc .py and ftype Python.File.

Just place this batch file in the same directory as foo.py and call it foo.bat.

 @setlocal
 @if (%_echo%)==()  set _echo=off
@echo %_echo% :: You must explicitly invoke python.exe, rather than rely on the :: file association for .py, if you want stdin redirection to work. :: See http://mail.python.org/pipermail/python-bugs-list/2004-August/024920.html :: The -u flag to python.exe specifies unbuffered, binary stdin, :: so '\r\n' is not remapped to '\n'. call :FindPythonExe if "%PythonExe%"=="" (
echo Can't find python.exe exit /B 1 ) :: Replace the extension of this batch file with .py: s/.bat$/.py/ set PythonFile=%~dpn0.py

"%PythonExe%" -u %PythonFile% %* goto :EOF :: :: Find python.exe in the path or via the .py association :: :FindPythonExe set PythonExe= :: Search for python.{cmd,bat,exe} in %PATH% for %%i in (cmd bat exe) do (
if "%PythonExe%"=="" (
for %%j in (python.%%i) do set PythonExe=%%~$PATH:j ) ) :: Extract path to python.exe from .py association if "%PythonExe%"=="" call :AssocPy2Exe goto :EOF :: :: Return the executable associated with .py in %PythonExe% :: :AssocPy2Exe call :AssocExtn2Exe .py
set PythonExe=%_exe% goto :EOF :: :: Return the executable associated with file extension %1 in %_exe% :: :AssocExtn2Exe :: assoc .py -> .py=Python.File for /f "usebackq tokens=2 delims==" %%i in (`assoc %1`) do set _ftype=%%i :: ftype Python.File -> Python.File="C:\Python24\python.exe" "%1" %* :: Grab everything after the '=' for /f "usebackq tokens=2 delims==" %%i in (`ftype %_ftype%`) do set _rhs=%%i :: Get the first token of the space-separated list for /f "tokens=1" %%i in ("%_rhs%") do set _exe=%%i goto :EOF

Now you can run foo.bat < bar.jpg with the expected results.

Enjoy!

Update 2007/01/03: The batchfile now searches %PATH% before looking up the .py association.

Update 2007/01/12: See here for a significantly improved batchfile and for py2cmd.

posted on Friday, December 29, 2006 1:47:28 AM (Pacific Standard Time, UTC-08:00) 
#    Comments [0]