Re: Python portability planning ( and prototypes )

Steven D. Majewski (sdm7g@aemsun.med.Virginia.EDU)
Wed, 5 Feb 92 15:15:08 EST

me> Using the success/failure of modules posix|mac|dos|{whatever} to
me> determine what machine & system the program is executing on
me> ( which is needed to do un-portable things portably ) is not
me> a good idea because it will conflict with 'aliasing' the modules
me> to provide portability. ( for example: making a posix.py module
me> for mac that attempts to provide some posix compatible emulation. )

guido> This depends on what you want to do. If the attempt at posix
guido> compatible emulation is good enough it doesn't matter whether
guido> os.system == 'posix' or not, as long as I can say posix.stat(filename).

Agreed!
But what I'm aiming for is that source code libraries be portable
*without* renaming or editing files. [ It is not a big bother right
now, but it will grow as the libraries grow. I'm trying to nip library
management in the bud.] So if there IS posix emulation, then either
POSIXposix, macposix, or dosposix will be loaded. ( Not exactly accurate,
I know posix & mac are "built-in", but you get the idea. ).

True: it is not a *frequently* needed feature. It will (probably) only
be used (occasionally) when module are imported.

guido> I don't see much use for knowing the CPU type, byte order, word size
guido> and floating point format unless pack/unpack get implemented. Come
guido> on, most C code doesn't know or care about these (surely 99% of my own
guido> code doesn't make any assumptions beyond what ANSI C guarantees), so
guido> why should Python programs care?

You got it backward: If there is a low level implementation of pack/unpack,
then we *DON'T* need to know byte-order or work size. If pack/unpack and
other network & native binary conversions are done in python-source, THEN
we DO need byte-order and word size. [ I don't really care about CPU type -
that was only because posix uname ( or arch ) seemed to be the only
(indirect) way of inferring the byte-order & (possibly) work size. ]

Again, I admit: Infrequently needed. But essential for a whole class of
problems. C network programs use htons(), etc. to convert to/from native
and network byte orders. C programs are FULL of "sizeof()"'s [ The need
for which disappears in Python 98+% of the time, but again, that 1 or 2%
either shuts out a whole class of problems or it forces some one to dig
into internals ( either of Python or the machine ) to find out the answer
and HARD-WIRE it into their program - the extreme of non-portability! ]

I think the minimum necessary functionality and the most portable
way of providing it is still an open question. [ And it is the question
I was trying to raise in the original pack/unpack discussion. ]

I may be more concerned about these things than some other folks:
I'm currently trying to move 10 YEARS and HUNDREDS of TAPES worth
of non portable vax vms record structured files some with VAX ( non IEEE )
floating point data into a portable network-neutral (i.e. readable by any
machine without any intermediate conversion program) format, so I may
be a bit more obsessed about this than the average programmer.
[ Plus all of the data from PC's & Mac's that get ftp-ed to and from
Sparc's, Vaxen & RS6000's et.al. ]

The ideal is to make both program and data portable. But we're not there
yet, so the next best thing is to provide a "portable" was to handle
non-portable objects. [ ntohs(),htons(),etc. is an example:
network-byte-order-16-bit-to-host-byte-order-short ( and vice versa )
For some machines that function is a no-op. But all portable network
programs that use binary data NEED that function to BE portable. ]

[ I think that aesthetically, Guido, you want things to be *SO*
portable that you DON'T NEED to know WHAT machine you are running on,
( or what byte-order is native, etc. ) but I'm argueing that the only
way to MAKE some things have a portable interface is for some level
to know that information, and modify it's actions accordingly. ]

You might reply that binary file conversion is not a proper job for Python.
If I want to get down and twiddle bits, I should be using C!
However one of the typical uses of Awk/Perl/et.al. ( and I would think
Icon & Python both ) is to read files in one format and spit them out
in another format. Usually these are ascii character files, but the
problems with binary files are not very different.

[ BTW: A different but related problem is a "portable" 'file' program.
Someone was working on a Perl solution a while back. 'file' itself
is limited: it matches fixed fields and there is a double byte order
problem - the byte order of the file and the byte order of the
machine that 'file' is running on, that made it impossible to
make a portable version of /etc/magic' that could be shared by
different machines. The best solution I saw ( other that the perl
re-write of file ) was a version of /etc/magic that was run through
the C pre-processor to make local variants. But any one local copy
was not sharable. Before I ran into all these problems, I was
considering a crusade to convince developers of archiver,compression,
graphics,etc. programs to provide a "magic" file and make their
installation scripts (if any) append their "magic" to "/etc/magic".
I nice idea that turns out not to be as simple as it sounds. ]

[ various interesting comments by Guido on portable pathnames deleted ... ]

That is also something I'm only STARTING to think about!
It is a similar problem. How to make the different path name syntax
and conventions appear portable, without making some native files
un-nameable and un-reachable.
*IF* we need a routine to convert to/from a cannonical ( probably
posix style ) pathname to/from a local style pathname ( I'm not sure
that we do, but I think so, unless everyone remembers to only use
the least common denominator path names, or unless that conversion
is built into the builtin module's "open" . ) then that module has to
have some local information!

( I haven't even read all the threads in comp.lang.{tcl,perl} about
handling unix-ism's - only noted that there was a discussion.
Maybe we need a comp.port !!! [ for cross language/systems
discussions of portable ( from the user syntax/interface ) ways
to handle non-portable features. ] )

- Steve