Re: embedding, getargs/mkvalue asymmetry

Mark Lutz (mlutz@KaPRE.COM)
Wed, 15 Feb 1995 13:44:40 +0700

> I would have thought the same solution can hold. Obviously I dont userstand
> your application fully, but my first thoughts would be to:
>
...[run a python module to register callable objects on C start-up,
and use call_object() on these, not module/function names ]

Right!-- great idea. It adds a level of indirection between Python
and C, and offers more flexibility. This scheme makes a lot of sense,
in many (most?) cases.

But it's not really usable here, for 3 reasons:
- The extra impact of Python on the C++ system.
Most areas won't use Python: we want to minimize C++ changes.

- The required end-user knowledge.
We want the Python interface to be as simple as possible.

- Changing the database versus changing Python code.
We get the same effect by allowing users to add/modify function
names and expression strings in a persistent object store.

I'll explain more below. You may not agree with these [I'm not
even sure I do! :-)], but these are some real constraints.

Shipping another configuration file (the "startup.py" default
registration module) can be a concern [if you change and re-ship
it, you may blow-away on-site changes!], but this is a minor issue.

[Side-bar: there's a number of ways to structure indirection without
start-up-time registration. You might instead just always call a
single Python switcher routine, with the name of the action/function,
and the arguments passed in; roughly:
actions.py:
import defaults

def router(action, args):
if action == 'THE_EXPRESSION':
apply(mycust.MyExpression, args) # changed
elif action == 'SOMETHING_ELSE":
apply(defaults.something_else, args) ...

or just index a standard dictionary:
actions.py:
import defaults

actions = {
'THE_EXPRESSION' : mycust.MyExpression, # changed
'SOMETHING_ELSE' : defaults.something_else, ... }

etc., and let users edit these. If you're going to open-up
name-to-callable mapping to the end-user, these are equivalent
to "registration", are more 'python-like', and may by simpler (?).
They also don't require a startup action from C++. We looked at
these, but dismissed them for the same reasons. Of course, none
of this applies to string-expressions...]


> For example - lets say you provide "customise.py". This contains:
> def SomeExpression():
> return something1 + something2 * something3
>
> If the string "customise.SomeExpression" is effectively hard-coded in 'C',
> users will have no choice but to modify your Python code.

For our users, this may be a Good Thing: it keeps life simple-- both
theirs and ours! ;-)...


> Somehow, it doesnt seem right for magic function names to be imbedded in the
> application C code. Would the above be an option?

Yes; in fact it's already possible in my system, but I have to
give you some more context to explain why...

Users are allowed to 'register' Python function/string actions that
the C++ system doesn't know about in advance, by storing function
names (or expression strings) in persistent data-base objects.

For instance, one Python client application allows the user to setup
arbitrary lists of function names on-site; at run-time, these are all
called as tests (and receive passed-in C++ objects).

Whether or not to 'hard-code' magic function names in the C code is
really a end-user question for us, and varies per application area.

- In some cases, we really want to hard-code both module and function
names, to make things simple: users go right to a file, and code or
tailor an expected routine.

- Sometimes, only the module-name is predefined: modules are used to
logically partition python actions (functions or strings), and
group them by application area.

- And other times, we leave it wide open, and let the user define the
module, and function-name or expression strings in the database.

This last case is similar to your idea, but there's no notion of
calling a Python registration routine on C++ application start-up.
That's a reasonable approach, but not usable here, because:

- We want Python impact to be minimal, since it will only be
used fairly rarely; the only place where our C++ system should
even touch Python is when an action is really called. Most app's
won't use Python, but all share start-up code (well, roughly...).

- We want to make things simple for the end-users: we're assuming
that on-site users will be relatively 'technically-naive' about
Python. A worst-case-complexity action might look like:

file PostActions.py:
import tools

def validateAccount(acct, company):
if acct.balance > 100000:
tools.error("balance too big!")
return 0
if acct.bookType not in company.defaults.bookTypes:
tools.message("check the book-type")
return 1

C++ caller:
Python::run_function("PostActions", "validateAccount",
"i", &result,
"(O&O&)", toThing, acct, toThing, company);

This can get arbitrarily complex (users can call their own logic
from the called action), but in the common case, it's pretty simple
for the on-site customizer-- they don't need to know about a start-up
routine, registering functions, first-class object semantics...

Users just add/change things in the database, to schedule/register
new actions. This is equivalent to changing registration code, but
makes more sense for our product/customers.

There's a lot more to this scheme, and there's no way I can explain
it all here (I've already gotten much deeper than I wanted to! :-).
Your scheme is sound, but won't apply here unless we change the
problem definition.

The point I was really after was whether or not an embedded-call
API based on module/function-names would be of interest. Sounds
like there's a wide variety of ways people are doing embedding,
so standard higher-level API functions may or may not be useful.

I don't care either way [they were really slated to be discussed
in this book I'm trying to get published... :-)]. Anyone else
have any experience they care to relate?

Thanks for the input,
Mark L.