I love posting about problems on my weblog. Inevitably, if there's a solution out there, someone points me at it.

This is exactly what happened with my previous entry. Thanks, Frederik, and thanks, Patrick!

Needless to say, it will be possible to do things the way I want to do them, importing python modules directly from the Frontier object database.

Now the frustrating part is waiting until I can get home to do the work.

1:30:37 PM    comments ()  trackback []  

python integration status (two steps forward, one step back)

I've been quiet lately, for a couple of reasons. One reason is that I haven't been making the progress that I'd like on the integration -- life keeps getting in the way. Another is because I'm close. Very close.

I finally came up with a satisfactory way (at least for me) of handling Frontier internal types that don't map nicely to Python types (like outlines or wp objects). If you look at the UserTalk code that plays with these kinds of types, you'll notice that most of them play with these types indirectly. So as long as you can call the appropriate UserTalk code to manipulate these types, you are fine treating them in an opaque fashion.

I've also come up with a way to call any standard UserTalk routines. Before any Python code can play with Frontier internals, it needs to import the new "frontier" builtin library. This library provides some helper routines, but most importantly it provides access to UserTalk code through the "UserTalk" (synonyms "usertalk" and "ut") object. If you want to call dialog.notify(), you would say UserTalk.dialog.notify("foo"). If you didn't want to type all of that, you can use one of the synonyms (I use "ut" everywhere), or do the Python trick of "renaming" the routine, like this: dn = ut.dialog.notify; dn("foo")

I don't know why it took me so long, but I finally integrated error reporting into the system, so when the Python code fails to compile (or fails to successfully run) you will get a standard Frontier error. I still need to go the other way -- UserTalk errors need to propagate Python exceptions if you call UserTalk from Python.

nasty bugs

I finally ran into a really nasty bug. I'd been successful, up to this point, at avoiding errors that trashed the database, but I finally hit one. Once I figured out the problem (and the solution), it amazed me that I had gone so long without running into something like this.

Frontier stores data using a structure called a "valuerecord". Valuerecords contain a tag that describes the type of the data, and a handle to the actual data.

My method of handling opaque Frontier datatypes is to create a Python object that holds on to a valuerecord. As an example, if I want to assign a Frontier odb address to a Python variable (I treat addresses opaquely), I can execute the following Python statement:

adr = ut.address("scratchpad.output")

After that statement, adr is an object of type frontier.InternalValue. At this point, I am free to pass that object to other bits of UserTalk code that operate on address, such as "wp.newTextObject"

The problem was that once I started playing with code of this sort, strange things started occuring. I would call wp.newTextObject, and the new text object woul dbe created, but not in the spot I expected it. Specifically, at one point I actually ended up blowing away the whole "wp" table, replacing it all with a wptext object.

As you can imagine, this is not good, and led to much swearing and trepidation, as well as a search of my hard drive for a backup of my Frontier.root file.

The problem was this -- I was not telling Frontier that I had stashed that valuerecord. It went ahead a reused it for something else, and that something else became the target of the wp.newTextObject script. The result speaks for itself.

I've checked in the code that fixes this. It was a simple fix, once I'd figured out what the problem was.

importing code from the object database

I'd really like to be able to somehow hook the import code to make it search the Frontier object database for Python libraries. It would go far in helping me create a complete integration of Python and Frontier.

As far as I can tell right now, I would need a custom build of Python to be able to do that. I really don't want to go in that direction -- I want Frontier to be able to use the Python you already have on your system. In fact, thanks to Andre Radke, we have written our code in a way that should support any Python installation that is at least version 2.3.

I've looked into the "imp" library, and contemplated writing my own __import__ method, but there's a basic problem -- no matter what direction you take, eventually Python wants a file to process. The problem is that I want to import Python that's in the object database, and it's not in the form of a file, it's a string.

When I wrote the Python tool (the "Using Radio as a Python IDE" tool), I got around this by mirroring the Python code to the file system. I could do the same thing here, but it just doesn't feel right. It feels like a stopgap solution. I want it to be seamless. If I need to write the code to the disk so that I can import it, I want it to happen without any action from the user, and I'm still trying to figure out the best way to do that.

In the meantime, I suppose I could come up with an "export Python library" script that people could invoke directly. This would have the added benefit of easily allowing people to export Python code from the object database. For symmetry, I would need to provide an "import Python library" script, and an "import/export Python file" script to be complete.

10:32:43 AM    comments ()  trackback []