I wish I could find a job that allowed me to work on Frontier and/or Python, that didn't require me to move. I like where I live. I like my house.

But still. I'd love to do this stuff full-time.

12:15:22 PM    comments ()  trackback []  

Perhaps it's obvious, but when I'm done, I really need to write up how I did the integration.

I had the thought that when I was done with Python, it would be interesting to see what other languages I could get to work with Frontier. Perl and Ruby are obvious candidates, but I don't know what's involved in embedding and extending them, and that's exactly what the integration job is -- you embed the language within Frontier, and then write extensions to allow them to play with the Frontier environment.

TCL comes to mind as a simple candidate, since it was designed from the beginning as an embedded language.

I suppose I could appeal to the Paul Graham crowd and drop in Scheme or some implementation of Common LISP...

Once I formalize the steps needed, there's no reason why Frontier couldn't be the most programming language agnostic scripting platform around.

11:16:49 AM    comments ()  trackback []  

You know, I'm just not comfortable with my code unless I've had to fix something.

I was working on the integration into the Frontier scripting environment, and I wanted to make sure that Python threads worked in the system. As an experiment, I tried the "one line webserver" trick. As expected, it didn't really work, and locked up the UI in the process. I'm not quite sure how I'm going to solve that problem, but I decided to work around the issue by spawning a thread that launches the webserver. It worked, kind of, but I was not getting any execution time on the spawned thread. I was getting some, but not much.

It turns out that if you don't handle the Global Interpreter Lock (GIL) yourself, Python errs on the side of caution, and holds it for you while it has called back into your code. This is fine for your normal Python extension class, but Frontier is not your normal Python extension class. I want to allow Python to run freely as long as I'm not doing anything to it.

I looked at the Python docs, and it looked like the construct offered by Py_BEGIN/END_ALLOW_THREADS macros was where I need to look.

But they do it backwards -- the thought is that you are writing a Python extension that needs to do something time consuming, so for this small block of code, allow the threads to run. But what I want is the exact opposite, I want the threads to run freely by default, and then I want to stop them when I need to access the interpreter.

My first implementation, written using the entry points behind the macros (PyEval_Restore/SaveThread), seemed to work. Threads ran freely. But then I started running some regression tests. I ran my tests that access the Frontier ODB. Almost immediately, Frontier crashes. Right before the crash, I saw the message:

Fatal Python error: PyThreadState_Get: no current thread

I do a quick Google search on the error message, and see lots of complaints, and no real resolutions.

I go back to the docs, and try to find an alternative. After reading around for 10 minutes, I discover that I've glossed over exactly the solution I was looking for. I can use PyEval_Acquire/ReleaseLock to handle the GIL the way I originally wanted to handle it. The problem is that unless someone has started a thread, there is NO GIL. Quick solution -- right after starting the interpreter, call PyEval_InitThreads to ensure that we are in a threaded mode, and that the GIL exists.

I made those changes, ran into a few deadlocks, fixed them, and then everything just works. The regression tests all pass, threaded code runs freely, life is good.

I love it when a plan comes together. :)

11:08:50 AM    comments ()  trackback []