No Time To Play

Building A Game Application Server (part 4)

by on Jun.17, 2011, under Gamedev

Python Powered Logo from Python Site

Alright!… so now we’re close to having a somewhat functional product.  In that, we will have something that can accept connections and have some interactivity between those connections.

We talked about architecture, interthread communications, and network code.  Now, let’s integrate a script language.  The script language is going to be responsible for the game logic — basically, anything we want to be able to change really easily.  For this example, we’ll use Python because it is very easy to integrate.

The python interpreter is going to run in its own thread so it is not required to block when we do other operating-system level things like connection I/O or database/file access.  As a word of warning — Python and threads don’t really play so well together.  Using this model that I am about to describe, only one thread may use Python C-library calls! Meaning, you can’t have your connection thread or the main thread or anything else do a Python C call.  Chances are, the program will segmentation fault if you try.

The reason is because python doesn’t really multi-thread all that well.  You can actually run separate python VMs in different threads if you want, or you can use python’s critical section controls — but if you do that, good freaking luck.  Poorly documented and bad performance because python’s critical section handling is really primitive.  This doesn’t necessarily make Python bad for what we’re going to do — many, if not most, games are not multi-threaded anyway.  And we’re still getting thread benefits in the rest of our system, it’s just the game logic will be single threaded.  Personally, I don’t think that’s a bad thing.  However, a point that we should make clear here is that you must understand how well your chosen scripting language works in a threaded environment.  Some work better than Python, some are even worse — that is important research to do up front.

We will make a python bridge.  It will work much like our network thread; we will have a function that kicks off the python thread, and then use input/output queues to communicate with python.  Our header file for this is actually very simple with just three calls in it:

/**
 * Initializes Python -- this gets us set up to run.  Returns 0 on success,
 * -1 on error.  Errors will be printf'd and should never happen on a well
 * behaved system.
 */
int     pybridge_init();

/**
 * Starts the python game thread.  Uses the global pybridge_running to
 * determine if it's running or not -- 1 means it's running, 0 means it's
 * off.  It is set to 1 before kicking off it's thread, and set to 0 before
 * the thread exits.
 */
void    pybridge_start();

/**
 * Cleans up after python bridge system, assuming bridge thread has already
 * terminated.
 */
void    pybridge_cleanup();

We could make a structure with state information if we wanted and put the I/O queues in the state structure, but for now I will make the queues global.  You may think that’s a really bad idea; however, my thought at this time is I’m not sure what parts of the rest of my program will need to talk to python.  I’d rather start very permissive (with the queues being global) and later tighten it up then paint myself in a corner.  I would need to use a state structure (kind of like SERVER in the network library) if I was going to use multiple python VM’s…. however, multiple python VMs would be really heavy in both memory and CPU, and therefore I am not inclined to do that unless there is no other way to do what I want to do.

Before we go into implementation of the methods, let’s learn a little about how python interacts with its “parent” C program.  We will provide calls to python by presenting our program as a python module that we will call “pybridge”.  As far as python is concerned, the program it is running inside is just another python module; in fact, our scripts will “import” pybridge to access our pybridge methods.  As such, our pybridge_init call will set up a “pybridge” module and export a set of methods to pybridge.  Here’s the implemtation:

/**
 * Initializes Python -- this gets us set up to run.  Returns 0 on success,
 * -1 on error.  Errors will be printf'd and should never happen on a well
 * behaved system.
 */
int     pybridge_init() {
    PyObject        *psys,*ppath,*empty;

    /* Include our import */
    PyImport_AppendInittab("pybridge", PyInit_pybridge);

    /* Boot up our Python and our threads */
    Py_Initialize();
    PyEval_InitThreads();

    /* Auto-import our module */
    PyImport_ImportModule("pybridge");

    /* Add the current directory to the path */
    psys = PyImport_ImportModule("sys");

    if(!psys){
        printf("Could not load python sys module\n");
        return (int) -1;
    }

    ppath = PyObject_GetAttrString(psys,"path");

    if(!ppath){
        Py_DECREF(psys);
        printf("Could not get python path attribute.\n");
        return (int) -1;
    }

    /* Try to append to the path. */
    empty = PyUnicode_FromString("");

    PyList_Append(ppath,empty);

    /* Clear refs */
    Py_DECREF(ppath);
    Py_DECREF(empty);
    Py_DECREF(psys);

    return (int) 0;
}

Points of interest!  PyInit_pybridge is a method defined in pybridge.c which bootstraps our python module — the python module bootstrap consists of a couple different C structs which provide the PyDocs for the module and the mapping of C methods to python module methods.  There’s a lot of documentation already out there for doing this, so I won’t go too deep into it — read the code, read the links, and you’ll get it!

I’m developing using Python 3.  Python 3 has a couple of finnicky differences vs. Python 2, and unfortunately there is not much documentation going into the ins and outs of Python 3 integration yet.  One oddity is that the current working directory of the project is not part of the include path by default when you boot up python 3.  pybridge_init has a block of code (starting from PyImport_ImportModule(“sys”) and ending at PyList_Append(ppath,empty)) which adds the current working directory to the file path.  This is good for development, but later we may want our python files to live in a different directory from our executable file — a simple tweaking of this code will permit us to load our python files from whatever directory we want.

The next point of interest is the method that spins up our python thread:

/**
 * Starts the python game thread.  Uses the global pybridge_running to
 * determine if it's running or not -- 1 means it's running, 0 means it's
 * off.  It is set to 1 before kicking off it's thread, and set to 0 before
 * the thread exits.
 */
void    pybridge_start() {
    if(pybridge_running) { /* Already running? */
        return;
    }

    /* Set up the global queues */
    if(pybridge_system_in) {
        q_free(pybridge_system_in);
    }

    if(pybridge_system_out) {
        q_free(pybridge_system_out);
    }

    if(pybridge_io_in) {
        q_free(pybridge_io_in);
    }

    if(pybridge_io_out) {
        q_free(pybridge_io_out);
    }

    pybridge_system_in = q_create(pybridge_system_in);
    pybridge_system_out = q_create(pybridge_system_out);
    pybridge_io_in = q_create(pybridge_io_in);
    pybridge_io_out = q_create(pybridge_io_out);

    /* Set running flag, spin thread up */
    pybridge_running = 1;
    if(pthread_create(&pybridge_thread, NULL, pybridge_main, NULL)) {
        perror("Could not spin python thread up");
        pybridge_running = 0;
    }
}

This looks remarkably similar to our network.c server init call — we set up our queues, we spin the thread, and we exit. pybridge_main actually runs the Python:

/**
 * This is the main python game thread.  It runs the python game loop
 * and provides the 'shell' for python to run in.
 */
void*   pybridge_main(void* pyqueues) {
    PyObject*   type, *value, *traceback, *str, *by, *ret, *loopModule, *loopMethod;

    loopModule = PyImport_ImportModule("loo");

    if(loopModule == NULL) {
        printf("Could not load python module\n");
        pybridge_running = 0;
        return (void*) NULL;
    }

    /* Run the module */
    loopMethod = PyObject_GetAttrString(loopModule, "mainloop");

    if(loopMethod && PyCallable_Check(loopMethod)) {
        ret = PyObject_CallObject(loopMethod, NULL);
        Py_XDECREF(ret);

        PyErr_Fetch(&type,&value,&traceback);
        if(type || value || traceback) {
            PyErr_NormalizeException(&type,&value,&traceback);

            str = PyObject_Str(type);
            by = PyUnicode_AsEncodedString(str,"utf-8","Error ~");

            printf("Python Exception: %s\n",PyBytes_AS_STRING(by));
            Py_DECREF(str);
            Py_DECREF(by);

            if(value){
                str = PyObject_Str(value);
                by = PyUnicode_AsEncodedString(str,"utf-8","Error ~");

                printf("Python Exception message: %s\n",PyBytes_AS_STRING(by));

                Py_DECREF(str);
                Py_DECREF(by);
            }

            if(traceback && (traceback != Py_None)){
                by = PySys_GetObject("stderr");

                if(by && (by != Py_None)) {
                    PyTraceBack_Print(traceback, by);
                }
            }

            Py_XDECREF(type);
            Py_XDECREF(value);
            Py_XDECREF(traceback);
        }
    } else {
        printf("Could not call mainloop method");
    }

    Py_XDECREF(loopMethod);
    Py_XDECREF(loopModule);

    pybridge_running = 0;
    return (void*) NULL;
}

This looks for a file called “loo.py” (Loopy for our game’s main loop … get it? Yeah, okay, I’m terrible) and tries to call a method called “mainloop” in it. If mainloop throws an exception (i.e. has an error) we will display it.

Now, we just need some methods and a python loop script! If you’d like to get ahead of the curve, the full pybridge.c is here. Next installment, we’ll finish up and have something that actually runs.  We’ll also go a little into Python reference counting and some more fun details around memory management on our whirlwind tour.

:, ,

Comments are closed.

Posts by date

June 2011
M T W T F S S
« May   Jul »
 12345
6789101112
13141516171819
20212223242526
27282930  

Posts by month