Christian Simms's Software R&D

Introducing Botonomy LLC

I recently started a company called Botonomy LLC, and our first product is ProjectPipe, a subscription service to provide project-management thru the web. I will be updating my (Christian Simms) weblog more frequently than this one.

Evaluation of 3 Python Tools to Develop Apps with Postgresql

No, these aren't just Object Relational mapping tools. After doing applications that use O-R for several years now, I've decided that you can't completely hide the relational features from the object interface, so don't try. I looked at the following tools:

Roundup issue tracker This is an issue tracker whose design is so good you can build an application on it. It's basically a Python app server.
- Clever Features
  - Really simple way to specify schema. Below creates 2 tables where second table has foreign key to first:
```
						 project = Class(db,"project", name=String(), creationDate=Date())
						 task = Class(db,"task", name=String(), project=Link("project"))
					  
```
    The above causes 2 main tables to be created, plus 2 shadow audit tables, plus integer primary keys. Plus it creates indexes for the foreign keys.
  - Works out of the box: just cd to its directory and run "python demo.py" and you have a fully functional bug tracking system.
  - Automatic schema upgrader -- when app server starts up, it compares current python model (as defined above) with database's schema (stored in one row of special table schema which contains a repr()'ed dictionary of information about each Class that maps to a table
  - Mail integration - has a mail filter to receive bug reports through email. This filter also archives email-based bug discussions, attaching the discussion to the appropriate bug. Wow this is a killer feature for a bug-tracking system.
- Flaws
  - Instead of db-level triggers, it has app-level triggers -- changes made through python are stored in audit tables, but those directly through db are not.
  - Inflexible naming - for class project, it creates tables: _project and project__journal.
  - Hardcoded to use psycopg postgresql db driver (minor flaw).
Modeling (Object-Relational Bridge for python) This is a heavy-weight modelling tool, this is what a committee would design. Reminds me of EMF (Eclipse Modelling Framework) which is just as heavyweight. If you think the way to start app development is to write a big UML model, then this is for you!
- Clever Features
  - Has a Zope product which lets you edit your model through Zope. This sounded cool to me, but when I tried it out, it's basically a thin client data entry app -- sorry I don't need that, I like having my model defined in a text file that I can version and use in text editors.
  - Really flexible - let's you specify lots of details about each object and field, either in XML or python model. Also supports run-time APIs or generated code APIs.
- Flaws
  - Heavyweight and complex. For instance, instead of writing SQL queries, it has its own SQL-like (but not SQL!) language.
Twisted Enterprise ROW (R)elational (O)bject (W)rapper. A really simple way to do twisted programming against an existing db schema (yes it doesn't create the schema for you). It requires you to write a class for each table, and in this class write out how to map db columns + their types to python attributes, including PK and FK relationships.
- Clever Features
  - Maps to whatever your existing db schema is.
- Flaws
  - Does not create schema for you.

Intro to Business Strategy

One of the leaders of modern business strategy is Michael E. Porter, professor at Harvard Business School. He presents a framework for analyzing any industry by breaking looking at five forces:

Degree of Rivalry
Threat of Substitutes
Buyer Power
Supplier Power
Barrers to Entry

If one was to start a new business, Ken Hess recommends that you apply this type of analysis to your choice of business.

Interesting Async Programming Paradigm

Here's a good paper describing the approach that the Twisted networking framework uses when structuring asynchronous client programs. The paper mentions the influence of Smalltalk, one of my favorite languages. If you want to play with SmallTalk, go to Squeak and play with it! You can also learn about an alternative to MVC programming, because that's what Squeak has. The MVC successor is called Morphic programming. It's hard to describe, but it's basically direct manipulation development. What you're working with under your mouse *is* the object that you're developing. I believe that Apple's Interface Buildercopied this model. I say copy because Morphic programming comes from the people who originally developed SmallTalk, and Apple grabbed the ideas we're used to like overlapped windows, mouse, IDEs, from the original SmallTalk developers.

Build Automation

I've had great success in the past setting up automated build systems for Java projects. The one I've personally used several times is CruiseControl. It takes a while to configure it, but once it's running it's pretty stable. It seems like each major release they make they redo the internal architecture. You can set it up to monitor activity in your source control system, and when commits happen, it can then kick off a new build (after waiting for a specified quiet period of say 60 seconds). This build process can include:

checking out the latest code
doing a clean build
starting the app server
running unit tests
shutting down the app server
updating the build statistics page
sending an email about success / failure of build and tests

I found an interesting real-time build monitoring system called CIA Open Source Notification System. It's an open source GPL software application (based on Python, Twisted, among others). Besides being a software project, it also has a live demo where it monitors about 200 open source systems. Their demo site (or is it an open source ASP?) shows live web pages showing overall activity on a per project basis. You can drill down on each project to see statistics of build / bug tracking activity for each project. Each project can define rulesets to control routing of raw events into its knowledge scheme. They also provide RSS feeds of the aggregated data. They have IRC bots, I think to monitor projects.

A competitor to CruiseControl is the BuildBot, which is written in Python and uses the Twisted network framework. Here's a sample page. I don't find the presentation intuitive. But one major feature it has over CruiseControl is that it supports distributing the build over multiple machines in a network to test multiple platforms.

My buddy Mike has a technology-related weblog as well.

Easiest way to create OpenGL scripts

The standardized OpenGL API is a pain in the arse to use, it's too low-level for the casual programmer -- e.g., it requires you to do your own double-buffering, plus no scenegraph API. There are many projects which provide easier to use wrappings of this API, but even these are often rather low-level. The simplest and highest-level wrapper I've found is VPython. It has some really impressive demos for such short programs. Its biggest drawback I believe is that it's difficult to extend it (e.g., image/texture mapping is missing) because it's basically python method wrappers of a C library the authors wrote.

Inside Story on SWT vs. Swing

I've spent some time recently learning Swing. It has a steep learning curve. I've worked w/GUI toolkits before, and it's rather complicated compared to the others.

I found an interesting email from Nov. 2002 about the history of Swing. Bottom line: when IBM was porting their Visual Age to Java, they found the Swing toolkit too buggy and difficult to use, so they developed SWT instead. I'm inclined to agree with them. Maybe it's time to learn SWT!

Clever way to XML Schema

I don't know about you, but I find xml dtd's quite cumbersome to define -- so cumbersome that I have never written one. And xml schema language surely sucks in its verbosity. Here's another approach which I find pretty clever, where you specify an example xml document and the program (would you believe they implemented the program in xsl!?!) generates a Relax NG validation file.

Lisp-level macro system for Java: OpenJava

When I learned the C language years ago, I learned how to use its macro system, but also learned the nasty side effects that macros could cause if used or defined improperly.

Years later, I learned what Lisp macros can do, and I realized what a pale imitation C macros are by comparison. I believe the definitive description of Lisp macros is On Lisp by Paul Graham.

Apparently someone has gone ahead and provided a macro system for Java which is as powerful as Lisp's system instead of C's: OpenJava.

JBoss has integrated some sort of Aspect-Oriented features to JBoss. I like the idea of transforming source code at compile time, but I don't know if I like the idea of deploying an application, and then at run-time modifying the bytecodes. Then your app is some black box jumble of byte codes that have been mangled.

In my spare time, I've been developing a J2EE code generator using python.

However, I noticed that as I added new features, the performance was degrading considerably, so I decided to do some benchmarks. This is what I found.

Below, each column name is a link to the source code for the given language. Each program is written in the style for that language, so for instance the Python version uses the built-in dictionary for hash tables, whereas the Java version uses HashMap class. Notice how these tests don't do something simple like test using hashes of int's, because I haven't had to do that kind of thing since college.

Test	Python	Java (classic)	Java (hotspot)	C#	g++	g++ (-O3)
Dynamic Object Allocation	141,000/sec	168,000/sec	1,400,000/sec	52,300,000/sec	1,060,000/sec	1,450,000/sec
Dynamic Object Allocation plus hash table get/set	9,880/sec	4,670/sec	56,300/sec	53,800/sec	19,100/sec	30,200/sec

So what can we learn from this?

Java with hotspot compares favorably to C#
Java with hotspot is something like 10x faster than without
Python's about 5x slower than Java/C#
And most surprising (to me): Java and C# were both about twice as fast as C++! I had read that could be true, but seeing is believing.

I tried speeding up the Python code using the Psycho JIT compiler, but without much luck. Actually, it was slightly slower (maybe 10% slower), and I think I know why. The Python versions of these tests primarily test the memory allocator and built-in dictionary type, neither of which can be JIT'ed because they're already native in the Python language, so I just saw the overhead of the JIT).