Improvise.it Code Dissection

Improvise.it intends to make a wiki-editable general simulator website and then harness the emergent phenomena for educational and entertaining purposes.

So what is a wiki? What is a general simulator? A wiki is a web site that allows users to modify the content and contribute to the content with submissions of their own. A general simulator is like the foundation of almost any game, it is the code that handles a numerical representation of a virtual world and presents it to the end user.

To create a wiki-editable general simulator one must construct an interface and a database, and the interface must enable modifications to the database. The information in the database that makes up the general simulator must be organized in a manner that mimics the real world.

The part of the code that makes modifications to the database is called the 'content management system (CMS)'. The initial code for the site came from an ancient CMS I had written that bore some superficial similarities to the modern wiki. It was a set of PHP scripts that interacted with a database and sorted documents in an infinitely traversable hierarchy. It also had a user management system that was tied to the document tree which restricted access based on 'authorship' or an arbitrary numerical access level. Many of the features of the CMS are still useful for Improvise.it.

This code was quite ugly and needed much rewriting, as I had written it before I knew the concept of "standards compliance"! I shudder to think at the number of tags I sent to a digital afterlife. I also lacked knowledge of PDO and had to replace a great many mysql calls.

So to design this it is best to break it down into components. If you are familiar with the MVC (Model-View-Controller) software pattern, you will perhaps suggest adopting it at this stage. The virtual world would need 'a model', this model would consist of information organized in the database. It would also need 'a view', some way of presenting that information to the user. And finally it would need 'a controller' or the piece of code that mediates the conversation between the model and the view.

Because I am not an experienced software architect ( "Gasp!" you gasp.), I chose instead to hack and slash my way through the task with the simple motto of spaghetti kludgists everywhere "if it works, move on." In the far recesses of mind, though, I knew there were silly gaps in the architecture, but I wanted to mash up a satisfactory prototype as quick as possible.

So instead of making a complicated graphical view, I chose instead to keep all the interface elements hyper-textual (anything the web can do, we can do- to present information from the model). This simplicity is important for scalability and for ease in the initial stages of construction. Graphic elements can be added within the context of the hyper-textual elements anyway, so why bother with them until the ball is actually rolling.

So to present textual things to the user we would need a chat. I, being the incompetent derelict that I am, searched for 'simplest php chat' and found one I liked and then began the task of merging the updated CMS code with the simple chat code. The chat code used polling, which means it was constantly asking the server to display the list of latest chat messages. This would turn out to be inefficient and was later replaced at the encouragement of redditor bastawhiz for a method called 'long polling' (which means asking the server 'Hey any new messages?' and the server would ignore the request until a message arrived).

Soon I had the chat placed squarely above the CMS. When a user logged into the CMS, they logged into the chat and a message was displayed to the other users. Then I had the chat notifying other users of CMS-events, like document creation, editing, deletion, user creation, user deletion.

At this point all the code is procedural (strictly functions- no classes or objects). It was a pure architecturally-lazy bliss, to be honest. I would just throw useful functions in a giant ugly file and always go by the path of least resistance. r/PHP mocked my directoryless file structure and chastened me for my blatent PDO-less insecurities. I wasn't trying to win any code fashion contests, that's for sure. I haven't even tried learning Haskell.

So at this point we have the mutant offspring of a chat and a content management system...

So what do you do to the mutant offspring of a chat and a content management system to make it a general simulator. We need to add MUD-like elements. Younglings might not remember MUDS, but the oldbies certainly do. MUD stands for "Multi-User Domain" and were the precursor to all massively multiplayer online games. It was a world represented in text only, because telnet didn't do images.

So at this point I'm wondering whether or not I should find MUD code and try to port it to the web, or if it would be easier to write it all from scratch. I wanted to know the code really well, and worried that if I used existing code I would become less able to implement new changes. So I chose to write it all myself, frameworks and years of MUD-evolution be damned.

To handle the influx of world-simulating display code, I decided to create the concept of an 'aspect'- which in general just means some part of the site that can be displayed (or hidden) on demand. Soon the chat/cms had a functioning command line. Commands could be entered directly into the chat to toggle the display of each aspect on a per-aspect basis. So the chat became an aspect among equals, and could be hidden or shown with the /chat command. The command line itself became the only non-aspect part of the site.

It was around this time I noticed how useful it would be if the code for the aspects was held within the CMS itself. This would mean that I could edit the code easily and also gain the benefits of CMS such as versioning history, user access, and hierarchy organization. In addition, because the chat was monitoring the cms, it would inform the community of modifications to the aspect documents.

So I made a root level in the cms called 'Code' and another level under that called 'Aspect Code' and the first document in that level was called 'Chat' and contained the PHP/HTML code for displaying chat messages. In the database I made a table called 'aspects' and in that had a reference to to the document for the code (the cms gives every document a unique reference integer). The aspect table also has a number of text fields to indicate the command word to display the aspect, a brief description of the intent of the aspect (For the help section), and other technical details pertaining to making aspects 'session sticky' which means having the state of each users aspects maintained across sessions.. ie: if you log out with only the chat visible, that is what you see when you log in next.

When a user has an aspect visible, they are viewing the code that is stored in the CMS document (which is referenced in the aspects table) after being evaluated. If this makes you go cross-eyed with "security nightmare" scenarios- please for the love of crap help me secure this nasty beast. If you go cross-eyed for other reasons, perhaps I can explain that better.

The site, at this point, is a command line and a generated list of evaluated blocks of web code which are stored in a db, organized in a cms, and, hidden or shown on command. The list of aspects itself (ie: what commands will work, what blocks can be shown) becomes an editable part of the cms, so that there is an aspect to create a new aspect! Once the information pertaining to the aspect has been saved in the database, all site users can access it with the command (provided they have access).

So how could I make the smallest, easiest step from the mutant offspring chat/cms hybrid to the chat/cms/mud superhybrid. This would, at a minimum, require the concept of an avatar that has a position and an inventory. But what is an inventory without items? What items can be made? And what is a position without locations?

To avoid any trademark disputes I used the word 'character' to mean 'avatar' in my database design. I felt that the users would appreciate being individualized so I incorporated a 'character_type' and added a few basic types (like 'human' and 'dog'). When I say I added these I mean I added a table the database to store information relating to these concepts as they are to appear in the virtual world. The characters table contains information about a character's type and position, and there is a one-to-one mapping between users (in the users table) and characters (in the character_data table) and the character type table contained things like description, or max inventory carryable.

To make these concepts fully wiki-editable (to give users the ability to choose their character type, and create new character types, and to modify the position of the character.. etc) I would have to make aspects that contained forms that linked to scripts which then modified the database. So I created an aspect to display a form accepting information about a new character type, and then a script to handle the database modification which took its data from that form. Also I modified the cms' new user form to allow for a selection from the list of available character types. I added an aspect called 'move' which allowed users to change the position of their character (using the least realistic instant-hop-to-anywhere technique as an initial simplification).

At this point I had to decide how many dimensions I wanted in my virtual world. Do characters exist along a single line (position represented by one number only) .. or do they exist on a 2D plane (two numbers), or in a 3D space? I settled on 4D for no particular reason. I called the fourth number 'domain' in a similar manner to MUDS.

The habit of treating code as just another document within the CMS began to take shape, and I soon added a document for the Javascript under 'Code', and I also made a level under Code for potential style sheets. I added an aspect called /setstyle which displayed these in a form to the end user, and made a column in the user table which referenced the selected style document. That way each person could create and select the style sheet which they see for the entire site!

If you are unfamiliar with the concept of a style sheet, I will explain. It is the code that tells the web browser how a website should look- completely independent from what content is contained in the page. It is where one defines colors, borders, fonts, background images.

There were other things that just seemed both easily do-able and functionally useful. The /me command, for example, would allow a user to send a chat message in the style of "username does something." I also added a /go # command to jump directly to a document if you knew the number. I tried to make every old CMS function (like edit document) accessible from a command instead of a link in the page. This provided greater encapsulation and segmentation, and made the whole thing more intuitive (provided you found command lines intuitive).

So the CMS was chopped up into aspects- /nav displays the list of root levels, /ls displays the list of sublevels, /edit displays the edit document aspect, /delete displays the delete document confirmation, /pwd shows your current location in the document hierarchy and so on.

We needed a command to show which users were online, so I wrote the /who aspect, which displays a list of users who have been active within the last five minutes. I added the /zero command to hide all aspects, which is quite useful when trying to calmly navigate an ever-expanding series of aspects.

I added features of link aggregation, so that users could share links with each other, with each link being sent to the chat as well as to a place for reference in the database. I also added an RSS aggregator which was connected to a database of potential rss feeds, so once a user 'finds' a feed and shares it with the site, other users can see that it is there and perhaps add it to the list they read. (RSS is a way for publications to syndicate content by providing a list of latest links, RSS aggregation is 'reading the news' on steroids).

In addition to the wholesale creation of a style sheet, I wanted a more granular method for users to customize their experience, so I implemented the /set command which allowed users over-write specific CSS rules on a per-rule basis. So a user could have a red background even if the style sheet they are using has a black one by default, by typing only a single command into the chat.

Soon came the /help aspect, which automatically generates a list of aspects and their commands, and a description of the aspect. In addition to the automatically generated list, manually entered information on non-aspect commands is contained in the document for the help aspect.

And then it became apparent that some grouping of aspects was needed- a way to turn on a bunch at once or jump between sets of visible aspects. For this I created 'site views' which are selected from a menu under the command line. When selected, a site view specifies the complete set of aspects visible to the user.

So now we have a web-based semi-windowing command-line content-managed chat-console style-customizable hyper-textual web-based multi-user domain! La-de-fricking-dah. We need to go deeper!

It was easy enough to make a representation of a character, that had a certain type, at a position specified by four numbers. But it doesn't make for a very interesting world if positions are not accompanied by some landscape-esque description. To make this happen I added a table called 'locations' whose records uniquely identified positions, extending the information stored about them.

For example, even though a character may be at the position (0,0,0,0) there may not be any location there, and therefore the position has no unique name, nor unique description. If we add a record to the locations table for that position, we can specify then a name and a description.

Also, what fun is an avatar that can't carry virtual items? Or a world that has no virtual items? We will need ways for users to create and manipulate these items! And what fun are items that can't do anything?

So we need a table for all the types of items that can be instantiated (made to exist in the simulation). For this we have the 'item_types' table.

And we need a table that lists all the currently-instantiated item types. For this we have the 'item_instances' table.

And we need these tables to be populated via user-submitted forms. So we have an aspect to accept descriptions of new items and then create a record in the item_types table. We also need a way to display the position and location information to a user about their current position.

And so the /newitem aspect was made to let users virtually-invent item types, and the /create aspect to make them from thin air, and the /inventory aspect to list currently-held item instances, and the /map aspect to show position coordinates, location descriptions, items-at-locations, /give allows users to share virtual items.

I went a little wild at this point and wrote up some market code. So I created a 'markets' table and specified a name for the market, an owner, and a token item type which enabled access to the market. Also at around this point I added a 'site currency' field in the user's table to hold a virtual account balance on a per user basis.

Certain objects also permit characters to perform certain actions. To do this I would require a table of 'actions' which held a name for the action and a reference to the document which contained the code to be evaluated at the invocation of the action. For this we need some mapping between the item types and the allowable actions. Again I added a level under 'Code' and called it 'Actions' and a table called 'item_actions' to link item types to actions.

There are some actions that can be performed on all items, such as 'drop at this position' or 'destroy', and these are accessible under the /do command. If the component relation is specified for an item type, it can be recycled into components, or constructed from components. The /map aspect also lists item instances that have been dropped at positions and provides the interface to pick items up from a position (clicking on the object name).

It was around this point that I probably didn't think generally enough- because instead of generically representing any relations between the item types, I focused on one only- the component relation. There are many more relations between items (such as 'is a tool used to construct') and these should be easily expandable by the users, by that I mean if somebody can think of a way that items can be related to each other, then they should be able to add that to the database.

Okay, let's stop talking about the actual technical creation of the virtual world. Perhaps there are millions of bit-monkeys out there who could type at millions of computers and produce a perfect result, a thing of sublime creation, a cathedral on the web, a glorious library of world-emulating simulation. Let's assume we've got that already. What then?

But first what is a perfect simulation? It would be both visually realistic and semantically realistic. The number of things represented in the database would grow with each step towards a greater realism. It would have to contain representations of nature, the artificial world..

The perfect simulator would teach any aspect of reality, from farming, to welding, to barn-raising, butter-churning. It would know of three-toed sloths, of weevils, beetles, and fireflies. It would know in what climate these were found and place them only in that climate. It would know, and could therefore teach by display, the life-cycles of dragonflies and even the mating rituals of humans.

But we don't have the perfect simulation, at the moment we have a simulation that only 'does' some of reality, but from that small amount of stuff we can construct educational and entertaining online-community interactions. The item and component relation can be used to teach the composition of objects, with players memorizing a growing list of item-component graphs and then demonstrating that knowledge on the simulated world by synthesizing the virtual objects from their virtual components and then listing them on markets (or leaving them freely about the land). We could reward virtual currency per synthesis as an incentive!

So, to summarize, the task at the moment is to define a web site with a world simulation that has built-in mechanisms for growth. These mechanisms should be accessible to all and should result in modifications to both front and back end, the interface and the database. We obviously can't expect everyone to be programmers, so the interface to the mechanisms must be intuitive enough that it doesn't alienate a majority of potential contributors.

But this is, admittedly, a huge project, but it is a simple idea. From a user's perspective the list of things to learn about the site would be growing with every addition of a 'real thing' that it simulates. So I think the the list of 'things' should be explicitly defined in a manner independent of aspects, such that future things and the group decision of exactly how to implement them is displayed for all to see.

So I will construct a table for 'things' which can act as both changelog and sounding board. The table should have the following columns: id, name, type of thing, suggestion date, suggestion user, implementation date, implementation user, required database modifications, required interface modifications. Now I hope at this point you aren't confused between virtual items and these new things- by thing I mean the smallest reasonable unit representing some modification or improvement to the site. Somebody could log in and create a future thing called 'birds' and then the developers would fill in the 'required changes' parts in the table. They wouldn't have to program them right away, but the description of how to program them would be the starting point for another developer coming along afterward.

This will also require a table to hold the list of potential types of things. Records in that table will read something like 'addition to realism', 'interface simplification', 'information display', 'database optimization', 'graphical component', 'audio component', 'gameplay element', ...

Here is the concept of things presented as a thing!