pl0t engine - introduction

supershort version: The pl0t system makes it possible to play realtime generated music (midi with custom soundbank) in a webpage, the generated music is a result of the users input - like mouse movement and keyboard. It is a multi-user system, in other words: people interact with each other. Ergo - it's WAY COOOL! ;)
simple figure of the pl0t system


The whole idea was to make an interactive system for the web that would use the users input, such as pressing the keyboard and moving the mouse, and from this input generate music (and graphics, but that is not extensivly discussed on these pages) that the user would experience.

Also, it should be possible for more then one user to interact with each other, in other words the input from one user would also affect the music and graphics another user experienced.

applets and javascript

There are a lot of problems trying to make such a system, one is the required machine and bandwith a user has to have in order to get something wortwhile out of the pages. We decided to go for streaming midi instead of other technology like RealAudio or ShoutCast because the bandwitdh requirements for streaming midi are conciderably lower! Now, we all know that GM stinks, so - we had to find a system that had a softwaresynth that was ok, and hey presto JAVA ver1.3 includes the Beatnik/Headspace engine (that also is included in the Beatnik plugin). Therefore, applets (java web-applications) seemed a good solution since they had the possibility to easily include a softwaresynth.

Now, to capture the events, the applets could ofcourse be used, but then we were restricted to the canvas of the applets, and we were not sure if the graphics would be done in the applets at all, so therefore using LiveConnect (Netscape java<->javascript communication) seemed like a good solution.

server & communication

The events captured in javascript (sent through the applets) had to be sent somewhere so that all the local users input could be processed and send back as graphics and streaming midi. Therefore a server was needed. The servers job is to receive UDP packets from the applets, and to stream small midi-files back to the applets through a TCP socket connection. Also, the events are gathered and sent further to MAX (a midi-generator/composer/thingy) which in turn sent back small midi-files to the server. All the communication with MAX is done using UDP.


As the music streamed back to the applets should be a result or at least influenced by the users input it could ofcourse not be precomposed. MAX is a midi-program that lets you 'program' certain responses (like notes being played) based on the input that it receives. Therefore it seemed like a good tool for composers to use making patches (that is: interactive compositions) used together with the rest of the pl0t system.