/55\Notes from BEK nato.0+55 workshop

Kurt Ralske 55 at plot.bek.no
Fri, 11 Aug 2000 14:57:46 +0200


BEK (Bergen Senter For Elektronish Kunst) in Bergen, Norway hosted a
workshop in live video art utilizing nato.0+55 on July 31 - August 5. At
the workshop's end, the participants did a networked group performance on
Aug 5 at Kvarteret in Bergen.

The participants were HC Gilje, Kurt Ralske, Gisle Fr°ysland, Trond
Lossius, Reinert Mithassel and Thomas Sivertsen.

Two of the participants are nato.0+55 veterans (HC Gilje and Kurt Ralske)
and one is a very accomplished Max/MSP programmer (Trond Lossius). The
others brought their diverse computer graphics knowledge but less experience
with Max or nato.

A typical day included a few hours of demonstration and tutorial by HC
Gilje (leader of the workshop), using small patches demonstrating different
capabilities of nato. The group would then switch between individual and
collaborative work while testing techniques, objects, and patches; working
on patches and source material for the perfomance; and defining the
parameters of the performance.

Some of the patches and utilities used in the workshop are available at:
www.bek.no/55/files.html

....

The participants + their contributions:

HC Gilje has been using nato in collaboration with dance and theatre
companies throughout Europe. His broad and deep knowledge of nato helped
open many doors for the other workshop participants. HC works a lot with
live video input into nato; his camera was rarely off. He has made patches
that intermittently record a few seconds of live input, then display a
processed version, before sampling new input. A new patch throws a few
seconds of camera input into a 242.buffer controlled by the "beat" output
of Sound Tracker (see below). This had the delightful effect of making
anyone who passed in front of the camera appear to be "grooving" to
the beat, hard.
For the performance, HC created a very imaginative arrangement of multiple-
screens-in-one, displaying variations of the same source. The main screen
was a processed version of live input: the screen was dark, except for
anything in motion, which was displayed in coloured light. Faster motion
served to "illuminate" stationary objects.
HC also showed a video of his use of nato in a theatre piece: images were
projected on the floor in a 7x10 matrix grid. Images could be resized to
grid multiples and moved around the grid, sometimes tracking the location of
the actors.

Trond Lossius and his deep Max/MSP knowledge contributed greatly to the
workshop. He devoted a lot of attention to the 242.lut object (look-up
table), using it to effect various types of color cycling. Thinking of the
color table as analogous to the audio frequency spectrum, he wrote various
distribution curves (Gaussian, tanh, etc.) and used them to shift red,
green, and blue levels at different frequencies and amplitudes. Trond also
wrote the Sound Tracker patch, which was the core of the group's
networked performance. It is a MSP patch that takes an audio input and
provides real-time information about the music's level, transients (beat)
and frequency distribution. This info is sent out over a TCP/IP connection
to the network, and connected machines running nato can use the info
to have images be inflected by the audio. The beauty of the set-up is that
each nato operator could individually choose which elements of the audio
data would affect his images, in what way, and to what degree.

(Trond also made a pursuasive case for the use of Peter Elsea's list
objects, and made several people eager to use them as powerfully as he
does.)

Kurt Ralske has been using a very large nato patch for live interactive
video performances in New York City. His aim is a transparent user interface
that allows powerful and flexible processing to be controlled in an
improvisatory way. In April, Kurt wrote the "copymode_explorer" tutorial
patch, which seems to be a valuble tool for nato users. (Another essential
tool would be the Bernstein/DeKam "modular.ref".) HC Gilje distributed his
update to "copymode_explorer", which is now a laboratory for exploring
three-layer collage, instead of the original two. Kurt plans to return the
compliment by re-revising the revision!
Kurt demonstrated his performance patch, which uses a Wacom tablet as an
expressive interface for collaging and non-linear playback. He advocates
the 242.buffer object as the real playground of the "power nato user",
because it allows a speed and a ultra non-linearity not available off disk.
He distributed his "narrative_destroyer" stochastic editing patch, and
utilities for automatic 2nd monitor output and for keyboard selection of
source films. He also assisted HC with tutorial duties a bit.

Gisle Fr°yland is the developer of GifJam software for real-time video
performance on the Windows platform. He chose to meet nato on its own terms
and create a patch that had nothing in common with his own software.
He exploited Max's input capabilities by using an I-Cube with sensors for
motion, proximity, and pressure. Data from these were used to control the
blending of layers in collage effects, and other effects. Additionally small
light sensors were placed on the monitor surface, creating a feedback loop
of data. A 242.buffer waswas constantly refreshed with output, and re-routed
back into the collage input, creating a second feedback mechanism within the
system.

Reinart Mithassel, an installation artist, chose an interesting approach to
exploring nato: he worked with the simplest of still images, so as to be
sure what affect nato was having, without being dazzled or confused by good
or bad source material. For a while he used one circle and one line, and
was able to find techniques for collaging and effects that achieved his
goals, even with such minimal sources.

Thomas Sivertsen is a3d animator whose main tool is Maya. He explored the
nato 3d capabilities, and wrote a very nice patch demonstrating some of
them, which was distributed. Kurt suggested it would be possible to use the
3d patch with sensor input to allow a "virtual camera", but there wasn't
time to explore this.

Maja Kuzmanovic and Phillip Blank, two other nato operators (who
happened to be in Bergen for a different multimedia conference), briefly
dropped by BEK to say hello and see what was going on.

....

Several themes emerged:

Processing power -- It's no secret: working with real-time video processing
makes one's computer work very hard indeed. Reinert was intially concerned
that a slow frame rate might result in a large nato patch, with lots of
effects. (This was not an issue when he ran his patch on a G4.) He also
realized that working with compressed source material (instead of raw DV) is
advisable for most situations.

Trond has written Max externals in C, and so was able to provide insight
into what might be going on "under the hood" in nato. He made a "frequency"
object that gives an accurate reading of current frame rate, which allowed
the group to test out exactly how expensive various nato processes are (in
terms of CPU cycles).

Trond proposed a theory about the inner workings of nato, and seems to have
proved it: most nato objects draw no CPU cycles until the are passed an
image; they do not idly draw power. The exception are the image producer
objects: 242.film, 242.qd, 242.buffer, etc. If an image producer is at the
front of an effect chain, essentially it's actually the image producer (and
its frame rate) that determines how much load is placed on all the objects
"downstream".

Trond also pointed out that nato is more Max-like than others had realized:
for example, image output can be used to trigger a bang, and all
initializing arguments to nato objects can be typed directly on the object
(just like all other Max objects).

Networking -- The group tested out the new 242.wto and 242.fireuire objects.
(These are not part of the standard nato distribution.) They work as
advertised. It is indeed shocking to see one's images come up on another's
screen. Using the 242.wto objects for streaming via OT/UDP protocol, an
image stream was sent from one machine to a second, and back: the returned
image appeared to lag ~100 ms behind the sent one. One concern over these
objects is that they seem to be processor-intensive; on a 333mHz machine,
there seemed to be not a lot of CPU left over to do other amusing things.
Thus they're most useful if two machines can be dedicated to transmitting
and receiving. They might work less well to "daisy-chain" nato machines
together for accumulative nato super-processing. HC suggested that this
goal (necessary for only the "industrial-strength super-power nato users")
would be better achieved by connecting the s-video output of one machine to
the analog capture card of the next, etc.

....

And then the performance! After a mildly agonizing transport of two
carloads of g4 towers, monitors, projectors etc, a local network was set up
for the eight computers. All participant's video output was routed to a 4x4
video switcher, then to four projectors around the room. The group took
turns displaying their images, and each image was occasionally moved to a
new projector, or displayed on multiple projectors. A dazzling and immersive
environment was formed. At times the individual artists' images congealed
into a powerful synergystic whole. Audio was courtesy the rather radical
dj-ing of John Hegre.

HC's multiple 242.ekran arrangement looked lovely. Kurt was finally able to
try out the new improvements to the patch he has worked on for four months,
and discovered they did even more than he anticipated. Gisle used his
multiple sensors to control blending of image layers, to excellent effect:
complex and dynamic. Reinert's bold images were very strong, his large eye
was pleasing and disturbing. Thomas had a last-minute inspiration and
assembled a new performance patch from scratch in the ten minutes before
the show! He used it to collage 3d sources he had previously made in nato.
(Surely this disproves some rumors about nato's steepness.) And Trond had
his cycling colors tightly synched to the audio, sometimes fast, sometimes
slow, always otherworldly and beautiful. However, his late-night
manipulation of "the singing penis video" was so completely out of
character that one simply has to accept his explanation that he loaded that
source material by accident.

In all, an enjoyable and productive week! Those new to nato were impressed,
and expressed a desire to continue working with it.

The group's final verdict on nato.0+55...Steep? A bit. Deep? Yes -- and
lovely too. Worthwhile and exciting? Ja danke!++