Throwing Beam for Fun and Profit
http://panelpicker.sxsw.com/ideas/view/7548

Motion Paintings, Visual Storytelling and Deaf Film
http://panelpicker.sxsw.com/ideas/view/7886

Beyond 3D: How Interactive Screenings Will Save Cinema
http://panelpicker.sxsw.com/ideas/view/7020

DIY Motion Tracking for Public Spaces
http://panelpicker.sxsw.com/ideas/view/7003

People as Peripherals: The Future of Gesture Interface
http://panelpicker.sxsw.com/ideas/view/5716

Developing Futuristic Multi-Touch / Multi-Screen Applications
http://panelpicker.sxsw.com/ideas/view/6154

Interactive Narratives: Creating the Future of Literature
http://panelpicker.sxsw.com/ideas/view/6462

3D Interfaces: A Primer on Stereoscopic Interaction Design
http://panelpicker.sxsw.com/ideas/view/5934

Data Visualization: Sci-Fi Interfaces in the Real World
http://panelpicker.sxsw.com/ideas/view/7358

Toss the Projector: Redefining the Presenter/Audience Dynamic
http://panelpicker.sxsw.com/ideas/view/6389

Hybridizing Interaction with Motion Design: User Interfaces
http://panelpicker.sxsw.com/ideas/view/5568

The Future of Touch User Interface Design
http://panelpicker.sxsw.com/ideas/view/5984

Fun With The Lights Off: Interactivity Without Graphics
http://panelpicker.sxsw.com/ideas/view/7208

Getting Touchy: Going Beyond the Keyboard and Mouse
http://panelpicker.sxsw.com/ideas/view/6662

Creating Interactive Art Through Social Media Collaboration
http://panelpicker.sxsw.com/ideas/view/7239

The Music of Interaction Design
http://panelpicker.sxsw.com/ideas/view/6643

Realtime Interactivity for Music, Broadcast and Art
http://panelpicker.sxsw.com/ideas/view/5400

Realtime Mixing In Video Games
http://panelpicker.sxsw.com/ideas/view/6648

The application itself is an AIR app, so it’s all written in Actionscript 3, with the graphics designed in Flash. So there are actually three versions – one for the Installation, which runs as a full-screen AIR app on my netbook, a web version which will be up soon (which drops the saving capabilities), and my favorite – an Android app which runs on my HTC Incredible.

If you’re brave, you can download the file to run on your own phone – you’ll need an Android phone running FroYo (Android OS 2.2). You’ll also need to install the pre-release Air for Android runtime. This stuff will all be more accessible once Air for Android goes official later this year. If that’s not enough to scare you off, then Download Bridge Invaders for Android!

The Future

I wanted to get the Android app out there to any early adopters and interested parties, while I get the desktop and web versions ready. There is, technically, a desktop version working – the one I used for the installation – but it requires a touchscreen for use. I developed it on my Lenovo S10-3t, which is a nifty little convertible touchscreen netbook – it’s a great Android dev machine, as it will do two-finger-touch, and run AIR apps in either portrait or landscape mode.

Beyond the installation, the underlying engine (which I call PixelAnimator, original, eh?) is designed for musical performance. The idea is that I can make these minimal animations wherever I am, and then save each one out, and load them into a performance application for playback alongside a musician, projected onto a proper backdrop. Thus PixelAnimator becomes a visual instrument, allowing me to work on my Performance Visuals anywhere I like. Additionally, the HTC Incredible allows me to send composite video out of the USB port, meaning I could perform directly from the phone itself!

Major thanks to Adobe for putting out the prerelease, WKE for sponsoring some of the development, FDT for their awesome dev tools, Robotlegs for keeping the code manageable, the Hype Framework for the shortcuts, and to Jared Tarbell and Invader for the inspiration.

The Problem

As is often the case with banners, we’ve got a lot of text that may change multiple times before we ship. Every foreign studio will need to switch out the text within each banner as well. I’ve been doing lots of localization work using AS3 loading language for XML files, but we don’t have that luxury in the world of banners, where any extra kilobyte of load code may take us over our filesize limits.

The Solution

Enter JSFL. For those not yet in the know, it’s scripting for the Flash IDE – Javascript files which can control many aspects of the interface, like manipulating the library, timeline, tweens, renaming, organizing and more. I had a hunch that I could build something that would perform the text-switching functions of XML Language loading, but at authoring time instead of runtime, simplifying the resulting XML file.

How it works

You create a text file which lists object names and the text they should have (the download includes an example file). It’s XML-like, but simplified to be easy to read an edit. You run the CopySwitch script (by double-clicking on it) and select the language file you just created when prompted. Flash will switch out all the text you described in any open Flash file. So you can change the copy in 20 banners in a matter of seconds. Works with static or dynamic text.

Download CopySwitch v1.0

I’m getting ready to head to Seattle tomorrow for the Decibel Festival. I’ll be performing graphics live for Dilo on Friday night and Daedalus on Saturday. I’ve spent much of the last two days prepping content and working on (yet another) new performance system/style, and decided to take a break from staring at the screen today to build prototype three of my VJ Box.

This version centers around the APC40, which can face the artist or the audience with a simple flip of the box. Inside the box will be my external hard drive(s), USB Hub, Power Strips and lots of cables. The idea is to plug in as much as possible beforehand, so you have only a few last connections to make once you’ve arrived at the venue.

Inspirational props go to Middleman for his amazing Plexiglass Table, Pleasuretek for making me want a mobile rig, Daedalus for tilting his monome towards the audience for all to see, and VJ Exavior for showing off his Velcro-Filled VJ Coffin all those years ago.

I wanted the ability to pan sounds for the game I’m working on. I wrote a quick note to Joe Berkovitz, who developed the Audio library I’m using. He suggested I write the filter myself

I’ve never extended a library before, but he included great example filters and assured me it wouldn’t be hard, so I gave it a try. What do you know – it wasn’t bad at all. So I’m happy to give my teensy contribution back – you can download my PanFilter below and drop it in your StandingWave2 Filters directory.

It works just like the Gain filter, taking in one Number between -1 (hard left pan) and 1 (hard right pan). Zero will give you centered audio, and you can use any float in between -1 and 1 for some subtle effects.

See interactive movie at top of page for a hands-on example (sounds pan from left to right as you move left to right across keyboard).

Download PanFilter.as

I’d originally browsed iStockPhoto for squid illustrations, but wound up liking my version better, anyway. I’m no illustrator, but what can I say – I like my designs.

I’ve currently got about half of the game mechanics working, so I should be able to post up a demo within a week or so.

I’ve been working on a Simon-like game in order to stretch my AS3 knowledge, and I’ve been running up against some of Flash’s known issues regarding playing sounds at exact times. I stumbled upon StandingWave2 while searching for developments in the sound-in-Flash arena, and after just a few hours of messing with it, I am HOOKED. I’m not yet sure that it’s going to solve the particular issue I was having with the Simon game, but it’s spurred a few tangential ideas that I may try out first.

In any case, the movie at the top of this post is my first go with the StandingWave2 library, made in the course of a few hours on Sunday. I’m looking forward to more experiments.

The Master Control System is a 2-Player Interactive Video Station designed for temporary installation near loud music.

Each player has an array of buttons and sliders, along with a scratch wheel. Every press, slide, scroll and twitch changes the video, which can be simultaneously displayed on the screen and projected as large as you like. The interface has some items which are quite intuitive, and others which require more experimentation and produce nuance through extended play and exploration.

Inside the Box

It’s a relatively simple setup.

• A 15″ MacBook Pro runs a customized Quartz Composer scene.
• A MixMan DM2 plugs into the laptop via USB.
• Two light sources illuminate the box from the inside.
• One power strip provides power to all devices, so the box requires only one three-prong grounded outlet.
• A Keyboard and Trackball provide easy access to setup via the hinged door.

My current conclusion: Oscar for OSC, Red5 for MIDI. Read on for the details.

Oscar

In Short: OSC to Flash, one-way, Mac-only.
Details: The slickest of the controller translation apps, this one is great for connecting up the Monome and any other controllers that support OSC natively. Additionally, if none of the MIDI solutions are working for your setup, you can convert your MIDI to OSC using Quartz Composer and then route it through Oscar and into Flash. Perhaps a forthcoming post on this?

Red5 Flash Server

In Short: Open source Flash Server, includes MIDI functions.
Details: This Java server supports streaming audio, video, and lots of extra stuff. It seems like it’s perhaps overkill if you just want to get MIDI into flash, but hey, it works! The newest stable release (0.8) is available for OSX, Windows, or as source to build on your own box. Unfortunately, the Mac 0.8 installable version needs Java 1.6, which was never released for 32-bit Mac systems. Thankfully, you can download the 0.6 release pre-compiled, and it works just fine. Last updated June 2009.
Starter Code: Lukasz Karluk put together a great starting point: AS3 Code for Red5

MidiToFlash

In Short: A Java applet and bridge that enables MIDI input into Flash, including through a webserver.
Details: I really wish I could have gotten this to work. He’s got an online example that worked for me, but I couldn’t get my local version to connect properly. After messing about with it, I can no longer get his example to see my MIDI messages either (though my devices still show up properly). To be fair, I updated my OS in between the working and not-working tries, so it could be that, or something else I messed up. Let me know if you’ve had success with this project – there’s lots of possibility for fun multi-player MIDI games here.

FlashMIDI

In Short: MIDI input and output from Flash, last updated 2005.
Details: Assumed abandoned, this was a plug-in based on some sort of undocumented feature. It allowed MIDI control, as well as playback of MIDI files. It was mostly Windows-Only, though there was an Alpha version of a Mac plug-in made.

FlashServer

In Short: Max/MSP external allows bi-directional communication- last updated 2006
Details: Assumed abandoned, this was an external for Max/MSP (so you need to write a Max/MSP program that would use it). Mac and PC versions still available for download, current compatibility unknown (written for Flash 5).

flOSC

In Short: A Java server that creates a bi-directional channel for communication between OSC devices and Flash.
Details: I did get it working, but Oscar is much slicker. If you’re on a PC, this is a great (perhaps the only) solution for OSC to Flash communication. There’s a new flOSC by tranxete, but like Red5 0.8, it requires Java 1.6, so it’s a non-starter for me.
Starter Code: From tim, based on the fwiidom classes: flOSC As3 Classes.

Tim, of the flOSC classes above, also has a good round-up on using MIDI in Flash.

Split-Screen

Full-Screen

The basic data flow is this – Sliced Audio Tracks play back samples, and Dummy Tracks listen to their MIDI and pipe it out through a virtual MIDI port. An external video program (Isadora, in this case) reads the incoming MIDI and plays back video in sync.

Ultimately, I think people are going to develop some awesome Live AV plugins via Max4Live. At that points, I can probably scrap this system and use one of theirs. In the meantime, I’m figuring out what it is I want in an AV system so I can pass that along to the real developers.