My house/office has a store window and I try to make use of it as much as possible. I had some projections etc. in there before, but now I wanted to make something interactive. We have a lot of loitering youth and many people passing by. To appeal to a wide audience, I thought it would be nice to make an old fashion arcade box like thing.

It’s now on the street for one day and people really seem to like it. Age or background doesn’t make a difference.

To make it a little bit more challenging,  I thought it would be fun to try to run the hole thing on an Arduino without an external computer. Just to see how much power it actually has. And while I was at it, I decided to try to make everything myself. So I also made my own led screen, with his own “display driver” and display list, a 8-bit sound library, and of course the game itself.

The only thing I didn’t make was the background music for the game, that was composed by the lovely Annegreet Sledsens: Thanks sweety!  You can hear it in the video.

The led screen is very bright which made it very hard to film and photograph with my cheap camera. Even after some photoshopping, I couldn’t  make it look as good as in real life. So if you’re in the neighbourhood (Antwerp Belgium, Provinciestraat 60, behind the Zoo), feel free to come check it out and play a game with the locals!

Here the video and some more pics:

The Game

I needed something that would appeal to a large audience (the people in my street) and something was fun to play .
So I took the good old gaming cliché, where the world gets invaded by aliens and you have to fight your way to the end boss, save the world and the human race . With my minimal resolution of 16*90 pixels, I didn’t have much other choice than making it pixel-art style.

The game has 3 modes, a single-player and a multiplayer brawling/fighting the aliens mode, and an extra fighting mode where the 2 players can battle with each other.

Every mode has just one level, but since it is just a casual “play on your way to work” kind of game, I thought it would be more then ok.

some screen-shots:

The game-mode selection:

Two players fighting the the final boss:

The girl kicking the boys ass in fighting mode:

Game over!

Development

To make the development easy and fast, I cross developed the game as a Cinder c++ app  and an Arduino app, that way i didn’t had to upload the whole thing on an Arduino every time I wanted to test something.

I didn’t use any external memory, so I had to store everything on the Arduino.  I made a small app that generates c++ classes from bitmaps with some gamma correction for the led screen. I used indexed colours to save some of that precious SRAM. For driving the leds, I modified the Adafruit neopixel lib, to support alpha-blending and make it a little bit faster for my specific case.

I don’t really have much experience with generating sound and music. So the 8-bit sound was quite challenging. I’m happy with the result, but my oscilloscope broke during development of a more advanced sound lib, so its quite simple for now. But I really enjoyed  myself working on the sound aspect. I have a feeling that my next project is going to be something sound/music related

I’m not going to elaborate too much on the rest of the code. But you’re free to check the source code yourself  (warning: its messy).

The full source code and game assets on Github (Code-> MIT, Assets-> CC attribution)

The Hardware

Like I said, I made my own led display.

I used strips of  WS2812 LEDs, those are individually-addressable RGB LEDs. The lay in a 16*90 grid (=1440 leds)

I added a laser cut raster over the leds to make the pixels square and put a layer of plexi to diffuse the light some more.

Beneath the leds is the main Arduino Due and the power source. This is the Arduino that drives the display and runs the game. Those leds suck a lot of power , so I had to add a 60A 5V power supply.

The screen/main arduino is connected with the arcade box through a simple tx-rx serial line, which was fast enough to send the button commands.

the arcade box is just some painted MDF with a steel frame inside, and a steel plate on-top (it has to be a little bit solid if i leave it outside )
The Arduino in the arcade box generates the sound and music and handles the raw button/joystick input.

If your interested,  the full “schematic” of the project:

Its pretty straight forward.: Pulldowns for the buttons and joystick, the led connections and a pot for the screen brightness and a tx-rx line between the two Arduinos.

on top of that there is a DC coupling for the speaker which is connected to the Arduino DAC . But I still have to put some kind of amplifier between the arduino and the speakers. The sound is hard to hear because of the street noise.

I was always interested to remove the perfection from computer graphics. So thought it would be fun to try to make the most obvious thing first. A machine that uses a paint brush to print a drawing.

It runs on an Arduino DUE  and an Android tablet.

I had a Samsung galaxy note 10.1 laying around from a previous project, so i used that one, The nice thing about that tablet is that it comes with a pen with pressure sensitive input. So I could translate the pen pressure to the brush height/size on my machine for more interesting brush strokes.

A poor video of the machine in action:

Its far from perfect, I made a lot of mistakes . But since almost everything I’ve done for this project was a first time, I’m still happy with the result

Physical Setup

I only have some basic tools. A drill, a jigsaw and a grinder machine and a couple of wrenches. So I improvised a constructions with bolds and nuts that I could adjust as needed. But I learned that precision really is a thing in mechanics so next time I think I’ll have to extend my tool-set, (and make a plan before I start )

For the arduino/electronics part the basic is just some stepper motor drivers (big easy driver) , stepper motors and a servo motor .
Because it was running from an Android tablet, a had to add some displays for debugging, anyway this is the setup I use now:

The same thing in real life:

Some Code stuff

Arduino

the Arduino is really a fun tool and actually quite easy. You write classes for it in basic c++ (no std etc.).
But almost all existing libraries for Arduino use delay() (a sleep that pauses the main and only thread). I don’t get why they work that way, sure It’s easy for quick testing, but it makes it impossible to do multiple things at once (like adjusting the brush height while moving the brush in my case)

So I had to write everything myself, the stepper driver, servo driver  and display drivers. so they worked on timesteps instead of sleeps.

Something like this in the main loop:

unsigned long timeCurrent=micros();
unsigned long timeElapsed=timeCurrent-timePreviuos;
timePreviuos =timeCurrent;

servo.updateTime(timeElapsed);
stepper.updateTime(timeElapsed);
pauze.updateTime(timeElapsed);
display.updateTime(timeElapsed);
ledDisplay.updateTime(timeElapsed);

And for example in the servo class:

void Servo::updateTime(unsigned long timeStep)
{
  currentTime+=timeStep;

  if(isLow)
  {
    if(currentTime>=waitLow)
    {
      currentTime=0;
      isLow =false;
      digitalWrite(_pin, HIGH); 

    }
  }
  else
  {
    if(currentTime>=  waitHigh)
    {
      currentTime=0;
      isLow =true;
      digitalWrite(_pin, LOW);
    }
  }

}

The Android-Arduino communication

The android connection to the arduino whas quite annoing. I used the naive android USB host, but unlike a regular computer, you don’t have a driver for the arduino.
So you have to handle the USB connection yourself (send controle bytes etc) and since it took me so long to get it working: a small example (not sure if this is the best way to do it…)

To make the connection:

View.OnClickListener tryConnect = new View.OnClickListener()
{
	public void onClick(View v)
	{
		manager = (UsbManager) getSystemService(Context.USB_SERVICE);
		HashMap deviceList = manager.getDeviceList();
		Iterator deviceIterator = deviceList.values().iterator();
		while (deviceIterator.hasNext())
		{
			UsbDevice device = deviceIterator.next();
			manager.requestPermission(device, mPermissionIntent);
		}
	}
};
private final BroadcastReceiver mUsbReceiver = new BroadcastReceiver()
{
	public void onReceive(Context context, Intent intent)
	{
		String action = intent.getAction();
		if (ACTION_USB_PERMISSION.equals(action))
		{
			synchronized (this)
			{
				device = (UsbDevice) intent.getParcelableExtra(UsbManager.EXTRA_DEVICE);

				if (intent.getBooleanExtra(UsbManager.EXTRA_PERMISSION_GRANTED, false))
				{
					if (device != null)
					{
						connection = manager.openDevice(device);
						connection.controlTransfer(0x21, 34, 0, 0, null, 0,0);
						connection.controlTransfer(0x21, 32, 0, 0,new byte[] { (byte) 0x80, 0x25, 0x00, 0x00,0x00, 0x00, 0x08 }, 7, 0);

						UsbInterface usbIf = device.getInterface(1);
						for (int i = 0; i < usbIf.getEndpointCount(); i++)
						{
							if (usbIf.getEndpoint(i).getType() == UsbConstants.USB_ENDPOINT_XFER_BULK)
							{
								if (usbIf.getEndpoint(i).getDirection() == UsbConstants.USB_DIR_IN)
									epIN = usbIf.getEndpoint(i);
								else
									epOUT = usbIf.getEndpoint(i);
							}
						}
						if (epIN == null)
						{
							//fail
						}
					}
				} else
				{
					//permision denide
				}
			}
		}
	}
};

After that, I used an asynctask to send bytes to the arduino and wait for the response (when the command is finished, and android can send a new one)

connection.bulkTransfer(epOUT, new byte[] { (byte) command,servoValue,val1B256,val1B1,val2B256,val2B1,speedByte,0}, 8, 0);
new WaitForResult().execute();

private class WaitForResult extends AsyncTask;
{
protected String doInBackground(byte[]... params)
{

    String s2 = "0";
    int len = 0;
    while (len <= 0)
    {
        len = connection.bulkTransfer(epIN, buffer, 40, 5000);
        s2 = new String(buffer);
        try
        {
            Thread.sleep(5);
        }  catch (InterruptedException ie){}
    }
    return s2;
  }
  protected void onPostExecute(String result)
  {
    setResult(result);
  }
}

The full source code: (this is no production code, just some proof of concept crappy code):
https://github.com/neuroprod/npPaintingMachine

So one sunny afternoon, I decided to do just that and I started to make my own 3D game from scratch.

I wanted to write a blog-post when it was finished. But I hugely underestimated the work (Now I understand why indie gamedevelopers take +3 years to finish a game). And I’ve already done so much it wouldn’t fit in one blog post. So I’m going to try to make multiple blog-posts, to light out different technical things I’ve used for my game, and keep you updated about the progress of the game itself.

The game:

Or more game engine, since I don’t really have any game play at the moment:

As you can see, there are still a lot of glitches and things to be done, but I’m happy where it’s going.

Some of the things I’ve done so far (not always complete, final or clean).

-The terrain generation, it’s an infinite/live generated terrain.
-Seamless day-night transitions.
-Cascade shadow maps, SSAO (screen space ambient occlusion).
-Normal maps, alpha textures, alpha shadows.
-Some simple LOD (level of detail) setup for the terrain mesh and the 3D objects.
-Bone animations in the vertex shader + interpolations between the animations. (models are loaded with Assimp)
-The main part of the rendering pipeline.
-Learned the basics of modeling and animating in Blender
-and a lot of other stuff.

The current Render pipeline

It’s very WIP at this moment, but here is a diagram of how the current rendering is done. More about this below.
(click for the full size image, open in new tab if you get the popup)

Deferred Shading:

One of the things I wanted to try was deferred shading, just for the heck of it. But I’m really glad I did. Since it seems much faster, and has some additional advantages (there are also some disadvantages)

How does it work:

You just render all the main geometry once, In 3 textures. a depth texture, a color texture with just the diffuse. and a color texture with the normals in worldspace.
you can do that in OpenGL all at once, with an FBO (frame buffer object) with multiple color attachments. And since you don’t do any lightning, its pretty fast. After that, you use those textures to calculate the lighting etc.

The big advantage is that you only have to calculate the lightning on the pixels that are actually visible on the screen. So you have no overhead of overlapping objects. And if you want to do some post processing stuff, like ambient occlusion or depth of field, you already have the depth texture and normal texture in place.

To get back from the depth to the worldspace, you just multiply the screen space with the inverse of the perspective matrix:

float depth = texture2D(depthTexture, uv_var).x*2.0-1.0;
vec3 viewSpace ;
viewSpace .xy = uv *2.0 -1.0;
viewSpace.z=depth;
vec4 worldSpace = perspectiveInvMatrix* vec4(viewSpace,1.0);
worldSpace.xyz/=worldSpace.w;

But what I really liked about it, is that you can have a lot of dynamic lights in your scene. When you use forward shading, your happy if you can use a couple of lights. But but with deferred shading you can have hundreds. You just render the lights as volumes in a separate pass (sphere for a point light, a cone for a spotlight etc). So you only have to calculate the light on the position where it really affects things, and since you use the normal and depth texture as input, only on the geometry you really see.

The light volumes rendered as normal geometrie:

The fragment shader calculation I use for the light volumes:

vec3 dir = worldPos.xyz-lightCenter; // worldPos from the depth
float l = length( dir);
float dist =1.0-pow((clamp (l ,0.0,lightSize_var)/lightSize_var),2.0); // quick light falloff
float lambert  =clamp(dot(normalize(dir) , -normal ),0.0,1.0); //shading
gl_FragColor = vec4(lightColor *dist * lambert   ,1.0);

The result of the Light pass:

I just use a blendfunct GL_ONE, GL_ONE (add) for all the lights, although technically not correct. But it looks good enough.
The lights in the final scene:

Lambert texutre lookup

An other neat trick I discovered while doing research for this, is using a Texture to lookup your shading color (found it in the Valve Team Fortress 2 Paper).
The great thing about this is that you can edit your shading in photoshop. In my case I used a 2D texture, so I can have al the subtle light differences between day and night.

This is the texture I use now, but it still needs some work:

I just use the shading color for the u and the time for the v lookup.

Here is the plain old shading:

And the shading after the look-up:
Night:

Day:

That’s it for now, hopefully some more blog-posts to follow this up

Source code and assets at Github ( at your own risk )