Close
0%
0%

Rasterphonic Glove

Analog video synthesizer you can touch with a five finger light gun. Makes music too.

Similar projects worth following
This project combines light guns and video synthesizers to produce a new type of interactive audio/video art. Analog circuitry is used to get the X,Y,Z position of five finger-mounted light sensors near a CRT monitor. Position values feed into video synthesizers to produce effects on the monitor manipulated by hand movement. Audio synthesizers turn the finger movements into music.


Project Goals:
- invent a new form of video synth art.
- Invent a new musical instrument.
- Pretend it's 1987 and do everything with discrete logic chips and analog circuits.
- Only use through-hole components and design PCBs in a way that can be copied on perfboard.

It's easier to show what the system does than try to explain it.

Video synthesizer controlled by light sensor position in three dimensions.


Example of the video effects that become possible with individual sensing on all five fingers.


Finger position sensors can be used to control audio synthesizers.


See build log for schematics, PCBs, and design explanations.

Before building the glove I did a simple proof of concept using the NES Zapper.

DisplayInchPresentable.brd

Second version of the LED display board.

brd - 43.93 kB - 05/06/2016 at 04:54

Download

ZmixPres2.brd

Z volume Mixer

brd - 24.46 kB - 04/25/2016 at 06:30

Download

LineVideoAnewPresentable.brd

Warp line generator v2

brd - 24.01 kB - 03/28/2016 at 18:48

Download

WarpGen.brd

Warp signal generator v2

brd - 43.00 kB - 03/28/2016 at 18:47

Download

RasterPhonicGcode1.zip

Gcode files for the CNC machine. Everything up to XYZ sensor board version 2.

x-zip-compressed - 266.62 kB - 03/15/2016 at 08:54

Download

View all 16 files

  • Display Board Version 2

    Russell Kramer05/06/2016 at 03:12 0 comments

    This is a new version of the indicator board used to satisfy my love of blinky lights. It shows the XYZ position of each finger on three LED bar graphs. This is a much simpler design than the last one because I'm using one LM3914 or KA2284 chip for each bar instead of using an an overly complicated system of analog multiplexers and LED matrices to control multiple bars with one chip. This version also puts the Z axis on its own bar instead of using it to control the LED brightness of the X,Y bars.


    Green: X
    Yellow: Y
    Blue: Z

  • Layout and boxing

    Russell Kramer04/30/2016 at 21:10 0 comments

    The system has a lot of modules to keep track of now so I've made an overview diagram showing how they're all connected. I'll keep this updated as new modules are added.

    I've decided to drop the design goal of an entirely wearable system. The power glove was a sensor interface to the NES not a standalone game system after all. The first two builds had a lot of problems because I was trying to keep everything on the hand and forearm: The first used a stack of small boards that became a nightmare of loose wires in tight spaces and the second was bulky and made hand movement feel clumsy. I could make things smaller by using SMD components and double sided boards, but I think it's more important to use hobbyist friendly DIP components.

    The only electronics on the glove are the phototransistors. A five foot cable connects the glove to the sensing and synthesizer circuit boards.

    I'm making panels for my modules by cutting punched flatbar into 9" pieces. The flatbar is 1.25" wide with holes 0.75" apart. I've been designing the PCBs so that switches and potentiometers align to these holes. I'll redo the entire panel as a solid piece of acrylic once I'm satisfied with the layout.

    From left to right: Audio Board 3, Atari Punk, Z volume Mixer, Rise/Fall speed control, Rise/Fall direction control, Display Board, Warp signal Generator, Warp Line board, Gradient Board.

    Wiring is quite messy. I'll make it clean and presentable after I'm past the point of adding new modules and moving things around.

  • Z volume Mixer Revision 2

    Russell Kramer04/25/2016 at 04:24 0 comments

    The last version of the Z-volume mixer was integrated into the main XYZ control board and there was no way to adjust its behavior.

    The new design is on a seperate board and uses a differential amplifiers to control a voltage-controlled-attenuators. The differential amplifier's amplification and common suppression are controlled by two separate potentiometers. A high level of differential amplification causes the attenuator to transition from fully closed to fully open between a few millimeters of finger movements. This produces tones that go off and on quickly ( a feature common to chiptunes). A low level of differential amplification means gradual increases in volume over several centemeters of finger movement. This is similar to the effect possible with theremins.


    The suppression control potentiometer sets the distance at which the amplification begins. The fingers can start making sound a millimeter from the screen or several centimeters back. This means the instrument can be played by tapping fingers against the screen, wiggling them near it, or a combination of both.



    A dual potentiometer is used to control the amplification. A change to a differential amplifier's amplification feedback resistor also produces a change to the common suppression. By using a dual potentiometer I can use one potentiometer to increase the amplification and the second to counter it's effect on suppression. This makes the control scheme simpler. I don't have to re-adjust the suppression every time I make a change to amplification.

    Video showing the new Z volume mixer.

    This is also the first video showing the new parts from other recent log entries:

    XYZ Sensor Board, Horizontal Warp Signal generator , Warp Line Video Circuit, and Audio Circuit 3. The Gradients circuit board has not changed.

  • Horizontal Warp Signal generator rev2.

    Russell Kramer03/28/2016 at 08:14 0 comments

    With the old Warped line video circuit the line generator and warp signal were on the same board with one unique warp signal generator per finger. In the new version the warp signal generation has been moved to a separate board. For each finger there are six different warp signal generation circuits that can be selected through a rotary switch. The rotary switch has a seventh position which I've left open for future use. The Z control voltage effects the warp signals so the line patterns also change as the finger moves closer to the screen. A DPDT switch swaps the comparator inputs which flips the line patterns vertically.

    Examples of the new warped line effects:

  • Warp Line Video Circuit revision 2

    Russell Kramer03/28/2016 at 06:49 0 comments

    This is the second version of the warped line video effect board. Take a look at the last one to see how much part count and complexity has come down. A big part of this is the decision to put the monostable alignment circuitry into the XYZ sensor board instead of the video boards. I plan to have several different video effects that respond to hand motions. By fixing alignment issues on the sensor board I can keep every video board fairly simple and straightforward.

    This board does not contain the signal generators to produce curved warping line effects. Those have been moved to a separate board (coming up next). With no warp signal applied the result is vertical lines that follow the finger sensors left and right.

    Another improvement is an sp3t switch to select one of two different 555 astable charge resistors (10k, 100k) or no resistor. No resistor results in a single line. Smaller resistor results in more densely packed lines and a greater effect from the warp signals.

  • Video Source 1 (T5200 + VCR)

    Russell Kramer03/26/2016 at 04:17 0 comments

    This has been completely stand alone video synthesiser up to now, but I've had a lot of input encouraging me to use the glove system to add distortion effects to an external video signal.

    I went to an E-waste recycling center looking for things that make video that wouldn't be suspicious in 1987. I came back with a T5200 and VCR. My goal was to combine their video outputs.

    Composite video and 640x480 VGA have the same timing parameters and number of scanlines. The difference is that composite combines the sync signals, and colour channels into one wire. The R,G,B uses a chroma conversion where the amplitude of the signal represents brightness and phase shift gives the hue. Phase shift is based on a known frequency given during the vertical porch time known as "colour burst".

    I tried sending the T5200's Hysnc and Vsync signals to the monitor while using the composite output to control R,G,B. This produces a black and white image (sort of).

    I had to take a dozen photos just to capture an image that was more than random squiggles and noise. 640x480x60Hz VGA and composite video have the same timing parameters in theory, but in real life there's a huge amount of instability in each signal. The composite video image is thrashing all over the monitor because it's not synchronized with the VGA signal.

    I know the VCR will be using a phase comparator to keep its head and tape feed motors running at a constant speed relative to the VHS control track. The phase comparator takes a waveform from the VHS control tracks, compares it to a reference, then uses a low-pass of the result to adjust the head and tape feed motor speeds. What I want to do is find that circuit and replace the reference waveform with one generated from the external VGA sync pulses. This should synchronize the composite output from the VCR with the VGA output of the T5200.

    I found an AN6344 chip, which must be the phase comparator I'm looking for. I was able to find a short Japanese datasheet too!!

    This chip is more than just a phase comparator. It's specifically designed for running a VCR and has several additional features. There's two different reference waveform inputs. One is used when recording and is extracted from an external video signal. The other is used for playing a tape and uses a reference created internally by the VCR. I want to cut this one and replace it with VGA sync.

    From the datasheet I gathered that pin 24 is playback reference waveform and pin 25 is recording reference waveform. I tested it out and found it's reversed (pin 25 is playback reference waveform). I'm either reading the datasheet wrong or the designers of this VCR decided to wire it backwards and use the recording motor control circuitry to run the motors during playback and vice versa.

    I also wired it up so the VGA red channel was coming from the T5200 while the green and blue channels came from the VCR. I found a pin inside the VCR that gave a sin wave at four times the frequency of the drive motor RPM and hooked that into the audio input while making the movie. You can hear the frequency change as I fiddle with the fast forward and rewind buttons.

    The VCR vertically synchronization is very good. Horizontal synchronization is better now, but not as stable as I'd hoped. This comes from instability in the tape drive and head motors. You don't see the instability during normal usage because the TV gets its sync from the tape. The video from the tape is stable relative to the sync pulses in the tape.

    I wanted the video to be more stable and attempted to design my own phase control circuitry. I was hoping that one referencing both the Hsync Vsync pulses would have better horizontal stability, but I couldn't produce anything noticeably better than the AN6344 circuitry. I think I've hit the hard limit on how stable the motors in a VCR can operate.

    This is one of several designs I tried. It uses a comparator to extract the composite sync pulses from the VCR's output, then produce a second composite Sync...

    Read more »

  • XYZ Sensor Board (version 2)

    Russell Kramer03/15/2016 at 06:15 0 comments

    I'm going into a second iteration of the project. The first step was a redesign of the XYZ sensor board. These are the major improvements:


    Increased sensitivity
    The sensors felt slow and unresponsive on some of the darker video patterns, particularly those generated by the blob circuit. More sensitivity means darker patterns can still trigger the XY sample-and-hold circuits.

    Monostable Delay
    This solves the signal delay between the light detectors and XY that caused X values to be a few inches to the right of the sensor. There is now a delay in the detector pulse lasting the time taken to draw one scanline less the signal delay time. This means the X sampling occurs at the correct position one scanline down from the one that triggered the sensor.


    Better Layout
    All XYZ outputs are in a row on the PCB instead of scattered all over the board.

    Better Z Axis sensing.
    The Z axis circuitry now uses a Sallen Key low pass filter to remove the 60Hz signal. It's amplified to a 0-5v range and buffered now too. The Z-axis audio mixer has been removed. It's going on a separate PCB now.

    Ramp Circuitrs trigger from Hsync, Vsync.

    In the previous design I was partially correcting the signal delay by generating seperate timing pulses on the VGA board instead of triggering the ramps with Hsync and Vsync. The monostable delay completely corrects the signal delay, so this is no longer needed. An additional transistor went into the pulse triggers because Hsync, Vsync are normally high pulse low.

    Schematic of the redesigned sample-and-hold trigger circuit and the Z axis circuit.

    This board does not contain VGA signal generation circuits like the last one did. It takes Hsync and Vsync from an external source and contains the circuitry to produce five XYZ values with phototransistors.

  • Audio Circuit 3

    Russell Kramer03/15/2016 at 05:57 0 comments

  • Audio Circuit 2 (Atari Punk).

    Russell Kramer03/11/2016 at 04:09 0 comments

    I wasn't a fan of how the last audio board turned out, so I tried again using an old classic. This board is just five Atari Punk consoles (one per finger). The only alteration is using X,Y control voltages as 555 VCO inputs. Those pins aren't used on the regular Atari Punk circuit.

    I also added switches to allow the Z axis video control to be switched off. When the switch is up the line board will not make lines disappear when the finger moves away from the screen. Making "music" means tapping fingers against the screen which produces a flickering mess when the Z axis has such a huge effect on the video output. I plan to redo the video board and use the Z position to change the output in a more subtle way.

    I'm not a huge fan of how this circuit turned out either. I get different sounds from fingers in different positions, but they don't behave in a predictable/controllable way. That's just the Atari Punk being an Atari Punk though. It's a noise maker not a musical instrument.

  • Audio Circuit 1

    Russell Kramer03/11/2016 at 03:58 0 comments

    This is the first audio circuit I came up with. I don't like it. The design is similar to the one used in the Zapper project, but I over-engineered it. There's four oscillator circuits per finger total, but only two can be active at a time because I tried to be clever and have them share a variable capacitor. Of the two active VCOs one is controlled by the finger's Y position, and one is controlled by the finger's X position. One VCO is constructed with a 74HC4046 (phase locked loop) and one constructed with a 74HC4093 (NAND schmitt). The two oscillators respond somewhat differently to changes in finger position.

    The base frequency of each VCO is controlled by a variable capacitor. The variable capacitor is built with a set of regular capacitors that connect in parallel through a pair of binary thumb wheels. I really should have only used one binary thumb wheel per variable capacitor; the fine tuning of the lower digits is unnecessary.

    The circuit board also uses LM3914 to produce an LED dot display of the Y axes. This time one LM3914 per measured voltage instead of the overly-complex matrix multiplex design.

    The PCB design isn't great. Way too many jumper wires.

    Here's a test of the audio circuit board. It's dirty buzzing sounds not music. I like the sound each finger makes by itself, but when multiple fingers touch the screen there's no distinguishable tones or beat. The video circuits produce the best patterns when multiple fingers are touching the screen simultaneously, so there's discord with how I want to use the glove.

    From left to right: Audio circuit, Video circuit, main control board.

View all 23 project logs

Enjoy this project?

Share

Discussions

Bruno Laurencich wrote 10/20/2017 at 10:13 point

I can't belive it, this is so amazing!!! are you (or someone) using it for making performances or something currently?

  Are you sure? yes | no

0xCAFEAFFE wrote 05/05/2017 at 17:25 point

this is amazing! i love your work of art! :)

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates