Optic Research

Good lighting is the key to a well produced ReacTable.

Two things MUST be taking into accord when thinking about lighting 

1. To track the objects the table needs to be is well illuminated. Allowing the camera and ReacTIVision to see the markers clearly.
2. The glass screen must be dark enough so that the user can see the projected visual aid.

So I ask myself this question.....

How can I have a dark surface and illuminate the table at the same time?

Sound impossible?

All light from the visable specrum must be filtered through the camera, so that the projector image does not interfere with the tracking of the markers. 

As infrared light is just below the visible spectrum in frequency, an IR lamp can be used to illuminate the table so that the markers can be seen by the camera. 

IR lamps on ebay.co.uk go from about £20. I need to know the exact distance between the camera and the marker objects to know exactly how many LED IR's I need in order to get enough light. There is a shop on ebay that sells IR lamps, I will email the owner and see if I can get any further information on this.

If I am using Inferred light then I must think about how I will filter out all other visible light. The light from the projection will interfere with the camera. A filter can be used to do this, but I have found an alternative method.

The PS3 eye toy which I have decided to use in this project has a IR blocking lense inside the sencer.  I aim to take the camera apart and initially remove the IR blocking lense from the web cam.

below is a very helpfull video tutorial on how to do this:

part 2

The video tells me I can use an old floppy disc as a visible light blocking filter.

After watching this tutorial I decided to take my eye toy and have a go............

My ReacTable Status 3

Building the cabinate

I now have all he conponants to begin to construct the cabinate where inside my video camera projector and light sorce will sit.  I have the follow

ing items;

1. MDF 
2. Glass
3. Screws 
4. Glue
5. Woodern battons 
6. Electronic screwdriver

The picture below shows the top of my cabinate where my glass will sit. It is of 2 pieces of square wood one slightly bigger than the other, creating a lip for the glass to sit on.

 

The picture below shows me marking the width of the board that will form the sides to my cabinet.  By doing this I can glue wooden batons on the the board to screw all the pieces together, making it like an IKEA flat pact ReacTable. 

The next picture shows me marking the width of  the batons  so when I glue them down they will be in the exact place.

Here I can be seen marking the spot for the screws to be placed, 3 times on each baton, equalling it out to 9 screws for 3 sides.

Here I am using the electric screw driver to make indents for the screws, this will make it easer for me to screw it all together.

This picture shows me using the hand screw driver to make the indents in which the screws will sit.

The final picture is of me putting glue on the batons, which I will stick onto the side the board to screw all 3 sides together

My ReacTable Status 2

The hardware side of things...

With regards to this I have so far been working via my laptop built in isight, I have just bought the following items;

1. Round glass Tabletop which I had custom made from Trefforest glass and cost me £30. The glass has a diameter of 80cm 
2. MDF wood which I had cut into 4 peices.

below is a diagram of the design of my further ReacTable cabinet

Glass table top case Side *4

I have decided to make my ReacTable square but keeping to the original circular peice of glass for the surface. 

I have chosen this design for a few reasons, 

1. It is easier to construct than a circular cabinate
2. It is cheaper 
3. I can also project of to it's sides if need be.

I aim to perform my ReacTable for my final year recital in late May, and I feel that it is more important to master the audio side of the project before moving onto glass surface visual feed back element.

Yet.... 

If I produce a square box and paint it white I can project on the box more easily than on to its surface.  Instead of having the visual feedback I can program Pure Date to trigger pictures and videos onto the side of my box via the fiducial marker method.

This will it interesting for the audience as well as giving it a new edge that Is my own design.

This is the main reason why I have made my cabinate square.

I will post pictures of the finished product when I start to construct my reactable cabinet.

   

My ReacTable Status 1

The software side of things...

So far I have printed out the fiducial markers and I have downloaded ReacTIVision and TUIO protocol for pure data. I  have decided to work with Pure Data and my computer's built in eye site that I am using temperally until I get a external camera (this will follow in my research later).

Pure Data has been programmed to reconise fiducial marker 16. I will run some tests to see if 

Pure Data and ReacTIVision respond in the correct way.

Test 1, Does ReacTIVision recognise fiducial number 16 and send it's imformation to Pure Data via TUIO?

Test 1 compleate

Yes ReacTivision does see fidutial 16, the picture below shows this, note how the number of fiducial is displayed in the center of the marker. Pure Data does detect its x/y possitioning.

Test 1.2, Can osc sound be heard once marker is held up?

Test 1.2 complete 

Yes when I hold up a marker 16 osc sound can be heard, demonstration video can be seen below.

Test 1.3,  Turn fiducial left, does pitch increase? 

Left turn = Pitch Increase

Test 1.4, Turn fiducial marker right, does pitch decrease?

Right turn =pitch decrease

(note video is inverted)

As you can see from my video's my tests were a success, the next thing I shall do with regards to programming the music software will be to add another affect such as reverb or delay to the x position of my markers.  Meaning that when I move the marker to the left or right it will effect the amount of musical effect on the sound. I will post more of my progress in a further blog.

Further Camera Research

Searching for the right camera proved slightly disheartening, especially when knowing that the cameras that will give the best performance carrys a double zero price tag...

ouch....

At the moment I am just waving fiducial markers in front of my built in isight on my laptop, but when I finally build my table I will need something that is;

1. Cheap to Buy
2. Robust
3. Works with ReacTIVision
4. Easy to assemble
5. Captures the entire glass tabletop

Ruling out the AVT and faster frame rate cameras, I have come up with two possibilities 

The apple isight

The eye sight costs about £80 and was build for apple computers before they started building them into the framework of the computer.  They have been discontinued since but are available to buy on Ebay.co.uk they go from £40.

Test 1, does ReacTIVision see this camera?

 

Test 1 complete 

Yes this camera works through my computer via a fire wire, and ReacTIVision responds reasonably well.

Things to consider at this point,

1. Will The camera capture the entire table?
2. Will the camera work quickly enough in real time to track the markers?

The eye sight roughly works at 25-30 frames per second.

As I aim to have a 80cm diameter table top, and because the eye sight do

es not have a wide eye lens, I predict that the camera will not capture the whole of the glass. Which will mean I will have to experiment with mirrors.

The second camera is the Sony PS3 eye toy.

The eye toy has a frequency of 640*480 at 60Hz and 320*240 at 120Hz, according to the spec on the ReacTIVision website it may work quite nicely.

I bought this little camera from Game and it cost me £25. I do not know if my computer will respond to the PS3 eye because it is USB and not firewire. I will run some tests to find out.

Test 2, does the PS3 eye toy work with ReacTivision?

My computer recognises the USB camera, yet this camera does not respond at all with applications on my laptop such as quicktime, ichat and reacTIVision.

The eye sight roughly works at 25-30 frames per second.

The ReacTIVision website states "A very affordable and relatively good camera for this purpose is the Sony PS3eye, which is also working well under Windows,Linux and Mac OSX."

This sounds promising, if my computer detects it and if the ReacTIVision website says that it works well under OSX then there must be a way of it working. I found and downloaded a driver from http://webcam-osx.sourceforge.net/. This driver allows my computer to use the USB eye toy as a webcam.

Test 2.2, After downloading Macam does the PS3 eye with ReacTIVision?

No, after I downloaded Macam my computer responds to the PS3 eye like a webcam, yet still does not respond at all with ReacTIVision.

Now I can use my PS3 eye toy with my computer but ReacTIVision still refuses to recognize the eye toy as a camera. I must have missed something....

Below is a picture of the PS3 eye toy working as a web cam on my computer.

Looking back at my method of installation in regards to the Macam driver I discovered one simple rule that i didn't follow. When installing Macam you must copy it's component into your /library/quicktime folder for it to work with other applications such as ichat and hopefully ReacTIVision.

Test 2.3, after copying a componant into /library/quicktime folder, does the PS3 eye toy work with ReacTIVision?

Yes, I simply need to follow the following rules;

1. Open Macam (working)
2. Close Macam
3. Open ReacTIVision (defalt built in isight appears)
4. Press (H) = ReacTIVision option mode
5. Press (O) = ReacTIVision camera settings
6. Select Macam Sony HD Eye for PS3 

Test 2 Complete, 

Status: 

1. PS3 working with ReacTIVision
2. Fiducial markers are recognised from a close distance.

Thing to consider at this point...

Will the camera cover the entire range of the glass table top and successfully tract the fiducial markers?

How will the camera behave when considering optics and the projection side of things?

 

Camera Research

For the best results the ReacTIVision website says;

"You should make sure that the camera has an acceptable lens and sensor size. For lowest latency and best performance we recommend firewire cameras from the top range, such as industrial cameras with a high framerate, resolution and sensor size. These cameras usually also come with high quality C-mount lenses. Cheaper firewire cameras, such as the unibrain fire-i also allow optional wide-angle lenses. From the large range of available USB cameras we recommend to use high end models with a native resolution of at least 640x480 at a frame rate of 30Hz. A very affordable and relatively good camera for this purpose is the Sony PS3eye, which is also working well under Windows, Linux and Mac OSX. DV cameras supporting full-frame mode are suitable, while those with interlaced mode only, will not work at all."

If I had a much bigger budget  for my project I would of liked to get a ATV pike or AVT guppy, my suspicions tell me that the true ReacTable works with this sort of camera.

These AVT guppy cameras are small firewire CMOS and CCD cameras, designed to perform very well. They are pretty pricey though and start from around £550 - £1000. You can find them here... 

http://www.edmundoptics.com/Onlinecatalog/displayproduct.cfm?productID=2716

I couldn't find a quote online for the AVT Pike cameras but I assume the are similar in price to the AVT Guppy.

 

CMOS and CCD

This is taken from (http://www.dalsa.com/corp/markets/CCD_vs_CMOS.aspx)

"CCDs have traditionally provided the performance benchmarks in the photographic, scientific, and industrial applications that demand the highest image quality (as measured in quantum efficiency and noise) at the expense of system size. CMOS imagers offer more integration (more functions on the chip), lower power dissipation (at the chip level), and the possibility of smaller system size, but they have often required tradeoffs between image quality and device cost."

Things to consider.......

After researching different camera's I have come to the conclusion that I will not be able to afford a fast camera such as the AVT Guppy just yet. In the meantime I will find an alternative camera to use.

TUIO Demo

Starting the patch....

Martin Kaltenbrunner of the music techonology group pro

vides a TUIO demo which will be my starting point of creating my music software. 

Test 1, does Pure Data reconise fiducial marker number 15?

Test complete, Pure Data prints addObject: 15

Test 1.2, does Pure data reconise when fiducial marker is taken away from isight?

Test complete, Pure Data prints removeObject: 15

With the help of my teacher we will begin to design the music sequencer and possibly a visual patch too.. 

Ill keep you posted

So You Want to Build a ReacTable?

The music technology group over in Barcelona have done most of the work, but even so I have to a substantial amount of the work myself, which includes;

1. Software research
2. Camera and projector research
3. Optics research
4. Design and construction of glass cabinet
5. Writing the synthesizer software
6. Testing
7. Composition
8. Performance

A Little Magic

The ReacTable live in Oslo