programming video manipulation

[Olivia Robinson]

Hi,
I am an artist who creates interactive video installations that are
controlled using a set of sensors, a microcontroller , computer and
video projector. Currently I am using MAX/MSP and Jitter to program
how information collected from the sensors then is used to manipulate
pre-recorded video footage.

I have been investigating Python as an alternative to using
MAX/MSP/Jitter to program the collection of information from the
sensors and the resulting video manipulation. I have not been able to
find information about manipulating video with Python, or collecting
information from a microcontroller . I have heard that I may need a
library of commands already written in C or C++. Does anyone have any
information about using Python with microcontroller s or manipulating
video? Ideas?

My current programming experience is limited to using MAX/MSP/Jitter
and Macromedia Director.

Thank you much for your time,
Olivia

[Doug Holton]

Re: programming video manipulation

Olivia Robinson wrote:[color=blue]
> Hi,
> I am an artist who creates interactive video installations that are
> controlled using a set of sensors, a microcontroller , computer and video
> projector. Currently I am using MAX/MSP and Jitter to program how
> information collected from the sensors then is used to manipulate
> pre-recorded video footage.
>
> I have been investigating Python as an alternative to using
> MAX/MSP/Jitter to program the collection of information from the sensors
> and the resulting video manipulation. I have not been able to find
> information about manipulating video with Python, or collecting
> information from a microcontroller .[/color]

Hi, here's my best guess at some tips for doing this, but it would be
nowhere as easy as using Jitter I bet (but free instead of $850).

First, see this related project that used sensors to create music, and
is programmed with Python. Simon Burton could give you some advice.

http://www.pcworld.idg.com.au/index.php?id=716362867

http://arrowtheory.com/hypersense/

The most difficult thing to program on your own may be reading the
sensor data from the computer. It used to be simple using serial ports,
but now we use USB. There are no solutions here, just some pointers to
related resources:

http://developers.coedit.net/UsbProgramming

http://developers.coedit.net/AudioProgramming

For the video and real-time video transformations , you would likely want
to use OpenGL from Python (pyopengl). It is possible to load a frame
from a video file as a texture in OpenGL. And you can manipulate it
just like any other texture or surface. Here are demos of this below.
The code can be adapted to pyopengl, but there is also some
Windows-specific and Mac-specific code to read the video format.

http://nehe.gamedev.net/data/lessons/lesson.asp?lesson=35

http://developer.apple.com/samplecode/OpenGL_Movie/OpenGL_Movie.html

[Ian]

Re: programming video manipulation

On Tue, 25 May 2004 15:15:39 -0400, Olivia Robinson <robino@rpi.edu >
wrote:

[color=blue]
>I have been investigating Python as an alternative to using
>MAX/MSP/Jitter to program the collection of information from the
>sensors and the resulting video manipulation. I have not been able to
>find information about manipulating video with Python, or collecting[/color]

Hi Olivia,

I don't have answers for all your questions, but you may be interested
in this project:

http://aid.interaccess.org/drupal/

We are developing a standard microcontroller module for use by
artists, and have developed Python interfaces for it. We've also used
Director and Pure Data (similar to Max I understand). Currently the
module uses a serial interface, we use usb-to-serial adapters. We
have an experimental usb board interface and will be implementing usb
support for our Python module. All source will be available on our
site, as well as recipes for software and hardware configuration.
There are a variety of hardware interface modules available for the
board.

By the way you can build your own board from the info on our site,
or soon buy finished boards as well. We don't have any direct support
for video at the moment, but that should change soon.

Ian