That sounds crazy, and yet that is exactly what we do using the "jit.catch~" object. This object samples a sound and stores it as numbers in the cells of a matrix which of course can be read as video. Although this is fairly easy to do, the matrix that "jit.catch~" makes is one dimensional, so we have to convert it into a matrix with both length and height in order to combine it with other matrices (which we also do).
