Differences

This shows you the differences between two versions of the page.

--- processing:video_capture [2013/04/12 00:00] – mithat
+++ processing:video_capture [2013/08/23 04:10] (current) – [Video capture examples] mithat
@@ Line 1: / Line 1: @@
-====== Video Capture Examples ======
+====== Video capture examples ======
-The following have been adapted from the [[http://www.processing.org/reference/libraries/video/Capture.html|Processing 2.0 library reference]] and from examples at [[http://www.learningprocessing.com/|Learning Processing]].
+The examples below lean heavily on documentation and code from the [[http://www.processing.org/reference/libraries/video/Capture.html|Processing 2.0 library reference]] and at [[http://www.learningprocessing.com/|Learning Processing]].
+===== List available capture devices =====
+You'll need at least one available device to do video capture. The following sketch will print out a list of available capture devices to the console. On Linux (the only platform I tested) different resolutions show up as different devices.
-===== Available capture devices =====
 <file java list_capture_devices.pde>
-/** Print out a list of available capture devices
+/** Print out a list of available capture (i.e., video) devices
  * @author Mithat Konar
  */
@@ Line 20: / Line 22: @@
     exit();
   }
-  else {
-    println("Available cameras:");
+  println("Available cameras:");
-    for (int i = 0; i < cameras.length; i++) {
+  for (int i = 0; i < cameras.length; i++) {
-      println("camera " + i + ": " + cameras[i]);
+    println("camera " + i + ": " + cameras[i]);
+  }
+}
+</file>
+===== Show video =====
+There are two ways you can set up video capture: by requesting capture parameters or specifying the device.
+==== By requesting capture parameters ====
+<file java show_capture_by_parameters.pde>
+/** Show video using requested capture parameters
+ * @author Mithat Konar
+ */
+import processing.video.*;
+Capture cam;
+final int CAM_WIDTH = 320;
+final int CAM_HEIGHT = 240;
+final int FPS = 15;
+void setup() {
+  size(CAM_WIDTH, CAM_HEIGHT);
+  if (Capture.list().length == 0) {
+    println("There are no cameras available for capture.");
+    exit();
+  }
+  // Instantiate a new Capture object, requesting the specs:
+  cam = new Capture(this, CAM_WIDTH, CAM_HEIGHT, FPS);
+  cam.start();  // In Processing 2.0, you need to start the capture device
+}
+void draw() {
+  if (cam.available() == true) {
+    cam.read();
+  }
+  image(cam, 0, 0);
+}
+</file>
+==== By specifying device ====
+Processing will crop the capture to the canvas size if it doesn't fit.
+<file java show_capture_by_devnum.pde>
+/** Show video using specified capture device
+ * @author Mithat Konar
+ */
+import processing.video.*;
+Capture cam;
+final int CANVAS_WIDTH = 320;
+final int CANVAS_HEIGHT = 240;
+final int camnum = 55;  // Pick a camnum that makes sense given the canvas size!
+void setup() {
+  size(CANVAS_WIDTH, CANVAS_HEIGHT);
+  String[] camera_list = Capture.list();
+  if (camera_list.length == 0) {
+    println("There are no camera_list available for capture.");
+    exit();
+  }
+  println("Available cameras:");
+  for (int i = 0; i < camera_list.length; i++) {
+    println("camera " + i + ": " + camera_list[i]);
+  }
+  println();
+  println("Using camera " + camnum + ", " + camera_list[camnum] + ".");
+  // Instantiate a Capture object by specifying the device name stored
+  // in camera_list array:
+  cam = new Capture(this, camera_list[camnum]);
+  cam.start();  // In Processing 2.0, you need to start the capture device
+}
+void draw() {
+  if (cam.available() == true) {
+    cam.read();
+  }
+  image(cam, 0, 0);
+}
+</file>
+<WRAP center round tip 80%>
+===== Capture is a PImage =====
+Something the [[http://processing.org/reference/libraries/video/Capture.html|documentation]] on the Capture class doesn’t mention is that Capture is derived from [[http://processing.org/reference/PImage.html|PImage]]. This is the key to manipulating Capture output. You need to dig into the Capture source code to figure that out:
+<code java>public class Capture extends PImage implements PConstants { ...</code>
+This means all the methods and fields available to PImage objects should also be available to Capture objects.
+</WRAP>
+===== Reduce resolution =====
+The following example will reduce the resolution of the captured video in both space and time. In addition, it adds a filter that renders the output in black and white. (This might be useful for determining roughly the minimum resolution required for useful processed video results.)
+<file java capture_reduce_res.pde>
+/** Reduce resolution of captured image.
+ * @author Mithat Konar
+ */
+import processing.video.*;
+Capture cam;  // The video capture device.
+PImage img;   // Buffer image.
+// Canvas parameters
+final int CANVAS_WIDTH = 320;
+final int CANVAS_HEIGHT = 240;
+final int CANVAS_FPS = 4;
+// Video capture parameters (adjust as neeeded for your
+// platform's available capture options):
+final int CAM_WIDTH = 320;
+final int CAM_HEIGHT = 240;
+final int CAM_FPS = 15;
+// Rendering parameters: number of cells the rendered
+// image should be in each direction:
+final int REDUCED_WIDTH = 128;
+final int REDUCED_HEIGHT = 96;
+void setup() {
+  frameRate(CANVAS_FPS);
+  size(CANVAS_WIDTH, CANVAS_HEIGHT);
+  if (Capture.list().length == 0) {
+    println("There are no cameras available for capture.");
+    exit();
+  }
+  // Instantiate a buffer image used for subsampling, etc.
+  img = createImage(REDUCED_WIDTH, REDUCED_HEIGHT, RGB);
+  // Instantiate a new Capture object, requesting the specs:
+  cam = new Capture(this, CAM_WIDTH, CAM_HEIGHT, CAM_FPS);
+  cam.start();  // In Processing 2.0, you need to start the capture device
+}
+void draw() {
+  // Grab a frame
+  if (cam.available() == true) {
+    cam.read();
+  }
+  // The following should work but doesn't.
+  //  cam.resize(REDUCED_WIDTH, REDUCED_HEIGHT);
+  // So we use an interim buffer image instead:
+  img.copy(cam, 0, 0, CAM_WIDTH, CAM_HEIGHT, 0, 0, REDUCED_WIDTH, REDUCED_HEIGHT);
+  // And draw the image (full canvas):
+  image(img, 0, 0, width, height);
+  filter(GRAY);  // Render image() in B&W.
+}
+</file>
+This version walks through each pixel in the reduced image and renders its value as an NxN block. (N is called OUTPUT_SCALE in the code below.)
+<file java capture_reduce_res-2.pde>
+/** Reduce resolution of captured image. Display results in blocks.
+ * @author Mithat Konar
+ */
+import processing.video.*;
+// === Program constants === //
+// Rendering parameters:
+// number of cells the rendered image should be in each direction:
+final int REDUCED_WIDTH = 32;
+final int REDUCED_HEIGHT = 24;
+// Canvas parameters:
+// number of times you want REDUCED image blown up:
+final int OUTPUT_SCALE = 10;
+// frame rate of rendered output:
+final int CANVAS_FPS = 8;
+// Video capture parameters
+// (adjust as neeeded for your platform's available capture options):
+final int CAM_WIDTH = 320;
+final int CAM_HEIGHT = 240;
+final int CAM_FPS = 15;
+// === Global variables ===//
+Capture cam;  // The video capture device.
+PImage img;   // Buffer image.
+// === GO! === //
+void setup() {
+  frameRate(CANVAS_FPS);
+  size(REDUCED_WIDTH*OUTPUT_SCALE, REDUCED_HEIGHT*OUTPUT_SCALE);
+  if (Capture.list().length == 0) {
+    println("There are no cameras available for capture.");
+    exit();
+  }
+  // Instantiate a buffer image used for subsampling, etc.
+  img = createImage(REDUCED_WIDTH, REDUCED_HEIGHT, RGB);
+  // Instantiate a new Capture object, requesting the specs:
+  cam = new Capture(this, CAM_WIDTH, CAM_HEIGHT, CAM_FPS);
+  cam.start();  // In Processing 2.0, you need to start the capture device
+}
+void draw() {
+  // Grab a frame
+  if (cam.available() == true) {
+    cam.read();
+  }
+  // Using a buffer because
+  // cam.resize(REDUCED_WIDTH, REDUCED_HEIGHT);
+  // doesn't work :-(
+  img.copy(cam, 0, 0, CAM_WIDTH, CAM_HEIGHT, 0, 0, REDUCED_WIDTH, REDUCED_HEIGHT);
+  img.loadPixels();
+  // For each column in img:
+  for (int col = 0; col < REDUCED_WIDTH; col++) {
+    // For each row in img:
+    for (int row = 0; row < REDUCED_HEIGHT; row++) {
+      // Get color from pixel at col, row
+      color c = img.pixels[col + row*img.width];
+      fill(c);
+      stroke(c);
+      rect(col*OUTPUT_SCALE, row*OUTPUT_SCALE, OUTPUT_SCALE, OUTPUT_SCALE);
     }
   }
@@ Line 29: / Line 264: @@
 </file>
+You can render grayscale instead of color above by replacing the line <code java>color c = img.pixels[col + row*img.width];</code> with <code java>float c = brightness(img.pixels[col + row*img.width]);</code>
+===== Brightest pixel =====
+The following example takes the reduced resolution camera feed above and draws a colored circle inside the brightest pixel found. If there is more than one pixel with the same brightness, then it shows the first one (scanning from top-left to bottom-right). It also adds some code to indicate frame timing.
+<file java capture_brightest_pixel.pde>
+/** Reduce resolution of captured image and indicate brightest pixel.
+ * @author Mithat Konar
+ */
+import processing.video.*;
+// === Program constants === //
+// Rendering parameters:
+// number of cells the rendered image should be in each direction:
+final int REDUCED_WIDTH = 32;
+final int REDUCED_HEIGHT = 24;
+// Canvas parameters:
+// number of times you want REDUCED image blown up:
+final int OUTPUT_SCALE = 20;
+// frame rate of rendered output:
+final int CANVAS_FPS = 6;  // should divide evenly into CAM_FPS
+// to avoid jitter.
+// Video capture parameters
+// (adjust as neeeded for your platform's available capture options):
+final int CAM_WIDTH = 320;
+final int CAM_HEIGHT = 240;
+final int CAM_FPS = 30;
+// === Global variables ===//
+Capture cam;  // The video capture device.
+PImage img;   // Buffer image.
+int blink_state = 0;
+// === GO! === //
+void setup() {
+  frameRate(CANVAS_FPS);
+  size(REDUCED_WIDTH*OUTPUT_SCALE, REDUCED_HEIGHT*OUTPUT_SCALE);
+  ellipseMode(CORNER);
+  if (Capture.list().length == 0) {
+    println("There are no cameras available for capture.");
+    exit();
+  }
+  // Instantiate a buffer image used for subsampling, etc.
+  img = createImage(REDUCED_WIDTH, REDUCED_HEIGHT, RGB);
+  // Instantiate a new Capture object, requesting the specs:
+  cam = new Capture(this, CAM_WIDTH, CAM_HEIGHT, CAM_FPS);
+  cam.start();  // In Processing 2.0, you need to start the capture device
+}
+void draw() {
+  int brightestCol = 0;
+  int brightestRow = 0;
+  float bightestIntensity = 0.0;
+  // Grab a frame
+  if (cam.available() == true) {
+    cam.read();
+  }
+  // We are using a buffer img because
+  // cam.resize(REDUCED_WIDTH, REDUCED_HEIGHT);
+  // doesn't work :-(
+  img.copy(cam, 0, 0, CAM_WIDTH, CAM_HEIGHT, 0, 0, REDUCED_WIDTH, REDUCED_HEIGHT);
+  img.loadPixels();
+  // For each column in img:
+  for (int col = 0; col < REDUCED_WIDTH; col++) {
+    // For each row in img:
+    for (int row = 0; row < REDUCED_HEIGHT; row++) {
+      // Draw the pixel intensity.
+      float pixelIntensity = brightness(img.pixels[col + row*img.width]);
+      fill(pixelIntensity);
+      stroke(pixelIntensity);
+      rect(col*OUTPUT_SCALE, row*OUTPUT_SCALE, OUTPUT_SCALE, OUTPUT_SCALE);
+      // Determine whether this is the brightest pixel in this frame.
+      if (pixelIntensity > bightestIntensity)
+      {
+        bightestIntensity = pixelIntensity;
+        brightestCol = col;
+        brightestRow = row;
+      }
+    }
+  }
+  // Highlight brightest pixel.
+  fill(#000000, 0);
+  stroke(#ff0000);
+  strokeWeight(2);
+  ellipse(brightestCol*OUTPUT_SCALE, brightestRow*OUTPUT_SCALE,
+       OUTPUT_SCALE, OUTPUT_SCALE);
+  // Frame timer
+  fill(#000099, 100);
+  stroke(#0000ff, 100);
+  strokeWeight(1);
+  blink_state = ++blink_state % CANVAS_FPS;
+  rect(0, 0, blink_state*OUTPUT_SCALE, OUTPUT_SCALE);
+}</file>