Background Replacement

11 Jul 2011

This is the first episode of my new Computer Vision for the Robotic Age podcast. This episode is on replacing the background of a live video with another video. The background replacement algorithm is implemented live using the HornetsEye real-time computer vision library for the Ruby programming language.

You can also download the video here

hornetseye-background-replacement.m4v (27.63 MB)

I am new to podcasting. So feel free to let me know any suggestions.

See Also:

Camera Calibration

23 May 2011

I am currently working on camera calibration. Many implementations require the user to manuallly point out corners. Here is an idea on how to detect and label corners automatically.

Apply Otsu Thresholding to input image.
Take difference of dilated and eroded image to get edge regions.
Label connected components.
Compute corners of input image (and use non-maxima suppression).
Count corners in each component.
Look for a component which contains exactly 40 corners.
Get largest component of inverse of grid (i.e. the surroundings).
Grow that component and find all corners on it (i.e. corners on the boundary of the grid).
Find centre of gravity of all corners and compute vectors from centre to each boundary corner.
Sort boundary corners by angle of those vectors.
Use non-maxima suppression on list of length of vectors to get the 4 “corner corners” (convexity).
Use the locations of the 4 “corner corners” to compute a planar homography mapping the image coordinates of the 8 times 5 grid to the ranges 0..7 and 0..4 respectively.
Use the homography to transform the 40 corners and round the coordinates.
Order the points using the rounded coordinates.

Further work is about taking several images to perform the actual camera calibration.

Thanks to Manuel Boissenin for suggesting convexity for finding the “corner corners”.

Update:

After calibrating the camera the ratio of focal length to pixel size is known (also see Zhengyou Zhang’s camera calibration). Once the camera is calibrated, it is possible to estimate the 3D pose of the calibration grid in every frame.

I have created a screencast on how to locate the chequerboard calibration pattern.

See also:

Broken Tonight

09 May 2011

Histogram-based classification

06 Apr 2011

Here is a small presentation showing histogram-based classification with Ruby.

Here is the program to capture the reference pictures

#!/usr/bin/env ruby
require 'rubygems'
require 'hornetseye_v4l2'
require 'hornetseye_rmagick'
require 'hornetseye_xorg'
include Hornetseye
BOX = [220 ... 420, 140 ... 340]
input = V4L2Input.new
mask = MultiArray.bool(640, 480).fill!
mask[*BOX] = true
labels = ['reference', 'dragon', 'knight', 'camel']
labels.each do |label|
  X11Display.show(:title => label.capitalize) do
    img = input.read_ubytergb
    mask.conditional img, img >> 1
  end[*BOX].save_ubytergb "#{label}.jpg"
end

The program for live classification is shown below

#!/usr/bin/env ruby
require 'rubygems'
require 'hornetseye_v4l2'
require 'hornetseye_rmagick'
require 'hornetseye_xorg'
include Hornetseye
PI = Math::PI
RAD = 2.0 * PI
VAL = 117.0
HBINS = 8
CBINS = 8
THRESHOLD = 32
N = 5
BOX = [220 ... 420, 140 ... 340]
class Node
  def hsv_hist(hbins = HBINS, cbins = CBINS)
    alpha = 2 * r.to_sint - g - b
    beta = Math.sqrt(3) / 2 * ( g.to_sint - b )
    h = ( (Math.atan2(beta, alpha) + PI) * (HBINS / RAD) ).to_int.clip 0 .. HBINS - 1
    c = ( Math.sqrt(alpha ** 2 + beta ** 2) * (CBINS / VAL) ).to_int.clip 0 .. CBINS - 1
    [h, c].histogram(hbins, cbins).to_int
  end
  def cmp(other)
    (self - other).abs / (self + other).major(1.0)
  end
end
input = V4L2Input.new
mask = MultiArray.bool(640, 480).fill!
mask[*BOX] = true
labels = ['reference', 'dragon', 'knight', 'camel']
hists = labels.collect do |object|
  MultiArray.load_ubytergb("#{object}.jpg").hsv_hist
end
history = ['reference'] * N
X11Display.show do
  img = input.read_ubytergb
  img_hist = img[*BOX].hsv_hist
  similarities = hists.collect { |hist| img_hist.cmp(hist).abs.sum }
  label = labels[similarities.index(similarities.min)]
  history = [label] + history[0 ... N - 1]
  if history == [label] * N
    system "echo '#{label}' | festival --tts" if label != 'reference'
  end
  history != ['reference'] * N ? mask.conditional(img, img >> 1) : img
end

And here is a demonstration video of the two programs

Bitcoin peer-to-peer currency

27 Feb 2011

I watched this interview with Steve Gibson about the Bitcoin crypto currency. The Bitcoin software for online peer-to-peer banking was developed by Satoshi Nakamoto (中本哲史) and it is available at bitcoin.org.

Basically every block of Bitcoins is a solution of a cryptographic equation. I.e. instead of a scarce metal (such as gold), the currency uses hard computational problems as a proof of work. Standard asymmetric encryption is used to digitally sign transactions. The proof of work is also used to cryptographically strengthen a chain of transactions in order to prevent double-spending of coins (see Bitcoin publication for more details). The only known attack requires the attacker to have more computational power at his/her disposal than the entire network of Bitcoin clients.

There’s a number of organisations and shops which already accept Bitcoins (see Bitcoin.it for a list of sites that accept Bitcoins). Furthermore there are several traders which will exchange Bitcoins for US dollars, Euros, and other currencies (there is an early review of Bitcoin exchanges). According to Bitcoin Charts the exchange rate currently is around 0.9 USD/BTC

Update:

There are concerns that Bitcoin will suffer a deflationary spiral because the total amount of currency is limited. Of course if this is a real problem one could start a new peer-to-peer currency with a built-in controlled inflation.

Update:

Here’s a nice video giving a quick introduction to Bitcoin.

See Also:

Jan Wedekind

Background Replacement

Camera Calibration

Broken Tonight

Histogram-based classification

Bitcoin peer-to-peer currency