MEncoder options

Encoding DVDs with MEncoder

Here is an example on how to encode a DVD using MEncoder.

mencoder dvd://1 -o test.avi -ss 120 -endpos 10 -alang en -slang en -aid 130 -psprobe 1000000 -vf crop=720:446:0:66 -aspect 1.78 -af volume=15 -ovc x264 -x264encopts crf=21:threads=2 -oac mp3lame -lameopts q=2

You can get hints for the audio id, supported languages, aspect ratio, and other values by playing the DVD with mplayer dvd://1 and inspecting the resulting console output.

• dvd://1: read the main track of the DVD. Sometimes the main video is on another track (e.g. dvd://3, dvd://5, dvd://81, …).
• -o test.avi: selects the output file and AVI as container format. You could also specify an MP4 file here.
• -ss 120 -endpos 10: this tells the encoder to skip the first 120 seconds and then stop after encoding 10 seconds of video. This is used initially to test on a small representative part of the video that the encoding options are correct. Afterwards you can drop these options.
• -alang en: choose the English audio track.
• -slang en: add the English subtitles to the video.
• -aid 130: choose the audio track with id 130. Specifying this is not necessary in most cases.
• -psprobe 1000000: use this option to force MEncoder to extend the search for the audio stream. Only necessary if MEncoder says no audio stream found.
• -vf crop=720:446:0:66: you can use this to crop of black borders.
• -aspect 1.78: often it is necessary to set the pixel aspect ratio of the output video.
• -af volume=15: with many DVDs it is necessary to boost the audio volume.
• -ovc x264 -x264encopts crf=21:threads=2: this selects the H.264 video encoder and configures it.
• -oac mp3lame -lameopts q=2: this selects the MP3 audio encoder and configures it. You can also use -oac faac -faacopts quality=300 if you prefer to use the AAC codec.

Once you have found the right options, rerun the command without the -ss and -endpos option to encode the full video.

See also:

• MPlayer

Arduino hello world

Using an Arduino board with a GNU/Linux personal computer has become pretty straightforward. The following command will install the required software:

sudo aptitude install arduino-mk

In order to get the file-system permissions to use the USB-serial it is necessary to add the current user to the dialout group. It is necessary to log out and back in after this.

usermod $USER -a -G dialout

For the Arduino Diecimila you then create a Makefile with the following content (see /usr/share/arduino/hardware/arduino/boards.txt for other supported boards).

ARDUINO_DIR = /usr/share/arduino
BOARD_TAG    = diecimila
ARDUINO_PORT = /dev/ttyUSB0
ARDUINO_LIBS =
include /usr/share/arduino/Arduino.mk

Make sure that /dev/ttyUSB0 is the right device! You can do this by inspecting the output of dmesg after plugging in the Arduino device.

Now you can create a sketch file (ino file extension) for programming the micro controller. E.g. a file Blink.ino with the following content:

int led = 13;

void setup() {
  pinMode(led, OUTPUT);
}

void loop() {
  digitalWrite(led, HIGH);
  delay(50);
  digitalWrite(led, LOW);
  delay(50);
}

Finally compile and upload the code:

make
make upload

The source is available on Github: github.com/wedesoft/arduino-hello-world

Update:

It is also possible to switch the LED according to instructions send over the USB serial channel. The following program facilitates this:

int led = 13;

void setup() {
  Serial.begin(9600);
  pinMode(led, OUTPUT);
}

void loop() {
  char c;
  if (Serial.available()) {
    c = Serial.read();
    if (c == '1') {
      digitalWrite(led, HIGH);
      Serial.write("LED on\r\n");
    } else if (c == '0') {
      digitalWrite(led, LOW);
      Serial.write("LED off\r\n");
    } else
      Serial.write("Switch LED on/off using keys '1' and '0'\r\n");
  };
}

After opening screen as a serial terminal, the LED can be controlled using keys ‘1’ and ‘0’:

screen /dev/ttyUSB0 9600

Exit with Ctrl-A \

See also:

• Uploading a Sketch from the command line
• Control LEDs with lower level IO
• Three Alternatives to Arduino’s Serial Monitor
• Arduino ProtoShield

Graph colouring

Currently I am looking into register allocation using graph colouring (also see Chaitin’s paper on the subject). The graph colouring problem also occurs when trying to colour countries in a political world map where adjacent countries must have different colours. I thought it would be interesting to try a minimal implementation in Scheme. Feel free to suggest improvements in the comment section below .

A (undirected) graph is usually represented by a list of nodes (vertices) and a list of edges. If there are no insular nodes, a list of edges is sufficient. In Scheme (here GNU Guile implementation) one can do this as follows.

'((b . a) (a . c) (d . c))

The graph can be visualised on the command line using Graphviz and ImageMagick as follows.

echo 'graph test { b -- a; a -- c; d -- c; }' | dot -Tpng | display -

The following helper function graphviz uses a system call to display a graph from within the REPL.

(define (dot graph colors)
  (apply string-append
         (append (list "graph g {")
                 (map (lambda (color) (format #f " ~a [style=filled, fillcolor=~a];" (car color) (cdr color))) colors)
                 (map (lambda (edge) (format #f " ~a -- ~a;" (car edge) (cdr edge))) graph)
                 (list " }"))))
(define (graphviz graph colors) (system (format #f "echo '~a' | dot -Tpng | display -" (dot graph colors))))
(graphviz '((b . a) (a . c) (d . c)) '())

One can get the nodes of the graph by extracting all elements and suppressing any duplicates. The definition of delete-duplicates is part of SRFI-1 (Scheme Request for Implementation 1).

(use-modules (srfi srfi-1))
(define (nodes graph) (delete-duplicates (append (map car graph) (map cdr graph))))
(nodes '((b . a) (a . c) (d . c)))
; (b a d c)

The adjacency list of a node is simply the list of nodes of the sub-graph obtained by filtering for edges connecting to this node.

(use-modules (ice-9 curried-definitions))
(define ((has-node? node) edge) (or (eq? (car edge) node) (eq? (cdr edge) node)))
(define (adjacent graph node) (nodes (filter (has-node? node) graph)))
(adjacent '((b . a) (a . c) (d . c)) 'c)
; (a d c)

Chaitin’s graph coloring algorithm works by successively removing nodes with a low adjacency count from the graph. Removing a node from our graph can be done as follows.

(define (remove-node graph node) (filter (compose not (has-node? node)) graph))
(remove-node '((b . a) (a . c) (d . c)) 'c)
; ((b . a))

Using the argument of the minimum one can determine the node with lowest adjacency count.

(define (argmin fun lst)
  (let* [(vals   (map fun lst))
         (minval (apply min vals))]
    (list-ref lst (- (length lst) (length (member minval vals))))))

Now one can recursively remove the node with lowest adjacency count and then assign colours starting with the last node and working backwards. If an adjacent node has a colour already, another colour must be used.

(use-modules (srfi srfi-26))
(define (assign-colors graph nodes colors)
  (if (null? nodes) '()
    (let* [(target    (argmin (compose length (cut adjacent graph <>)) nodes))
           (coloring  (assign-colors (remove-node graph target) (delete target nodes) colors))
           (blocked   (map (cut assq-ref coloring <>) (adjacent graph target)))
           (available (lset-difference eq? colors blocked))]
      (cons (cons target (car available)) coloring))))
(define (coloring graph colors) (assign-colors graph (nodes graph) colors))
(coloring '((b . a) (a . c) (d . c)) '(red green blue))
; ((b . red) (a . green) (d . green) (c . red))
(let [(graph '((b . a) (a . c) (d . c)))] (graphviz graph (coloring graph '(red green blue))))

And here is an example of coloring a graph with a few more nodes.

(let [(graph '((run . intr)
               (intr . runbl)
               (runbl . run)
               (run . kernel)
               (kernel . zombie)
               (kernel . sleep)
               (kernel . runmem)
               (sleep . swap)
               (swap . runswap)
               (runswap . new)
               (runswap . runmem)
               (new . runmem)
               (sleep . runmem)))]
  (graphviz graph (coloring graph '(red green blue yellow))))

OOP with GNU Guile and GOOPS

Having worked with the Ruby programming language for many years I started to get interested in Scheme. The Scheme programming language has a small and powerful core supporting closures, hygienic macros, first-class continuations, and of course the famous interpreter functions eval and apply.

However one thing I don’t like about Scheme is that there are different function names for each type of arguments. E.g. adding numbers is done with +, adding lists is done with append, and adding strings is done with string-append.

(+ 2 3)
; 5
(append '(1) '(2 3))
; (1 2 3)
(string-append "a" "bc")
; "abc"

The same program would be much less verbose if + was extended to work with strings and lists, too:

(+ 2 3)
; 5
(+ '(1) '(2 3))
; (1 2 3)
(+ "a" "bc")
; "abc"

GOOPS: The Guile extension for object-oriented programming

It turns out that GNU Guile (the Scheme interpreter of the GNU project) has a mature extension which facilitates this. GOOPS is inspired by the Common Lisp Object System (CLOS). GOOPS not only provides polymorphic methods, it even lets you replace existing functions with polymorphic ones:

(use-modules (oop goops))
(define-method (+ (a <list>) (b <list>)) (append a b))
(define-method (+ (a <string>) (b <string>)) (string-append a b))
(+ 2 3)
; 5
(+ '(1) '(2 3))
; (1 2 3)
(+ "a" "bc")
; "abc"

The define-class method is the normal way to define a class. GOOPS requires you to define the instance attributes when defining the class. The following example defines a class <a> with attribute x and a + method. The make method is used to create the instance a of the class <a>.

(use-modules (oop goops))
(define-class <a> ()
  (x #:init-value 0 #:init-keyword #:x #:getter get-x))
(define-method (+ (u <a>) (v <a>))
  (make <a> #:x (+ (get-x u) (get-x v))))
<a>
; #<<class> <a> 2103c30>
(define a (make <a> #:x 123))
a
; #<<a> 22a2440>
(get-x a)
; 123
(+ a a)
; #<<a> 23713e0>
(get-x (+ a a))
; 246

Constructors and the next method

One can define a shorthand for instantiating objects. E.g. one can define a method which sets the attribute #:x to the argument multiplied by two.

(use-modules (oop goops))
(define-class <a> ()
  (x #:init-value 0 #:init-keyword #:x #:getter get-x))
(define (make-a x2) (make <a> #:x (* 2 x2)))
(define a2 (make-a 123))
; #<<a> 1689740>
(get-x a2)
; 246

IMHO it is better though to define a modified constructor directly. This is more involved but also possible.

(use-modules (oop goops) (ice-9 optargs))
(define-class <a> ()
  (x #:init-value 0 #:init-keyword #:x #:getter get-x))
(define-method (initialize (self <a>) initargs)
  (let-keywords initargs #f (x2)
    (next-method self (list #:x (* 2 x2)))))
(define a3 (make <a> #:x2 123))
(get-x a3)
; 246

Note the call to next-method. This essentially calls the next less specialized method for that generic function. Here is another example using an inheriting class <b> to illustrate the concept.

(use-modules (oop goops))
(define-class <a> ())
(define-class <b> (<a>))
(define-method (test (self <a>)) "a")
(define-method (test (self <b>)) (string-append (next-method) "b"))
(test (make <a>))
; "a"
(test (make <b>))
; "ab"

Metaclasses

Note that GOOPS does not implicitly create a metaclass. The following example shows how to define a metaclass <m<c>> with a class method.

(use-modules (oop goops))
(define-class <m<c>> (<class>))
(define-method (test <m<c>>) "m")
(define-class <c> ()
  (x #:init-keyword #:x #:getter get-x) #:metaclass <m<c>>)
(define o (make <c> #:x 5))
o
; #<<c> 2825d40>
(test (class-of o))
; "m"

Inspection

One can also use GOOPS to change the way how objects get displayed and what the REPL writes to the terminal. E.g. one can define the method write for the class <a> to change the way the Guile REPL displays objects of that class.

(use-modules (oop goops))
(define-class <a> ()
  (x #:init-value 0 #:init-keyword #:x #:getter get-x))
(define a (make <a> #:x 5))
a
; #<<a> 2c64140>
(define-method (write (self <a>) port)
  (format port "(make <a> #:x ~a)" (get-x self)))
a
; (make <a> #:x 5)

Furthermore one can implement the method display to define the way objects get displayed in formatted output.

(define-method (display (self <a>) port)
  (format port "*~a*" (get-x self)))
(format #t "~a~&" a)
; *5*

Conclusion

I have now used GOOPS for a little while. Coming from Ruby there are a few gotchas when using GOOPS and Guile’s module system. For example one needs to use a #:re-export statement when using a module to replace the core binding for the + operator.

All in all GOOPS makes quite a mature impression and I think it makes Scheme much more amenable to developers who are used to thinking in terms of objects and classes.

To quote Guile’s foreign function interface documentation:

The more one hacks in Scheme, the more one realizes that there are actually two computational worlds: one which is warm and alive, that land of parentheses, and one cold and dead, the land of C and its ilk.

Any comments and suggestions are welcome

Further remarks

If necessary it is also possible to create objects, classes, and metaclasses dynamically. The following example instantiates the object v of class <v> of metaclass <m<v>>. Furthermore the generic test is implemented for <v>.

(use-modules (oop goops))
(define-generic test)
(define <m<v>> (make <class>
                     #:dsupers `(,<class>)
                     #:slots '()
                     #:name "<m<v>>"))
(define <v> (make <m<v>>
                  #:dsupers `(,<object>)
                  #:slots '()
                  #:name "<v>"))
(add-method! test (make <method>
                        #:specializers `(,<v>)
                        #:procedure (lambda (self) "v")))
(define v (make <v>))
v
; #<<v> 2d5a750>
(test v)
; "v"

Update:

I tweeted about the article and submitted it to Hackernews and the GUILE user list.

I have reduced the use of slot-ref during a discussion on the GUILE user list.

See also:

• GNU Guile
• GOOPS
  • Generic functions and methods example
  • Guile object properties
• Structure and Interpretation of Computer Programs
• The Little Schemer
• The Seasoned Schemer
• The Reasoned Schemer
• COOPS object system for Chicken Scheme
• Clojure multi-methods

Implement an Interpreter using Bison, Flex, and Automake

This is a small example on how to implement an interpreter using Flex, Bison (formerly known as Yacc), and the Autotools. The example is based on Ben Reichard’s course material.

Dependencies

First you need to install the C compiler, the lexer, and the LALR parser generator for this project. It also helps to install the readline wrapper utility.

sudo aptitude install build-essential bison flex rlwrap

Build system

You need to create the file Makefile.dist with the following content.

all:
	aclocal
	autoheader
	libtoolize --force
	automake -a --foreign
	autoconf

configure:
	./configure

Then you create the file configure.ac with the following content.

AC_INIT(aclocal.m4)
AM_INIT_AUTOMAKE([calculator], [1.0.0])
AC_CONFIG_MACRO_DIR([m4])
AC_PROG_CC
AC_PROG_INSTALL
AC_PROG_LN_S
AC_PROG_YACC
AC_PROG_LEX
AM_PROG_LIBTOOL
AM_CONFIG_HEADER(config.h)
AC_CHECK_HEADERS([stdio.h])
AC_OUTPUT(Makefile)

Finally you create the file Makefile.am with the following content.

SUFFIXES = .c .h .y .l

ACLOCAL_AMFLAGS = -I m4

AM_YFLAGS = -d

bin_PROGRAMS = calculator

calculator_SOURCES = calculator.c calc_bison.y calc_flex.l
calculator_LDFLAGS = 
calculator_LDADD =

noinst_HEADERS = calculator.h

BUILT_SOURCES = calc_bison.h

EXTRA_DIST = Makefile.dist configure.ac

CLEANFILES = *~

MAINTAINERCLEANFILES = aclocal.m4 config.guess config.sub configure \
	install-sh ltmain.sh Makefile.in missing mkinstalldirs stamp-h.in \
	libtool config.cache config.h config.h.in acinclude.m4 depcomp \
	ylwrap

maintainer-clean-local:
	-rm -rf m4

This completes the setup of the build system.

Implementation

First create the file calc_bison.y with the implementation of the parser.

%{
#include <stdio.h>

void yyerror(char *s) {
  fprintf(stderr, "%s\n", s);
}

int sym[26];
%}

%union {
  int number;
  int var;
};

%type <number> expression
%token <var> VAR
%token <number> NUMBER

%%

start: expression '\n' { printf("%d\n\n", $1); } start
     | /* NULL */
     ;

expression: NUMBER
          | VAR                       { $$ = sym[$1]; }
          | '-' expression            { $$ = -$2; }
          | expression '+' expression { $$ = $1 + $3; }
          | expression '-' expression { $$ = $1 - $3; }
          | expression '*' expression { $$ = $1 * $3; }
          | '(' expression ')'        { $$ = $2; }
          | VAR '=' expression        { sym[$1] = $3; $$ = $3; }
          ;

Then create the file calc_flex.l with the implementation of the lexer (tokenizer).

%{
#include "calc_bison.h"
%}

%option noyywrap
%option always-interactive

%%

[0-9]+     { yylval.number = atoi(yytext); return NUMBER; }

[a-z]      { yylval.var = *yytext - 'a'; return VAR; }

[-+()*\n=] { return *yytext; }

[ \t]      ;

.          yyerror("Unknown character");

%%

Then create the header file calculator.h for the parser.

#ifndef CALCULATOR_H
#define CALCULATOR_H

int yyparse(void);
extern int sym[26];

#endif

Finally create the file calculator.c with the main program.

#include "calculator.h"

int main(void)
{
  int i;
  for (i=0; i<26; i++) sym[i] = 0;
  yyparse();
  return 0;
}

Compiling and running it

Above program is built using the following steps.

make -f Makefile.dist
./configure
make

You can run the calculator as follows.

./calculator

Alternatively you can run the interpreter with rlwrap to get command line history.

rlwrap ./calculator

Here is a sample session using the calculator program.

1
1

1 + 2
3

a = 1 + 2
3

b = a * 3
9

(1 + 2) * 3
9

(x = b) + 1
10

x
9

The code is also available on Github. You can get a copy using Git:

git clone https://github.com/wedesoft/calculator.git

Enjoy!

See also

• OMeta (Parsing Expression Grammars)
• Treetop (PEG parser for Ruby)
• Sample LALR(1) parser implemented in Racket
• Earley parser
• Lexers in Racket