Stretchable Models Code

Download

This project is hosted on github. If you have and want to use git, run the command

$ git clone git@github.com:bensapp/Stretchable-Models-for-Motion-Parsing.git

from a console. Alternatively, download a static ZIP'ed copy from

https://github.com/bensapp/Stretchable-Models-for-Motion-Parsing/zipball/master

or, just browse the source.

Version 1.0, October 4, 2011.

Usage

To run on a new image, see demo_stretchable_model.m. If everything goes according to plan, you should get the following output on the example clip provided:

The required input is a sequence of video filenames and a list of torso detection boxes; see the example input in demo_stretchable_model.m. The code then computes a variety of features, runs the Cascaded Pictorial Structures code, and performs the simplest, most efficient of our tree ensemble inference methods from our paper

Parsing Human Motion with Stretchable Models. Benjamin Sapp, David Weiss and Ben Taskar. CVPR. June 2011.

Mex compilation issues

The code makes heavy use of mex and also requires a slightly modified version of OpenCV (easier read/write and status updates in the boosting library). I have included all source code, but have compiled binaries only for 64 bit linux (hence, *.mexa64 files). Please ask if you have any difficulties compiling any of the mex files in any other environment. Also, if you DO compile binaries in other environments (e.g., mexmaci, mexglx, windows), please let me know and I can add them to the download to make it easier for others.

If you're running in a 64 bit linux environment, make sure your LD_LIBRARY_PATH is set to correctly use the OpenCV libs included in stretchable-models/cps/thirdparty/OpenCV-2.0.0/lib/

You can check this inside matlab using the command '!ldd mex_file.mexa64', for example.

Most common issue:

Invalid MEX-file '/some/path/cps/utils/mex_opencv_boosting.mexa64': libcxcore.so.4: cannot open shared object file: No such file or directory

Fix: Make sure this directory is in your library path: cascaded_pictorial_structures/thirdparty/OpenCV-2.0.0/lib/ and in particular, that libcxcore.so.4 is in that directory.

You can check if mex_opencv_boosting.mexa64 knows where to locate libcxcore.so.4 by running

>> !ldd utils/mex_opencv_boosting.mexa64 | grep cxcore inside matlab. If it's not found, run something like

>> setenv('LD_LIBRARY_PATH', [getenv('LD_LIBRARY_PATH') ';thirdparty/OpenCV-2.0.0/lib/']);

Then, you should see something like

>> !ldd utils/mex_opencv_boosting.mexa64 | grep cxcore

	libcxcore.so.4 => thirdparty/OpenCV-2.0.0/lib/libcxcore.so.4 (0x00007f5c37aad000)

Processing Pipeline & Timing

For a video sequence, here is a breakdown of the steps and time each step takes. NOTE, most of these steps is trivially parallelizable at the level of video frames, noted in the code comments.

• Run Cascaded PS (including HoG detectors, pb, ncut and associated features): ~5 minutes / frame
• Compute optical flow: 15-20 seconds / frame
• Compute additional features, collect all features and discretize: 1 minute / frame and 90MB / frame
• Inference: 1 second / frame

This was evaluated on this machine:
Linux 2.6.31.5-0.1-default x86_64 GNU/Linux
Intel(R) Xeon(R) CPU E5450 @ 3.00GHz (w/ 8 cpus)
MATLAB R2010b

License

MIT