Stretchable Models CodeEdit
This project is hosted on github. If you have and want to use git, run the command
$ git clone firstname.lastname@example.org:bensapp/Stretchable-Models-for-Motion-Parsing.git
from a console. Alternatively, download a static ZIP'ed copy from
or, just browse the source.
Version 1.0, October 4, 2011.
To run on a new image, see
demo_stretchable_model.m. If everything goes according to plan, you should get the following output on the example clip provided:
The required input is a sequence of video filenames and a list of torso detection boxes; see the example input in
demo_stretchable_model.m. The code then computes a variety of features, runs the Cascaded Pictorial Structures code, and performs the simplest, most efficient of our tree ensemble inference methods from our paper
Parsing Human Motion with Stretchable Models. Benjamin Sapp, David Weiss and Ben Taskar. CVPR. June 2011.
Mex compilation issues
The code makes heavy use of mex and also requires a slightly modified version of OpenCV (easier read/write and status updates in the boosting library). I have included all source code, but have compiled binaries only for 64 bit linux (hence, *.mexa64 files). Please ask if you have any difficulties compiling any of the mex files in any other environment. Also, if you DO compile binaries in other environments (e.g., mexmaci, mexglx, windows), please let me know and I can add them to the download to make it easier for others.
If you're running in a 64 bit linux environment, make sure your LD_LIBRARY_PATH is set to correctly use the OpenCV libs included in
You can check this inside matlab using the command '!ldd mex_file.mexa64', for example.
Most common issue:
Invalid MEX-file '/some/path/cps/utils/mex_opencv_boosting.mexa64': libcxcore.so.4: cannot open shared object file: No such file or directory
Fix: Make sure this directory is in your library path: cascaded_pictorial_structures/thirdparty/OpenCV-2.0.0/lib/ and in particular, that libcxcore.so.4 is in that directory.
You can check if
mex_opencv_boosting.mexa64 knows where to locate
libcxcore.so.4 by running
>> !ldd utils/mex_opencv_boosting.mexa64 | grep cxcore inside matlab. If it's not found, run something like
>> setenv('LD_LIBRARY_PATH', [getenv('LD_LIBRARY_PATH') ';thirdparty/OpenCV-2.0.0/lib/']);
Then, you should see something like
>> !ldd utils/mex_opencv_boosting.mexa64 | grep cxcore
libcxcore.so.4 => thirdparty/OpenCV-2.0.0/lib/libcxcore.so.4 (0x00007f5c37aad000)
Processing Pipeline & Timing
For a video sequence, here is a breakdown of the steps and time each step takes. NOTE, most of these steps is trivially parallelizable at the level of video frames, noted in the code comments.
- Run Cascaded PS (including HoG detectors, pb, ncut and associated features): ~5 minutes / frame
- Compute optical flow: 15-20 seconds / frame
- Compute additional features, collect all features and discretize: 1 minute / frame and 90MB / frame
- Inference: 1 second / frame
This was evaluated on this machine:
Linux 220.127.116.11-0.1-default x86_64 GNU/Linux
Intel(R) Xeon(R) CPU E5450 @ 3.00GHz (w/ 8 cpus)