Conclusion
In this project we implemented parallelized versions of DeepFlow algorithm. The main work that we achieved are the follows:
- We studied optical flow algorithms and analyzed a state-of-the-art implementation of optical flow algorithm (DeepFlow). We further identified the computational-heavy parts in this algorithm and implemented the parallelizations.
- At image level, we implemented Red-Black SOR method, a parallelizable method to replace the original SOR which is not parallelizable and their accuracy and convergence don't differ much.
- At video level, we implemented a robust MapReduce pipeline for image processing on Hadoop platform using HDFS to utilize the power of clusters.
- We tried other various kinds of parallelization algorithms and programming models that we studied this semester and did comprehensive evaluation of these methods. We learned that OpenMP and MapReduce achieve significant speedup for our task, while MPI and OpenACC might not be suitable for our case.
- We created a high-performance hybrid model based on OpenMP and MapReduce for video processing.
- We built two applications, namely fake slow motion video generation and video stylization with our parallelized DeepFlow algorithm.
Insight: Computer vision is one of the most important area where big compute and big data techniques can be heavily used. This is the reason why we choose a problem in this area as the topic of our final project. Our work shows that OpenMP and MapReduce can make significant improvement in optical flow computation. We also learnt GPU acceleration and MPI are not always helpful. Although both of them bring in more computation power, they might add even more overheads.
Future work: There are still many open problems in optical flow regarding how to improve the latency and the accuracy. Several future work includes making MPI more efficient in our use case, and applying parallelization in other state-of-the-art implementations of optical flow estimations.
We also hope our work can give some insights to reasearchers and help them better incoroperate the idea of big compute and big data into the solution of many related computer vision problems.
Citations
- DeepFlow: Large displacement optical flow with deep matching Philippe Weinzaepfel, Jerome Revaud, Zaid Harchaoui and Cordelia Schmid, Proc. ICCV'13, December, 2013
- DeepMatching: Hierarchical Deformable Dense Matching Jerome Revaud, Philippe Weinzaepfel, Zaid Harchaoui and Cordelia Schmid, IJCV 2016.
- B. K. Horn and B. G. Schunck. Determining optical flow. Proc. SPIE 0281, Techniques and Applications of Image Understanding, 1981.
- B. D. Lucas and T. Kanade. An iterative image registration technique with an application to stereo vision. In IJCAI, volume 81, pages 674–679, 1981