Author(s):
Barron, Jonathan T.
Abstract:
We present a generalization of the Cauchy/Lorentzian, Geman-McClure, Welsch/Leclerc, generalized Charbonnier, Charbonnier/pseudo-Huber/L1-L2, and L2 loss functions. By introducing robustness as a continuous parameter, our loss function allows algorithms built around robust loss minimization to be generalized, which improves performance on basic vision tasks such as registration and clustering. Interpreting our loss as the negative log of a univariate density yields a general probability distribution that includes normal and Cauchy distributions as special cases. This probabilistic interpretation enables the training of neural networks in which the robustness of the loss automatically adapts itself during training, which improves performance on learning-based tasks such as generative image synthesis and unsupervised monocular depth estimation, without requiring any manual parameter tuning.
Document:
https://doi.org/10.1109/CVPR.2019.00446
References:
1. Nasir Ahmed, T_ Natarajan and Kamisetty R Rao, “Discrete cosine transform”, IEEE Transactions on Computers, 1974. Show Context View Article Full Text: PDF (571KB) Google Scholar
2. Michael J Black and Paul Anandan, “The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields”, CVIU, 1996. Show Context CrossRef Google Scholar
3. Michael J. Black and Anand Rangarajan, “On the unification of line processes outlier rejection and robust statistics with applications in early vision”, IJCV, 1996. Show Context CrossRef Google Scholar
4. Andrew Blake and Andrew Zisserman, Visual Reconstruction, MIT Press, 1987. Show Context CrossRef Google Scholar
5. Pierre Charbonnier, Laure Blanc-Feraud, Gilles Aubert and Michel Barlaud, “Two deterministic half-quadratic regularization algorithms for computed imaging”, ICIP, 1994. Show Context View Article Full Text: PDF (575KB) Google Scholar
6. Qifeng Chen and Vladlen Koltun, “Fast mrf optimization with application to depth reconstruction”, CVPR, 2014. Show Context View Article Full Text: PDF (593KB) Google Scholar
7. Albert Cohen, Ingrid Daubechies and J-C Feauveau, “Biorthogonal bases of compactly supported wavelets”, Communications on pure and applied mathematics, 1992. Show Context CrossRef Google Scholar
8. Alexey Dosovitskiy and Thomas Brox, “Generating images with perceptual similarity metrics based on deep networks”, NIPS, 2016. Show Context Google Scholar 9. David J. Field, “Relations between the statistics of natural images and the response properties of cortical cells”, JOSA A, 1987. Show Context CrossRef Google Scholar
10. John Flynn, Ivan Neulander, James Philbin and Noah Snavely. Deepstereo, “Learning to predict new views from the world’s imagery”, CVPR, 2016. Show Context Google Scholar
11. Ravi Garg, BG Vijay Kumar, Gustavo Carneiro and Ian Reid, “Unsupervised cnn for single view depth estimation: Geometry to the rescue”, ECCV, 2016. Show Context Google Scholar
12. Andreas Geiger, Philip Lenz and Raquel Urtasun, “Are we ready for autonomous drivingƒ the kitti vision benchmark suite”, CVPR, 2012. Show Context View Article Full Text: PDF (339KB) Google Scholar
13. Stuart Geman and Donald E. McClure, “Bayesian image analysis: An application to single photon emission tomography”, Proceedings of the American Statistical Association, 1985. Show Context Google Scholar
14. Clement Godard, Oisin Mac Aodha and Gabriel J. Brostow, “Unsupervised monocular depth estimation with left- right consistency”, CVPR, 2017. Show Context View Article Full Text: PDF (1061KB) Google Scholar
15. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, et al., “Generative adversarial nets”, NIPS, 2014. Show Context Google Scholar
16. Frank R. Hampel, Elvezio M. Ronchetti, Peter J. Rousseeuw and Werner A. Stahel, Robust Statistics: The Approach Based on Influence Functions, Wiley, 1986. Show Context Google Scholar
17. Trevor Hastie, Robert Tibshirani and Martin Wainwright, Statistical Learning with Sparsity: The Lasso and Generalizations, Chapman and Hall/CRC, 2015. Show Context CrossRef Google Scholar
18. Peter J. Huber, “Robust estimation of a location parameter”, Annals of Mathematical Statistics, 1964. Show Context CrossRef Google Scholar
19. Peter J. Huber, Robust Statistics, Wiley, 1981. Show Context CrossRef Google Scholar
20. John E. Dennis Jr. and Roy E. Welsch, “Techniques for nonlinear least squares and robust regression”, Communications in Statistics-simulation and Computation, 1978. Show Context CrossRef Google Scholar
21. Diederik P. Kingma and Jimmy Ba, “Adam: A method for stochastic optimization”, ICLR, 2015. Show Context Google Scholar
22. Diederik P. Kingma and Max Welling, “Auto-encoding variational bayes”, ICLR, 2014. Show Context Google Scholar
23. Philipp Krahenbuhl and Vladlen Koltun, “Efficient nonlocal regularization for optical flow”, ECCV, 2012. Show Context CrossRef Google Scholar
24. Anders Boesen Lindbo Larsen, Søren Kaae Sønderby, Hugo Larochelle and Ole Winther, “Autoencoding beyond pixels using a learned similarity metric”, ICML, 2016. Show Context Google Scholar
25. Yvan G Leclerc, “Constructing simple stable descriptions for image partitioning”, IJCV, 1989. Show Context CrossRef Google Scholar
26. Ziwei Liu, Ping Luo, Xiaogang Wang and Xiaoou Tang, “Deep learning face attributes in the wild”, ICCV, 2015. Show Context View Article Full Text: PDF (1187KB) Google Scholar
27. Stephane Mallat, “A theory for multiresolution signal decomposition: The wavelet representation”, TPAMI, 1989. Show Context View Article Full Text: PDF (1797KB) Google Scholar
28. Saralees Nadarajah, “A generalized normal distribution”, Journal of Applied Statistics, 2005. Show Context CrossRef Google Scholar
29. Javier Portilla, Vasily Strela, Martin J. Wainwright and Eero P. Simoncelli, “Image denoising using scale mixtures of gaussians in the wavelet domain”, IEEE TIP, 2003. Show Context View Article Full Text: PDF (1087KB) Google Scholar
30. Danilo Jimenez Rezende, Shakir Mohamed and Daan Wierstra, “Stochastic backpropagation and approximate inference in deep generative models”, ICML, 2014. Show Context Google Scholar
31. Sohil Atul Shah and Vladlen Koltun, “Robust continuous clustering”, PNAS, 2017. Show Context CrossRef Google Scholar
32. Jianbo Shi and Jitendra Malik, “Normalized cuts and image segmentation”, TPAMI, 2000. Google Scholar
33. M Th Subbotin, “On the law of frequency of error”, Matematicheskii Sbornik, 1923. Show Context Google Scholar
34. Deqing Sun, Stefan Roth and Michael J. Black, “Secrets of optical flow estimation and their principles”, CVPR, 2010. Show Context View Article Full Text: PDF (800KB) Google Scholar
35. Rein van den Boomgaard and Joost van de Weijer, “On the equivalence of local-mode finding robust estimation and mean-shift analysis as used in early vision tasks”, ICPR, 2002. Show Context View Article Full Text: PDF (279KB) Google Scholar
36. Yi Yang, Dong Xu, Feiping Nie, Shuicheng Yan and Yueting Zhuang, “Image clustering using local discriminant models and global integration”, TIP, 2010. View Article Full Text: PDF (992KB) Google Scholar
37. Christopher Zach, “Robust bundle adjustment revisited”, ECCV, 2014. Show Context CrossRef Google Scholar
38. Wei Zhang, Deli Zhao and Xiaogang Wang, “Agglomerative clustering via maximum incremental path integral”, Pattern Recognition, 2013. CrossRef Google Scholar
39. Zhengyou Zhang, “Parameter estimation techniques: A tutorial with application to conic fitting”, 1995. Show Context Google Scholar
40. Qian-Yi Zhou, Jaesik Park and Vladlen Koltun, “Fast global registration”, ECCV, 2016. Show Context Google Scholar
41. Tinghui Zhou, Matthew Brown, Noah Snavely and David G. Lowe, “Unsupervised learning of depth and ego-motion from video”, CVPR, 2017. Show Context View Article Full Text: PDF (3068KB) Google Scholar