An Application of Kolmogorov's Superposition Theorem to Function Reconstruction in Higher Dimensions

Jürgen Braun



In this thesis we present a Regularization Network approach to reconstruct a continuous function ƒ:[0,1]nR from its function values ƒ(xj) on discrete data points xj, j=1,…,P. The ansatz is based on a new constructive version of Kolmogorov's superposition theorem.

Typically, the numerical solution of mathematical problems underlies the so--called curse of dimensionality. This term describes the exponential dependency of the involved numerical costs on the dimensionality n. To circumvent the curse at least to some extend, typically higher regularity assumptions on the function ƒ are made which however are unrealistic in most cases. Therefore, we employ a representation of the function as superposition of one--dimensional functions which does not require higher smoothness assumptions on ƒ than continuity. To this end, a constructive version of Kolmogorov's superposition theorem which is based on D. Sprecher is adapted in such a manner that one single outer function Φ and a universal inner function ψ suffice to represent the function ƒ. Here, ψ is the extension of a function which was defined by M. Köppen on a dense subset of the real line. The proofs of existence, continuity, and monotonicity are presented in this thesis. To compute the outer function Φ, we adapt a constructive algorithm by Sprecher such that in each iteration step, depending on ƒ, an element of a sequence of univariate functions { Φr}r is computed. It will be shown that this sequence converges to a continuous limit Φ:RR. This constructively proves Kolmogorov's superposition theorem with a single outer and inner function.

Due to the fact that the numerical complexity to compute the outer function Φ by the algorithm grows exponentially with the dimensionality, we alternatively present a Regularization Network approach which is based on this representation. Here, the outer function is computed from discrete function samples (xj,ƒ(xj)), j=1,…,P. The model to reconstruct ƒ will be introduced in two steps. First, the outer function Φ is represented in a finite basis with unknown coefficients which are then determined by a variational formulation, i.e. by the minimization of a regularized empirical error functional. A detailed numerical analysis of this model shows that the dimensionality of ƒ is transformed by Kolmogorov's representation into oscillations of Φ. Thus, the use of locally supported basis functions leads to an exponential growth of the complexity since the spatial mesh resolution has to resolve the strong oscillations. Furthermore, a numerical analysis of the Fourier transform of Φ shows that the locations of the relevant frequencies in Fourier space can be determined a priori and are independent of ƒ. It also reveals a product structure of the outer function and directly motivates the definition of the final model. Therefore, Φ is replaced in the second step by a product of functions for which each factor is expanded in a Fourier basis with appropriate frequency numbers. Again, the coefficients in the expansions are determined by the minimization of a regularized empirical error functional.

For both models, the underlying approximation spaces are developed by means of reproducing kernel Hilbert spaces and the corresponding norms are the respective regularization terms in the empirical error functionals. Thus, both approaches can be interpreted as Regularization Networks. However, it is important to note that the error functional for the second model is not convex and that nonlinear minimizers have to be used for the computation of the model parameters. A detailed numerical analysis of the product model shows that it is capable of reconstructing functions which depend on up to ten variables.

Complete version (4 MB) Hier können Sie den Adobe Acrobat Reader downloaden


© Universitäts- und Landesbibliothek Bonn | Published: 01.12.2009