Classical versus Data-Driven Approximation of a Damped Oscillatory Function: Convergence Rates, Uniform Error, and Extrapolation Behaviour
DOI:
https://doi.org/10.65405/jzkq8q88Keywords:
function approximation; uniform error; convergence rate; Chebyshev polynomials; cubic-spline interpolation; random forest; multilayer perceptron; extrapolationAbstract
We study the approximation of the entire function on from 5,000 exact, equally spaced samples. A random forest and two multilayer perceptrons (MLPs), regressors now widely used as general-purpose approximators, are compared with classical approximants: least-squares polynomials in the monomial and Chebyshev bases, piecewise-linear interpolation and cubic-spline interpolation. Accuracy is assessed by MAE, RMSE and and by the uniform error , and the observed behaviour is related to a-priori error estimates. Under an 80/20 random partition, the random forest achieved the lowest test error among the original data-driven configurations (, ), ahead of a degree-8 polynomial () and an MLP trained with early stopping (). On the same data, however, piecewise-linear interpolation, cubic-spline interpolation and a degree-40 Chebyshev polynomial reached , and , respectively. Under mesh refinement, the observed convergence rates were (random forest), (piecewise-linear) and (cubic spline), consistent with the , and estimates, whereas the Chebyshev error decayed super-geometrically with the degree. The degree-8 result was sensitive to the ill-conditioning of the monomial basis, MLP accuracy depended strongly on the optimiser, and no method extrapolated reliably beyond the sampled interval ( on for all methods). A coefficient of determination close to one therefore indicates accurate in-domain interpolation, not approximation-theoretic superiority.
Downloads
References
[1] K. Weierstrass, “Über die analytische Darstellbarkeit sogenannter willkürlicher Functionen einer reellen Veränderlichen,” Sitzungsberichte der Königlich Preußischen Akademie der Wissenschaften zu Berlin, pp. 633–639 and 789–805, 1885.
[2] L. N. Trefethen, Approximation Theory and Approximation Practice, extended ed. Philadelphia, PA, USA: SIAM, 2019, doi: 10.1137/1.9781611975949.
[3] C. de Boor, A Practical Guide to Splines, rev. ed. New York, NY, USA: Springer, 2001.
[4] J. P. Boyd, Chebyshev and Fourier Spectral Methods, 2nd ed. Mineola, NY, USA: Dover, 2001.
[5] G. E. Karniadakis, I. G. Kevrekidis, L. Lu, P. Perdikaris, S. Wang, and L. Yang, “Physics-informed machine learning,” Nature Reviews Physics, vol. 3, pp. 422–440, 2021, doi: 10.1038/s42254-021-00314-5.
[6] Z. Azarhoosh and M. Ilchi Ghazaan, “A review of recent advances in surrogate models for uncertainty quantification of high-dimensional engineering applications,” Computer Methods in Applied Mechanics and Engineering, vol. 433, Art. no. 117508, 2025, doi: 10.1016/j.cma.2024.117508.
[7] A. Marrel and B. Iooss, “Probabilistic surrogate modeling by Gaussian process: A review on recent insights in estimation and validation,” Reliability Engineering & System Safety, vol. 247, Art. no. 110094, 2024, doi: 10.1016/j.ress.2024.110094.
[8] G. Cybenko, “Approximation by superpositions of a sigmoidal function,” Mathematics of Control, Signals and Systems, vol. 2, no. 4, pp. 303–314, 1989, doi: 10.1007/BF02551274.
[9] K. Hornik, “Approximation capabilities of multilayer feedforward networks,” Neural Networks, vol. 4, no. 2, pp. 251–257, 1991, doi: 10.1016/0893-6080(91)90009-T.
[10] A. Pinkus, “Approximation theory of the MLP model in neural networks,” Acta Numerica, vol. 8, pp. 143–195, 1999.
[11] M. T. Augustine, “A survey on universal approximation theorems,” arXiv:2407.12895, 2024.
[12] T. Bui-Thanh, “A unified and constructive framework for the universality of neural networks,” IMA Journal of Applied Mathematics, vol. 89, no. 1, pp. 197–230, 2024, doi: 10.1093/imamat/hxad032.
[13] M. E. Valle, W. L. Vital, and G. Vieira, “Universal approximation theorem for vector- and hypercomplex-valued neural networks,” Neural Networks, vol. 180, Art. no. 106632, 2024, doi: 10.1016/j.neunet.2024.106632.
[14] Z. Li, H. Li, J. Wang, and K. Tang, “Approximation to smooth functions by low-rank Swish networks,” in Proc. 42nd Int. Conf. Machine Learning (ICML), PMLR vol. 267, pp. 35259–35291, 2025.
[15] T. De Ryck and S. Mishra, “Numerical analysis of physics-informed neural networks and related models in physics-informed machine learning,” Acta Numerica, vol. 33, pp. 633–713, 2024, doi: 10.1017/S0962492923000089.
[16] L. Breiman, “Random forests,” Machine Learning, vol. 45, no. 1, pp. 5–32, 2001, doi: 10.1023/A:1010933404324.
[17] G. Biau and E. Scornet, “A random forest guided tour,” TEST, vol. 25, no. 2, pp. 197–227, 2016, doi: 10.1007/s11749-016-0481-7.
[18] G. H. Golub and C. F. Van Loan, Matrix Computations, 4th ed. Baltimore, MD, USA: Johns Hopkins Univ. Press, 2013.
[19] C. A. Hall and W. W. Meyer, “Optimal error bounds for cubic spline interpolation,” Journal of Approximation Theory, vol. 16, no. 2, pp. 105–122, 1976, doi: 10.1016/0021-9045(76)90040-X.
[20] N. Rahaman, A. Baratin, D. Arpit, F. Draxler, M. Lin, F. Hamprecht, Y. Bengio, and A. Courville, “On the spectral bias of neural networks,” in Proc. 36th Int. Conf. Machine Learning (ICML), PMLR vol. 97, pp. 5301–5310, 2019.
[21] D. C. Liu and J. Nocedal, “On the limited memory BFGS method for large scale optimization,” Mathematical Programming, vol. 45, pp. 503–528, 1989, doi: 10.1007/BF01589116.
[22] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in Proc. 3rd Int. Conf. Learning Representations (ICLR), 2015, arXiv:1412.6980.
[23] F. Pedregosa et al., “Scikit-learn: Machine learning in Python,” Journal of Machine Learning Research, vol. 12, pp. 2825–2830, 2011.
