Combinations of Fast Activation and Trigonometric Functions in Kolmogorov–Arnold Networks

Combinations of Fast Activation and Trigonometric Functions in Kolmogorov–Arnold Networks

For years, many neural networks have been developed based on the Kolmogorov-Arnold Representation Theorem (KART), which was created to address Hilbert’s 13th problem. Recently, relying on KART, Kolmogorov-Arnold Networks (KANs) have attracted attention from the research community, stimulating the us...

全面介绍

Đã lưu trong:
书目详细资料
Những tác giả chính: Linh, Trần Thị Phương, Tạ, Hoàng Thắng, Thai Duy Quy
格式: Conference paper
语言:English
出版: 2025
主题:
Kolmogorov-Arnold Networks
function combinations
activation functions
trigonometric functions
在线阅读:https://scholar.dlu.edu.vn/handle/123456789/4901
标签: 添加标签
没有标签, 成为第一个标记此记录!
Thư viện lưu trữ: Thư viện Trường Đại học Đà Lạt
id oai:scholar.dlu.edu.vn:123456789-4901
record_format dspace
institution Thư viện Trường Đại học Đà Lạt
collection Thư viện số
language English
topic Kolmogorov-Arnold Networks
function combinations
activation functions
trigonometric functions
spellingShingle Kolmogorov-Arnold Networks
function combinations
activation functions
trigonometric functions
Linh, Trần Thị Phương
Tạ, Hoàng Thắng
Thai Duy Quy
Combinations of Fast Activation and Trigonometric Functions in Kolmogorov–Arnold Networks
description For years, many neural networks have been developed based on the Kolmogorov-Arnold Representation Theorem (KART), which was created to address Hilbert’s 13th problem. Recently, relying on KART, Kolmogorov-Arnold Networks (KANs) have attracted attention from the research community, stimulating the use of polynomial functions such as B-splines and RBFs. However, these functions are not fully supported by GPU devices and are still considered less popular. In this paper, we propose the use of fast computational functions, such as ReLU and trigonometric functions (e.g., ReLU, sin, cos, arctan), as basis components in Kolmogorov–Arnold Networks (KANs). By integrating these function combinations into the network structure, we aim to enhance computational efficiency. Experimental results show that these combinations maintain competitive performance while offering potential improvements in training time and generalization.
format Conference paper
author Linh, Trần Thị Phương
Tạ, Hoàng Thắng
Thai Duy Quy
author_facet Linh, Trần Thị Phương
Tạ, Hoàng Thắng
Thai Duy Quy
author_sort Linh, Trần Thị Phương
title Combinations of Fast Activation and Trigonometric Functions in Kolmogorov–Arnold Networks
title_short Combinations of Fast Activation and Trigonometric Functions in Kolmogorov–Arnold Networks
title_full Combinations of Fast Activation and Trigonometric Functions in Kolmogorov–Arnold Networks
title_fullStr Combinations of Fast Activation and Trigonometric Functions in Kolmogorov–Arnold Networks
title_full_unstemmed Combinations of Fast Activation and Trigonometric Functions in Kolmogorov–Arnold Networks
title_sort combinations of fast activation and trigonometric functions in kolmogorov–arnold networks
publishDate 2025
url https://scholar.dlu.edu.vn/handle/123456789/4901
_version_ 1845408491564433408
spelling oai:scholar.dlu.edu.vn:123456789-49012025-07-02T13:47:10Z Combinations of Fast Activation and Trigonometric Functions in Kolmogorov–Arnold Networks Linh, Trần Thị Phương Tạ, Hoàng Thắng Thai Duy Quy Kolmogorov-Arnold Networks function combinations activation functions trigonometric functions For years, many neural networks have been developed based on the Kolmogorov-Arnold Representation Theorem (KART), which was created to address Hilbert’s 13th problem. Recently, relying on KART, Kolmogorov-Arnold Networks (KANs) have attracted attention from the research community, stimulating the use of polynomial functions such as B-splines and RBFs. However, these functions are not fully supported by GPU devices and are still considered less popular. In this paper, we propose the use of fast computational functions, such as ReLU and trigonometric functions (e.g., ReLU, sin, cos, arctan), as basis components in Kolmogorov–Arnold Networks (KANs). By integrating these function combinations into the network structure, we aim to enhance computational efficiency. Experimental results show that these combinations maintain competitive performance while offering potential improvements in training time and generalization. 2025-07-02T13:45:25Z 2025-07-02T13:45:25Z 2025-07 Conference paper Bài báo đăng trên KYHT trong nước (có ISBN) https://scholar.dlu.edu.vn/handle/123456789/4901 en ICT2025 [1]Z. Liu, Y. Wang, S. Vaidya, F. Ruehle, J. Halverson, M. Soljaciˇ c,´ T. Y. Hou, and M. Tegmark, “Kan: Kolmogorov-arnold networks,” arXiv preprint arXiv:2404.19756, 2024. [2]Z. Liu, P. Ma, Y. Wang, W. Matusik, and M. Tegmark, “Kan 2.0: Kolmogorov-arnold networks meet science,” arXiv preprint arXiv:2408.10205, 2024. [3]Z. Li, “Kolmogorov-arnold networks are radial basis function networks,” arXiv preprint arXiv:2405.06721, 2024. [4]A. Delis, “Fasterkan,” https://github.com/ AthanasiosDelis/faster-kan/, 2024. [5]Z. Bozorgasl and H. Chen, “Wav-kan: Wavelet kolmogorov-arnold networks,” arXiv preprint arXiv:2405.12832, 2024. [6]S. SS, “Chebyshev polynomial-based kolmogorov-arnold networks: An efficient architecture for nonlinear function approximation,” arXiv preprint arXiv:2405.07200, 2024. [7]H.-T. Ta, “Bsrbf-kan: A combination of b-splines and radial basis functions in kolmogorov-arnold networks,” arXiv preprint arXiv:2406.11173, 2024. [8]Z. Yang, J. Zhang, X. Luo, Z. Lu, and L. Shen, “Activation space selectable kolmogorov-arnold networks,” arXiv preprint arXiv:2408.08338, 2024. [9]M. G. Altarabichi, “Rethinking the function of neurons in kans,” arXiv preprint arXiv:2407.20667, 2024. [10]R. Jie, J. Gao, A. Vasnev, and M.-n. Tran, “Regularized flexible activation function combination for deep neural networks,” in 2020 25th International Conference on Pattern Recognition (ICPR). IEEE, 2021, pp. 2001– 2008. [11]L. Xu and C. P. Chen, “Comparison and combination of activation functions in broad learning system,” in 2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE, 2020, pp. 3537–3542. [12]L. M. Zhang, “Genetic deep neural networks using different activation functions for financial data mining,” in 2015 IEEE International Conference on Big Data (Big Data). IEEE, 2015, pp. 2849–2851. [13]H.-T. Ta, D.-Q. Thai, A. B. S. Rahman, G. Sidorov, and A. Gelbukh, “Fc-kan: Function combinations in kolmogorov-arnold networks,” arXiv preprint arXiv:2409.01763, 2024. [14]A. N. Kolmogorov, “On the representation of continuous functions of many variables by superposition of continuous functions of one variable and addition,” in Doklady Akademii Nauk, vol. 114. Russian Academy of Sciences, 1957, pp. 953–956. [15]J. Braun and M. Griebel, “On a constructive proof of kolmogorov’s superposition theorem,” Constructive approximation, vol. 30, pp. 653–675, 2009. [16]H. Hao, X. Zhang, B. Li, and A. Zhou, “A first look at kolmogorov-arnold networks in surrogate-assisted evolutionary algorithms,” arXiv preprint arXiv:2405.16494, 2024. [17]A. Xu, B. Zhang, S. Kong, Y. Huang, Z. Yang, S. Srivastava, and M. Sun, “Effective integration of kan for keyword spotting,” arXiv preprint arXiv:2409.08605, 2024. [18]D. W. Abueidda, P. Pantidis, and M. E. Mobasher, “Deepokan: Deep operator network based on kolmogorov arnold networks for mechanics problems,” arXiv preprint arXiv:2405.19143, 2024. [19]A. Kundu, A. Sarkar, and A. Sadhu, “Kanqas: Kolmogorov arnold network for quantum architecture search,” arXiv preprint arXiv:2406.17630, 2024. [20]H. Wakaura and A. Suksmono, “Variational quantum kolmogorov-arnold network,” 2024. [21]W. Troy, “Sparks of quantum advantage and rapid retraining in machine learning,” arXiv preprint arXiv:2407.16020, 2024. [22]W. Knottenbelt, Z. Gao, R. Wray, W. Z. Zhang, J. Liu, and M. Crispin-Ortuzar, “Coxkan: Kolmogorov-arnold networks for interpretable, high-performance survival analysis,” arXiv preprint arXiv:2409.04290, 2024. [23]R. Genet and H. Inzirillo, “Tkan: Temporal kolmogorovarnold networks,” arXiv preprint arXiv:2405.07344, 2024. [24]K. Xu, L. Chen, and S. Wang, “Kolmogorov-arnold networks for time series: Bridging predictive power and interpretability,” arXiv preprint arXiv:2406.02496, 2024. [25]C. J. Vaca-Rubio, L. Blanco, R. Pereira, and M. Caus, “Kolmogorov-arnold networks (kans) for time series analysis,” arXiv preprint arXiv:2405.08790, 2024. [26]R. Genet and H. Inzirillo, “A temporal kolmogorovarnold transformer for time series forecasting,” arXiv preprint arXiv:2406.02486, 2024. [27]X. Han, X. Zhang, Y. Wu, Z. Zhang, and Z. Wu, “Kan4tsf: Are kan and kan-based models effective for time series forecasting?” arXiv preprint arXiv:2408.11306, 2024. [28]K. Xu, L. Chen, and S. Wang, “Kan4drift: Are kan effective for identifying and tracking concept drift in time series?” in NeurIPS Workshop on Time Series in the Age of Large Models, 2024. [29]C. Li, X. Liu, W. Li, C. Wang, H. Liu, and Y. Yuan, “Ukan makes strong backbone for medical image segmentation and generation,” arXiv preprint arXiv:2406.02918, 2024. [30]M. Cheon, “Demonstrating the efficacy of kolmogorovarnold networks in vision tasks,” arXiv preprint arXiv:2406.14916, 2024. [31]R. Ge, X. Yu, Y. Chen, F. Jia, S. Zhu, G. Zhou, Y. Huang, C. Zhang, D. Zeng, C. Wang et al., “Tc-kanrecon: Highquality and accelerated mri reconstruction via adaptive kan mechanisms and intelligent feature scaling,” arXiv preprint arXiv:2408.05705, 2024. [32]C. De Boor, “On calculating with b-splines,” Journal of Approximation theory, vol. 6, no. 1, pp. 50–62, 1972. [33]S. S. Bhattacharjee, “Torchkan: Simplified kan model with variations,” https://github.com/1ssb/torchkan/, 2024. [34]J. Xu, Z. Chen, J. Li, S. Yang, W. Wang, X. Hu, and E. C.-H. Ngai, “Fourierkan-gcf: Fourier kolmogorovarnold network–an effective and efficient feature transformation for graph collaborative filtering,” arXiv preprint arXiv:2406.01034, 2024. [35]S. T. Seydi, “Unveiling the power of wavelets: A waveletbased kolmogorov-arnold network for hyperspectral image classification,” arXiv preprint arXiv:2406.07869, 2024. [36]S. Teymoor Seydi, “Exploring the potential of polynomial basis functions in kolmogorov-arnold networks: A comparative study of different groups of polynomials,” arXiv e-prints, pp. arXiv–2406, 2024. [37]M. G. Altarabichi, “Dropkan: Regularizing kans by masking post-activations,” arXiv preprint arXiv:2407.13044, 2024. [38]Q. Qiu, T. Zhu, H. Gong, L. Chen, and H. Ning, “Relu-kan: New kolmogorov-arnold networks that only need matrix addition, dot multiplication, and relu,” arXiv preprint arXiv:2406.02075, 2024. [39]A. V. Chernov, “Gaussian functions combined with kolmogorov’s theorem as applied to approximation of functions of several variables,” Computational Mathematics and Mathematical Physics, vol. 60, pp. 766–782, 2020. [40]J. Schmidt-Hieber, “The kolmogorov–arnold representation theorem revisited,” Neural networks, vol. 137, pp. 119–126, 2021. [41]L. Deng, “The mnist database of handwritten digit images for machine learning research [best of the web],” IEEE signal processing magazine, vol. 29, no. 6, pp. 141–142, 2012. [42]H. Xiao, K. Rasul, and R. Vollgraf, “Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms,” arXiv preprint arXiv:1708.07747, 2017. [43]Blealtan, “efficient-kan,” https://github.com/Blealtan/ efficient-kan, 2024.

相似书籍