Hyperparameter Optimization in Machine Learning Models on Sky Survey Data Classification
-
Efraim Kurniawan Dairo Kette(1*)
Universitas Nusa Cendana
(*) Corresponding Author
Abstract
Discovering the optimal model in today's popularity of various machine learning applications remains an essential challenge. Besides data dependency, the performance of classification models is also affected by deciding on suitable algorithm with optimal hyperparameter settings. This study conducted a hyperparameter optimization process and compared the accuracy results by applying various classification models to the observation dataset. This study obtains data from the Sloan Digital Sky Survey Data Release 18 (SDSS-DR18) and Sloan Extension for Galactic Understanding and Exploration (SEGUE-IV). The SDSS-DR18 and SEGUE-IV provide observational data of space objects, such as stellar spectra with corresponding positions and magnitudes of galaxies or stars. The SDSS-DR18 dataset contains magnitude and redshift data of celestial objects with target features of stars, Quasi Stellar Objects (QSOs), and galaxies. The SEGUE-IV dataset contains equivalent-width parameters, inline indices, and other features to the radial velocity of the corresponding star spectrum. This study utilized several machine learning models, such as k-Nearest Neighbor (KNN), Gaussian-Naive Bayes, eXtreme Gradient Boosting (XGBoost), Random Forest, Support Vector Machine (SVM), and Multi-Layer Perceptron (MLP). This study utilized Bayesian, Grid, and Random-based approaches to find the optimal hyperparameters to maximize the performance of the classification model. This study proved that some classification models have improved accuracy scores through the Bayesian-based hyperparameter optimization settings. This study discovers the XGBoost model shows the highest classification results after hyperparameters optimization compared to other models for both datasets with an average accuracy of 99.10% and 95.11%, respectively.
Downloads
References
[2] I. H. Sarker, “Deep Learning: A Comprehensive Overview on Techniques, Taxonomy, Applications and Research Directions,” SN Comput. Sci., vol. 2, no. 6, pp. 420, 2021, doi: 10.1007/s42979-021-00815-1.
[3] A. Almeida et al., “The Eighteenth Data Release of the Sloan Digital Sky Surveys: Targeting and First Spectra from SDSS-V,” Astrophys. J. Suppl. Ser., vol. 267, no. 2, pp. 44, 2023, doi: 10.3847/1538-4365/acda98.
[4] S. Ethiraj and B. K. Bolla, “Classification of Quasars, Galaxies, and Stars in the Mapping of the Universe Multi-modal Deep Learning,” Deep Learning Developers Conference, 2022, doi: 10.48550/arXiv.2205.10745.
[5] W. Xiao-Qing and Y. Jin-Meng, “Classification of star/galaxy/QSO and Star Spectral Types from LAMOST Data Release 5 with Machine Learning Approaches,” Chin. J. Phys., vol. 69, pp. 303–311, 2021, doi: 10.1016/j.cjph.2020.03.008.
[6] S. Raschka, “Model Evaluation, Model Selection, and Algorithm Selection in Machine Learning,” 2020, doi: 10.48550/arXiv.1811.12808.
[7] L. Yang and A. Shami, “On hyperparameter optimization of machine learning algorithms: Theory and practice,” Neurocomputing, vol. 415, pp. 295–316, 2020, doi: 10.1016/j.neucom.2020.07.061.
[8] J. Bergstra and Y. Bengio, “Random search for hyper-parameter optimization,” J. Mach. Learn. Res., vol. 13, pp. 281–305, 2012. [Online]. Available at: https://dl.acm.org/doi/pdf/ 10.5555/2188385.2188395.
[9] R. Turner dkk., “Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning: Analysis of the Black-Box Optimization Challenge 2020,” 2021, doi: 10.48550/arXiv.2104.10201.
[10] J. Kollmeier dkk., “SDSS-V Pioneering Panoptic Spectroscopy,” vol. 51, pp. 274, 2019. [Online]. Available at: https://arxiv.org/pdf/1711.03234.
[11] F. Oktariani, F. A. Adziima, and E. K. D. Kette, “Application of Classification Algorithms in Machine Learning on Digital Sloan Data Survey (SDSS) and Sloan Extension for Galactic Understanding and Exploration (SEGUE) data,” 13th International Symposium on Computational Science, 2022. [Online]. Available at: https://iscs2022.site/event/1.
[12] M. Ben-Bassat, K. L. Klove, and M. H. Weil, “Sensitivity Analysis in Bayesian Classification Models: Multiplicative Deviations,” IEEE Trans. Pattern Anal. Mach. Intell., vol. PAMI-2, no. 3, pp. 261–266, Mei 1980, doi: 10.1109/TPAMI.1980.4767015.
[13] G. H. John and P. Langley, “Estimating continuous distributions in Bayesian classifiers,” Proceedings of the Eleventh conference on Uncertainty in artificial intelligence, 1995, pp. 338–345.
[14] P. Domingos and M. Pazzani, “On the Optimality of the Simple Bayesian Classifier under Zero-One Loss,” Mach. Learn., vol. 29, no. 2, pp. 103–130, 1997, doi: 10.1023/A:1007413511361.
[15] P. K. Syriopoulos, N. G. Kalampalikis, S. B. Kotsiantis, and M. N. Vrahatis, “kNN Classification: a review,” Ann. Math. Artif. Intell., 2023, doi: 10.1007/s10472-023-09882-x.
[16] T. Evgeniou and M. Pontil, “Support Vector Machines: Theory and Applications,” Machine Learning and Its Applications: Advanced Lectures, 2001, pp. 249–257. doi: 10.1007/3-540-44673-7_12.
[17] A. Roy and S. Chakraborty, “Support Vector Machine in Structural Reliability Analysis: A Review,” Reliab. Eng. Syst. Saf., vol. 233, pp. 109126, 2023, doi: 10.1016/j.ress.2023.109126.
[18] P. Dutta, S. Paul, and A. Kumar, “Chapter 25 - Comparative Analysis of Various Supervised Machine Learning Techniques for Diagnosis of COVID-19,” dalam Electronic Devices, Circuits, and Systems for Biomedical Applications, 2021, pp. 521–540. doi: 10.1016/B978-0-323-85172-5.00020-4.
[19] I. A. Osman, A. N. Ahmed, M. F. Chow, Y. F. Huang, and A. El-Shafie, “Extreme Gradient Boosting (XGBoost) Model to Predict the Groundwater Levels in Selangor Malaysia,” Ain Shams Eng. J., vol. 12, no. 2, pp. 1545–1556, 2021, doi: 10.1016/j.asej.2020.11.011.
[20] S. S. Chai, W. L. Cheah, K. L. Goh, Y. H. R. Chang, K. Y. Sim, and K. O. Chin, “A Multilayer Perceptron Neural Network Model to Classify Hypertension in Adolescents Using Anthropometric Measurements: A Cross-Sectional Study in Sarawak, Malaysia,” Comput. Math. Methods Med., vol. 2021, no. 1, pp. 2794888, 2021, doi: 10.1155/2021/2794888.
[21] T.T. Wong, “Performance Evaluation of Classification Algorithms by k-Fold and Leave-One-Out Cross Validation,” Pattern Recognit., vol. 48, no. 9, pp. 2839–2846, 2015, doi: 10.1016/j.patcog.2015.03.009.
Copyright (c) 2025 Efraim Kurniawan Dairo Kette
This work is licensed under a Creative Commons Attribution 4.0 International License.
The author submitting the manuscript must understand and agree that if accepted for publication, authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution (CC-BY) 4.0 License that allows others to share the work with an acknowledgment of the work’s authorship and initial publication in this journal.
