AI-Driven Makeup Suggestions Leveraging Mediapipe Face Landmarks For Eye Shape Detection: Rekomendasi Tampilan Riasan Mata Berbasis AI Menggunakan Landmark Wajah Mediapipe Untuk Mendeteksi Bentuk Mata

Faustin Devanda; Handri Santoso

doi:10.33050/tmj.v10i1.2316

Authors

Faustin Devanda , Universitas Pradita
Handri Santoso , Universitas Pradita https://orcid.org/0000-0002-3265-8179 (unauthenticated)

DOI:

https://doi.org/10.33050/tmj.v10i1.2316

Abstract

In the world of beauty, makeup is not only a form of self expression but also a creative skill that requires precision and an understanding of facial structures. Among all facial features, the eyes play a crucial role in defining makeup styles. Each individual has unique eye shapes such as round, monolid, upturned, almond, and downturned, which require different makeup techniques to enhance their appearance. However, many individuals struggle to identify their eye shape, leading to suboptimal makeup results. This research aims to develop an intelligent system for eye shape classification using image processing and artificial intelligence technologies. MediaPipe, a robust and lightweight framework for facial landmark detection, was employed to extract key features from the eye region, including Eye Aspect Ratio (EAR), Eye Corner (angle), and Eye Distance. A total of 1,250 images were used from various datasets including personal archives, Kaggle, and GitHub MUCT. The classification process used a Support Vector Machine (SVM) with a non-linear RBF kernel, and its performance was validated using K-Fold Cross Validation with 10 folds. The system demonstrated high accuracy for almond, downturned, monolid, and round eyes. However, classification for upturned eyes showed less optimal results, likely due to limitations in the current feature set. This study also introduces an integrated open camera interface that detects eye shape in real time and recommends suitable eye makeup styles. This research contributes to inclusive beauty technology by providing personalized makeup suggestions based on eye shape, aligning with SDG 5 (Gender Equality) and SDG 9 (Industry, Innovation, and Infrastructure). Future work will focus on improving accuracy, particularly for upturned eye classification.

References

R. Nabila and P. T. Prihatin, “Adaptasi makeup cleopatra pada tata rias wajah kesempatan pesta malam,” Journal of Education Research, vol. 5, no. 2, pp. 1956–1967, 2024.

V. Agarwal, M. Lohani, A. S. Bist, L. Rahardja, M. Hardini, and G. Mustika, “Deep cnn–real esrgan: An innovative framework for lung disease prediction,” in 2022 IEEE Creative Communication and Innovative Technology (ICCIT). IEEE, 2022, pp. 1–6.

T. Supiani and S. R. Salsa, “Pembuatan video tutorial teknik winged eyeliner pada tata rias wajah korektif untuk mata bulat,” J. Tata Rias, vol. 10, no. 2, pp. 91–102, 2023.

C. Yang, W. He, Y. Xu, and Y. Gao, “Elegant: Exquisite and locally editable gan for makeup transfer,” arXiv preprint arXiv:2207.09840, 2022.

F. Dwinuriyani, “Tata rias korektif,” pp. 86–100, 2022.

H. Wang, H. Xie, and K. Miyata, “Iloveeye: Eyeliner makeup guidance system with eye shape features,” arXiv preprint arXiv:2203.13592, 2022.

U. Rusilowati, U. Narimawati, Y. R. Wijayanti, U. Rahardja, and O. A. Al-Kamari, “Optimizing human resource planning through advanced management information systems: A technological approach,” Aptisi Transactions on Technopreneurship (ATT), vol. 6, no. 1, pp. 72–83, 2024.

H. Balim, S. Park, X. Wang, X. Zhang, and O. Hilliges, “Efe: End-to-end frame-to-gaze estimation,” arXiv preprint arXiv:2305.05526, 2023.

F. Thabtah and D. Peebles, “Assessment for alzheimer’s disease advancement using classification models with rules,” Applied Sciences, vol. 13, no. 21, p. 12152, 2023.

M. Faisal, S. A. M. Hidayat, A. R. Basrida, M. T. Fazrin et al., “Prototype of water level and rainfall detection system as flood warning based on blynk iot application,” International Transactions on Education Technology, vol. 2, no. 1, pp. 1–10, 2023.

M. L. Amit, A. C. Fajardo, and R. P. Medina, “Recognition of real-time hand gestures using mediapipe holistic model and lstm with mlp architecture,” in 2022 IEEE 10th Conference on Systems, Process & Control (ICSPC), 2022, pp. 292–295.

S. Sulistio, “Assessing the factors influencing cybersecurity effectiveness: A pls-sem approach,” International Transactions on Education Technology, vol. 2, no. 1, pp. 49–58, 2023.

P. K. Espandiah, P. A. Mayuni, and M. D. Angendari, “Aplikasi eyeshadow 3d pada tata rias pengantin bali agung modifikasi di salon tutde wedding,” J. BOSAPARIS Pendidik. Kesejaht. Kel., vol. 12, no. 3, pp. 107–117, 2021.

G. Maulani, E. W. Musu, Y. J. W. Soetikno, and S. Aisa, “Education management using blockchain as future application innovation,” IAIC Transactions on Sustainable Digital Innovation (ITSDI), vol. 3, no. 1, pp. 60–65, 2021.

S. Hervella, J. Rouco, J. Novo, and M. Ortega, “Multimodal image encoding pre-training for diabetic retinopathy grading,” Computational Biology and Medicine, vol. 143, p. 105302, 2022.

A. S. Bist, B. Rawat, A. G. Prawiyogi, N. Septiani, M. Fakhrezzy, D. B. Saputra et al., “Ai-enabled blockchain for supply chain in agriculture,” in 2022 IEEE Creative Communication and Innovative Technology (ICCIT). IEEE, 2022, pp. 1–5.

E. A. Nabila, S. Santoso, Y. Muhtadi, and B. Tjahjono, “Artificial intelligence robots and revolutionizing society in terms of technology, innovation, work and power,” IAIC Transactions on Sustainable Digital Innovation (ITSDI), vol. 3, no. 1, pp. 46–52, 2021.

D. Kumar and C. Ravindra, “Transfer learning and hybrid models for the classification of eye diseases on retinal images,” SPIE Applications of Machine Learning 2024, Proc. SPIE 13138, 2024.

F. Kruger, Z. Queen, O. Radelva, and N. Lawrence, “Comparative analysis of scientific approaches in computer science: A quantitative study,” International Transactions on Education Technology (ITEE), vol. 2, no. 2, pp. 120–128, 2024.

K. Syahida, “Penggunaan make up terhadap kepercayaan diri wanita dewasa awal (study pada mahasiswi fakultas ekonomi bisnis islam uin sunan kalijaga yogyakarta),” Pap. Knowl., pp. 1–75, 2021.

I. Putu, W. Dwinata, and Y. J. Pambudi, “Penggunaan teknologi artificial intelligence dalam pemilihan produk kecantikan oleh konsumen wanita,” vol. 12, no. 7, pp. 733–752, 2023. [Online]. Available: https://doi.org/10.24843/EJMUNUD.2023.v12.i07.p05

I. Handayani, D. Apriani, M. Mulyati, A. R. A. Zahra, and N. A. Yusuf, “Enhancing security and privacy of patient data in healthcare: A smartpls analysis of blockchain technology implementation,” IAIC Transactions on Sustainable Digital Innovation (ITSDI), vol. 5, no. 1, pp. 8–17, 2023.

K. Nguyen, H. Proenc¸a, and F. Alonso-Fernandez, “Advances and prospects of multi-modal ophthalmic artificial intelligence based on deep learning: a review,” Eye and Vision, vol. 11, p. xx–xx, 2024.

P. Das, D. C. Dobhal, A. S. Bist, F. P. Oganda, I. S. Fadli, H. Nusantoro et al., “A review on feature selection approaches and few prominent research gaps for image classification,” 2022 IEEE Creative Communication and Innovative Technology (ICCIT), pp. 1–7, 2022.

A. H. Khaleel, T. H. Abbas, and A. W. S. Ibrahim, “Best low-cost methods for real-time detection of the eye and gaze tracking,” I-Com, pp. 1–16, 2024.

V. Meilinda, S. A. Anjani, and M. Ridwan, “A platform based business revolution activates indonesia’s digital economy,” Startupreneur Business Digital (SABDA Journal), vol. 2, no. 2, pp. 155–174, 2023.

B. Domini, A. S. Dewi, and G. P. Cesna, “The effects of artificial intelligence (ai) on company performance including the value of ai-based business transformation initiatives,” IAIC Trans. Sustain. Digit. Innov, vol. 5, no. 1, pp. 24–38, 2023.

S. Sugeng and T. N. Nizar, “Deteksi aktivitas mata, mulut dan kemiringan kepala sebagai fitur untuk deteksi kantuk pada pengendara mobil,” Komputika J. Sist. Komput., vol. 12, no. 1, pp. 83–91, 2023.

J. Williams, A. G. Prawiyogi, M. Rodriguez, and I. Kovac, “Enhancing circular economy with digital technologies: A pls-sem approach,” International Transactions on Education Technology (ITEE), vol. 2, no. 2, pp. 140–151, 2024.

K. A. A. Manurung, H. Siregar, I. Fahmi, and D. B. Hakim, “Value chain and esg performance as determinants of sustainable lending in commercial bank: A systematic literature review,” Aptisi Transactions on Technopreneurship (ATT), vol. 6, no. 1, pp. 41–55, 2024.

D. Jonas, E. Maria, I. R. Widiasari, U. Rahardja, T. Wellem et al., “Design of a tam framework with emotional variables in the acceptance of health-based iot in indonesia,” ADI Journal on Recent Innovation, vol. 5, no. 2, pp. 146–154, 2024.

D. E. Rose, J. Van Der Merwe, and J. Jones, “Digital marketing strategy in enhancing brand awareness and profitability of e-commerce companies,” APTISI Transactions on Management, vol. 8, no. 2, pp. 160–166, 2024.

M. Chakim, M. Mulyati, C. Komalasari, M. Madani, and R. Raihan, “Sustainable competitive advantage through market and entrepreneurial orientation in indonesian retail msmes during the covid-19 recovery stage,” in 2022 IEEE Creative Communication and Innovative Technology (ICCIT). IEEE, 2022, pp. 1–6.

G. S. Putra, I. I. Maulana, A. D. Chayo, M. I. Haekal, R. Syaharani et al., “Pengukuran efektivitas platform e-learning dalam pembelajaran teknik informatika di era digital,” Jurnal MENTARI: Manajemen, Pendidikan dan Teknologi Informasi, vol. 3, no. 1, pp. 19–29, 2024.

S. Doijad, A. Bhalerao, and S. Khan, “Eye irises tracking using mediapipe and opencv,” Int. J. Progress. Res. Eng. Manag. Sci., vol. 2, pp. 319–322, 2022.

M. W. Wicaksono, M. B. Hakim, F. H. Wijaya, T. Saleh, E. Sana et al., “Analyzing the influence of artificial intelligence on digital innovation: A smartpls approach,” IAIC Transactions on Sustainable Digital Innovation (ITSDI), vol. 5, no. 2, pp. 108–116, 2024.

M. F. Rasyid, M. S. Mustafa, A. A. M. Suradi, M. Rizal, M. Mushaf, and A. Arifin, “Deteksi mata di video smartphone menggunakan mediapipe python,” JOINTECS (Journal Inf. Technol. Comput. Sci.), vol. 8, no. 2, p. 49, 2023.

I. Khong, N. Aprila Yusuf, A. Nuriman, and A. Bayu Yadila, “Exploring the Impact of Data Quality on Decision-Making Processes in Information Intensive Organizations,” APTISI Transactions on Management (ATM), vol. 7, no. 3, pp. 253–260, 2023.

C. Dewi, R. C. Chen, C. W. Chang, S. H. Wu, X. Jiang, and H. Yu, “Eye aspect ratio for real-time drowsiness detection to improve driver safety,” Electron., vol. 11, no. 19, 2022.

A. Nuche, O. Sy, and J. C. Rodriguez, “Optimizing efficiency through sustainable strategies: The role of management and monitoring in achieving goals,” APTISI Transactions on Management, vol. 8, no. 2, pp. 167–174, 2024.

H. Wang, “Eye-makeup guidance system based on eye-shape analysis,” 2021.

Q. Aini, D. Manongga, U. Rahardja, I. Sembiring, and Y.-M. Li, “Understanding behavioral intention to use of air quality monitoring solutions with emphasis on technology readiness,” International Journal of Human–Computer Interaction, pp. 1–21, 2024.

Ara, “Tutorial smokey eyes for round eyes,” https://s.lemon8-app.com/s/UFQhMRFrQR, 2022, accessed May 27, 2024.

Monolidtreasure, “Daily neutral korean eyeshadow palette — monolid makeup,” https://s.lemon8-app. com/s/FpefeRFrQR, 2022, accessed May 27, 2024.

Fricillia, “Reverse cat eye makeup tutorial — upturned makeup,” https://s.lemon8-app.com/s/yUYjeRFrQR, 2022, accessed May 27, 2024.

K. Putrii, “Tutorial wing liner for almone eyes,” https://s.lemon8-app.com/s/dYsjeRFrQR, 2023, accessed May 27, 2024.

charlottaeve.com, “How to: makeup for downturned eyes,” https://www.charlottaeve.com/2019/11/18/how-to-makeup-for-downturned-eyes/, 2019, accessed May 27, 2024.

E. Maitria and M. Yanita, “Jahit bulu mata dan eyeshadow smokey pada mata turun terhadap rias,” J. Tata Rias dan Kecantikan, vol. 4, no. 1, p. 15, 2022.

G. R. Prillacaprienta, “Pengaruh teknik jahit mata dan penggunaan eyelid tape terhadap mata monolid untuk rias pengantin modern,” e-jurnal Ed. Yudisium, vol. 10, no. 3, pp. 1–7, 2021. [Online]. Available:https://ejournal.unesa.ac.id/index.php/jurnal-tata-rias/article/view/43668

R. Tarmizi, N. Septiani, P. A. Sunarya, and Y. P. A. Sanjaya, “Harnessing digital platforms for entrepreneurial success: A study of technopreneurship trends and practices,” Aptisi Transactions on Technopreneurship (ATT), vol. 5, no. 3, pp. 278–290, 2023.