NURNANINGSIH, Desi and Adi, Kusworo and Surarso, Bayu (2026) PENGEMBANGAN MODEL DETEKSI KANTUK BERDASARKAN ANALISIS WAJAH DAN KEMIRINGAN KEPALA DENGAN METODE VIDEO VISION TRANSFORMER (ViViT). Doctoral thesis, UNIVERSITAS DIPONEGORO.

	Text cover aja.pdf Download (135kB)
	Text Cover lengkap.pdf Restricted to Repository staff only Download (1MB)
	Text BAB I.pdf Download (1MB)
	Text BAB II.pdf Download (1MB)
	Text BAB III.pdf Restricted to Repository staff only Download (945kB)
	Text BAB IV.pdf Restricted to Repository staff only Download (1MB)
	Text BAB V.pdf Restricted to Repository staff only Download (808kB)
	Text Daftar Pustaka.pdf Download (405kB)
	Text Lampiran.pdf Restricted to Repository staff only Download (627kB)

Abstract

Kecelakaan lalu lintas akibat kantuk masih menjadi masalah keselamatan yang serius karena penurunan kewaspadaan sering terjadi secara bertahap dan sulit dikenali secara dini. Metode deteksi kantuk berbasis sinyal fisiologis seperti electroencephalography (EEG) memang mampu memberikan akurasi tinggi, tetapi bersifat intrusif dan kurang praktis untuk implementasi di kendaraan. Pendekatan visual non-intrusif lebih mudah diterapkan, namun metode berbasis citra konvensional umumnya masih sensitif terhadap perubahan pencahayaan, penggunaan kacamata, oklusi wajah, dan variasi pose kepala. Penelitian ini mengusulkan model deteksi kantuk berbasis Video Vision Transformer (ViViT) yang dirancang untuk menganalisis tiga indikator visual utama secara simultan, yaitu kondisi wajah seperti mata, mulut dan kemiringan kepala, melalui pemodelan spasial dan temporal pada data video. Pada penelitian menggunakan NTHU Driver Drowsiness Detection Dataset (NTHU-DDD) yang mencakup beragam kondisi, seperti siang, malam, penggunaan kacamata, dan variasi tingkat kewaspadaan pengemudi. Tahapan penelitian meliputi ekstraksi frame, deteksi wajah, normalisasi citra, pembentukan klip video, tokenisasi patch spasial -temporal, pemrosesan dengan spatial dan temporal transformer encoder, serta klasifikasi akhir menggunakan multilayer perceptron head. Model yang diusulkan kemudian dievaluasi menggunakan metrik accuracy, precision, recall, F1-score, confusion matrix, ROC-AUC, dan waktu inferensi. Hasil pengujian menunjukkan bahwa model ViViT yang diusulkan mampu mencapai precision 90,6%, recall 89,9%, AUC 0,87 dan F1-score 90,0%, serta waktu inferensi 28,9 ms per frame. Hasil tersebut menunjukkan bahwa model memiliki kemampuan yang baik dalam mengenali kondisi waspada dan kantuk secara cepat dan stabil pada berbagai kondisi visual. Kontribusi utama penelitian ini terletak pada pengembangan model deteksi kantuk non-intrusif berbasis video yang memadukan informasi kondisi wajah seperti mata tertutup serta mulut menguap dan kemiringan kepala dalam kerangka ViViT, sehingga memberikan alternatif yang lebih adaptif dan potensial untuk diterapkan sebagai sistem peringatan dini pada kendaraan cerdas.
Kata Kunci : Deteksi kantuk, kemiringan kepala, kondisi mata, mulut menguap, nthu-ddd, video vision transformer

Traffic accidents caused drowsiness remain a critical safety concern, as the decline in alertness often occurs gradually and is difficult to detect early. Although physiological signal-based detection methods, such as electroencephalography (EEG), offer high accuracy, they are intrusive and impractical for in-vehicle implementation. While non-intrusive visual approaches are more feasible, conventional image-based methods generally remain sensitive to lighting fluctuations, the use of eyeglasses, facial occl usion, and variations in head pose. To address these challenges, this study proposes a driver drowsiness detection model based on Video Vision Transformer (ViViT). The model is designed to simultaneously analyze two primary visual indicators facial conditions (eyes and mouth) and head tilt through spatial and temporal modeling of video data. This research utilizes the NTHU Driver Drowsiness Detection (NTHU-DDD) dataset, which encompasses diverse scenarios, including day and night conditions, drivers wearing glasses, and varying levels of alertness. The research stages involve frame extraction, face detection, image normalization, video clip construction, spatial-temporal patch tokenization, processing via Spatial and Temporal Transformer Encoders, and final classification using a multilayer perceptron head. The proposed model was evaluated using metrics such as accuracy, precision, recall, F1-score, confusion matrix, ROC-AUC, and inference time. The experimental results demonstrate that the proposed ViViT model achieves a precision of 90.6%, a recall of 89.9%, an AUC of 0.87 and an F1-score of 90.0%, with an inference time of 28.9 ms per frame. These findings indicate that the model effectively recognizes alert and drowsy states rapidly and consistently across various visual conditions. The primary contribution of this research lies in the development of a non-intrusive, video-based drowsiness detection model that integrates facial features (closed eyes and yawning) with head tilt within the ViViT framework, offering a more adaptive and potential alternative for early warning systems in smart vehi cles.
Keywords: Drowsiness Detection, eye condition, head movement, video vision transformer, nthu-ddd, yawning.

Actions (login required)

View Item