Functional Testing of Vision-based ML Applications
Test scenarios and test cases for an ML model and app to detect drowsiness
Drowsiness and its effects on driving form the context for this series of test scenarios and test cases. The objective is to test a computer vision-based application that identifies drowsiness by monitoring changes in driver’s face. If the ML model detects drowsiness, the application alerts the driver.
Following are the test scenarios.
Test scenario 1: Illumination conditions
Test case 1: With ambient light
First, the driver’s face and eyes must be detected accurately in all conditions in which they are illuminated by ambient light, including:
- From natural sunlight during the day
- From on-board LED lights at night
- From the lights (whether normal, high beam or fog lights) of oncoming vehicles
Test case 2: When poorly illuminated
In conditions of poor illumination, the task of accurately detecting the driver’s facial expression is of even greater importance than in Test case 1. Poor illumination is an issue at night and in night-like conditions such as driving through a tunnel or a long, unlighted underpass, where daylight cannot penetrate.
With poor illumination, it is best to use infrared-enabled cameras for capturing data from the driver’s face and eyes.
Test Scenario 2: Position of the driver’s head and face
The position and angle of the driver’s head, along with the face and eye movements, are essential to detecting drowsiness. To recognize them exactly, they must be captured in all possible directions.
Test Scenario 3: Eyewear
Drowsiness can be easily identified by analyzing the eyelid movements of the driver or by detecting whether the eyes are open or closed. The test scenario must compensate with higher precision in case the driver is using any sort of eyewear such as eyeglasses, sunglasses, spectacles or even contact lenses.
Test scenario 4: Driver behavior
Test case 1: Frequent yawning
The main indicator of drowsiness is frequent yawning, which the app should detect and track.
Test case 2: Percentage of eye closure
Generally, the movement and position (open/closed) of eyelids denote the state of drowsiness. When the eyes are half-closed or completely closed for a certain number of seconds, it’s a clear indicator of drowsiness. Therefore, the app should detect and measure the percentage of time that the driver’s eyes are closed.
Test case 3: Nodding or swinging head
It is a common sign of drowsiness when the driver’s head nods or swings regularly, so it is important to detect head movements. But it is also important to detect eyelid movements because head movements alone could be attributed to listening to music or to mere casual activity. A nodding/swinging head in combination with blinking eyes is a much more accurate indicator of drowsiness.
Test case 4: Eye color
Tiredness and lack of sleep cause the sclera, or white, of the driver’s eye to take on a reddish hue. Therefore, eye color is an important indicator of drowsiness.
Test scenario 5: Software/hardware liability
Raising the alarm without any delay is as important as detecting the state of drowsiness accurately. In a real-time application like this, no software or hardware errors should arise. When the drowsiness level exceeds a certain threshold value, the alarm should be raised without any delay.