Functional Testing of Vision-based ML Applications

Test scenarios and test cases for an ML model and app to detect drowsiness

Drowsiness and its effects on driving form the context for this series of test scenarios and test cases. The objective is to test a computer vision-based application that identifies drowsiness by monitoring changes in driver’s face. If the ML model detects drowsiness, the application alerts the driver.

Following are the test scenarios.

Test scenario 1: Illumination conditions

Test case 1: With ambient light

First, the driver’s face and eyes must be detected accurately in all conditions in which they are illuminated by ambient light, including:

  • From natural sunlight during the day
  • From on-board LED lights at night
  • From the lights (whether normal, high beam or fog lights) of oncoming vehicles

Test case 2: When poorly illuminated

In conditions of poor illumination, the task of accurately detecting the driver’s facial expression is of even greater importance than in Test case 1. Poor illumination is an issue at night and in night-like conditions such as driving through a tunnel or a long, unlighted underpass, where daylight cannot penetrate.

With poor illumination, it is best to use infrared-enabled cameras for capturing data from the driver’s face and eyes.

Test Scenario 2: Position of the driver’s head and face

The position and angle of the driver’s head, along with the face and eye movements, are essential to detecting drowsiness. To recognize them exactly, they must be captured in all possible directions.

Test Scenario 3: Eyewear

Drowsiness can be easily identified by analyzing the eyelid movements of the driver or by detecting whether the eyes are open or closed. The test scenario must compensate with higher precision in case the driver is using any sort of eyewear such as eyeglasses, sunglasses, spectacles or even contact lenses.

Test scenario 4: Driver behavior

Test case 1: Frequent yawning

The main indicator of drowsiness is frequent yawning, which the app should detect and track.

Test case 2: Percentage of eye closure

Generally, the movement and position (open/closed) of eyelids denote the state of drowsiness. When the eyes are half-closed or completely closed for a certain number of seconds, it’s a clear indicator of drowsiness. Therefore, the app should detect and measure the percentage of time that the driver’s eyes are closed.

Test case 3: Nodding or swinging head

It is a common sign of drowsiness when the driver’s head nods or swings regularly, so it is important to detect head movements. But it is also important to detect eyelid movements because head movements alone could be attributed to listening to music or to mere casual activity. A nodding/swinging head in combination with blinking eyes is a much more accurate indicator of drowsiness.

Test case 4: Eye color

Tiredness and lack of sleep cause the sclera, or white, of the driver’s eye to take on a reddish hue. Therefore, eye color is an important indicator of drowsiness.

Test scenario 5: Software/hardware liability

Raising the alarm without any delay is as important as detecting the state of drowsiness accurately. In a real-time application like this, no software or hardware errors should arise. When the drowsiness level exceeds a certain threshold value, the alarm should be raised without any delay.