Subjective Evaluation on Viewing Experience of Drone Videos

During aerial video capture, drone and camera parameters and relative motion between camera and target could have a major influence on viewer experience. The relationship between various scenarios, shot types and drone parameters has yet to be fully understood. In this context, researchers from the University of Bristol have designed a series of psychovisual experiments in order to characterise the optimal drone parameters (or their optimal operating envelopes) for specific scenarios and shot types through subjective evaluation. The results can be used to help the community to understand the influence of these parameters in drone cinematography.

Figure 1: A sample frame of the Cyclist-Closing Scenario (header figure).

In this subjective study, various test video clips with different drone parameters need to be first produced. These videos can be either captured using actual UAVs from real scenes or generated by simulation engines. Due to the limited time and resource available and the requirement of accurate parameter configuration, all one hundred test videos for six individual shot types and two scenarios (Cyclists and Racing Car) were generated using a popular 3D game engine, Unreal Engine (UE4). For each scenario/shot type, five different drone parameter values (either for height or for speed) were carefully selected within wide and sensible ranges.

The subjective tests were conducted in a darkened, living room-style environment on a SONY KD-65ZD9 4K HDR TV controlled by Matlab R2012a and Psychtoolbox 3.0. The experimental configuration and methodology have followed the recommendations in ITU-R BT.500. Forty subjects (21 male and 19 female) with normal or corrected-to-normal vision were invited to participate in the experiment. They were asked to rate their viewing experience from 5 to 1 (5 Excellent, 4 Good, 3 Fair, 2 Poor, and 1 Bad), following a single stimulus discrete quality evaluation method.

Figure 2: The average opinion scores of various test height versions for the Cycling-Establishing scenario.

The experimental results show that the preference of drone height and speed parameters is related to the size of object, framing and shot types. After-test interviews also indicate the preference varies for viewers with different genders. The results can be used to define optimal drone parameter (height and speed) envelopes and parameters that should be avoided or used more sparingly. The future work will focus on conducting subjective evaluation on real footage to validate the simulation approach.