➤ Missed the tutorial? You may watch the whole workshop, with all sessions listed above, on youtube. Just click on the image below.

The aim of drone cinematography is to develop innovative intelligent single- and multiple-drone platform for media production to cover outdoor events (e.g. sports) that are typically distributed over large expanses, ranging, for example, from a stadium to an entire city.

The drone or drone team, to be managed by the production director and his/her production crew, will have:

a)increased multiple drone decisional autonomy, hence allowing event coverage in the time span of around one hour in an outdoor environment

and

b)improved multiple drone robustness and safety mechanisms (e.g. communication robustness/safety, embedded flight regulation compliance, enhanced crowd avoidance and emergency landing mechanisms), enabling it to carry out its mission against errors or crew inaction and to handle emergencies. Such robustness is particularly important, as the drones will operate close to crowds and/or may face environmental hazards (e.g., wind). Therefore, it must be contextually aware and adaptive, towards maximizing shooting creativity and productivity, while minimizing production costs.

Drone vision plays an important role towards this end, covering the following topics:

a)drone visual mapping and localization,
b)drone visual analysis for target/obstacle/crowd/POI detection,
c)2D/3D target tracking,
d)privacy protection technologies in drones (face de-identification).

The tutorial will offer an overview of all the above plus other related topics:

a)drone odometry and localization (IMU, GPS and other methods),
b)drone formation and flight control,
c)communication issues in drones,
d)security and ethics issues in drones.

ORBSLAM, whose source code is available under GPLv3 licence, is a paradigmatic visual SLAM system, it is able to compute, in real-time, the camera trajectory and a sparse 3D reconstruction of the scene in a wide variety of environments. The talk is focused in presenting the synergy among the main building blocks of ORBSLAM: initialization, tracking, mapping, large loop closing and relocalisation from wide baselines.

Target tracking is a very important functionality in a wide range of applications, including drone filming. There are not ideal sensors for tracking in unstructured environments: each sensor has its pros and cons. This presentation includes some previous decentralised multisensor tracking techniques developed by the presenter and discusses on different multidrone multisensor fusion schemes for efficient and accurate target tracking.

30min to grab something to drink and discuss ideas

Drones are rapidly evolving to become highly capable sensing platforms that can navigate and track trajectories with great accuracy. Flight feedback control plays a central role in providing position and velocity stabilization, as well as trajectory tracking capabilities, with robustness against measurement errors, poorly modeled dynamics and external disturbances. Feedback control methods range from linear control schemes based on linearized models to more evolved nonlinear control solutions. Under certain conditions, global stability properties can be obtained, while explicitly taking into account disturbances, such as wind and enforcing bounds on the actuation. As flight control of individual drones reaches its maturity, formation control of multiple drones is opening new research avenues. Formation control aims to drive multiple robots to achieve a group motion, while holding a reference geometric shape. This behavior is particularly suited for multiple drone audiovisual shooting scenaria, where multiple drones track a moving target, whose trajectory is not known a-priori for acquiring images from different views. Formation control solutions range from centralized to distributed, position-based to distance-based or bearing-based, considering single integrator to nonlinear vehicle dynamics, or addressing robustness issues, such as sensor or communication-related time delays or packet losses. In this tutorial, we will give an overview of existing solutions for flight control of both individual drones and multiple drones in formation, tested both indoors and outdoors and using different sensor suites.

Deep Convolutional Neural Networks (CNNs) are among the state-of-the-art techniques for Visual Information Analysis. CNNs can be used to perform several drone visual analysis tasks such as object detection and tracking, face detection and person identification, crowd detection for ensuring the safety of the flight, emergency landing point detection, etc. However, deploying such deep learning models on drones is not a straightforward task, since there are significant memory and model complexity constraints. To overcome these limitations several methodologies have been proposed:
a)training small lightweight CNNs,
b)using knowledge transfer techniques, such as neural-network distillation, layer hints and similarity embeddings, to reduce the size of CNNs
and
c)using neural region proposals for fast object detection and classification (faster R-CNN, YOLO, SSD).

Furthermore, gathering training data suitable for training the deep learning models is also a challenging task. Learning by using dataset augmentation techniques, such as hard negative and positive sample mining, can help to partially overcome this limitation, while allows us to further increase the performance of the trained models. Deep learning techniques can be also used for end-to-end drone control, allowing the deep model to control every aspect of the flight, from the visual information analysis to the drone and camera controls. Using multiple drones (multidrone setup) can increase the flexibility of drone cinematography (several multidrone use-cases are considered). Finally, there is a number of deep learning frameworks that can be used for deploying the aforementioned deep learning techniques on drones (Tensorflow, Caffe, Darknet). The above topics will be covered in the tutorial giving emphasis on specific use cases from drone cinematography.

Drones have already made their way into media production, be it cinema movies (Spectre, Captain America: Civil War etc) or  TV content (e.g. documentaries and news coverage), and they have done so for a good reason. Indeed, the versatility provided by camera-carrying drones is expected to revolutionize aerial shooting,  allowing faster and more flexible camera positioning  and movements (including low-altitude ones or shots close to the subject) than those provided by helicopters,  while at the same reducing cost and increasing safety and ease of operation. Drones are expected to enable film-makers and TV crews to develop a new cinematographic language, especially in combination with techniques that enable automated and intelligent shooting (a topic that has just started to emerge). This part of the tutorial will review characteristic  cases of drone usage in cinematography, provide a taxonomy of existing drone cinematography static & dynamic shot types and shot sequencing, delve into the new horizons that open for the creation of new visual effects and shot types and discuss technical/research  issues and challenges as well as issues related to viewer’s experience and perceived quality. It will also review recent approaches for automatic drone cinematography or approaches for the “virtual” planning drone shots. The opportunities and challenges stemming from the use of multiple drones will also be discussed.

Drones raise many important privacy issues: e.g., they should not overfly private space, they should avoid taking unsuspected persons’ closeup views. These ethics restrictions and the relevant legal provisions will be detailed in this lecture. Furthermore, relevant privacy protection tools will be overviewed, namely a) methods for face detection hindering, b) face de-identification, c) tools that detect faces, recognize humans and de-identify the faces of only these persons that are in medium close-ups or close-ups and are not actors.