In: Binocular Vision |
ISBN: 978-1-60876-547-8 |
Editors: J. McCoun et al, pp. 125-137 |
© 2010 Nova Science Publishers, Inc. |
Chapter 5
EVOLUTION OF COMPUTER VISION SYSTEMS
Vladimir Grishin*
Space Research Institute (IKI) of the Russian Academy of Sciences 117997, 84/32 Profsoyuznaya Str, Moscow, Russia
Abstract
Applications of computer vision systems (CVS) in the flight control of unmanned aerial vehicles (UAV) are considered. In many projects, CVS are used for precision navigation, angular and linear UAV motion measurement, landing (in particular shipboard landing), homing guidance and others. All these tasks have been successfully solved separately in various projects. The development of perspective CVS can be divided into two stages. The first stage of perspective CVS development is the realization of all the above tasks in a single full-scale universal CVS with acceptable size, weight and power consumption. Therefore, all UAV flight control tasks can be performed in automatic mode on the base of information that is delivered by CVS. All necessary technologies exist and the degree of its maturity is high. The second stage of CVS development is integration of CVS and control systems with artificial intelligence (AI). This integration will bring two great benefits. Firstly it will allow considerable improvement of CVS performance and reliability due to accumulation of additional information about the environment. Secondly, the AI control system will obtain a high degree of awareness about the state of the environment. This allows the realization of a high degree of control effectiveness of the autonomous AI system in a fast changing and hostile environment.
* E-mail address: vgrishin@iki.rssi.ru
126 |
Vladimir Grishin |
|
|
Introduction
The computer vision systems (CVS) revealed a great evolution during the last decades. This chapter attempts to estimate the nearest perspective for its development. Further analysis will be dedicated to usage of CVS in mobile robot control systems, mainly in unmanned aerial vehicles (UAV). This problem is one of the most challenging tasks of control theory and practice. However, the main principles of this analysis are applicable to most of the different kinds of mobile robots.
Present-Day Level of CVS Development
A large number of publications is devoted to the application of CVS to different tasks of UAV flight control. The enumeration of these publications may take many pages; so here we refer to a few arbitrarily chosen papers. Let’s list the key tasks of such CVS.
•High precision navigation [1–6]. This task can be solved by means of recognition (matching) beforehand specified objects (landmarks) whose coordinates are known [1]. Another demand which is imposed on these landmarks is reliability of detection and recognition process. Reliability had to be guaranteed in conditions of possible imitation and masking. Since these landmarks are selected in advance and their reference patterns can be carefully prepared, the process of recognition (matching) can be reliably performed. Reference patterns are prepared with the account of different distances, perspective aspect angles of observation and observation conditions. Reliability of landmark recognition can be subsequently increased by joint usage of landmark images and their 3D profiles. 3D profiles are widely used for navigation of missiles of different kinds (Tomahawk cruise missiles and others). The technologies for 3D profile reconstruction are well known. For instance, the complex of the Israeli firm VisionMap [2] can be referred. The complex allows reconstructing of a 3D profile with precision about 0.2–0.3 m from the altitude of 3250 m. This complex is heavy enough and has considerable size. Some weakening of the precision requirement will allow significant decrease in weight and size. Further increasing of navigation reliability can be achieved by selection of a redundant number of landmarks in the whole area of observation. Information from CVS is usually integrated
Evolution of Computer Vision Systems |
127 |
|
|
with information from inertial navigation system. Such integration allows serious decreasing of accumulated errors of the inertial navigation system [3]. The 3D profile of observed surface can be calculated simultaneously [4, 5]. In the presence of the on-board high precision inertial navigation system, it is possible to make a 3D reconstruction in the monocular mode (with longitudinal stereo base) [6]. In this case it is possible to make a 3D reconstruction of very distant objects and surfaces.
•Flight stabilization and control of angular orientation [7–12]. This task is solved by the measurement of angular and linear UAV motion relative to the observed surface or objects [7, 8]. The set of features (points in frame with good localization) are selected and traced in subsequent frames. Observed shifts of a selected set of points are used for calculation of relative angular and linear motion. In this aspect, the star trackers should be mentioned. These devices are the specialized CVS which are used in automatic spacecrafts for measurement of angular position with high precision and reliability [9, 10]. For flight control, it is important to estimate the UAV orientations with regard to local vertical (pitch and roll angles). These estimations are used for attitude stabilization and control. CVS sensors of local vertical use algorithms of horizon line detection, recognition and tracing [11–12]. Joint usage of CVS and inertial navigation systems allows significant improvement of precision and reliability of angular orientation [7, 8]. The pose estimation and 3D reconstruction are frequently realized in single algorithm, and this process is called SLAM (simultaneous localization and mapping) [4, 5, 7].
•Near-obstacle flight [13–17]. It is a very complicated flight control task. The control system had to guarantee high precision control with a short time delay. A good example of such control is the flight control of an airplane or helicopter between buildings in an urban environment in altitude about 10–15 m [15]. Another example is the ground hugging flight or on-the-deck flight. The CVS is able to provide all necessary information for solving such control tasks. In particular, the optical flow calculation allows us to evaluate the risk of collision and to correct the direction of flight to avoid collision (obstacle avoidance). The distance and 3D profile of observed objects can be calculated by stereo pairs. The optical flow is used for estimation of flight altitude, too.
•Landing [18–26]. Landing is the most dangerous stage of flight. During this stage, UAV can crash. Moreover, some persons can be injured or property can be damaged. CVS can provide all necessary information for
128 |
Vladimir Grishin |
|
|
the control system which should realize this maneuver. This task includes the recognition of landing stripe or landing site and flight motion control relative to the landing stripe. CVS is used also for landing site selection in conditions of forced landing; see for example [19, 21 and 25]. In particular, 3D reconstruction of observed surface allows selection of the most proper landing area. In other words, CVS supports landing on unprepared (non-cooperative) sites. CVS are used for autonomous landing on the surface of Solar system planets and small bodies [22, 24]. The most complicated task is the shipboard landing [26]. There are two main difficulties. The first – the landing stripe is small. The second – the ship deck is moving.
•Detection and tracking of selected moving targets [27–30]. A target can be a pedestrian, car, ship or other moving object. The selected target can make evolutions, attempts to hide from the observation and so on. In such a complicated condition, CVS should guarantee reliable tracing of specified target. In the case of automatic tracking collapse, the CVS should use effective search algorithms for target tracking restoration.
•Homing guidance to selected objects [31]. The homing task is similar to the task of navigation. It includes search, detection, recognition and tracking on the aim object. Significant problem is the multiple changing of distance during the homing process. During this process the observed size of the tracking object changes very significantly. In such case the correlation matching technologies are used. These technologies are used in smart weapons (precision-attack bombs). Another example is the Tomahawk cruise missile which is equipped with so-called DSMAC (Digital Scene Matching Area Corellator) system. The other scene matching terminal guidance systems exist.
All these tasks have been successfully solved separately in different projects. The larger part of these projects belongs to the on-board real-time systems. The remaining part will be realized in the form of on-board real-time systems in near future.
During the last decades, great attention had been paid to the pattern recognition problem. From the flight control CVS view, the pattern recognition problem can be divided into two problems.
•Recognition of preliminary specified objects. This problem in fact is being solved in the high precision navigation task, tracking of selected objects and homing guidance.