Article: F. M. JAKOBS, J. FRISCHMUTH, U. KREUTZMANN, A. WERNER (GERMANY)
Flying with Night Vision Devices
Night vision devices provide the unique capability to overcome limitations of the human sensory system under low-visibility flight conditions. However, as technological progress in avionics proceeds and integrated systems on board evolve, image intensifiers and thermosensors are transforming towards highly demanding and challenging applications at the human-machine interface. Visual perception through such devices considerably differs from natural vision in the dark. This review illuminates fundamentals of night vision physiology and provides insight into basic principles of a technology that has changed military tactics in general, and military air operations in particular.
“Our night vision capability provided the single greatest mismatch of the war.”
(General Barry McCaffrey, USA, Operation Desert Storm, 1991).1
This definitely is the most impressive way to describe the importance of the capability to successfully perform military operations even in the dark and/or under poor visibility. It should, however, be borne in mind that the technological development of systems providing this capability to human beings has made significant progress since Operation Desert Storm. This article is to provide an overview of the technology and development in this field and of the aerophysiological and aeromedical aspects of flying with night vision devices (NVDs). In order to understand the physiological and psychophysical phenomena occurring when flying with NVDs under military conditions, basic knowledge of the different "night vision" technologies and their development is required which will be provided at the beginning of this article.
| ANVIS|| Aviator's Night Vision Imaging System|
| HMD|| Helmet-Mounted Display|
| HUD|| Head-Up Display|
| IMC|| Instrument Meteorological Conditions|
| I2|| Image Intensifier|
| NVG|| Night Vision Goggles|
| PNVS|| Pilot's Night Vision System|
| PAPI|| Precision Approach Path Indicator|
| SAR|| Search and Rescue|
| VMC|| Visual Meteorological Conditions|
Table 1: Most important abbreviations
Development of Night Vision Devices (NVDs)
Development of Image Intensifier Systems
The foundation for the technology of the image intensifier systems was laid during World War II when the German Wehrmacht and the Allied forces were engaged in a race to develop infrared image converters. These elementary versions of NVDs referred to as "black light sniper devices"  by the Allies operated in spectral ranges in the vicinity of visible light which is why they would rather be classified as "near infrared devices" from today's point of view; however, the basic principle was already identical with that of today's NVDs. From a military point of view, the new technology had one decisive disadvantage: The devices basically represented infrared searchlights, i.e. active reflection systems, which became useless at the moment the enemy forces were equipped with the same technology (Figure 1).
Consequently, there was a need to develop passive systems that could operate without the conspicuous infrared searchlights. This was achieved by Vladimir Zworykin , who had the idea of designing a photoelectric image converter even before the outbreak of World War II; although he had it patented as early as 1935, he only realized it in the 1950s . The result exceeded all expectations. The amplification effect, which later was termed "image intensification" (I.I., subsequently called I2), was so strong that the light reflected at night by the moon and the stars sufficed for generating a monochrome image with the wavelength of the phosphorus used for the image tube. The concept of an ambient light intensifier had become reality. However, it was to take another decade until this technology became marketable (Figure 2).
The first generation of NVDs meeting wartime requirements (GEN-1 NVD) was employed in Vietnam in 1964 when the military leaders of the US troops realized that their troops were facing an enemy that preferred individual military actions under cover of darkness for reasons of material and nominal inferiority as well as specific local knowledge. Thus, the Vietnam conflict was one of the great driving forces for the development of powerful NVDs meeting wartime requirements. Consequently, a second and a third generation of image intensifiers (GEN-2/GEN-3 NVD) were successively placed on the market, which essentially were marked by an increase in sensitivity or luminescence and a longer service life. All the same, the applications were limited to weapon-mounted telescopic sights and hand-held night telescopes for a long time, thus being used by ground forces only. It was never in doubt, however, that the air forces also had a great interest in an adaptation of the new technology. NVDs would permit flying under visual meteorological conditions (VMC), and every pilot, then and now, would prefer this to flying under instrument meteorological conditions (IMC), apart from the fact that certain missions cannot be flown under IMC. However, this required that binocular vision was maintained and freedom of hands was ensured, resulting in the necessity to design a binocular lens and to mount it on the pilots' helmet systems. This in turn took some time. The first binocular helmet-mounted displays (HMDs) were introduced into the US Air Force at the beginning of the 1970s. They were based on second-generation image intensifiers and were given the appropriate designation "aviator's night vision imaging system" (ANVIS). Most of the NVDs for pilots commonly used today are based on advancements of this prototype. This also applies to the image intensifier systems of the Bundeswehr flying units.
Development of Forward-Looking Infrared (FLIR) Systems
Parallel to this development, there was an expansion into thermal infrared ranges near the visible spectrum at the end of the 1970s. The resulting technology based on multiple-array and linear-scan detectors was so precise that it could detect miniscule temperature differences between the scanned target and its ambient temperature. This was the birth of thermal imaging cameras. Their advancement resulted in the design of a high-performance target and navigation system designated FLIR that has become an integral part of numerous military aircraft (e.g. Tiger, NH90, Apache). This system owes its designation to its capability to quasi "see through" dense smoke and fog, stirred-up dust and snow, which makes it clearly superior to conventional ambient light intensifiers. It was no coincidence that the FLIR technology experienced its first big test within the scope of the Iraq conflict. As had already been the case in Vietnam before, lessons learned – this time from the two Gulf Wars – showed scientists the way, which consequently resulted in the implementation of FLIR during Operation Desert Storm in 1991. The accident-prone dust and fog landings, particularly by helicopters , which were due to air turbulences generated by the rotors, have been outdated since then. The Tiger helicopters of the German Army Aviation forces also benefited from the technical superiority of these systems during their Afghanistan operations.
In principle, there is a distinction between three types of NVDs :
- active systems,
- ambient light intensifiers and
- thermography systems.
Each of these systems is still in use today and has its specific advantages and disadvantages.
Active illumination systems are based on a reflection of invisible light from infrared sources. The reflected light originates from the lower infrared range (700–1000 nm) directly below the visible (red) spectrum of the human eye. The resulting image is monochrome and appears in green, as in the case of the ambient light intensifiers. However, the resolution is much higher than that of passive systems because the reflection increases proportionally to the illumination and the infrared sources used today have an extremely high output . Therefore, active illumination systems will always be employed if an image reproduction as true to detail as possible is desired and/or if there is too little residual light for the use of night vision goggles (NVG). Typical examples include crime control activities, which require a high resolution for recognition purposes, or the work of security services. The disadvantage is that the infrared source can easily be detected, even by means of conventional ambient light intensifiers. Thus, for example, two infrared laser attacks on Bundeswehr helicopters reported in the course of the "Hot Blade" NATO exercise – during which pilots flew with image intensifiers for training purposes – could be traced back to their sources without any difficulties (Portugal in 2014). With regard to tactical military aspects, particularly within the scope of military aviation, this technology is of minor importance.
Ambient Light Intensifiers
Ambient light intensifiers are image converters that convert invisible or weak light into visible or enhanced visible light. A precondition for this conversion is the existence of photons, which usually originate from natural light sources like the moon and the stars and are reflected from the objects to be observed. If these photons hit an electrically charged photocathode located behind the NVD lens, electrons will be released. The quantity of these electrons is proportional to the frequency of the photons hitting the photocathode. The released electrons are amplified in a downstream vacuum cell, the so-called photomultiplier, i.e. they in turn excite additional electrons from the wall coating, causing a cascade effect. When the electrons leave the photomultiplier, they hit a phosphor screen from which they release photons with a wavelength visible for the human eye. To put it simply, the energy of photons is transferred to electrons and then back to photons again. All image intensifier units (I2 units) are based on this principle, and only the semiconductor materials of the photocathodes and the method of electron amplification have been modified in the advanced units by using gallium arsenide derivatives  or microchannel plates . The image intensifier user does not see a real image, as when looking through goggles or a lens, but a display in the interior of the unit on which the real scenery is reconstructed in real time. In practice, this results in a number of optical illusions and incorrect assessments that will be described in more detail in the section "Binocular Perception and Stereopsis". The reconstructed images are green in accordance with the wavelength of the phosphorus used. Own refraction errors can be adjusted within a range of +2.0 to -3.0 diopters by means of a rotary control so that, in principle, simple and weak corrective glasses can be compensated for. However, this only applies to spherical corrections, i.e. to hyperopia and myopia, whereas astigmatic ametropia is not adjustable in order to prevent erroneous self-corrections. Whereas pilots can compensate for the resulting imaging deficit by adaptation through appropriate contact lenses, this option is not envisaged for ground forces and non-flying personnel. A possible solution to this problem consists in correcting the refraction error by refractive surgery, which, however, is currently not supported by the Bundeswehr. The image quality of the image intensifier goggles is similar to that of former tube monitors but very susceptible to disturbances caused by a number of influencing factors. As it directly depends on the amount of residual light reflected, the image quality will become the worse the darker it is. Artificial light sources, in turn, will result in blooming effects in the form of disturbing halos explicable by the simultaneous projection of sharp and blurred images onto the retina. These effects particularly occurred in first- and second-generation units. In GEN-3 units, these effects are less distinct due to the higher image brightness . What all of the above-mentioned units have in common is constant image noise, which is caused by the continuous emission of electrons and thus is inevitable.
Thermal Imaging Cameras
Thermal imaging cameras are photoelectric image converters that convert infrared light into visible light but – in contrast to ambient light intensifiers – do not require the detour via an electron transformation. The operating principle can be traced back to the work of William Herschel, who detected the electromagnetic radiation emitted by a body due to its temperature in about 1800 and called it thermal radiation . The respective wavelength ranges range from 1–2 μm (near infrared) and 3–5 μm (mid-wavelength infrared) to8–14 μm (far infrared). The operating principle is based less on the detection of "warm" objects than on the registration of temperature differences as compared to an ambient temperature reference . From a technological point of view, thermal imaging cameras are infrared cameras operating very similarly to digital CCD or CMOS cameras designed for visible light. Image generation is effected by using cooled or uncooled thermosensors and results in two-dimensional images in the range between 2 and 14 μm (white to gray). Modern cameras are capable of resolving up to 256 gray scales (8 bits) in contrast to the human eye, which is overcharged with such a resolution. Thermal images are therefore often represented in false colors which admittedly is spectacular, but inappropriate for military purposes due to the unfamiliar appearance. Thermosensors in the form of FLIR systems  are primarily employed in combat and police helicopters (Tiger, EC135) in order to retain operational readiness even under poor visibility conditions that cause conventional image intensifier systems to fail. Some civilian transport helicopters used for SAR purposes are also equipped with FLIR, like, for example, the Super Puma of the Swiss Army, which is employed in the search for avalanche victims and missing persons. The designation "forward-looking" refers to the emission direction of the infrared radiation with regard to the aircraft axis. Sideways-looking infrared (SLIR) and downward-looking infrared (DLIR) systems have by now become available as well. The best-known military FLIR system is the PNVS (pilot's night vision system), which is the standard system used in the US Apache helicopters.
Flying with NVDs
Vision is undoubtedly one of the most important information sources for pilots during flight. An above-average vision therefore is one of the basic prerequisites for qualifying for the profession of pilot. However, vision comprises much more than most pilot candidates are aware of at the time of their initial examination. Important as a central vision of 100% is – without peripheral vision, it would not be possible to correctly assess climb and descent rates or recognize obstacles in the environment. Color and contrast vision are as relevant for detail recognition and orientation in the outside world as they are for reading off cockpit displays and noticing warning signals, and good stereoscopic vision comprises not only takeoff and landing situations, but also flying in formations (jets), tactical low-level flying and hovering (hover flights with helicopters) in difficult terrain as well as distance assessment during air-to-air refueling (jets, fixed-wing aircraft). This list is not complete, but it may stand pars pro toto for the requirements profile to be met by a military pilot today, even under visual meteorological conditions (VMC). Flying at night poses a particular challenge. In contrast to many animals, particularly nocturnal ones, which have larger pupils, differently distributed photoreceptors or special reflection systems, like the tapetum lucidum, the human eye is adapted to vision in the dark to a very limited extent only. NVDs are capable of eliminating or reducing some of these human "shortcomings", but always at the price of an optical degradation of the generated image only. Thus, night operations, whether performed with or without appropriate NVDs, always pose particular challenges to the pilot.
Central and Peripheral Vision
When light enters the interior of the eye through the pupil, it must be focused on the retina in order to generate a sharp and recognizable image. The retina is a highly differentiated, membranous tissue lining the eye from inside and accommodating millions of nerve fibers with their photoreceptors, which pass the light impression on to the brain via the optic nerve. Two types of photoreceptors are available for processing the different light modalities: cones and rods. Whereas the cone system is responsible for color vision and vernier acuity under good light conditions (photopic vision), the rod system works most effectively in the dark (scotopic vision). There are a small number of cones densely crowded in the center of sharpest vision and an enormous number of rods distributed over the entire periphery (Table 2). The joint action of both receptor types forms the basis for mesopic vision .
| || Cones|| <----->|| Rods|
| Number|| 6 million|| Mesopic|| 120 million|
| Sensitivity|| Photopic|| Scotopic|
| Highest density|| Central retina|| Peripheral retina|
| Function|| Photopic vision|| Mesopic vision|| Scotopic vision|
Table 2: Functional architecture of the neurosensory retina
The three conclusions that can be derived from the architecture of this system should be known to every pilot:
- As the area of sharpest vision only covers 1–2° of the field of view, rapid head and eye movements must be performed by daylight to ensure that the entire scenery of the outside world can be perceived (scan vision technique) . Fixation on individual points (gazing) cannot be recommended in the VMC flight mode.
- As the same retinal area is practically blind at night, neither sharp nor color vision is possible in the dark. Thus, gazing will not be a successful strategy in this case either. In order to use the rod function in an optimum way, it is advisable to sight small targets slightly eccentrically. As this will be difficult in practice, the scan technique will be helpful in case of doubt under these circumstances as well.
- Flying under twilight conditions is demanding because both receptor systems are activated but are neither fully light-adapted nor fully dark-adapted. The unconscious effort to compensate for this deficiency results in fatigue symptoms much more quickly than often assumed . Consequently, situational awareness decreases so that the risk of pilot errors increases. In this case, concentration and communication are urgently required.
The use of image intensifier goggles always results in a condition of mesopic vision. This is good because it prevents a failure of the central field of view and bad because the receptors have not yet achieved a complete dark adaptation; it is rather a compromise that must be made in order to be able to maintain a rudimentary function of the visual overall concept. From the user's point of view, there is no alternative to the scan technique due to the restriction of the field of view to approximately 40° caused by the equipment. For this reason alone, visual acuity determinations with NVG are problematic. It seems to be doubtful whether the maximum achievable visual acuity can be determined by conventional test methods at all [16, 17] because the visual fixation of the equipment display invalidates the laws of geometrical optics and results in accommodation conditions resembling myopia (equipment myopia). The results of previous studies [18, 19], which assumed a visual acuity range between 0.4 for GEN-2 and 0.5 for GEN-3 NVG, should therefore be interpreted with caution. It can be calculated, however, that a telephone pole with a diameter of 30 cm can only just be perceived from a distance of 300 m when using third-generation image intensifier goggles, whereas the same pole can still be optically resolved from a distance of approximately 1,000 m with the naked eye by day. This implies that the image resolution capability of current image intensifier units is 3 to 4 times lower than the resolution capacity of a light-adapted emmetropic human eye . However, the cables attached to such poles are to be regarded as critical because they are extremely difficult to resolve by means of NVDs and can pose a big problem, particularly for low-flying helicopters.
Color and Contrast Vision
Like contrast vision, color vision is a function of the retinal cones. Human beings are typical trichromats, i.e. they have one cone type each for the perception of blue, red or green. Under natural visibility conditions, the activation of the cones is determined by the electromagnetic wavelength of the incoming photons. If an NVD is placed in front of the human eye, this mechanism will be changed. The operating principle of the I2 converters implies that any color information regarding the incoming light is lost due to the deceleration of the photons at the photocathode. The result is a monochrome image with contours that must be resolved by means of contrast vision . The color of this image will exclusively depend on the wavelength of the light emitted by the activated phosphorus . As detail recognition is in the focus of interest for military purposes, it has been agreed to use the color for which the human eye shows the highest sensitivity in the dark-adapted condition, i.e. green. From a technological point of view, red or blue could have been selected just as well. In principle, orientation in a color-degraded, green environment is not more difficult or unpleasant than watching a black-and-white movie from the era of tube monitors. As in this "world", contrast and image brightness play the decisive parts for the "image intensifier pilot" as well. The following rule applies: The darker the night and the more low-contrast the terrain, the worse the image. The worst experience is reported from night operations in desert areas, whereas operations in vegetation-rich or hilly terrain are perceived as comparatively pleasant . Difficulties are caused by interferences of the image intensifier images with colored light . This light may come either from the outside world or from the cockpit and may cause the following problems:
Light sources outside the cockpit can be encountered particularly when flying over densely populated areas, but they usually do not interfere with the pilot's color sense. Critical color information can be found in the form of anti-collision, position and navigation lights at airports. The precision approach path indicator (PAPI) used for approach control during the final approach is of particular importance, telling the pilot whether his height of approach is correct. However, the differentiation between red ("too low") and white ("correct") is no longer possible with image identifier goggles (Figures 3 and 4).
(Comment: Photo taken at the Night Vision Training System of the Königsbrück Aviation Physiology Training Center)
- The light sources in the cockpit comprise numerous multicolor displays and control and signal lights required by the pilot for instrument flying. The risk of a direct impairment of the pilot's color sense by these light sources is rather non-existent because he can recognize most of these indications/displays by looking past the NVD optics. However, the entire cockpit lighting is shifted to an image intensifier-compatible lighting during night flights in order to ensure the dark adaptation of the remaining crew. In helicopters, this lighting usually is red (aviation red). The lighting can make it extremely difficult to recognize color saturation differences on displays or maps .
In both cases, the pilot must learn that he cannot rely on his color sense any more under image intensifier conditions. Only communication and cooperation with the copilot will help.
Color discrimination during altitude exposure is the subject of a current research project performed at the German Air Force Aviation Physiology Training Center at Königsbrück. The most important consideration in this context is the discrepancy between the use of an image intensifier-compatible lighting for the pilot and the resulting impairment of the crew's color vision, which is further reduced by exposure to an oxygen deficiency, e.g. in case of in-flight emergencies [26, 27]. In addition, the pilot must look past the image intensifier optics for reading off the cockpit displays, thus possibly impairing his own dark adaptation. It is possible that future cockpits will have to be equipped with dual lighting systems (red or green) that can be activated alternatively. First preliminary results indicate that the susceptibility to errors is highest for blue displays.
Binocular Perception and Stereopsis
In contrast to previous models, current NVDs like the Bundeswehr image identifier goggles are equipped with binocular optics (Figure 5). The advantage of this advancement is the maintenance of stereopsis, which gets lost when using monocular vision systems. To the knowledge of the authors, German pilots have so far not reported any flattening of three-dimensional contours as described by US and Israeli pilots  or headaches and double images due to dislocated optical axes in the case of binocular image intensifier goggles . It is possible that this is due to differences between the NVG of the US and Israeli Air Forces and the image intensifier goggles of the German Air Force.
What is reported by pilots of all nationalities, however, are parallax problems manifesting themselves in the fact that objects tend to appear smaller and farther away than they really are [30, 31]. It can well be assumed that the "equipment myopia" already mentioned above is responsible for this effect. If an observer fixates an image at infinity which is actually reproduced at a distance of 15 cm from his retina, the brain will take a wrong image size as the basis for distance assessment, causing an overestimation of the real conditions. The misinterpretation is ultimately based on the assumption that, according to experience, the retinal image of a remote object must represent a larger object than the equal-size retinal image of a near object (Emmert's law) . The error is inherent in the equipment and can ultimately not be eliminated.
Another problem that occurs when using ambient light intensifiers is the perception of illusory movements in the outside world . The explanation for this phenomenon does not lie in the difference in distance between the visual object and the image intensifier image, but in the difference in distance between the image intensifier image and the retina. As the image intensifier image is approximately 15 cm more peripheral than the retinal image, it will move on a larger radius than the retinal image during turns of the head, thus developing a higher angular velocity during turns of the head as well. This will lead to an illusory movement in the opposite direction (autokinesis), with the subjective impression of an "unstable" outside world . This effect can best be avoided by calm and controlled movements of the head.
The integrated HMD/HUD image intensifier systems used in the Tiger and NH90 helicopters of the Bundeswehr may lead to even more complex perceptual disturbances. An example of such disturbances is the phenomenon of hyperstereopsis [35, 36] resulting from the lateral position of the image sensors on the pilot's helmet. This almost doubles the interpupillary distance on which depth perception is ultimately based (principle of binocular disparity ). Thus, a correct principle is transmitted to the visual cortex, but in a wrong dimension. This in turn may result in complex three-dimensional illusions: During landing or takeoff maneuvers, the ground seems to rise towards the pilot, whereas there seems to be a depression in the kurtosis area, making the pilot feel as if he were sitting in a hole. Specific training methods (perceptual learning) had to be developed in order to train Bundeswehr pilots in handling this system . This similarly applies to the FLIR systems, which partly caused significant problems during their introduction due to the optical divergence between the anatomical visual axis and the images of the thermosensors generated at a distance of 2 m below the aircraft nose, resulting in the requirement for appropriate training measures.
There is a broad range of subjective experiences and individual evaluations of flying with image intensifier goggles. Whereas some pilots get used to the unfamiliar strain extremely quickly, others develop ambivalent and, in individual cases, even averse feelings.
Visual fatigue conditions belong to the most frequent psychophysical experiences and are described by pilots as a heavy fatigue immediately after the end of the flight . The reasons for this can be manifold. It seems to be understandable that continuous focusing on green images of limited quality may be unpleasant and disturbing. Likewise, the monotony of uninterrupted movements of the head in connection with the scan vision technique, taking into account the helmet system as well, can undoubtedly also result in a certain physical fatigue effect.
However, these are rather superficial causes. The two decisive factors are psychological stress and a high workload based on non-routine work . Low-level flying through unknown terrain, accomplishment of the military mission, communication with the copilot, the concern to possibly overlook a cross-country transmission line: all of this, in connection with the responsibility for the crew, possibly even in a hostile environment, is part of a multitasking mission demanding the pilot's entire ability and aeronautical experience. Vision under image intensifier conditions is not simply a matter of "vision by day – only just at night" but rather an exceptional psychological situation that must be learned and trained. Flying with limited vision is an ability requiring concentration, the capability to cope with multiple stress factors and an extraordinary degree of situational awareness. Therefore, even the most experienced Bundeswehr pilots participate in regular training measures aimed at maintaining this ability. As far as helicopters are concerned, the handling of ambient light intensifiers has already been integrated into the simulator training so that the capability profile required can be complemented and reviewed under extreme conditions as well.
Summary and Outlook
The invention of NVDs is a remarkable example of overcoming the natural limits of human sensory physiology. The consequent advancement of these NVDs under the pressure of historical key events has led to fundamental operational changes, particularly in the area of air forces. Meanwhile, advanced infrared thermosensors are available in addition to the ambient light intensifiers, extending the range of applications to the area of visibility impairments caused by dust and haze as well. It can be assumed that this development has not ended yet. Future generations will combine both systems in the form of overlaid images and will provide the option of a data display resembling the head-up displays (HUDs) . Further approaches include the enlargement of the field of view, improvement of the wearing comfort and protection against defects caused by high-energy light (laser). All the same, vision under image intensifier conditions will never reach the same quality as natural vision by day. Anyone who has ever flown with an image intensifier will always remember how pleasant it was afterwards to see the light of the sun again.
| Key Statements|
1 Barry McCaffrey served in the United States Army from 1964 to 1996. He commanded the 1st Cavalry Division in the Vietnam War and the 24th Infantry Division in the First Gulf War. At the time of his retirement, he was the youngest and most highly decorated four-star general of the United States.
Source: MCIF 1/16