Digital ICE: Defect Detection and Correction Using Infrared-enabled ScannersDr. Gabriel Fielding, Eastman Kodak Company
Dust-busting and scratch correction are a familiar part of film-restoration projects. There are huge numbers of movies in archives and film vaults and there is a growing demand for more content to fill hundreds of high-definition channels available to viewers of cable television at home. But not all films have been stored properly over the years; for some films, the accumulated dust and scratches make a full restoration too expensive to attempt. Most of this expense occurs because high-quality correction of defects can be a time-consuming and tedious process. And most automatic software tools still require a considerable amount of human interaction. Thus, there is a desire to improve the speed and quality of restoration tasks required to bring these films "back to life."
Recent advances in infrared scanner technology as well as improved image processing algorithms have dramatically reduced the time needed to repair dust and scratches on motion picture film. In the marketplace, this combined hardware/software technology is often known by its marketed name "Digital ICE," where ICE is an acronym for "image correction and enhancement." This technology has already made a significant impact on restoration workflows around the world and may result in more feature films being restored in the future.
How it works
Infrared (IR) enabled film scanners have been available in the consumer world for several years. These scanners use an IR illumination source and a sensor capable of measuring into the IR range to provide an additional channel along with the traditional red, green, and blue (RGB) channels. This IR channel is subsequently analyzed to detect minor defects in the scanned film image. These defects are then corrected by a robust filtering method.
The principal behind IR scanning is relatively simple: if the base and color dyes in color negative film absorb no IR light, then any deviation in an IR image is caused by defects (dust and scratches). Scratches and small dust tend to scatter incident light rather than block it completely; hence, some color information still reaches the sensor in the film scanner. Image processing algorithms can analyze the residual color information in "defective pixels" and make a correction based on the estimated scattering of light.
Figure 1 shows pictorially the process by which defects may be detected using IR. If light passing through the film strikes a defect, then the amount of light reaching the sensor is reduced. Under ideal circumstances, a processed film frame will transmit a constant level of IR light when illuminated with an IR source. Defects will transmit some lesser amount of IR light. A simple threshold can be used to separate defective pixels by comparing the measured IR level to the expected constant IR level. Determination of the threshold parameter is a function of measured noise in the IR channel as well as a function of the desired false alarm/missed detection ratio. This value can be chosen during the hardware design and calibration phase.
The motion picture community has a much higher quality requirement than average consumers. Implementing Digital ICE on motion picture film scanners required improvements in both quality and speed. Working with film scanner manufacturers, Kodak was able to identify and implement hardware and software requirements that would provide unsurpassed defect detection and correction.
HardwareDigital ICE requires that a film scanner have the following characteristics:
The techniques used to achieve these requirements will depend on the scanner system hardware. The quality of defect detection and correction is largely determined by how well the hardware can meet the requirements listed above. Some variation from the ideal can be compensated by software but this typically requires increased computational time.
The Digital ICE software is designed to partially compensate for hardware that fails to achieve the ideal assumptions listed above. One significant issue that must be dealt with is that the dyes in most color negative films have some response into the IR range. The dominant contribution to this IR response is from the cyan dyes whereas the spectral density of the magenta and yellow dyes is negligible in most color films. The impact of absorption of light in the IR range by the cyan dye is that the IR channel often shows some image content, referred to as leakage, which must be removed prior to defect detection and correction.
Figure 2 shows the spectral dye density curves for Kodak's 5229 Expression 500T film. The red curve indicates that the cyan dye absorbs well into the infrared while the green and blue curves (magenta and yellow dyes) are virtually zero.
A mixture model for IR absorption by both defects and by cyan dye can be used to eliminate crosstalk prior as seen in Figure 3. Characterization of each film stock provides estimates for the leakage; thus we can de-correlate the IR channel from the scanned color records.
The power of Digital ICE can be seen clearly through some examples. Figure 4 shows a "before" and "after" pair of images in panels (a) and (b). These samples, from color negative film, were intentionally scratched to simulate the effects of improperly handled color negative film rolls. The images below have been inverted and color-adjusted for the purposes of display/printing, hence defects appear lighter in the images. It can be seen in the "after" image on the right that the scratches are virtually gone and that all image details have been preserved.
Figure 5 shows another "before" and "after" pair of images. Clearly, Digital ICE has corrected a significant amount of light loss caused by scratches and dust on the film. It should be pointed out that these films were intentionally scratched and allowed to accumulate dirt and dust for testing purposes. The amount of defects seen in Figures 4 and 5 are more representative of very old film that has not been handled properly.
Many of the real images encountered in restoration workflows have much less obvious defects. In fact, some images when viewed as still frames appear to have no defects at all; but when viewed as a movie, the scratches and dust are far more visible. Figure 6 shows an original frame of film on the left and then shows the defect matte on the right revealing the dust and scratches detected by Digital ICE. Correcting even these seemingly "invisible" defects will result in far superior image quality.
Infrared enabled dust busting and scratch removal have become a reality with the extension of Digital ICE technology to motion picture film scanners. The marriage of superior hardware design with robust signal processing algorithms allows for the automatic detection and correction of a wide variety of film artifacts. Moreover, the software is capable of indicating, through the defect matte, those defects that are severe enough to require manual intervention. This will significantly reduce the costs of dustbusting and scratch removal while improving overall image quality.