Applications of Intelligent SWaP3 Cameras

Applications of Intelligent SWaP³ Cameras: History

Please note this is an old version of this entry, which may differ significantly from the current revision.

Contributor:

Driven by the constantly evolving and iterative SWaP³, manufacturers have focused on some advanced directions such as miniaturization, lightweight, high performance (high-resolution, hyperspectral), and intelligence of IRCs. In applications such as surveillance globalization satellites and much strategic military equipment, the requirement for performance is unlimited. Additionally, the focus of scenarios dominated by high performance is to improve time resolution, spatial resolution, spectral resolution, and other core performance parameters such as sensitivity, frame rate, signal-to-noise ratio, as well as the degree of intelligence of the imaging devices.

infrared camera
intelligent
SWaP³

1. Introduction

At present, due to the improvement of infrared detection and imaging technology, intelligent infrared detection and imaging products gradually tend to “SWAP”, namely size, weight, and power consumption. “SWAP” along with the high-performance (such as band, spectral information, signal-to-noise ratio, dynamic range, field of view (FOV), resolution, etc.) and the low-price constitute the new concept of “SWaP³” ^[1], as shown in Figure 1. SWaP³ reflects the current market variation tendency from the pursuit of ultimate performance to product availability, usability, manufacturability, and cost performance.

Figure 1. The concept diagram of SWaP³.

Intelligent SWaP³ cameras are now widely used in military fields, including military equipment, space remote sensing (RS) of key targets, space-sensitive target surveillance, planetary exploration, and in civil fields such as medical thermal imaging and temperature measurement, agricultural production, industrial security monitoring, forestry disaster prevention, and living entertainment ^[2]. The ultimate goal of current product development is to reduce the size, weight, power, and cost while meeting the performance requirements of the camera as much as possible. In terms of spaceborne applications, the microsatellite and CubeSat, with relatively lower weight, volume, and cost, shorter development cycles, and more widely applicable scenarios, have been rapidly developed in recent years. At the same time, with the rise of new materials and the continuous improvement of the detector, electronics, optical and other technologies, the corresponding payloads aboard the satellites above have also made breakthroughs, and micro miniaturization is gradually gaining attention ^[3]. Additionally, the miniaturization of imaging payloads undertaking critical tasks is one of the major difficulties during the micro miniaturization of payloads ^[4]. In ground scenarios, more and more applications tend to the edge scene, so small size, portability, low research and development (R&D) cost, mass production, and low power consumption have become the main demands nowadays. Additionally, the requirements for lightweight, power consumption, and cost of camera systems are getting higher and higher ^[5]. Therefore, the technical methods for each element of “SWaP³” are especially critical.

There are many universities, research institutes, and companies around the world focusing on the R&D of cameras led by SWaP³. In 2011, aiming at significantly reducing the size, weight, power, and production costs of uncooled cameras ^[6], DARPA initiated a Low-Cost Thermal Imager-Manufacturing (LCTI-M) program, in which DRS provided solutions for the LCTI-M program ^[7]. In particular, in terms of material, a silicon integration scheme was adopted. For the optical system, DRS eliminated most discrete components and used chip-level optics. In electronics, DRS is partitioned according to the electronic functions and uses state-of-the-art low-power FPGAs and memory devices. Additionally, in terms of packaging technology, a new 3D wafer-level package was adopted to achieve miniaturization. Finally, the solutions mentioned above facilitated the development of the SWaP³ module. BAE Systems developed the Stacked Modular Architecture High-Resolution Thermal (SMART) chip camera to reach the LCTI-M index requirements ^[8]. The SMART chip camera is manufactured using a wafer-on-wafer process. BAE Systems used a ROIC and an ASIC chip to reduce the size, which does not affect the FPA performance. Additionally, BAE Systems used wafer-level vacuum packaging to realize miniaturization. In addition, in recent years, DARPA also proposed a number of programs related to the R&D of intelligent SWaP³ infrared imaging equipment. The programs including the limits of thermal sensors (LOTS), focal arrays for curved infrared imagers (FOCII), and Blackjack all contain the objectives developing high-performance, small size, low weight, power and price detector chip, camera or satellite imaging payload. As shown in Table 1, in addition to DARPA, the organizations, agencies, and companies with strong capabilities in development of detector and camera including Leonardo DRS, BAE, ESA, NASA, Arizona Space Institute, FLIR, Sofradir, and Semi-Conductor Devices (SCD) have done a lot of works in the R&D of SWaP³ cameras.

Table 1. Technical indicators and characteristics of typical intelligent SWaP³ infrared cameras (IRCs) around the world in recent years.

Development/ On-Board Deployment Time	Camera Name	Institution/ Company	Band	Scenes	Size, Weight, Power	Solution	Cooled or Not (Y/N)	Highlighting Technology
2010	Chang’e-2 Advanced Photo System’s (APS) landing camera ^[9]	Beijing Institute of Space Mechanics & Electricity (BISME)	VNIR	Satellite surface imaging	Size: 100 × 80 × 81 mm³ Weight: 502 g Power: 4 W	1280 × 1024 640 × 512	N	Dual mode: High resolution and high frame rate mode
2011	ATOM1024 XGA	Sofradr EC	LWIR	Large FOV, situation awareness, surveillance	Size: 61 × 69 × 66 mm³ Weight: <110 g Power: <3.7 W	1024 × 768	N	Uncooled micro-radiometer camera wide detection range
2013	Tamarisk 320 Cameras	DRS	LWIR	High sensitivity and high image quality requirements	Size: <34 × 30 × 30 mm³ Weight: <37 g Power: <1.1 W	320 × 240	N	With non-uniform correction, real-time video processing
2013	Tamarisk 640 Cameras	DRS	LWIR	High sensitivity and high image quality requirements	Size: 46 × 40 × 31 mm³ Weight: <73 g Power: <1.5 W	640 × 480	N	With non-uniform correction, real-time video processing
2014	Banpil Camera	Banpil Photonics	VIS-SWIR Multispectral	Identification and detection in all weather and visibility conditions	Size: Not mentioned Weight: Not mentioned Power: <1 W	640 × 512	N	Low dark current Multispectral imaging High-performance FPA HDR
2015	SMART Chip Camera	BAE Systems	LWIR	Edge scenarios such as handheld	Size: 2.9 cm³ Weight: 5.1 g Power: <0.5 mW	640 × 480 1280 × 1024	N	“SMART” Architecture Android GUI
2016	TEC-less SWIR Camera	UTC Aerospace Systems	SWIR	Various scenarios requiring SWIR	Size: Not mentioned Weight: Not mentioned Power: <1.4 W	1280 × 1024	N	no TEC Current mirror-type pixel
2016	Hot Blackbird XBn detector	SCD	MWIR	Surveillance, missile warning	Size: Not mentioned Weight: 730 g Power: 7.5 W	1920 × 1536	Y	10 μm small pixel size Smaller cooled device
2018	SWIR Imager based on Cardinal 640	SCD	SWIR	Day and night imaging: strong daylight and LLL	Size: 31 × 31 × 32 mm³ Weight: 50 g Power: <1.3 W	640 × 512	Y	Video engine image processing algorithm HDR LLL High Adaptation
2018	FLIR Tau 2+ ^[10]	TELEDYNE FLIR	LWIR	In-orbit observation	Size: 82 × 82 × 145.08 mm³ Weight: 422.8 g Power: <1.2 W	640 × 512	N	High sensitivity Multiple video formats
2018	CIRES cameras ^[10]	TELEDYNE e2v	VNIR	In-orbit observation	Size: 40.8 × 36.6 × 34.1 mm³ Weight: 70 g Power: 1.5 W	1280 × 1024	N	Multi-image processing algorithms High frame rate
2018	MCAMv3 camera	microcameras.space	VNIR	In-orbit monitoring on solar panels and critical loads	Size: 63.1 × 45 × 85 mm³ Weight: 327 g Power: <1.8 W	4 million	N	Embedded image compression High Resolution HDR
2020	HyperScout 2	Cosine	VNIR/LWIR Hyperspectral	Fire detection, heat island effect, etc.	Size: 1.8 L Weight: 1.8 kg Power: 11 W	4000 × 1850/1024 × 768	N	VNIR/TIR Dual channel Hyperspectral/AI Chip BEE/OBDH Subsystem
2021	Griffin-5 HOT MWIR	Attollo Engineering	MWIR	MWIR edge scenario	Size: 166 cm³ Weight: 250 g Power: <4 W Power with cooling: <10 W	640 × 512	Y	5 μm small pixel pitch sensor
2022	HyperScout M	Cosine	VNIR Hyperspectral	Fire detection, heat island effect, etc.	Size: 1.2 L Weight: 1.2 kg Power: 9 W	4096 × 1850	N	Hyperspectral AI Chip BEE/OBDH Subsystem
2022	Griffin-8 HD	Attollo Engineering	SWIR/MWIR dual Band	SWIR and MWIR edge scenarios	Size: 97 × 50 × 50 mm³ Weight: <250 g Power: <6 W Power with cooling: <10 W	1280 × 720	Y	8 μm small pixel pitch sensor
2022	Multi-Mode Tracking (MMT) Camera	Collins Aerospace	SWIR	Multimodal scenes	Size: 25 × 25 × 7 mm³ Weight: Not mentioned Power: 1.5 W	1280 × 1024	Not mentioned	Multimode: Support passive and ALPD active imaging

At present, intelligent SWaP³ IRCs are gradually applied to edge scenarios with the extensive application of AI, embedded, and other technologies. Additionally, the cameras are increasingly focused on lightweight and low power consumption while improving their performance. However, it should be noted that the lightweight and miniaturization of the camera are contradictory to the improvement of the resolution, FOV, and other performance parameters. Therefore, how to find the right balance among these elements is a critical task for the development of intelligent SWaP³ IRCs. Currently, in almost all the scenarios, the overall development idea of new intelligent SWaP³ cameras is to realize miniaturization, low power consumption, and high reliability on the premise of achieving the target function and performance, but according to different application requirements, the emphasis is different. What is more, in terms of spaceborne applications, radiation resistance should also be considered.

2. Applications

2.1. Military and Scientific Field

The applications of intelligent SWaP³ cameras in the military and scientific fields can be divided into ground-level applications and space-borne applications. The former includes surveillance and reconnaissance, detection, identification, and tracking of key targets and military weapons development, and the latter consists of RS of sensitive objects, planetary and space exploration, and spaceborne surveillance. Many devices, including reconnaissance and surveillance cameras, are gradually developing in the direction of SWaP³. Additionally, various intelligent image processing and optimization algorithms are effectively deployed at the edge.

In ground-level military systems, intelligent SWaP³ cameras are used in many scenarios, including military infrared vision enhancement equipment, maritime target surveillance, and target tracking. However, researchers focus on the space military and scientific domain. The intelligent SWaP³ cameras are widely used in the monitoring and inspection of satellites themselves (including the monitoring of solar panels, antennas, and other key payloads) and in-orbit service with visual support (including the monitoring of critical events such as docking and separation of spacecraft) ^[3]. In the ESA’s BepiColombo program, the spacecraft carried a set of BepiColombo Selfie Cameras consisting of three MCAMs (micro-cameras) developed by MCSE, which are attached to a CAM Box (an interface box). The full name of the camera is MCAMv3 Digital Space Micro-Camera, and it is a multi-functional, modular, radiation-resistant SWaP³ module suitable for harsh environments. Furthermore, the system is responsible for capturing images of solar panel deployment during the LEOP (Launch and Early Orbit Phase) and the core module MTM (Mercury Transfer Module) MEPS (Micro-Satellite Electric Propulsion System) during the cruise phase. Additionally, the system can also be used for planetary surface photography ^[11]^[12]^[13]. As shown in Figure 2, in the European radar imaging satellite Sentinel-1A of the Copernicus program, MCSE also developed a Monitoring Camera System ^[14] with three miniature cameras and control processing units to monitor the deployment of solar panels and antennas during LEOP. Furthermore, one camera of the system is dedicated to the observation of the Earth with a weight of 110 g, power consumption of 1.8 W, and a large FOV of 70°.

Figure 2. Deployment of Sentinel-1A’s cameras and images taken by the Monitoring Camera System ^[15]^[16].

What is more, in planetary research, intelligent SWaP³ cameras can be used to photograph the surface of planets or deep-space targets for research and analysis; in space debris detection and identification, SWaP³ cameras can detect foreign objects such as space debris and collect information from them to take avoidance measures or record them for research. In terms of space situation awareness, SWaP³ cameras can realize the identification and monitoring of various space targets such as high and low-orbit satellites (including active and abandoned satellites) and approaching satellites. Then the cameras obtain information and data from the above satellites, thus avoiding space collisions and conducting autonomous sensing and detection of in-orbit threats for avoidance or countermeasures. In regard to extraterrestrial planet detection, there are many intelligent SWaP³ IRCs developed for rover systems applying for exploration of Mars, the Moon, etc., which usually undertake tasks such as surface environment detection and analysis of planets ^[17]^[18]^[19].

2.2. Civil Field

Intelligent SWaP³ cameras have wide applications in the civil field, including security monitoring, assisted driving, vehicle vision enhancement, and disaster prevention. For example, microthermal imaging cameras for monitoring abnormal areas of equipment are used in daily preventive testing. Additionally, in product R&D and electronics manufacturing, developers often use high-precision detectors to test and assess. Additionally, temperature guns are used for medical temperature measurement. Moreover, infrared imaging has the characteristics of long working distance and high imaging contrast, and it can work without external light sources, and IR imaging is less affected by abnormal weather. So, the IRC can be used as an auxiliary supplement to enhance the visual effect for high-performance and miniaturized monitoring edge equipment. Huawei “XMC” (extra, magic, credible) series ^[20] intelligent SWaP³ surveillance cameras use supplementary infrared light for auxiliary visual enhancement in addition to its own super starlight imaging capabilities. This series product can realize perimeter intrusion detection and behavior analysis.

Intelligent SWaP³ cameras are also widely used in agriculture. Unispectral’s Monarch is the first portable edge-end tunable spectral NIR SWaP camera in the world ^[21]. In the camera, Unispectral’s proprietary tunable Fabry–Perot filter (μFPF) and miniature IR camera module (including optics, image sensor, and controller) are integrated together on a 60 × 40 × 14.5 mm PCB. Furthermore, the module weighs 30 g in total and consumes less than 0.85 W, simplifying spectral imaging and eliminating the need for a bulky, complex, and expensive spectrometer. Additionally, it facilitates instant detection, monitorization, and classification. It has a wide range of prospects in agricultural scenarios, such as pest control, grading, and classification, as well as industrial and commercial scenarios.

In terms of medical scenarios, intelligent SWaP³ cameras can be used to detect body temperature in a non-contact, rapid, and non-hazardous way, which is suitable for the screening of large numbers of people. In recent years, infrared thermography and TIR products have been widely used during epidemics. In addition, intelligent SWaP³ IRCs can be used to assist in surgeries or medical examinations for achieving intelligent medical treatment along with many devices equipped with high technology. For example, the infrared radiation of the human body can be captured and converted into a dedicated thermogram by a small and light camera, thus facilitating the localization and detection of lesions, which can provide a basis for clinical diagnosis. Furthermore, the smaller size and lower weight of SWaP³ cameras can be used in more complex and difficult medical scenarios such as gastrointestinal examinations. The SWaP³ cameras can achieve more clear monitoring of lesions and accomplish more reliable and on-time localization of difficult diseases such as cancer. Then, doctors can save lives by taking medical measures or doing operative planning in advance.

Moreover, hyperspectral imaging (HSI) devices are wildly used in medical fields. HSI can acquire both two-dimensional spatial information and one-dimensional spectral information of an object with a high spectral resolution, thus enabling the identification of various pathological conditions. It can provide a wealth of spectral information about patients’ tissue samples or disease sites. As a non-contact optical diagnostic technology, HSI can detect the distribution of lesions in body skin tissues, ex vivo cancer tissues, and organs such as eyes and teeth. It can play an important role in medical diagnostic scenarios such as cancer, heart diseases, retinal lesions, and dental diseases. In addition to medical diagnostic scenarios, HSI holds great promise for disease mechanism research and even surgery assistance. Compared with general imaging methods, HSI can acquire both image and spectral information, with the advantages of a wider range of bands, higher spectral resolution, and more comprehensive information acquisition. So, HSI can provide more accurate and dimensional information of the disease site in spatial and spectral dimensions. Not only can HSI obtain external information such as size and shape that can also be identified by other detection imaging methods, but HIS can also get the differentials of the internal structure. It also has a very high sensitivity. However, due to the effect of too much information and overlong processing time, it is difficult to achieve rapid diagnosis in real-time. Therefore, how to capture effective information quickly by fusing algorithms is one of the main research directions currently. In addition, there are few HSI devices and mass-produced products that are actually used for medical detection, and even the existing products are often very large in size and difficult to adapt to the needs of medical conditions. The main reason is that it is difficult to develop hyperspectral products for SWaP³-constrained platforms, which makes them inconvenient to be used in the actual diagnosis and surgery assistance. Therefore, developing SWaP³ HSI instruments with higher performance is an important direction for R&D in the field of medical devices.

This entry is adapted from the peer-reviewed paper 10.3390/s23094189

References

Ye, Z.; Chen, Y.; Zhang, P. Overview of latest technologies of HgCdTe infrared photoelectric detectors. Infrared Technol. 2014, 35, 8.
Mai, L. Outline on development and application of FPA thermal imaging sets. Infrared Technol. 2006, 28, 497–502.
Pan, C.; Cang, L.; Luo, M.; Tao, L.; Chen, S.; Chen, B.; Bai, Z.; Cui, H.; Xu, C.; Zhao, J. Development status and application of space infrared camera optical technology. Infrared Technol. 2022, 44, 9.
Jiao, B. Status and trends of development of satellite-based compact CCD cameras. Space Electron. 1995, 7, 34–40.
Ye, Z.; Li, H.; Wang, J.; Chen, X.; Sun, C.; Liao, Q.; Huang, A.; Li, H.; Zhou, S.; Lin, J. Recent hotspots and innovative trends of infrared photon detectors. J. Infrared Millim. Waves 2022, 41, 15–39.
Dhar, N.K.; Elizondo, L.A.; Dat, R.; Elizondo, S.L. Advanced imaging systems programs at DARPA MTO. In Proceedings of the Infrared Sensors, Devices, and Applications III, San Diego, CA, USA, 19 September 2013; p. 886802.
Li, C.; Han, C.; Skidmore, G.D.; Cook, G.; Kubala, K.; Bates, R.; Temple, D.; Lannon, J.; Hilton, A.; Glukh, K. Low-cost uncooled VOx infrared camera development. In Proceedings of the Infrared Technology and Applications XXXIX, Baltimore, MD, USA, 29 April–3 May 2013; pp. 465–474.
Sengupta, L.; Auroux, P.-A.; McManus, D.; Harris, D.A.; Blackwell, R.J.; Bryant, J.; Boal, M.; Binkerd, E. BAE systems’ SMART chip camera FPA development. In Proceedings of the Infrared Technology and Applications XLI, Baltimore, MD, USA, 8 June 2015; pp. 338–344.
Yue, T.; Zhang, H.; Huang, C.; Chen, X. The application of Chang’ E-2 CMOS camera technologies. Space Recovery Remote Sens. 2011, 32, 12–17.
Pack, D.; Ardila, D.; Herman, E.; Rowen, D.; Welle, R.; Wiktorowicz, S.; Hattersley, B. Two aerospace corporation CubeSat remote sensing imagers: CUMULOS and R3. In Proceedings of the 31st Annual AIAA/USU Conference on Small Satellites, Logan, UT, USA, 4–10 August 2017.
Schulz, R.; Benkhoff, J. BepiColombo: Payload and mission updates. Adv. Space Res. 2006, 38, 572–577.
Benkhoff, J.; Van Casteren, J.; Hayakawa, H.; Fujimoto, M.; Laakso, H.; Novara, M.; Ferri, P.; Middleton, H.R.; Ziethe, R. BepiColombo—Comprehensive exploration of Mercury: Mission overview and science goals. Planet. Space Sci. 2010, 58, 2–20.
Benkhoff, J.; Murakami, G.; Baumjohann, W.; Besse, S.; Bunce, E.; Casale, M.; Cremosese, G.; Glassmeier, K.-H.; Hayakawa, H.; Heyner, D. BepiColombo-mission overview and science goals. Space Sci. Rev. 2021, 217, 90.
Krag, H.; Serrano, M.; Braun, V.; Kuchynka, P.; Catania, M.; Siminski, J.; Schimmerohn, M.; Marc, X.; Kuijper, D.; Shurmer, I. A 1 cm space debris impact onto the sentinel-1a solar array. Acta Astronaut. 2017, 137, 434–443.
ESA. BepiColombo Monitoring Cameras. Available online: https://www.esa.int/ESA_Multimedia/Images/2018/10/BepiColombo_monitoring_cameras (accessed on 21 March 2023).
ESA. BepiColombo’s first images from space. Available online: https://www.esa.int/ESA_Multimedia/Images/2018/10/BepiColombo_s_first_images_from_space (accessed on 21 March 2023).
Li, F. Development of small infrared camera on board. Space Recovery Remote Sens. 2004, 25, 5.
Thangavelautham, J.; Asphaug, E.; Dektor, G.; Kenia, N.; Uglietta, J.; Ichikawa, S.; Choudhari, A.; Herreras-Martinez, M.; Schwartz, S. An Interplanetary CubeSat Mission to Phobos. In Proceedings of the 48th Annual Lunar and Planetary Science Conference, The Woodlands, TX, USA, 20–24 March 2017; p. 1707.
Schwartz, S.; Nallapu, R.T.; Gankidi, P.; Dektor, G.; Thangavelautham, J. Navigating to Small-Bodies Using Small Satellites. In Proceedings of the 2018 IEEE/ION Position, Location and Navigation Symposium (PLANS), Monterey, CA, USA, 23–26 April 2018.
Huawei. Available online: https://www.huawei.com (accessed on 21 March 2023).
Unispectral. Available online: https://www.unispectral.com/product/monarch-nir-pcb/ (accessed on 21 March 2023).

© Text is available under the terms and conditions of the Creative Commons Attribution (CC BY) license; additional terms may apply. By using this site, you agree to the Terms and Conditions and Privacy Policy.