1. ZED Depth Sensors

The ZED depth camera is a passive depth ranging tool without an active ranging device because it does not contain an IR laser projector. This depth camera employs a binocular camera to capture 3D scene data, measures the disparity of the objects using a stereo matching algorithm, and finally calculates the depth map according to the sensor parameters [16,17,18,19,20].

The ZED depth sensor is composed of stereo cameras with a video resolution of 2560 × 1440 pixels (2K) with dual 4 mega-pixel RGB sensors. The two RGB cameras are at a fixed base distance of 12 cm. This base distance allows the ZED camera to generate a depth image up to 20 m (40 m is the maximum distance in the new updated firmware, according to the Stereolabs datasheet) [16]. The camera contains a USB video device class supported USB 3.0 port backward compatible with the USB 2.0. standard. It should be noted that the ZED 3D depth sensor is optimized for real-time depth calculation using NVidia Compute Unified Device Architecture technology [16]. Therefore, a corresponding graphical processing unit (GPU) and appropriate computer hardware are required to use it [16,17,18].

The ZED sensor uses wide-angle optical lenses with a FOV of 110°, and it can stream an uncompressed video signal at a rate up to 100 fps in Wide Video Graphics Array (WVGA) format. The depth image is provided with a 32-bit resolution. Hence, the camera gives a very accurate and precise depth image that describes the depth differences, i.e., the different distances from the plane of the camera. Right and left video frames are synchronized and streamed as a single uncompressed video frame in the side-by-side format. Various configurations and capturing parameters of the on-board image signal processor, such as brightness, saturation, resolution, and contrast, can be adjusted through the SDK provided by Stereolabs [16]. Furthermore, ZED devices support several software packages, called “wrappers,” such as ROS, MATLAB, Python, etc. All these software packages allow the modification of different parameters depending on the user requirements, such as the image quality, depth quality, sensing mode, name of topics, quantity of frames per second, etc. [16,17,18,19,20]. Figure 3 shows the ZED depth sensors [16].

Figure 4 presents the accuracy graph of the ZED depth sensor depending on the distance of an object from the depth camera. As shown, the depth resolution, i.e., the depth precision, is impaired with increasing distance [16].

Finally, it should be remarked that the ZED depth sensor comes with a unique factory calibration file, which is downloaded automatically. The recommendation is to use the Stereolabs factory settings, but users can also calibrate the ZED sensor with the ZED SDK software package [16,17,18,19,20].

The new ZED 2i depth camera has some similarities and shares some properties with the previous version of the ZED depth camera. However, the new ZED 2i sensor includes several significant improvements.

ZED 2i is the first stereo depth camera that uses artificial neural networks (ANNs) to reproduce human vision, bringing stereo perception to a new level [16]. It contains a neural engine that significantly improves the captured depth image or depth video stream. This ANN is connected to the image DSP, and they contribute jointly to creating the best possible depth map [16]. Furthermore, the ZED 2i camera has a built-in object detection algorithm [16]. This algorithm detects objects with spatial context. It combines artificial intelligence with 3D localization to create next-generation spatial awareness [16]. There is also a built-in skeleton tracking option that uses 18x body key points for the tracking application. The algorithm detects and tracks human body skeletons in real time. The tracking result is displayed via a bounding box, and the algorithm works up to a 10 m range [16].

Next, this ZED 2i camera has an enhanced positional tracking algorithm that is a significant improvement suitable for robotic applications [16,17,18,19,20,21]. This benefit arises from a wide 120° angle FOV, advanced sensor stack, and thermal calibration for greatly improved positional tracking precision and accuracy [16]. The ZED 2i depth camera has a built-in inertial measurement unit, barometer, temperature sensor, and magnetometer. All these sensors provide extraordinary opportunities for easy and accurate multi-sensor capturing. These sensors are factory calibrated on nine axes with robotic arms [16]. The data rates of the position sensors, i.e., the barometer and magnetometer, are 25 Hz/50 Hz. The built-in motion sensors, accelerometer, and gyroscope contribute significantly to the development of robotic applications since there is no need to install any other sensor except the ZED 2i camera itself. The data rate of these sensors is 400 Hz. The thermal sensors monitor the temperature and compensate for the drifts caused by heating. In this way, gathered real-time synchronized inertial, elevation, and magnetic field data along with image and depth are collected. These sensors also contribute to accurate and precise depth sensing. The all-aluminum frame reduces the camera heating that induces changes in focal length and motion sensor biases [16]. The case in aluminum allows to better dissipate the internal heat generated by the electronic components reducing the internal temperature of the depth camera. Additionally, the case deformations cannot affect the measure of the depth in any way because the lenses do not move. Furthermore, the software is factory calibrated in order to use the temperature information provided by the internal sensors and modify the data accordingly [16]. All the mentioned motion and positional features indicate that the ZED 2i depth camera is extremely suitable for development of autonomous and industrial robotic applications [21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40].

Figure 5 shows the accuracy graph of the ZED 2i depth sensor, depending on the distance of an object from the depth camera. As shown, the depth resolution, i.e., the depth precision, decreases with increasing distance [16]; however, for instance, at the 1 m range, the accuracy is better than that of the previous ZED, as seen in Figure 4.

One of the most important improvements in ZED 2i is the inclusion of new ultra-sharp eight-element all-glass lenses able to capture video and depth up to a 120° FOV, with optically corrected distortion and a wider ƒ/1.8 aperture, which allows capturing 40% more light [16]. Furthermore, the ZED 2i has an optional feature: the polarizing filter. This built-in polarizing filter gives the highest possible image quality in various outdoors applications. This lens helps to reduce glare and reflections and increases the color depth and quality of the captured images [16]. The effect of the polarizer can be seen in Figure 6.

The ZED 2i stereo camera also has two lens options. It is possible to choose between a 2.1 mm lens for a wide FOV or a 4 mm lens for increased depth and image quality at long range, according to the manufacturer [16]. However, the 4 mm lens option entails delays in delivery [16]. These are principal features related to all depth cameras since the lenses, aperture, and light significantly affect the image quality of any camera, not only the depth camera. These features enable high-quality depth images to be obtained, upon which promising robotic vision applications can be developed [41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59]. According to the Stereolabs [16], the main features of ZED cameras are listed in Table 2.

Furthermore, the ZED 2i has an Ingress Protection 66 (IP66)-rated enclosure that is highly resistant to humidity, water, and dust, since the ZED 2i depth sensor is designed for outdoor applications and challenging industrial and agricultural environments, etc. [16], while the ZED camera, such as the D415 and D435 RealSense cameras, is mainly designed for indoor applications [7]. Moreover, the ZED 2i has multiple mounting options and a flat bottom, and it can be easily integrated into any system and environment [16].

Since the ZED 2i can be cloud-connected, there is an option to monitor and control the camera remotely. Using a dedicated cloud platform, capturing and analyzing the 3D data of the depth image is possible from anywhere in the world [16]. It is also possible to monitor live video streams, remotely control the cameras, deploy applications, and collect data.

Finally, it should be mentioned that there is also a ZED two-depth camera provided by Stereolabs [16]. The main features and the accuracy graph of the ZED 2i are the same as those of the ZED 2 as the internal sensor stack and the lens configuration of the two sensors are the same [16]. The only differences are the external enclosure for the ZED 2i, which is IP66 and hence more robust, and the option of a built-in polarizing filter [16].

The essential features of ZED and ZED 2i depth devices are summarized and compared in Table 2 [16].