Earthworks Using Audio-Visual and Location-Sensing Technology: History
View Latest Version
Please note this is an old version of this entry, which may differ significantly from the current revision.
Subjects: Engineering, Civil
Contributor:
Martina Šopić
,
Mladen Vukomanović
,
Diana Car-Pušić

During earthworks, monitoring and controlling the actual productivity of construction machines enables insight into the progress of tasks, calculation of expected duration and costs, favorable use and allocation of machines, and the application of appropriate decisions and corrective measures, which is of great interest to contractors. Excavators and tipper trucks are primarily used in earthworks. Manual collection of data from the construction site to assess the actual productivity of machines is today considered an outdated, time-consuming, and subjective method.

  • earthworks
  • productivity
  • video camera
  • global positioning system (GPS)

1. Introduction

The productivity of construction machinery plays an essential role in the progress of earthworks [1]. The actual machine productivity achieved during earthworks performance significantly differs from the expected machine productivity estimated in the planning phase [2]. Tracking and monitoring of earthworks, based on collected quality data from the construction site, is necessary to detect deviations between the planned productivity and the actual productivity of the machines so that appropriate corrective measures can be taken in time and thereby reduce potential damages caused by deviations [3].
Manual (traditional) methods for collecting data from construction sites are subjective and time-consuming and can result in the delayed application of corrective measures and increased costs [4]. In recent years, to measure and evaluate the actual machine productivity on construction sites, researchers have used methods and tools of rapidly growing wireless technologies, more precisely, representatives of sensing technologies [4][5][6][7][8][9] or audio–visual technologies [10][11][12].
One of the representatives of location-sensing technologies is the global positioning system (GPS). Civil engineering researchers use GPS technology because it offers a cost-effective solution for automated data collection [13]. GPS technology is a valuable tool for earthworks [14]. Research in the field of application of GPS technology includes collecting tipper truck driving data to assess the actual productivity of tipper trucks in near real-time [6][7][8][9]. Although in numerous studies GPS technology was used as an independent tool, in most of these studies it was emphasized that GPS technology, as an independent tool, cannot meet all the requirements for solving research problems because the data is limited to time and location [15].
Video recording devices, like video cameras, are widely used on construction sites for better insight into work performance, productivity improvement, and safety monitoring [16][17]. Compared to sensing technologies, audio–visual technologies have the possibility of providing insight into the real activity of machines, thus making it easier to analyze the reasons that lead to low productivity and reducing the generation of incorrect data and conclusions [11]. Audio–visual technologies have significant potential for automated data collection from construction sites to monitor progress and safety, analyze productivity, and visually survey facilities [18].
Resource detection in audio–visual data from construction sites is a frequent research focus because it gives promising results [19]. Despite the efforts of researchers and achievements so far in the application of audio–visual technologies, further research is needed, especially in tracking and monitoring construction [20]. To solve the shortcomings of audio–visual technologies, further research should focus on the application of audio–visual technologies together with other technologies like GPS technology, radio frequency identification (RFID), accelerometers, etc. [12].

2. Audio-Visual and Loaction-Sensing Technology in Earthworks

2.1. Location-Sensing Technology for Productivity Assessment

Montaser and Moselhi [6] proposed a practical and simple system, called Truck+, for tracking, monitoring, controlling, and estimating the actual productivity of tipper trucks in earthworks in near real-time. In the Truck+ system, the integration of GPS and GIS system technology is used to calculate the duration of the tipper truck time cycle. The authors pointed out that the application of the Truck+ system can be improved by integrating with other sensing or audio–visual technologies, such as video cameras, weight, speed, or movement sensors, RFID technology, etc.
Ibrahim and Moselhi [7] proposed a method for estimating the actual productivity of tipper trucks in near real-time. The method uses the integration of GPS technology with sensors and a microcontroller to collect data on the operation of the tipper truck.
Alshibani and Moselhi [9] proposed a system for estimating the actual productivity of tipper trucks and estimating the required cost and time when performing earthworks (under the influence of tipper trucks) in near real-time. The system integrates GPS and GIS technology, consisting of five modules and four algorithms. For simplicity and efficiency of tracking, only one GPS receiver is placed on a tipper truck that drives on the same route and has the same box volume as other tipper trucks. The authors noted that their system is currently being researched and is limited to work involving tipper trucks. Also, they stated that the system applies only to the open area of operation, so there is no interference in receiving satellite radio signals. They suggest using RFID or other sensing technologies to improve the system.
Salem and Moselhi [15][21] proposed a model for tracking and monitoring the productivity of earthworks (under the influence of tipper trucks). The application of the model is the collection of data on the driving of tipper trucks to calculate the actual productivity of tipper trucks and analyze driver habits and road conditions for timely detection of unwanted driver behavior and/or road disturbances. The authors pointed out that only trial tests of the model were performed and that the model needs to be validated on an actual construction site.

2.2. Audio–Visual Technology for Productivity Assessment

Bügler et al. [10] proposed a methodology that combines two different audio–visual technologies, i.e., a combination of photogrammetry and video analysis. Their methodology aims to monitor progress and assess the productivity of earthworks in the case of extensive (deep) excavations of construction pits.
Kim J. and Chi [20] presented a vision-based framework for excavator action recognition that considers sequential pattern analysis for automated cycle time and productivity analysis. They emphasized that the experiments confirm the positive effects and applicability of the proposed framework.
Chen et al. [11] pointed out that the application of audio–visual technologies is primarily based on the automatic detection of work activities of construction machines, with little application for real problems such as tracking and monitoring the productivity of earthworks. Other limitations, which they mentioned, are difficulties in automatically detecting work activities in the case of a long video or a large number of machines on the construction site. To overcome some of the limitations, they proposed a research framework with the possibility of automatically recognizing work activities and analyzing the productivity of a large number of excavators on a construction site.
Kim J. and Chi [12] proposed a methodology for monitoring earthmoving productivity using multiple (non-overlapping) cameras on a construction site. They especially emphasized the fact that their proposed methodology is, to their knowledge, the first attempt to monitor and control the productivity of earthworks with the help of a large number of cameras and that the shortcomings of their methodology (such as overlap and tracking errors) can be solved by integrating with Internet of Things (IoT) technologies, such as GPS, RFID, accelerometers, and the like.
Chen et al. [22] presented a computer vision (CV) method to identify the leading causes of excavator and truck idling in excavator operations. They pointed out that the proposed method aims to control the work efficiency and productivity of construction machines and that the validation results are promising.
Šopić et al. [23] proposed a simple research framework for quick and practical estimates of excavator cycle time and actual productivity using a video camera at the construction site and performing video analysis. Video analysis involves labeling the excavator’s working activities using a label automation algorithm. They highlighted that the simple research framework should be integrated with non-vision-based technologies (such as GPS, RFID, accelerometers, and sensors) in further research.
Xiao et al. [24] described a vision-based method for automatically generating video highlights from construction videos by integrating machine tracking and convolutional neural networks (CNN) feature extraction. They pointed out that useful video footage is beneficial for productivity analysis and safety control. Experiments with the proposed method proved its accuracy and potential advantages.
Cheng et al. [25] introduced a novel, autonomous vision-based framework for excavator action recognition and productivity measurement based on deep learning and average cycle time calculation. They emphasized that the implementation of the proposed framework was successful, feasible, economical, and fast, with the ability to measure the productivity of the excavator in real-time.
Chen et al. [26] described a vision-based method for identifying excavators and loaders activities without pre-training or fine-tuning by adopting zero-shot learning. They pointed out that testing the proposed method for activity recognition and productivity evaluation on videos recorded from real construction sites showed feasibility and high accuracy.

2.3. Integration of Audio–Visual and Sensing Technology

Kim H. et al. [27] investigated the feasibility of measuring excavator cycle times using a smartphone-embedded inertial measurement unit (IMU). The IMU included an accelerometer and a gyroscope. The excavator’s operation was videotaped using a GoPro camera. They highlighted that the test results of the proposed research demonstrate its applicability and cost-effectiveness. In future research, they suggested the combined utilization of IMU and GPS for collecting data and monitoring equipment status.
Kavaliauskas et al. [28] compared the workflow of three unmanned aerial vehicle (UAV)-based photogrammetry techniques: real-time kinematic (RTK), post-processing kinematic (PPK), and GPS-based for efficiency, reliability, and geometric accuracy of earthwork quantity estimations.
Šopić [29] proposed an early warning system model for approaching the marginal cost-effectiveness of construction machinery during earthworks. Among other things, audio–visual (smartphone) and location-sensing (GPS) technology were used to collect data from the construction site. The application of the model on the actual construction site of the mega infrastructure project of state road construction and its verification proved the model’s innovativeness, reliability, and practicality.

3. Methodologies for Earthworks Productivity Assessment

The deterministic approach to assessing machine productivity is a simple and easy one [30]. Based on the literature review, a significant application of methodologies with a deterministic approach for assessing machine productivity can be observed, of which the methodologies contained in books, such as Peurifoy et al. [31], Nunnally [32], Nichols and Day [33], and manuals of world machine manufacturers, such as Komatsu’s specifications and application handbook [34] and Caterpillar’s performance handbook [35], stand out. Some of the researchers who used the Peurifoy et al. [31] methodology are Montaser et al. [36], Kang and Seo [37], Sarkar and Shah [38], and Singla and Gupta [39]. Furthermore, some of the research that used the Nunnally [32] methodology is at Attoh-Okine [40], Sağlam and Bettemir [41], and Sabillon et al. [42], while some of the research that used the Nicholas and Day [33] methodology is at Lewis et al. [43][44]. Finally, some of the researchers who used Komatsu’s specifications and application handbook [34] or Caterpillar’s performance handbook [35] are Bhurisith and Touran [45], Panas and Pantouvakis [46][47], Rafsanjani [48], and Pantouvakis [49].
For the purpose of estimating the productivity of construction machinery, it would be significant to compare several different methodologies for assessing productivity, thereby preventing unconditional and uncritical acceptance of the results obtained from only one methodology [46]. The theory and associated formulas for machine productivity from books and manuals of world machine manufacturers provide an excellent basis and, together with experience, serve to evaluate the productivity of machines on the construction site [50].
The productivity of earthmoving machines can be measured by the volume of excavated soil and (crumbly or broken) rock per unit of time, which can be obtained based on the number of tipper trucks that transport the excavated material in one day [51]. Therefore, the criterion for the precision of the methodologies for machine productivity can be the number of tipper truck laps from the construction site to the unloading place (with return), tracked by GPS technology. Figure 1 shows a deterministic approach to productivity assessment.
Sustainability 15 12358 g001
Figure 1. Deterministic approach to productivity assessment.

This entry is adapted from the peer-reviewed paper 10.3390/su151612358

References

  1. Salem, A.; Salah, A.; Ibrahim, M.; Moselhi, O. Study of factors influencing productivity of hauling equipment in earthmoving projects using fuzzy set theory. Int. J. Innov. Manag. Technol. 2017, 8, 151–154.
  2. Hola, B.; Schabowicz, K. Estimation of earthworks execution time cost by means of artificial neural networks. Autom. Constr. 2010, 19, 570–579.
  3. Navon, R.; Shpatnitsky, Y. Field experiments in automated monitoring of road construction. J. Constr. Eng. Manag. 2005, 131, 487–493.
  4. Montaser, A.; Moselhi, O. Tracking hauling trucks for cut-fill earthmoving operations. In Proceedings of the International Symposium on Automation and Robotics in Construction (ISARC), Montreal, QC, Canada, 11 August 2013; IAARC Publications: Edinburgh, UK, 2013; Volume 30.
  5. Montaser, A.; Moselhi, O. RFID+ for tracking earthmoving operations. In Proceedings of the Construction Research Congress 2012: Construction Challenges in a Flat World, West Lafayette, Indiana, 21–23 May 2012; American Society of Civil Engineering (ASCE) Publication: Reston, Virginia, 2012; pp. 1011–1020.
  6. Montaser, A.; Moselhi, O. Truck+ for earthmoving operations. J. Inf. Technol. Constr. 2014, 19, 412–433.
  7. Ibrahim, M.; Moselhi, O. Automated productivity assessment of earthmoving operations. J. Inf. Technol. Constr. 2014, 19, 169–184.
  8. Alshibani, A.; Moselhi, O. Automated System for Productivity Assessement of Earthmoving Operations. In Proceedings of the International Symposium on Automation and Robotics in Construction (ISARC), Bratislava, Slovakia, 25–27 June 2010; IAARC Publications: Edinburgh, UK, 2010; Volume 27.
  9. Alshibani, A.; Moselhi, O. Productivity based method for forecasting cost & time of earthmoving operations using sampling GPS data. J. Inf. Technol. Constr. 2016, 21, 39–56.
  10. Bügler, M.; Borrmann, A.; Ogunmakin, G.; Vela, P.A.; Teizer, J. Fusion of photogrammetry and video analysis for productivity assessment of earthwork processes. Comput. Aided Civ. Infrastruct. Eng. 2017, 32, 107–123.
  11. Chen, C.; Zhu, Z.; Hammad, A. Automated excavators activity recognition and productivity analysis from construction site surveillance videos. Autom. Constr. 2020, 110, 10304.
  12. Kim, J.; Chi, S. Multi-camera vision-based productivity monitoring of earthmoving operations. Autom. Constr. 2020, 112, 103121.
  13. Lu, M.; Chen, W.; Shen, X.; Lam, H.C.; Liu, J. Positioning and tracking construction vehicles in highly dense urban areas and building construction sites. Autom. Constr. 2007, 16, 647–656.
  14. Morley, D.; Lu, M.; AbouRizk, S. Identification of invariant average weighted haul distance to simplify earthmoving simulation modeling in planning site grading operation. J. Constr. Eng. Manag. 2014, 140, 04014057.
  15. Salem, A.; Moselhi, O. AI-based cloud computing application for smart earthmoving operations. Can. J. Civ. Eng. 2021, 48, 312–327.
  16. Chi, S.; Caldas, C.H. Automated object identification using optical video cameras on construction sites. Comput. Aided Civ. Infrastruct. Eng. 2011, 26, 368–380.
  17. Kim, J.; Ham, Y.; Chung, Y.; Chi, S. Systematic camera placement framework for operation-level visual monitoring on construction jobsites. J. Constr. Eng. Manag. 2019, 145, 04019019.
  18. Kim, H.; Ham, Y.; Kim, W.; Park, S.; Kim, H. Vision-based nonintrusive context documentation for earthmoving productivity simulation. Autom. Constr. 2019, 102, 135–147.
  19. Roberts, D.; Golparvar-Fard, M. End-to-end vision-based detection, tracking and activity analysis of earthmoving equipment filmed at ground level. Autom. Constr. 2019, 105, 102811.
  20. Kim, J.; Chi, S. Action recognition of earthmoving excavators based on sequential pattern analysis of visual features and operation cycles. Autom. Constr. 2019, 104, 255–264.
  21. Salem, A.; Moselhi, O. Automated monitoring and assessment of productivity in earthmoving projects. Can. J. Civ. Eng. 2018, 45, 958–972.
  22. Chen, C.; Zhu, Z.; Hammad, A.; Akbarzadeh, M. Automatic identification of idling reasons in excavation operations based on excavator–truck relationships. J. Comput. Civ. Eng. 2021, 35, 04021015.
  23. Šopić, M.; Vukomanović, M.; Car Pusić, D.; Završki, I. Estimation of the excavator actual productivity at the construction site using video analysis. Organ. Technol. Manag. Constr. Int. J. 2021, 13, 2341–2352.
  24. Xiao, B.; Yin, X.; Kang, S.C. Vision-based method of automatically detecting construction video highlights by integrating machine tracking and CNN feature extraction. Autom. Constr. 2021, 129, 103817.
  25. Cheng, M.Y.; Cao, M.T.; Nuralim, C.K. Computer vision-based deep learning for supervising excavator operations and measuring real-time earthwork productivity. J. Supercomput. 2023, 79, 4468–4492.
  26. Chen, C.; Xiao, B.; Zhang, Y.; Zhu, Z. Automatic vision-based calculation of excavator earthmoving productivity using zero-shot learning activity recognition. Autom. Constr. 2023, 146, 104702.
  27. Kim, H.; Ahn, C.R.; Engelhaupt, D.; Lee, S. Application of dynamic time warping to the recognition of mixed equipment activities in cycle time measurement. Autom. Constr. 2018, 87, 225–234.
  28. Kavaliauskas, P.; Židanavičius, D.; Jurelionis, A. Geometric Accuracy of 3D Reality Mesh Utilization for BIM-Based Earthwork Quantity Estimation Workflows. ISPRS Int. J. Geo-Inf. 2021, 10, 399.
  29. Šopić, M. Early Warning System of Approaching the Marginal Cost-Effectiveness of Construction Machinery during Earthworks. Ph.D. Thesis, University of Zagreb, Faculty of Civil Engineering, Zagreb, Croatia, 2022.
  30. Han, S.; Lee, S. Quantified comparison and analysis of different productivity measurements. J. Asian Archit. Build. Eng. 2008, 7, 309–316.
  31. Peurifoy, R.; Schexnayder, C.; Schmitt, R.; Shapira, A. Construction Planning, Equipment, and Methods; McGraw-Hill Education: New York, NY, USA, 2006.
  32. Nunnally, S.W. (Ed.) Construction Methods and Management; Prentice-Hall: Hoboken, NJ, USA, 2007.
  33. Nichols, H.L., Jr.; Day, D. Moving the Earth: The Workbook of Excavation; McGraw-Hill Education: New York, NY, USA, 2005.
  34. Komatsu. Specifications and Application Handbook, 31st ed.; Komatsu: Tokyo, Japan, 2013.
  35. Caterpillar. Performance Handbook, 49th ed.; Cat® Publication by Caterpillar Inc.: Peoria, IL, USA, 2019.
  36. Montaser, A.; Bakry, I.; Alshibani, A.; Moselhi, O. Estimating productivity of earthmoving operations using spatial technologies. Can. J. Civ. Eng. 2012, 39, 1072–1082.
  37. Kang, S.; Seo, J. GIS method for haul road layout planning in large earthmoving projects: Framework and analysis. J. Constr. Eng. Manag. 2013, 139, 236–246.
  38. Sarkar, D.; Shah, D. A Framework for Application of Genetic Algorithm in Productivity Optimization of Highway Equipment Using Evolver Software. Eur. Int. J. Sci. Technol. 2013, 2, 151–163.
  39. Singla, A.; Gupta, P.K. Cost and Productivity Analysis of Equipments for Flexible Pavement-A Case Study. Int. Res. J. Eng. Technol. 2016, 3, 965–969.
  40. Attoh-Okine, N.O. Rule induction in productivity analysis: A rough set approach. Eng. Constr. Archit. Manag. 2003, 10, 413–417.
  41. Sağlam, B.; Bettemir, Ö.H. Estimation of duration of earthwork with backhoe excavator by Monte Carlo Simulation. J. Constr. Eng. 2018, 1, 85–94.
  42. Sabillon, C.; Rashidi, A.; Samanta, B.; Davenport, M.A.; Anderson, D.V. Audio-based bayesian model for productivity estimation of cyclic construction activities. J. Comput. Civ. Eng. 2020, 34, 04019048.
  43. Lewis, P.; Leming, M.; Rasdorf, W. Impact of engine idling on fuel use and CO 2 emissions of nonroad diesel construction equipment. J. Manag. Eng. 2012, 28, 31–38.
  44. Lewis, P.; Rasdorf, W.; Frey, H.C.; Leming, M. Effects of engine idling on national ambient air quality standards criteria pollutant emissions from nonroad diesel construction equipment. Transp. Res. Rec. 2012, 2270, 67–75.
  45. Bhurisith, I.; Touran, A. Case study of obsolescence and equipment productivity. J. Constr. Eng. Manag. 2002, 128, 357–361.
  46. Panas, A.; Pantouvakis, J.P. Comparative analysis of operational coefficients’ impact on excavation operations. Eng. Constr. Archit. Manag. 2010, 17, 461–475.
  47. Panas, A.; Pantouvakis, J.P. Efficiency multipliers for construction productivity: A Comparative Evaluation. Organ. Technol. Manag. Constr. Int. J. 2015, 7, 1186–1196.
  48. Rafsanjani, H.N. Comparison between Nominal and Actual Hourly Production of Crawler-Type Dozer: A Case Study. Comput. Civ. Eng. 2015, 2015, 17–24.
  49. Pantouvakis, J.P. Data Analysis and Design of Construction Productivity Efficiency Multipliers. In International Conference on Engineering, Project, and Product Management; Springer: Cham, Switzerland, 2017; pp. 119–128.
  50. Pantouvakis, J.P.; Maravas, A. Composite Criticality in Machinery Fleet Management of Construction Projects. MATEC Web Conf. EDP Sci. 2020, 312, 02003.
  51. Belayutham, S.; González, V.A.; Yiu, T.W. Lean-based clean earthworks operation. J. Clean. Prod. 2017, 142, 2195–2208.
More
© Text is available under the terms and conditions of the Creative Commons Attribution (CC BY) license; additional terms may apply. By using this site, you agree to the Terms and Conditions and Privacy Policy.
This entry is offline, you can click here to edit this entry!
ScholarVision Creations
Feedback