Construction equipment activity recognition for simulation input modeling using mobile sensors and machine learning classifiers

Although activity recognition is an emerging general area of research in computer science, its potential in construction engineering and management (CEM) domain has not yet been fully investigated. Due to the complex and dynamic nature of many construction and infrastructure projects, the ability to detect and classify key activities performed in the field by various equipment and human crew can improve the quality and reliability of project decision‐making and control. In particular to simulation modeling, process‐level knowledge obtained as a result of activity recognition can help verify and update the input parameters of simulation models. Such input parameters include but are not limited to activity durations and precedence, resource flows, and site layout. The goal of this research is to investigate the prospect of using built‐in smartphone sensors as ubiquitous multi‐modal data collection and transmission nodes in order to detect detailed construction equipment activities which can ultimately contribute to the process of simulation input modeling. A case study of front‐end loader activity recognition is presented to describe the methodology for action recognition and evaluate the performance of the developed system. In the designed methodology, certain key features are extracted from the collected data using accelerometer and gyroscope sensors, and a subset of the extracted features is used to train supervised machine learning classifiers. In doing so, several important technical details such as selection of discriminating features to extract, sensitivity analysis of data segmentation window size, and choice of the classifier to be trained are investigated. It is shown that the choice of the level of detail (LoD) in describing equipment actions (classes) is an important factor with major impact on the classification performance. Results also indicate that although decreasing the number of classes generally improves the classification output, considering other factors such as actions to be combined as a single activity, methodologies to extract knowledge from classified activities, computational efficiency, and end use of the classification process may as well influence one’s decision in selecting an optimal LoD in describing equipment activities (classes).