Operational Data Extraction from Frontal Vehicular Camera using Computer Vision

Abstract: A data-driven understanding of how a vehicle is used can help transportation companies improve their products and provide better service to their customers. Sensors have been the usual source of the operational data of vehicles. However, in order to improve and provide new services to customers it is often necessary to understand the operation of the vehicle in new ways with collection of new data that the vehicle sensors do not capture today. We hypothesize that a camera alone would be able to capture a large number of operational variables, thereby eliminating the need of multiple, costly sensors. This project investigates the feasibility of collecting valuable operational data through image analysis of photos or videos from the on-board camera(s) and evaluate the best techniques to collect and analyse operational data from the vision of the vehicle. The goal of the project is to extract five variables- road curvature, traffic density, pedestrian density, nature of area and motion. Several experiments were carried out to determine the most suitable architecture for each of the variables as well as ensemble techniques, smoothing techniques and data storage techniques. We evaluate the models based on their performance on test frames from three different datasets. Additionally, we also evaluate the smoothing techniques based on analysis of data vectors over a short section of selected videos.