Similarity metric for crowd trajectory evaluation on a per-agent basis : An approach based on the sum of absolute differences

Abstract: Simulation models that replicate realistic crowd behaviours and dynamics are of great societal use in a variety of fields of research and entertainment. In order to evaluate the accuracy of such models there is a demand for metrics and evaluation solutions that measure how well they simulate the dynamics of real crowds. A crowd similarity metric is a performance indicator which quantifies the similarity of crowd trajectories. Similarity metrics may be used to evaluate the validity of simulation models by comparing the content they produce to real-world crowd trajectory data. This thesis presents and evaluates a similarity metric which employs an approach based on the Sum of Absolute Differences to compare two-dimensional crowd trajectories. The metric encapsulates the similarity of crowd trajectories by iteratively summing time-wise positional differences on a per-agent basis. The resulting metric is simple, highly reproducible and simulatorindependent. Its accuracy in quantifying similarity is evaluated by means of a user study investigating the correlation between metric values and human perception of similarity for real and simulated crowd scenarios of varying density, trajectory, speed, and presence of environmental obstacles. The user study explores different aspects of crowd perception by dividing similarity ratings on a five-point Likert scale into four categories: overall, in terms of trajectories, speeds, and positions. Scenarios and rating categories that indicate high and low degrees of correlation between metric values and perceived similarity are identified and discussed. Furthermore, the findings are compared to previous research on crowd trajectory similarity metrics. The results indicate that the metric shows promising potential for accurate similarity measurement in simple and sparse scenarios across all rated categories. Moreover, the metric is strongly correlated with the trajectory-ratings of crowd motion similarity. However, it appears to not correlate well with the perception of overall similarity for large and dense crowds.