Construction of the Berkeley Innovation Index: A Higher-Order Item Response Theory Model Approach

Abstract: The Berkeley Innovation Index (BII) is a tool developed for assessment of individual innovation capability. The index is based on responses to a survey that constitutes of questions linked to domain abilities, i.e., sub-traits, that are hypothesized to govern an individual's overall innovation ability. The underlying algorithm for the BII produces a score representing the test-takers' proficiency on the domain ability continua as well as a score associated with their general innovation ability. In this thesis, the algorithm for the BII is constructed by applying a Higher-Order Item Response Theory model for hierarchical latent trait estimation. Simultaneous estimation of the vast amount of model parameters is done by employing a Markov Chain Monte Carlo (MCMC) method that utilizes a multi-level bayesian inference sampling technique. The validity, feasibility, and usefulness of the approach is analyzed throughout the thesis. The statistical relevance of the obtained results is evaluated by examining the Deviance Information Criteria, the Item Response Theory Information Criteria, the posterior predictive values, different convergence criteria for the MCMC chains etc. In order to reduce the amount of questions, and make the index more user-friendly, feature selection techniques are applied to explore the possibility of discarding items that contribute with the least amount of information. An easily implementable and scalable algorithm is presented, and the advantages/disadvantages of the acquired model are discussed. Lastly, recommendations on how to further improve the Berkeley Innovation Index are proposed.