DeepMACSS : Deep Modular Analyzer for Creating Semantics and generate code from Sketch

Abstract: Scientific areas such as artificial intelligence have exploded in popularity and more advanced techniques such as deep learning has been applied in various areas in order to automate tasks. As many software developers know creating prototypes can both be daunting and very time consuming. In this thesis we explore the possibility of utilizing deep learning techniques to automate the task of creating prototypes from hand drawn sketches. We will cover a method of attempting to automate daunting tasks that follow with using deep learning techniques such as labelling of data. This is automated by utilizing image editing techniques and results in both the automatic labelling of new data being efficient and the ability to create new data artificially to extend the data obtained. The thesis compares three different deep learning architectures which are trained and evaluated to obtain the best resulting model. It will also investigate how performance changes depending on data set size, pre-processing steps, architectures, and extendibility. The architectures utilize transfer learning to be able to be extended with new components without great loss to the overall performance. To read text written on the provided image optical character recognition is utilized with different pre-processing techniques to obtain the best result possible. A code generator built using template-based design is also proposed which is built using one main generator which then utilizes a language specific generator for generating the code. The reasoning behind splitting the generator into two is to provide a more extendable solution by having the language specific generator being able to be switched out for any language independent of the results of the deep learning architecture. The provided solution heavily aims on being a modular approach for the solution as a whole to be more future proof. The results show that using the proposed deep learning model might not have enough prediction accuracy to be used in a production environment. Due to the low prediction accuracy conclusions are made on how the accuracy can be increased which would lead to better results for the solution.