Automating the boiling of carbohydrate food through machine learning

Abstract: There are scenarios in the modern world of today when several things are being cooked at the same time on the stove in the kitchen. You typically have a saucepan that is boiling a carbohydrate. This requires attention and can result in elevated levels of mental exertion. Would it not be useful then to aid the cooking process by removing the boiling process as a point of attention by automating the boiling process?   Food to be boiled can be identified through image recognition. There is thus a possibility to automate boiling by using machine learning. In this project machine learning is used to automate the boiling of carbohydrates. A prototype has been developed which consists of a camera and a Raspberry Pi in which a convolutional neural network (CNN) model has been implemented. The prototype can identify pasta, potato, rice, their corresponding boiling states, and give correct indication when any of them is ready. A dataset has been created from scratch, containing 5607 images that were taken and labeled, and then used to train the CNN model.   The CNN model has been evaluated through a confusion matrix applied to an image dataset which was captured by the prototype. It was also evaluated through tables of successful boiling trials. The evaluation results show that the performance of the CNN model can identify carbohydrates in limited stove scenarios. The confusion matrix shows that the precision scores are 0.846, 0.959, 0.870, 0.688 for pasta, potato, rice and "no boiling item", respectively. Recall scores are 0.967, 0.848, 0.844 and 0.681 for pasta, potato, rice and "no boiling item", respectively. But it is not sufficiently reliable to be able to work in a wide range of scenarios because of the limited dataset. It has also been shown that it is possible to use the CNN model to guide the boiling of carbohydrates. But still the dataset is not sufficiently large to quantify the error rate of the boiling system. There is potential for this type of application but further work is needed.