Digits Classifier — Discarded
So, today I worked on building “Digit classification AI” which is supposed to be used for the Parser / Parsing Library — it is used for splitting questions from a pdf document.
Didn’t really have any issues while building the neural network since it was such a simple problem and I had prior experience of collecting and building the dataset — it was a straightforward process for me as I had spent a month doing something similar in 2017 — maybe not so much for beginners.
The training did not take a lot of time since I was using an NVIDIA GPU laptop to train the model and the dataset was big enough that I was already about to overfit the model by the 3rd iteration.
However, I did make one mistake — how does the model detect whether something in a picture is a number or something else. We just taught the AI how to distinguish a certain set of numbers but the fundamental question still remains unsolved. I can literally give it an alphabet and the model will give me a prediction among the number which is not a good sign thus, I will have to make one change to the model to classify something as gibberish if it cannot pinpoint its numerical value. That is for tomorrow — I am so tired right now, my eyes are barely functioning.
Tomorrow will be a day filled with problems — solvable ones — although every day of mine is filled with such. I will have to fix the AI and retrain the model.
Also need to figure out if there’s any site that can host our images for free which is the main problem for this project to take off right now. If I cannot find something, the only option for me is the software way — although I will try not to go down that path.
Build the dataset of Mathematics or Physics subject. Although the current system is pretty functional with physics — all 3 papers can be classified at a near-perfect level. On the other hand, Mathematics is — messy — but the machine learning algorithm for this subject is pretty good. However, the odds are in the favor of Physics. So, it is very likely that I will build the model for Physics P1 and build the first version of the site’s image catalog. Let’s see.
I have realized that I am improving something that already kind of works. I’ve been reading a book called “The Lean Startup” in which the author wrote the initial startup should have their product not fully functional but rather meet the need of the people just about right for time being. Then start on improving it with the help of input from people. Now, the state of my project is that I can pretty much build what I think is the baseline product and then focus on improving it in the future. It might take a week to do this but I think it’s plausible.
Initially, I am considering Physics multiple-choice questions since the Parsing Library can work fully well on that and I can build the chapter classification model within 2 days — perhaps 1 but the collection of data is such a tedious process. Then build the site and launch it. Let’s see how this plan goes.
Originally published at http://manishgotame.wordpress.com on August 10, 2020.
