IdentiBreed 0.3.0

GitHub   Click to open the GitHub repository

Table of Contents:

• Introduction
• Exploring the GUI
• Evaluation prompt
• Evaluation = False
• Evaluation = True
• Current State of the App
• Upload Your Log Files!

Introduction

IdentiBreed is my first major project which started as a notebook I worked on during a machine learning and deep learning course. However the people behind it went onto other topics right after completing the workflow. Don't get me wrong - working alongside someone when creating my first neural network was a great way to catch the basic ideas and steps it takes to build one. They took me from making tensors, through set up process and then experimenting with the model. However I decided that just creating the network isn't enough. I had a sense that it deserved to be interacted with, it just had to be able to work with and for the user. That's where I stepped in and took this simple learning experience into the next level.

Say hello to the launch screen of the IdentiBreed in its 0.3.0 version. This is an app created entirely by one person - me. It is a dog breed identifier powered by transfer learning neural network. Thanks to Tensorflow Hub I imported a MobileNet V2 model and trained it on the Kaggle Dog Breed Identification dataset. Currently, the app only detects breeds within the 120 present in said dataset.

Exploring the GUI

Operating the app is fairly straight forward. The UI is divided into labeled boxes, let's discuss each of them here:

• Input Options - Users can decide on using one of three options on feeding the model images to predict on. There is the first option "Input Folder" which means a folder located within the IdentiBreed files made specifically for the user to put images there. Second option "Specific" allows the user to either specify a directory or singular file to predict on. Third option "Web Image" is an empty text field into which the user can paste a URL to an image they'd like to make prediction on.
• Predict Breeds - Here is a string of text and a button which makes the model create predictions. However if the input folder/specific directory is empty, there are no files to predict on or the URL is broken that's where you'll get an appropriate notification.
• Select Save Method - This box becomes active only when the predictions were correctly made. Here you can choose one of three options - you can save all predictions, save none of them or hand pick which predictions to save. Last button opens the output directory, that is a folder within the IndetiBreed directory. Once selected, the options get disabled for the current viewing session and are reset when new predictions are made (even on the same images).
• Navigation Controls - Here you can flick through all predictions made on the images given. There is a counter of the images available, and if the user selected the "Select Manually" save method a button to save image appears. In case of saving files the Navigation Bar is also a place where the path to the saved prediction is displayed.

Evaluation Prompt

Before we dive into displaying the predictions the app asks the user if they want to contribute to the development of IdentiBreed. It looks a little something like this...

This is a one time question, that happens on the very first prediction made by a new user. Simultaneously the app creates the config.ini file, which (for now) contains only the version number, users ID (a 16 character long alpha-numeric string) and the evaluation value (True/False). Currently removing the config.ini file or manually setting the value of "Evaluation" are the only ways to change your decision. The settings tab is comming in one of the future updates. Becuase of the binary options down below we will look at what happens for both options:

Evaluation = False

Let's say you've decided that you don't want to spend your time evaluating the predicted labels. That's okay! Maybe you'll change your mind in the future. After deciding on the saving method you are greeted with one of these - a displayed prediction and the top 5 possible breeds! You can navigate through the predictions using the "< Prev" and "Next >" buttons. At the time of this screenshot I decided to manually save each predictions that I wanted, and that's why at the bottom of the navigation bar I was asked if I wanted to save an image.

That's really all that there is to it! You can give the app different pictures and just play around seeing if the predictions are correct. Who knows, if you've taken a picture of a dog yourself maybe it will tell you the correct breed... If not, well, that's why I ask for help at the first run - there are only 120 breeds in the Kaggle dataset. I will do my best to expand it with more breeds, but I'm just a single man and help from other developers will go a long way! Speaking about help, let's see how the experience differs for a user that wants to spend some time calling the breeds right or wrong!

Evaluation = True

This is where you get to experience sort of a "secret" menu. An easter egg if you will. Let's take a look at what changes in the app after the prediction is displayed:

As you can see you can't just skip to the next prediction now. Before the app will allow it you have to tell it if the prediction is correct, incorrect or if you simply don't know the breed yourself. This one displayed above is the right one, so if we click "Yes" the app will add this to the log file and kindly display the next prediction for you. If you don't know the actual breed that's alright - the app will keep that in mind and move you along to the next prediction as well. Let's see now what happens if you do know the breed, but the AI was incorrect!

This time the bottom half of the evaluation window becomes available for interaction. In this combobox you can check if the correct breed appears in the 120 currently recognized. If it's there just select it and click "Done". If it's not there however you can type the correct breed into this combobox and it will be saved into the log file. Simple as that! Right then, now let's see what happens after the very last image is evaluated:

And that's how the ending looks. A new button appears and when clicked it saves the log file in the "/user/evaluation_logs" directory. I call it a "log file" while in reality it's a .csv file which we'll discus below. At the same time a tab is opened in your browser that redirects you to this very page, just a tad lower. You can add your evaluation log file and upload it, which adds it into the database. Let's discus this file in detail now...

Current State of the App

In the current state this app works only on MacOS machines. At least that's what I assume as I still haven't tested it on a Windows computer. Thanks to the help from my friends I realised that in its 0.1.X versions the app looked silly on large monitors, that's why in version 0.2.0 I updated it, so that both the display and the app rescale based on the screen's resolution.

Move from 0.2.X to 0.3.0 was a big step in the apps functionality. Now the app allows user to make evaluations on the predictions. Since this version users can check whether the given label is correct, incorrect or if they don't really know the correct breed. Right now, users can choose out of 120 breeds available in the original dataset, but if the breed is missing they can write down their own suggestion. After all predictions were evaluated a .csv log file is saved to the "IdentiBreed/user/evaluation_logs" directory. At the bottom of the page they can submit their log to expand the database.

What I would like to do in near future is take the original Kaggle dataset and build on it, adding more and more breeds as it's missing quite a few well known and popular breeds. Finally, what I'm going to do after the app is running and there are no additions to be made is rebuilding the model behind the predictions from the ground up! Without the Tensorflow Hub, simply creating my own convolutional network. But that's the goal for another time...

If you wish to download IdentiBreed for yourself and try it out, remain updated about the changes that take place and give me feedback about the app please visit the repository linked here. Feel free to email me using the contact sheet or catch me on websites linked below.

Upload Your Log Files!

If you decided to take part in evaluating the predictions here is a bit of information about these log files and how your data is handled.
To begin with - unless you changed the user_id in the config.ini file to something personal all your data is anonymous and does not allow me or anyone else to trace it back to you. The file itself is a .csv file containing the user_id (makes it easier to create unique filenames), the "image" in a numpy.array format, the breed and the boolean value telling if the prediction was correct or not. The file itself is not stored anywhere - upon submitting it, an automated process expands the existing database. After this process is finished your file is removed from the server.
If you'd still like to upload your evaluation logs please use the form below to select your file and then click the "UPLOAD LOG FILE" button. Your evaluations can be found under the path "IdentiBreed/user/evaluation_logs". Each file is named after your very own id, date and time of predictions.

Log File: