Creating a Computer Vision-Powered API for a Petcare Mobile App

Solution

The solution we proposed to the customer involved unfolding the project in the next distinct stages.

1. Data Collection: at the first stage, we gathered a dataset of images containing dog excrement samples with a variety of textures, colors, and conditions representing different health states. We also ensured that the images within the dataset are of high quality, being captured from different angles and light conditions.
2. Data Exploration and Analysis:  our initial focus was on understanding the intricacies of the dataset. Through thorough data exploration and analysis, we identified instances of noise in the labels and recognized the significant imbalance between different data classes. This critical step laid the foundation for devising effective strategies to handle the challenges.
3. Instance Segmentation Model Training: at the third stage, we embarked on training an instance segmentation model. This model excelled at identifying and segmenting individual instances within images, offering a more granular understanding of the data. In such a way, this algorithm identifies and segments the fecal sample in the image, distinguishing it from other objects and ensuring further analysis correctness. At this stage, we also annotated the images to mark the areas of interest (the poop) for training the instance segmentation model and to label the conditions or health states for regression model training. The outcomes of this stage included improved object localization accuracy, laying the groundwork for subsequent phases.
4. Deep Imbalanced Regression Model Training: the next move was to train a deep imbalanced regression model. We assigned a condition score to each pool instance, and then proceeded with training the model to assess the health of the dog based on its excrement appearance or other factors. A regression model is particularly suitable for this task because it can predict continuous values, such as the degree of healthiness or abnormalities in the poop. The result was a robust model capable of making accurate predictions across diverse data classes.
5. QA and Data Labeling Review: after the models were trained, we conducted thorough testing and validation of the feature before deployment. We checked it for false positives and false negatives to refine the model’s accuracy and ensured the model’s predictions align with expert veterinary opinions. In this critical quality assurance step, our team also conducted a meticulous review of the data labeling. This involved identifying and rectifying instances of noisy labels that could potentially impact the model’s performance. 
6. Integration and Testing: at the last stage, we integrated the trained models into the mobile application, ensuring they can process images and provide results in real-time. For this goal, we tested the feature with a separate validation dataset and made sure that it provides accurate results for a variety of poop samples. As a result, we delivered a computer vision-powered API that incorporated the trained instance segmentation and imbalanced regression models.
7. Feedback Loop Implementation: to ensure continuous improvement of the model’s performance, we implemented a feedback loop so that the users can share their opinion on the accuracy of the analysis.