Creating a Computer Vision-Powered API for a Petcare Mobile App

# API development # Data Analytics # Mobile Application
Discover more

Highlights

  • Two Significant Challenges in Data Structure: successfully addressing noisy labeled data and its high imbalance between classes, ensuring data consistency and reliability for subsequent neural network development.
  • Two Advanced Neural Network Models: created an instance segmentation and deep imbalance regression models that laid the foundation for the computer vision-enabled API.
  • Lightning-Fast Image Analysis: enabled the company to significantly reduce operational costs and deliver effective user experience with almost-instant in-app image processing.

Client

Established in 2015 as a startup, our client has rapidly evolved to become a prominent leader in the Israeli pet tech industry. The company designed a technology platform to actively engage dog owners in monitoring and improving their pets’ well-being. Their solution goes beyond conventional tracking, fostering engagement with gamification. The users can earn coins for being proactively involved in their dogs’ lives, and then, exchange them for dog food, treats, toys or veterinary services within the partners’ network.

Country:
Industry:
Team Size:

Product

The company offers mobile applications for Android and iOS. Dog’s fitness tracker is the main feature of the app; it allows the owners to track walk duration and dog activity, competing with other members within their neighborhood. Active users can earn coins that can be redeemed for discounts for pet care products, and for every mile walked with the dog, the company donates a meal to a shelter. In 2022, the company decided to add more advanced features to their application, and SPD Tech became the technology provider of choice.

Goals and objectives

  • Introduce a Remote Veterinary Service

The client aimed at bringing artificial intelligence capabilities to the app, enchanting the pet care service being provided and solidifying the company’s position as a leader in the Israeli pet tech industry.

  • Build a Computer Vision Model

To make a client’s app more innovative and effective for users, we were tasked to build an image analysis neural network that should be deployed as an API. 

  • Drive User Engagement

Since before the development started, the application had a dog fitness tracker feature only, driving user engagement by expanding service offering became one of the primary goals. 

Project challenge

  1. Noisy Labeled Data: the presence of incorrect or mislabeled instances in a dataset can have detrimental effects on the model’s performance and reliability. To address the issue, we had to use a combination of careful data curation, employing robust learning algorithms, and implementing strategies to identify and mitigate the impact of mislabeled instances.
  2. High Imbalance Between Data Classes: since non-uniform distribution of labels across different classes can lead to biases towards the majority classes, we had to choose the right tactic to address the issue.

Solution

The solution we proposed to the customer involved unfolding the project in the next distinct stages, each of which strategically tailored to address the challenges posed by noisy data and a high imbalance between data classes:

  1. Data Exploration and Analysis: our initial focus was on understanding the intricacies of the dataset. Through thorough data exploration and analysis, we identified instances of noise in the labels and recognized the significant imbalance between different data classes. This critical step laid the foundation for devising effective strategies to handle the challenges.
  2. Instance Segmentation Model Training:  at the second stage, we embarked on training an instance segmentation model. This model excelled at identifying and segmenting individual instances within images, offering a more granular understanding of the data. The outcomes of this stage included improved object detection and segmentation accuracy, laying the groundwork for subsequent phases.
  3. Deep Imbalanced Regression Model Training: the next strategic move was to train a deep imbalanced regression model. This model was tailored to address the skewed distribution of classes, ensuring that the model’s learning process accounted for the challenges posed by the dataset. The result was a more robust and balanced model capable of making accurate predictions across diverse data classes.
  4. Data Labeling Review: in a critical quality assurance step, our team conducted a meticulous review of the data labeling. This involved identifying and rectifying instances of noisy labels that could potentially impact the model’s performance.

As a result, we delivered a computer vision-powered API that incorporated the trained instance segmentation and imbalanced regression models.

Tech Stack

  • Python Python
  • PyTorch  PyTorch

Our results

Our collaboration with the customer goes on. Our ML developer is proactively engaged in image analysis model support and its further training. For now, SPD Tech delivered the next results: 

  1. Shortened Image Analysis Time to Several Seconds:
    Our team has successfully coped with data inconsistency challenges and built a robust neural network, capable of an almost-instant image analysis.
  2. Rid Company’s Employees From Manual Work:
    The customer shared that after releasing an image recognition model, the time employees spent on manual image processing was reduced significantly. 
  3. Created a Revenue Stream for the Customer:
    An AI-supported remote veterinary service was introduced as a paid feature, thereby creating an additional source of income for our client. 
Next Project
# AWS infrastructure, # Fintech, # Legacy Application Improvement, # Web development
Transforming the U.S. Asset Manager’s Investment Analytics App with Serverless Architecture

Highlights Client Our client is one of the U.S.’s largest asset management companies, with 12 000 + employees in...

Explore Case