Leveraging AI/ML for Expanded Product Range and Enhanced eCommerce Product Listings 

Solution

Performance and SEO Enhancement

Our focus on performance optimization encompassed a multifaceted approach aimed at enhancing usability and site responsiveness. This included strategies such as optimizing page loading speed, streamlining navigation pathways, and refining user interface elements to ensure a seamless browsing experience.

Some of the existing SQL queries took too long to execute even with the current volume of products and there would have been at least 10 times more products in the future. Therefore, we analyzed database queries and applied different optimization techniques: created multiple indexes, rewritten some queries and created materialized views achieving lower execution times than in the beginning with a larger amount of products.   

In parallel, our SEO strategy was meticulously crafted to boost visibility and search engine rankings. Through comprehensive keyword research, content optimization, and strategic backlinking efforts, we aimed to elevate customer’s products’ online presence. Additionally, we implemented measures such as cleaning up 404 pages, optimizing robot.txt files, and refining indexing practices to enhance search engine crawlability and indexing efficiency. By improving semantics and ensuring alignment with search engine algorithms, we aimed to position the products favorably in search engine results pages, driving organic traffic and fostering increased conversions.

UI/UX enhancements and the uniqueness of the product detail page provide significant attractiveness from SEO perspective.

Furthermore, we developed customized email forms tailored for marketing campaigns, enabling targeted communication with potential customers and fostering deeper engagement with the client’s brand.

Custom Parsing Solution

In response to the client’s ambitious goal of expanding their product range by over a million items sourced from the major online retailer, we embarked on creating a parsing solution. Despite extensive searches for pre-existing solutions, none could handle the scale required, peaking at 300,000 products. Thus, we devised a tailored approach, executed in three stages:

• Product Collection: We initiated by gathering product data from a curated list of links.
• Data Extraction: Each product’s vital information, including name, description, image, and tags, was meticulously extracted from the provided links.
• Categorization: Leveraging the GhatGPT categorization capability, we seamlessly categorized the parsed products, ensuring efficient organization and accessibility.

The peak of our efforts was the staggering throughput of 300,000 items sourced within three hours, demonstrating the scalability, speed, and automation capabilities of our solution.

Enhanced Product Categorization with ChatGPT

In our quest for precise product categorization, we harnessed the capabilities of the ChatGPT API. Our first step involved crafting a tailored list of business-specific categories and subcategories, laying the groundwork for efficient organization. 

To enhance this categorization structure, we developed a script that worked along with ChatGPT, fine-tuned to classify products into specific categories. Subsequently, we submitted this classification along with over a million of product items, comprising names, descriptions, and metadata to the ChatGPT model.

By meticulously selecting prompts and optimizing parameters such as Temperature and top_p sampling, we optimized the model’s performance, achieving an impressive 90% accuracy in categorization. This strategic approach ensured that ChatGPT effectively assigned each product to its appropriate category with remarkable precision.

Product Cleanup with Tesseract OCR and HuggingFace Salesforce Blip Image Captioning Model

In our pursuit of excellent product listings sourced from the major retailer, we encountered challenges such as irrelevant text in product descriptions and names, as well as redundant content within product images. To rectify this, we implemented a comprehensive cleanup system designed to ensure product data integrity.

• Description and Name Cleanup: Leveraging ChatGPT, we systematically identified and removed irrelevant text from both product descriptions and names. To make the cleanup more accurate, we additionally programmed the logic for curating lists of forbidden words and relevant keywords, meticulously removing irrelevant data while preserving essential product description. 
• Image Cleanup: We used Tesseract OCR and a custom-programmed solution to extract text descriptions from product images, determining their relevance (e.g., containing details like size and dimensions) for inclusion on the client’s eCommerce site. In cases where images lacked text but provided visual product information, we employed the Salesforce Blip image captioning model to assess their business relevance. After cleaning up the images, we utilized ChatGPT to associate images with corresponding products, enabling seamless image-to-product mapping. 

Autosuggestions with Algolia and ChatGPT

Initially, our efforts to implement auto-suggestions were impeded by the lack of access to search history on the eCommerce site. So, we developed a logic wherein autosuggestions were based on keywords related to products and categories, prioritizing relevance. However, to enhance the effectiveness of autosuggestions, we devised a more sophisticated approach. This involved leveraging ChatGPT for search category suggestions and utilizing another major retailer website API for category suggestions.

Moreover, suggestions for products were generated by ChatGPT, powered by Algolia. Algolia enables developers to implement fast and relevant search experiences in their applications, offering a range of tools such as typo tolerance and faceted search.

Scalability

Our solution leverages map reduce algorithms and AWS Batch with Fargate Spot Fleet to run the web scrapping process in parallel on multiple server instances. To distribute processes such as categorization and cleanup, ensuring optimal performance and resource utilization even during peak loads, we used AWS Elastic Beanstalk and Java CompletableFuture parallelism capabilities.