Enhancing Global Investment Data Analysis Platform with AI

Highlights

Ongoing Collaboration with the Top Global Provider of Financial Data: Previously, we have migrated the company’s platform to cloud, engineered a microservice architecture for it and developed a dedicated mobile app.
Leveraging AI for Advanced Data Insights: introduced AI-powered features for growing markets spotting and market trends detection, leading to a significant improvement in company’s customers satisfaction.

Client

Our client is a leading financial data and software company that provides comprehensive insights on the private and public capital markets, helping clients make informed business decisions. They offer a robust platform that delivers data, research, and technology to private equity and venture capital professionals, investment banks, law firms, and corporate development teams. Their services include detailed financial information, analytics, and tools for sourcing deals, performing due diligence, and tracking investments.

In 2016, our client’s company was acquired by a global financial services firm, and now it operates as a subsidiary. Our software development company has had the privilege of working with the client for over a decade, supporting their mission of providing top-tier financial intelligence to their customers.

Country

the USA

Industry

Business Research & Data Analytics

Team Size:

Product

The product is a sophisticated software platform designed to provide in-depth insights into the private and public capital markets. The key features of the product include but aren’t limited to:

Advanced Search and Filtering: Powerful search capabilities allow users to filter and find specific data points quickly and efficiently.
Robust Analytics and Visualization: Tools to analyze trends, build financial models, and create visual representations of data for easier interpretation.
Customizable Dashboards: Personalized dashboards that let users track key metrics, monitor market activity, and stay updated on relevant developments.
In-depth Reports and Research: Access to exclusive reports and research conducted by the client’s team of analysts, providing deep insights into various sectors and market trends.

The solution stands out for its extensive data coverage, offering detailed information on companies, investors, deals, funds, and service providers across global markets. As the volume of information grew from approximately 100 sources of news to 2,000-3,000 sources, it became impossible for the team to process such a large amount of information with the help of a crawling mechanism.

Therefore, the company realized the need to automate data processing and design a comprehensive dashboard for data visualization, laying the foundation for the feature allowing for instantly finding emerging investment opportunities.

With the help of this functionality, the users can easily find niche markets within relevant sectors and spot promising investment trends. The feature shows the number of companies founded in a new area each year, total deals over time, and year-over-year growth metrics for better insights.

During its development, it became evident that this feature could also be utilized as an additional microservice, demonstrating clear business value.

Goals and objectives

Extending Functionality with AI-Enabled Market Analysis: Develop an AI-powered feature for investment opportunities detection and designing a dashboard for data visualization.
Meeting Customers Expectations: In addition to the growing number of data sources that required automated processing, the client’s customers also requested more advanced data analysis and visualization capabilities.
Developing Trend Detection Feature: The feature was intended to spot impactful trends with the help of AI and let the client’s customers stay updated on the current investment landscape.
Creating a Competitive Edge: In 2019, none of the client’s competitors had an ML-powered functionality for data analysis.

Project challenge

Data Volume: The company needed to handle approximately 50,000 articles per day, amounting to 18 million articles annually. To manage this extensive data load efficiently, the implementation and fine-tuning of the appropriate set of machine learning algorithms were crucial.
Data Quality: Another significant challenge was ensuring data quality for building the AI-enabled market analysis functionality.

Solution

Initially, we gathered a large volume of data from publicly available sources. As a result, we had a dataset of 3-4 million companies but critical information about their capital and investment series was often missing. To address this, a web crawling mechanism was employed to gather data about the companies. Subsequently, a machine learning summarization model was used to automatically generate the missing information based on the crawled data. This generated information then underwent manual validation by a US-based team to ensure accuracy and reliability.

At the next stage, we used Word2Vec, a popular technique in natural language processing, for converting words into vectors and identifying similarities between companies based on how they are described. Based on these similarities, we trained the algorithm to group companies in categories, adjusting the optimal size of the group. At the last stages, the model was integrated and optimized.

As for the trend detection feature, this ML-enabled functionality was designed to analyze news mentions of companies across various sources and provide valuable insights into significant events in the business world.

Using a Logistic Regression model, this feature identifies and highlights instances where specific events are repeatedly mentioned in the news. For example, it can detect noteworthy events such as a global corporation acquiring a unicorn (a startup valued at over a billion dollars), based on the frequency and context of these mentions across different news outlets. This capability helps users stay informed about major developments impacting companies, markets, and industries in real-time.

Tech Stack

Python

Our results

6 Months to Completion: We successfully delivered AI features for promising markets spotting and evolving trends detection within a 6-month timeframe.
50% Decrease in Manual Data Processing Workload: Implemented automation resulted in a significant 50% reduction in manual data processing efforts.
Newly Created Features Rapid Scaling: When the feature was launched in late 2019, the company tracked roughly 70 emerging spaces. Since then, the number has expanded by nearly 50% to 125. Now, the algorithm is processing around 60 000 news topics a day.

Our fruitful collaboration with the client is going on.