How HaulHub SupplierCI delivered a Game-Changing Analytical system

Highlights

Data Analytics System Development: successfully developed a robust Data Analytics system, managing and analyzing an extensive dataset comprising 70+ million records.
High-Performing OLAP Data Lake System: engineered a client-facing OLAP Data Lake system, specifically tailored for the transport construction domain, ensuring high performance and near real-time data availability.
Cost-Effective and Scalable Solution: implemented a cost-effective and scalable solution utilizing a Serverless architecture, minimizing infrastructure costs while maintaining optimal performance.

Client

HaulHub is a comprehensive platform tailored for heavy construction companies, revolutionizing transportation and digital ticketing processes for producers and contractors. Their platform plays a pivotal role in digitizing an expanding network of suppliers and contractors throughout the United States.

HaulHub platform stands out as the exclusive solution fully embraced by multiple Transportation Agencies for managing their central ticketing operations. At present, it holds the position as the foremost digital construction delivery platform for Transportation Agencies and authorities. It caters to material producers, general contractors, fleet owners, and carriers nationwide, reaffirming its status as an industry-leading solution.

Country:

the USA

Industry:

Logistic

Team Size:

10+

Product

SupplierCI https://www.haulhub.com/supplierci/

A continuous intelligence service designed to generate valuable insights within the construction industry. This service intelligently monitors diverse activities from plant to jobsite, all without relying on GPS.

The primary goal is to empower material ticketing data processing through user-friendly visualization tools, facilitating a rapid analysis of business trends and enabling accurate forecasts of future performance.

Goals and objectives

Eliminate Manual Reports and Enhance Data Presentation:

Replace cumbersome manual reports and extensive spreadsheets with sophisticated, customizable visual tools. Introduce widgets tailored for both mobile and desktop platforms, ensuring accessibility and convenience for users.

Enhance Customer Visibility through Data:

Provide comprehensive data on tonnages, materials, projects, plants, trucks, and more to enhance customer visibility and offer a holistic view of operations.

Custom Dashboard and Watchlists Creation:

Develop custom dashboards and watchlists for users to monitor top-performing plants and customers, facilitating strategic decision-making with readily available key information.

Simplify Material Ticketing Data Visualization:

Streamline the visualization of material ticketing data, presenting information in a clear and actionable manner to unlock additional margin potential.

Fine-Grained Project Workload Insights:

Offer fine-grained insights into project workloads, enabling accurate monitoring of trucks and prediction of project completion time.

Project challenge

Vast Ticketing Data Volume: manage and analyze a massive dataset comprising over 70+ million ticketing records. The challenge involves ensuring prompt updates, near real-time processing of continuously incoming new tickets, and addressing the ongoing growth in data volume.
Client-Facing OLAP System Development: construct a client-facing OLAP system designed for optimal performance and near real-time data availability. This involves aggregating both current and historical ticketing data across various dimensions within specified time ranges.
Integration of Complex Business Logic: implement intricate business logic to address the nuanced requirements of the project. This includes accommodating complicated data aggregation use cases that necessitate a sophisticated and flexible system architecture.

Solution

In collaboration with the HaulHub team, SPD Technology has successfully developed Web and mobile applications to achieve the predefined business objectives. We have also effectively implemented custom charts and widgets using ReactJs with ChartsJS with Recharts libraries.

To establish a comprehensive Data Lake for aggregating all collected data, we rigorously evaluated various solutions, including AWS Athena, Glue, RedShift, and ClickHouse. Due to the stringent non-functional requirements, and, particularly, the imperative need for rapid response times, PostgreSQL emerged as the optimal choice for this specific use case.

To ensure swift performance and robust data availability, we meticulously implemented substantial database optimization techniques t. Notable strategies encompassed data denormalization, proficient database index management, partitioning, and efficient caching mechanisms.

Data integration from diverse sources was orchestrated through AWS Database Migration Service (DMS) and supplemented by the implementation of custom Extract, Transform, Load (ETL) processes. These efforts were pivotal in transforming raw data into structured, relational formats accessible within milliseconds.

Regarding the mobile application, we made a strategic decision to leverage native mobile technologies that afford the flexibility to incorporate custom features, tailored to specific requirements.

Our project team built the API layer of the solution using a Serverless architecture we chose for its cost-effectiveness and scalability.

As far as Java development is concerned, the choice of supersonic and subatomic Quarkus, complemented by GraalVM native images, has ensured exceptional performance and efficiency.

In addition to the development efforts, SPD Technology’s project team diligently configured the comprehensive monitoring systems, enabling proactive identification and swift resolution of potential issues. This includes monitoring for performance degradation, data outages for each client, high-traffic incidents, and internal system anomalies.

Our results

We have successfully developed business intelligence apps to leverage powerful visual data analytics and better understand business needs.

Business Intelligence App for Enhanced Data Analytics: Successfully developed business intelligence application, empowering users with powerful visual data analytics tools to gain deeper insights into business needs.
Efficient Data Processing Solution:
Developed a solution capable of processing millions of analytical data points swiftly, delivering lightning-fast performance with a remarkable response time of less than 2 seconds.
Zero API Compute and Frontend Infra Cost:
Achieved a milestone of zero API compute and frontend infrastructure costs, ensuring an efficient and cost-effective operational model.
Infrastructure Cost Savings Through Optimization:
Achieved infrastructure cost savings by implementing an efficient Data Lake solution and executing database optimizations, contributing to overall operational efficiency.
Modern and Comprehensive UI/UX:
Successfully developed a modern, all-encompassing user interface and user experience (UI/UX) that provides users with the flexibility to create a tailored user journey.