Overhauling HaulHub’s Reporting System with Advanced Data Engineering

Highlights

  • Leveraged Data Engineering: We merged data from several sources, connecting all the HaulHub’s components and ensuring seamless integrations with third-party services. 
  • Designed Data Architecture: Our team made sure the system can handle high volumes of data, remaining stable and scalable.
  • Equipped the Platform with Value-Driven Features: Engineered and implemented the main features agencies required for construction inspection, live activity monitoring, and more.

Client

HaulHub is a B2B2C enterprise that offers digital platforms to the transportation construction sector. The company focuses on the research and development of cutting-edge data collecting, AI, and software systems for the public and private sectors. We have a long-standing and productive partnership with HaulHub. Previously, we created business intelligence web and mobile applications and built a reliable data processing system that could effectively handle millions of analytical data points. Additionally, we provided a ticketing system driven by AI to gather information on contraction supplies.

Country
Industry
Team Size:

Product

EDOT is a unified reporting platform for transportation construction agencies, vendors, contractors, and suppliers. HaulHub’s EDOT system allows collaboration between all of the parties involved in the contraction process, conducting digital inspection of the activities on sites, enhancing coordination between departments, and checking construction in progress in real-time. The DOT Slip is a part of the EDOT platform that helps connect a contraction agency to the rest of the contraction ecosystem.

Goals and objectives

We specifically needed to work on the DOT Slip part. To make it work up to HaulHub’s requirements, we were asked to:

  • Create Management Solution for Transportation Agencies:

Develop a system with several functionalities such as project tracking, digital inspection, and collaboration tools to empower agencies to manage their projects more efficiently and ensure that all aspects of construction are seamlessly coordinated. 

  • Provide Integrity Across Construction Activities:

Linking contractors, inspectors and suppliers to establish a transparent and cohesive workflow. This will enable real-time communication and data sharing among all the participants involved in the construction.

  • Simplify Material Ticketing Data Visualization:

 Deliver granular insights into tonnages, materials, projects, and equipment to optimize operations and inform strategic decisions.

  • Improve Data Presentation:

Incorporate customizable reports, interactive maps, and detailed analytics to help users monitor progress, and make informed decisions.

Project challenge

  1. Legacy Code Base: We faced the necessity to revise some outdated and complex parts of the system to meet current standards of industry and business needs.
  2. Data Integration: HaulHub has a big infrastructure that merges data from different sources and systems. So, one of the challenging things was to provide connectivity and data visualization in the DOT Slip component from other client systems.
  3. Scalability: The HaulHub system had a huge amount of projects, eTickets and photos. We needed to perform complex work to ensure system stability with a large amount of data. 
  4. Seamless Authorization: Users didn’t have the right accesses. We set them up, configuring accesses separately for agencies, vendors, suppliers, and contractors. From now on users just need to sign in into the EDOT portal and get access to all HaulHub systems in one place.

Solution

Our team leveraged our data engineering expertise to merge data from several sources and systems for seamless integration and visualization in the DOT Slip component. Our data engineers built pipelines that integrate data from multiple sources into a unified system. We ensured that data from contractors, suppliers, inspectors, and other stakeholders is accurately ingested, processed, and stored in the platform. Additionally, we developed APIs and data exchange protocols to facilitate real-time communication between the platform and third-party systems, both internal and external to the DOT Slip component.

We also needed to utilize our data engineering skills to make the platform handle a large volume of projects and eTickets since the system required a stable and scalable system to manage this data. With this goal in mind, our engineers designed the platform’s data architecture to handle high volumes of data. They also implemented distributed data storage solutions and use technologies like Data Lakes or Data Warehouses to efficiently store and retrieve large datasets without sacrificing the platform’s stability. 

Moreover, our team of Data Engineers, Data Analysts, and front-end developers transformed raw data into visual formats for easy-to-understand ticketing data visualization. SPD Technology’s experts used React.js for the frontend, which leverages Chart.js and Recharts for data visualization. Then, Mapbox was integrated for mapping functionalities. The UI components were styled using MUI (Material-UI), and a custom flex grid table is implemented to handle complex data display and manipulation.

  • React.js  React.js
  • Chart.js  Chart.js
  • Recharts Recharts
  • Mapbox Mapbox
  • MUI MUI

Proсess

  1. Analysis of Existing Code and Functional Requirements: We conducted a thorough analysis of the current codebase and its alignment with the functional requirements of the EDOT platform. This involved reviewing the architecture, identifying technical debt, and understanding how the existing features fulfill business needs. 
  2. Improvement of Codebase Quality and System Stability: Based on the analysis, we implemented a series of activities aimed at enhancing code quality and system stability. This included refactoring outdated or inefficient code, improving documentation, and implementing automated testing. 
  3. Integration with Third-Party Systems: We developed APIs, configuring data exchange protocols, and ensuring that the platform can communicate effectively with internal components and external software, databases, and services. 
  4. Extending Feature Set: We prioritized feature development based on business needs and impact, implementing these features in a way that aligns with the overall system architecture.

Our results

We successfully completed the project and delivered an enhanced EDOT platform for HaulHub. 

  1. Delivered Key Features: Enabled such functionality as digital inspection, live construction activity view, activation and deactivation of digital work zones, worker presence analysis, automated traveler alerts, environmental product declarations reporting, as-built tracking, and more.
  2. Secured the EDOT Portal with Correct Permissions and Roles: Users can login to one portal and get access to all main HaulHub services that include agencies (DOT Slip), contractors, vendors, suppliers (Supplier CI). 
  3. Ensured Stable Integration with External APIs: Enabled the platform to function within the broader ecosystem of transportation management tools.