The client operates in the mid-market finance sector. Working with large quantities of historical financial data is labour-intensive and prone to errors due to duplication and inconsistencies across multiple systems. With hundreds of people-days required to complete such tasks, the client embarked on testing a strong hypothesis: that an AI-powered approach could significantly reduce time and improve accuracy.
Over time, financial data accumulates across multiple internal and external systems, creating issues with data integrity and duplication that require manual intervention to manage effectively.
The primary challenge was to determine whether Artificial Intelligence (AI) and Machine Learning (ML) techniques could effectively accelerate and improve the accuracy of identifying similar or identical financial information across disparate systems, mitigate human intervention, and free up time for professionals.
We designed a ‘Proof of Concept’ (POC) project to test the feasibility of using AI to address industry challenges while laying the groundwork for future iterations and improvements.
1. AI-Driven Matching Model: Our development team created a custom AI clustering model, enhanced with Natural Language Processing (NLP) techniques, to analyse and group similar datasets. We integrated and tailored an existing open-source financial AI model (FinBERT), which provided the ability to generate higher-dimensional data representations. This allowed the system to identify semantic similarities between text and data structures.
2. Graph Structure Integration: To increase the accuracy of data matching, the AI model analysed and incorporated the data structure. A graph-based approach enabled an understanding of the relationships between datasets and the propagation of critical information, including parent, sibling, and child nodes. This structure provided a more contextual understanding of the data, allowing the process to improve over time.
3. Iterative Learning and Data Enhancement: The system initially worked with a limited amount of labelled data. The focus was on building a scalable model that could evolve as more data was attained. The system supports CSV and Excel uploads to add more data. The solution provided multiple potential matches for datasets. As an early POC product, human oversight was required to confirm or reject matches, providing feedback that enabled the AI model to learn and refine its predictions as more labelled data became available.
The project produced promising results, demonstrating the potential of AI to transform financial data migration processes.
1. Increased Efficiency: After feedback and refinement, the POC product achieved a 77% match accuracy with the initial dataset, demonstrating that the AI-powered product could significantly reduce the time required for humans to perform matchmaking. The clustering model’s ability to present multiple potential matches allowed users to quickly select the correct accounts, accelerating the overall process.
2. Insights into Model Limitations: The project highlighted the need for more labelled training data to further enhance the model’s performance. While the initial results were positive, it is recognised that the addition of more comprehensive training data would improve both the accuracy and scalability of the solution.
3. Foundation for Future Development: The model’s current architecture provided a strong foundation for future iterations. With additional training data and fine-tuning of the AI model, the solution has the potential to achieve even higher accuracy and eventually automate more of the process with limited human intervention.
The project significantly demonstrated that a time-intensive task could be mitigated using AI and ML. The client was satisfied with the initial performance and saw the potential for further improvements.
As the model evolves, we expect it to deliver even greater efficiency and accuracy, making it a valuable tool for businesses facing the complex challenge of matching financial data.
We want to hear your questions, your ambitions and understand the requirements of technology to support the people within your organisation. Schedule a no commitment short introductory call.