by Ingo Wonner (CTO & Co-Founder)
in Process automation, Data validation
July 2024
The growing demand of their clients resulted in a growing number of suppliers to meet the demand. This significantly increased the amount of human effort necessary to process all the different types and formats of data provided by those suppliers. The client was not able to handle the workload on a regular basis anymore which lead to less update cycles and stale data.
For our client we automated a process for price listings sent in from suppliers. The idea was to not only make the process faster and more streamlined but to also validate the actual data and alert any inconsistencies in format or unusual changes in patterns.
We observed and interviewed employees involved in the recurring process to understand which steps can be easily automated or improved. Spoiler alert: We found a few things!
Visualisation of the flow used between Preparation and Review.
In the manual workflow a JIRA ticket for each new data file was created. This took a lot of time, but was necessary for the developer who performs the transformation task.
By fully automating this task the need for JIRA tickets became obsolete.
Instead of performing this task once a month manually by a developer, we run a fully automated workflow on each upload of a new data file and can provide the transformed data in seconds.
To ensure data integrity we implemented a stricter data validation step and created an analysis script to compare the new data set with the current one to be able to find duplicates and unusual changes and alert the process owner.
For steps we could not fully automate yet we improved the way to manually perform them by creating a easy to scan and meaningful report.
It was not possible to reduce this to just 1min to reach the ultimate goal of 2min human effort for the whole workflow, but in the end this was a huge improvement and a big win.
One ecosystem for customers, partners and delivery services
