How we reduced workloads from 2 days to (almost) 2 minutes

An automated process to read, validate and transform different sources and formats of data for internal use.

Challenge

The growing demand of their clients resulted in a growing number of suppliers to meet the demand. This significantly increased the amount of human effort necessary to process all the different types and formats of data provided by those suppliers. The client was not able to handle the workload on a regular basis anymore which lead to less update cycles and stale data.

Mission

For our client we automated a process for price listings sent in from suppliers. The idea was to not only make the process faster and more streamlined but to also validate the actual data and alert any inconsistencies in format or unusual changes in patterns.

Solution

We observed and interviewed employees involved in the recurring process to understand which steps can be easily automated or improved. Spoiler alert: We found a few things!

Visualisation of the flow used between Preparation and Review.

Step 1 : Prepare Data

In the manual workflow a JIRA ticket for each new data file was created. This took a lot of time, but was necessary for the developer who performs the transformation task.

By fully automating this task the need for JIRA tickets became obsolete.

Step 2 : Transform Data & Create Report

Instead of performing this task once a month manually by a developer, we run a fully automated workflow on each upload of a new data file and can provide the transformed data in seconds.

To ensure data integrity we implemented a stricter data validation step and created an analysis script to compare the new data set with the current one to be able to find duplicates and unusual changes and alert the process owner.

Step 3 : Review Report

For steps we could not fully automate yet we improved the way to manually perform them by creating an easy to scan and meaningful report.

It was not possible to reduce this to just 1min to reach the ultimate goal of 2min human effort for the whole workflow, but in the end this was a huge improvement and a big win.

Services

Data Consulting
Tech Consulting
User Experience Design
Software Development
DevOps

Results

  • We empowered the client to provide up-to-date data on a daily-basis for their customers in a fast and secure way.
  • In the end, we did not reach the 2 minutes mark but were able to reduce the process to 6 minutes workload, which is a speedup by a factor of 1440x. Pretty impressive!

How we setup easy digital access methods for shared rooms and buildings

From tedious to trivial: Using digital locking systems to achieve a seamless check-in and check-out process for users of shared resources.

-> See the story

How we helped to choose the best software solution from multiple options

Well considered requirements facilitate the provider selection.

-> See the story