During the COVID-19 pandemic, a part of the Danish authorities was facing a desperate need for expert insight, guidance and hands-on development to help collect, structure and confirm the correctness of critical data across Denmark. There was a lack of transparency in national stock levels and supplies of essential medical resources, which posed significant challenges in minimising potential disruptions caused by the outbreak.
To address this urgent requirement, the Danish authorities partnered with the Halfspace expert team. Halfspace established fully automated ETL (Extract, Transform, Load) pipelines to aggregate data from over 100 daily sources. They orchestrated this using Azure Data Factory and Apache Spark in Databricks, enabling the authorities to access comprehensive COVID-19-related data across the entire country.
As a result of this collaboration, the Danish Medicines Agency now has full transparency into their supply chains and stock levels of critical medical equipment. This provides them the ability to navigate future outbreaks with high precision and speed, ensuring they can make well-informed decisions to safeguard the availability of vital resources and ultimately save lives.
Remember the importance of establishing robust data integration capabilities that can ingest, cleanse, and harmonise information from diverse systems and sources. Having this centralised, high-quality data foundation was essential for providing the Danish authorities with a transparent, real-time view of their critical supplies.
Data-driven platforms can empower organisations to navigate complex, rapidly evolving situations with agility and precision. Access to trusted, data-driven insights is crucial for public sector entities tasked with protecting public health and safety.
The use of Azure Data Factory and Apache Spark in Databricks to power the data pipelines highlights the value of cloud-based, scalable data infrastructure. These modern, enterprise-grade platforms allow the deployment of robust, high-performance data solutions that can ingest and process large volumes of information. The flexibility and scalability of the cloud-based architecture were instrumental in supporting the Danish authorities' evolving data and analytics needs during the pandemic.