ATSICHS
ATSICHS sought to provision a data lake and modern data warehouse platform to remove the challenges they faced with integrating and reporting on data dispersed across many disconnected platforms.
The challenge
ATSICHS’s IT strategic roadmap identified that its reporting capabilities were significantly limited by the heterogeneous nature of its technical platforms and reporting solutions. Haze Software was engaged to deliver a homogenous analytics solution to assist ATSICHS in achieving its strategic priorities.
The solution
A Microsoft Azure data platform was implemented incorporating:
- an Azure Data Lake Storage (ADLS) Gen2 solution, organised into logical zones, hosting approximately fifteen data sources in formats such as Parquet and CSV
- a centralised data warehouse provisioned on Azure Synapse Analytics with change tracking, and table/column/row level security
- two sets of data processing pipelines built and orchestrated using Azure Data Factory – the first set ingests and transforms data from sources to the lake while the second set loads data from the lake to the warehouse
- varying schedules set up to ingest and load data on a regular basis with appropriate monitoring to ensure pipelines are executed as expected.
The outcome
Ultimately the provisioned data platform has addressed ATSICHS’s key challenges by providing a centralised data repository and enabling a variety of analytical capabilities.
More specifically, the data lake has already delivered value to a variety of projects by acting as a simple standardised data source for downstream consumers. ATSICHS is currently building-out the capabilities of their data team and they have started to extend the lake by applying the pipeline design patterns to new data sources.