Azure data factory engineer

Job Overview

Collaborate with team members and stakeholders to design data integration solutions that meet business requirements, utilizing Microsoft Azure tools such as Azure Data Factory and SQL.

Develop, implement, and maintain Extract Transform Load (ETL) pipelines to ensure efficient and secure data transfer between various systems, adhering to best practices for data processing and transformation.

Design and maintain data models, ensuring data accuracy, consistency, and availability while considering scalability and performance optimizations.

Implement data validation and quality checks to identify and rectify data anomalies, errors, and inconsistencies within integrated systems.

Continuously monitor and optimize data integration processes to enhance performance, reduce latency, and minimize resource consumption.

Create and maintain comprehensive documentation for data integration solutions, including technical specifications, data flow diagrams, and operational procedures.

Work closely with cross-functional teams, including developers and system analysts, to understand data requirements and ensure integrations align with business objectives.

Troubleshoot and resolve data integration issues, working collaboratively with the team to implement effective solutions and prevent recurrence.

Stay updated on emerging technologies and industry trends related to data engineering and integration, applying this knowledge to improve integration processes and tools.


Requirements

* Strong Microsoft Azure Data Factory skills and experience.
* Proven experience in data engineering, including data integration, ETL processes, and data modeling.
* Strong SQL skills and experience with data warehousing concepts.
* Knowledge of data governance, security, and compliance best practices.
* Excellent problem-solving skills and the ability to work effectively in a collaborative team environment.
* Effective communication skills for collaborating with technical and non-technical stakeholders.

Candidates must be Australian Citizens.

Federal Government Security Clearances are highly desirable.