About the Role: Our Federal Government client is looking for an experienced Data Factory Engineer to join its dynamic ICT team.
This role offers the opportunity to work on a range of exciting ICT projects and programs, focusing on delivering high-quality data integration solutions that connect both on-premises and cloud-based applications.
You will leverage Microsoft Azure technologies to design, develop, and maintain data pipelines, ensuring performance, security, and compliance.
As a Data Factory Engineer Developer, you'll work closely with cross-functional teams to support systems through troubleshooting, testing, and monitoring, contributing to the ongoing success of critical projects.
Key Responsibilities: Design & Develop Data Integration Solutions: Collaborate with stakeholders to create data integration solutions that meet business requirements, using tools such as Azure Data Factory and SQL .
Develop ETL Pipelines: Implement and maintain efficient ETL pipelines to transfer data securely between systems while ensuring the best practices for data transformation and processing.
Data Modelling & Quality Assurance: Design and maintain data models, ensuring data accuracy, consistency, and availability, while optimising performance.
Data Validation: Implement validation and quality checks to identify and resolve data errors, ensuring seamless system integration.
Continuous Monitoring & Optimisation: Proactively monitor data integration processes to enhance performance, reduce latency, and minimise resource consumption.
Documentation: Create and maintain technical documentation for data integration solutions, including specifications and data flow diagrams.
Collaboration: Work with developers, system analysts, and other stakeholders to align data integration processes with business objectives.
Issue Resolution: Troubleshoot and resolve data integration issues, ensuring effective solutions are implemented and recurrence is prevented.
Stay Current with Emerging Technologies: Keep abreast of industry trends and apply new knowledge to improve processes and tools.
Key Skills & Experience: Mandatory Requirements: Expertise in Microsoft Azure Data Factory and related tools.
Proven experience in data engineering, including data integration, ETL processes, and data modelling .
Strong skills in SQL and knowledge of data warehousing .
Solid understanding of data governance, security, and compliance best practices.
Excellent problem-solving skills and the ability to collaborate effectively within a team.
Strong communication skills for both technical and non-technical stakeholders.
Demonstrated ability to build and maintain effective working relationships.
Ability to work independently with strong self-motivation.
Desirable Requirements: Experience with Azure SQL Database and other Microsoft Azure tools and services.
Bachelor's degree in Computer Science, Information Technology, or a related field (or equivalent experience).
Relevant certifications (e.g., Microsoft Azure Data Engineer, AWS Certified Data Analytics ) are a plus.