Summary
We are seeking an experienced Data Engineer with a proven track record in building and maintaining data pipelines, developing applications, and implementing data management solutions. The ideal candidate combines technical expertise with hands-on development skills to drive our data engineering initiatives.
Requirements
1. 4-5 years of professional experience in data engineering
2. Strong proficiency in Python for data processing, ETL/ELT development, and application building
3. Advanced SQL skills for complex data querying, optimization, and database design
4. Experience with Apache Airflow for workflow orchestration and scheduling
5. Knowledge of low-code development platforms (such as Tooljet) for rapid application creation
6. Ability to build and deploy data applications using Streamlit
7. Extensive data management experience including data governance, quality control, and metadata management
Key Responsibilities
1. Design, build, and maintain efficient data pipelines to process structured and unstructured data
2. Create and optimize SQL queries for data extraction and transformation
3. Develop and manage Airflow DAGs for reliable workflow orchestration
4. Utilize low-code tools to accelerate application development where appropriate
5. Build interactive dashboards and tools using Streamlit for data visualization and exploration
6. Implement data management best practices throughout the data lifecycle
7. Collaborate with data scientists, analysts, and other stakeholders to understand requirements
Technical Skills
1. Programming Languages: Python, Javascript
2. Database Technologies: SQL (PostgreSQL, MySQL, or similar)
3. Orchestration: Apache Airflow
4. Low-Code Development: Tooljet or similar platforms
5. ETL/ELT Tools: Apache Nifi, KNIME, Apache Spark
6. Lakehouse Solution: Snowflake
7. Application Development: Streamlit
8. Version Control: Git
9. Data Processing: Experience with batch and streaming data processing
Preferred Qualifications
1. Experience with cloud platforms (AWS, Azure, or GCP)
2. Knowledge of containerization technologies (Docker, Kubernetes)
3. Familiarity with big data technologies (Spark, Hadoop)
4. Understanding of data modeling and dimensional modeling concepts
Success in this Role
1. The successful candidate will bridge the gap between raw data and valuable insights by building reliable data infrastructure and intuitive applications, ensuring that data is accessible, trustworthy, and actionable across the organization.
#J-18808-Ljbffr