Data Engineer

The Data Engineer is responsible for designing, building, and maintaining scalable data pipelines and architectures that enable effective data collection, transformation, storage, and analysis. This role involves working closely with data analysts, data scientists, and other stakeholders to ensure data accessibility, accuracy, and reliability for business insights and decision-making.

Key Responsibilities:

Data Pipeline Development:

• Design, build, and maintain scalable ETL/ELT pipelines to process structured and unstructured data.
• Integrate data from multiple sources into data warehouses, lakes, or other storage systems.

Data Modeling and Architecture:

• Develop and maintain data models to support analytics and reporting needs.
• Implement data architecture best practices for high availability and performance.

Data Quality and Governance:

• Ensure data integrity, consistency, and accuracy through quality checks and validation processes.
• Collaborate with stakeholders to establish and enforce data governance policies.

Performance Optimization:

• Monitor and optimize the performance of data systems and pipelines.
• Troubleshoot and resolve issues related to data processing and storage.

Collaboration and Support:

• Work with data scientists and analysts to understand data needs and deliver appropriate solutions.
• Provide technical support and documentation for data systems and processes.

Qualifications:

Education: Bachelor’s or Master’s degree in Computer Science, Engineering, Data Science, or a related field.

Experience:

• 2-5 years of experience in data engineering, software development, or database management.
• Experience with cloud platforms like AWS, Azure, or Google Cloud is a plus.

Technical Skills:

• Proficiency in programming languages such as Python, Java, or Scala.
• Strong SQL skills and experience with relational and non-relational databases (e.g., MySQL, PostgreSQL, MongoDB).
• Familiarity with big data tools and frameworks (e.g., Hadoop, Spark, Kafka).
• Knowledge of data warehousing concepts and tools (e.g., Snowflake, Redshift, BigQuery).
• Understanding of data pipeline tools such as Apache Airflow or AWS Glue.

Soft Skills:

• Strong problem-solving and analytical skills.
• Excellent communication and ability to work collaboratively with cross-functional teams.
• Attention to detail and a commitment to data accuracy and reliability.

Working Conditions:

• Flexible work arrangements, including remote or hybrid options, depending on the organization.
• Occasional availability for on-call support during critical data operations