Senior Cloud Data Engineer - SCDE24-13276
Duration: 6 months (Tentative)
Location: Quincy (Hybrid)
Summary:
A Senior Cloud Data Engineer is needed to play a pivotal role in leading the design and implementation of a data platform architecture. This role is part of a central data modernization program aimed at improving and coordinating data capabilities across multiple agencies, including the Department of Public Health and the Department of Mental Health.
The ideal candidate is an expert in designing and implementing data pipelines within decentralized architectures, such as Data Mesh. They must be skilled at communicating with both business and technical stakeholders, ensuring cohesive and collaborative solutions.This role will involve leading cloud data platform design, development activities, and establishing best practices for data modernization. The Senior Cloud Data Engineer will work with the Chief Data Officer and other data management leaders to advance data modernization efforts using modern tools such as Snowflake, IICS, and Airflow, while providing KPI reporting through Tableau.
Duties and Responsibilities:
- Design and develop cloud-based data management systems.
- Implement data pipelines within decentralized data architectures, such as Data Mesh.
- Ensure adherence to data security policies and standards in cloud data solutions.
- Collaborate with cross-functional teams to create scalable, available, and performant cloud data solutions.
- Evaluate and recommend cloud products and services for efficient data management.
- Monitor cloud data solution performance and implement optimization strategies.
- Ensure compliance with data governance policies and cloud best practices.
- Implement data management processes and document cloud data engineering strategies.
- Perform system performance assessments and recommend improvements.
- Lead the design and development of data layers, including reviewing ETL requirements.
- Facilitate team design sessions and lead decision-making processes.
- Ensure timely delivery of solutions that meet requirements and expectations.
Qualifications:
Minimum Experience/Skills:
- 5+ years of experience in cloud data architecture, data warehousing, data modeling, and system engineering.
- Extensive experience with data warehouse and enterprise data architecture principles, including decentralized architectures like Data Mesh.
- 3-5 years of experience developing cloud-based data solutions, particularly with AWS.
- 5+ years of experience with cloud database platforms, such as Snowflake.
- Expertise in creating and maintaining end-to-end data pipelines.
- Strong communication skills to articulate solutions to technical stakeholders and business consumers.
- Proficiency with Unix, Linux, and shell scripting.
- Experience managing large, multi-terabyte enterprise data platforms in the cloud.
- Thorough understanding of system administration, database administration, storage architectures, backup/recovery, and networking.
- 3-5 years of experience with Informatica (IICS) ETL, Snowflake Data Warehousing, and Airflow orchestration.
- Experience working with sensitive healthcare information, and familiarity with HIPAA and other data privacy controls.
Preferred Experience/Skills:
- AWS Certification or Cloud Data Engineer Certification.
- Snowflake certification.
- Experience with Cognos and/or Tableau reporting.