Senior Manager Data Engineer - Robert Half

We are looking for a highly skilled Data Engineering and Software Engineering professional to design, build, and optimize our Data Lake and Data Processing platform on AWS. This role requires deep expertise in data architecture, cloud computing, and software development, as well as the ability to define and implement strategies for deployment, testing, and production workflows.

Key Responsibilities:

• Design and develop a scalable Data Lake and data processing platform from the ground up on AWS.
• Lead decision-making and provide guidance on code deployment, testing strategies, and production environment workflows.
• Define the roadmap for Data Lake development, ensuring efficient data storage and processing.
• Oversee S3 data storage, Delta.io for change data capture, and AWS data processing services.
• Work with Python and PySpark to process large-scale data efficiently.
• Implement and manage Lambda, Glue, Kafka, and Firehose for seamless data integration and processing.
• Collaborate with stakeholders to align technical strategies with business objectives, while maintaining a hands-on engineering focus.
• Drive innovation and cost optimization in data architecture and cloud infrastructure.
• Provide expertise in data warehousing and transitioning into modern AWS-based data processing practices.

Required Skills & Experience:

• Extensive experience in data engineering, software development, and cloud-based data solutions.
• Proven track record in migrating and optimizing data warehouses in AWS.
• Expertise in Python and Java, with a strong understanding of data structures and algorithms.
• Hands-on experience with modern data architecture, including Data Lakes, Change Data Capture, and high-scale processing frameworks.
• Strong knowledge of software development life cycle (SDLC) and Agile methodologies.
• Deep understanding of data governance, architecture, and best practices for scalable data solutions.
• Experience in building highly efficient data pipelines and working with real-time data processing tools.

Preferred Qualifications:

• Experience with data visualization tools and advanced analytics, including machine learning.
• Familiarity with cloud cost optimization strategies.
• Certifications in AWS or data engineering.