[ref. c86966716] Raritan - Platform Automation Engineer 24-12578
APN Consulting Inc Raritan
APN Consulting has an immediate need for a direct client requirement: Title - Platform Automation Engineer Location: Raritan/Bridgwater/Trenton NJ Contract Remote/Hybrid(1 day onsite) Note - Candidate needs to be 1 day onsite to any of the location (Raritan/Bridgwater/Trenton) About the Role We are seeking a skilled Platform Automation Engineer to design, implement, and maintain automated workflows and scalable solutions for our Databricks and Data Lake environments.
You will play a pivotal role in optimizing data platform performance, ensuring data integrity, and enabling efficient data pipelines through automation. Key Responsibilities Platform Automation & Management: Develop and maintain Infrastructure-as-Code (IaC) solutions for Databricks and cloud-based Data Lakes using tools like Terraform or CloudFormation.
Automate deployment, scaling, and management of Databricks clusters and services. Implement monitoring, logging, and alerting systems for platform performance and uptime. Data Engineering Support: Collaborate with data engineering teams to design and deploy automated ETL pipelines for data ingestion, transformation, and processing.
Optimize Data Lake storage patterns and structures for high performance and cost efficiency. Cloud Integration: Integrate Databricks and Data Lake solutions with cloud platforms (AWS, Azure, GCP Configure secure connections for external data sources and downstream applications.
Security & Compliance: Implement automated security controls, ensuring compliance with organizational and regulatory standards. Conduct regular audits to verify access controls, encryption, and data integrity. Continuous Improvement: Evaluate new tools and technologies to enhance platform efficiency and automation.
Design and implement CI/CD pipelines for Databricks notebooks and data workflows. Required Qualifications Education: Bachelor's degree in Computer Science, Engineering, or related field. Experience: 3+ years of experience in platform automation, infrastructure engineering, or DevOps roles.
Hands-on experience with Databricks, Spark, and large-scale data platforms. Expertise in Data Lake architecture and management (e.g., Delta Lake, Apache Iceberg Proficiency in scripting/programming languages (Python, Scala, Shell Strong knowledge of cloud platforms (AWS S3, Azure Data Lake, GCP BigQuery Preferred Skills Familiarity with orchestration tools like Apache Airflow or Azure Data Factory.
You will play a pivotal role in optimizing data platform performance, ensuring data integrity, and enabling efficient data pipelines through automation. Key Responsibilities Platform Automation & Management: Develop and maintain Infrastructure-as-Code (IaC) solutions for Databricks and cloud-based Data Lakes using tools like Terraform or CloudFormation.
Automate deployment, scaling, and management of Databricks clusters and services. Implement monitoring, logging, and alerting systems for platform performance and uptime. Data Engineering Support: Collaborate with data engineering teams to design and deploy automated ETL pipelines for data ingestion, transformation, and processing.
Optimize Data Lake storage patterns and structures for high performance and cost efficiency. Cloud Integration: Integrate Databricks and Data Lake solutions with cloud platforms (AWS, Azure, GCP Configure secure connections for external data sources and downstream applications.
Security & Compliance: Implement automated security controls, ensuring compliance with organizational and regulatory standards. Conduct regular audits to verify access controls, encryption, and data integrity. Continuous Improvement: Evaluate new tools and technologies to enhance platform efficiency and automation.
Design and implement CI/CD pipelines for Databricks notebooks and data workflows. Required Qualifications Education: Bachelor's degree in Computer Science, Engineering, or related field. Experience: 3+ years of experience in platform automation, infrastructure engineering, or DevOps roles.
Hands-on experience with Databricks, Spark, and large-scale data platforms. Expertise in Data Lake architecture and management (e.g., Delta Lake, Apache Iceberg Proficiency in scripting/programming languages (Python, Scala, Shell Strong knowledge of cloud platforms (AWS S3, Azure Data Lake, GCP BigQuery Preferred Skills Familiarity with orchestration tools like Apache Airflow or Azure Data Factory.
Experience with containerization and Kubernetes for scalable application deployments. Certification in Databricks, AWS, Azure, or GCP.
Intone NetworksLansdale, 41 mi from Raritan (NJ)
Job Title: Automation Engineer who will work NIGHT shiftLocation:THIS ROLE IS ONSITE IN LANSDALE, PADuration:9+MonthsExperience: 4-5 yearsTravel: NoBGV Fully ClearedClient laptop Must Have Skills: Experience with having worked on night shift...
Michael PageLong Island (NY), 40 mi from Raritan (NJ)
and product quality.
• Collaborate with cross-functional teams, including engineering, production, and quality, to ensure smooth automation implementation.
• Manage the selection, installation, and validation of automated machinery, robotics, and software...
Robert HalfAllentown, 44 mi from Raritan (NJ)
We are offering an Automation Engineer a long-term contract employment opportunity in Allentown, Pennsylvania. As part of our team, you will have the opportunity to apply your strong computer skills, engineering knowledge and programming abilities...