PySpark

apartmentSage IT INC placeNew York calendar_month1/8/25

ETL Developer (5+years) to create data pipeline ETL Jobs using AWS Glue and PySpark within the financial services industry.Responsibilities:Work with a scrum team(s) to deliver product stories according to priorities set by the business and the Product (link removed)teract with stakeholders.Provide knowledge transfer to other team members.Creating and testing pipeline jobs locally using aws glue interactive session.Performance tuning of PySpark jobs.AWS Athena to perform data analysis on Lake data populated into aws glue data catalog through aws glue crawlers.Must Haves:Responsible for designing, developing, and maintaining ETL processes to support data integration and business intelligence initiatives.Need to closely work with stakeholders to understand data requirements and ensure efficient data flow and transformation using ETL tools and PySparkDevelop and implement ETL processes using with one of ETL tool and PySpark to extract, transform, and load data.4+ years of experience in ETL development with knowledge on Pyspark5+ years as an ETL DeveloperSQL expertAWS Glue WITH Python ( PySpark )PySpark Dataframe APISpark SQLKnowledge in AWS services (e.g. DMS, S3, RDS, Redshift, Step FunctionNice to Haves:Etl development experience with tools e.g. SAP BODS, Informatica.Good understanding of version control tools like Git, GitHub, TortoiseHg.Financial services experienceAgile Digital : PySpark ETL Developer (5+years) to create data pipeline ETL Jobs using AWS Glue and PySpark within the financial services industry.Responsibilities:Work with a scrum team(s) to deliver product stories according to priorities set by the business and the Product (link removed)teract with stakeholders.Provide knowledge transfer to other team members.Creating and testing pipeline jobs locally using aws glue interactive session.Performance tuning of PySpark jobs.AWS Athena to perform data analysis on Lake data populated into aws glue data catalog through aws glue crawlers.Must Haves:Responsible for designing, developing, and maintaining ETL processes to support data integration and business intelligence initiatives.Need to closely work with stakeholders to understand data requirements and ensure efficient data flow and transformation using ETL tools and PySparkDevelop and implement ETL processes using with one of ETL tool and PySpark to extract, transform, and load data.4+ years of experience in ETL development with knowledge on Pyspark5+ years as an ETL DeveloperSQL expertAWS Glue WITH Python ( PySpark )PySpark Dataframe APISpark SQLKnowledge in AWS services (e.g. DMS, S3, RDS, Redshift, Step FunctionNice to Haves:Etl development experience with tools e.g. SAP BODS, Informatica.Good understanding of version control tools like Git, GitHub, TortoiseHg.Financial services experienceAgile ,ETL Developer (5+years) to create data pipeline ETL Jobs using AWS Glue and PySpark within the financial services industry.Responsibilities:Work with a scrum team(s) to deliver product stories according to priorities set by the business and the Product (link removed)teract with stakeholders.Provide knowledge transfer to other team members.Creating and testing pipeline jobs locally using aws glue interactive session.Performance tuning of PySpark jobs.AWS Athena to perform data analysis on Lake data populated into aws glue data catalog through aws glue crawlers.Must Haves:Responsible for designing, developing, and maintaining ETL processes to support data integration and business intelligence initiatives.Need to closely work with stakeholders to understand data requirements and ensure efficient data flow and transformation using ETL tools and PySparkDevelop and implement ETL processes using with one of ETL tool and PySpark to extract, transform, and load data.4+ years of experience in ETL development with knowledge on Pyspark5+ years as an ETL DeveloperSQL expertAWS Glue WITH Python ( PySpark )PySpark Dataframe APISpark SQLKnowledge in AWS services (e.g. DMS, S3, RDS, Redshift, Step FunctionNice to Haves:Etl development experience with tools e.g. SAP BODS, Informatica.Good understanding of version control tools like Git, GitHub, TortoiseHg.Financial services experienceAgile, 6-8 Experience, The Job start date is Feb-3-2025 , 4 Months Duration , ETL lead with PySpark expertise

thumb_up_altRecommended

Python, Digital : Amazon Web Service(AWS) Cloud Computing, Digit

apartmentSage IT INCplaceNew York

Developer Digital : Python, Digital : Amazon Web Service(AWS) Cloud Computing, Digital : PySpark, MySQL AWS Glue, PySpark, AWS CDK, Python, SQL, Snowflake, Kubernetes ,AWS Glue, PySpark, AWS CDK, Python, SQL, Snowflake, Kubernetes, 8-10 Experience...

electric_boltImmediate start

Informatica Cloud Developer - EDP

apartmentSeven Seven SoftwaresplaceNewark, 9 mi from New York

option based on project needs-Translate business needs into master data solutions utilizing process flows and data flow techniques.-Have good experience with PYSPARK, Spark ETL, SPARK SQL, Data Sets and Data Frames. Work with IDQ/BDQ Address doctor, IDQ...

check_circleNew offer

AL/ML Engineer

apartmentRadiant DigitalplaceRochelle Park, 14 mi from New York

with experience in related tools (Pig, Spark, Scoop, Oozie, HDFS, Bteq, FastExport, MLOAD, etc.) • Extensive Experience in: SQL, Hive, Sqoop, OOZIE, Unix/Linux KSH, deploying PySpark or Scala models in production Hadoop cluster. • Experience in large scale...