Senior Data Engineer
EtiVenture, Inc San Antonio
Identify and evaluate emerging technologies, tools, and trends that can drive innovation and improve the efficiency and effectiveness of our data engineering processes. Design, develop, and own robust, scalable data processing and data integration pipelines using Python, Airflow, PySpark, SparkSQL, and REST API endpoints to ingest data from various external data sources to Data Lake.
Design and develop batch and streaming data pipelines using Apache Kafka, Flink etc. Design and develop dimensional data model techniques (snowflake and star schemas, dimension, fact tables, slowly changing dimensions, periodic snapshot tables etc.) Design and build data models for optimal storage and retrieval.
Collaborate with cross-functional teams, such as data scientists, software engineers, and business stakeholders to understand data requirements and deliver solutions that meet business needs. Enforce data governance policies and practices to maintain data integrity, security, and compliance with relevant regulations.Work with developers to implement software processes and tools such as Git, CI/CD pipelines, and orchestration tools like Airflow.
REQUIREMENTS: Bachelor degree in Computer Science, Information Technology or related or foreign equivalent + 60 months of experience in the job offered. Employer will accept an Associate's degree in Computer Science or foreign equivalent or three years of academic studies or foreign equivalent towards a Bachelor of Science Degree in Computer Science.Experience in using Kafka, Hive and Sqoop tools are required.
Global Channel Management, IncSan Antonio
Security Clearance Data Engineer needs 9 years total relevant experience and security clearance
Security Clearance Data Engineer requires: Security Clearance-Current active TS, SCI eligible
9 years total relevant experience plus the following: BS...
Grassroots CarbonSan Antonio
Villages to reduce their carbon impact and reach their sustainability goals.
About the Lead Data Science role:
As the Lead Data Scientist/Data Engineer, you will play a pivotal role in designing, building, and managing the data infrastructure that powers...
Robert HalfSan Antonio
with Business Intelligence tools
• Address model performance issues such as overfitting, underfitting, and bias
• Develop and maintain data pipelines for efficient data retrieval and processing
• Collaborate with applications and data engineering teams to deploy...