Data Engineer
Position Summary:
We are seeking a Data Engineer who is passionate about creating highly scalable and efficient data infrastructure that drives key business insights. As a Data Engineer, you will work with cutting-edge technologies and collaborate with data analysts, and cross-functional engineering teams to design, build, and optimize data architectures that empower data-driven innovation across the organization.
This role requires someone who thrives in an agile, fast-paced environment and has a deep understanding of building data infrastructure that scales with rapid business growth and change.
Responsibilities:
Build and Optimize Data Pipelines: Analyze and organize raw data from various sources. Design, develop, and manage scalable, high-performance and resilient data pipelines that process, transform, and make data accessible for real-time analytics and reporting.
Data Infrastructure Development: Design, build, and maintain data infrastructure to ensure data is reliable, accessible, and secure for various applications and stakeholders.
Data Integration: Ingest data from various internal and external data sources (APIs, databases, flat files, documents) into centralized data storage systems using modern ELT tools and techniques.
Streaming & Batch Processing: Build streaming or batch data processing solution that is reusable, fault-tolerant and scalable.
Data Quality & Governance: Implement data validation and quality control checks to ensure accuracy, consistency, and completeness across all data pipelines and datasets.
Collaboration & Cross-Functional Partnerships: Collaborate with data analysts, product and platform engineers, and business stakeholders to define and implement data requirements and deliver end-to-end solutions.
Data Warehouse Design: Design, implement, and optimize data lake, data warehouse and data mart to meet analytics and reporting needs.
Innovation & Automation: Identify opportunities to automate data workflows, streamline processes, and introduce best-in-class tools and frameworks that enhance productivity and efficiency.
Performance Tuning: Continuously monitor and tune data pipelines and infrastructure for optimal performance, scalability, and cost-efficiency.
Qualifications:
Experience: 4+ years of experience as a Data Engineer or in a similar role, with a proven track record of building and maintaining data pipelines and infrastructure at scale.
Programming Skills: Strong proficiency in SQL, JavaScript, Python, DAX, Power Query or similar language for data processing and automation.
Databases & Warehouses: Strong experience in a variety of SQL and cloud-based data platform like Snowflake. Advanced proficiency with SQL skills writing complex queries and optimizing performance.
ELT Tools: Experience with modern ELT tools and frameworks for building scalable data workflows (e.g., Apache Airflow, Azure DataFactory, dbt, Fivetran or similar).
Data Lake & Lakehouse Design: Expertise in designing and managing data warehouse, data lake, or data lake-house storage architecture for analytics and reporting.
Data Streaming: Experience with real-time data processing and streaming platforms such as Kafka, Spark Streaming, Kinesis or similar technologies.
Automation & DevOps: Familiarity with CI/CD practices, infrastructure as code (IaC), and containerization (Docker, Kubernetes) to automate and manage data infrastructure.
Data Governance: Knowledge of data governance practices, including data lineage, metadata management, and compliance with regulatory standards (e.g., GDPR, HIPAA).
Agile Methodologies: Experience working in Agile or Scrum teams with the ability to iterate and deliver rapidly.
Preferred Qualifications:
Machine Learning Support: Knowledge of building data pipelines that support machine learning and generative AI models and AI-driven products.Data Visualization: Experience working with data visualization tools like PowerBI, Sigma and Domo to make data insights accessible
Company Overview:
Vista is a leading global investment firm that exclusively invests in enterprise software, data and technology-enabled organizations across private equity, permanent capital, credit and public equity strategies, bringing an approach that prioritizes creating enduring market value for the benefit of its global ecosystem of investors, companies, customers and employees.Vista’s investments are anchored by a sizable long-term capital base, experience in structuring technology-oriented transactions and proven, flexible management techniques that drive sustainable growth. Vista believes the transformative power of technology is the key to an even better future – a healthier planet, a smarter economy, a diverse and inclusive community and a broader path to prosperity.
Further information is available at vistaequitypartners.com. Follow Vista on LinkedIn, @Vista Equity Partners, and on X, @Vista_Equity.