Senior GCP Data Engineer - Databricks

December 5

Apply Now

Receive Emails with Similar Jobs

Xebia Poland

WebsiteLinkedInAll Job Openings

Software Development • Agile Development • DevOps • Scrum • Mobile Applications

1001 - 5000 employees

Founded 2001

🤖 Artificial Intelligence

☁️ SaaS

Description

• responsible for at-scale infrastructure design, build and deployment with a focus on distributed systems • building and maintaining architecture patterns for data processing, workflow definitions, and system to system integrations using Big Data and Cloud technologies • evaluating and translating technical design to workable technical solutions/code and technical specifications at par with industry standards • driving creation of re-usable artifacts • establishing scalable, efficient, automated processes for data analysis, data model development, validation, and implementation • working closely with analysts/data scientists to understand impact to the downstream data models • writing efficient and well-organized software to ship products in an iterative, continual release environment • contributing and promoting good software engineering practices across the team • communicating clearly and effectively to technical and non-technical audiences • defining data retention policies • monitoring performance and advising any necessary infrastructure changes.

Requirements

• 3+ years’ experience with GCP (BigQuery, Dataflow, Pub/Sub, Bigtable or other NoSQL database, Dataproc, Storage, Kubernetes Engine) • 5+ years’ experience with data engineering or backend/fullstack software development • strong SQL skills • Python scripting proficiency • experience with data transformation tools - Databricks and Spark • Data manipulation libraries (such as Pandas, NumPy, PySpark) • experience in structuring and modelling data in both relational and non-relational forms • ability to elaborate and propose relational/non-relational approach • normalization / denormalization and data warehousing concepts (star, snowflake schemas) • designing for transactional and analytical operations • experience with CI/CD tooling (GitHub, Azure DevOps, Harness etc.) • good verbal and written communication skills in English • Work from European Union region and work permit are required. • Nice to have: Apache Hadoop • experience with data modelling tools, preferably DBT • Enterprise Data Warehouse solutions, preferably Snowflake • familiarity with ETL tools (such as Informatica, Talend, Datastage, Stitch, Fivetran etc.) • experience in containerization and orchestration (Docker, Kubernetes etc.) • cloud (Azure, AWS, GCP) certification