Data Engineer - Spark
October 3
Description
• A Data Engineer's role involves crafting, constructing, and upholding the structure, tools, and procedures essential for an organization to gather, store, modify, and scrutinize extensive data amounts. • This position involves creating data platforms using typically provided infrastructure and establishing a clear path for Analytics Engineers who utilize the system. • Development and maintenance of ETL and data platforms (Python, Scala, Spark, HDFS, Hive) • Development and maintenance of access applications for business users and user support (Airflow, Jupyterhub, Trino, Superset, MLFlow) in the context of Kubernetes, Docker, ArgoCD • Automation and CICD (Gitlab-CI) • Monitoring (Prometheus) • R&D, maintenance, and monitoring of the platform's components • Implementing and executing policies aligned with the company's strategic plans concerning used technologies, work organization, etc.
Requirements
• Proficiency in a programming language like Python and Scala • Working with Spark messaging systems • Experience with Hadoop • Strong programming skills with a solid understanding of software engineering principles, best practices, and solutions • Experience with Version Control System, preferably GIT • Ability to actively participate/lead discussions with clients to identify and assess concrete and ambitious avenues for improvement
Benefits
• 100% remote work • Flexible working hours • Possibility to work from the office located in the heart of Warsaw • Opportunity to learn and develop with the best Big Data experts • International projects • Possibility of conducting workshops and training • Certifications • Co-financing sport card • Co-financing health care • All equipment needed for work
Apply Now