Senior Data Engineer

17 hours ago

Apply Now

Receive Emails with Similar Jobs

Xmartlabs

WebsiteLinkedInAll Job Openings

Startup Development • UX/UI Design • Computer Vision • Machine Learning • dApps Design & Development

51 - 200

Description

• Architect, implement, and optimize end-to-end data pipelines to support machine learning workflows. • Gather, clean, and preprocess large volumes of data. • Design and implement strategies for real-time data ingestion and integration. • Build scalable and reliable data pipelines from systems including CRM and Ecommerce. • Drive data quality initiatives across the organization. • Develop and enforce best practices for standard naming conventions and coding practices. • Continuously improve our data infrastructure and stay ahead of technology advancements. • Build strong cross-functional partnerships with Data Scientists, Machine Learning Engineers, Product Managers, and Software Engineers to understand data needs and deliver on those needs. • Perform production support and deployment activities for data platforms.

Requirements

• Excellent written and verbal communication skills in English. • Proficiency in Spark and Python. • Databricks knowledge is a plus. • Experience in SQL programming, Postgres, MySql. • Experience with various data formats like Parquet, HUDI, etc. • Experience with Azure Data Factory or another orchestration tool such as airflow. • Knowledgeable in at least one cloud provider solution. • Expertise in Azure is a plus. • Strong problem-solving skills and ability to analyze and translate business requirements into technical solutions. • Excellent communication and collaboration skills to work effectively with cross-functional teams. • Ability to drive data quality initiatives and establish best practices. • Self-motivated with a passion for continuous learning and staying ahead of industry trends.