Senior GCP Data Engineer

December 14

Apply Now

Receive Emails with Similar Jobs

Experience.com

WebsiteLinkedInAll Job Openings

Social Media • Automated Workflow • Real Estate • Mortgage • Enterprise Reputation Management

51 - 200 employees

Founded 2015

☁️ SaaS

🤝 B2B

🏢 Enterprise

Description

• Come Join Us • Experience.com - We make every experience matter more • Position: Senior GCP Data Engineer • Job Location: Chennai (Base Location) / Remote • Employment Type: Full Time • Summary of Position: A Senior Data Engineer is a professional who specializes in preparing big data infrastructure for analytical or operational uses. He/She is responsible for develops and maintains scalable data pipelines and builds out new API integrations to support continuing increases in data volume and complexity. They collaborate with data scientists and business teams to improve data models that feed business intelligence tools, increasing data accessibility and fostering data-driven decision making across the organisation. • Responsibilities: Collaborate with cross-functional teams to define, prioritize, and execute data engineering initiatives aligned with business objectives. Design and implement scalable, reliable, and secure data solutions by industry best practices and compliance requirements. Drive the adoption of cloud-native technologies and architectural patterns to optimize the performance, cost, and reliability of data pipelines and analytics solutions. Mentor and lead a team of Data Engineers. Demonstrate a drive to learn and master new technologies and techniques. Apply strong problem-solving skills with an emphasis on building data-driven or AI-enhanced products. Coordinate with ML/AI and engineering teams to understand data requirements.

Requirements

• 8+ years of Strong experience in ETL and ELT data from various sources in Data Warehouses. • 8+ years of experience in Python, Pandas, Numpy, and SciPy. • 5+ years of Experience in GCP. • 5+ years of Experience in BigQuery, PySpark, and Pub/Sub. • 5+ years of Experience working with and creating data architectures. • Certified in Google Cloud Professional Data Engineer. • Advanced proficiency in Google Cloud services such as Dataflow, Dataproc, Dataprep, Data Studio, and Cloud Composer. • Proficient in writing complex Spark (PySpark) User Defined Functions (UDFs), Spark SQL, and HiveQL. • Good understanding of Elastic search. • Experience in assessing and ensuring data quality, data testing, and addressing data quality issues. • Excellent understanding of Spark architecture and underlying frameworks including storage management. • Solid background in database design and development, database administration, and software engineering across full life cycles. • Experience with NoSQL data stores like MongoDB, DocumentDB, and DynamoDB. • Knowledge of data governance principles and practices, including data lineage, metadata management, and access control mechanisms. • Experience in implementing and optimizing data security controls, encryption, and compliance measures in GCP environments. • Ability to troubleshoot complex issues, perform root cause analysis, and implement effective solutions in a timely manner. • Proficiency in data visualization tools such as Tableau, Looker, or Data Studio to create insightful dashboards and reports for business users. • Strong communication and interpersonal skills to effectively collaborate with technical and non-technical stakeholders, articulate complex concepts, and drive consensus. • Experience with agile methodologies and project management tools like Jira or Asana for sprint planning, backlog grooming, and task tracking.