Software Engineer, Batch Compute

2 days ago

Apply Now

Receive Emails with Similar Jobs

Tecton

WebsiteLinkedInAll Job Openings

Machine Learning • Data Science • Feature Store • Data Engineering • Artificial Intelligence

11 - 50

Description

• Tecton helps companies unlock the full potential of their data for AI applications. • The platform streamlines the complex process of preparing and delivering data to models. • Tecton is funded by Sequoia Capital, Andreessen Horowitz, and Kleiner Perkins. • Our team has years of experience building and operating business-critical machine learning systems at leading tech companies like Uber, Google, Meta, Airbnb, Lyft, and Twitter. • As a staff-level engineer on the Batch Compute team, you’ll play a critical role in architecting, designing, and scaling the core compute engines and storage architecture used by every Tecton customer. • You will be working in one or more of the following areas to build the next generation of Tecton infrastructure: Distributed compute and resource management, Query optimization and distributed execution, Cross-platform integrations with state-of-the-art data platforms.

Requirements

• Experience working in large Python, Java, Kotlin, or Go codebases and running cloud-native Spark systems (e.g. AWS EMR, Databricks, GCP Dataproc) • Experience in performance tuning of Spark, Ray, Maestro, or Airflow jobs • Knowledge of data formats such as Parquet, Avro, Arrow, Iceberg, or Delta Lake and object storage (e.g. S3, GCS) • Expertise with cloud-scale query performance, query optimization, query planning, heuristic query execution techniques, and cost-driven optimizations • Experience with internals of distributed systems, SQL/NoSQL databases, data lakes, or data warehouses • Strong communication skills and ability to write detailed technical specifications • Excitement about coaching and mentorship of junior engineers • BSc, MS, or PhD in Computer Science or related fields • 8+ years of experience in building product software systems • 5+ years of technical leadership experience for a group of engineers

Benefits

• Competitive equity • Comprehensive benefits such as medical, dental, vision, life, 401(K) • Flexible paid time off • 10 paid holidays each calendar year • Sick time • Leave of absence as per the FMLA and other relevant leave laws