Big Data Engineer - Apache Spark

February 10

Apply Now

Receive Emails with Similar Jobs

Innosphere

WebsiteLinkedInAll Job Openings

Innosphere is a solutions company dedicated to turning innovative ideas into successful outcomes through technology enablement. We specialize in providing tailored professional services, staff augmentation, and tech recruitment aimed at helping enterprises and scale-ups address their technology-related challenges. Our focus is on delivering cloud-first solutions and measurable business impact, enabling clients to achieve their objectives efficiently and effectively while fostering long-term relationships to understand their needs and aspirations.

51 - 200 employees

Founded 1997

☁️ SaaS

🏢 Enterprise

📋 Description

• This is a remote position. • We are seeking an experienced Big Data Engineer with a strong proficiency in Apache Spark to join our team. • The ideal candidate will have strong SQL skills, proficiency in programming languages such as Python, Scala or Java, and experience with cloud platforms and related data services. • Design, develop, and optimize scalable big data processing solutions using Apache Spark. • Build and maintain robust data pipelines to support advanced analytics and data-driven insights. • Collaborate closely with software engineers to understand data requirements and deliver efficient, scalable solutions. • Optimize Spark applications for performance, including tuning, scaling, and performance improvement strategies. • Implement data quality checks and ensure data integrity across large datasets. • Identify and resolve performance bottlenecks in Spark applications to enhance efficiency. • Ensure data security, compliance, and governance standards are met.

🎯 Requirements

• 5+ years of experience in big data engineering roles. • Strong proficiency with Apache Spark ( PySpark /Scala/Java) for large-scale data processing. • Solid understanding of distributed computing principles. • Proven experience in scaling, tuning, and optimizing Spark applications for performance improvement. • Nice to have experience with big data technologies like Hadoop, Hive, Kafka, HDFS, etc. • Proficiency in programming languages such as Python, Scala, or Java; knowledge of C# is a bonus. • Strong SQL skills for data querying and manipulation. • Experience with cloud platforms (AWS, GCP, Azure) and related data services. • Familiarity with CI/CD pipelines and version control systems (Git). • Nice to Have: Experience with real-time data processing and streaming technologies. • Knowledge of containerization and orchestration tools (Docker, Kubernetes). • Exposure to data lake architectures and modern data stack tools. • Strong problem-solving skills and attention to detail.

🏖️ Benefits

• Generous Time Off • Above Average Benefits (Dental, Health, Vision, STD, LTD, AD&D, Life, Dependent Life) • Home Office Benefit Program • Yearly Professional Development Budget • Annual Fitness Benefit • Lunch & Learns and Virtual Events • Annual In-Person Celebrations • RRSP with Company Match • Volunteer Days • Sports Tickets for Blue Jays games • Birthday Club