Senior Data Engineer - Equity

December 24, 2024

Apply Now

Receive Emails with Similar Jobs

Fortyx

WebsiteLinkedInAll Job Openings

2 - 10 employees

Description

• We’re seeking a skilled Senior Data Engineer with experience in recommendation systems to join our team. • This role is pivotal in enhancing our backend systems, managing databases, and optimizing recommendation algorithms. • You’ll play a critical role in the architecture and performance of our recommendation infrastructure, ensuring accurate, relevant, and efficient recommendations for our users. • Data and Database Management: Design, optimize, and maintain tables and data structures to support recommendation and trending content data. • Work with structured data storage solutions, including PostgreSQL and JSONB, to manage recommendation and interaction data. • Recommendation Systems: Implement and refine recommendation algorithms (e.g., collaborative filtering, content-based, and hybrid approaches) to enhance relevancy. • Use similarity search libraries like Annoy or Faiss to optimize recommendation speed and accuracy. • Continuously evaluate recommendation logic to better serve user preferences, ensuring real-time delivery. • Data Aggregation and Analysis: Aggregate, analyze, and process user interaction data to support recommendations and trending content. • Design efficient queries and implement aggregation methods to capture relevant data and insights for recommendations. • Debugging and Optimisation: Identify, troubleshoot, and resolve data handling issues to ensure accurate recommendation delivery. • Optimise queries, processing workflows, and containerised services for high performance and scalability within AWS ECS.

Requirements

• Proven experience as a Data Engineer or Backend Engineer, with a focus on recommendation systems. • Proficient in SQL and database management, especially with PostgreSQL and JSONB for structured data handling. • Solid understanding of recommendation algorithms (collaborative filtering, content-based, hybrid approaches). • Experience with similarity search libraries such as Annoy or Faiss. • Strong programming skills in Python, with experience in building backend logic for data-intensive applications in a containerised environment. • Familiarity with AWS ECS for container management, including task scheduling and scaling. • Experience using AWS EventBridge to trigger workflows or automate tasks in response to application events. • Analytical skills for data aggregation, querying, and insights generation. • Strong debugging and optimisation skills for handling large-scale data processing in cloud-based environments.