Data Engineer

Yesterday

Apply Now

Receive Emails with Similar Jobs

Mediastream

WebsiteLinkedInAll Job Openings

CDN • Media • Broadcast • OTT • Online Video Platform

51 - 200

Description

• The hire will be responsible for expanding and optimizing our data and data pipeline architecture and data flow and collection for cross-functional teams. • Create and maintain optimal data pipeline architecture • Assemble large, complex data sets that meet functional / non-functional business requirements • Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc. • Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and big data technologies • Build analytics tools that utilize the data pipeline to provide actionable insights into customer behavior, operational efficiency, and other key business performance metrics, ensuring these tools are optimized to work within big data environments and distributed systems • Design, implement, and maintain real-time streaming pipelines • Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs • Work with data and analytics experts to strive for greater functionality in our data systems

Requirements

• Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) and working familiarity with various databases • Experience building and optimizing big-data data pipelines, architectures, and data sets • Strong analytic skills related to working with unstructured datasets • Build processes supporting data transformation, data structures, metadata, dependency, and workload management • A successful history of manipulating, processing, and extracting value from large, disconnected datasets • Proficiency in developing and maintaining RESTful APIs to integrate BI tools and machine learning models into the data ecosystem • Working knowledge of message queuing, stream processing, and highly scalable big-data data stores • Experience with column-oriented and analytical databases (e.g. ClickHouse) • Experience with big data tools: Hadoop, Spark, Kafka, RabbitMQ • Experience with relational SQL and NoSQL databases, including MySQL • Experience with data pipeline and workflow management tools • Experience with Machine Learning model (e.g. regression, classification) • Experience with stream-processing systems: Storm, Spark-Streaming, etc • Proficiency in version control systems such as Git and Bitbucket • Understanding of business requirements and transformation into data queries

Benefits

• Company and team events • Ongoing training and further education • An energetic atmosphere and opportunity to grow within and with the company