Databricks Architect

January 24

Apply Now

Receive Emails with Similar Jobs

CoEnterprise

WebsiteLinkedInAll Job Openings

CoEnterprise is a leading provider of supply chain and business analytics software and services that enable companies to make faster and smarter decisions. The company offers a suite of solutions including Syncrofy, a platform that provides supply chain and B2B/EDI visibility, compliance, and managed services. CoEnterprise specializes in helping businesses with real-time data access, EDI/B2B integration, and data analytics, leveraging advanced technologies such as AI and machine learning to improve decision-making and operational efficiency. They are recognized for their expertise in the implementation and management of IBM Sterling solutions and for providing comprehensive data analytics services with tools like Tableau.

Sterling Integrator • Sterling Selling and Fulfillment • Tableau • B2B software • supply chain

51 - 200 employees

Founded 2010

🤝 B2B

☁️ SaaS

📋 Description

• Design and implement scalable, high-performance data architectures using Databricks and the Medallion Architecture. • Lead requirement-gathering sessions with clients and internal teams to understand business needs and define best practices for solutions including data workflows. • Collaborate with cloud platform teams to optimize data storage and retrieval in environments like AWS S3, Azure Data Lake, and Delta Lake. • Translate complex data processes, such as those in Alteryx and Tableau, into optimized Databricks workflows using PySpark and SQL. • Develop reusable automation scripts to streamline workflow migrations and improve operational efficiency. • Provide hands-on development and troubleshooting support to ensure smooth implementation and optimal performance. • Partner with cross-functional teams to establish data governance frameworks, best practices, and standardized reporting processes. • Deliver training, documentation, and ongoing support to empower users and enhance organizational data literacy. • Stay ahead of industry trends, identifying opportunities to integrate new tools and methods that enhance the practice's capabilities.

🎯 Requirements

• 7+ years’ experience in data engineering or software development, with a strong focus on data architecture • Extensive experience (3-5+ years) with Databricks and the Medallion Architecture. • Expertise in Python, including developing production-grade data pipelines, reusable automation scripts, and leveraging libraries such as Pandas, and NumPy for advanced data manipulation and analytics. • Proficiency in PySpark and SQL with in-depth experience integrating a wide variety of data sources with Databricks. • Expertise in cloud platform services (AWS, Azure, GCP), including their data storage solutions (e.g., S3, Azure Data Lake Storage, Google Cloud Storage) • In-depth experience with Delta Lake, including schema enforcement, time travel, and optimization for ACID transactions. • Strong understanding of data pipelines, transformation workflows, and modeling for analytics and reporting, including scheduling, monitoring, and error handling in distributed data processing environments. • Experience in building automation tools or reusable scripts to accelerate data engineering processes. • Knowledge of Alteryx, Tableau, and Tableau Server integration is a plus.

🏖️ Benefits

• Comprehensive Health Insurance with generous employer contribution • Matching 401(k) - $$$$ • Generous PTO Policy • Virtual Events and Team Meetings • Wellness Program