AWS Databricks Platform Administrator

2 days ago

Apply Now

Receive Emails with Similar Jobs

The Baldwin Group

WebsiteLinkedInAll Job Openings

Commercial Risk Management • Private Risk Management • Personal Insurance • Employee Benefits • Asset and Income Protection

1001 - 5000 employees

Founded 2011

💸 Finance

Description

• Manage and maintain role-based access to data and features in the Databricks Platform using Unity Catalog • Implement external access controls for outside teams using service principles, SQL warehouses and Delta Sharing • Work with platform users to solve problems and facilitate work • Create infrastructure for AWS and Databricks using Terraform: S3, IAM roles, Instance Profiles, KMS Keys • Improve processes and systems used to manage infrastructure, users and external access • Keep track of unused assets for pruning • Understand and implement best practices for security and compliance • Implement service principles and access tokens for external users • Understand how to utilize Databricks APIs to automate administrative tasks • Create queries and dashboards to monitor critical systems and processes • Create documentation for users and admins • Automate common admin tasks using Databricks notebooks • Manage workflow tags, cluster tags, workflow naming convention enforcement • Clusters health check and best practices implementation • Regular back up & recovery • Privilege reviews on users and resources

Requirements

• 4 or more years of experience administering the Databricks platform • Strong understanding of data engineering needs • Experience creating AWS infrastructure with Terraform • Strong understanding of AWS infrastructure required by Databricks • Experience with compliance audits/audit documentation • Experience using Databricks APIs with tools like Postman • Understanding of the Databricks workspace and development environment • Experience using GitHub to manage source code • Strong SQL skills for creating admin related queries and dashboards • Strong debugging skills • Ability to do risk analysis for 3rd party integrations with AWS resources • Knowledge of Python a plus • Familiarity with Datadog