Databricks Administrator

Role overview A platform administration position focused on operating and securing an Azure Databricks environment that supports enterprise analytics workloads in a regulated financial services setting. The role blends day-to-day cluster and workspace administration with broader security governance, cost optimization, and collaboration with data engineering teams.

Responsibilities - Configure, maintain, and administer Azure Databricks workspaces and clusters across development, staging, and production environments. - Monitor cluster health, performance metrics, resource utilization, and overall platform availability, troubleshooting infrastructure, connectivity, and workload failures as they arise. - Tune cluster, SQL warehouse, and workspace configurations to balance performance with cost efficiency. - Administer user access, workspace permissions, group-based security controls, and SQL warehouse settings, including Unity Catalog permissions, metastores, catalogs, schemas, tables, volumes, and external locations. - Govern workspace standards such as job scheduling, notebook organization, audit logging, compute policies, instance pools, and service principal ownership. - Coordinate recurring access reviews, audit evidence collection, and remediation of excessive or outdated permissions, while supporting disaster recovery planning, capacity planning, and platform improvement initiatives. - Manage secrets, tokens, storage credentials, and Key Vault-backed configurations, and collaborate with data engineering teams to establish usage standards and Delta Lake performance practices.

Requirements - Hands-on experience administering Azure Databricks workspaces, clusters, jobs, notebooks, SQL warehouses, instance pools, compute policies, and audit logs. - Working knowledge of Unity Catalog administration, workspace security, service principals, and platform governance controls. - Experience supporting Databricks in financial services or another regulated environment with audit, access review, and compliance requirements. - Familiarity with identity and access management concepts, including RBAC, SCIM provisioning, and Microsoft Entra ID integration. - Background in database, cloud platform, or data platform operations supporting enterprise analytics workloads. - Ability to build runbooks, operational standards, monitoring procedures, and administrator documentation for Databricks environments.

Nice to have - Databricks Platform Administrator certification or equivalent demonstrated expertise. - Experience with Azure platform integration, including storage, Key Vault, private endpoints, virtual networks, firewalls, Azure Monitor, and Log Analytics. - Comfort with automation tooling such as the Databricks CLI, REST APIs, Terraform, Python, PowerShell, and SQL for scripting and operational tasks.