Azure Databricks Platform Admin

Azure Databricks Platform Administration:

Administer and manage Azure Databricks workspaces, clusters, and jobs
Configure and maintain Databricks environments (Dev, QA, Prod)
Monitor cluster performance, job execution, and system health
Manage notebooks, libraries, and dependencies
Perform upgrades, patches, and platform maintenance

Cluster Management & Optimization:

Design and manage cluster configurations (auto-scaling, instance types, pools)
Optimize job performance, resource utilization, and execution times
Troubleshoot cluster failures, memory issues, and job bottlenecks
Implement best practices for Spark performance tuning

Security & Governance:

Implement and enforce data security and access controls (RBAC, ACLs, IAM)
Manage workspace security, secrets (Key Vault integration), and credential passthrough
Ensure compliance with enterprise security and governance policies
Implement data governance, auditing, and lineage tracking

Azure Cloud Integration:

Integrate Databricks with Azure services such as:

Azure Data Lake Storage (ADLS Gen2)

Azure Key Vault:

Azure Data Factory (ADF)

Azure Synapse Analytics:

Manage connectivity and networking (VNet, Private Endpoints, NSGs)

✅ Core Skills

10–15 years of IT experience with strong Azure Databricks administration
Hands-on expertise in Apache Spark and Databricks platform management
Strong experience with Azure cloud services (ADLS, ADF, Key Vault, Networking)
Proven experience in production support and platform operations

✅ Technical Skills

Deep knowledge of cluster configuration, job scheduling, and performance tuning
Strong understanding of Spark architecture and optimization techniques
Experience with Terraform / ARM / Bicep (IaC tools)
Knowledge of security models, IAM, RBAC, and data governance
Familiarity with CI/CD tools and DevOps practices

Preferred Skills:

Experience with Delta Lake and Lakehouse architecture:

Exposure to Python / Py Spark / Scala
Familiarity with monitoring tools (Azure Monitor, Splunk)
Knowledge of multi-cloud environments (AWS/GCP)
Experience in data engineering or ETL pipelines

Education: MCA,MSc,MTech,BCA,BSc,BTech

Preferred skills: Technology->Cloud Platform->Azure Analytics Services->Azure Databricks