Overview

Technology: Senior Cloud Engineer

Responsibilities

• Infrastructure as Code (Terraform-first): Author and maintain reusable Terraform modules provisioning the
• Network Security Architecture: Design private networking and security posture - private endpoints, hub-
• Data Lake Infrastructure: Own ADLS Gen2 at the infrastructure layer - storage architecture, hierarchical namespace, ACL strategy, lifecycle/tiering, encryption, and access patterns.
• Observability Reliability: Build and maintain the monitoring layer - Azure Monitor, Log Analytics,
• FinOps Capacity Planning: Own cost visibility and optimization across DBU and storage spend - tagging,
• CI/CD Environment Management: Own deployment pipelines and the promotion path across dev, test,
• Databricks Account Workspace Administration: Account console configuration, workspace provisioning and topology, admin role delegation, and multi-workspace strategy (day-to-day Databricks operations sit with a dedicated admin team; this role sets the standards they work from).
• Unity Catalog: Metastore design, catalog/schema/table permission models, storage credentials,
• Identity Access: Entra ID integration, SCIM provisioning, identity federation, service principals, and token policy.
• Compute Governance: Cluster policies, instance pools, autoscaling/autotermination standards, and SQL warehouse sizing.
• Cost Usage Analysis: System tables, usage attribution, and DBU forecasting, with the ability to recommend remediation for cost spend.
• Technical Partnership: Publish standards, reference configurations, and self-service patterns; support data teams on cross-cutting issues.
• Cross-Boundary Diagnostics: Investigate job failures, cluster/connectivity/permission issues, and isolate root cause across infrastructure, Databricks, or workload. ROLE SCOPE
• Production ETL/ELT pipeline development and Spark transformation work - owned by the Data Engineering team.
• Model development, training, and tuning - owned by the Data Science team.
• Dimensional modeling, dbt development, and BI/semantic layer work - outside this role.

Qualifications

• Face to Face Connect - Once a candidate is finally selected, they must visit Client s TVM office for a face-to-face interaction. This will be more of a formal/casual discussion rather than an additional interview round. The client will bear the expense.
• Compliance: For the background verification (BGV) process, PAN card and Aadhaar card of the proposed resources will be required.
• The UAN number is required for selected candidates.
• 6+ years (min. 4 years on Azure)
• Focus Areas
• Azure, Terraform, Databricks Administration
• ABOUT THE ROLE
• We are seeking a Senior Cloud Engineer to design, implement, and administer the Azure infrastructure behind our enterprise data and AI platform - centered on Azure Databricks, ADLS Gen2, Azure Data Factory, and Azure
• Machine Learning. You will own the platform foundation (infrastructure as code, network and identity architecture, compute governance, observability, and cost management) and partner closely with our
• Databricks administration, data engineering, and data science teams.
• This role suits an engineer who enjoys being the technical reference point for a platform - setting architectural direction, partnering with adjacent teams on complex problems, and translating business requirements into platform capabilities.
• Terraform: Expert-level, hands-on. Authored and maintained production Terraform modules, managed remote state, and run IaC through CI/CD - a core daily skill.
• Databricks Administration: Demonstrable hands-on experience - account/workspace administration, Unity
• Technical Partnership: Proven track record as senior technical resource to adjacent teams, guiding stakeholders from request to requirement and building consensus.
• Azure Networking Security: Deep knowledge of private endpoints, VNets/hub-spoke design, NSGs, Entra
• Data Lake Architecture: ADLS Gen2 design and access control at enterprise scale.
• Azure Data Factory: Platform-side experience with integration runtimes, managed VNet, credential management, and deployment automation.
• Automation Scripting: Strong Python, plus PowerShell and/or Bash, for platform tooling and automation.
• Spark SQL Literacy: Sufficient working knowledge to diagnose infrastructure / configuration-level performance issues; deep Spark development is not required.

Nice to have

• Databricks Asset Bundles, the Terraform Databricks provider, and workspace-as-code patterns.
• Experience leading a Unity Catalog migration or multi-workspace consolidation.
• Azure Machine Learning, MLflow, or model-serving infrastructure experience.
• Kubernetes / AKS and containerized workloads.
• Policy as code - Azure Policy, OPA, Sentinel, or Checkov.
• Event-driven infrastructure - Event Hubs, Kafka, or Stream Analytics.
• Formal FinOps practice experience.
• Multi-region or multi-tenant Databricks deployments. PREFERRED CERTIFICATIONS
• Databricks Certified Data Engineer Professional, or a Databricks platform administrator credential.
• Microsoft Certified: Azure Solutions Architect Expert (AZ-305).
• Microsoft Certified: Azure Administrator Associate (AZ-104).
• Microsoft Certified: DevOps Engineer Expert (AZ-400).
• HashiCorp Certified: Terraform Associate.