Overview

AIA: Pune Skill: Azure databricks with Pyspark Experience: 6.5- 9.5Years

Responsibilities

• Design robust data models and Databricks SQL queries that transform raw data into reliable structured datasets enabling analytics teams to derive accurate insights and measurable business outcomes
• Develop scalable and efficient data pipelines in Databricks that process large datasets with high performance ensuring timely availability of curated data for downstream applications and reporting
• Optimize Databricks SQL code and workspace configuration to improve query performance resource utilization and cost efficiency while maintaining stability and consistency across environments
• Collaborate with product owners and data consumers to translate business requirements into technical specifications ensuring that every data solution directly supports clear organizational goals and priorities
• Implement rigorous data quality checks and validation rules within Databricks workflows ensuring that all delivered datasets meet defined accuracy completeness and consistency standards for critical decision making
• Create reusable components and standardized patterns for Databricks SQL development enabling faster delivery of new data solutions and promoting consistency across projects and teams
• Automate deployment and scheduling of Databricks jobs using available tools to ensure reliable execution monitoring and recovery processes that support uninterrupted day shift operations
• Document data pipelines data models and transformation logic in a clear and structured manner so that other team members can easily understand maintain and extend implemented solutions
• Work closely with cloud and platform teams to ensure secure and compliant use of Databricks resources aligning data solutions with enterprise policies and industry regulations that protect customer data and trust
• Perform root cause analysis for data issues and performance bottlenecks in Databricks environments providing targeted fixes and long term improvements that enhance overall system resilience
• Support business stakeholders during testing and rollout phases by explaining data logic helping with query usage and ensuring that delivered analytics solutions are fully adopted and generate practical impact
• Continuously research new Databricks features and data engineering techniques and apply relevant improvements to existing solutions so that the organization stays current and competitive in its use of data
• Contribute to a culture of collaboration and shared responsibility by providing constructive feedback participating in code reviews and helping peers solve complex Databricks SQL challenges that affect project success

Qualifications

• Job Summary
• Sr Developer with 5 to 8 years of experience will design and optimize data solutions using Databricks SQL within a hybrid work model focused on day shift work. The role will build scalable data pipelines and analytics assets that enable business teams to make reliable decisions and deliver measurable value to customers and society through responsible data use and insight generation.
• Demonstrate strong hands on expertise in Databricks SQL including complex joins window functions aggregations and performance tuning to deliver high quality analytical datasets
• Show proven experience building and maintaining Databricks workflows and jobs for end to end data ingestion transformation and delivery across hybrid working arrangements and day shift schedules
• Apply solid knowledge of relational databases data warehousing concepts and data modeling techniques to design structures that support reporting self service analytics and advanced data products
• Utilize practical experience with version control and collaborative development practices to manage Databricks artifacts ensuring traceability and controlled evolution of data solutions
• Exhibit effective communication skills that enable clear discussion of technical topics with nontechnical stakeholders ensuring that data work remains aligned with business and societal value objectives