Kagool
Create a free account
One signup unlocks everything for this job:
- Apply to this job and track your application
- See your personalized profile match score
- Generate an AI cover letter for this role
Enterprise Software & Network Solutions · 201 to 500 Employees · Birmingham, United Kingdom
Hiring: Engineering
What employees say
+ Literally none. Awful leaders. No hold on their finance and takes them months to pay
− I honestly would run away. It’s just filled with people who went to college together and refuse to pay you
More reviews on the company profile.
Interview experience · Data Engineer
First HR will call you after screening Next round with tech lead/architect for 30-45 mins The interviewers was soft spoken, respectful and trying to understand your hands on live experience with tech stack and projects I
India · 6 reports
More interview insightsJob description
Overview
About Kagool \nWe are a fast-growing IT consultancy specializing in the transformation of complex Global enterprises that use SAP.
Responsibilities
- About Kagool
- \nWe are a fast-growing IT consultancy specializing in the transformation of complex Global enterprises that use SAP. We are looking for hard working individuals to help deliver for our global customer base. We embrace the opportunities of the future and work proactively to make good use of technology. As you can imagine, this means that we have a vibrant and diverse mix of skills and people making Kagool a great place to work.
- \nJob Summary
- \nWe are looking for an experienced Senior Data Engineer with 7+ years of experience in designing, developing, and supporting scalable data engineering solutions.
- \nThe ideal candidate must have strong hands-on experience with Apache Spark, PySpark, Kafka, Spark Structured Streaming, Python, SQL, and Microsoft Azure. The candidate should have practical experience implementing and managing Kafka-based streaming solutions on Azure and working with real-time data processing pipelines.
- \nAzure experience is mandatory for this role.
- \nKey Responsibilities
- \n\n • Design, develop, and maintain scalable batch and real-time data pipelines using Apache Spark and PySpark.
- \n • Develop and support Kafka-based real-time streaming pipelines on Microsoft Azure.
- \n • Implement real-time data processing using Spark Structured Streaming.
- \n • Work with Kafka topics, partitions, consumer groups, offsets, retention, and message delivery mechanisms.
- \n • Implement Kafka producers and consumers for high-volume data ingestion and processing.
- \n • Integrate Apache Kafka with Azure data services and downstream data platforms.
- \n • Develop data transformation, cleansing, aggregation, and enrichment processes.
- \n • Implement checkpointing, fault tolerance, error handling, retry mechanisms, and recovery strategies for streaming applications.
- \n • Design and optimize Spark jobs for performance, scalability, and efficient resource utilization.
- \n • Work with Azure data services such as Azure Databricks, ADLS Gen2, Azure Event Hubs, Azure Data Factory, and Azure Synapse Analytics.
- \n • Develop SQL queries and work with relational and analytical databases.
- \n • Implement data quality and validation frameworks.
- \n • Troubleshoot production issues and perform root-cause analysis.
- \n • Collaborate with Data Architects, Application Teams, DevOps, and Business stakeholders.
- \n • Participate in technical design discussions, code reviews, and CI/CD activities.
- \nMandatory Technical Skills
- \n\n • 7+ years of experience in Data Engineering / Big Data.
- \n • Strong hands-on experience with Apache Spark and PySpark.
- \n • Strong hands-on experience with Apache Kafka.
- \n • Mandatory hands-on experience with Kafka on Azure.
- \n • Mandatory Microsoft Azure experience.
- \n • Strong experience with Spark Structured Streaming.
- \n • Strong programming skills in Python.
Verify Python for this role
Free PRACTICE for Python adds a verified skill signal that can boost how roles like this rank for you — optional, not required to browse or apply.
Sign up — verify Python free →Want match scores and deeper insight? Create a free candidate profile
Job application FAQ
Last updated 3 October 2026
- How do I apply for Senior Data Engineer - Spark Streaming & Kafka at Kagool?
- Open this job on Feedback AI and sign up free (or sign in). You can apply from the job detail page; match scores and skill gaps appear when your profile is complete. Kagool may attach structured assessments to interview rounds after you apply.
- Does Feedback AI show a match score for this Senior Data Engineer - Spark Streaming & Kafka role?
- Yes after you add a resume or skills. Match score ranks this role against your profile — skills met, gaps, level, and location — so you prioritise applications where fit is strongest.
- Is Senior Data Engineer - Spark Streaming & Kafka remote or on-site in Indore, Madhya Pradesh?
- This listing is marked on_site with location Indore, Madhya Pradesh. Confirm details on the job description before applying.
- What skills matter for this Senior Data Engineer - Spark Streaming & Kafka role at Kagool?
- This is a senior Senior Data Engineer - Spark Streaming & Kafka role. Key skills include skills listed in the job description. Feedback AI highlights matched vs missing skills on your profile after signup.