Kagool
Create a free account
One signup unlocks everything for this job:
- See your personalized profile match score
- See the skills this role is missing for you
- Apply to this job and track your application
- Generate an AI cover letter for this role
Enterprise Software & Network Solutions · 201 to 500 Employees · Birmingham, United Kingdom
Hiring: Engineering
What employees say
+ Literally none. Awful leaders. No hold on their finance and takes them months to pay
− I honestly would run away. It’s just filled with people who went to college together and refuse to pay you
More reviews on the company profile.
Interview experience · Data Engineer
First HR will call you after screening Next round with tech lead/architect for 30-45 mins The interviewers was soft spoken, respectful and trying to understand your hands on live experience with tech stack and projects I
India · 6 reports
More interview insightsJob description
Overview
About Kagool We are a fast-growing IT consultancy specializing in the transformation of complex Global enterprises that use SAP.
Responsibilities
- Design, develop, and maintain scalable batch and real-time data pipelines using Apache Spark and PySpark.
- Develop and support Kafka-based real-time streaming pipelines on Microsoft Azure.
- Implement real-time data processing using Spark Structured Streaming.
- Work with Kafka topics, partitions, consumer groups, offsets, retention, and message delivery mechanisms.
- Implement Kafka producers and consumers for high-volume data ingestion and processing.
- Integrate Apache Kafka with Azure data services and downstream data platforms.
- Develop data transformation, cleansing, aggregation, and enrichment processes.
- Implement checkpointing, fault tolerance, error handling, retry mechanisms, and recovery strategies for streaming applications.
- Design and optimize Spark jobs for performance, scalability, and efficient resource utilization.
- Work with Azure data services such as Azure Databricks, ADLS Gen2, Azure Event Hubs, Azure Data Factory, and Azure Synapse Analytics.
- Develop SQL queries and work with relational and analytical databases.
- Implement data quality and validation frameworks.
- Collaborate with Data Architects, Application Teams, DevOps, and Business stakeholders.
- Participate in technical design discussions, code reviews, and CI/CD activities.
Qualifications
- Good understanding of distributed computing and Big Data architecture.
- Hands-on experience with Kafka:
- Topics and partitions
- Producers and consumers
- Consumer groups
- Partitioning
- Error handling and recovery
- Strong understanding of Spark concepts including:
- Partitioning
- Serialization
- Performance tuning
- Hands-on experience with Azure Databricks and/or Azure data engineering services.
- Azure Skills
- Candidates must have practical experience with one or more of the following:
- Azure Databricks
- Azure Event Hubs
- Azure Data Factory
- Azure Data Lake Storage Gen2
- Azure Synapse Analytics
- Azure Functions
- Azure Monitor
- Azure Key Vault
- Microsoft Entra ID / Azure authentication
- Azure Kafka Requirement
- The candidate should have hands-on implementation/support experience with Kafka in an Azure environment, including experience with:
- Kafka deployment/integration on Azure
- Kafka producers and consumers
- Kafka topics and partitions
- Consumer groups and offset management
- Kafka-to-Spark streaming integration
Verify Python for this role
Free PRACTICE for Python adds a verified skill signal that can boost how roles like this rank for you — optional, not required to browse or apply.
Sign up — verify Python free →Want match scores and deeper insight? Create a free candidate profile
Job application FAQ
Last updated 8 October 2026
- How do I apply for Senior Data Engineer Spark Streaming & Kafka at Kagool?
- Open this job on Feedback AI and sign up free (or sign in). You can apply from the job detail page; match scores and skill gaps appear when your profile is complete. Kagool may attach structured assessments to interview rounds after you apply.
- Does Feedback AI show a match score for this Senior Data Engineer Spark Streaming & Kafka role?
- Yes after you add a resume or skills. Match score ranks this role against your profile — skills met, gaps, level, and location — so you prioritise applications where fit is strongest.
- Is Senior Data Engineer Spark Streaming & Kafka remote or on-site in Mumbai, Maharashtra?
- This listing is marked on_site with location Mumbai, Maharashtra. Confirm details on the job description before applying.
- What skills matter for this Senior Data Engineer Spark Streaming & Kafka role at Kagool?
- This is a senior Senior Data Engineer Spark Streaming & Kafka role. Key skills include skills listed in the job description. Feedback AI highlights matched vs missing skills on your profile after signup.