Browse Jobs
Explore open roles across India.Explore active software roles. Sign up to apply and get AI match scores.
Top MatchesTop Job Matches
Loading jobs…
Back to Browse Jobs
YMinds.AI
Data Engineer Level 2 | 3-5 YOE | Snowflake,Azure Data Factory , Python, SQL , Azure Functions|Immediate Joiners | Bangalore
YMinds.AI
Bengaluru•On-site•1-3 Yrs•13 hours ago
Job description
Overview
Our client is seeking a Data Engineer to build high-performance pipelines that power their Explainable AI engine.
Responsibilities
- Ingest raw structured and semi-structured data from customer cloud environments (AWS, Azure, GCP) into Snowflake staging layers.
- Configure ADF Linked Services, Integration Runtimes, and pipeline triggers for reliable data movement across multi-cloud boundaries.
- Optimize pipeline performance for high-volume datasets, ensuring low-latency delivery to downstream transformation layers.
- Pull data from REST APIs and third-party data sources as required by product needs.
- Transform and normalize API responses using Python and SQL to produce clean, structured tables in Snowflake.
- Ensure transformed data is structured and optimized for direct consumption by front-end applications and BI tools.
- Build reusable Python utilities for API pagination, error handling, and incremental data extraction.
- Collaborate with product and domain experts to gather and analyze business requirements for new product modules.
- Translate business requirements into clean, scalable data models.
- Design analytics-ready schemas (Star Schema, dimensional models) for ML and front-end consumption.
- Implement data quality frameworks to ensure accuracy, completeness, and consistency across all pipeline layers.
- Propose refresh strategies for each table, identifying appropriate SCD types.
- Implement automated testing and watermarking logic to prevent data gaps in Financial Reporting.
- Instrument observability metadata for monitoring data quality and performance.
Qualifications
- 3–5 years in a Data Engineering role; SaaS or analytics product experience is a plus.
- Hands-on experience with Azure Data Factory (Linked Services, IR, triggers, parameterization).
- Proficiency in Snowflake, including virtual warehouses and role-based access.
- Expert-level SQL for complex transformations; strong Python for API integrations and data validation.
- Ability to design Star Schemas and handle SCDs (Type 1 & 2).
- Knowledge of data quality checks and freshness monitoring.
- Familiarity with AWS, Azure, or GCP environments.
- Knowledge of query optimization and indexing in distributed environments.
- Exposure to real-time data tools like Kafka, Kinesis, or Spark Streaming.
- Familiarity with Docker and Kubernetes from a data workload perspective.
- Strong analytical thinking and problem-solving skills.
- Detail-oriented with a focus on data integrity.
- Excellent communication skills and ability to empower team collaboration.
- Intellectual curiosity and a fast learning ability.
- Gain exposure to enterprise-scale cloud billing datasets.
- Become a subject matter expert in Multi-Cloud FinOps.
- Acquire hands-on experience in AI-native product building.
- Enjoy architectural ownership and influence the long-term roadmap in a high-growth startup.
- Develop business and financial acumen, bridging technical infrastructure with business ROI.