Browse Jobs
Explore open roles across India.Explore active software roles. Sign up to apply and get AI match scores.
Top MatchesTop Job Matches
Loading jobs…
Back to Browse Jobs
Volga Infotech
Sr Data Engineer ( AWS) with Pharma Experience
Volga Infotech
Anywhere•Remote•Salary•3-5 Yrs•12 hours ago
Job description
Remote
Overview
Senior Data Engineer –Data Engineering
Responsibilities
- As the Technical Lead Cloud & Data Engineering, you will be responsible for handling large, complex, multi-dimensional datasets, including structured, unstructured, and real-time data. You will develop complex data transformation ETLs and work extensively with Python, Py Spark, Apache Spark, as well as cloud services from both AWS and Azure. Additionally, you will demonstrate strong expertise in cloud services related to the data domain and have a solid
- understanding of On-Prem/Cloud Data warehouse databases. You will also exhibit technical leadership capabilities, leading and delivering projects independently.
- Handle large, complex, multi-dimensional datasets, including structured, unstructured, and real-time data.
- Develop complex data transformation ETLs using Python, Py Spark, and Apache Spark.
- Utilize cloud services from both AWS and Azure related to the data domain, demonstrating a strong understanding of their respective ecosystems.
- Have a solid understanding of On-Prem/Cloud Data warehouse databases and their architecture.
- Provide technical leadership, leading and delivering projects independently while ensuring high-quality results.
- Stay updated with emerging trends in data tools, analysis techniques, and data usage, understanding their potential impact.
- Understand the concepts and principles of data modelling and create, maintain, and update relevant data models for specific business needs.
- Possess knowledge of cloud data engineering tools/components/technologies from both AWS and Azure, such as AWS Glue, EMR, Azure Data Factory, etc.
- Have an additional advantage with knowledge of Snowflake, Dataiku, Alteryx, or similar technologies.
- Collaborate effectively with cross-functional teams, including data scientists, analysts, and business stakeholders, to understand requirements and deliver optimal data engineering solutions.
- Ensure the scalability, reliability, and efficiency of data pipelines and processes.
- Troubleshoot and resolve issues related to data engineering, performance, and scalability.
- Document and maintain clear and comprehensive technical specifications, design documents, and best practices.
- Collaborate with team members to complete tasks and achieve project objectives.
- Ideal Candidate
Qualifications
- Working hours
- 1:00 pm to 10:00 pm IST ( Mon - Fri )
- Qualification
- A Bachelor's Degree in Computer Science, or a similar discipline (or comparable practical experience) is required.
- A Bachelor's Degree in Computer Science, or a similar discipline (or comparable practical experience) is required.
- Extensive experience in handling large, complex, multi-dimensional datasets, including structured, unstructured, and real-time data.
- Strong experience in developing complex data transformation ETLs using Python, PySpark, and Apache Spark.
- In-depth understanding of cloud services from both AWS and Azure related to the data domain, such as AWS Glue, EMR, S3, Athena, Azure Data Factory, etc.
- Solid understanding of On-Prem/Cloud Data warehouse databases and their architecture.
- Demonstrated technical leadership capabilities, with the ability to lead and deliver projects independently.
- Up-to-date knowledge of emerging trends in data tools, analysis techniques, and data usage.
- Communication skills that are effective in conveying complicated design concepts.