Overview

We are seeking a Senior AI Engineer with strong expertise in Generative AI (GenAI), Databricks, and end-to-end ML/LLM systems. You will be responsible for designing, building, and deploying intelligent, scalable GenAI solutions integrated into enterprise-grade data and analytics platforms.

Responsibilities

• Design and implement end-to-end Generative AI solutions on Databricks, leveraging Unity Catalog, MLflow, Delta Lake, and Vector Search.
• Architect LLM-based multi-agent frameworks for intelligent automation, chatbot systems, and document reasoning tasks.
• Integrate Cortex AI, OpenAI, or Anthropic APIs for retrieval-augmented generation (RAG), conversational reasoning, and workflow orchestration.
• Fine-tune and evaluate LLMs and domain-specific NLP models (NER, Risk Assessment, Question Answering).
• Develop pipelines for prompt engineering, context management, model evaluation, and hallucination detection.
• Optimize inference performance, latency, and cost across multi-cloud and Databricks environments.
• Collaborate with data engineering teams to ensure clean, well-governed, and vectorized data pipelines.
• Build and maintain feature stores and embeddings stores using Databricks or Snowflake.
• Implement data validation, lineage, and monitoring using Delta Live Tables and Unity Catalog.
• Build reusable ML pipelines using Databricks Repos, MLflow, and Feature Store.
• Automate deployment, monitoring, and retraining workflows for continuous model improvement.
• Partner with product managers, data scientists, and business stakeholders to translate ideas into production-ready AI systems.
• Review code, mentor junior engineers, and enforce best practices in scalable AI/ML development.
• Contribute to internal knowledge bases, documentation, and reusable component libraries.

Qualifications

• Strong background in Machine Learning, NLP, and LLMs (Transformers, RAG, embedding models).
• Proven experience fine-tuning or implementing models using Hugging Face, LangChain, LlamaIndex, or OpenAI API.
• Knowledge of retrieval-augmented generation, multi-agent orchestration, and context management.
• Expertise in Databricks (Delta Lake, MLflow, Unity Catalog, Feature Store, Vector Search).
• Familiarity with Azure Databricks, Azure OpenAI, or Snowflake Cortex AI.
• Experience integrating external APIs and cloud-native microservices (FastAPI, REST, or gRPC).
• Strong proficiency in Python, SQL, PySpark, and Databricks Notebooks.
• Experience building modular codebases, deploying APIs, and working with CI/CD pipelines (GitHub Actions, Azure DevOps).
• Hands-on experience with Streamlit, Gradio, or other UI frameworks for AI app development.
• Hands-on with MLflow tracking, model registry, and experiment management.
• Experience in AI validation, faithfulness scoring, drift detection, and integrity match metrics.
• Working knowledge of Docker, Kubernetes, and inference scaling techniques.

Nice to have

• Strong communication, stakeholder management, and ability to translate business problems into AI solutions.
• Comfort working in agile, multi-disciplinary environments.
• Passion for innovation, experimentation, and applied AI problem-solving.