Overview

We are seeking a versatile engineer to build the data foundation of our agentic AI analytics platform, built on LangGraph, Azure OpenAI, AWS Bedrock, Databricks and Postgres and to establish how we measure the quality of what that platform produces.

Responsibilities

• Design, build, and maintain data pipelines that deliver structured, AI-ready data to analytical workflows
• Integrate internal and external data sources through APIs, cloud data platforms, and modern agent-tool
• Design, migrate, and maintain relational database schemas supporting workflow persistence, metadata, and
• Extend and modernize analytical and refining databases as coverage grows, with automated update
• Normalize and structure raw data so that it can be consumed directly by AI agents within automated
• Build monitoring and validation that maintains data quality across sources and refresh cycles, including
• Work with subject matter experts and analysts to understand domain data, its provenance, and its
• Document data flows, schemas, and integration points so that the work is transferable and auditable.
• Support deployment and security readiness of data components, including access control, entitlements, and
• Design and build evaluation harnesses that score AI-generated output systematically for accuracy,
• Establish ground-truth and gold-standard reference sets in partnership with subject matter experts,
• Build regression test suites so that changes to prompts, models, retrieval, or workflow logic are validated
• Run structured comparisons across AI models and configurations, measuring quality, latency, and compute
• Define and detect the characteristic failure modes of generative systems, including unsupported statements,
• Build discrepancy detection that surfaces conflicts between sources as explicit, reviewable flags.
• Work with domain experts to translate expert quality judgments into testable, repeatable criteria.
• Report findings clearly to both engineers and senior leadership, and drive prioritization of the resulting
• Supply evidence of systematic validation to governance, risk, and security review processes.
• Proficiency in Python and SQL, sufficient to build both data pipelines and test harnesses and to analyze the
• Practical experience building data pipelines or automated data processes that other people relied on in
• Working knowledge of relational databases and data modeling, including schema change against systems
• Demonstrated experience evaluating, validating, or testing analytical output, models, or systems against a
• Sound grasp of statistics and experimental design, including sampling, controls, baselines, and sources of
• Practical understanding of generative AI applications and prompt engineering, shown through something
• Intellectual honesty and precision: a willingness to report an unwelcome result and defend the method that
• Clear written communication, including the ability to explain a measurement or a data model to someone
• Data or analytics engineering with production ownership of the pipelines you built.
• AI or machine learning evaluation, large language model evaluation, or applied data science.
• Model validation, model risk management, or quantitative audit in banking, insurance, energy, or a similar
• Scientific or academic research background with strong experimental methodology, in any discipline.

Qualifications

• Understanding of token and compute cost economics in AI systems.
• data such as crude and refined products, trade flows, or price assessment.