AIML Data & ETL Data Architect
Charlotte, VT - USA
Job Summary
We offer products and solutions in Cloud Data Engineering Data Governance AI/ML DevOps and Blockchain to large corporates across the globe. Strategic Partners with AWS Collibra cloudera neo4j DataRobot Global IDs tableau MuleSoft and Talend.
Key Responsibilities
- Architect feature stores training/inference pipelines and MLOps workflows for insurance use cases fraud detection claims triage underwriting risk scoring loss reserving and customer churn/retention.
- Design RAG and GenAI solution patterns for claims summarization policy/document intelligence and underwriter/agent copilots.
- Establish model lifecycle controls: versioning lineage drift monitoring evaluation and human-in-the-loop review.
- Define responsible-AI and governance guardrails appropriate to a regulated insurance environment (auditability explainability bias monitoring).
- Own the end-to-end target-state architecture for the insurance data platform policy administration claims billing underwriting actuarial and reinsurance domains across raw curated and analytics-ready layers.
- Design lakehouse and AI/ML reference architectures (Bronze/Silver/Gold Medallion) that unify structured semi-structured and streaming insurance data.
- Define data domain boundaries source-to-target mappings and canonical insurance data models for shared enterprise consumption.
- Produce architecture diagrams design decision records and patterns that engineering teams can implement consistently.
- Make build-vs-buy cloud service selection and cost/performance trade-off decisions and defend them to client architecture review boards.
- Design scalable production-grade ETL/ELT frameworks (PySpark Spark SQL Delta Live Tables / equivalent orchestrated Workflows).
- Define ingestion patterns for batch micro-batch and streaming insurance feeds (policy claims payments third-party/bureau data).
- Establish orchestration monitoring alerting and automation standards for the engineering team.
- Design dimensional models (star/snowflake) and canonical/conformed models for analytical and actuarial workloads.
- Apply normalization/denormalization strategies balancing performance usability and regulatory traceability.
- Ensure data quality integrity and alignment with enterprise and insurance regulatory governance policies.
- Embed PII/PHI handling masking tokenization and least-privilege access models into platform design.
- Align architecture with insurance regulatory and audit requirements (e.g. NAIC model standards state DOI HIPAA where health lines apply SOC 2 GDPR/CCPA).
- Define metadata management data lineage and cataloging strategy (Unity Catalog or equivalent).
- Advanced hands-on data engineering: Spark Delta Lake / lakehouse Workflows Unity Catalog (or cloud-native equivalents).
- AI/ML tooling: MLflow or equivalent feature stores model serving and GenAI/RAG frameworks (LangChain/LangGraph or similar).
- Strong SQL and Python programming with performance tuning skills.
- Cloud platform depth (AWS / Azure / GCP) including managed data and ML services.
- Hands-on AI/ML pipeline and MLOps experience including at least one production GenAI/RAG deployment.
- Strong command of Medallion architecture (Bronze/Silver/Gold) and modern data modeling for warehousing and analytics.
- Proficiency with PySpark SQL ETL/ELT frameworks and Delta Lake (or equivalent) optimization.
- Experience with CI/CD Git and job orchestration tooling.
- Insurance financial services or other regulated-industry delivery experience.
- Demonstrated ability to present and defend architecture to senior client and review-board stakeholders.
- Data governance metadata management and Unity Catalog (or equivalent) advanced features.
- Streaming technologies (Auto-Loader / Structured Streaming / Kafka / Event Hubs / Kinesis).
- Data security regulatory compliance and fine-grained access models.
- Cost optimization and performance tuning in cloud environments.
- Responsible-AI / model governance frameworks (e.g. NIST AI RMF).
- Tools such as Airflow Databricks Workflows dbt or similar.
- Data architecture leadership lakehouse / Medallion (Bronze/Silver/Gold) target-state design
- Strong Python (PySpark) and SQL programming with performance tuning
- Databricks (or equivalent) Spark Delta Lake Workflows Unity Catalog
- ETL/ELT framework design and data modeling (dimensional star/snowflake canonical)
- AI/ML pipelines MLOps plus at least one production GenAI/RAG deployment
- Cloud experience AWS Azure or GCP (managed data ML services)
- CI/CD Git job orchestration
- 12 years total; 3 years as architect/lead; regulated-industry delivery
Required Skills:
Data architecture leadership lakehouse / Medallion (Bronze/Silver/Gold) target-state design Strong Python (PySpark) and SQL programming with performance tuning Databricks (or equivalent) Spark Delta Lake Workflows Unity Catalog ETL/ELT framework design and data modeling (dimensional star/snowflake canonical) AI/ML pipelines MLOps plus at least one production GenAI/RAG deployment Cloud experience AWS Azure or GCP (managed data ML services) CI/CD Git job orchestration 12 years total; 3 years as architect/lead; regulated-industry delivery