Senior Machine Learning Engineer

Company Overview

Pantheon Data (a Kenific Holding company) is a private small business based in the Washington DC area. Pantheon Data was founded in 2011 initially providing acquisition and supply chain management services to the US Coast Guard. Our service offerings have grown in the past ten years including infrastructure resiliency contact center operations information technology software engineering program management strategic communications engineering and cybersecurity. We have also grown our customer base to include commercial clients. The company has used this experience to expand our service offerings to other agencies within the Department of Homeland Security (DHS) the Department of Defense (DoD) and other Federal Civilian Agencies.

Position Overview

Pantheon Data is seeking a Senior Machine Learning Engineer to design build and operate production AI systems for federal clients - including hybrid retrieval-augmented generation (RAG) applications Intelligent Document Processing (IDP) pipelines and LLM-backed decision-support tools running in AWS GovCloud.

This is a hands-on production-focused engineering role. You will work with GovCloud-hosted foundation models on Amazon Bedrock build hybrid retrieval over relational and vector data stores design rigorous evaluation harnesses and ship secure traceable evidence-grounded systems in FedRAMP High / DoD IL4IL5 environments. This is not a notebook-only prompt-only or research-only role: successful candidates can walk through real systems they have built - the data flow retrieval and inference architecture deployment approach evaluation strategy failure modes and what they personally implemented.

What This Role Will Work On

• Design and build hybrid RAG applications combining semantic (vector) search lexical/full-text search and structured SQL retrieval with rank fusion evidence grounding and source-cited answer generation.
• Integrate and operate GovCloud-hosted LLMs and embedding models via Amazon Bedrock (e.g. Anthropic Claude Amazon Titan embeddings) including prompt/version management guardrails streaming inference and cost/latency tuning.
• Build Intelligent Document Processing capabilities: OCR post-processing layout-aware parsingtableand form extraction NLP/LLM structured extraction chunking and embedding strategies and human-in-the-loop review workflows.
• Develop serverless event-driven ingestion pipelines (AWS Step Functions Lambda S3 Aurora PostgreSQL withpgvector) that turn messy source documents into reliable traceablequeryableinformation.
• Design and run evaluation programs for retrieval quality extraction accuracy answergroundedness hallucination risk and end-to-end system performance - including golden datasets automated eval harnesses in CI and regression testing across model and prompt versions.
• Engineer for security and compliance from the start: least-privilege IAM private VPC endpoints and restricted-egress network boundaries guardrails and prompt-injection defenses output validation audit logging and CUI-aware data handling in support of ATO requirements.
• Make practical engineering decisions about when to use deterministic logic classical NLP OCR embeddings LLMs fine-tuned or distilled models or humanreview -and defend those tradeoffs to technical and non-technical stakeholders.

Responsibilities

• Design implement andmaintainML/AI software components for RAG IDP and generative AI systems serving federal customers.
• Build andoperatedata pipelines for unstructured and semi-structured data: ingestion extraction cleaning enrichment validation quality scoring quarantine/review and storage.
• Develop and evaluate NLP OCR computer vision embedding retrieval and LLM-based approaches for document understanding and question answering.
• Contribute production-quality Python with clear structure tests logging error handling and documentation;participatein code review and team-based delivery.
• Deploy and support AI services in cloud and containerized environments including Bedrock model integration batch processing workflow orchestration and observability (CloudWatch tracing structured logs).
• Define evaluationmethodologyand metrics; build eval datasets and harnesses; analyze errors; and drive iterative improvement of retrieval and generation quality.
• Implement responsible-AI and security controls: Bedrock Guardrails or equivalent PII/sensitive-data redaction content filtering prompt-injection mitigation and output traceability to source evidence.
• Troubleshoot system behavior across model output data quality retrieval schema design infrastructure latency cost and user workflow.
• Mentor other engineershelpestablishstandards for evaluation reproducibility and responsible AI use and raise the technical quality of the team.
• Communicate clearly with technical and non-technical stakeholders including project managers customers and executive leadership.

Required Skills and Experience

• Bachelors degree in Computer Science Engineering Mathematics Data Science or a related technical field or equivalent professional experience.
• 8 years of professional software engineering experience including 5 years hands-on in machine learning / AI engineering or a closely related role.
• Demonstrated experience shipping AI/ML systems beyond notebooks and demos: production pipelines services APIs inference endpoints evaluation harnesses or customer-facingtools youbuilt and supported.
• Hands-on experience building LLM-backed applications with managed foundation-model services - Amazon Bedrock strongly preferred (Claude Titan or similar models) ideally in AWS GovCloud or another regulated/isolated environment.
• Deep experience with RAG architectures: embeddings vector search (pgvectoror comparable) hybrid retrieval (semantic lexical structured) chunking strategies re-ranking or rank fusion and citation/evidence grounding.
• Experience designing and running LLM/RAG evaluations: retrieval and answer-quality metricsgroundednessand hallucination measurement golden datasets automated regression evals and human-in-the-loop validation.
• Strong Python engineering skills: readable maintainable tested code; debugging; packaging; and integration with other systems.
• Experience with AWS services relevant to this stack: Lambda Step Functions S3 IAM CloudWatch and relational databases (PostgreSQL preferred).
• Experience working with unstructured or semi-structured data: PDFs scanned documents forms tables technical manuals drawings or engineering documentation.
• Working knowledge of AI security practices: guardrails prompt-injection defenses output validation data redaction andleast-privilegeaccess patterns.
• Ability to design and reasonaboutend-to-end data flow: source data preprocessing retrieval model/inference step persistence API/service boundary evaluation and user-facing output.
• Experience with Git-based workflows code review CI/CD and team-based software delivery.
• Clear written and verbal communication including explaining tradeoffs limitations and failure modes.
• Ability to work effectively remotely in cross-functional teams.
• Ability to meet deadlines and produce quality work.
• Proficient in Microsoft Suite software including Outlook Word Excel SharePoint and PowerPoint.

Preferred Skills and Experience

• Direct experience operating LLM workloads in AWS GovCloud under FedRAMP High or DoD IL4/IL5 including private VPC endpoints zero- or restricted-egress boundaries and ATO-supporting documentation.
• Experience with Bedrock features beyond basic inference: Guardrails Knowledge Bases Agents Model Evaluation provisioned throughput and cross-model routing.
• Intelligent Document Processing depth: OCR pipelines (e.g.Textract) layout-aware models table/form extraction and document quality scoring.
• Familiarity with federal compliance frameworks and their engineering implications: NIST 800-53 CMMC CUI handling SBOM/supply-chain controls and STIG-hardened environments.
• Experience with infrastructure as code (Terraform) containerized deployment (Docker/ECS) and CI/CD pipelines (GitLab CI or GitHub Actions) in regulated environments.
• Experience with observability andLLMOpstooling: structured logging tracing model/prompt version management drift detection and cost/latency dashboards.
• Experience building internal validation tools review interfaces dashboards or lightweight full-stack applications (e.g. front ends over Python services).
• Familiarity with common ML frameworks and tooling:PyTorch Hugging Face scikit-learnMLflow or similar.
• Advanced degree ina technicaldiscipline.
• Demonstrated mentorship of junior engineers or contribution to team technical direction.

Clearance Requirements

Applicants selected will be subject to a security investigation and may need to meet eligibility requirements. Secret Clearance is required for continued employment.

Work Location: Reston VA - Remote

• Our company prioritizes the benefits of flexibility and collaboration whether that happens in person or remotely.
• If the position is remote or hybrid you may periodically work from a Pantheon Data office location or client site.
• If this position is assigned to a Pantheon Data office location or client site youll work with colleagues and clients in person as needed for specific client requirements.

Interview Requirement: Candidates who are local to the area should be prepared to participate in an in-person interview as part of the selection process. Candidates outside the local area may be considered for a virtual interview.

Compensation

The salary range for this position is $140000 - $200000. This is not however a guarantee of compensation or salary. Rather salary will be set based on experience geographic location and possibly contractual requirements and could fall outside of this range.

Benefits Overview

We are always looking for good people! Pantheon Data is committed to providing its employees with competitive salaries and benefits in order to increase employee satisfaction and addition to our benefits we also offer SmartBenefits through the Washington Metro Area Transportation Authority where you specify an amount of your pre-tax wages be paid directly to your SmarTrip some cases tuition assistance may be available for continuing education expenses and certifications related to their position. Additional details may be found at Data Important Information

All qualified applicants will be considered for employment without regard to disability status as a protected veteran or any other status protected by applicable federal state local or international law.

As part of the application process you are expected to be on camera during interviews and assessments. We reserve the right to take your picture to verify your identity and prevent fraud.

If you require reasonable accommodation in completing this application interviewing completing any pre-employment testing or otherwise participating in the employee selection process please direct your inquiries to our Talent Team at or by phone .

This company uses E-Verify to confirm each employees work authorization. For more information click here E-Verify Participation Poster

Required Experience:

Senior IC