Sr. QA Test Engineer

PS: Qualified QA Test Engineer candidates located near Alpharetta GA to be considered due to the position requiring an onsite presence.


Required Skills Experience & Abilities:
- 3 years in QA automation or SDET-type work (adjust by level); 1 year exposure to AI/LLM or ML-driven features is a plus.
- Strong test automation in Python and/or Java/TypeScript.
- We are a platform team testing APIs for high performance automation will be primary focus.
- Strong communication and analytical skills.

Additional skills required:
- Hands-on with frameworks/tools such as: UI: Playwright / Cypress / Selenium and API: pytest requests Postman/Newman REST Assured
- CI/CD integration: Git GitHub Actions/Jenkins/GitLab CI test reporting gating.
- Test design: equivalence partitioning boundary testing risk-based testing defect triage. AI-Specific Testing Competencies (Key)
- LLM/application behavior testing: validating correctness when outputs are probabilistic.
- Evaluation strategies: golden datasets scoring rubrics human-in-the-loop reviews.
- Non-determinism handling: statistical assertions repeated runs variance thresholds.
- Prompt and regression management: versioning prompts detecting prompt drift replay tests.
- RAG testing (if applicable): retrieval quality (recall/precision) grounding checks citation validation doc freshness.
- Safety & quality checks: hallucination detection toxicity/PII leakage checks policy compliance tests.

Data & Observability:
- Ability to create and maintain test datasets (structured unstructured) including edge cases.
- Familiarity with telemetry for AI systems: - logging prompts/outputs safely traceability correlation IDs - tools like OpenTelemetry ELK/Splunk Datadog/Grafana (any equivalent)
- Understanding of data privacy constraints (masking/redaction) and secure test data practices.
- API / Microservices / Cloud
- Comfortable testing distributed systems: microservices async workflows queues/events.
- Basic cloud proficiency (AWS/Azure/GCP) and containerization (Docker optional Kubernetes). Performance & Reliability Testing (AI-Aware)
- Load/performance testing for inference endpoints (latency throughput concurrency).
- Cost-aware testing (token usage rate limits fallbacks).
- Resilience tests: retries circuit breakers model timeouts degraded-mode behavior.

Nice-to-Have:
Domain Knowledge:
- Familiarity with NLP concepts (embeddings context windows temperature/top-p).
- Experience with AI tooling: LangChain/LlamaIndex evaluation tools model gateways.
- Knowledge of regulatory/security needs relevant to the telecom domain.

Soft Skills / Ways of Working:
- Strong communication
-able to explain AI quality issues clearly to product and engineering.
- Comfortable partnering with data science/ML engineers and backend teams.
- Ownership mindset: building reusable test harnesses improving quality metrics preventing regressions.

Education:
- Bachelors in Computer Science Engineering Data/Information Systems or equivalent practical experience.