Sr QA Automation Engineer

Smart Ship Hub


Job Location:

Pune - India

Monthly Salary: Not Disclosed
Posted on: 9 hours ago
Vacancies: 1 Vacancy

Job Summary

Role Overview

The Senior QA Automation Engineer (Performance Database & Scale Testing) is responsible for designing building and operating the performance and scale testing capability across SmartShipHub's cloud-native platform. The role owns the full performance engineering lifecycle — from requirements and test design through execution analysis and actionable recommendations to engineering teams.

The successful candidate will architect reusable maintainable performance test frameworks; build AI-augmented test generation pipelines leveraging Gen AI tooling; validate database query performance under realistic fleet-scale loads; and champion performance as a first-class quality gate in CI/CD pipelines.

JMeter / Gatling / k6

Locust / Artillery

Prometheus / Grafana

PostgreSQL / MongoDB

Cloud: GCP / AWS / Azure

Gen AI: Claude / ChatGPT

K8s Scale Testing

Database Benchmarking

CI/CD Integration

IoT / Maritime SaaS

Experience Requirements

Core Experience

  • 7–12 years of total QA / quality engineering experience with meaningful progression
  • 5 years of hands-on performance load and scale testing experience in cloud-native SaaS or product-based environments
  • Proven experience building performance test frameworks from scratch — not just running existing scripts
  • Experience at a product-based company or MNC with global SLAs and high-availability commitments is strongly preferred
  • Experience contributing to or owning performance engineering as a dedicated function (not combined QA generalist)

Domain & Environment

  • Experience testing cloud-based products deployed on GCP AWS or Azure — Kubernetes-hosted microservices strongly preferred
  • Experience with IoT maritime industrial or high-volume telemetry platforms is a significant plus
  • Comfort working in polyglot engineering environments (Java Python Go) across test tooling
  • Experience collaborating with SRE platform and database engineering teams on performance RCA
  • Exposure to maritime vessel data AIS feeds or time-series telemetry data pipelines is a bonus.

Technical Skills & Competencies

1. Performance & Load Testing Frameworks

  • Expert-level proficiency with two or more: Apache JMeter Gatling k6 Locust Artillery Tsung — test plan design parameterisation distributed execution
  • Test scenario design: steady-state load ramp-up/ramp-down spike tests soak tests breakpoint / stress tests — mapped to real user journey profiles
  • Distributed load generation: JMeter master-slave clusters on Kubernetes; k6 operator on GKE/EKS; Artillery cloud generating realistic global load from multiple regions
  • Protocol coverage: HTTP/HTTPS REST gRPC WebSocket MQTT (IoT telemetry) GraphQL Server-Sent Events (SSE) — performance testing beyond simple HTTP
  • Realistic data generation: parameterised virtual user datasets correlated parameters (login → session token → API call chains) data pool management
  • Correlation and dynamic extraction: response token extraction session correlation JSON/XML/regex extractors — no hard-coded test values
  • Think time and pacing: realistic inter-request delays user pacing constant throughput timers — preventing artificial test results
  • Performance test environment management: traffic shaping network throttling simulation database seeding with production-scale data volumes

2. Database Performance & Query Optimisation Testing

  • PostgreSQL performance testing: EXPLAIN / EXPLAIN ANALYSE query plan inspection index usage validation connection pool benchmarking (PgBouncer)
  • MongoDB performance: aggregation pipeline performance analysis index strategy validation read/write throughput benchmarking under shard-scale data volumes
  • Time-series databases: TimescaleDB / InfluxDB / BigQuery — benchmarking ingestion throughput for IoT telemetry (vessel sensor data at fleet scale)
  • Database load simulation: realistic concurrent query load generation (pgbench mongo-perf custom Python/Java load drivers) replicating production query mix
  • Slow query identification: automated slow query log analysis long-running transaction detection lock contention identification
  • N1 query detection: integration of database call tracing into API performance tests — identify ORM-generated N1 patterns under load
  • Connection pool exhaustion testing: simulate pool saturation queue depth connection timeout behaviour under peak concurrency
  • Database backup performance: validate backup and restore duration under production data volumes against RTO/RPO targets
  • Data volume scaling: populate test environments with statistically representative data volumes (millions of vessel events port calls voyage records)

3. Cloud Scale Testing & Infrastructure Validation

  • Kubernetes scale testing: HPA (Horizontal Pod Autoscaler) validation — trigger scale-up latency scale-down behaviour; PodDisruptionBudget compliance under load
  • Cloud service limits: API rate limit validation cloud quota headroom testing throttling behaviour under burst traffic
  • Multi-region load testing: latency profiling across GCP / AWS / Azure regions; CDN / global load balancer behaviour validation
  • Chaos engineering for performance: Chaos Mesh / LitmusChaos — pod kill network latency injection disk I/O stress during load tests to validate graceful degradation
  • Storage performance: GCS / S3 / Azure Blob throughput testing; PVC / persistent volume I/O benchmarking in Kubernetes
  • Message queue throughput: Kafka / Pub/Sub / RabbitMQ consumer lag benchmarking; producer throughput at fleet-scale IoT message rates
  • Cache performance validation: Redis / Memcached hit ratio eviction rate latency percentiles under production-representative cache workloads
  • CDN and edge performance: cache-hit rate validation origin offload percentage global TTFB (Time to First Byte) benchmarking
  • Autoscaling cost analysis: correlate scale events with cloud billing impact — performance vs cost trade-off reporting

4. Gen AI — Augmented Test Design & Automation (Mandatory)

  • Claude (Anthropic): using Claude API / to generate realistic load test scripts from API specifications (OpenAPI / Swagger) user story descriptions or production traffic samples
  • ChatGPT / GPT-4o: prompt engineering for test scenario generation edge case identification performance test data synthesis test result narrative generation
  • AI-assisted test code generation: using LLM prompts to generate k6 / JMeter / Gatling scripts database test queries and data seed scripts — with human review and validation
  • AI for anomaly analysis: using LLMs to analyse Grafana/Prometheus alert output slow query logs and JMeter result summaries — generate natural-language RCA narratives
  • Prompt engineering discipline: structured prompts for consistent reproducible test artefact generation; version-controlled prompt libraries; output validation pipelines
  • AI-powered test maintenance: LLM-based script update generation when APIs change — diff-aware prompt templates that preserve existing scenario logic
  • Gen AI evaluation: critical evaluation of LLM-generated test output for correctness security (no credential leakage) and representativeness — AI assists engineer decides
  • Emerging tooling: staying current with AI-native testing tools (Testim AI Mabl Functionize Applitools) and evaluating their applicability to performance use cases

5. Performance Observability Metrics & Analysis

  • Prometheus: custom metrics instrumentation (client libraries for Java / Python / ) performance-relevant alerting rules histogram / percentile analysis
  • Grafana: performance test result dashboards (live during test execution) SLO compliance dashboards regression comparison across test runs
  • Distributed tracing: OpenTelemetry / Jaeger / Zipkin — trace slow paths under load identify latency contributions per microservice during test execution
  • APM integration: Datadog / New Relic / Dynatrace — correlate load test activity with APM traces for root cause identification
  • JMeter / k6 result analysis: response time percentile analysis (p50 p90 p95 p99) error rate analysis throughput vs latency correlation
  • Flame graph analysis: async-profiler / perf / py-spy — CPU and memory hotspot identification under load in JVM and Python services
  • Statistical analysis: performance regression detection using Mann-Whitney U test or t-test — automated baseline comparison in CI pipeline
  • Performance test reporting: executive summary generation (automated via Gen AI) trend reports capacity planning recommendations

6. CI/CD Pipeline Integration & Shift-Left Performance

  • GitHub Actions / GitLab CI: integrate k6 / Gatling performance tests as CI pipeline stages — block merges on performance regression (p99 threshold breach)
  • Performance baselines: automated baseline capture on main branch; PR-triggered comparative tests with pass/fail decision against baseline
  • Performance budget enforcement: define and enforce response time throughput and error rate budgets per API endpoint — surfaced as PR status checks
  • Staging environment performance gates: automated nightly performance regression suite in staging — failures block production deployments
  • Artefact management: performance test scripts versioned in git; test result artefacts stored in GCS / S3 with retention policies; baseline database in Postgres
  • Container-native test execution: k6 operator / JMeter Docker containers orchestrated in Kubernetes — ephemeral reproducible test infrastructure

7. Cloud-Native Product Testing Expertise

  • REST API performance testing: complete CRUD lifecycle load testing pagination performance bulk operation throughput concurrent user simulation
  • WebSocket / real-time performance: sustained connection load testing (vessel tracking dashboards receiving live AIS updates) message delivery latency validation
  • Multi-tenancy performance isolation: validate that one tenant's load does not degrade other tenants' performance — noisy-neighbour detection
  • SLA / SLO validation: automated validation that p99 API response times meet contractual SLAs under defined load profiles
  • Geographical latency testing: Cloud-native load generation from GCP / AWS regions closest to vessel fleet locations; latency profiling per maritime route
  • Serverless cold-start benchmarking: Cloud Functions / Lambda cold-start latency measurement and optimisation impact validation

Key Responsibilities

Performance Engineering Ownership

  • Own the end-to-end performance test strategy for SmartShipHub's cloud platform — from SLA definition through test design execution analysis and remediation tracking
  • Build and maintain reusable modular performance test frameworks covering API database messaging and WebSocket layers
  • Establish and maintain performance baselines per sprint; detect and report regressions before they reach production
  • Define performance NFRs (Non-Functional Requirements) in collaboration with product and engineering — translate into testable acceptance criteria
  • Lead performance RCA investigations — coordinate with platform database and backend engineering to resolve identified bottlenecks

Database & Scale Testing

  • Design and execute database performance test suites for PostgreSQL MongoDB and time-series stores — query benchmarking index validation connection pool testing
  • Populate test environments with statistically representative production-scale data volumes — automated data seeding pipelines
  • Validate scale-out behaviour of Kubernetes-hosted services under fleet-scale IoT telemetry loads (millions of vessel events per hour)
  • Validate BCP/DR database failover performance — measure RTO/RPO achievement under controlled failure scenarios

Gen AI Integration

  • Build and maintain Gen AI–augmented test generation pipelines (Claude ChatGPT) — accelerating test script creation data synthesis and RCA reporting
  • Develop prompt engineering standards and libraries for performance test artefact generation — version-controlled reviewed and auditable
  • Evaluate and adopt emerging AI-native testing tools appropriate for SmartShipHub's platform context
  • Advocate for responsible AI use in testing — ensuring LLM-generated output is validated not blindly trusted

Tools & Technology Stac

Performance & Testing Tools

  • Apache JMeter Gatling k6 Locust Artillery
  • pgbench mongo-perf TimescaleDB bench
  • Chaos Mesh LitmusChaos
  • OpenTelemetry Jaeger Zipkin
  • Datadog / New Relic / Dynatrace APM
  • async-profiler py-spy (flame graphs)
  • Prometheus Grafana (dashboards)
  • BlazeMeter / (cloud load generation)

Gen AI & Automation

  • Claude (Anthropic API / )
  • ChatGPT / GPT-4o (OpenAI API)
  • GitHub Copilot (AI code completion)
  • Testim AI Mabl Functionize (evaluation)
  • Python / JavaScript / Java (test frameworks)
  • GitHub Actions GitLab CI (pipeline integration)
  • Docker / Kubernetes (test infrastructure)
  • GCP / AWS / Azure (cloud environments)

SmartShipHub is an equal-opportunity employer. All applicants are considered regardless of race gender religion nationality disability or age.


Required Skills:

Apache JMeter Gatling k6 Locust Artillery Tsung Distributed Load Generation HTTP/HTTPS REST gRPC WebSocket MQTT GraphQL Server-Sent Events Realistic Data Generation Parameterisation Correlation Dynamic Extraction Think Time Pacing Traffic Shaping Network Throttling Database Seeding PostgreSQL EXPLAIN EXPLAIN ANALYZE Query Plan Inspection Index Usage Validation Connection Pool Benchmarking PgBouncer MongoDB Aggregation Pipeline Performance Analysis Index Strategy Validation Read/Write Throughput Benchmarking TimescaleDB InfluxDB BigQuery Database Load Simulation pgbench mongo-perf Slow Query Identification Slow Query Log Analysis Long-Running Transaction Detection Lock Contention Identification N1 Query Detection Connection Pool Exhaustion Testing Database Backup Performance Data Volume Scaling Kubernetes Scale Testing HPA Validation PodDisruptionBudget Cloud Service Limits Validation Multi-Region Load Testing Latency Profiling Chaos Engineering Chaos Mesh LitmusChaos Storage Performance Testing GCS S3 Azure Blob PVC I/O Benchmarking Message Queue Throughput Kafka Pub/Sub RabbitMQ Cache Performance Validation Redis Memcached CDN Performance Cache-Hit Rate Origin Offload TTFB Autoscaling Cost Analysis Gen AI Claude ChatGPT AI-Assisted Test Code Generation Prompt Engineering AI for Anomaly Analysis Gen AI Evaluation Performance Observability Prometheus Custom Metrics Instrumentation Grafana Dashboards Distributed Tracing OpenTelemetry Jaeger Zipkin APM Integration Datadog New Relic Dynatrace JMeter Result Analysis k6 Result Analysis Response Time Percentile Analysis Error Rate Analysis Throughput vs Latency Correlation Flame Graph Analysis async-profiler py-spy Statistical Analysis Mann-Whitney U Test t-test Performance Regression Detection CI/CD Integration GitHub Actions GitLab CI Performance Baselines Performance Budget Enforcement Staging Environment Performance Gates Artefact Management Container-Native Test Execution Docker Kubernetes Cloud-Native Product Testing REST API Performance Testing WebSocket Performance Testing Multi-Tenancy Performance Isolation SLA/SLO Validation Geographical Latency Testing Serverless Cold-Start Benchmarking BlazeMeter GitHub Copilot Testim AI Mabl Functionize Python Java Go Test Framework Design Performance RCA Cross-Functional Collaboration Communication Capacity Planning IoT Maritime Industrial Telemetry AIS Feeds Time-Series Telemetry Performance Engineering Load Testing Scale Testing Database Testing Cloud Testing API Testing Microservices Testing Performance Budgeting Shift-Left Performance Performance NFRs Non-Functional Requirements

Role OverviewThe Senior QA Automation Engineer (Performance Database & Scale Testing) is responsible for designing building and operating the performance and scale testing capability across SmartShipHub's cloud-native platform. The role owns the full performance engineering lifecycle — from requirem...