Datahub- Backend Developer Data Engineer

Summary

Designs and implements REST APIs backend services and data access layers. Develops and maintains data pipelines (Excel/CSV import/export DB migrations). Integrates AWS services (Lambda/S3/ECS). Handles debugging incident analysis and performance tuning.

Mandatory Skills

1. TypeScript / JavaScript
2. ()
3. PostgreSQL (SQL query optimization) & Drizzle ORM
4. ETL / data pipeline design ( Streams exceljs csv-parse)
5. Database schema migrations (Drizzle Kit)
6. AWS: Lambda S3 ECS ECR operating & deploying data-processing services
7. IAM Secrets Manager access rights & credential handling
8. Integration tests (Jest DB test setups) for data pipelines
9. Docker containerizing batch/ETL jobs for ECS

Advantageous Skills

• Python (for AWS Lambda functions)
• CloudWatch (log analysis for pipeline debugging)
• VPC basics (network access DB <-> Lambda)
• GitHub Actions (CI/CD for migration/deployment pipelines)
• Node-Cron/scheduling Redis caching
• Unit tests (Jest) for transformation functions
• Report generation (PPTX/Excel export)

Soft Skills

• Analytical thinking for complex data structures
• Diligence around data quality/consistency
• Clear documentation of interfaces

Required Experience:

Senior IC