Senior Data Engineer – Python, PySpark & AWS
Job Location:
Monthly Salary:
Experience Required:
Posted:
Application Deadline:
Vacancies:
Job Summary
Senior Data Engineer Python PySpark & AWS
Location: Plano TX
Relocation : Open
Open for W2 only- No c2c/1099 or third party resumes
Due to the nature of the roles client is only accepting USC/GC /GC EAD only no other visas accepted.
Role Overview
We are seeking a hands-on Senior Data Engineer with strong Python PySpark and AWS experience to develop and maintain scalable data pipelines and distributed data processing solutions. The ideal candidate will have experience supporting production data engineering workloads cloud infrastructure and data orchestration.
Key Responsibilities
- Develop test and maintain ETL/ELT data pipelines using Python and PySpark.
- Build and support big-data processing solutions on AWS using S3 EMR and Lambda.
- Implement infrastructure automation using Terraform.
- Develop data integrations using REST APIs Kafka and MQ messaging.
- Support data storage and retrieval using Cassandra.
- Orchestrate and monitor workflows using Airflow Control-M and native schedulers.
- Troubleshoot pipeline failures and optimize performance reliability and data quality.
Required Skills
- Strong hands-on Python and PySpark experience.
- Experience with big-data processing and production data pipelines.
- AWS experience with S3 EMR and Lambda.
- Working knowledge of Terraform.
- Familiarity with Kafka MQ messaging REST APIs Java and Cassandra.
- Experience with Airflow Control-M or other workflow scheduling tools.
Preferred: Databricks experience and exposure to machine-learning workflows.
Required Skills:
Senior Data Engineer Python PySpark & AWS Location: Plano TX Relocation : Open Open for W2 only- No c2c/1099 or third party resumes Due to the nature of the roles client is only accepting USC/GC /GC EAD only no other visas accepted. Role Overview We are seeking a hands-on Senior Data Engineer with strong Python PySpark and AWS experience to develop and maintain scalable data pipelines and distributed data processing solutions. The ideal candidate will have experience supporting production data engineering workloads cloud infrastructure and data orchestration. Key Responsibilities Develop test and maintain ETL/ELT data pipelines using Python and PySpark. Build and support big-data processing solutions on AWS using S3 EMR and Lambda. Implement infrastructure automation using Terraform. Develop data integrations using REST APIs Kafka and MQ messaging. Support data storage and retrieval using Cassandra. Orchestrate and monitor workflows using Airflow Control-M and native schedulers. Troubleshoot pipeline failures and optimize performance reliability and data quality. Required Skills Strong hands-on Python and PySpark experience. Experience with big-data processing and production data pipelines. AWS experience with S3 EMR and Lambda. Working knowledge of Terraform. Familiarity with Kafka MQ messaging REST APIs Java and Cassandra. Experience with Airflow Control-M or other workflow scheduling tools. Preferred: Databricks experience and exposure to machine-learning workflows.
