Data Engineer – Python, PySpark, SQL & ETL

Job Summary

Synechron is seeking a Data Engineer with 5 years of experience in building data marts ETL pipelines and large-scale data-processing solutions.

The role will focus on designing developing optimizing and supporting scalable data pipelines using Python PySpark Spark SQL and data-warehousing technologies. The position will cover the end-to-end software development lifecycle including development testing UAT support production deployment issue resolution and post-production support.

The role will contribute to business objectives by delivering reliable and efficient data solutions improving data quality and accessibility supporting informed decision-making and enabling consistent data operations across enterprise environments.

Software Requirements

Required Software Skills

• Python: Strong hands-on experience developing data-processing applications ETL workflows automation scripts and production services.
• PySpark and Spark: Strong experience building distributed data-processing and ETL solutions.
• Pandas: Experience with data preparation analysis transformation validation and profiling.
• SQL: Strong experience with data extraction transformation validation analysis and query optimization.
• Oracle SQL: Strong knowledge of Oracle SQL complex queries joins aggregations subqueries stored procedures and performance optimization.
• ETL development: Experience designing developing testing deploying and supporting ETL and data-pipeline solutions.
• Data warehousing: Strong understanding of data-warehouse concepts dimensional modelling data marts transformations fact tables dimensions and data quality.
• Hadoop Hive and MapReduce: Experience working with distributed data-processing and big-data technologies.
• SQL and NoSQL databases: Experience working with structured semi-structured and unstructured data.
• Jupyter Notebook: Experience using notebooks for data analysis development validation troubleshooting and technical exploration.
• CI/CD pipelines: Familiarity with automated build testing deployment and release processes.
• Testing and data validation: Experience validating data transformations pipeline results data quality and application behavior.
• Software engineering practices: Understanding of coding standards version control documentation code reviews testing and maintainable development practices.

Preferred Software Skills

• Experience with banking or financial-services data and reporting requirements.
• Exposure to cloud-based data platforms and data-processing services.
• Familiarity with workflow scheduling orchestration monitoring and alerting tools.
• Experience with automated data-quality checks reconciliation lineage and metadata management.
• Exposure to infrastructure automation containerization and modern DevOps practices.
• Familiarity with tools supporting data pipeline observability and performance analysis.

Version and Experience Guidance

• Specific software versions are not mandated in the requirements.
• Candidates should have experience with current production-supported versions of Python PySpark Spark Pandas Oracle SQL Hadoop Hive MapReduce and related tools.
• Candidates should be able to assess compatibility manage dependencies support upgrades and follow version-control practices.
• Experience should include production data engineering pipeline support and end-to-end SDLC participation rather than only academic or prototype development.

Overall Responsibilities

• Design develop maintain and optimize scalable ETL and data-pipeline solutions using Python and PySpark.
• Build and enhance data marts and data-transformation processes.
• Develop distributed data-processing solutions using Spark Hadoop Hive and MapReduce.
• Work with structured semi-structured and unstructured data from multiple sources.
• Develop and optimize Oracle SQL queries for data extraction transformation analysis and validation.
• Analyze and debug data issues using Oracle SQL PySpark Pandas and structured data-analysis techniques.
• Perform end-to-end SDLC activities including development testing UAT support bug fixing production deployment and post-production support.
• Collaborate with cross-functional teams to define data requirements validate solutions resolve defects and improve data quality.
• Implement data validation reconciliation completeness accuracy and consistency checks.
• Support CI/CD implementation for data pipelines applications and deployment workflows.
• Monitor production environments and troubleshoot pipeline failures data-quality issues performance problems and processing delays.
• Create and maintain technical documentation data mappings transformation logic support procedures and operational runbooks.
• Apply software engineering best practices coding standards testing practices and maintainable design principles.
• Contribute to performance improvements through query optimization efficient transformations appropriate partitioning resource management and pipeline tuning.
• Identify opportunities to improve automation reliability data quality maintainability and sustainable use of processing resources.
• Contribute to measurable outcomes including reliable data delivery reduced processing time improved data accuracy fewer production incidents and successful release execution.

Technical Skills (By Category)

Programming Languages

Essential

• Strong Python development experience.
• Strong PySpark and Spark programming experience.
• Experience using Pandas for data analysis transformation profiling and validation.
• Ability to write modular maintainable testable and production-ready code.
• Ability to develop reusable data-processing components ETL logic validation utilities and automation scripts.

Preferred

• Experience with additional programming or scripting languages used in data engineering and distributed processing.
• Familiarity with asynchronous or event-driven data-processing patterns.
• Experience developing reusable libraries and shared data-engineering components.

Databases/Data Management

Essential

• Strong Oracle SQL experience.
• Experience with SQL and NoSQL databases.
• Understanding of data-warehouse concepts and data-mart design.
• Experience working with structured semi-structured and unstructured data.
• Understanding of data modelling dimensional modelling schemas fact tables dimensions keys joins aggregations and transformations.
• Experience with data quality validation reconciliation completeness accuracy and consistency checks.
• Ability to optimize queries and investigate data-related application or pipeline issues.
• Understanding of data lineage source-to-target mapping metadata data retention and controlled data access.

Preferred

• Experience with banking or financial-services data structures and reporting requirements.
• Familiarity with data lakes analytical data stores and distributed data architectures.
• Experience with partitioning indexing compression schema evolution and high-volume data processing.
• Exposure to automated data-quality frameworks data reconciliation and metadata management.

Cloud Technologies

Essential

• Understanding of scalable data-processing and data-pipeline deployment environments.
• Ability to support development testing UAT staging and production data environments.
• Understanding of resource management performance availability monitoring and operational support for data platforms.

Preferred

• Experience with cloud-based data platforms and distributed processing services.
• Familiarity with cloud storage managed data-processing services data warehouses and orchestration tools.
• Exposure to cloud-native CI/CD infrastructure automation and containerized data workloads.
• Understanding of processing optimization resource utilization and sustainable data-engineering practices.

Frameworks and Libraries

Essential

• Python.
• PySpark and Spark.
• Pandas.
• Oracle SQL.
• Hadoop.
• Hive.
• MapReduce.
• SQL and NoSQL database technologies.
• Jupyter Notebook.
• ETL and data-pipeline development frameworks.
• CI/CD tools and automated deployment workflows.
• Data validation testing monitoring and troubleshooting utilities.

Preferred

• Workflow orchestration and scheduling frameworks.
• Data-quality reconciliation lineage and metadata-management tools.
• Pipeline observability logging metrics and alerting frameworks.
• Libraries supporting performance profiling distributed processing and automated testing.
• Tools supporting cloud-based data integration and analytics.

Development Tools and Methodologies

Essential

• Experience across the data-engineering lifecycle including requirements analysis design development testing UAT support deployment monitoring troubleshooting and production support.
• Experience with ETL development and data-mart implementation.
• Familiarity with version control code reviews coding standards technical documentation and collaborative development practices.
• Experience implementing data validation unit testing integration testing regression testing and pipeline testing.
• Experience supporting CI/CD implementation for data applications and deployment workflows.
• Ability to create data mappings transformation specifications test plans support procedures and operational runbooks.
• Familiarity with Agile or iterative delivery practices.
• Experience performing root-cause analysis and resolving production data-pipeline issues.

Preferred

• Experience with automated data-pipeline deployment and release management.
• Familiarity with infrastructure as code containers and DevOps practices.
• Experience with pipeline monitoring operational dashboards alerting and service-level reporting.
• Exposure to automated rollback recovery retry and failure-handling patterns.
• Experience supporting continuous improvement and process standardization.

Security Protocols

Essential

• Understanding of secure data access authentication authorization and least-privilege principles.
• Ability to protect sensitive business customer financial and operational data.
• Understanding of secure database connections credential management data masking encryption and controlled data handling.
• Ability to follow security privacy compliance change-management and access-control requirements.
• Awareness of risks related to unauthorized access data leakage insecure scripts exposed credentials and uncontrolled data movement.
• Ability to maintain auditability and traceability across data transformations deployments and production changes.

Preferred

• Experience supporting data-security reviews audit activities access reviews and compliance controls.
• Familiarity with secrets management encryption in transit and at rest and security monitoring.
• Experience integrating security and quality checks into CI/CD pipelines.
• Knowledge of security requirements for banking and financial-services data.

Experience Requirements

• Overall experience: Minimum 5 years of professional experience in data engineering ETL development data warehousing big-data engineering or a related technical role.
• Data-engineering experience: Strong hands-on experience building data marts ETL pipelines data transformations and large-scale data-processing solutions.
• Technical experience: Strong experience with Python PySpark Spark Pandas SQL Oracle SQL Hadoop Hive MapReduce and SQL and NoSQL databases.
• Data-warehouse experience: Experience with data-warehouse concepts data modelling data marts source-to-target mapping data transformations and data-quality controls.
• SDLC experience: Experience across development UAT support bug fixing production deployment monitoring and post-production support.
• Production experience: Experience troubleshooting data-pipeline failures query issues data-quality defects performance problems and production incidents.
• Domain preference: Banking and financial-services experience is highly preferred.
• Industry preference: Experience supporting enterprise-scale high-volume regulated or business-critical data environments is preferred.
• Alternative experience pathways: Equivalent combinations of relevant data-engineering experience technical training substantial pipeline implementations open-source contributions or demonstrated production outcomes may be considered.
• Outcome evidence: Candidates should demonstrate contributions to reliable data delivery improved pipeline performance enhanced data quality reduced processing time successful production deployments or reduced recurring support issues.

Day-to-Day Activities

• Design develop test and optimize Python PySpark Spark SQL and ETL solutions supporting data marts and large-scale data processing.
• Analyze source data debug Oracle SQL and PySpark issues validate transformations support UAT resolve defects and monitor production pipelines.
• Collaborate with data application testing operations and business teams through planning sessions requirement discussions technical reviews release meetings and issue-resolution activities.
• Deliver production-quality code data mappings test results validation reports deployment artifacts technical documentation and recommendations within the assigned scope.

Qualifications

• Bachelors degree in Computer Science Information Technology Engineering Data Science or a related field; equivalent relevant experience may be considered.
• Minimum 5 years of hands-on experience in data engineering ETL development data warehousing big-data processing or a related technical role.
• Demonstrated experience with Python PySpark Spark Pandas Oracle SQL ETL development data marts data warehousing Hadoop Hive MapReduce and SQL and NoSQL databases.
• Experience with Jupyter Notebook CI/CD pipelines testing data validation SDLC activities production deployment and post-production support is required.
• Commitment to continuous professional development through relevant training technical experimentation knowledge sharing and ongoing learning in data engineering distributed processing cloud technologies data quality and automation.

Professional Competencies

• Applies structured analytical thinking to investigate data-quality issues pipeline failures query performance concerns processing errors and production incidents.
• Collaborates effectively with data application testing operations and business teams to deliver reliable data solutions and resolve issues.
• Communicates technical findings data limitations defects delivery status risks and recommendations clearly to technical and non-technical stakeholders.
• Adapts to evolving data sources processing technologies business requirements delivery priorities and platform constraints through continuous learning.
• Identifies practical opportunities to improve pipeline performance data quality automation maintainability reliability resource efficiency and sustainability.
• Manages time and priorities effectively across development testing UAT support production deployment incident resolution documentation and continuous improvement.

SYNECHRONS DIVERSITY & INCLUSION STATEMENT

Diversity & Inclusion are fundamental to our culture and Synechron is proud to be an equal opportunity workplace and is an affirmative action employer. Our Diversity Equity and Inclusion (DEI) initiative Same Difference is committed to fostering an inclusive culture promoting equality diversity and an environment that is respectful to all. We strongly believe that a diverse workforce helps build stronger successful businesses as a global company. We encourage applicants from across diverse backgrounds race ethnicities religion age marital status gender sexual orientations or disabilities to apply. We empower our global workforce by offering flexible workplace arrangements mentoring internal mobility learning and development programs and more.

All employment decisions at Synechron are based on business needs job requirements and individual qualifications without regard to the applicants gender gender identity sexual orientation race ethnicity disabled or veteran status or any other characteristic protected by law.

Candidate Application Notice