Enter a job title or keyword
Big Data Platform Engineer
Job Location:
Job Location:
Jersey, NJ - USA
Monthly Salary:
Not provided by the employer
Monthly Salary:
Posted:
11 November 2026 (2 hours ago)
Posted:
Application Deadline:
8 February 2027
Application Deadline:
Vacancies:
1 Vacancy
Vacancies:
Job Summary
Big Data Platform Engineer - Jersey City NJ
Jersey City NJ
Seeking an experienced Spark HDFS and Kafka Administrator/Engineer to manage support and optimize enterprise-grade big data platforms. Deep hands-on expertise with Apache Spark HDFS and Kafka is required and will be the primary focus of this role. Experience administering these technologies within a Cloudera environment is preferred but secondary to strong expertise in the underlying technologies.
Main areas of responsibilities:
Spark Administration: Deploy configure monitor tune and troubleshoot Apache Spark environments and workloads. Support Spark job performance resource management capacity planning and production operations.
HDFS Administration: Manage and support HDFS storage including capacity performance availability permissions replication and troubleshooting. Strong understanding of HDFS architecture and integration with distributed compute platforms is required.
Kafka Administration: Deploy configure manage monitor and troubleshoot Apache Kafka clusters topics producers and consumers. Support performance capacity availability security and production operations.
Monitoring and Performance Tuning: Monitor platform connectivity health capacity and performance using tools such as Grafana Splunk and relevant platform management tools. Proactively identify and resolve performance bottlenecks across Spark HDFS and Kafka.
Security and Compliance: Implement and manage security configurations including Kerberos SSL/TLS authorization policies HDFS ACLs and enterprise IAM/LDAP integrations.
Automation and Scripting: Develop and maintain automation using Python Bash or Perl for routine administration configuration management monitoring and operational support.
Troubleshooting and Support: Provide expert-level troubleshooting and technical support for Spark HDFS Kafka and related big data technologies including participation in an on-call rotation.
Platform Administration: Support installations patching upgrades migrations and platform lifecycle activities while minimizing downtime. Cloudera Hadoop/CDP administration will be part of the role but is secondary to expertise in Spark HDFS and Kafka.
Requirements:
Excellent problem-solving analytical and communication skills.
Minimum 3 years of hands-on production experience administering and supporting Apache Spark HDFS and Kafka.
Strong demonstrable expertise in Spark administration configuration performance tuning and troubleshooting.
Strong demonstrable expertise in HDFS administration storage management security performance and troubleshooting.
Strong demonstrable expertise in Kafka administration topic management producer/consumer architecture performance security and troubleshooting.
Experience with Linux operating systems and command-line tools.
Experience with enterprise monitoring and operational tooling such as Grafana and Splunk.
Experience with Cloudera Hadoop/CDP and Cloudera Manager is preferred but is not the primary skill requirement.
Cloudera certifications or equivalent big data certifications are desirable but not required.
Solid understanding of Kerberos SSL/TLS IAM/LDAP and security concepts for distributed data platforms.
Jersey City NJ
Seeking an experienced Spark HDFS and Kafka Administrator/Engineer to manage support and optimize enterprise-grade big data platforms. Deep hands-on expertise with Apache Spark HDFS and Kafka is required and will be the primary focus of this role. Experience administering these technologies within a Cloudera environment is preferred but secondary to strong expertise in the underlying technologies.
Main areas of responsibilities:
Spark Administration: Deploy configure monitor tune and troubleshoot Apache Spark environments and workloads. Support Spark job performance resource management capacity planning and production operations.
HDFS Administration: Manage and support HDFS storage including capacity performance availability permissions replication and troubleshooting. Strong understanding of HDFS architecture and integration with distributed compute platforms is required.
Kafka Administration: Deploy configure manage monitor and troubleshoot Apache Kafka clusters topics producers and consumers. Support performance capacity availability security and production operations.
Monitoring and Performance Tuning: Monitor platform connectivity health capacity and performance using tools such as Grafana Splunk and relevant platform management tools. Proactively identify and resolve performance bottlenecks across Spark HDFS and Kafka.
Security and Compliance: Implement and manage security configurations including Kerberos SSL/TLS authorization policies HDFS ACLs and enterprise IAM/LDAP integrations.
Automation and Scripting: Develop and maintain automation using Python Bash or Perl for routine administration configuration management monitoring and operational support.
Troubleshooting and Support: Provide expert-level troubleshooting and technical support for Spark HDFS Kafka and related big data technologies including participation in an on-call rotation.
Platform Administration: Support installations patching upgrades migrations and platform lifecycle activities while minimizing downtime. Cloudera Hadoop/CDP administration will be part of the role but is secondary to expertise in Spark HDFS and Kafka.
Requirements:
Excellent problem-solving analytical and communication skills.
Minimum 3 years of hands-on production experience administering and supporting Apache Spark HDFS and Kafka.
Strong demonstrable expertise in Spark administration configuration performance tuning and troubleshooting.
Strong demonstrable expertise in HDFS administration storage management security performance and troubleshooting.
Strong demonstrable expertise in Kafka administration topic management producer/consumer architecture performance security and troubleshooting.
Experience with Linux operating systems and command-line tools.
Experience with enterprise monitoring and operational tooling such as Grafana and Splunk.
Experience with Cloudera Hadoop/CDP and Cloudera Manager is preferred but is not the primary skill requirement.
Cloudera certifications or equivalent big data certifications are desirable but not required.
Solid understanding of Kerberos SSL/TLS IAM/LDAP and security concepts for distributed data platforms.
Required Skills:
PYTHONCLOUDERA HADOOPAPACHE SPARKKERBEROSPERLHDFSAUTOMATIONSPARKBIG DATASPLUNK
