Staff Data Engineer

We are the Risk and Vulnerability Management (RVM) team in Apple Services Engineering (ASE) Security. We manage security risk for the infrastructure platforms and services behind iCloud App Store Apple Music TV and Commerce. Findings reach us from scanners red team engagements design reviews vendor advisories threat intelligence and bug bounty. Our job is to turn all of that into one prioritized backlog engineering teams can work from and a posture picture leadership can of that job is data work. Signal arrives from dozens of systems at different quality and age and it rarely says which asset which service or who owns it so someone pieces that together by hand before anyone can act on it. That manual step sets the ceiling on how fast we identify and triage risk and on how fast our partners can fix it. We are consolidating this onto one security data platform: a system of record for findings a graph for ownership and blast radius and a lakehouse for posture metrics and analytics.

We are looking for a Staff Data Engineer to lead the data engineering and data architecture behind it. You will own the data model and the pipeline contracts other teams build against move pipelines from prototype into production the business depends on and make ownership attribution and risk enrichment a service instead of repeated one-off work. Success here takes deep distributed data engineering experience real data judgment and the ability to hold a technical direction across a large matrixed engineering organization.

15 years of experience working with Spark and other distributed data technologies (e.g. Hadoop Presto Flink Druid) for building efficient u0026 large scale data pipelinesnHighly proficient in at least one of Java Python or ScalanDeep expertise in Data Principles Data Architecture u0026 Data Modeling Strong SQL skillsnStrong problem solver with meticulous attention to detail capable of taking on loosely defined problemsnExperience working in a complex matrixed organization involving cross-functional and/or cross-business projectsnStrong communication and collaboration skills u0026 ability to lead high-level discussions on technology strategy and approachnConceptually familiar with AWS cloud resources (S3 EC2 RDS etc)nConceptually familiar with OSI model and understanding of how network works (Load Balancer Routers network tagging NetFlow)nConceptually familiar of the full technology stack (from BMCs Firmware to OS layer to containers and applications) primitivesn

Experience with Cloud Computing platforms like Amazon AWS Google CloudnExperience with building stream-processing applications using Apache Flink Spark-Streaming Apache Storm Kafka Streams or othersnExperience with Search systems (such as ElasticSearch Solr) NoSQL datastores (such as HBase Cassandra MongoDB)nExperience building distributed high-volume data services is a plus