Data Engineer, Active Grid Response
San Francisco, CA - USA
Job Summary
- Building ETL/ELT pipelines that ingest transformer pole and sensor telemetry into Gridwares Data Lake and Lakehouse
- Developing and maintaining real-time and batch ingestion processes using Python SQL Databricks and Spark
- Implementing data quality checks validation rules and automated testing for stable operations
- Collaborating with Software Firmware and Data Science teams to define ingestion schemas and transformations
- Working with cloud-native tools to optimize pipeline throughput and cost efficiency
- Monitoring pipelines for reliability troubleshooting issues and contributing to on-call rotations
- Writing documentation for data processes models and metadata
- 24 years of experience as a Data Engineer (or Backend Engineer with heavy data exposure)
- Strong proficiency inPythonandSQL
- Familiarity with data warehouses Lakehouse platforms or big data tools (Databricks Spark or equivalent)
- Experience with pipeline orchestration tools (Airflow Dagster Prefect etc.)
- Understanding of event-driven systems or streaming platforms (Kafka Kinesis Pub/Sub)
- Solid foundation in data modeling testing and version control
- Ability to work collaboratively in a high-autonomy fast-paced environment
- Experience with IoT telemetry ingestion or time-series data
- Exposure to Unity Catalog governance or schema enforcement
- Understanding of Protobuf Avro Parquet or serialization formats
- Hands-on experience with observability tools (Grafana OpenTelemetry)
Required Experience:
IC
About Company
**At this time, Gridware is unable to provide visa sponsorship or immigration support for this role. We’re only able to consider candidates who are currently authorized to work in the country of employment without visa sponsorship now or in the future.** This describes the ideal candi ... View more