Senior Data Engineer
Core
Design, build, and maintain scalable data pipelines and security data platforms to ingest telemetry data, support AI/ML use cases, and enable analytics and detection capabilities.
Role type
Senior IC Data Engineer (Security & Cloud)
Builds
Scalable data pipelines (batch and real-time), security data platforms, and data products for AI/ML and security detection.
Domain
Financial Services / Cyber Security / Cloud Data Engineering
Deliverable
production ML models | product features | dashboards & analysis
Required skills
AWS services (EC2, S3, Lambda, Athena, Kinesis, Redshift, Glue, EMR, DynamoDB, IAM, Step Functions, SQS, SNS, CloudWatch), Python framework development, PySpark/Spark, SQL, Terraform, Docker, Kubernetes, streaming data processing, event-driven architectures, data encryption (SSL/TLS, at rest/in transit), CI/CD, infrastructure-as-code.
Preferred skills
Gen AI tools (Claude, GitHub Copilot, VS Code Co-pilot), complex automation and engineering framework development.
Technologies
AWS, Python, PySpark, SQL, Terraform, Jenkins, Docker, Kubernetes, Kafka, Snowflake, DBT, ServiceNow SecOps.
Responsibilities
Drive development of scalable data solutions and pipelines; design and manage data models and products for analytics and detection; ensure data quality, reliability, and observability; integrate data solutions into enterprise security platforms; contribute to DevSecOps practices including CI/CD and IaC; own and develop technical strategy for medium to complex engineering initiatives.
Seniority
Senior, hands-on IC with strategic ownership