Senior Security Data Engineer

Added
1 hour ago
Type
Full time
Salary
Salary not provided

Related skills

json sql python kafka regex

📋 Description

  • Maintain high-throughput streaming pipelines to ingest logs from various sources (Firewalls, Cloud
  • Write parsers to convert raw, messy logs into standard schemas (e.g., OCSF or ECS) for consistent
  • Implement routing logic to send "high-value" data to the SIEM and "bulk" data to low-cost Object
  • Clean and structure data to enable AI/ML detection models and advanced analytics.

🎯 Requirements

  • 5-10 years of experience.
  • Proficiency in Python (for ETL) and SQL (for complex querying).
  • Experience with Message Queues (e.g., Kafka, Pub/Sub) and stream processing concepts.
  • Mastery of Regex and log parsing strategies for standard formats (Syslog, CEF, JSON).
  • Understanding of Data Lake principles (Parquet/Avro formats) vs. Data Warehouses.
Share job

Meet JobCopilot: Your Personal AI Job Hunter

Automatically Apply to Data Jobs. Just set your preferences and Job Copilot will do the rest — finding, filtering, and applying while you focus on what matters.

Related Data Jobs

See more Data jobs →