Senior Data Engineer

petroapp

📍 egypt🕐 16d ago🔗 himalayas

Job Description

1. Data platform engineering: Design and maintain scalable batch and near-real-time data pipelines across mobile applications, NFC/fuel transactions, station integrations, ERP integrations, payments, support systems, and operational databases. 2. Data modeling: Create clean, reusable data models for core entities such as customers, vehicles, drivers, stations, transactions, wallets, limits, invoices, products, maintenance services, and geographic coverage. 3. Reliability and quality: Implement data validation, lineage, observability, alerting, reconciliation, and automated quality checks to ensure business-critical dashboards and reports are accurate and timely. 4. Analytics enablement: Partner with analytics, product, finance, operations, and customer success teams to deliver self-service datasets, metrics layers, and well-documented data marts. 5. Performance and cost optimization: Tune queries, storage layouts, orchestration schedules, and cloud resources to improve platform performance and manage infrastructure cost. 6. Data governance and security: Apply data access controls, PII handling, retention practices, auditability, and compliance-aware engineering patterns across the data lifecycle. 7. Integration engineering: Build robust ingestion patterns for APIs, webhooks, CDC, files, event streams, third-party integrations, and partner station data feeds. 8. DevOps for data: Use CI/CD, version control, automated testing, infrastructure-as-code, and deployment standards for data pipelines and transformations. 9. Incident management: Troubleshoot data incidents, conduct root-cause analysis, reduce recurring failures, and communicate impact clearly to stakeholders. 10. Technical mentorship: Review designs and code, establish engineering standards, mentor junior team members, and raise the quality bar for data engineering at [PetroApp](https://himalayas.app/companies/petroapp). ### Requirements ### Required qualifications * 5+ years of professional experience in data engineering, analytics engineering, platform engineering, or backend engineering with strong data ownership. * Advanced SQL skills, including query optimization, data modeling, window functions, incremental transformations, and large-table performance tuning. * Strong Python programming experience for data pipelines, automation, testing, and production-grade data workflows. * Hands-on experience with workflow orchestration such as Airflow, Dagster, Prefect, or similar tools. * Experience with modern data warehouses or lakehouse platforms such as BigQuery, Snowflake, Redshift, Databricks, Delta Lake, Iceberg, or equivalent. * Experience building reliable ELT/ETL pipelines using tools such as dbt, Spark, Kafka, Flink, Fivetran, Stitch, custom API ingestion, or CDC frameworks. * Practical understanding of data quality, schema evolution, monitoring, alerting, backfills, idempotency, and failure recovery. * Experience designing dimensional, wide-table, and event-based data models for BI, analytics, and operational reporting. * Comfort working with cloud platforms such as AWS, GCP, or Azure, plus Git-based engineering workflows. * Strong communication skills with the ability to translate business requirements into clear technical designs and delivery plans. ### Preferred qualifications * Experience in fintech, payments, fleet management, logistics, mobility, marketplace, fuel, or high-volume transaction platforms. * Knowledge of event-driven architectures, streaming data, CDC, API integrations, data contracts, and data mesh or domain-oriented data ownership. * Experience supporting BI tools such as Power BI, Looker, Tableau, Metabase, Superset, or similar platforms. * Familiarity with MLOps or feature engineering for fraud detection, anomaly detection, forecasting, customer segmentation, or optimization use cases. * Experience with data privacy, access control, encryption, secrets management, and compliance expectations in the Middle East or multi-country operations. ### Core technical stack expectations The exact stack may evolve, but the successful candidate should be comfortable operating across the following categories: * Languages: SQL, Python; optional Scala or Java for distributed processing. * Transformation and modeling: dbt or equivalent; dimensional modeling; metrics layers. * Orchestration: Airflow, Dagster, Prefect, or similar. * Storage and compute: cloud warehouse, data lake/lakehouse, object storage, distributed processing. * Streaming and integration: Kafka or equivalent, CDC, APIs, webhooks, files, partner data feeds. * Engineering practices: Git, CI/CD, automated tests, Docker, Kubernetes or containerized deployment, Terraform or infrastructure-as-code. * Observability: data quality checks, lineage, pipeline monitoring, logs, alerts, runbooks, and service-level objectives for data products. ### Benefits * Competitive salary and benefits package. * Opportunity to work on cutting-edge technology with a passionate team. * Career growth and development opportunities. * A collaborative and inclusive work environment. Originally posted on [Himalayas](https://himalayas.app)