Senior Data Engineer
petroapp
📍 egypt🕐 16d ago🔗 himalayas
Job Description
1. Data platform engineering: Design and maintain scalable batch and near-real-time data pipelines across mobile applications, NFC/fuel transactions, station integrations, ERP integrations, payments, support systems, and operational databases.
2. Data modeling: Create clean, reusable data models for core entities such as customers, vehicles, drivers, stations, transactions, wallets, limits, invoices, products, maintenance services, and geographic coverage.
3. Reliability and quality: Implement data validation, lineage, observability, alerting, reconciliation, and automated quality checks to ensure business-critical dashboards and reports are accurate and timely.
4. Analytics enablement: Partner with analytics, product, finance, operations, and customer success teams to deliver self-service datasets, metrics layers, and well-documented data marts.
5. Performance and cost optimization: Tune queries, storage layouts, orchestration schedules, and cloud resources to improve platform performance and manage infrastructure cost.
6. Data governance and security: Apply data access controls, PII handling, retention practices, auditability, and compliance-aware engineering patterns across the data lifecycle.
7. Integration engineering: Build robust ingestion patterns for APIs, webhooks, CDC, files, event streams, third-party integrations, and partner station data feeds.
8. DevOps for data: Use CI/CD, version control, automated testing, infrastructure-as-code, and deployment standards for data pipelines and transformations.
9. Incident management: Troubleshoot data incidents, conduct root-cause analysis, reduce recurring failures, and communicate impact clearly to stakeholders.
10. Technical mentorship: Review designs and code, establish engineering standards, mentor junior team members, and raise the quality bar for data engineering at [PetroApp](https://himalayas.app/companies/petroapp).
### Requirements
### Required qualifications
* 5+ years of professional experience in data engineering, analytics engineering, platform engineering, or backend engineering with strong data ownership.
* Advanced SQL skills, including query optimization, data modeling, window functions, incremental transformations, and large-table performance tuning.
* Strong Python programming experience for data pipelines, automation, testing, and production-grade data workflows.
* Hands-on experience with workflow orchestration such as Airflow, Dagster, Prefect, or similar tools.
* Experience with modern data warehouses or lakehouse platforms such as BigQuery, Snowflake, Redshift, Databricks, Delta Lake, Iceberg, or equivalent.
* Experience building reliable ELT/ETL pipelines using tools such as dbt, Spark, Kafka, Flink, Fivetran, Stitch, custom API ingestion, or CDC frameworks.
* Practical understanding of data quality, schema evolution, monitoring, alerting, backfills, idempotency, and failure recovery.
* Experience designing dimensional, wide-table, and event-based data models for BI, analytics, and operational reporting.
* Comfort working with cloud platforms such as AWS, GCP, or Azure, plus Git-based engineering workflows.
* Strong communication skills with the ability to translate business requirements into clear technical designs and delivery plans.
### Preferred qualifications
* Experience in fintech, payments, fleet management, logistics, mobility, marketplace, fuel, or high-volume transaction platforms.
* Knowledge of event-driven architectures, streaming data, CDC, API integrations, data contracts, and data mesh or domain-oriented data ownership.
* Experience supporting BI tools such as Power BI, Looker, Tableau, Metabase, Superset, or similar platforms.
* Familiarity with MLOps or feature engineering for fraud detection, anomaly detection, forecasting, customer segmentation, or optimization use cases.
* Experience with data privacy, access control, encryption, secrets management, and compliance expectations in the Middle East or multi-country operations.
### Core technical stack expectations
The exact stack may evolve, but the successful candidate should be comfortable operating across the following categories:
* Languages: SQL, Python; optional Scala or Java for distributed processing.
* Transformation and modeling: dbt or equivalent; dimensional modeling; metrics layers.
* Orchestration: Airflow, Dagster, Prefect, or similar.
* Storage and compute: cloud warehouse, data lake/lakehouse, object storage, distributed processing.
* Streaming and integration: Kafka or equivalent, CDC, APIs, webhooks, files, partner data feeds.
* Engineering practices: Git, CI/CD, automated tests, Docker, Kubernetes or containerized deployment, Terraform or infrastructure-as-code.
* Observability: data quality checks, lineage, pipeline monitoring, logs, alerts, runbooks, and service-level objectives for data products.
### Benefits
* Competitive salary and benefits package.
* Opportunity to work on cutting-edge technology with a passionate team.
* Career growth and development opportunities.
* A collaborative and inclusive work environment.
Originally posted on [Himalayas](https://himalayas.app)