Senior Data Engineer

### Senior Data Engineer **Experience:** 6+ Years **Location:** WFH / Remote (India) **Open Positions:** Multiple **Qualification:** B.Tech / M.Tech / MCA orequivalent **Work Timings:** 1:30 PM IST – 10:30 PM IST **Functional Area:** Data Engineering ### About the Role Weare seeking an experienced Senior Data Engineer to design, develop, and maintain scalable, reliable, andhigh-performance data platforms and pipelines that support businessintelligence, analytics, and data-driven decision-making. Theideal candidate will have strong hands-on expertise in Python,Apache Spark, Apache Airflow, Amazon Redshift, SQL, and modern data engineeringpractices, along with mandatory experience in the Healthcare domain. The candidate should haveexperience building end-to-end ETL/ELT pipelines, processing large-scaledatasets, implementing data quality frameworks, and supporting enterpriseanalytics and reporting needs. This rolerequires a technically strong professional who can collaborate with businessstakeholders, data analysts, architects, and application teams to deliverrobust and scalable data solutions. ### Key Responsibilities * Design, develop, and maintain scalable data pipelines and ETL/ELT workflows for structured and unstructured data. * Build and optimize data ingestion, transformation, and data integration processes using Python and Apache Spark. * Develop, schedule, and monitor workflows using Apache Airflow. * Design and manage enterprise data warehouse solutions using Amazon Redshift. * Implement data models, data marts, and reporting datasets to support analytics and business intelligence requirements. * Optimize data processing jobs and database performance for scalability and efficiency. * Work with large healthcare datasets, ensuring data accuracy, consistency, and compliance with industry standards. * Collaborate with business stakeholders to understand reporting and data requirements. * Implement data quality checks, validation frameworks, and monitoring processes. * Troubleshoot data pipeline issues and perform root cause analysis. * Ensure data security, governance, and compliance requirements are adhered to. * Participate in code reviews, development best practices, and continuous improvement initiatives. * Create and maintain technical documentation, data flow diagrams, and data mapping documents. * Work closely with cross-functional teams including Data Science, Analytics, Product, and Engineering teams. ### Skills, Knowledge & Experience ### Mandatory Skills * 6+ years of experience in Data Engineering, Big Data Engineering, or related roles. * Mandatory experience working in the Healthcare domain. * Strong hands-on experience in Python development for data engineering solutions. * Strong hands-on experience with Apache Spark (PySpark) for large-scale data processing. * Hands-on experience with Apache Airflow for workflow orchestration and scheduling. * Strong experience designing, developing, and optimizing solutions on Amazon Redshift. * Advanced SQL skills with experience in query tuning and performance optimization. * Experience building ETL/ELT pipelines and modern data integration frameworks. * Strong understanding of data warehousing concepts, dimensional modelling, and data architecture. * Experience handling large-scale datasets and complex data transformation requirements. * Strong debugging, troubleshooting, and performance tuning skills. * Experience working with cloud-based data platforms and services (AWS or Azure) is preferred. * Excellent analytical and problem-solving skills. * Strong communication and stakeholder management capabilities. ### Preferred Skills * Experience with AWS data services such as S3, Glue, EMR, Athena, Lambda, or Kinesis * Experience with Azure Data Factory, Azure Databricks, ADLS, Synapse Analytics, or other Azure data services * Exposure to Databricks, Snowflake, or modern Lakehouse architectures * Experience with CI/CD for Data Engineering pipelines * Knowledge of Healthcare data standards such as HL7, FHIR, EHR/EMR, Claims, Provider, Member, or Clinical Data * Experience with data governance, data quality, and compliance frameworks * Experience with Git and DevOps best practices for data engineering This jobdescription is not intended to cover or contain a comprehensive listing of allresponsibilities, duties, or activities that are required. Responsibilities,duties, and/or activities may change, or new ones may be added at any time withor without notice. If you are amotivated professional with a passion for delivering impactful solutions, we'dlove to hear from you. Apply today to be part of a dynamic and forward-thinkingteam at NextGen Invent. Originally posted on [Himalayas](https://himalayas.app)