Senior Engineer - Data Engineering
ksb tech pvt
📍 pune india🕐 3mo ago🔗 workday
Job Description
Key Responsibilities
--------------------
* Contribute to the design and development of scalable data pipelines and a growing data lake
* Build and extend data processing workflows using Python, Apache Spark, and Databricks
* Define technical standards, best practices, and reusable frameworks for data engineering
* Ensure data quality, reliability, performance, and maintainability across data solutions
* Support data modeling, data integration, and transformation processes for analytics and reporting
* Drive automation, monitoring, and CI/CD improvements to ensure operational excellence
* Collaborate across teams, acting as a technical interface between the data platform and engineering, analytics, and business stakeholders.
* Contribute to architecture decisions and long-term data platform strategy
Your Profile
------------
* Outstanding programming experience, preferably in Python; ability to write clean, testable, production-grade code; able to write clean, testable, production-grade code
* Strong SQL skills and familiarity with structured and semi-structured data formats (JSON, Protobuf, Delta format)
* Hands-on experience with Apache Spark, ideally on Databricks, and understanding of the medallion architecture
* Solid grasp of data lakehouse principles, data modeling, and data governance concepts
* Experience building and maintaining CI/CD pipelines (e.g. GitLab CI); familiarity with IaC and deployment
* Cloud Platforms: Experience with AWS or comparable cloud providers; familiarity with Databricks as a managed Lakehouse platform
* Experience with event-driven architectures or streaming platforms (e.g. Kafka)
* Proven track record deploying, monitoring, and maintaining data pipelines and services in
production environments; experience with testing practices
* Able to work autonomously and take ownership of tasks end-to-end
* Clear and concise communicator — comfortable working across engineering and data teams
Education & experieance
-----------------------
* Bachelor's degree in Software Engineering, Mathematics, Physics, or a related field
* 3 years of project/coding experience in a company
* Cloud experience:
* Databricks, AWS/Azure
* Staging, testing, Git, pipelines
* Distributed systems, data pipelines
* Python experience:
* Versioning, package management, requirements/environments
* Change data capture
* Data engineering experience:
* SQL
* Partitioning
* Data structures (tables, relations)
* Lakehouse