DevOps Data Engineer
pivotal
📍 fargo north dakota united states🕐 22d ago🔗 devopsjobs
Job Description
*
Drive the infrastructure strategy alongside the rest of the DevOps team
*
Be responsible for building, managing, and automating our AWS infrastructure
*
Oversee the maintenance and growth of our data lakehouse infrastructure and implementation
*
Champion security by adopting necessary tools and processes
*
Provide leadership to the entire engineering team on various infrastructure and big data topics
*
Identify and adopt high-level technologies, processes, and services that impact the organization on a system-wide scope
* Participate in the L2 rotation for engineering support
### Requirements
*
8+ years of experience as a DevOps Engineer
*
5+ years of experience with the AWS cloud platform, including proficiency with AWS SDK, AWS CLI/UI, and services such as VPC, Route53, Private and Public subnets, route tables, IGW, EC2 Instances, CloudFront, API Gateway, IAM, ELB, Autoscaling, CloudWatch, EFS, NFS, EBS, S3, RDS, Lambda, SQS/SNS, Kafka, Security groups, etc.
*
5+ years of experience with Terraform/Terragrunt
*
5+ years of experience with event processing infrastructure
*
2+ years of experience as a Data Engineer with Databricks, including data pipelines, Debezium, Spark, and data modeling
*
Expert proficiency with Python
*
Model serving experience
*
Expert proficiency in deploying, maintaining, and scaling applications
*
Proficiency with Docker, Kubernetes, and Serverless functions
*
Expert proficiency in setting up CI/CD pipelines (tools/processes/governance)
*
Experience creating and maintaining fully automated CI/CD pipelines for code deployment using tools such as Git, CircleCI, AWS Code build, and Code Deploy / Code pipeline
*
Proficient with both Relational and Non-Relational databases
*
Experience with Security vulnerability scanning tools (Guard Duty, Inspector)
*
Experience building Monitoring dashboards and leveraging tools such as Cloudwatch, Datadog, and Sumologic