DevOps Data Engineer

pivotal

📍 fargo north dakota united states🕐 22d ago🔗 devopsjobs

Job Description

* Drive the infrastructure strategy alongside the rest of the DevOps team * Be responsible for building, managing, and automating our AWS infrastructure * Oversee the maintenance and growth of our data lakehouse infrastructure and implementation * Champion security by adopting necessary tools and processes * Provide leadership to the entire engineering team on various infrastructure and big data topics * Identify and adopt high-level technologies, processes, and services that impact the organization on a system-wide scope * Participate in the L2 rotation for engineering support ### Requirements * 8+ years of experience as a DevOps Engineer * 5+ years of experience with the AWS cloud platform, including proficiency with AWS SDK, AWS CLI/UI, and services such as VPC, Route53, Private and Public subnets, route tables, IGW, EC2 Instances, CloudFront, API Gateway, IAM, ELB, Autoscaling, CloudWatch, EFS, NFS, EBS, S3, RDS, Lambda, SQS/SNS, Kafka, Security groups, etc. * 5+ years of experience with Terraform/Terragrunt * 5+ years of experience with event processing infrastructure * 2+ years of experience as a Data Engineer with Databricks, including data pipelines, Debezium, Spark, and data modeling * Expert proficiency with Python * Model serving experience * Expert proficiency in deploying, maintaining, and scaling applications * Proficiency with Docker, Kubernetes, and Serverless functions * Expert proficiency in setting up CI/CD pipelines (tools/processes/governance) * Experience creating and maintaining fully automated CI/CD pipelines for code deployment using tools such as Git, CircleCI, AWS Code build, and Code Deploy / Code pipeline * Proficient with both Relational and Non-Relational databases * Experience with Security vulnerability scanning tools (Guard Duty, Inspector) * Experience building Monitoring dashboards and leveraging tools such as Cloudwatch, Datadog, and Sumologic
DevOps Data Engineer at pivotal | MergeJobs