Data Engineer

Company: IMC Trading

Location: Mumbai, Maharashtra, India

Position: Data Engineer

Type: Full-time, On-site

Role Overview

As a Data Engineer, you'll manage and enhance IMC’s data infrastructure used by India-based trading teams. This includes distributed systems, storage platforms, and data pipelines supporting critical analytics, simulations, and reporting. The role emphasizes automation, scalability, and high availability.

Key Responsibilities

• Manage and monitor systems like HDFS, Kubernetes, and Dremio
• Maintain multi-petabyte data infrastructure
• Develop automation for hardware/software deployments (CI/CD)
• Support cross-functional teams: traders, developers, network/systems engineers
• Stay current with technology trends and propose improvements

Required Skills & Experience

• 5–7 years in large-scale data infrastructure roles
• Strong Linux system administration and troubleshooting skills
• Deep expertise in at least one: Kafka, Spark, Cassandra/Scylla, HDFS
• Experience with Docker, Kubernetes, and Helm
• Familiarity with Dremio, Presto, Airflow, or Prefect
• Cloud experience (AWS, GCP, or Azure)
• Proficient in Git and CI/CD practices
• Strong understanding of data security and compliance
• Excellent troubleshooting and problem-solving skills
• High adaptability and fast learning ability