Technical Lead

Job Summary

Design and deliver scalable real-time data and machine learning solutions by building robust ingestion and transformation frameworks across Hadoop ecosystems. Enable end-to-end ML model operationalization and performance optimization, while supporting multi-modal data processing and development of engineering tools and applications.

Key Responsibilities

1. To be responsible for providing technical guidance / solutions ;define, advocate, and implement best practices and coding standards for the team.

2. To develop and guide the team members in enhancing their technical capabilities and increasing productivity

3. To ensure process compliance in the assigned module| and participate in technical discussions/review as a technical consultant for feasibility study (technical alternatives, best packages, supporting architecture best practices, technical risks, breakdown into components, estimations).

4. To prepare and submit status reports for minimizing exposure and risks on the project or closure of escalations.

Skill Requirements

• Design and develop highly scalable, Real time systems using Hadoop ecosystem components(Iceberg, Spark, Ozone, Trino, Hive, Ranger, Kafka, Flink and Nifi)

• Build robust data ingestion and transformation frameworks using Java, Spark, Python, and shell scripting for ingesting multi model data(image, audio, video, unstructured documents) with both batch and real-time.

• Develop full‑stack applications and internal engineering tools using Python, shell scripting, and modern web frameworks (e.g., Flask, React).

• Collaborate closely with data scientists to operationalize machine learning models using Cloudera Machine Learning (CML).

• Perform performance tuning and optimization of data applications on Hadoop to ensure optimal resource utilization.

• Experience working with ML platforms such as CML, Spark MLlib, and Python ML libraries (scikit‑learn, XGBoost), including model deployment.

• Design and develop highly scalable, Real time systems using Hadoop ecosystem components (Iceberg, Spark, Ozone, Trino, Hive, Ranger, Kafka, Flink and Nifi)

• Build robust data ingestion and transformation frameworks using Java, Spark, Python, and shell scripting for ingesting multi model data(image, audio, video, unstructured documents) with both batch and real-time.

• Develop full‑stack applications and internal engineering tools using Python, shell scripting, and modern web frameworks (e.g., Flask, React).

• Collaborate closely with data scientists to operationalize machine learning models using Cloudera Machine Learning (CML).

• Perform performance tuning and optimization of data applications on Hadoop to ensure optimal resource utilization.

Other Requirements

null