Senior Apache Spark Technical Lead - Scala, Python

Job Summary

The Senior Technical Lead at HCL will be responsible for leading technical teams and projects related to apache spark, scala, and python. The role involves overseeing the design, development, and implementation of scalable and efficient solutions using these technologies.

Key Responsibilities

As a Sr. Data Engineer, you will:

Lead the scoping, design and implementation of complex features
Lead and push the boundaries of analytics and powerful, scalable applications
Design and implement intuitive, responsive UIs that allow issuers to better understand data and analytics
Build and maintain analytics and data models to enable performant and scalable products
Ensure a high-quality code base by writing and reviewing performant, well-tested code
Mentor junior software engineers and teammates
Drive innovative improvements to team development processes
Partner with Product Managers and Customer Experience Designers to develop a deep understanding of users and use cases and apply that knowledge to scoping and building new modules and features
Collaborate across teams with exceptional peers who are passionate about what they do

Skill Requirements

5+ years of full stack engineering experience in an agile production environment
Experience leading the design and implementation of large, complex features in full-stack applications
Ability to easily move between business, data management, and technical teams; ability to quickly intuit the business use case and identify technical solutions to enable it

• Experience leveraging open source tools, predictive analytics, machine learning, Advanced Statistics, and other data techniques to perform analyses
• High proficiency in using Python or Scala, Spark, Hadoop platforms & tools (Hive, Impala, Airflow, Ni Fi, Scoop), SQL to build Big Data products & platforms
• Experience with cloud data platforms such as Databricks, and Snowflake
• Experience in building and deploying production-level data-driven applications and data processing workflows/pipelines and/or implementing machine learning systems at scale in Java, Scala, or Python and deliver analytics involving all phases like data ingestion, feature engineering, modeling, tuning, evaluating, monitoring, and presenting
  Experience coaching and mentoring junior teammates
  Experience leading a large technical effort that spans multiple people and teams
  Proficiency with Java/Spring Boot, .NET/C#, React, Redux, Typescript, and SQL Server or other object-oriented languages, front-end frameworks, and/or relational database technologies
  Strong technologist with proven track record of learning new technologies and frameworks
  Customer-centric development approach
  Passion for analytical / quantitative problem solving
  Experience identifying and implementing technical improvements to development processes
  Collaboration skills with experience working with people across roles and geographies
  Motivation, creativity, self-direction, and desire to thrive on small project teams
  Superior academic record with a degree in Computer Science or related technical field
  Strong written and verbal English communication skills

Other Requirements

Proficiency with Java/Spring Boot, .NET/C#, React, Redux, Typescript, and SQL Server or other object-oriented languages, front-end frameworks, and/or relational database technologies