Jobs

Big Data / PySpark Engineering Lead - Vice President
PUNE, Mahārāshtra, India
·
On-site
·
Full-time
·
6d ago
The Applications Development Technology Lead Analyst is a senior level position responsible for establishing and implementing new or revised application systems and programs in coordination with the Technology team. The overall objective of this role is to lead applications systems analysis and programming activities.
Key Responsibilities
Architecture & Design
- Design and implement scalable, fault-tolerant batch and real-time data processing pipelines.
- Develop robust data models and schema designs optimized for both performance and storage efficiency.
- Evaluate and integrate emerging tools and frameworks (e.g., Spark, Flink, Kafka) into the existing stack.
- Provide in-depth analysis with interpretive thinking to define issues and develop innovative solutions
- Develop comprehensive knowledge of how areas of business, such as architecture and infrastructure, integrate to accomplish business goals
Data Modernization & Migration Leadership
- Legacy Systems Decommissioning: Lead the strategic migration of data and logic from legacy platforms (e.g. on-premises SQL Servers) to a modern Data Lakehouse environment.
- ETL/ELT Transformation: Re-engineer existing stored procedures and complex legacy ETL jobs into scalable, distributed processing frameworks using Spark (Python) and Starburst/Trino.
- Validation & Parity Testing: Design and implement automated frameworks for Data Parity Testing to ensure 100% accuracy and consistency between legacy outputs and new big data results.
- Schema Evolution: Map and transform rigid, legacy relational schemas into flexible, high-performance formats optimized for the cloud (e.g., Parquet, Avro, or Iceberg).
- Phased Cutover Management: Orchestrate a phased migration strategy (Parallel Run, Shadow Execution) to ensure zero downtime for downstream business applications and reporting tools.
- Performance Benchmarking: Establish performance baselines on legacy systems and ensure the new Big Data architecture meets or exceeds those benchmarks at scale.
- Resolve variety of high impact problems/projects through in-depth evaluation of complex business processes, system processes, and industry standards
- Engineering Excellence
- Write clean, high-performance code in Python.
- Optimize complex SQL queries and fine-tune distributed computing clusters to reduce latency and costs.
- Ensure data integrity and security by implementing rigorous validation and encryption standards.
Dev
Ops & Reliability:
- Build and maintain CI/CD pipelines for automated testing and deployment of data jobs.
- Monitor system health and troubleshoot performance bottlenecks across the data lifecycle.
Leadership & Strategy:
- Provide technical mentorship and conduct code reviews for junior and mid-level engineers. Serve as advisor or coach to mid-level developers and analysts, allocating work as necessary.
- Translate complex business requirements into technical specifications.
- Collaborate with Product Managers to ensure data availability for downstream analytics, business models and users
- Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency.
- Partner with multiple management teams to ensure appropriate integration of functions to meet goals as well as identify and define necessary system enhancements to deploy new products and process improvements
Required Skills & Qualifications:
- Highly experienced and skilled technical lead with 12+years of experience with software building and platform engineering.
- Experience in Data Engineering, focused on Big Data ecosystems.
- Knowledge in Hadoop, YARN, Hive, Impala, Spark, and Spark SQL with extensive high volume of data processing pipeline development. Programming Expert level and hand on experience in Python.
- Familiarity with data formats like Avro, Parquet, CSV, JSON.
- Hands-on experience in writing SQL queries.
- Highly experienced with Unix based operating systems and shell scripting.
- Experience with source code management tools such as Bitbucket, Git etc.
- Big Data Tech Proficiency and hands-on in Hadoop, Spark, Hive, Kafka, and NoSQL databases (MongoDB, HBase).
- Experience working with query engines like Trino, Presto, Starburst
- Strong computer science fundamentals in data structures, algorithms, databases, and operating systems.
- Reverse Engineering, ability to read "spaghetti" SQL or old scripts and document the business logic before moving it.
- Data Lineage, Experience using tools (like Collibra or Informatica) to track where data comes from and where it’s going.
- Change Management, Experience managing the technical "shock" to the business when switching from legacy BI tools to modern query engines like Starburst.
Preferred Qualities
- Problem Solver: You don't just fix bugs; you identify the root cause to prevent recurrence.
- Communicator: You can explain the "why" behind a technical decision to non-technical stakeholders.
- Automation and AI Mindset: You believe that if a task has to be done twice, it should be automated. Familiarity with AI tools to expedite deliveries.
Job Family Group:
Technology
Job Family:
Applications Development
Time Type:
Full time
Most Relevant Skills
Please see the requirements listed above.
Other Relevant Skills
Py Spark.
Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law.
If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi.
View Citi’s EEO Policy Statement and the Know Your Rights poster.
Total Views
0
Apply Clicks
0
Mock Applicants
0
Scraps
0
Similar Jobs

Staff Data Scientist
PayPal · Dublin, County Dublin, Ireland

Principal Associate, Data Scientist - Risk Management Product
Capital One · 5 Locations

Vice President - Data Transformation & Strategy Lead
JPMorgan Chase · Bournemouth, United Kingdom

2026 BNY Analyst Program - Engineering Data Science (Manchester)
BNY Mellon · Greater Manchester, United Kingdom

Risk Data Aggregation, (Firm Risk Management)(Risk Management) : Job Level - Executive Director
Morgan Stanley · New York, NY
About Citigroup

Citigroup
PublicCitigroup Inc. or Citi is an American multinational investment bank and financial services company based in New York City. The company was formed in 1998 by the merger of Citicorp, the bank holding company for Citibank, and Travelers; Travelers was spun off from the company in 2002.
10,001+
Employees
New York City
Headquarters
Reviews
3.3
4 reviews
Work Life Balance
3.0
Compensation
3.2
Culture
2.8
Career
2.5
Management
2.7
35%
Recommend to a Friend
Pros
Compensation increases for investment banking roles
Legitimate investment banking employer
Internship opportunities available
Cons
Unclear career progression paths
Limited meaningful experience in internships
Compensation raises lower than competitors
Salary Ranges
28 data points
Mid/L4
Senior/L5
Staff/L6
Mid/L4 · Business Risk Intermediate Analyst
1 reports
$77,165
total / year
Base
$67,100
Stock
-
Bonus
-
$77,165
$77,165
Interview Experience
5 interviews
Difficulty
2.8
/ 5
Duration
14-28 weeks
Experience
Positive 0%
Neutral 40%
Negative 60%
Interview Process
1
Application Review
2
Recruiter Screen
3
Programming Assessment
4
Hiring Manager Interview
5
Panel/Superday Interviews
6
Final Decision
Common Questions
Technical Knowledge
Case Study
Behavioral/STAR
Past Experience
Culture Fit
News & Buzz
National Pension Service Raises Stake in Citigroup Inc. $C - MarketBeat
Source: MarketBeat
News
·
5w ago
Form 424B2 CITIGROUP INC - StreetInsider
Source: StreetInsider
News
·
5w ago
Citigroup or Wells Fargo: Which Bank Stock Has More Upside in 2026? - TradingView
Source: TradingView
News
·
5w ago
Citigroup Inc. (C) is Attracting Investor Attention: Here is What You Should Know - Yahoo Finance
Source: Yahoo Finance
News
·
5w ago