We have an opportunity to impact your career and provide an adventure where you can push the limits of what's possible.

• As a Lead Software Engineer at JPMorgan Chase within the Chief Technology Office
• Identity and Access Management
• Data Service Engineering team, you are an integral part of an agile team that works to enhance, build, and deliver trusted market-leading technology products in a secure, stable, and scalable way. As a core technical contributor, you are responsible for conducting critical technology solutions across multiple technical areas within various business functions in support of the firm’s business objectives.

Job responsibilities:

• Executes creative software solutions, design, development, and technical troubleshooting with ability to think beyond routine or conventional approaches to build solutions or break down technical problems
• Architect and deliver high-throughput, low-latency data pipelines on Databricks and Apache Spark (Core, SQL, Structured Streaming)
• Develops secure high-quality production code, and reviews and debugs code written by others
• Define and implement Lakehouse patterns with Delta Lake (ACID, schema evolution, time travel, Z-ordering, compaction) to achieve performance at scale
• Manage Databricks compute and cluster configurations: runtime selection, autoscaling, driver/executor sizing, Spark configs, init-scripts, cluster policies, pools, and instance profiles including orchestrate jobs with Databrick workflows to integrate with AWS
• Design secure data ingestion and transformation on AWS using S3 (data lake and lifecycle), Glue (catalog/ETL), IAM and Secrets Manager (RBAC/credentials), CloudWatch (logging/metrics/alerting), Lambda (serverless utilities), and Kinesis/Kafka/MSK (streaming)
• Implement data quality, lineage, and governance with Unity Catalog and/or Glue Catalog - embed expectations and validations within pipelines
• Optimize spark performance and cost via partitioning strategies, file sizing, AQE, broadcast joins, shuffle tuning, caching, spill/memory management, and job right - sizing
• Leads evaluation sessions with external vendors, startups, and internal teams to drive outcomes-oriented probing of architectural designs, technical credentials, and applicability for use within existing systems and information architecture
• Collaborate with platform security and networking teams to enforce encryption, network controls, and least-privilege access; ensure compliance with organizational policies
• Add to team culture of diversity, opportunity, inclusion, and respect

Required qualifications, capabilities, and skills:

• Formal training or certification on software / data engineering concepts and 3+ years applied experience
• Hands-on Databricks expertise across Delta Lake, Unity Catalog, Workflows, Repos/notebooks, and SQL Warehouses, including cluster setup/optimization and Spark performance tuning
• Advanced in one or more programming language(s), framework(s), and tools (i.e., Python and or Java, Spark and Databricks)
• Proficient in all aspects of the Software Development Life Cycle, and proficient in automation and continuous delivery methods
• Advanced understanding of agile methodologies such as CI/CD Git-based workflows and Terraform IaC, Application Resiliency, and Security
• Experience with incident responses and root - cause analysis; maintaining SLAs, observability, and runbooks to drive continuous improvement in reliability and cost efficiency, participating in technical design reviews and collaborating with product owners and cross - functional teams on roadmaps and backlog prioritization
• Demonstrated proficiency in software applications and technical processes within a technical discipline (e.g., cloud, artificial intelligence, machine learning, mobile, etc.)
• Strong AWS experience with S3, IAM (including roles/instance profiles), Glue, CloudWatch, Kinesis/MSK, and DynamoDB
• Proven track record architecting and operating ETL/ELT pipelines (batch and streaming), covering schema design/evolution, SLAs, and reliability engineering
• Strong SQL and analytics data modeling (dimensional/star schemas) aligned with Lakehouse best practices
• Program with a security-first mindset in secret management, encryption at rest/in transit, and network controls

Preferred qualifications, capabilities, and skills:

• Experience with Databricks: Delta Live Tables and advanced governance (catalogs, grants, auditing)
• Practical cloud native experience - using AWS networking and egress: VPC, subnets, routing, security groups, and data egress controls
• Analyze IaC, CI/CD, and test automation: Terraform for infrastructure deployments; Git workflows, artifact management; testing frameworks (pytest, JUnit).
• Knowledge in cost optimization with autoscaling strategies, spot vs on-demand, auto-termination, storage layouts, and compaction
• Drive real-time ingestion with Kafka/MSK or Kinesis Data Streams/Firehose
• Comprehend observability metrics, lineage, SLAs, and alerting for data systems
• Experience in financial services or similarly regulated environments