About this role

• Improve Black Rock’s ability to enhance our retail sales distribution capabilities and services suite by creating, expanding and optimizing our data and data pipeline architecture.
• You will create and operationalize data pipelines to enable squads to deliver high quality data-driven product.
• You will be accountable for managing high-quality datasets exposed for internal and external consumption by downstream users and applications.
• Top technical / programming skills – Python, Java and Scala with ability to work across big data frameworks such as Spark, Hadoop Suite, Py Spark, Hive, Cloud Data Platforms Preferably Snowflake and SQL.  Experience working with flat files (e.g., csv, tsv, Excel), Database API sources is a must to both ingest and create transformations.
• Given the highly execution-focused nature of the work, the ideal candidate will roll up their sleeves to ensure that their projects meet deadlines and will always look for ways to optimize processes in future cycles.
• The successful candidate will be highly motivated to create, optimize, or redesign data pipelines to support our next generation of products and data initiatives.
• You will be a builder and an owner of your work product.

Responsibilities:

• Lead in the creation and maintenance of optimized data pipeline architectures on large and complex data sets.
• Assemble large, complex data sets that meet business requirements.
• Act as lead to identify, design, and implement internal process improvements and relay to relevant technology organization.
• Work with stakeholders to assist in data-related technical issues and support their data infrastructure needs.
• Automate manual ingest processes and optimize data delivery subject to service level agreements; work with infrastructure on re-design for greater scalability.
• Keep data separated and segregated according to relevant data policies.
• Demonstrated ability to join a complex global team, collaborate crossfunctionally (data scientists, platform engineers, business stakeholders), and take ownership of major components of the data platform ecosystem and develop data ready tools to support their job.
• Be up-to-date with the latest tech trends in the big-data space and recommend them as needed.
• Identify, investigate, and resolve data discrepancies by finding the root cause of issues; work with partners across various cross-functional teams to prevent future occurrences.

Qualifications:

• Overall 4+ years of hands-on experience in computer/software engineering with majority in big data engineering.
• 4+ years of strong Python or Scala programming skills (Core Python and Py Spark) including hands-on experience creating and supporting UDFs and modules like pytest.
• 4+ years of experience with building and optimizing ‘big data’ pipelines, architectures, and data sets. Familiarity with data pipeline and workflow management tools (e.g., Airflow, DBT, Kafka).
• 4+ years of hands-on experience on developing on Spark in a production environment. Expertise on parallel execution, deciding resources and different modes of executing jobs is required.
• 4+ years of experience using Hive (on Spark), Yarn (logs, DAG flow diagrams), Sqoop. Proficiency bucketing, partitioning, tuning and handling different file formats (ORC, PARQUET & AVRO).
• 4+ years of experience using Transact SQL (e.g., MS SQ Server, MySQL), No-SQL and GraphQL.
• Strong experience implementing solutions on Snowflake
• Experience with data quality and validation frameworks, especially Great Expectations for automated testing.
• Strong understanding and use of Swagger/OpenAPI for designing, documenting, and testing RESTful APIs.
• Experience in deployment, maintenance, and administration tasks related to Cloud (AWS, Azure Preferred), Open Stack, Docker, Kafka and Kubernetes. Familiarity with CI/CD pipelines for data pipeline automation and deployment (Jenkins, GitLab CI, Azure DevOps)
• Experience with data governance, metadata management, and data lineageusing tools like Axonand Unity Catalog**.**Expertise in managing business glossaries, data access control, auditing, and ensuring centralized governance across data assets in both cloud and hybrid environments.
• Hands-on experience with Databricks, including notebooks, workflows, and ML integrations.
• Experience with working with global teams across different time zones
• Plus
• Experience with Machine Learning and Artificial Intelligence
• Plus
• Experience with Generative Artificial Intelligence

For New York, NY Only the salary range for this position is USD**$132,500.00** - USD**$162,000.00** . Additionally, employees are eligible for an annual discretionary bonus, and benefits including healthcare, leave benefits, and retirement benefits. Black Rock operates a pay-for-performance compensation philosophy and your total compensation may vary based on role, location, and firm, department and individual performance.

Our benefits

To help you stay energized, engaged and inspired, we offer a wide range of benefits including a strong retirement plan, tuition reimbursement, comprehensive healthcare, support for working parents and Flexible Time Off (FTO) so you can relax, recharge and be there for the people you care about.

Our hybrid work model

Black Rock’s hybrid work model is designed to enable a culture of collaboration and apprenticeship that enriches the experience of our employees, while supporting flexibility for all. Employees are currently required to work at least 4 days in the office per week, with the flexibility to work from home 1 day a week. Some business groups may require more time in the office due to their roles and responsibilities. We remain focused on increasing the impactful moments that arise when we work together in person – aligned with our commitment to performance and innovation. As a new joiner, you can count on this hybrid model to accelerate your learning and onboarding experience here at Black Rock.

About Black Rock

At Black Rock, we are all connected by one mission: to help more and more people experience financial well-being. Our clients, and the people they serve, are saving for retirement, paying for their children’s educations, buying homes and starting businesses. Their investments also help to strengthen the global economy: support businesses small and large; finance infrastructure projects that connect and power cities; and facilitate innovations that drive progress.

This mission would not be possible without our smartest investment – the one we make in our employees. It’s why we’re dedicated to creating an environment where our colleagues feel welcomed, valued and supported with networks, benefits and development opportunities to help them thrive.

For additional information on Black Rock, please visit @blackrock | Twitter: @blackrock | LinkedIn: www.linkedin.com/company/blackrock

Black Rock is proud to be an equal opportunity workplace. We are committed to equal employment opportunity to all applicants and existing employees, and we evaluate qualified applicants without regard to race, creed, color, national origin, sex (including pregnancy and gender identity/expression), sexual orientation, age, ancestry, physical or mental disability, marital status, political affiliation, religion, citizenship status, genetic information, veteran status, or any other basis protected under applicable federal, state, or local law. View the EEOC’s Know Your Rights poster and its supplement and the pay transparency statement.

Black Rock is committed to full inclusion of all qualified individuals and to providing reasonable accommodations or job modifications for individuals with disabilities. If reasonable accommodation/adjustments are needed throughout the employment process, please email Disability.

Assistance@blackrock.com. All requests are treated in line with our privacy policy.

Black Rock will consider for employment qualified applicants with arrest or conviction records in a manner consistent with the requirements of the law, including any applicable fair chance law.