Staff Software Engineer, Batch Data

33 Days Old

At Tecton , we solve complex data problems in production machine learning. Tecton’s feature platform simplifies data activation for smarter models and predictions, abstracting complex engineering to accelerate innovation.
Our founders developed the first Feature Store during their work on Uber’s Michelangelo ML platform. We now bring these capabilities to organizations worldwide.
Funded by Sequoia Capital, Andreessen Horowitz, Kleiner Perkins, Snowflake, and Databricks, we have a rapidly growing, distributed team with offices in San Francisco and New York City. Our team members have extensive experience building and operating critical ML systems at Uber, Google, Meta, Airbnb, Lyft, and Twitter.
We are developing Rift ( talk , blog post ), a managed compute environment enabling data scientists to build batch and streaming pipelines in Python, leveraging open-source technologies like Ray, Arrow, and DuckDB, with integrations across data platforms and warehouses.
As a senior engineer on the Batch Data team, you will architect, design, and scale core compute engines and storage architectures used by our customers. You will improve query optimizer performance, from parsing to plan selection, acting as the "beating heart" of Tecton.
This role offers a unique blend of customer-focused product development, platform innovation, and data engineering, helping companies accelerate their AI initiatives. Your work may involve:
Distributed compute and resource management
Query optimization and distributed execution
Cross-platform data platform integrations
Responsibilities
Lead technical domains from problem definition to implementation and maintenance
Manage strategic projects across teams including product management and engineering
Enhance engineering practices, tooling, and processes; mentor junior engineers
Understand customer challenges in building ML systems
Be versatile within our small, growing team
Qualifications
Experience with large Python, Java, Kotlin, or Go codebases and cloud-native Spark systems (AWS EMR, Databricks, GCP Dataproc)
Performance tuning expertise in Spark, Ray, Maestro, or Airflow
Knowledge of data formats like Parquet, Avro, Arrow, Iceberg, Delta Lake, and object storage (S3, GCS)
Expertise in query performance, optimization, planning, and cost-driven techniques
Experience with distributed systems internals, SQL/NoSQL databases, data lakes, or warehouses
Strong communication skills and ability to write detailed specs
Interest in coaching and mentoring
BSc, MSc, or PhD in Computer Science or related fields
8+ years in building product software systems
The estimated US base salary range is $197,000 - $266,000 annually for employees in California & New York. We offer competitive equity and benefits, including medical, dental, vision, life insurance, 401(k), flexible PTO, holidays, sick leave, and leave of absence options. Compensation depends on location, role, skills, experience, and other factors.
Tecton is a remote-friendly company with a hybrid work policy for employees in SF, NY, and Seattle. We value in-person collaboration and require SF & NY employees to work in-office at least two days per week, and Seattle employees at least two days per month.
We are committed to diversity and inclusion, providing accommodations upon request. We participate in E-Verify to confirm employment eligibility in the U.S.
#J-18808-Ljbffr
Location:
San Francisco, CA, United States
Salary:
$200,000 - $250,000
Category:
IT & Technology