Infrastructure at Orb: Hard problems, real ownership and impact

Before leading the infrastructure team at Orb, I spent eight years at Asana building and scaling the reactive data loading platform powering the product and pushing real-time updates to clients through distributed caches and pub/sub infrastructure. The challenges were deeply technical in nature, but I found myself wanting to be closer to the business problems.

Orb is where those two threads came together for me: hard, unsolved infrastructure challenges and a very direct line to the product and customers.

If you’re an engineer interested in infrastructure or wondering what infra looks like at a company where usage-based billing is the core product, here’s what the role actually entails.

From early-stage scrappiness to serious scale

When I joined Orb two and a half years ago, there was no “infrastructure team.” We were 15 people in total, and focused on building out the product and getting new customers live.

As we started signing larger customers, and existing customers were growing quicker than ever, it became clear that reliability and performance weren’t background concerns. They were the product. If your billing system isn’t fast and reliable, you can’t trust it. Downtime for Orb means lost revenue for our customers — and we take that seriously.

We spun up a small, two-person infrastructure team. Early on, our mission included:

• Stabilizing reliability: Ensuring load spikes and noisy neighbors have isolated impact and effective failure recovery mechanisms.
• Digging into performance issues: Improving performance of user-impacting surface areas, from invoice issuance speed to APIs powering usage dashboards to alerting pipelines backing spend caps and fraudulent activity detection.
• Laying a scalable foundation: Making architecture decisions to eliminate scaling bottlenecks and allow us to seamlessly serve millions of usage events per second over an ever-increasing number of customers.

I joined because I wanted exactly that kind of work: technically challenging and directly tied to business outcomes, in an environment where minutes matter. At Orb, infra isn’t a back-office function. Our decisions show up directly in what customers experience and in how the business scales.

Infrastructure is core to the product

Orb was purpose-built from the start to be a dynamic billing and monetization platform. We’ve taken an opinionated (and ambitious!) approach by treating billing as the output of a deterministic query over the raw set of usage events, rather than just the result of incrementing state in-stream.

Why is this important?

• Flexibility: By keeping the full history of events, we can recalculate invoices from scratch. Common actions in the real world, like backfilling/amending historical usage data, and retroactively creating subscriptions and modifying pricing models, have first-class support within Orb.
• Extensibility: Usage data is critical for more than just billing — it provides insight into your product and customers for things like simulating pricing changes and forecasting revenue.

What makes this interesting from an infrastructure perspective? Under the hood, we store usage events in ClickHouse and calculate metrics with a SQL query system. Building this architecture to handle the scale and domain complexity of billing at Orb is no easy task — there are serious infrastructure problems worth solving:

• How do we store and query billions of events efficiently?
• How do we keep latency low enough to power real-time analytics and alerting?
• How do we ensure costs scale sub-linearly as the business continues to grow?

These are the types of problems the infrastructure team tackles on a daily basis. We’ve built efficient caching layers, scoped invalidation systems, and complementary high-throughput stream processing pipelines, but we’re just getting started.

A week in the life of infrastructure at Orb

This isn’t a “disappear for six months and then ship” team. The work is collaborative, cross-functional, and fast-paced.

A typical week looks like:

• Daily standups: We hold fifteen-minute team standups every day to cover current work, prioritize new urgent work, and discuss technical decisions.
• Weekly retros and planning: On Fridays, we run a retrospective — reflecting on how the week went, celebrating wins, and discussing team learnings — followed by reviewing the roadmap and scoping out what we’re tackling next.
• On-call rotation: Everyone on infra participates in the infra on-call rotation for a week at a time. Because our services touch so much of the product, it’s not uncommon to work together with the product and customer success teams to resolve incidents.
• Design and whiteboarding sessions: We spend meaningful time whiteboarding architectures, comparing trade-offs, and pressure-testing designs with other teams.
• Customer and cross-functional interaction: Infra collaborates with product, customer success, and sometimes directly with customers in varying capacities. At the end of the day, it’s our responsibility to both understand and inform the direction of the product to be able to build the foundational infrastructure.

Today, the infra team is based in San Francisco with regular in-office collaboration (typically Monday, Thursday, and Friday), and we’re also opening roles to remote engineers.

Who thrives in this role?

Working at a fast growing startup in a high-complexity domain in billing isn’t for everyone. They type of engineer who thrives here:

• Own problems end to end: You’re comfortable taking a fuzzy, high-impact problem and seeing it through, from design to rollout and follow-up.
• Enjoy hard, messy debugging: You’re energized by complex performance issues, subtle correctness bugs, and tricky data edge cases.
• Have a knack for building distributed, scalable systems: You love the challenge of scaling a system to handle 10x the load on a short time horizon.
• Balance idealism with pragmatism: You care about good abstractions and high quality code, and also effectively balance incremental improvements that help customers immediately.
• Like working with others, not just code: You’re comfortable collaborating with product, support, and customers when that’s the fastest way to understand and solve a problem.

If you’re the type of person who makes everything faster, more reliable, and more scalable, while also shaping where the company can go next, you’ll feel at home here.

Want to join us?

Orb is at an inflection point. We’re past the “Does this even work?” stage and firmly in “How far can we take this?” You’ll play a significant role in scaling the platform and shaping how modern companies price, bill, and monetize their products, from usage events to revenue.

Check out our careers page — we’re hiring!

‍