You don't need to build observability tooling from scratch. The open source community has already produced mature tools that can be adapted for LLM workflow monitoring. OpenTelemetry provides a standardised framework for collecting traces, metrics, and logs. SigNoz and similar platforms offer dashboards and alerting on top of that data. Honeycomb excels at exploring high-cardinality event data — exactly the kind of data LLM interactions produce.
Leveraging what already exists
The key insight is that LLM observability isn't a fundamentally new problem. It's a variant of application performance monitoring applied to a new kind of workload. The same patterns that work for tracking API calls, database queries, and microservice interactions work for tracking Claude's inputs, outputs, connector activity, and decision chains.
Most mid-market businesses don't know these tools exist, let alone how to apply them to AI workflows. Part of the consulting value is bringing this ecosystem to businesses that need visibility but don't have the expertise to set it up. The tools are there — they just need to be configured and connected to the right data sources.