How AI is applied across API Evangelist and APIs.io. Read my AI disclosure →
API Evangelist API Evangelist
Discovery
Learnings
Guidance
Toolbox
Alignment
API Evangelist LLC

How can I debug why 99.99% of my spans are being dropped

calendar_today February 11, 2025 person MarkR7232213139 domain lightstep

My service, IMS, is getting a ton of errors trying to push to traces to Lightstep.

 

Here's what I know

* Some pushes work, so I have a few but not many, traces in LS

* I have a consistently high `rate(lightstep_error_event_total)` metric on all my pods, so traces are being continously dropped

* I have a lot of `flush failed, could not send report to Collector` errors

* I am getting a lot of `status code (400) is not ok` in my logs for the trace.

 

Is there a way to change the lightstep micro-satellite to log the requests that produce these 400 errors? For the LS go client library, can I turn up the debugging level to get more details on why this is failing?

open_in_new Read original post