DigitalDownConverter — Troubleshooting
Pipeline stuck in configuring
Cause: mkrecv cannot bind to the multicast group, or the input stream address is unreachable.
Resolution: Check that the multicast group in input_data_streams[0].ip is reachable
from the host NIC. Verify that the packetizer is transmitting.
Diagnostic commands:
docker logs ddc_processor
# look for "mkrecv" bind errors or "no data received"
output_data_streams count mismatch
Cause: len(output_data_streams) != 2 × len(f_lo). The pipeline raises
EDDPipelineError during configure.
Resolution: Ensure the configuration contains exactly two output stream keys per entry
in f_lo (one _lower, one _upper).
NUMA check failure
Cause: The pipeline detects that the GPU and input NIC are on different NUMA nodes, which causes suboptimal PCIe transfers.
Resolution: This is a warning by default. Set nonfatal_numacheck: true to suppress it
if the system cannot be reconfigured. For production, ensure GPU and NIC share a NUMA node.
High heap drop rate in mkrecv
Cause: Input buffer (dada_nslots) is too shallow for the incoming data rate, or the
GPU processing falls behind.
Resolution: Increase dada_nslots. Check GPU utilisation and ensure no other process
competes for GPU resources.
Diagnostic commands:
# inside container
dada_db -l # list PSRDADA ring buffers and fill levels
output-rate sensor reports 0
Cause: The pipeline has not yet reached running state, or output_type is set to
[].
Resolution: Confirm the pipeline state via ?sensor-value pipeline-status. Ensure
output_type contains at least one valid entry.
Log locations
Log |
Location |
|---|---|
Container stdout |
|
mkrecv output |
included in container stdout |
InfluxDB metrics |
Grafana → dada_monitor panel |