Exporter: The success rate is too low.

[winlinvip]

Description

When enable prometheus exporter, there are lots of error clients events, so that the success rate is too low.

1. SRS Version: SRS 5.0

2. Start SRS origin server:

env SRS_LISTEN=19350 SRS_MAX_CONNECTIONS=1000 \
  SRS_PID=./objs/origin.pid SRS_DAEMON=off SRS_SRS_LOG_TANK=console \
  SRS_EXPORTER_ENABLED=on \
  ./objs/srs -e

3. Start SRS edge server:

./objs/srs -c conf/edge.conf

4. Prometheus config prometheus.yml

scrape_configs: 
  - job_name: "srs"
    metrics_path: "/metrics"
    scrape_interval: 15s
    static_configs:
      - targets: ["host.docker.internal:9972"]

5. Run prometheus:

docker run --rm -v `pwd`/prometheus.yml:/etc/prometheus/prometheus.yml  \
    -p 9090:9090 prom/prometheus

Replay

Please describe how to replay the bug?

Step 1: Publish to SRS

ffmpeg -stream_loop -1 -re -i doc/source.flv -c copy -f flv rtmp://localhost/live/livestream

Step 2: Stop FFmpeg.

Step 3: There is always an error

[2023-01-06 13:35:33.282][ERROR][65245][p4q038w5][4] edge push get server control message failed. err=code=1007(SocketRead)(Socket read data failed) : recv interlaced message : read basic header : basic header requires 1 bytes : read bytes : read
thread [65245][p4q038w5]: recv_message() [./src/protocol/srs_protocol_rtmp_stack.cpp:341][errno=4]
thread [65245][p4q038w5]: recv_interlaced_message() [./src/protocol/srs_protocol_rtmp_stack.cpp:792][errno=4]
thread [65245][p4q038w5]: read_basic_header() [./src/protocol/srs_protocol_rtmp_stack.cpp:887][errno=4]
thread [65245][p4q038w5]: grow() [./src/protocol/srs_protocol_stream.cpp:162][errno=4]
thread [65245][p4q038w5]: read() [./src/protocol/srs_protocol_st.cpp:566][errno=4](Interrupted system call)

Step 4: Query prometheus by:

srs_clients_errs_total

Expect

Ignore some disconnect events.

[chundonglinlin]

Will place the error code error_code in the metrics section, similar to node_exporter.

# HELP promhttp_metric_handler_errors_total Total number of internal errors encountered by the promhttp metric handler.
# TYPE promhttp_metric_handler_errors_total counter
promhttp_metric_handler_errors_total{cause="encoding"} 0
promhttp_metric_handler_errors_total{cause="gathering"} 0

TRANS_BY_GPT3