Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Dpservice crashes due to early telemetry in multiport-eswitch mode #610

Open
PlagueCZ opened this issue Sep 26, 2024 · 1 comment · May be fixed by #613
Open

Dpservice crashes due to early telemetry in multiport-eswitch mode #610

PlagueCZ opened this issue Sep 26, 2024 · 1 comment · May be fixed by #613
Assignees
@vlorinc
Labels
bug Something isn't working

Comments

@PlagueCZ
Copy link
Contributor

This happened in OSC environment when using multiport-eswitch and dpservice-exporter.

By asking for /ethdev/xstats during initialization, dpservice crashes.

Easily reproducible by running while true; do echo "/ethdev/xstats,0" | dpdk-telemetry.py; sleep 1; done and then starting dpservice.

Stack trace is showing corrupted stack trace on the telemetry thread, so I would think it's some sort of stack overflow. But there are no debug messages from telemetry to know more.
@PlagueCZ PlagueCZ added the bug Something isn't working label Sep 26, 2024
FlorinPeter added a commit that referenced this issue Sep 26, 2024
@FlorinPeter
fixes #610 Add TCP port check for DPDK telemetry connection
b8ca3ef 
Implemented a check to verify if TCP port 1337 on localhost is open before attempting to write to the DPDK connection. This ensures more robust error handling and potentially resolves connectivity issues earlier in the process.
vlorinc pushed a commit that referenced this issue Sep 26, 2024
@FlorinPeter @vlorinc
fixes #610 Add TCP port check for DPDK telemetry connection
f1cf8c5 
Implemented a check to verify if TCP port 1337 on localhost is open before attempting to write to the DPDK connection. This ensures more robust error handling and potentially resolves connectivity issues earlier in the process.
vlorinc pushed a commit that referenced this issue Sep 30, 2024
@FlorinPeter @vlorinc
fixes #610 Add TCP port check for DPDK telemetry connection
680ef63 
Implemented a check to verify if TCP port 1337 on localhost is open before attempting to write to the DPDK connection. This ensures more robust error handling and potentially resolves connectivity issues earlier in the process.
PlagueCZ pushed a commit that referenced this issue Sep 30, 2024
@FlorinPeter @PlagueCZ
fixes #610 Add TCP port check for DPDK telemetry connection
c30feb2 
Implemented a check to verify if TCP port 1337 on localhost is open before attempting to write to the DPDK connection. This ensures more robust error handling and potentially resolves connectivity issues earlier in the process.
@vlorinc vlorinc linked a pull request Oct 1, 2024 that will close this issue
Open
@vlorinc vlorinc linked a pull request Oct 1, 2024 that will close this issue
Fix/exporter port #613
Open
PlagueCZ pushed a commit that referenced this issue Oct 1, 2024
@FlorinPeter @PlagueCZ
fixes #610 Add TCP port check for DPDK telemetry connection
82abea1 
Implemented a check to verify if TCP port 1337 on localhost is open before attempting to write to the DPDK connection. This ensures more robust error handling and potentially resolves connectivity issues earlier in the process.
@byteocean
Copy link
Contributor

Steps to reproduce using standard tools:

  1. configure the machine using dpservice's prepare script with --multiport-eswitch --pf1-proxy;
  2. start testpmd sudo ./dpdk-testpmd -a 3b:00.0,dv_flow_en=2,representor=pf[0-1]vf0 -- -i --rxq=1 --txq=1 --disable-device-start;
  3. running while true; do echo "/ethdev/xstats,0" | dpdk-telemetry.py; sleep 1; done leads to the crashing of testpmd

vlorinc pushed a commit that referenced this issue Oct 17, 2024
@FlorinPeter @vlorinc
fixes #610 Add TCP port check for DPDK telemetry connection
080e47b 
Implemented a check to verify if TCP port 1337 on localhost is open before attempting to write to the DPDK connection. This ensures more robust error handling and potentially resolves connectivity issues earlier in the process.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@vlorinc vlorinc

Labels
bug Something isn't working
Projects
Networking
Status: In Progress
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix/exporter port ironcore-dev/dpservice
3 participants
@byteocean @PlagueCZ @vlorinc