Refreshing/navigating directories 7 times causes failures

[paul-hansen]

Simply refreshing a directory or navigating to a different directory quickly 7 times causes failures. Repeated additional refreshes will mostly fail but some will randomly will succeed.

I tired this with Dolphin (KDE's file manager) and the failures exhibit as a "Loading canceled" screen.

It happens in FileZilla as well, though filezilla caches directory listings so you run into it less often.
FileZilla logs this when the requests fail:

Command:	PASV
Response:	425 No data connection established
Command:	PORT 127,0,0,1,153,97
Response:	502 Active mode not enabled.
Error:	Failed to retrieve directory listing

unFTP logs this when the requests fail:

Aug 11 18:19:31.345 INFO SIZE: Successful size command for file "/": (size: 192)

When they succeed this is logged by unFTP:

Aug 11 18:52:05.787 INFO Listening on passive port 172.17.0.4:50005 username: PaulH
Aug 11 18:52:05.788 INFO Successful LIST "/"; Duration 0ms; Bytes copied 1.03 KiB; Transfer speed 3.58 MB/s

I tried hosting the same directory using vsftpd and did not have this problem.

Here's how I'm running unftp:

AUTH_ADD=192.168.1.128
docker run \
  -e ROOT_DIR=/ftp \
  -e UNFTP_ROOT_DIR=/ftp \
  -e UNFTP_LOG_LEVEL=debug \
  -e UNFTP_PASSIVE_PORTS=50000-50005 \
  -e UNFTP_AUTH_TYPE=rest \
  -e UNFTP_AUTH_REST_METHOD=POST \
  -e UNFTP_AUTH_REST_URL=http://$AUTH_ADD:42069/ftp/auth \
  -e UNFTP_AUTH_REST_BODY='{"username":"{USER}","password":"{PASS}"}' \
  -e UNFTP_AUTH_REST_SELECTOR=/status \
  -e UNFTP_AUTH_REST_REGEX=success \
  -e UNFTP_USR_HTTP_URL=http://$AUTH_ADD:42069/ftp/user/ \
  -p 2121:2121 \
  -p 50000:50000 \
  -p 50001:50001 \
  -p 50002:50002 \
  -p 50003:50003 \
  -p 50004:50004 \
  -p 50005:50005 \
  -p 8080:8080 \
  -v /home/paul/.local/share/my_app/ftp/:/ftp:rw \
  -ti \
  bolcom/unftp:v0.14.7-alpine-debug

[paul-hansen]

This appears to be because the passive port pool is exhausted. Increasing the size of the passive port pool to 50 it takes much longer to happen. Attempting this with vsftpd I couldn't make it happen spamming refresh with only 10 passive ports assigned so this seems to be undesired/unexpected behavior.

It seems we need to be reusing these ports or releasing them so they can be reused by a new connection.

These issues seem to be related:
bolcom/libunftp#439
bolcom/libunftp#459

[paul-hansen]

Off topic: This project is awesome btw. I love having auth over REST to my webserver and the new usr-http-url feature makes that usable for my use case. Thanks!

[hannesdejager]

Thank you @paul-hansen for the detailed information (and the kind words)