Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

serial: Add --device virtio-serial,pty support #113

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

serial: Add --device virtio-serial,pty support #113

wants to merge 4 commits into from

Conversation

cfergeau
Copy link
Collaborator

This new virtio-serial option allocates a pseudo-tty for the VM console. It
can then be accessed using screen for example. This is a bit similar to
the --device virtio-serial,stdio option, except that the console is not
tied to the terminal running vfkit, it's possible to connect/disconnect
from the pseudo-tty from any terminal.

This fixes #48

@cfergeau
Use gocleanup package
0589358 
pkg/vf/virtionet.go uses local adhoc solution to run cleanup tasks when
vfkit exits.
github.com/onsi/gocleanup achieves the same result in more generic code.

Since there are more things I'd like to cleanup at exit time, let's make
use of this package. It hasn't been changed in a long time, but its code
is a short single file, at worse we can import this file in vfkit
repository and make the changes we need.

Signed-off-by: Christophe Fergeau <[email protected]>
@cfergeau
rest: Use gocleanup
9989850 
When using the REST API over a unix socket, this ensures the unix socket
is removed from the filesystem when vfkit exits.
@cfergeau
serial: Add --device virtio-serial,pty support
6836da5 
This new virtio-serial option allocates a pseudo-tty for the VM console.
It can then be accessed using `screen` for example.
This is a bit similar to the `--device virtio-serial,stdio` option,
except that the console is not tied to the terminal running vfkit, it's
possible to connect/disconnect from the pseudo-tty from any terminal.

This fixes crc-org#48

Signed-off-by: Christophe Fergeau <[email protected]>
@cfergeau
virtio: Simplify setRawMode
a570c1b 
This uses more helper APIs from github.com/pkg/term/termios, and makes
the code closer to Apple's recommendations in
https://developer.apple.com/documentation/virtualization/running_linux_in_a_virtual_machine?language=objc#:~:text=Configure%20the%20Serial%20Port%20Device%20for%20Standard%20In%20and%20Out
This also removes direct use of `syscall` in pkg/vf/virtio.go

Signed-off-by: Christophe Fergeau <[email protected]>
@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Mar 15, 2024

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from cfergeau. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@cfergeau cfergeau mentioned this pull request Mar 15, 2024
Open
@anjannath
Copy link
Member

tested and working as expected..

% ./out/vfkit \
    --cpus 2 --memory 2048 \
    --bootloader efi,variable-store=efi-variable-store,create \
    --device virtio-blk,path=vfkit-test-image.raw --device virtio-serial,pty
INFO[0000] &{2 2048    {[efi variable-store=efi-variable-store create] true}  [virtio-blk,path=vfkit-test-image.raw virtio-serial,pty] none://  false}
INFO[0000] boot parameters: &{EFIVariableStorePath:efi-variable-store CreateVariableStore:true}
INFO[0000]
INFO[0000] virtual machine parameters:
INFO[0000] 	vCPUs: 2
INFO[0000] 	memory: 2048 MiB
INFO[0000]
INFO[0000] Adding virtio-blk device (imagePath: vfkit-test-image.raw)
INFO[0000] Using PTY (pty path: /dev/ttys002)
INFO[0000] virtual machine is running
INFO[0000] waiting for VM to stop

after this i was able to see the fedora login prompt by running % screen /dev/ttys002

@anjannath
Copy link
Member

/lgtm

@openshift-ci openshift-ci bot added the lgtm label Mar 18, 2024
@praveenkumar
Copy link
Member

I am testing it with crc bundle with the script part of contrib/script with following change but not able to get any login prompt

$ git diff
diff --git a/contrib/scripts/start-crc-bundle.sh b/contrib/scripts/start-crc-bundle.sh
old mode 100644
new mode 100755
index 34a28ee..965a497
--- a/contrib/scripts/start-crc-bundle.sh
+++ b/contrib/scripts/start-crc-bundle.sh
@@ -15,6 +15,6 @@ cp -c ${BUNDLE_PATH}/${DISKIMG} overlay.img
     --initrd "${BUNDLE_PATH}/${INITRD}" \
     --kernel-cmdline "${CMDLINE}" \
     --device virtio-blk,path=overlay.img \
-    --device virtio-serial,logFilePath=start-bundle.log \
+    --device virtio-serial,pty \ 
     --device virtio-net,nat,mac=72:20:43:d4:38:62 \
     --device virtio-rng

$ contrib/scripts/start-crc-bundle.sh /Users/prkumar/.crc/cache/crc_vfkit_4.13.14_arm64
[...]
INFO[0000]                                              
INFO[0000] virtual machine parameters:                  
INFO[0000] 	vCPUs: 2                                    
INFO[0000] 	memory: 2048 MiB                            
INFO[0000]                                              
INFO[0000] Adding virtio-blk device (imagePath: overlay.img) 
INFO[0000] Using PTY (pty path: /dev/ttys000)            <= tried to connect this with screen and nothing
INFO[0000] Adding virtio-net device (nat: true macAddress: [72:20:43:d4:38:62]) 
INFO[0000] Adding virtio-rng device                     
INFO[0000] virtual machine is running                   
INFO[0000] waiting for VM to stop

I will try with fedora also same what @anjannath tested and put another comment.

@praveenkumar
Copy link
Member

To me even steps mentioned by @anjannath didn't work after screen /dev/ttys000 I just see the blank without any prompt.

@cfergeau
Copy link
Collaborator Author

To me even steps mentioned by @anjannath didn't work after screen /dev/ttys000 I just see the blank without any prompt.

You need to make sure you have console=hvc0 on the kernel cmd line if you are booting with external kernel. If I connect late in the boot process, sometimes I cannot get output, not clear why.

@anjannath
Copy link
Member

I tried again and it is working for me still, maybe you have to attach to the pty quickly otherwise you miss the login prompt and it doesn't re-appear?

pty.mov

but another thing i noticed is that the keyboard only works sometimes, a few times (2 out of 4 runs) during testing keys are not echoed in screen, which i assumed to be the keyboard not working

@praveenkumar
Copy link
Member

maybe you have to attach to the pty quickly otherwise you miss the login prompt and it doesn't re-appear?

I also tried to attach pty quickly but still nothing, waiting more that 5 mins.

@praveenkumar
Copy link
Member

but another thing i noticed is that the keyboard only works sometimes, a few times (2 out of 4 runs) during testing keys are not echoed in screen, which i assumed to be the keyboard not working

So after 5-6 retry finally I am able to get the login prompt for fedora and crc bundle but then on crc I am not able to use the keyboard but on fedora able to do.

@cfergeau
Copy link
Collaborator Author

but another thing i noticed is that the keyboard only works sometimes, a few times (2 out of 4 runs) during testing keys are not echoed in screen, which i assumed to be the keyboard not working

So after 5-6 retry finally I am able to get the login prompt for fedora and crc bundle but then on crc I am not able to use the keyboard but on fedora able to do.

Maybe virtio: Simplify setRawMode is causing issues. Or maybe it would be better to use https://github.com/pkg/term/blob/1a4a3b719465afccedae7788469c0b54a43417e0/termios/termios.go#L41-L50 , but I could not get this to work correctly.
Really not familiar with pty/terminal input/output :-/

@praveenkumar
Copy link
Member

but another thing i noticed is that the keyboard only works sometimes, a few times (2 out of 4 runs) during testing keys are not echoed in screen, which i assumed to be the keyboard not working

So after 5-6 retry finally I am able to get the login prompt for fedora and crc bundle but then on crc I am not able to use the keyboard but on fedora able to do.

Maybe virtio: Simplify setRawMode is causing issues. Or maybe it would be better to use https://github.com/pkg/term/blob/1a4a3b719465afccedae7788469c0b54a43417e0/termios/termios.go#L41-L50 , but I could not get this to work correctly. Really not familiar with pty/terminal input/output :-/

I am also not sure what causing the issue but as of now this functionality is quite buggy and we might not able to consume it when required like debugging an broken ssh connection. Do you think we can use tty along with log to a file because in tty I also don't see boot logs ?

@openshift-merge-robot
Copy link
Collaborator

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@anjannath anjannath Awaiting requested review from anjannath

@praveenkumar praveenkumar Awaiting requested review from praveenkumar

Assignees

@anjannath anjannath

Labels
lgtm needs-rebase
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Access guest console when it is running
4 participants
@cfergeau @anjannath @praveenkumar @openshift-merge-robot