Refactor Customize IBus

[HollowMan6]

Project Title: Refactor Customize IBus

Description: Customize IBus is a GNOME extension that aims to fully customize the appearance, behavior, system tray and input source indicator for IBus. It was a GSoC 2021 project and has 100+ downloads on average per week on the official GNOME Shell Extensions store.

As time goes by, we need help to refactor this plugin so that it can be easier to maintain in the long run. A to-do list is as follows:

1. Migrate the plugin's preference panel from GTK to Adw. (GUI port to Adw (libadwaita) for GNOME 42 Customize-IBus#17)
2. Improve the coding style and separate the code into different files.
3. Clean up the codebase and remove the code for supporting the older versions of GNOME (Plugins that support GNOME 45 are no longer backwards compatible).
4. (Optional) Improve the input source indicator feature.
5. (Optional) Use your imagination and contribute new features to the plugin.
6. (Optional) You can also define your tasks that can help improve Customize IBus in your proposal.

Deliverable: Separate PRs for each task in the to-do list. After GSoC'24 finishes, it would be great if you have the ambition to continue maintaining the plugin for each new release of GNOME Shell.

Mentor: @HollowMan6 ([email protected])

Skills:

• GNOME Javascript (GJS), GTK, Adw
• Understanding how IBus works in GNOME-Shell (ibusCandidatePopup.js)
• Having experience in maintaining/creating GNOME-Shell extensions is preferred

Skill Level: Medium

Project Size: Large Sized Project (350 hours)

Get started:

1. Try Customize IBus yourself in GNOME.
2. Check all the links in the skills section.

[ddemaio]

@HollowMan6 I'll add you do the list of emails I'm sending out to mentors. You'll have one or two backups I imagine.

[jasisboredsojoinedgithub]

Hi @HollowMan6,

My name is Jiamin Wang, and I am a graduate student specializing in software engineering at SIT. I have reviewed all the materials in the skills section and gained a basic understanding of how IBus and Customize IBus functions within GNOME Shell. Regarding the tasks mentioned, I am comfortable working with migrate the current Preference Panel UI from its existing framework to a new one. Additionally, I am interested in refactoring extension.js breaking it down into smaller modules and removing codes for backward support.

I also want to propose a new feature that automatically switches the input source based on the URL or application context, inspired by services like https://inputsource.pro/. Could we explore the possibility of this feature? I would greatly appreciate your guidance and feedback as I develop my proposal.

Thank you for your time and consideration. I look forward to the possibility of collaborating on this project.

Best regards,
Jiamin Wang

[HollowMan6]

Hi @jasisboredsojoinedgithub! Thank you for your interest and great to hear that! Let me know if you need other help to develop your proposal. When you are ready, feel free to send me your draft proposal either by posting the link here or via email, and I can help review it!

About the new feature you proposed, we already have the input mode auto-switch (e.g. English and Chinese mode switch in ibus-libpinyin, just like you pressed SHIFT) based on the application (enable Auto switch ASCII mode and set Remember Input State) https://github.com/openSUSE/Customize-IBus/blob/main/GUIDE.md#general I'm not sure about the feasibility for doing that based on the URL (I'm not aware of any API for doing that), but it can be a great addition if you successfully have URL based input mode auto-switch implemented.

If the new feature is about input source switch (e.g. German keyboard and US keyboard), GNOME already has that feature implemented based on windows, I guess any improvement can go to GNOME Settings App directly if they support this change (based on the URL as well), but anyway, it's still good to have this:

Looking forward to seeing your proposal soon!

[thefool76]

Hi @HollowMan6,
I hope this message finds you well. My name is Bhavesh, and I'm thrilled at the opportunity to contribute to the Customize IBus project for GSoC '24. Having extensively explored the extension in GNOME, I'm keen on leveraging my skills to refactor and enhance its functionality for better long-term maintenance.

Upon reviewing the to-do list for the project, I'm confident in my ability to tackle each task effectively. I plan to start by migrating the plugin's preference panel from GTK to Adw, a crucial step towards modernizing its user interface and ensuring compatibility with GNOME 42. I'll ensure that the codebase is cleanly structured and adhere to best coding practices, thereby improving readability and maintainability. Additionally, I'll remove any deprecated code and streamline the extension's compatibility with newer versions of GNOME, focusing on GNOME 45 onwards.

Moreover, I'd like to propose the inclusion of two optional features during the refactoring process. Firstly, I believe integrating theme support would greatly enrich the user experience by allowing users to customize the appearance of the extension according to their preferences. Secondly, implementing clipboard integration would enhance usability, enabling seamless copy-paste functionality between IBus input and other applications.

I'm eager to collaborate with you and the community to realize these goals and contribute towards making Customize IBus a robust and user-friendly extension. I'm committed to delivering high-quality code and ensuring that the extension remains up-to-date with future releases of GNOME Shell.

Looking forward to your feedback and guidance as we embark on this exciting journey together!

Warm regards,
Bhavesh

[HollowMan6]

Hi @thefool76!

a crucial step towards modernizing its user interface and ensuring compatibility with GNOME 42

One correction here, we hope that your work is compatible with GNOME 45 (ideally also for the coming GNOME 46) since plugins that support GNOME 45 are no longer backward-compatible. Although I mentioned in the "GUI port to Adw (libadwaita)" issue to support GNOME 42, that was just some outdated information. Sorry if that confused you.

About the optional features you are proposing:

Firstly, I believe integrating theme support would greatly enrich the user experience by allowing users to customize the appearance of the extension according to their preferences.

I'm not sure what theme support you are going to add further for IBus / into this plugin. We already have the theme customization support in the plugin https://github.com/openSUSE/Customize-IBus/blob/main/GUIDE.md#theme My GSoC '21 main focus was to integrate theme support as well: https://summerofcode.withgoogle.com/archive/2021/projects/6295506795364352 It will be great if you can explain this with details in your proposal.

Secondly, implementing clipboard integration would enhance usability, enabling seamless copy-paste functionality between IBus input and other applications.

Not sure what you are trying to achieve by mentioning "clipboard integration", is that something similar to searching the "Clipboard history" feature that is available in ibus-tweaker https://github.com/tuberry/ibus-tweaker?tab=readme-ov-file#features? If so, one requirement from me to get that integrated into customize-ibus is that ideally we should have no third-party dependency for that (no python-pinyin)

Again, let me know if you need other help to develop your proposal. When you are ready, feel free to send me your draft proposal either by posting the link here or via email, and I can help review it! Looking forward to seeing your proposal soon, too!

[thefool76]

hey @HollowMan6
Firstly, I want to express my gratitude for your prompt response and clarification regarding the compatibility requirements for GNOME versions 45 and above. I appreciate the correction and will ensure that my work aligns with the latest standards. I have been using customize IBus extension for few days. I wanted to bring to your attention a couple of issues I encountered while using the Customize IBus extension. I believe addressing these bugs will significantly improve the user experience and contribute to the overall refinement of the extension.

• The first issue pertains to a bug I noticed within the IBus extension manager. Upon clicking on tabs within the manager, I observed that the window automatically scrolls down, which disrupts the user's navigation experience. I have provided a video demonstration for clarity

Window.bug.mp4

• The Second bug, I encountered difficulty copying emojis from the tray after selecting them. Despite referring to the Customize IBus user guide, I couldn't find a solution to this problem. Here's a video demonstration showcasing the issue

emoji.mp4

Linux distro I used
Ubuntu 20.04
Gnome version 42

Your guidance on resolving these issues would be greatly appreciated. Additionally, if there are specific steps or troubleshooting methods you recommend, I'm eager to implement them.

Thank you for your attention to these matters, and I look forward to your feedback and guidance.

[HollowMan6]

Hey @thefool76! Great to learn that you have started exploring and noticed some issues!

• The first issue pertains to a bug I noticed within the IBus extension manager. Upon clicking on tabs within the manager, I observed that the window automatically scrolls down, which disrupts the user's navigation experience. I have provided a video demonstration for clarity

Yes, this is exactly the "scrolling issue" I mentioned in #17, hopefully porting to Adw can get this fixed somehow.

• The Second bug, I encountered difficulty copying emojis from the tray after selecting them. Despite referring to the Customize IBus user guide, I couldn't find a solution to this problem. Here's a video demonstration showcasing the issue

This is interesting, it works well in GNOME 45 with IBus version v1.5.29 on my side. This should not be caused by "Customize IBus", since that dialog is triggered directly by spawning ibus emoji

https://github.com/openSUSE/Customize-IBus/blob/68abab1bbc8450072f618d1fdefe0aaba0bc77d4/customize-ibus%40hollowman.ml/extension.js#L2919

You might want to confirm your ibus version and see if it's the latest one (ibus version). Also check if you have the same issue by executing ibus emoji directly with CLI. If you find anything that needs to get fixed, you might want to send a PR to https://github.com/ibus/ibus.

[thefool76]

Thank you for your continued support @HollowMan6 I followed your suggestion and tried executing ibus emoji directly from the terminal. While I did receive the message indicating that the emoji was copied to the clipboard, I encountered the same issue when attempting to paste it. Despite the notification, the emoji wasn't successfully pasted into the desired application.

Ibus version v1.5.26 installed it via Gnome shell extension.
looks like I need to update it then.

[jasisboredsojoinedgithub]

Hi @HollowMan6,

I hope this message finds you well. Sorry for the delay in submitting my proposal draft. I was too busy with my midterm exams over the last two weeks.

I have managed to prepare a draft of my proposal and have added you as an editor for your feedback. Here is the link to the document: https://docs.google.com/document/d/1ZMLcEOHoKjDYtn_G2903VDJzyjdYz35G5WygEzWQv_8/edit?usp=sharing. I would really appreciate it if you could review my proposal and provide any suggestions or feedback. It is still a draft version, and I haven’t done any formatting yet. I will address the formatting issues and finalize the document after incorporating with your feedback before the final submission.

Best,
Jiamin

[HollowMan6]

Here is one obvious improvement for the input source indicator feature: In some applications (e.g. Google Chrome), the indicator doesn’t always follow the cursor when you focus by mouse

Screencast.from.2024-03-26.21-22-08.webm

While it works perfectly fine in other applications such as GNOME Text Editor:

Screencast.from.2024-03-26.21-23-12.webm

It would be great if you have any idea to implement this fix/enhancement on those applications such as Google Chrome.

I also remember we have several other bugs here and there, if you find them, do mention them even if you haven't come up with an immediate solution.

[jasisboredsojoinedgithub]

Understood! I'll investigate the source of the issue and incorporate a solution into the proposal. I'll also make adjustments based on your feedback. Thank you for your guidance!

[thefool76]

Here is one obvious improvement for the input source indicator feature: In some applications (e.g. Google Chrome), the indicator doesn’t always follow the cursor when you focus by mouse
Screencast.from.2024-03-26.21-22-08.webm

While it works perfectly fine in other applications such as GNOME Text Editor:
Screencast.from.2024-03-26.21-23-12.webm

It would be great if you have any idea to implement this fix/enhancement on those applications such as Google Chrome.

I also remember we have several other bugs here and there, if you find them, do mention them even if you haven't come up with an immediate solution.

Hello folks! After conducting thorough research, including reviewing discussions on the Ubuntu and Arch Linux forums, I attempted the troubleshooting steps outlined in the provided resources. This included launching Chromium with the --gtk-version=4 flag and exploring the possibility of enabling Wayland support via chrome://flags, as suggested. Unfortunately, these efforts did not resolve the issue.
Further investigation led me to discover that while Chrome does support Wayland, it still relies on GTK3, which could be the underlying cause of the compatibility issue with the IBus extension. Moreover, attempting to switch to Wayland via chrome://flags resulted in instability, making it an impractical workaround.Regrettably, after testing IBus on Chrome, Chromium, and Brave browsers, I can confirm that the extension does not function as expected. It appears that we must await official support for GTK4 on Chrome and Chrome-based browsers to address this compatibility issue effectively.

As part of ongoing efforts to resolve this issue, I plan to test IBus on other Electron-based applications to assess its functionality on it and will update you all.

Links for the forums
Ubuntu forum
wayland support
Arch Forum

[HollowMan6]

Thank you for your research! I don't think it's related to the GTK version (3/4), since I remember this issue existed already in GNOME 3.36, back when I started to develop this plugin during GSoC 2021, and at that time GNOME was still using GTK 3. In addition, actually, I was reproducing this under X11, so it's not something specific to Wayland.

The links to the forum you mentioned are mostly about solving the IBus typing issue (can't type at all), while here the typing works just fine. My rough guess for the cause here is that, maybe we shouldn't rely on the API here at all / introduce other API as an addition

https://github.com/openSUSE/Customize-IBus/blob/68abab1bbc8450072f618d1fdefe0aaba0bc77d4/customize-ibus%40hollowman.ml/extension.js#L1371-L1391

Or you might want to implement some changes for IBus/GNOME Shell so that this becomes consistent for those apps again.

[jasisboredsojoinedgithub]

Hi @HollowMan6!

After a quick research, I believe the problem is either the "set-cursor-location" or "set-cursor-location-relative"function from the IBus.panelservice API, which you pointed out earlier, or the get_compositor_private method at here.
Either way, I have come two plans for the fix:

1. use get_frame_rect on top of the window object we received from the get_compositor_private method to get the real position of the window's edge(window obejct and get_frame_rect).;
2. instead of using IBus.panelservice, we can use Meta API directly get_pointer.

I am still trying to test this possibility, because my parallel desktop ubuntu seems unable to install a google chrome (I'll modify my clone and test on chrome in a few days). Please let me know your thoughts on the potential solutions above.

[HollowMan6]

That sounds promising! I haven't played with those 2 APIs, and for the proposal phase, since the deadline is approaching, you don't need to verify the specific APIs and see if it's working or not, just propose and let's leave the actual work to the implementation phase. You can focus solely on your proposal now and what we really would like to see is your ability in problem identifying and debugging, since the main part of this idea is about refactoring, and it's crucial to have the right debugging skills, since refactoring can easily cause something to break or regress. So, you can document your research process in detail in your proposal, like what things have you done for the research, how exactly you are sure that the problem lies in those parts, and that will be a great plus:

After a quick research, I believe the problem is either the "set-cursor-location" or "set-cursor-location-relative"function from the IBus.panelservice API, which you pointed out earlier, or the get_compositor_private method at here.

Google Chrome is not a necessity to reproduce this bug, at least I know that the LibreOffice Writer also has this bug. You can try what's available in your virtual machine and find one app that has this bug, then starts there.

By the way, one trick for debugging in case you don't know, you can see all the logs of GNOME Shell (including extensions) from "journalctl -f /usr/bin/gnome-shell".

[jasisboredsojoinedgithub]

Hi @HollowMan6:

Here is the update of Task 4: to improve the input source indicator.

I followed your tip and used "journalctl -f /usr/bin/gnome-shell" to see all the logs. And I found that the set-cursor-location-relative event is not being triggered when updating the cursor's position in chrome. Now we can confirm the problem is is with the API set-cursor-location. I have documented these findings and outlined the debugging proccess in the draft proposal for Task 4, which you can review here.

I will be refining the proposal and formatting the final document using LaTeX. I anticipate having everything ready for submission with the deadline. Thank you for your guidance and support!

Best,
Jiamin