Add Wayland docking support

[mliberty1]

Thanks for your great work on Qt-ADS! I have been using it in the Joulescope UI for many years, and it has been excellent. I have a number of customers affected by my UI issue #316, poor Wayland support. I tried to get someone on Upwork to fix this, but failed. Today, I worked with Claude Code / Fable to get this done.

This PR is a self-contained fix for Wayland. It's hopefully relatively easy to accept since most changes are gated to only affect Wayland. It does touch a bunch of files, though. The addition of GlobalPos parameter to a few methods should be a flow-through no-change on other platforms.

The overall experience is functional, but definitely not as slick as Windows and macOS. However, functional is an improvement and good enough to address my customers' concerns about slow performance using xcb.

Curated AI-generated summary follows:

Summary

This PR adds support for docking (undocking, moving, and re-docking floating widgets) on Wayland. It implements the approach outlined in #714 and follows the same mechanism that Qt itself uses for QDockWidget/QToolBar on Wayland.

Wayland forbids the techniques the docking system relies on elsewhere: a client cannot move its own top level windows in screen coordinates, cannot query the global cursor position, and cannot position one top level over another. Instead of the mouse tracked drag preview, undocking now performs a compositor driven drag:

• Floating containers always use native window decorations on Wayland, so the compositor moves the window (the custom QWidget title bar cannot).
• Undocking executes a QDrag carrying the MIME types that QWaylandDataDevice translates into an xdg_toplevel_drag_v1 request, so the real floating window follows the cursor (no translucent preview).
• CDockContainerWidget accepts drops and drives the drop overlays from the drag-and-drop event positions instead of QCursor::pos(), which is stale during a drag on Wayland.
• Dragging the tab or dock-area title bar of a floating widget drags the existing floating window, so it can be docked back into another container.

Platform impact

Every change is guarded by a new ads::internal::isWayland() helper, so X11, Windows and macOS are unaffected. Where a function is shared (e.g. the drop overlay setup, now factored into CDockContainerWidget::showDropOverlays()), the behavior is preserved on the other platforms. I verified the X11 mouse-drag path still works unchanged.

This change is self-contained and does not depend on any other PR. On the non-Wayland paths the existing top-level Qt::Tool overlays are kept exactly as before; only on Wayland are the overlays child widgets.

Requirements and behavioral notes (also updated in README)

• Requires Qt 6.6.3+ for the xdg_toplevel_drag_v1 support; on older Qt the code is harmless but the window will not follow the cursor, so X11/XWayland is still recommended there.
• On Wayland: native decorations only (no custom title bar), the real window is dragged instead of a translucent preview, a floating window takes the dock manager's stylesheet when it is created (a later application-wide stylesheet change does not update an already-floating window), and the compositor owns window stacking (no forced stays-on-top).

Testing

Tested on GNOME / Mutter 50.1 with Qt 6.11 (PySide6): undock, move via the decoration, drop on an indicator to re-dock, drop outside to keep floating, re-dock a floating window via its title bar, floating-to-floating drop, and auto-hide undock. The X11/xcb path was regression tested and is unchanged.

Other compositors are not yet covered; on KDE/KWin a benign Qt warning (This plugin supports grabbing the mouse only for popup windows) is emitted by Qt's own drag-and-drop internals and does not block docking.

References

• Addresses Support floating docks on Wayland #714 (Support floating docks on Wayland).
• Wayland: render drag preview and overlays as child widgets instead of top-level Qt::Tool #837 proposes rendering the drag preview and overlays as child widgets on all platforms. This PR is independent of it: it keeps the existing top-level overlays on X11/Windows/macOS and only uses child-widget overlays on Wayland.
• fix docking drag drop problem on Linux wayland #789 is an earlier workaround attempt for the same problem.

[githubuser0xFFFF]

Thank you very much for your comprehensive documentation. Could you please tell me, in which way your merge request is related to this one: #837

Do I need to merge both or is your merge request a more comlpete replacement?

Thank you.

[mliberty1]

This PR is related to #837, but they are different. This PR allows undocked windows to be separate from the original window. You can then move widgets between windows to reparent them. It allows Qt ADS to work correctly without crashes or corruption while allowing undocked window resizing. It does somewhat degrade the Qt-ADS experience, but you can accomplish all widget positioning you may want. This PR is also written to only affect Wayland code and deliberately avoid touching code for other platforms.

PR 837 improves the drag & drop widget experience within the original window on Wayland. It does not fix the multiple window experience.

Here's a video attempting to show the differences:
http://download.joulescope.com/other/qt_ads_wayland.mp4
Sorry about the low audio volume!

After playing with this some more, I am working on seeing if we can keep the existing widget drag behavior within the window and only activate this 844 behavior when we need to create a new window.

[mliberty1]

I just pushed an additional commit that effectively combines 837 into 844. The result is much nicer, and Wayland feels like a first-class platform for Qt ADS! The PR is implemented in a way that only affects Wayland. Other platforms should be functionally unchanged. Here is a demo:

http://download.joulescope.com/other/qt_ads_wayland_02.mp4

I am happy to squash and force commit if you want a single commit to review and merge. Let me know!

---

Detailed docs on this second commit:

Summary

This PR adds Wayland support to the docking system. On Wayland a client cannot reposition top-level windows and QCursor::pos() is unreliable, so the normal mouse-tracked drag of a floating window does not work. #844 addresses this by driving cross-window dock drags through the compositor's xdg_toplevel_drag_v1 protocol (CFloatingDockContainer::startPlatformDrag()), with the drop overlays rendered as child widgets so they align correctly on Wayland.

Latest commit: keep the in-window drag preview, go native only when leaving the window

The original approach converted every dock drag into a native platform drag (a new top-level window) the moment the drag threshold was crossed, which replaced the familiar in-window rearrange UX. The latest commit makes the behavior hybrid: while the cursor stays inside the source top-level window, the drag uses the normal in-window CFloatingDragPreview + drop overlays (no new window); only when the cursor leaves the window does it tear down the preview, create a real CFloatingDockContainer, and hand it to startPlatformDrag() — all in one continuous gesture, with the original press's implicit pointer grab still held. This is wired for both the tab (CDockWidgetTab) and title-bar (CDockAreaTitleBar) drag paths, and applies equally to rearranging widgets inside an already-floating window.

On Wayland the in-window preview is rendered as a child of the source window and driven by the reliable event-supplied position from the grabbing widget (not QCursor::pos()), with drop-target hit-testing confined to the source container.

One important protocol detail: xdg_toplevel_drag_v1 only honors the attach offset that places the new window under the cursor while the toplevel is still unmapped. show() creates the xdg_toplevel role but the surface only maps once the event loop is pumped, so startPlatformDrag() deliberately does not wait for exposure before QDrag::exec() — waiting would map the window first and the compositor would then ignore the offset and place the window at its default position (screen center).

Impact on other platforms

None. Every behavioral change is gated on internal::isWayland(). On X11/Windows/macOS the preview stays a Qt::Tool top-level using the existing QCursor::pos()-driven path; the shared-code refactors (a moveFloating(QPoint) overload, explicit-position overlay calls, the finishDragging position source) are behavior-preserving off Wayland (the no-arg overlay methods already delegate with QCursor::pos(), and the event-position stash is only set on Wayland), and startPlatformDrag() is only ever invoked from Wayland-gated paths.

Relationship to #837

#844 and #837 solve the same Wayland problem in overlapping code, and with the latest commit #844 now fully absorbs #837. They remain co-mergeable (a merge produces two resolvable, same-intent conflicts in DockOverlay.cpp and FloatingDragPreview.cpp), but #837 no longer adds anything of substance on top of #844.

#837's real footprint is three files (~48 insertions / 38 deletions):

#837 change	Status in #844
DockOverlay.cpp — overlays/cross as child widgets on Wayland; devicePixelRatioF() fix	Superseded — #844 implements the same, isWayland()-gated, and keeps the X11 Qt::Tool path
FloatingDragPreview.cpp — preview as a child widget on Wayland	Superseded — the latest #844 commit implements this, gated, plus event-coordinate tracking
setWindowOpacity(1) removal in CDockOverlay ctor	Moot — in #844 that call already lives in the non-Wayland else branch, so it never runs on Wayland
setWindowOpacity(1) removal in CFloatingDockContainer::finishDragging	Near-moot — that path isn't reached in #844's Wayland drag flow; on X11 it sets opacity to its default (1.0), a no-op

In short: the only lines in #837 not already present in #844 are two setWindowOpacity(1) removals, both of which #844 already neutralizes via gating. Adopting #844 means #837 can be dropped without losing any behavior.

[Wing-summer]

Thanks for your hard working on wayland support on linux wayland. This solution is almost fantastic and I think I can close my PR, tested on my KDE wayland.

I find a minor bug: If you drag a docking window out of the main container and make it finally a floating window, sometime you cannot drag another docking window onto the recent newly created floating window.

Another workon maybe a wayland limitation: You can not actually control the position of floating window when it transforms from docking window just now. It's wired when you drag a window out but it has a large distance with you cursor.

[mliberty1]

Hi @Wing-summer - It is working really well for me on Ubuntu 26.04 LTS.

A few things:

1. Can you provide a picture or video of the issues you see? You might be doing something in a way that I have not tried.
2. What Linux distro do you use? I have a spare computer that I can use to try to duplicate what you see.

[mliberty1]

I find a minor bug: If you drag a docking window out of the main container and make it finally a floating window, > sometime you cannot drag another docking window onto the recent newly created floating window.

1. One known issue is that dragging the Window title bar does not drag and drop widgets. You need to drag the Widget tab or Widget title area. The area in the red rectangle does not support drag and drop:

Is this what you mean?

Another workon maybe a wayland limitation: You can not actually control the position of floating window when it transforms from docking window just now. It's wired when you drag a window out but it has a large distance with you cursor.

2. The widget maintains it's size when it transforms into a docking window. I am not quite sure how the existing code determines the location of the widget relative to your cursor. I suppose it should keep the same widget location where you started the drag under the cursor. I will investigate.

3. I noticed another issue. Arrange things so that you have a main window and a docking window, each with a few widgets. When you drag a widget in the docking window, the preview shows up on the main window. It should be on the docking window. I will investigate.

[Wing-summer]

Hi @Wing-summer - It is working really well for me on Ubuntu 26.04 LTS.

A few things:

1. Can you provide a picture or video of the issues you see? You might be doing something in a way that I have not tried.
2. What Linux distro do you use? I have a spare computer that I can use to try to duplicate what you see.

1. There is a video following to reproduce the bug:
   bug.webm
2. I use CachyOS with KDE Plasma 6.6.5.

[Wing-summer]

I find a minor bug: If you drag a docking window out of the main container and make it finally a floating window, > sometime you cannot drag another docking window onto the recent newly created floating window.

1. One known issue is that dragging the Window title bar does not drag and drop widgets. You need to drag the Widget tab or Widget title area. The area in the red rectangle does not support drag and drop:

Is this what you mean?

Another workon maybe a wayland limitation: You can not actually control the position of floating window when it transforms from docking window just now. It's wired when you drag a window out but it has a large distance with you cursor.

2. The widget maintains it's size when it transforms into a docking window. I am not quite sure how the existing code determines the location of the widget relative to your cursor. I suppose it should keep the same widget location where you started the drag under the cursor. I will investigate.
3. I noticed another issue. Arrange things so that you have a main window and a docking window, each with a few widgets. When you drag a widget in the docking window, the preview shows up on the main window. It should be on the docking window. I will investigate.

1. Not yet, I've uploaded a video. This problem is wayland limitation, We don't need to focus on it.
2. I think it's wm behavior. KDE Plasma can restore the window position and size now. But it's far-fetched to think it were the reason.
3. I noticed too recently. But somehow I think it's not a very big problem.

[mliberty1]

I just did a force push, which further improves this PR.

1. Thanks for the video @Wing-summer ! This is the expected behavior, and I am not sure how we would fix it. The widget stays internal to the Window as long as the cursor stays inside. As soon as you drag the widget out of the Window, it gets its own Dock Window. Alternatively, if you release the drag like you did on a non-docking region, it also gets it's own Dock Window. Therefore, when you dragged the widget to the Dock Window situated over the Main Window, the widget stays in the main Window. I can't duplicate this behavior on Ubuntu since the Dock Widget ends up behind the Main Window when I click on the widget.
2. Fixed! The cursor now stays over the same location on the widget where you started the drag. It may be off by a few pixels due to window styling, but it is very close.
3. Fixed. The preview now goes to the widget's current window.
4. Fixed stylesheet cascading, which was previously partially implemented.
5. Confirmed that Window position restore is not possible on Wayland and updated README.
6. Fixed use-after-free crash.

Overall, this PR makes Wayland very well supported by Qt ADS!

[githubuser0xFFFF]

@mliberty1 Thank your very much for your fantastic work. This is a really massive improvement for Linux users. @Wing-summer Thank you very much for reviewing the merge request,

[mliberty1]

@githubuser0xFFFF - Thanks for all your work on this great project, being so responsive, and merging this PR so quickly! I think a bunch of ADS users have struggled using xcb (X11 emulation). My customers will be happy that the Joulescope UI will soon have proper Wayland support!

[githubuser0xFFFF]

@mliberty1 Thank you again for your outstanding contribution. Today I released ADS 5.0.0 with all the contributed new features. I highlighted your contribution in the New and Noteworthy section and added a showcase section for your Joulescope application.

[mliberty1]

Excellent, and thank you for your work on this project, @githubuser0xFFFF !

[ciros88]

there is a reason why isWayland() function is based on

return QGuiApplication::platformName().startsWith(QLatin1String("wayland"), Qt::CaseInsensitive);

instead of something like

return qEnvironmentVariable("XDG_SESSION_TYPE") == "wayland";

?

anyway thanks for you great work