Skip to content

feat: Support ContactRect in X11 and Windows platform #16498

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Oct 6, 2024
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/Avalonia.Base/Input/PenDevice.cs
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ private void ProcessRawEvent(RawPointerEventArgs e)
}

var props = new PointerPointProperties(e.InputModifiers, e.Type.ToUpdateKind(),
e.Point.Twist, e.Point.Pressure, e.Point.XTilt, e.Point.YTilt);
e.Point.Twist, e.Point.Pressure, e.Point.XTilt, e.Point.YTilt, e.Point.ContactRect);
var keyModifiers = e.InputModifiers.ToKeyModifiers();

bool shouldReleasePointer = false;
Expand Down
10 changes: 8 additions & 2 deletions src/Avalonia.Base/Input/PointerPoint.cs
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,11 @@ public PointerPoint(IPointer pointer, Point position, PointerPointProperties pro
/// </summary>
public record struct PointerPointProperties
{
/// <summary>
/// Gets the bounding rectangle of the contact area (typically from touch input).
/// </summary>
public Rect ContactRect { get; }

/// <summary>
/// Gets a value that indicates whether the pointer input was triggered by the primary action mode of an input device.
/// </summary>
Expand Down Expand Up @@ -155,17 +160,18 @@ public PointerPointProperties(RawInputModifiers modifiers, PointerUpdateKind kin
}

public PointerPointProperties(RawInputModifiers modifiers, PointerUpdateKind kind,
float twist, float pressure, float xTilt, float yTilt
float twist, float pressure, float xTilt, float yTilt, Rect contactRect
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to extend the API? Can't relevant XI2 properties be converted to existing ones? How other platforms expose the same information? How browsers are handling that?

Copy link
Member

@kekekeks kekekeks Jul 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is the same info exposed by UWP?

Copy link
Contributor Author

@lindexi lindexi Jul 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kekekeks

Why do we need to extend the API?

As a touch application, the touch area is an indispensable parameter.

Can't relevant XI2 properties be converted to existing ones?

Sorry, I can not find any existing ones can be converted to.

How other platforms expose the same information?

It can be implemented in Windows platform, see https://learn.microsoft.com/en-us/windows-hardware/design/component-guidelines/touchscreen-required-hid-top-level-collections

How is the same info exposed by UWP?

See https://learn.microsoft.com/en-us/windows/windows-app-sdk/api/winrt/microsoft.ui.input.pointerpointproperties.contactrect?view=windows-app-sdk-1.5

And we can get this info from WM_Pointer.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, we can get the touch size in wpf by StylusPointProperties.Width and StylusPointProperties.Height

Copy link
Contributor Author

@lindexi lindexi Aug 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How other platforms expose the same information? How browsers are handling that?

I support the windows platform now. And I test my demo, and I find the behavior of Avalonia is same as WinUI and UWP.

) : this (modifiers, kind)
{
Twist = twist;
Pressure = pressure;
XTilt = xTilt;
YTilt = yTilt;
ContactRect = contactRect;
}

internal PointerPointProperties(RawInputModifiers modifiers, PointerUpdateKind kind, RawPointerPoint rawPoint)
: this(modifiers, kind, rawPoint.Twist, rawPoint.Pressure, rawPoint.XTilt, rawPoint.YTilt)
: this(modifiers, kind, rawPoint.Twist, rawPoint.Pressure, rawPoint.XTilt, rawPoint.YTilt, rawPoint.ContactRect)
{
}

Expand Down
8 changes: 8 additions & 0 deletions src/Avalonia.Base/Input/Raw/RawPointerEventArgs.cs
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,14 @@ public record struct RawPointerPoint
/// <inheritdoc cref="PointerPointProperties.YTilt" />
public float YTilt { get; set; }

/// <inheritdoc cref="PointerPointProperties.ContactRect" />
public Rect ContactRect
{
get => _contactRect ?? new Rect(Position, new Size());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it should be new Size(1,1) by default I believe

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@maxkatz6 Sorry, I do not think so. I think the size should be zero here as the WinUI do.

set => _contactRect = value;
}

private Rect? _contactRect;

public RawPointerPoint()
{
Expand Down
14 changes: 14 additions & 0 deletions src/Avalonia.X11/Screens/X11Screens.cs
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,20 @@ public Screen ScreenFromPoint(PixelPoint point)
return ScreenHelper.ScreenFromPoint(point, AllScreens);
}

internal PixelRect? GetScreenBoundsFromPoint(PixelPoint point)
{
// Why not ScreenFromPoint? Because the AllScreens property will create some Screen instances which will cause the stress of GC. This method will be called in every touch event, so we need to avoid the GC stress.
foreach (var screen in _impl.Screens)
{
if (screen.Bounds.ContainsExclusive(point))
{
return screen.Bounds;
}
}

return null;
}

public Screen ScreenFromRect(PixelRect rect)
{
return ScreenHelper.ScreenFromRect(rect, AllScreens);
Expand Down
4 changes: 2 additions & 2 deletions src/Avalonia.X11/X11Platform.cs
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,9 @@ internal class AvaloniaX11Platform : IWindowingPlatform
{
private Lazy<KeyboardDevice> _keyboardDevice = new Lazy<KeyboardDevice>(() => new KeyboardDevice());
public KeyboardDevice KeyboardDevice => _keyboardDevice.Value;
public Dictionary<IntPtr, X11PlatformThreading.EventHandler> Windows =
public Dictionary<IntPtr, X11PlatformThreading.EventHandler> Windows { get; } =
new Dictionary<IntPtr, X11PlatformThreading.EventHandler>();
public XI2Manager XI2;
public XI2Manager XI2 { get; private set; }
public X11Info Info { get; private set; }
public X11Screens X11Screens { get; private set; }
public Compositor Compositor { get; private set; }
Expand Down
61 changes: 60 additions & 1 deletion src/Avalonia.X11/XI2Manager.cs
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using Avalonia.Input;
using Avalonia.Input.Raw;
using Avalonia.Platform;
using static Avalonia.X11.XLib;

namespace Avalonia.X11
Expand Down Expand Up @@ -97,13 +99,15 @@ public bool HasMotion(ParsedDeviceEvent ev)
private AvaloniaX11Platform _platform;

private XIValuatorClassInfo? _pressureXIValuatorClassInfo;
private XIValuatorClassInfo? _touchMajorXIValuatorClassInfo;
private XIValuatorClassInfo? _touchMinorXIValuatorClassInfo;

public bool Init(AvaloniaX11Platform platform)
{
_platform = platform;
_x11 = platform.Info;
_multitouch = platform.Options?.EnableMultiTouch ?? true;
var devices =(XIDeviceInfo*) XIQueryDevice(_x11.Display,
var devices = (XIDeviceInfo*) XIQueryDevice(_x11.Display,
(int)XiPredefinedDeviceId.XIAllMasterDevices, out int num);
for (var c = 0; c < num; c++)
{
Expand All @@ -118,8 +122,16 @@ public bool Init(AvaloniaX11Platform platform)

if (_multitouch)
{
// ABS_MT_TOUCH_MAJOR ABS_MT_TOUCH_MINOR
// https://www.kernel.org/doc/html/latest/input/multi-touch-protocol.html
var touchMajorAtom = XInternAtom(_x11.Display, "Abs MT Touch Major", false);
var touchMinorAtom = XInternAtom(_x11.Display, "Abs MT Touch Minor", false);

var pressureAtom = XInternAtom(_x11.Display, "Abs MT Pressure", false);
_pressureXIValuatorClassInfo = _pointerDevice.Valuators.FirstOrDefault(t => t.Label == pressureAtom);

_touchMajorXIValuatorClassInfo = _pointerDevice.Valuators.FirstOrDefault(t => t.Label == touchMajorAtom);
_touchMinorXIValuatorClassInfo = _pointerDevice.Valuators.FirstOrDefault(t => t.Label == touchMinorAtom);
}

/*
Expand Down Expand Up @@ -256,6 +268,51 @@ private void OnDeviceEvent(IXI2Client client, ParsedDeviceEvent ev)
}
}

if(_touchMajorXIValuatorClassInfo is {} touchMajorXIValuatorClassInfo)
{
double? touchMajor = null;
double? touchMinor = null;
PixelRect screenBounds = default;
if (ev.Valuators.TryGetValue(touchMajorXIValuatorClassInfo.Number, out var touchMajorValue))
{
var screenBoundsFromPoint = _platform.X11Screens.GetScreenBoundsFromPoint(new PixelPoint((int)ev.RootPosition.X, (int)ev.RootPosition.Y));
Debug.Assert(screenBoundsFromPoint != null);
if (screenBoundsFromPoint != null)
{
screenBounds = screenBoundsFromPoint.Value;

// As https://www.kernel.org/doc/html/latest/input/multi-touch-protocol.html says, using `screenBounds.Width` is not accurate enough.
touchMajor = (touchMajorValue - touchMajorXIValuatorClassInfo.Min) /
(touchMajorXIValuatorClassInfo.Max - touchMajorXIValuatorClassInfo.Min) * screenBounds.Width;
}
}

if (touchMajor != null)
{
if(_touchMinorXIValuatorClassInfo is {} touchMinorXIValuatorClassInfo)
{
if (ev.Valuators.TryGetValue(touchMinorXIValuatorClassInfo.Number, out var touchMinorValue))
{
touchMinor = (touchMinorValue - touchMinorXIValuatorClassInfo.Min) /
(touchMinorXIValuatorClassInfo.Max - touchMinorXIValuatorClassInfo.Min) * screenBounds.Height;
}
}

if (touchMinor == null)
{
touchMinor = touchMajor;
}

rawPointerPoint.ContactRect = new Rect
(
ev.Position.X,
ev.Position.Y,
touchMajor.Value,
touchMinor.Value
);
}
}

client.ScheduleXI2Input(new RawTouchEventArgs(client.TouchDevice,
ev.Timestamp, client.InputRoot, type, rawPointerPoint, ev.Modifiers, ev.Detail));
return;
Expand Down Expand Up @@ -340,6 +397,7 @@ internal unsafe class ParsedDeviceEvent
public RawInputModifiers Modifiers { get; }
public ulong Timestamp { get; }
public Point Position { get; }
public Point RootPosition { get; }
public int Button { get; set; }
public int Detail { get; set; }
public bool Emulated { get; set; }
Expand Down Expand Up @@ -385,6 +443,7 @@ public ParsedDeviceEvent(XIDeviceEvent* ev)

Valuators = new Dictionary<int, double>();
Position = new Point(ev->event_x, ev->event_y);
RootPosition = new Point(ev->root_x, ev->root_y);
var values = ev->valuators.Values;
if(ev->valuators.Mask != null)
for (var c = 0; c < ev->valuators.MaskLen * 8; c++)
Expand Down
Loading