Add native TrayMenuController implementation for macOS

[oc7o]

What

Adds a native macOS implementation of the TrayMenuController SPI, replacing the AWT-based tray menu.

Why

Initially intended as a bug fix for this issue: The AWT tray menu has long-standing rendering and behavior issues on macOS.
When clicking on a submenu the whole tray menu closes which is unintended.
Endend up in creating a whole native AppKit-backed menu instead which enables us to fix this.

How

• MacTrayMenuController (Java) implements the TrayMenuController service and is registered via module-info (provides + opens).
• A JNI backend (org_cryptomator_..._Native.m) drives NSStatusItem and NSMenu directly, supporting:
  • action items, submenus, and separators
  • tray icon set/update (template image)
  • a "before open" menu listener
• Native source added to the Xcode project build.

Notes

• Just to make it clear: macOS only; no behavior change on other platforms.

[CLAassistant]

All committers have signed the CLA.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 78c77813-6310-4cc0-b10e-f9cc790be4c2

📥 Commits

Reviewing files that changed from the base of the PR and between 0b46807 and cd9c435.

📒 Files selected for processing (3)

• src/main/java/module-info.java
• src/main/java/org/cryptomator/macos/tray/MacTrayMenuController.java
• src/main/native/org_cryptomator_macos_tray_MacTrayMenuController_Native.m

💤 Files with no reviewable changes (2)

• src/main/java/module-info.java
• src/main/java/org/cryptomator/macos/tray/MacTrayMenuController.java

🚧 Files skipped from review as they are similar to previous changes (1)

• src/main/native/org_cryptomator_macos_tray_MacTrayMenuController_Native.m

---

Walkthrough

This PR implements macOS tray menu functionality for Cryptomator integrations. It registers MacTrayMenuController as a service provider in module-info.java, adds a Java MacTrayMenuController that converts icons to PNG bytes and delegates tray/menu operations to native methods, adds a machine-generated JNI header declaring seven native methods and ROOT_MENU, implements those methods in Objective‑C (image conversion, status item and menu management, action handlers, and before-open listener), updates the Xcode project to compile the new native source, and updates .gitignore to ignore .DS_Store.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and specifically describes the main change: adding a native macOS implementation of TrayMenuController to replace the AWT-based version.
Description check	✅ Passed	The description is directly related to the changeset, clearly explaining what was added (native macOS TrayMenuController), why (to fix AWT rendering issues), and how (via JNI backend with NSStatusItem/NSMenu).
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/main/java/org/cryptomator/macos/tray/MacTrayMenuController.java`:
- Around line 49-53: The loadPng method can return null if the provided
iconLoader never invokes the TrayIconLoader.PngData consumer; after calling
iconLoader.accept(...) check holder[0] and if it is null return an empty byte[]
(or otherwise non-null sentinel) instead of null to avoid passing a null array
to native code that calls GetArrayLength; update loadPng (and its single-use
holder/consumer logic) to ensure it always returns a non-null byte[].
- Around line 29-31: The native implementation drops the Java Runnable passed
from MacTrayMenuController.showTrayIcon (Native.showTrayIcon) so clicks do
nothing; update
Java_org_cryptomator_macos_tray_MacTrayMenuController_00024Native_showTrayIcon
to store the passed defaultAction (Runnable) in a global/reference table, set
gStatusItem.button.target and action to a native selector that, when invoked on
click, looks up that stored Runnable and calls its run method via JNI; ensure
you retain a global JNI globalRef to the Runnable and release it when removing
the tray icon or on shutdown, and use the existing gStatusItem.button,
Java_org_cryptomator_macos_tray_MacTrayMenuController_00024Native_showTrayIcon
symbol names to locate where to add the storage, selector hookup, and JNI
invocation.

In `@src/main/native/org_cryptomator_macos_tray_MacTrayMenuController_Native.m`:
- Around line 89-114: The native showTrayIcon currently ignores the
defaultAction jobject; capture and store it as a JNI global reference (e.g.,
create a static jobject gDefaultAction via (*env)->NewGlobalRef(env,
defaultAction)) so it can be invoked later, and ensure you ReleaseGlobalRef when
replacing/removing the tray (cleanup). Also wire gStatusItem.button.action and
.target to an Objective‑C selector (e.g., -handleDefaultAction:) implemented in
the same file that obtains a JNIEnv, finds the Java callback method (or
Runnable.run) and calls it on gDefaultAction; keep gRootMenu and gMenuDelegate
behavior but change menu presentation so left-click triggers the selector
(invoke defaultAction) and show the menu manually (popUpMenu:forEvent: or on
right/Ctrl-click) if you want both menu and defaultAction to work.
- Around line 22-27: callRunnable currently invokes Runnable.run via
CallVoidMethod but never checks for a pending Java exception; update
callRunnable to, immediately after CallVoidMethod, check for exceptions with
ExceptionCheck/ExceptionOccurred, clear the pending exception, rethrow the same
exception back into the JVM with Throw (and delete any local refs), so that a
Java exception from Runnable.run is not left pending and is propagated
correctly.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 5a07322f-1f0f-4204-97eb-ef3ef09071e0

📥 Commits

Reviewing files that changed from the base of the PR and between d560150 and 5e1b580.

📒 Files selected for processing (6)

• .gitignore
• src/main/headers/org_cryptomator_macos_tray_MacTrayMenuController_Native.h
• src/main/java/module-info.java
• src/main/java/org/cryptomator/macos/tray/MacTrayMenuController.java
• src/main/native/Integrations.xcodeproj/project.pbxproj
• src/main/native/org_cryptomator_macos_tray_MacTrayMenuController_Native.m

[oc7o]

@overheadhunter
Are the fixes alright?