thatrichan/revue-companion
thatrichan/revue-companion
Watch
1
0
Fork
You've already forked revue-companion
 0
Code Issues 5 Pull requests Projects Releases 7 Packages Wiki Activity Actions

JetBrains: run the sandbox UI tests headless, in Docker #7

New issue
Open
opened 2026-11-12 23:57:29 +01:00 by thatrichan · 2 comments
thatrichan commented 2026-11-12 23:57:29 +01:00
Owner
Copy link

The sandbox UI tests (task jetbrains:ui-test: IDE Starter + Driver walking a review in WebStorm 2026.3 against the fake revue, see jetbrains/README.md → Testing) need a real IDE window. On macOS a WebStorm window appears for each run (about 35 s); there's no virtual display to hide it in. The input already goes into the IDE's own event queue (jetbrains/src/uiTest/.../Events.kt), so a run never moves the mouse or types for you, and runs no longer raise the window or screenshot the whole screen. But a window still shows.

Ask

Run the same walk with nothing appearing on the computer: in a Linux container with a virtual display.

  • A Docker image with Xvfb, a JDK 25 and Linux WebStorm 2026.3 (platformVersion in jetbrains/gradle.properties); the arm64 Linux build on Apple silicon, so it doesn't run under x86 emulation.
  • The plugin, the fake revue (jetbrains/fake-revue) and the sample repository built as today, the walk run inside the container, screenshots and idea.log copied back to jetbrains/build/ui-screenshots/.
  • A Taskfile task (e.g. jetbrains:ui-test:docker), no shell scripts (CLAUDE.md); the IDE and Gradle caches kept in volumes so runs after the first are quick.
  • Works with Docker Desktop, OrbStack or Colima; says clearly when none is running.
  • Then CI can run it too: a step in the JetBrains workflow on every push touching jetbrains/.

Done when

  • task jetbrains:ui-test:docker passes on a Mac with no window appearing, and on Linux.
  • The screenshots look the same as a run on the desktop.
  • jetbrains/README.md → Testing says how to run it.
The sandbox UI tests (`task jetbrains:ui-test`: IDE Starter + Driver walking a review in WebStorm 2026.3 against the fake `revue`, see `jetbrains/README.md` → Testing) need a real IDE window. On macOS a WebStorm window appears for each run (about 35 s); there's no virtual display to hide it in. The input already goes into the IDE's own event queue (`jetbrains/src/uiTest/.../Events.kt`), so a run never moves the mouse or types for you, and runs no longer raise the window or screenshot the whole screen. But a window still shows. ## Ask Run the same walk with nothing appearing on the computer: in a Linux container with a virtual display. - A Docker image with Xvfb, a JDK 25 and **Linux WebStorm 2026.3** (`platformVersion` in `jetbrains/gradle.properties`); the arm64 Linux build on Apple silicon, so it doesn't run under x86 emulation. - The plugin, the fake `revue` (`jetbrains/fake-revue`) and the sample repository built as today, the walk run inside the container, screenshots and `idea.log` copied back to `jetbrains/build/ui-screenshots/`. - A Taskfile task (e.g. `jetbrains:ui-test:docker`), no shell scripts (CLAUDE.md); the IDE and Gradle caches kept in volumes so runs after the first are quick. - Works with Docker Desktop, OrbStack or Colima; says clearly when none is running. - Then CI can run it too: a step in the JetBrains workflow on every push touching `jetbrains/`. ## Done when - `task jetbrains:ui-test:docker` passes on a Mac with no window appearing, and on Linux. - The screenshots look the same as a run on the desktop. - `jetbrains/README.md` → Testing says how to run it.
thatrichan commented 2026-11-12 23:59:58 +01:00
Author
Owner
Copy link

Notes for when this is picked up:

  • Since 4eee36e the desktop runs already stay out of the way as far as macOS allows: no raising the window after launch, and screenshots painted by WebStorm itself (jetbrains/src/uiTest/.../Pictures.kt, Driver.paintFrame), so they don't depend on what's on screen. The container run can reuse that helper as is.
  • A Docker set up with colima's defaults (2 CPUs, about 2 GB of memory) is too small for Gradle plus WebStorm. The task should check the VM's resources and say what to raise them to (e.g. colima start --cpu 4 --memory 8), the same for Docker Desktop's and OrbStack's settings.
Notes for when this is picked up: - Since 4eee36e the desktop runs already stay out of the way as far as macOS allows: no raising the window after launch, and screenshots painted by WebStorm itself (`jetbrains/src/uiTest/.../Pictures.kt`, `Driver.paintFrame`), so they don't depend on what's on screen. The container run can reuse that helper as is. - A Docker set up with colima's defaults (2 CPUs, about 2 GB of memory) is too small for Gradle plus WebStorm. The task should check the VM's resources and say what to raise them to (e.g. `colima start --cpu 4 --memory 8`), the same for Docker Desktop's and OrbStack's settings.
thatrichan commented 2026-11-13 10:28:13 +01:00
Author
Owner
Copy link

task jetbrains:ui-test:docker runs the walk in a Linux container on Xvfb, nothing on screen (merged in ad557bd: 85b202e, 1312fb9).

  • Image jetbrains/ui-test.Dockerfile (eclipse-temurin:25-jdk, the computer's architecture); Gradle downloads the Linux WebStorm for platformVersion into a volume, so later runs take about 50 s.
  • The task says when Docker isn't there or isn't running, and when it has under 4 CPUs / 6 GiB, with what to raise for Colima, OrbStack or Docker Desktop.
  • Screenshots and idea-<test>.log land in jetbrains/build/ui-screenshots/; jetbrains/README.md → Testing says how to run it.
  • The walk itself got sturdier on Linux: notification balloons and Got it tooltips off, the embedded browser's sandbox off as root, the PR row clicked again before Enter, a real double click.
  • CI: .forgejo/workflows/jetbrains.yml walks it on Xvfb on every push touching jetbrains/ (without Docker-in-Docker, as the runner's socket isn't known to be available).

Passes on Linux. Left before closing: a run on a Mac (Apple silicon, arm64 WebStorm) and the first CI run.

`task jetbrains:ui-test:docker` runs the walk in a Linux container on Xvfb, nothing on screen (merged in ad557bd: 85b202e, 1312fb9). - Image `jetbrains/ui-test.Dockerfile` (eclipse-temurin:25-jdk, the computer's architecture); Gradle downloads the Linux WebStorm for `platformVersion` into a volume, so later runs take about 50 s. - The task says when Docker isn't there or isn't running, and when it has under 4 CPUs / 6 GiB, with what to raise for Colima, OrbStack or Docker Desktop. - Screenshots and `idea-<test>.log` land in `jetbrains/build/ui-screenshots/`; `jetbrains/README.md` → Testing says how to run it. - The walk itself got sturdier on Linux: notification balloons and Got it tooltips off, the embedded browser's sandbox off as root, the PR row clicked again before Enter, a real double click. - CI: `.forgejo/workflows/jetbrains.yml` walks it on Xvfb on every push touching `jetbrains/` (without Docker-in-Docker, as the runner's socket isn't known to be available). Passes on Linux. Left before closing: a run on a Mac (Apple silicon, arm64 WebStorm) and the first CI run.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
nvim-v0.1.0
zed-v0.1.0
vscode-v0.1.0
jetbrains-v1.2.1
jetbrains-v1.2.0
jetbrains-v1.1.0
jetbrains-v1.0.0
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
thatrichan/revue-companion#7
No description provided.