Split Queue Server into modular network, monitor, and edit-control packages

app/queue-server/README.md

@@ -1,100 +1,110 @@
-# Java Bluesky Interface (JBI)
+# Phoebus Queue Server Application
 
-Prototype Java client for the [Bluesky QueueServer](https://blueskyproject.io/bluesky-queueserver/) REST API.  
-It includes:
+A JavaFX-based client for the [Bluesky QueueServer](https://blueskyproject.io/bluesky-queueserver/) that provides a graphical interface for managing experiment queues, monitoring status, and editing plans at beamlines and scientific facilities.
 
-* **BlueskyHttpClient** – thread-safe singleton with rate-limiting, automatic retries and JUL logging
-* **BlueskyService**   – one blocking call per API route
-* **CLI / REPL** – *testing utilities only*
-* **JavaFX UI** – the part you actually launch in normal use
+## Features
 
----
+### Queue Management
+- **View & Edit Queue**: See all queued plans with parameters, reorder items via drag-and-drop
+- **Execute Plans**: Start, stop, pause, resume, and abort plan execution
+- **Real-time Status**: Live updates via WebSocket connections showing queue state, running plans, and RE Manager status
+- **History View**: Browse completed plans with execution results and metadata
 
-##   Prerequisites
+### Plan Editing
+- **Interactive Plan Editor**: Create and modify plans with type-safe parameter editing
+- **Python Type Support**: Full support for Python types (strings, lists, dicts, booleans, numbers)
+- **Schema Validation**: Parameters validated against plan schemas from the Queue Server
+- **Live Preview**: See plan parameters as they will be sent to the server
 
-* Java 17
-* Maven
-* Running Bluesky HTTP Server at `http://localhost:60610`  
-  *single-user API key `a` assumed below*
+### Plan Viewer
+- **Plan Details**: View plan parameters, metadata, and execution results
+- **Parameter Display**: All parameters shown with correct Python syntax and types
+- **Copy to Queue**: Duplicate plans with one click
 
-```bash
-# 1) Start RE Manager
-start-re-manager --use-ipython-kernel=ON --zmq-publish-console=ON
+### Console Monitor
+- **Live Console Output**: Real-time streaming of Queue Server console output
+- **WebSocket Support**: Efficient streaming via WebSocket connections (fallback to HTTP polling)
+- **Autoscroll**: Automatically scroll to latest output with toggle control
+
+## Quick Start
+
+### Prerequisites
 
-# 2) Start HTTP Server
-QSERVER_HTTP_SERVER_SINGLE_USER_API_KEY=a \
-    uvicorn --host localhost --port 60610 bluesky_httpserver.server:app
-````
+- **Java 17** or later
+- **Maven** (for building from source)
+- **Bluesky Queue Server** running and accessible
 
----
+### Starting a Local Queue Server
 
-##   Configure
+Use the provided Docker setup for local development:
 
 ```bash
-export BLUESKY_API_KEY=a 
+cd services/bluesky-services
+docker-compose --profile container-redis up -d
 ```
 
----
+This starts:
+- Bluesky Queue Server (RE Manager) on ports 60615/60625
+- HTTP Server REST API on port 60610
+- Redis database on port 6380
 
-##  Build & run
+For details, see [services/bluesky-services/README.md](../../services/bluesky-services/README.md)
 
-```bash
-mvn clean javafx:run
-```
+### Configuration
 
-### Verbose logging
+Set environment variables to connect to your Queue Server:
 
 ```bash
-mvn -Djava.util.logging.config.file=src/main/resources/logging.properties javafx:run
-```
+# Queue Server HTTP address (default: http://localhost:60610)
+export QSERVER_HTTP_SERVER_URI=http://localhost:60610
 
----
+# API Key authentication
+export QSERVER_HTTP_SERVER_API_KEY=a
 
-##   CLI / REPL (testing only)
+# Or use a file containing the API key
+export QSERVER_HTTP_SERVER_API_KEYFILE=~/.phoebus/qserver_api_key.txt
+```
 
-| Tool     | Start (quiet)                                                                         | Start with request tracing (`FINE`)                                                                                                                        |
-| -------- | ------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| **CLI**  | `mvn -q -Dexec.mainClass=util.org.phoebus.applications.queueserver.RunEngineCli  -Dexec.args="STATUS" exec:java` | `mvn -Djava.util.logging.config.file=src/main/resources/logging.properties -q -Dexec.mainClass=util.org.phoebus.applications.queueserver.RunEngineCli -Dexec.args="STATUS" exec:java` |
-| **REPL** | `mvn -q -Dexec.mainClass=util.org.phoebus.applications.queueserver.RunEngineRepl exec:java`                          | `mvn -Djava.util.logging.config.file=src/main/resources/logging.properties -q -Dexec.mainClass=util.org.phoebus.applications.queueserver.RunEngineRepl -Dexec.args="STATUS" exec:java`                                                                                                                         |
+### Building & Running
 
-*CLI examples*
+#### As Part of Phoebus
 
 ```bash
-# list endpoints
-mvn -q -Dexec.mainClass=com.jbi.util.RunEngineClili -Dexec.args="list" exec:java
-
-# start the queue
-mvn -q -Dexec.mainClass=com.jbi.util.RunEngineClili -Dexec.args="QUEUE_START" exec:java
+# From phoebus root directory
+mvn clean install -DskipTests
+cd phoebus-product/target
+./phoebus
 ```
 
-`ENDPOINT [body]` accepts a JSON literal, `@file.json`, or `@-` for stdin.
+Then open **Applications → Queue Server** from the menu.
 
----
+#### Standalone Build
 
-##   How logging works
+```bash
+# From queue-server directory
+cd app/queue-server
+mvn clean install
+```
 
-* Logger name: **`com.jbi.bluesky`**
-* Levels
+## Configuration Options
 
-  * `INFO` – API errors
-  * `WARNING` – transport retries
-  * `FINE` – each HTTP call + latency
-* Enable by passing JVM flag
-  `-Djava.util.logging.config.file=src/main/resources/logging.properties`
+Configuration via **Edit → Preferences → Queue Server** or environment variables:
 
----
+| Preference        | Environment Variable              | Default                  | Description                           |
+|-------------------|-----------------------------------|--------------------------|---------------------------------------|
+| `queue_server_url`| `QSERVER_HTTP_SERVER_URI`         | `http://localhost:60610` | Queue Server HTTP address             |
+| `api_key`         | `QSERVER_HTTP_SERVER_API_KEY`     | *(none)*                 | API key for authentication            |
+| `api_key_file`    | `QSERVER_HTTP_SERVER_API_KEYFILE` | *(none)*                 | Path to file containing API key       |
+| `use_websockets`  | *(none)*                          | `true`                   | Use WebSockets for streaming data     |
+| `connectTimeout`  | *(none)*                          | `5000`                   | HTTP connection timeout (ms)          |
+| `debug`           | *(none)*                          | `false`                  | Enable HTTP request/response logging  |
 
-##   Tuning
+## Contributing
 
-```java
-// rate limit (req/sec)
-BlueskyHttpClient.initialize("http://localhost:60610",
-                             System.getenv("BLUESKY_API_KEY"),
-                             3.0);
+When making changes:
 
-// retry/back-off
-// edit src/main/java/com/jbi/util/HttpSupport.java
-MAX_RETRIES        = 5;
-INITIAL_BACKOFF_MS = 500;
-BACKOFF_MULTIPLIER = 1.5;
-```
+1. Ensure proper Python type handling in parameter editor
+2. Test with both WebSocket and HTTP fallback modes
+3. Verify API key authentication works
+4. Update documentation for new features
+5. Follow existing code style and patterns

app/queue-server/edit-control/pom.xml

@@ -0,0 +1,31 @@
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
+                             http://maven.apache.org/xsd/maven-4.0.0.xsd">
+    <modelVersion>4.0.0</modelVersion>
+    <parent>
+        <groupId>org.phoebus</groupId>
+        <artifactId>app-queue-server</artifactId>
+        <version>5.0.3-SNAPSHOT</version>
+    </parent>
+    <artifactId>app-queue-server-edit-control</artifactId>
+    <name>${project.groupId}:${project.artifactId}</name>
+
+    <dependencies>
+        <dependency>
+            <groupId>org.phoebus</groupId>
+            <artifactId>app-queue-server-network</artifactId>
+            <version>${project.version}</version>
+        </dependency>
+        <dependency>
+            <groupId>org.phoebus</groupId>
+            <artifactId>core-framework</artifactId>
+            <version>${project.version}</version>
+        </dependency>
+        <dependency>
+            <groupId>org.phoebus</groupId>
+            <artifactId>core-ui</artifactId>
+            <version>${project.version}</version>
+        </dependency>
+    </dependencies>
+</project>

app/queue-server/edit-control/src/main/java/org/phoebus/applications/queueserver/editcontrol/QueueEditControlApp.java

@@ -0,0 +1,88 @@
+package org.phoebus.applications.queueserver.editcontrol;
+
+import java.net.URL;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.logging.Level;
+import java.util.logging.Logger;
+
+import org.phoebus.applications.queueserver.Preferences;
+import org.phoebus.applications.queueserver.client.RunEngineHttpClient;
+import org.phoebus.applications.queueserver.util.AppLifecycle;
+import org.phoebus.applications.queueserver.util.PythonParameterConverter;
+import org.phoebus.applications.queueserver.view.ViewFactory;
+import javafx.scene.Parent;
+
+import org.phoebus.framework.spi.AppInstance;
+import org.phoebus.framework.spi.AppResourceDescriptor;
+import org.phoebus.framework.workbench.ApplicationService;
+import org.phoebus.ui.docking.DockItem;
+import org.phoebus.ui.docking.DockPane;
+
+@SuppressWarnings("nls")
+public final class QueueEditControlApp implements AppResourceDescriptor {
+
+    public static final Logger logger = Logger.getLogger(QueueEditControlApp.class.getPackageName());
+
+    public  static final String NAME         = "queue_edit_control";
+    private static final String DISPLAY_NAME = "Edit & Control Queue";
+
+    static {
+        PythonParameterConverter.initializeInBackground();
+    }
+
+    @Override public String getName()        { return NAME; }
+    @Override public String getDisplayName() { return DISPLAY_NAME; }
+
+    @Override public URL getIconURL() {
+        return getClass().getResource("/icons/bluesky.png");
+    }
+
+    @Override public AppInstance create() {
+        initializeHttpClient();
+        AppLifecycle.registerApp();
+
+        Parent root = ViewFactory.EDIT_AND_CONTROL_QUEUE.get();
+        QueueEditControlInstance inst = new QueueEditControlInstance(this, root);
+
+        DockItem tab = new DockItem(inst, root);
+        tab.addClosedNotification(AppLifecycle::unregisterApp);
+        DockPane.getActiveDockPane().addTab(tab);
+
+        return inst;
+    }
+
+    @Override public AppInstance create(java.net.URI resource) {
+        return ApplicationService.createInstance(NAME);
+    }
+
+    private static void initializeHttpClient() {
+        String serverUrl = Preferences.queue_server_url;
+        if (serverUrl == null || serverUrl.trim().isEmpty() || serverUrl.startsWith("$(")) {
+            serverUrl = "http://localhost:60610";
+            logger.log(Level.INFO, "Using default Queue Server URL: " + serverUrl);
+        }
+        String apiKey = resolveApiKey();
+        RunEngineHttpClient.initialize(serverUrl, apiKey);
+    }
+
+    private static String resolveApiKey() {
+        String apiKey = Preferences.api_key;
+        if (apiKey != null && !apiKey.trim().isEmpty() && !apiKey.startsWith("$(")) {
+            return apiKey.trim();
+        }
+        String keyFilePath = Preferences.api_key_file;
+        if (keyFilePath != null && !keyFilePath.trim().isEmpty() && !keyFilePath.startsWith("$(")) {
+            try {
+                Path path = Paths.get(keyFilePath.trim());
+                if (Files.exists(path)) {
+                    return Files.readString(path).trim();
+                }
+            } catch (Exception e) {
+                logger.log(Level.WARNING, "Failed to read API key from file: " + keyFilePath, e);
+            }
+        }
+        return null;
+    }
+}

app/queue-server/edit-control/src/main/java/org/phoebus/applications/queueserver/editcontrol/QueueEditControlInstance.java

@@ -0,0 +1,23 @@
+package org.phoebus.applications.queueserver.editcontrol;
+
+import javafx.scene.Node;
+import org.phoebus.framework.persistence.Memento;
+import org.phoebus.framework.spi.AppDescriptor;
+import org.phoebus.framework.spi.AppInstance;
+
+final class QueueEditControlInstance implements AppInstance {
+
+    private final AppDescriptor desc;
+    private final Node          view;
+
+    QueueEditControlInstance(AppDescriptor desc, Node view) {
+        this.desc = desc;
+        this.view = view;
+    }
+
+    @Override public AppDescriptor getAppDescriptor() { return desc; }
+    public Node create() { return view; }
+
+    @Override public void restore(Memento m) { /* nothing */ }
+    @Override public void save   (Memento m) { /* nothing */ }
+}

app/queue-server/edit-control/src/main/java/org/phoebus/applications/queueserver/editcontrol/QueueEditControlMenuEntry.java

@@ -0,0 +1,20 @@
+package org.phoebus.applications.queueserver.editcontrol;
+
+import org.phoebus.applications.queueserver.Messages;
+import org.phoebus.framework.workbench.ApplicationService;
+import org.phoebus.ui.javafx.ImageCache;
+import org.phoebus.ui.spi.MenuEntry;
+import javafx.scene.image.Image;
+
+public final class QueueEditControlMenuEntry implements MenuEntry {
+
+    @Override public String getName()      { return Messages.EditControlQueue; }
+    @Override public Image  getIcon()      { return ImageCache.getImage(
+            QueueEditControlApp.class, "/icons/bluesky.png"); }
+    @Override public String getMenuPath()  { return Messages.EditControlQueueMenuPath; }
+
+    @Override public Void call() throws Exception {
+        ApplicationService.createInstance(QueueEditControlApp.NAME);
+        return null;
+    }
+}

app/queue-server/edit-control/src/main/resources/META-INF/services/org.phoebus.framework.spi.AppDescriptor

@@ -0,0 +1 @@
+org.phoebus.applications.queueserver.editcontrol.QueueEditControlApp

app/queue-server/edit-control/src/main/resources/META-INF/services/org.phoebus.ui.spi.MenuEntry

@@ -0,0 +1 @@
+org.phoebus.applications.queueserver.editcontrol.QueueEditControlMenuEntry

app/queue-server/monitor/pom.xml

@@ -0,0 +1,31 @@
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
+                             http://maven.apache.org/xsd/maven-4.0.0.xsd">
+    <modelVersion>4.0.0</modelVersion>
+    <parent>
+        <groupId>org.phoebus</groupId>
+        <artifactId>app-queue-server</artifactId>
+        <version>5.0.3-SNAPSHOT</version>
+    </parent>
+    <artifactId>app-queue-server-monitor</artifactId>
+    <name>${project.groupId}:${project.artifactId}</name>
+
+    <dependencies>
+        <dependency>
+            <groupId>org.phoebus</groupId>
+            <artifactId>app-queue-server-network</artifactId>
+            <version>${project.version}</version>
+        </dependency>
+        <dependency>
+            <groupId>org.phoebus</groupId>
+            <artifactId>core-framework</artifactId>
+            <version>${project.version}</version>
+        </dependency>
+        <dependency>
+            <groupId>org.phoebus</groupId>
+            <artifactId>core-ui</artifactId>
+            <version>${project.version}</version>
+        </dependency>
+    </dependencies>
+</project>