Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -134,6 +134,29 @@ allowed can still _show_ the MOTD; `/motd set` then reports the refusal instead
| `REGION` | `{{region}}`, and the region `/region` considers "here" |
| `CONTINENT` | `{{localzone}}` / `{{continent}}` |

### Bedrock device platforms

On a proxy that also runs Floodgate — only `velocity-bedrock` does — the endpoint additionally
publishes which platform Bedrock players are on:

```text
velocity_bedrock_players{device_os="ANDROID"} Floodgate's DeviceOs, by enum name:
ANDROID, IOS, OSX, XBOX, NX, PS4, UWP, …
```

Nothing else in the estate can answer this. The device travels in a Bedrock client's login chain,
Geyser hands it to Floodgate, and by the time Velocity sees the player they are an ordinary
Java-protocol connection with the platform stripped off — the game servers cannot tell a Switch
from a phone.

Floodgate is read **reflectively** and is not a build dependency: this plugin loads on every proxy
and only one of them has Floodgate, and GeyserMC publishes the API as a SNAPSHOT only. A proxy
without Floodgate publishes no `velocity_bedrock_players` series at all — absent rather than zero,
because "no Bedrock players" and "cannot see Bedrock players" are different states.

A platform that empties keeps its series and reports 0, so a graph shows nobody on a Switch rather
than a gap.

The NATS auth-callout scopes each pod to the subjects declared in its bundle `events:` block, so `proxy.system.*` and `proxy.transfer.*` must be listed there — an undeclared subject is denied and the message vanishes.

## Build
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
package gg.grounds.proxy.velocity.metrics

import org.slf4j.Logger

/**
* How many Bedrock players are on each device platform, asked of Floodgate.
*
* Only the Bedrock proxy runs Floodgate, and only Floodgate knows this: the device a Bedrock client
* runs on travels in its login chain, Geyser hands that to Floodgate, and by the time Velocity sees
* the player they are an ordinary Java-protocol connection with the platform stripped off. Nothing
* downstream can recover it — the game servers cannot tell a Switch from a phone.
*
* ## Why reflection rather than a dependency
*
* This plugin is loaded on **every** proxy, and only one of them has Floodgate. A compile-time
* dependency would be `compileOnly` anyway — Floodgate provides the classes at runtime — so the
* only thing it would buy is type safety against an artifact GeyserMC publishes as a **SNAPSHOT
* only**. A moving snapshot that changes an interface breaks the build of a plugin that has nothing
* to do with Bedrock, on a proxy that never loads Floodgate. Three calls do not justify that.
*
* The reflection is shallow on purpose: one static, one collection, one getter per player, and the
* enum is read as a name so `DeviceOs` — which lives in a different artifact again — never has to
* be resolved at all.
*
* ## Why it probes on every read
*
* Velocity's plugin classloaders can see each other, but load *order* is not guaranteed and this
* declares no dependency on Floodgate. Probing once at startup would mean a proxy that happened to
* initialise this plugin first concluded "no Floodgate" and stayed wrong for the life of the pod.
* Probing per read costs a cached `Class.forName` and heals itself.
*/
class BedrockDevices
internal constructor(
private val logger: Logger,
/**
* How the Floodgate API class is found. Production looks it up by name through this plugin's
* own classloader; a test hands in a stand-in of the same shape, so the reflection below is the
* code that runs rather than a copy of it written twice.
*/
private val lookup: () -> Class<*>?,
) {

/** Resolved on first success and kept — the class does not come and go once Floodgate is up. */
@Volatile private var api: Class<*>? = null

/** True once Floodgate has been seen, so a later failure is reported rather than swallowed. */
@Volatile private var seenFloodgate = false

/**
* Bedrock players per device platform, keyed by Floodgate's `DeviceOs` name (`ANDROID`, `IOS`,
* `XBOX`, `NX`, `PS4`, `UWP`, …).
*
* Empty when Floodgate is not loaded, which is every Java proxy — and empty is the honest
* answer there rather than zero, because "no Bedrock players" and "cannot see Bedrock players"
* are different states and only one of them is worth a graph.
*/
fun countsByDevice(): Map<String, Int> {
val floodgate = resolve() ?: return emptyMap()
return try {
val instance = floodgate.getMethod("getInstance").invoke(null) ?: return emptyMap()
val players =
floodgate.getMethod("getPlayers").invoke(instance) as? Collection<*>
?: return emptyMap()

val counts = LinkedHashMap<String, Int>()
for (player in players) {
if (player == null) continue
val device = deviceNameOf(player) ?: continue
counts[device] = (counts[device] ?: 0) + 1
}
counts
} catch (failure: ReflectiveOperationException) {
// Floodgate is present but does not look the way it did. Reported once per read is too
// noisy and never is too quiet; the endpoint keeps serving everything else either way.
logger.debug("Could not read Bedrock device platforms from Floodgate", failure)
emptyMap()
} catch (failure: RuntimeException) {
logger.debug("Floodgate refused to report its players", failure)
emptyMap()
}
}

/**
* The player's `DeviceOs` as its enum constant name.
*
* `name` rather than `toString`: Floodgate's enum overrides `toString` with a display name
* ("Android", "Nintendo Switch"), and a label that changes case and spacing between versions is
* a label that silently splits a series in two.
*/
private fun deviceNameOf(player: Any): String? {
val device = player.javaClass.getMethod("getDeviceOs").invoke(player) ?: return null
return (device as? Enum<*>)?.name ?: device.toString()
}

private fun resolve(): Class<*>? {
api?.let {
return it
}
val found = lookup() ?: return null
if (!seenFloodgate) {
seenFloodgate = true
logger.info("Floodgate found; publishing Bedrock device platforms")
}
api = found
return found
}

companion object {
private const val FLOODGATE_API = "org.geysermc.floodgate.api.FloodgateApi"

fun of(logger: Logger): BedrockDevices =
BedrockDevices(logger) {
try {
Class.forName(FLOODGATE_API, false, BedrockDevices::class.java.classLoader)
} catch (_: ClassNotFoundException) {
null
}
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import com.sun.net.httpserver.HttpExchange
import com.sun.net.httpserver.HttpServer
import io.micrometer.core.instrument.Gauge
import io.micrometer.core.instrument.MeterRegistry
import io.micrometer.core.instrument.MultiGauge
import io.micrometer.core.instrument.Tags
import io.micrometer.core.instrument.binder.jvm.ClassLoaderMetrics
import io.micrometer.core.instrument.binder.jvm.JvmGcMetrics
Expand All @@ -15,6 +16,7 @@ import io.micrometer.prometheusmetrics.PrometheusConfig
import io.micrometer.prometheusmetrics.PrometheusMeterRegistry
import java.net.InetSocketAddress
import java.nio.charset.StandardCharsets
import java.util.concurrent.ConcurrentHashMap
import java.util.concurrent.Executors
import org.slf4j.Logger

Expand Down Expand Up @@ -81,6 +83,7 @@ private constructor(
snapshot: ProxySnapshot,
logger: Logger,
region: String? = System.getenv("REGION"),
devices: BedrockDevices = BedrockDevices.of(logger),
): ProxyMetrics {
val registry = PrometheusMeterRegistry(PrometheusConfig.DEFAULT)
// `cluster` and `pod` are stamped by the satellite's metrics agent; the region as the
Expand All @@ -94,8 +97,23 @@ private constructor(
val closeables = bindJvmAndProcess(registry)
bindProxyGauges(registry, snapshot)

// One series per Bedrock device platform, and the row set is not known ahead of time —
// it is whatever players happen to be connected. A MultiGauge is the one meter that
// takes a changing set of label values without registering a meter per value by hand.
val bedrock =
MultiGauge.builder("velocity.bedrock.players")
.description("Bedrock players by the device platform they are playing on")
.register(registry)
val seenDevices = ConcurrentHashMap.newKeySet<String>()

val http = HttpServer.create(InetSocketAddress(config.host, config.port), 0)
http.createContext("/") { exchange -> handle(exchange, config.path, registry) }
http.createContext("/") { exchange ->
// Refreshed here rather than on a timer: the endpoint is the only reader, so a
// scrape gets the count as it is at that moment and an unscraped proxy pays
// nothing. The walk is one pass over the connected Bedrock players.
refreshDevices(bedrock, seenDevices, devices)
handle(exchange, config.path, registry)
}
http.executor =
Executors.newSingleThreadExecutor { runnable ->
Thread(runnable, "grounds-proxy-metrics").apply { isDaemon = true }
Expand All @@ -112,6 +130,30 @@ private constructor(
return metrics
}

/**
* Rewrite the device rows from what Floodgate reports right now.
*
* Every platform ever seen keeps a row, reporting 0 when nobody is on it. Registering only
* the platforms currently present would make a series go **stale** the moment its last
* player leaves, which Grafana draws as a gap — indistinguishable from the endpoint being
* down, and exactly wrong for the number that says "nobody is playing on a Switch".
*/
private fun refreshDevices(
gauge: MultiGauge,
seen: MutableSet<String>,
devices: BedrockDevices,
) {
val counts = devices.countsByDevice()
seen.addAll(counts.keys)
if (seen.isEmpty()) return
gauge.register(
seen.map { device ->
MultiGauge.Row.of(Tags.of("device_os", device), counts[device] ?: 0)
},
true,
)
}

private fun handle(
exchange: HttpExchange,
path: String,
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
package gg.grounds.proxy.velocity.metrics

import org.junit.jupiter.api.Assertions.assertEquals
import org.junit.jupiter.api.Assertions.assertTrue
import org.junit.jupiter.api.Test
import org.slf4j.LoggerFactory

/**
* Floodgate is deliberately not on this classpath — a plugin that builds without it is the whole
* reason the API is read reflectively. So the reflection runs against a stand-in of the same shape,
* handed in through the class lookup, and the not-installed case runs against its real absence.
*
* The shape asserted here is Floodgate's, taken from its published API jar: a static
* `FloodgateApi.getInstance()`, a `getPlayers()` returning a collection, and a
* `FloodgatePlayer.getDeviceOs()` returning an enum.
*/
class BedrockDevicesTest {

private val logger = LoggerFactory.getLogger(BedrockDevicesTest::class.java)

/** `DeviceOs` in the way that matters here: an enum whose `toString` is not its name. */
enum class FakeDeviceOs {
ANDROID,
NX;

override fun toString(): String = if (this == ANDROID) "Android" else "Nintendo Switch"
}

class FakePlayer(private val device: FakeDeviceOs?) {
fun getDeviceOs(): FakeDeviceOs? = device
}

object FakeFloodgateApi {
@JvmStatic var connected: Collection<Any?> = emptyList()

@JvmStatic fun getInstance(): FakeFloodgateApi = this

@JvmStatic fun getPlayers(): Collection<Any?> = connected
}

/**
* A Floodgate that is loaded but returns no instance, which is how it looks before it starts.
*/
object NotStartedFloodgateApi {
@JvmStatic fun getInstance(): Any? = null
}

/** Floodgate present but a different shape — a version that renamed or dropped a method. */
object ChangedFloodgateApi {
@JvmStatic fun getInstance(): ChangedFloodgateApi = this
}

private fun devices(api: Class<*>?) = BedrockDevices(logger) { api }

@Test
fun `reports nothing when Floodgate is not installed`() {
// The state of every Java proxy: the class is simply not there.
assertTrue(
BedrockDevices.of(logger).countsByDevice().isEmpty(),
"a proxy without Floodgate claimed to know about Bedrock players",
)
}

@Test
fun `counts Bedrock players by device platform`() {
FakeFloodgateApi.connected =
listOf(
FakePlayer(FakeDeviceOs.ANDROID),
FakePlayer(FakeDeviceOs.ANDROID),
FakePlayer(FakeDeviceOs.NX),
)

assertEquals(
mapOf("ANDROID" to 2, "NX" to 1),
devices(FakeFloodgateApi::class.java).countsByDevice(),
)
}

@Test
fun `a player whose device Floodgate does not know is skipped, not invented`() {
FakeFloodgateApi.connected = listOf(FakePlayer(FakeDeviceOs.NX), FakePlayer(null))

assertEquals(mapOf("NX" to 1), devices(FakeFloodgateApi::class.java).countsByDevice())
}

@Test
fun `the platform is the enum name, not its display string`() {
// Floodgate renders NX as "Nintendo Switch". A label that changes case and spacing between
// releases splits one series into two, and neither half is the whole truth afterwards.
FakeFloodgateApi.connected = listOf(FakePlayer(FakeDeviceOs.NX))

val counts = devices(FakeFloodgateApi::class.java).countsByDevice()

assertEquals(setOf("NX"), counts.keys)
assertEquals("Nintendo Switch", FakeDeviceOs.NX.toString())
}

@Test
fun `no players is an empty map rather than a failure`() {
FakeFloodgateApi.connected = emptyList()

assertTrue(devices(FakeFloodgateApi::class.java).countsByDevice().isEmpty())
}

@Test
fun `Floodgate loaded but not yet started reports nothing`() {
assertTrue(devices(NotStartedFloodgateApi::class.java).countsByDevice().isEmpty())
}

@Test
fun `a Floodgate that changed shape degrades instead of throwing`() {
// The endpoint has to keep serving every other proxy metric even when this one cannot be
// read — a NoSuchMethodError escaping here would take the whole scrape with it.
assertTrue(devices(ChangedFloodgateApi::class.java).countsByDevice().isEmpty())
}

@Test
fun `the class is resolved once and then remembered`() {
FakeFloodgateApi.connected = listOf(FakePlayer(FakeDeviceOs.ANDROID))
var lookups = 0
val devices =
BedrockDevices(logger) {
lookups++
FakeFloodgateApi::class.java
}

repeat(5) { devices.countsByDevice() }

assertEquals(1, lookups, "the Floodgate class was looked up on every scrape")
}
}
Loading