Skip to content

Fog in EffectCompositer erases all AO when the camera is far from the origin (view/world space mismatch) + uninitialized fogFactor when the scene has no fog #54

Description

@repka3
**Environment:** n8ao 2.0.0 (npm), three r185, WebGL2, Chrome, NVIDIA GTX 960M. Current `master` still contains the same code.

## Symptom

In an open world with standard linear fog (`THREE.Fog(color, 30, 250)`), SSAO silently disappears once the camera is a few hundred meters from the world origin. The pass still pays its full GPU cost, the beauty render is correct, but the AO term is 1.0 across the entire frame (verified by reading back the internal render targets — every knob: intensity, radius, falloff, halfRes, quality has no visible effect). Display mode "AO" shows a uniform white screen. Move the camera near the origin and AO comes back — which makes the bug look intermittent/machine-dependent and very hard to track down.

## Root cause 1 — fog distance mixes view space and world space

In `src/EffectCompositer.js`, the perspective branch of `getWorldPos()` reconstructs the position using only `projectionMatrixInv`:

```glsl
vec2 ndc = coord * 2. - 1.;
float ndcZ = depthToClipZ(depth);
mat4 Q = projectionMatrixInv;
vec3 view = vec3(Q[0][0] * ndc.x + Q[3][0], Q[1][1] * ndc.y + Q[3][1], Q[3][2]);
float invW = 1. / (Q[2][3] * ndcZ + Q[3][3]);
return view * invW;   // <-- VIEW-space position, despite the function name
```

but the fog depth is then computed against `cameraPos`, which is a **world-space** uniform (`camera.getWorldPosition()`):

```glsl
float fogFactor;
float fogDepth = distance(
    cameraPos,
    getWorldPos(depth, vUv)
);
```

So `fogDepth ≈ |cameraPos|` for every pixel. As soon as the camera sits beyond the fog `far` distance from the world origin, `fogFactor` saturates to 1 for the whole frame and

```glsl
finalAo = mix(finalAo, 1.0, fogFactor);
```

erases the AO that was just computed. (The `viewMatrixInv` uniform is already available in this shader but unused — a hint this was meant to be a world-space position.)

## Root cause 2 — `fogFactor` is read uninitialized when the scene has no fog

`float fogFactor;` is only assigned inside `if (fog) { ... }`, but the `mix()` above executes unconditionally. With `scene.fog == null` the shader reads an uninitialized value — undefined behavior that can (and on our hardware sometimes does) also wipe or corrupt the AO term.

## Suggested fix

In view space the camera **is** the origin, so the true eye distance is simply the length of the reconstructed position — no extra matrix needed:

```glsl
float fogFactor = 0.0;
float fogDepth = length(getWorldPos(depth, vUv));
```

We run exactly this as a local runtime patch (pinned to 2.0.0) in our game and verified via render-target readbacks: AO is restored at any camera position, and the fog fade now uses the correct per-pixel eye distance (near geometry keeps AO, distant geometry fades into fog as intended). Happy to open a PR if useful.

## Repro sketch

Any scene will do:

1. `scene.fog = new THREE.Fog(0x8899aa, 30, 250);`
2. Place geometry and camera at e.g. `(350, 2, -150)` (i.e. `length(camera.position)` > fog far).
3. `n8aopass.setDisplayMode("AO")` → uniform white, no geometric detail; all configuration knobs appear dead.
4. Move the same setup to the origin → AO appears.

## Unrelated observation

We also hit a separate issue where fractional render-target sizes (three's `EffectComposer.setSize` forwards `cssSize × devicePixelRatio` un-floored; Windows displays at 125% scale produce e.g. `852.5`) make the AO pass classify the whole frame as background. Flooring in `N8AOPass.setSize` would harden against it. Happy to file separately if you want it tracked.

## Workaround we ship today (for anyone else hitting this)

Runtime shader patch, applied right after constructing the pass — no fork, survives `npm install`, and re-applies itself because n8ao rebuilds the compositer material on every reconfigure (halfRes/depth-type changes). It throws loudly if a future n8ao version changes the shader, so it can never silently stop working:

```js
const COMPOSITER_FOG_BUG =
  /float fogFactor;\s*float fogDepth = distance\(\s*cameraPos,\s*getWorldPos\(depth, vUv\)\s*\);/;
const COMPOSITER_FOG_FIX =
  'float fogFactor = 0.0;\n        float fogDepth = length(getWorldPos(depth, vUv));';

const patchCompositerFog = (pass) => {
  const material = pass.effectCompositerQuad?.material;
  if (!material || !COMPOSITER_FOG_BUG.test(material.fragmentShader)) {
    throw new Error('n8ao fog patch: shader changed — re-audit (pinned to 2.0.0)');
  }
  material.fragmentShader = material.fragmentShader.replace(COMPOSITER_FOG_BUG, COMPOSITER_FOG_FIX);
  material.needsUpdate = true;
};

const pass = new N8AOPass(scene, camera, width, height);
const configure = pass.configureEffectCompositer.bind(pass);
pass.configureEffectCompositer = (...args) => {
  configure(...args);
  patchCompositerFog(pass);
};
patchCompositerFog(pass); // the constructor already built the quad once
```

And for the fractional-size observation, the app-side guard is to hand the composer whole drawing-buffer pixels only (mirroring `WebGLRenderer`'s own flooring):

```js
composer.setPixelRatio(1);
composer.setSize(Math.floor(cssWidth * dpr), Math.floor(cssHeight * dpr));
```

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions