Skip to content

Conversation

@Pranav-0440
Copy link

Summary

This PR adds a developer-only experimental recorder that captures the
main turtle/mouse animation canvas together with application audio into a
single WebM file using standard browser APIs:

  • canvas.captureStream()
  • existing AudioContext
  • createMediaStreamDestination()
  • MediaRecorder

There are no UI changes. The feature is opt-in, experimental, and
intended only for maintainer and developer testing.


Files Added / Modified

  • js/recorder-dev-only.js (new)
    Dev-only recording module exposing global hooks for testing.

  • index.html
    Includes the dev-only recorder script (experimental).


Behavior & Design Notes

  • Reuses the existing AudioContext (never creates a new one).
  • Captures visuals from the primary animation canvas.
  • Supports explicit audio wiring via a provided hook
    (connectAudioToDest) to avoid assumptions about the audio graph.
  • WebM-first with codec fallback:
    • VP9 → VP8 → plain WebM (via MediaRecorder.isTypeSupported)
  • No UI, menus, localization, or MP4 support included.

This PR implements a prototype solution for issue #1673 to validate
synchronized animation + audio export before any UI or format expansion.


Notes:

  • AudioContext resumed via user gesture (Tone.context.resume())
    due to browser autoplay policy.
  • Recorder prevents duplicate starts and cleans up on stop/cancel.

Summary

This PR adds a developer-only experimental recorder that captures the main turtle/mouse animation canvas together with application audio into a single WebM file.
The implementation uses browser-native APIs:

  • canvas.captureStream() for visuals
  • existing AudioContextcreateMediaStreamDestination() for audio
  • MediaRecorder for recording

There are no UI changes. The feature is opt-in and intended for maintainer/dev testing and validation.


Motivation

Addresses the long-standing request in #1673 to enable exporting:

  • mouse/turtle artwork animation alone, and
  • mouse/turtle animation with music combined into a video.

This PR provides a minimal, low-risk prototype to validate feasibility and browser behavior before any UI or production-facing export workflow is considered.


What’s included

  • New (dev-only) module: js/recorder-dev-only.js
  • Exposes two global hooks:
    • window.__startExportRecording()
    • window.__stopExportRecording()
  • Captures:
    • the active animation canvas
    • app audio routed through the existing AudioContext
  • WebM-only output with codec fallback:
    • VP9 → VP8 → plain WebM
  • Guards against:
    • duplicate recording starts
    • missing canvas or AudioContext
  • Cleans up streams and connections on stop/cancel

What’s NOT included

  • ❌ No UI buttons or menus
  • ❌ No localization changes
  • ❌ No MP4 export (can be a follow-up)
  • ❌ No new AudioContext creation

Browser & policy notes

  • Due to browser autoplay policies, audio recording requires a user gesture
    (e.g., clicking Run) and resuming the AudioContext:
    Tone.context.resume();
  • Tested primarily in Chromium-based browsers (Chrome).

Testing instructions (for reviewers)

  1. Run Music Blocks locally (Chrome recommended).
  2. Open DevTools Console.
  3. Resume audio (after a user gesture):
Tone.context.resume();
  1. Start recording (auto-detection):
window.__startExportRecording({
  frameRate: 30,
  filename: 'demo-short.webm'
});
  1. Run an animation with music for ~5 seconds.
  2. Stop recording:
window.__stopExportRecording({
  filename: 'demo-short.webm'
});
  1. Verify the downloaded .webm contains both video and audio and is roughly in sync.

Demos verified (Chrome)

Notes

  • The recorder intentionally throws if started twice (safety guard).
  • This module is meant to remain dev-only or gated behind an experimental flag.
  • MP4 export (e.g., via ffmpeg) can be explored as a follow-up once the approach is accepted.

…as+audio to WebM

Exposes window.__startExportRecording/__stopExportRecording and __cancelExportRecording.

Auto-detects and attempts to auto-connect Tone.Destination, p5.soundOut output, and Howler master gain to the MediaStreamDestination when possible. Uses existing AudioContext and defaults to #myCanvas for the canvas getter.

This is a developer-only prototype for maintainers to test and wire into the runtime.
@github-actions
Copy link
Contributor

❌ Some Jest tests failed. Please check the logs and fix the issues before merging.

Failed Tests:

SaveInterface.test.js

@github-actions
Copy link
Contributor

❌ Some Jest tests failed. Please check the logs and fix the issues before merging.

Failed Tests:

SaveInterface.test.js

@github-actions
Copy link
Contributor

❌ Some Jest tests failed. Please check the logs and fix the issues before merging.

Failed Tests:

SaveInterface.test.js

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant