Recreate a classic speech meme in real time • Generate animated accordion bellows from hand gestures • Play iconic melodies automatically • Preview in a minimalist dark web studio • Export comparison videos in multiple layouts
- 二次貝茲扇形風箱與雙手腕獨立傾角: 風箱會依雙手距離自然展開或收合,左右琴身則各自跟隨對應手腕旋轉。
- 名曲自動跟彈: 內建《Bella Ciao》、《玫瑰人生》與《俄羅斯方塊》(Korobeiniki),依手部移動距離和推拉轉折推進旋律。
- 極簡深色 Web Studio: 位於
web/,提供影片時間軸、快捷操作、靜音和匯出功能,無需 Python 後端。 - 多版型影片與 GIF 匯出: 支援單畫面、16:9 左右對比、9:16 直式短影音,以及調色盤最佳化 GIF。
需要 Python 3.10 以上版本(已使用 Python 3.12 測試):
pip install -r requirements.txt
python main.py執行 main.py 開啟桌面控制台,可選擇 Webcam 或影片檔、切換曲目、預覽效果,並匯出影片或 GIF。
Web 版是位於 web/ 的靜態網頁,不需要 Python 後端。可用以下命令在本機預覽:
python -m http.server 8000接著開啟 http://localhost:8000/web/。此網站也可部署至 GitHub Pages;即時手勢追蹤需要相機權限。
| 按鍵 | 功能 |
|---|---|
Q 或 ESC |
離開預覽 |
SPACE |
暫停或繼續播放 |
N |
切換下一首曲目:Bella Ciao → 玫瑰人生 → 俄羅斯方塊 |
P |
切換音高模式:名曲 → 和弦 → 滑音 → 固定 |
C |
切換貝茲彎曲風箱或直線風箱 |
M |
靜音或恢復手風琴音效 |
R |
重置追蹤濾波器與狀態 |
# 使用預設 Webcam 即時互動
python main.py --source 0
# 播放影片並指定曲目
python main.py --source speech.mp4 --song bella_ciao
# 匯出 16:9 左右對比影片
python main.py --source speech.mp4 --export compare.mp4 --layout side_by_side
# 匯出 9:16 直式影片
python main.py --source speech.mp4 --export reels.mp4 --layout vertical_9_16
# 同時輸出 MP4 與 GIF
python main.py --source speech.mp4 --export demo.mp4 --gif assets/demo.gif本專案採用 MIT License,歡迎自由 Fork、修改與分享。
- Quadratic Bézier fan bellows and independent wrist angles
- The bellows bend and fan out naturally along a quadratic Bézier curve as the distance between your hands changes.
- Each side of the accordion follows the rotation of its corresponding wrist for a more convincing 3D effect.
- Automatic melody mode
- Includes three classic accordion melodies: “Bella Ciao,” “La Vie en Rose,” and “Korobeiniki” (the Tetris theme).
- Hand movement advances the melody based on travel distance and push-pull transitions, so you can play by moving in front of the camera.
- Minimalist dark web studio (
web/)- An obsidian-black screening room with champagne-gold accents and ambient lighting.
- Includes a video timeline, keyboard controls, live mute, and one-click export.
- Video and GIF export for social media
- Solo: A clean composited accordion view.
- Side by side (16:9): Original footage next to the effect.
- Vertical (9:16): A split-screen layout for Instagram Reels, YouTube Shorts, and TikTok.
- GIF: Palette-optimized animations for GitHub and social platforms.
flowchart TD
subgraph Input["Inputs"]
Cam["Live webcam"]
Vid["Speech video (.mp4)"]
VidAudio["Extract original audio with FFmpeg"]
end
subgraph Core["Tracking and analysis (tracker.py)"]
MP["MediaPipe Hands + Pose occlusion recovery"]
EMA["EMA temporal filtering: hand positions, wrist angles, and speed"]
end
subgraph Audio["Audio synthesis and mixing (audio_engine.py, melodies.py)"]
MelodyFSM["Three-melody state machine"]
Synth["Musette dual-reed synthesizer: melody and chords"]
LiveMixer["Live mixer: accordion and original speech"]
end
subgraph Visual["Rendering (renderer.py)"]
Bezier["Quadratic Bézier fan bellows and independent wrist rotation"]
LayoutComp["Solo, side-by-side, and vertical layouts"]
end
subgraph Output["Output"]
Player["OpenCV live preview"]
Exporter["MP4 and GIF exporter"]
WebApp["Dark web studio (web/index.html)"]
end
Cam --> MP
Vid --> MP
Vid --> VidAudio
MP --> EMA
EMA --> MelodyFSM --> Synth
VidAudio --> LiveMixer
Synth --> LiveMixer
EMA --> Bezier --> LayoutComp
LiveMixer --> Exporter
LayoutComp --> Player
LayoutComp --> Exporter
EMA -.-> WebApp
PalmMotion requires Python 3.10 or later. It has been tested with Python 3.12.
pip install -r requirements.txtRun main.py to open the graphical control panel:
python main.pyFrom the control panel, you can:
- Choose a webcam or a video file as the input.
- Select a melody: “Bella Ciao,” “La Vie en Rose,” or “Korobeiniki.”
- Start the live preview, with original video audio playback and mute controls.
- Export a solo, 16:9 side-by-side, or 9:16 vertical video, or create a GIF.
The static web app lives in web/ and does not require a Python backend.
To preview it locally:
python -m http.server 8000Then open http://localhost:8000/web/ in your browser. The site can also be hosted on GitHub Pages. Camera access is required for live gesture tracking.
When the live preview window is open:
| Key | Action |
|---|---|
Q or ESC |
Exit the preview |
SPACE |
Pause or resume playback |
N |
Select the next melody: Bella Ciao → La Vie en Rose → Korobeiniki |
P |
Switch pitch mode: melody → chords → continuous → fixed |
C |
Switch between curved Bézier bellows and straight bellows |
M |
Mute or restore the accordion audio |
R |
Reset tracking filters and state |
Start live interaction with the default webcam:
python main.py --source 0Play a speech video with a selected melody:
python main.py --source speech.mp4 --song bella_ciaoExport a 16:9 side-by-side video:
python main.py --source speech.mp4 --export compare.mp4 --layout side_by_sideExport a 9:16 vertical video:
python main.py --source speech.mp4 --export reels.mp4 --layout vertical_9_16Export an MP4 and a preview GIF:
python main.py --source speech.mp4 --export demo.mp4 --gif assets/demo.gifPalmMotion/
├── main.py # Application entry point: GUI and CLI
├── requirements.txt # Python dependencies
├── assets/ # Accordion artwork, WAV samples, and demo GIFs
├── src/
│ ├── config.py # Shared configuration
│ ├── melodies.py # Melody notes and chord progressions
│ ├── tracker.py # MediaPipe hand/pose tracking and filtering
│ ├── renderer.py # Bellows rendering and video layouts
│ ├── audio_engine.py # Reed synthesis and audio mixing
│ ├── exporter.py # MP4 and GIF export
│ ├── player.py # OpenCV audio/video preview
│ ├── gui.py # Tkinter desktop interface
│ └── assets_generator.py # Accordion artwork and reference audio generation
├── web/ # Static web studio for GitHub Pages
│ ├── index.html # Web interface
│ ├── style.css # Web styles
│ └── app.js # MediaPipe, Web Audio, and Canvas rendering
└── tests/
└── test_pipeline.py # Pipeline and layout export tests
This project is licensed under the MIT License. Feel free to fork, modify, and share your work.
