Microsoft open-sources desktop tool that turns screen recordings into AI skills

Record yourself doing a task once, and AI can turn it into an automated script your agent runs forever.

Microsoft open-sourced Skill Recorder, a desktop app that captures your screen and turns manual work into AI automations. The tool runs locally on macOS and Windows 11. When you finish a task, the app uses the GitHub Copilot CLI to reconstruct your clicks, window switches, and optional spoken narration into an ordered step-by-step list. It then generates a reusable SKILL.md procedure or a scheduled automation.

Why it matters: Standard macro recorders break as soon as an interface changes. Skill Recorder maps your actions to high-level intent, leaning on native tools like the gh CLI or web fetches instead of replaying mouse coordinates. Doing a chore once teaches your agent how to handle it permanently.

Know this: All video capture and speech-to-text processing happen on your computer using an on-device Whisper model. Timeline data and screen snapshots only go to GitHub's cloud when you hit Analyze. Because that data leaves your machine, keep passwords and API keys off screen while recording.

The project picked up 766 stars this week, showing clear demand for automated workflow capture.