x-space-downloader/docs/plans/2025-10-26-x-space-downloader-implementation.md

X Space Downloader Implementation Plan

For Claude: REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.

Goal: Build a Python CLI tool that downloads X (Twitter) Space recordings and outputs MP3 files.

Architecture: Thin wrapper around yt-dlp with URL validation, error handling, and Click-based CLI. Cookie-based authentication for X access.

Tech Stack: Python, yt-dlp, Click, pytest

---

Task 1: Project Setup and Dependencies

Files:

• Create: requirements.txt
• Create: setup.py
• Create: src/__init__.py
• Create: .gitignore

Step 1: Create requirements.txt

Create the file with dependencies:

yt-dlp>=2024.0.0
click>=8.0.0

Step 2: Create setup.py

Create the file to make the package installable:

from setuptools import setup, find_packages

setup(
    name="x-space-downloader",
    version="0.1.0",
    packages=find_packages(),
    install_requires=[
        "yt-dlp>=2024.0.0",
        "click>=8.0.0",
    ],
    entry_points={
        "console_scripts": [
            "x-space-dl=src.main:cli",
        ],
    },
    python_requires=">=3.8",
)

Step 3: Create src package init file

mkdir -p src
touch src/__init__.py

Step 4: Create .gitignore

# Python
__pycache__/
*.py[cod]
*$py.class
*.so
.Python
env/
venv/
ENV/
*.egg-info/
dist/
build/

# IDEs
.vscode/
.idea/
*.swp
*.swo

# Testing
.pytest_cache/
.coverage
htmlcov/

# Downloads
*.mp3
*.m4a
cookies.txt

Step 5: Install in development mode

Run: pip install -e . Expected: Package installed successfully

Step 6: Commit

git add requirements.txt setup.py src/__init__.py .gitignore
git commit -m "feat: add project setup and dependencies"

---

Task 2: URL Validator (TDD)

Files:

• Create: tests/test_validator.py
• Create: src/validator.py

Step 1: Create test directory

mkdir -p tests
touch tests/__init__.py

Step 2: Write failing test for valid URL

Create tests/test_validator.py:

import pytest
from src.validator import validate_space_url


def test_validate_space_url_valid():
    """Test that valid X Space URL is accepted."""
    url = "https://x.com/i/spaces/1ZkKzZWnVRRKv"
    is_valid, space_id, error = validate_space_url(url)

    assert is_valid is True
    assert space_id == "1ZkKzZWnVRRKv"
    assert error is None

Step 3: Run test to verify it fails

Run: pytest tests/test_validator.py::test_validate_space_url_valid -v Expected: FAIL with "cannot import name 'validate_space_url'"

Step 4: Write minimal validator implementation

Create src/validator.py:

import re
from typing import Tuple, Optional


def validate_space_url(url: str) -> Tuple[bool, Optional[str], Optional[str]]:
    """
    Validates X Space URL format.

    Args:
        url: The URL to validate

    Returns:
        Tuple of (is_valid, space_id, error_message)
    """
    # Pattern for X Space URLs
    pattern = r"https://x\.com/i/spaces/([a-zA-Z0-9]+)"
    match = re.match(pattern, url)

    if not match:
        return False, None, "Invalid URL format. Expected: https://x.com/i/spaces/{SPACE_ID}"

    space_id = match.group(1)
    return True, space_id, None

Step 5: Run test to verify it passes

Run: pytest tests/test_validator.py::test_validate_space_url_valid -v Expected: PASS

Step 6: Commit

git add tests/__init__.py tests/test_validator.py src/validator.py
git commit -m "feat: add URL validator for X Space URLs"

---

Task 3: Validator Error Cases (TDD)

Files:

• Modify: tests/test_validator.py

Step 1: Write test for invalid URL formats

Add to tests/test_validator.py:

def test_validate_space_url_invalid_format():
    """Test that invalid URLs are rejected."""
    invalid_urls = [
        "https://twitter.com/i/spaces/1ZkKzZWnVRRKv",  # Old twitter.com
        "https://x.com/spaces/1ZkKzZWnVRRKv",  # Missing /i/
        "https://x.com/i/spaces/",  # Missing space ID
        "not a url",  # Completely invalid
        "",  # Empty string
    ]

    for url in invalid_urls:
        is_valid, space_id, error = validate_space_url(url)
        assert is_valid is False
        assert space_id is None
        assert "Invalid URL format" in error

Step 2: Run test to verify it passes

Run: pytest tests/test_validator.py::test_validate_space_url_invalid_format -v Expected: PASS (implementation already handles these)

Step 3: Write test for twitter.com URL support

Add to tests/test_validator.py:

def test_validate_space_url_twitter_domain():
    """Test that twitter.com URLs are also accepted."""
    url = "https://twitter.com/i/spaces/1ZkKzZWnVRRKv"
    is_valid, space_id, error = validate_space_url(url)

    assert is_valid is True
    assert space_id == "1ZkKzZWnVRRKv"
    assert error is None

Step 4: Run test to verify it fails

Run: pytest tests/test_validator.py::test_validate_space_url_twitter_domain -v Expected: FAIL

Step 5: Update validator to support twitter.com

Modify src/validator.py:

def validate_space_url(url: str) -> Tuple[bool, Optional[str], Optional[str]]:
    """
    Validates X Space URL format.

    Args:
        url: The URL to validate

    Returns:
        Tuple of (is_valid, space_id, error_message)
    """
    # Pattern for X Space URLs (supports both x.com and twitter.com)
    pattern = r"https://(x|twitter)\.com/i/spaces/([a-zA-Z0-9]+)"
    match = re.match(pattern, url)

    if not match:
        return False, None, "Invalid URL format. Expected: https://x.com/i/spaces/{SPACE_ID}"

    space_id = match.group(2)  # Changed from group(1) to group(2)
    return True, space_id, None

Step 6: Run all validator tests

Run: pytest tests/test_validator.py -v Expected: All tests PASS

Step 7: Commit

git add tests/test_validator.py src/validator.py
git commit -m "feat: add validator error cases and twitter.com support"

---

Task 4: Downloader Module (TDD)

Files:

• Create: tests/test_downloader.py
• Create: src/downloader.py

Step 1: Write failing test for downloader

Create tests/test_downloader.py:

import pytest
from unittest.mock import Mock, patch
from src.downloader import download_space


def test_download_space_basic(tmp_path):
    """Test basic download functionality."""
    space_url = "https://x.com/i/spaces/1ZkKzZWnVRRKv"

    with patch('src.downloader.yt_dlp.YoutubeDL') as mock_ytdl_class:
        # Mock the YoutubeDL instance
        mock_ytdl = Mock()
        mock_ytdl_class.return_value.__enter__.return_value = mock_ytdl
        mock_ytdl.extract_info.return_value = {'id': '1ZkKzZWnVRRKv'}

        # Call download_space
        result = download_space(
            space_url=space_url,
            output_dir=str(tmp_path),
            cookies_file=None
        )

        # Verify YoutubeDL was called
        mock_ytdl.extract_info.assert_called_once_with(space_url, download=True)

        # Verify result contains expected filename
        assert "space_1ZkKzZWnVRRKv" in result

Step 2: Run test to verify it fails

Run: pytest tests/test_downloader.py::test_download_space_basic -v Expected: FAIL with "cannot import name 'download_space'"

Step 3: Write minimal downloader implementation

Create src/downloader.py:

import os
import yt_dlp
from typing import Optional


def download_space(
    space_url: str,
    output_dir: str,
    cookies_file: Optional[str] = None
) -> str:
    """
    Downloads X Space recording using yt-dlp.

    Args:
        space_url: URL of the X Space recording
        output_dir: Directory to save the MP3 file
        cookies_file: Optional path to cookies file for authentication

    Returns:
        Path to the downloaded MP3 file
    """
    # Extract space ID from URL for filename
    space_id = space_url.split('/')[-1]

    # Configure yt-dlp options
    ydl_opts = {
        'format': 'bestaudio/best',
        'outtmpl': os.path.join(output_dir, f'space_{space_id}.%(ext)s'),
        'postprocessors': [{
            'key': 'FFmpegExtractAudio',
            'preferredcodec': 'mp3',
        }],
        'quiet': False,
        'no_warnings': False,
    }

    # Add cookies if provided
    if cookies_file:
        ydl_opts['cookiefile'] = cookies_file

    # Download with yt-dlp
    with yt_dlp.YoutubeDL(ydl_opts) as ydl:
        ydl.extract_info(space_url, download=True)

    # Return the expected output path
    return os.path.join(output_dir, f'space_{space_id}.mp3')

Step 4: Run test to verify it passes

Run: pytest tests/test_downloader.py::test_download_space_basic -v Expected: PASS

Step 5: Commit

git add tests/test_downloader.py src/downloader.py
git commit -m "feat: add downloader module with yt-dlp integration"

---

Task 5: Downloader Error Handling (TDD)

Files:

• Modify: tests/test_downloader.py
• Modify: src/downloader.py

Step 1: Write test for download errors

Add to tests/test_downloader.py:

from src.downloader import DownloadError


def test_download_space_error_handling(tmp_path):
    """Test that download errors are properly caught and wrapped."""
    space_url = "https://x.com/i/spaces/invalid"

    with patch('src.downloader.yt_dlp.YoutubeDL') as mock_ytdl_class:
        mock_ytdl = Mock()
        mock_ytdl_class.return_value.__enter__.return_value = mock_ytdl
        mock_ytdl.extract_info.side_effect = Exception("Video unavailable")

        with pytest.raises(DownloadError) as exc_info:
            download_space(
                space_url=space_url,
                output_dir=str(tmp_path),
                cookies_file=None
            )

        assert "Failed to download" in str(exc_info.value)

Step 2: Run test to verify it fails

Run: pytest tests/test_downloader.py::test_download_space_error_handling -v Expected: FAIL with "cannot import name 'DownloadError'"

Step 3: Add error handling to downloader

Modify src/downloader.py:

import os
import yt_dlp
from typing import Optional


class DownloadError(Exception):
    """Raised when download fails."""
    pass


def download_space(
    space_url: str,
    output_dir: str,
    cookies_file: Optional[str] = None
) -> str:
    """
    Downloads X Space recording using yt-dlp.

    Args:
        space_url: URL of the X Space recording
        output_dir: Directory to save the MP3 file
        cookies_file: Optional path to cookies file for authentication

    Returns:
        Path to the downloaded MP3 file

    Raises:
        DownloadError: If download fails
    """
    # Extract space ID from URL for filename
    space_id = space_url.split('/')[-1]

    # Configure yt-dlp options
    ydl_opts = {
        'format': 'bestaudio/best',
        'outtmpl': os.path.join(output_dir, f'space_{space_id}.%(ext)s'),
        'postprocessors': [{
            'key': 'FFmpegExtractAudio',
            'preferredcodec': 'mp3',
        }],
        'quiet': False,
        'no_warnings': False,
    }

    # Add cookies if provided
    if cookies_file:
        ydl_opts['cookiefile'] = cookies_file

    # Download with yt-dlp
    try:
        with yt_dlp.YoutubeDL(ydl_opts) as ydl:
            ydl.extract_info(space_url, download=True)
    except Exception as e:
        raise DownloadError(f"Failed to download Space: {str(e)}")

    # Return the expected output path
    return os.path.join(output_dir, f'space_{space_id}.mp3')

Step 4: Run test to verify it passes

Run: pytest tests/test_downloader.py::test_download_space_error_handling -v Expected: PASS

Step 5: Run all downloader tests

Run: pytest tests/test_downloader.py -v Expected: All tests PASS

Step 6: Commit

git add tests/test_downloader.py src/downloader.py
git commit -m "feat: add error handling to downloader"

---

Task 6: CLI Main Module (TDD)

Files:

• Create: tests/test_main.py
• Create: src/main.py

Step 1: Write test for CLI entry point

Create tests/test_main.py:

import pytest
from click.testing import CliRunner
from unittest.mock import patch, Mock
from src.main import cli


def test_cli_valid_url(tmp_path):
    """Test CLI with valid URL."""
    runner = CliRunner()

    with patch('src.main.validate_space_url') as mock_validate, \
         patch('src.main.download_space') as mock_download:

        mock_validate.return_value = (True, "1ZkKzZWnVRRKv", None)
        mock_download.return_value = str(tmp_path / "space_1ZkKzZWnVRRKv.mp3")

        result = runner.invoke(cli, ['https://x.com/i/spaces/1ZkKzZWnVRRKv'])

        assert result.exit_code == 0
        assert "Successfully downloaded" in result.output
        assert "space_1ZkKzZWnVRRKv.mp3" in result.output

Step 2: Run test to verify it fails

Run: pytest tests/test_main.py::test_cli_valid_url -v Expected: FAIL with "cannot import name 'cli'"

Step 3: Write minimal CLI implementation

Create src/main.py:

import click
import os
from src.validator import validate_space_url
from src.downloader import download_space, DownloadError


@click.command()
@click.argument('url')
@click.option('--cookies', '-c', type=click.Path(exists=True), help='Path to cookies file')
@click.option('--output', '-o', type=click.Path(), default='.', help='Output directory')
def cli(url, cookies, output):
    """
    Download X (Twitter) Space recordings as MP3 files.

    Example:
        x-space-dl https://x.com/i/spaces/1ZkKzZWnVRRKv
    """
    # Validate URL
    is_valid, space_id, error = validate_space_url(url)
    if not is_valid:
        click.echo(f"Error: {error}", err=True)
        raise click.Exit(1)

    # Download the Space
    try:
        click.echo(f"Downloading Space: {space_id}")
        output_path = download_space(url, output, cookies)
        click.echo(f"Successfully downloaded to: {output_path}")
    except DownloadError as e:
        click.echo(f"Error: {str(e)}", err=True)
        raise click.Exit(2)


if __name__ == '__main__':
    cli()

Step 4: Run test to verify it passes

Run: pytest tests/test_main.py::test_cli_valid_url -v Expected: PASS

Step 5: Commit

git add tests/test_main.py src/main.py
git commit -m "feat: add CLI main module with Click integration"

---

Task 7: CLI Error Cases (TDD)

Files:

• Modify: tests/test_main.py

Step 1: Write test for invalid URL

Add to tests/test_main.py:

def test_cli_invalid_url():
    """Test CLI with invalid URL."""
    runner = CliRunner()

    result = runner.invoke(cli, ['https://invalid.com/spaces/123'])

    assert result.exit_code == 1
    assert "Invalid URL format" in result.output

Step 2: Run test to verify it passes

Run: pytest tests/test_main.py::test_cli_invalid_url -v Expected: PASS (implementation already handles this)

Step 3: Write test for download failure

Add to tests/test_main.py:

def test_cli_download_failure():
    """Test CLI when download fails."""
    runner = CliRunner()

    with patch('src.main.validate_space_url') as mock_validate, \
         patch('src.main.download_space') as mock_download:

        mock_validate.return_value = (True, "1ZkKzZWnVRRKv", None)
        mock_download.side_effect = DownloadError("Video unavailable")

        result = runner.invoke(cli, ['https://x.com/i/spaces/1ZkKzZWnVRRKv'])

        assert result.exit_code == 2
        assert "Video unavailable" in result.output

Step 4: Run test to verify it passes

Run: pytest tests/test_main.py::test_cli_download_failure -v Expected: PASS (implementation already handles this)

Step 5: Write test for cookies option

Add to tests/test_main.py:

def test_cli_with_cookies(tmp_path):
    """Test CLI with cookies file."""
    runner = CliRunner()

    # Create a fake cookies file
    cookies_file = tmp_path / "cookies.txt"
    cookies_file.write_text("# Netscape HTTP Cookie File\n")

    with patch('src.main.validate_space_url') as mock_validate, \
         patch('src.main.download_space') as mock_download:

        mock_validate.return_value = (True, "1ZkKzZWnVRRKv", None)
        mock_download.return_value = str(tmp_path / "space_1ZkKzZWnVRRKv.mp3")

        result = runner.invoke(cli, [
            'https://x.com/i/spaces/1ZkKzZWnVRRKv',
            '--cookies', str(cookies_file)
        ])

        assert result.exit_code == 0
        mock_download.assert_called_once()
        assert mock_download.call_args[1]['cookies_file'] == str(cookies_file)

Step 6: Run all CLI tests

Run: pytest tests/test_main.py -v Expected: All tests PASS

Step 7: Commit

git add tests/test_main.py
git commit -m "test: add CLI error cases and cookies option tests"

---

Task 8: README Documentation

Files:

• Create: README.md

Step 1: Create comprehensive README

# X Space Downloader

A Python CLI tool that downloads X (Twitter) Space recordings and outputs them as MP3 files.

## Features

- Download X Space recordings as MP3 files
- Support for authenticated downloads using cookies
- Simple command-line interface
- Clear error messages

## Installation

### Prerequisites

- Python 3.8 or higher
- ffmpeg (required by yt-dlp for audio conversion)

Install ffmpeg:
```bash
# macOS
brew install ffmpeg

# Ubuntu/Debian
sudo apt install ffmpeg

# Windows
# Download from https://ffmpeg.org/download.html

Install the tool

git clone https://github.com/synapz-org/x-space-downloader.git
cd x-space-downloader
pip install -e .

Usage

Basic usage

x-space-dl https://x.com/i/spaces/1ZkKzZWnVRRKv

This will download the Space recording and save it as space_1ZkKzZWnVRRKv.mp3 in the current directory.

With authentication (recommended)

For better reliability and access to private Spaces, provide cookies from your logged-in X session:

x-space-dl --cookies cookies.txt https://x.com/i/spaces/1ZkKzZWnVRRKv

Specify output directory

x-space-dl --output ~/Downloads https://x.com/i/spaces/1ZkKzZWnVRRKv

Full options

x-space-dl --help

Exporting Cookies

To download Spaces that require authentication, you need to export cookies from your logged-in X session.

Method 1: Browser Extension (Easiest)

1. Install a cookie export extension:

   • Chrome/Edge: Get cookies.txt LOCALLY
   • Firefox: cookies.txt

2. Log in to X (twitter.com or x.com)

3. Click the extension icon and export cookies

4. Save as cookies.txt

Method 2: Manual Export (Advanced)

1. Log in to X
2. Open Developer Tools (F12)
3. Go to Application/Storage → Cookies
4. Copy cookies in Netscape format to a file

Requirements

• Python 3.8+
• yt-dlp
• click
• ffmpeg (system dependency)

Supported URLs

The tool supports X Space URLs in these formats:

• https://x.com/i/spaces/{SPACE_ID}
• https://twitter.com/i/spaces/{SPACE_ID}

Note: Only recordings are supported. Live Spaces must be recorded before downloading.

Error Messages

Error	Meaning
"Invalid URL format"	URL doesn't match X Space format
"Failed to download Space"	Recording not found or network error
"Authentication required"	Space requires login (use --cookies)

Development

Running tests

pip install pytest pytest-mock
pytest

Project structure

x-space-downloader/
├── src/
│   ├── main.py          # CLI entry point
│   ├── downloader.py    # yt-dlp wrapper
│   └── validator.py     # URL validation
├── tests/
│   ├── test_main.py
│   ├── test_downloader.py
│   └── test_validator.py
└── docs/
    └── plans/           # Design documents

License

MIT License - See LICENSE file for details

Contributing

Contributions are welcome! Please open an issue or submit a pull request.

Troubleshooting

"ffmpeg not found"

Install ffmpeg using your system's package manager (see Installation section).

"Video unavailable"

The Space recording may have been deleted or is private. Try using the --cookies option.

"Authentication required"

Export cookies from your logged-in X session and use the --cookies option.

Acknowledgments

Built with:

• yt-dlp - Media downloader
• Click - CLI framework

**Step 2: Commit README**

```bash
git add README.md
git commit -m "docs: add comprehensive README with usage and installation"

---

Task 9: Development Dependencies

Files:

• Create: requirements-dev.txt

Step 1: Create development requirements

pytest>=7.0.0
pytest-mock>=3.10.0

Step 2: Commit

git add requirements-dev.txt
git commit -m "chore: add development dependencies"

---

Task 10: Integration Test

Files:

• Create: tests/test_integration.py

Step 1: Write integration test

Create tests/test_integration.py:

import pytest
from click.testing import CliRunner
from src.main import cli


@pytest.mark.integration
def test_full_workflow_with_mocks(tmp_path):
    """Integration test for complete workflow."""
    runner = CliRunner()

    from unittest.mock import patch

    with patch('src.downloader.yt_dlp.YoutubeDL') as mock_ytdl_class:
        # Mock successful download
        mock_ytdl = mock_ytdl_class.return_value.__enter__.return_value
        mock_ytdl.extract_info.return_value = {'id': '1ZkKzZWnVRRKv'}

        # Run CLI
        result = runner.invoke(cli, [
            'https://x.com/i/spaces/1ZkKzZWnVRRKv',
            '--output', str(tmp_path)
        ])

        # Verify success
        assert result.exit_code == 0
        assert "Successfully downloaded" in result.output

        # Verify yt-dlp was called correctly
        mock_ytdl.extract_info.assert_called_once()
        call_args = mock_ytdl_class.call_args
        assert 'bestaudio' in call_args[0][0]['format']
        assert 'mp3' in call_args[0][0]['postprocessors'][0]['preferredcodec']

Step 2: Run integration test

Run: pytest tests/test_integration.py -v Expected: PASS

Step 3: Run all tests

Run: pytest -v Expected: All tests PASS

Step 4: Commit

git add tests/test_integration.py
git commit -m "test: add integration test for complete workflow"

---

Task 11: Push to GitHub

Files:

• N/A (git operations)

Step 1: Push all commits to GitHub

git push -u origin master

Expected: All commits pushed successfully

---

Verification Steps

After completing all tasks:

1. Install the package:

   pip install -e .
   

2. Run all tests:

   pytest -v
   

   Expected: All tests pass

3. Test CLI is installed:

   x-space-dl --help
   

   Expected: Help text displays

4. Test with invalid URL:

   x-space-dl https://invalid.com/test
   

   Expected: Error message about invalid URL format

Notes for Implementation

• TDD strictly: Write test first, watch it fail, implement, watch it pass
• Commit frequently: After each passing test
• YAGNI: Don't add features not in the design (no GUI, no batch downloads, etc.)
• DRY: Reuse validator and downloader functions, don't duplicate logic
• Error messages: Keep them user-friendly and actionable
• Testing: Mock yt-dlp in tests to avoid actual network calls