YouTube Scraper CLI

This is a command-line tool for downloading YouTube videos in various qualities, specifically transcoded to h.264 MP4 format.

Overview

The scrape tool allows users to easily download YouTube videos by providing a URL. It automatically fetches available video qualities, prompts the user for their choice, and handles the entire download and transcoding process. It is designed to be simple, efficient, and self-contained, requiring no manual installation of external software like ffmpeg.

Features

• Simple Video Downloads: Scrape any public YouTube video using its URL.
• Quality Selection: Automatically lists available MP4 video qualities (e.g., 1080p, 720p) and lets the user choose.
• h.264 MP4 Output: Downloads and merges streams into the standard h.264 MP4 format.
• Smart Output Location: Defaults to saving files in the user's Downloads folder on macOS. A custom output directory can be specified with the --output flag.
• Real-time Progress Bar: Displays a tqdm progress bar to monitor the download speed and progress for both video and audio streams.
• No External Dependencies: Automatically uses a Python-managed ffmpeg executable, so no system-wide installation is required.
• Flexible Usage: Supports two ways of providing the URL:
  1. As a direct command-line argument (requires quotes).
  2. Via an interactive prompt if the script is run without a URL.
• Simple Alias: Can be installed as a scrape command, making it accessible from anywhere in the terminal.

Workflow

The tool follows this sequence to process a video:

1. Initiation: The user runs the scrape command, either with a YouTube URL as an argument or without.
2. URL Input: If no URL is provided, the script prompts the user to paste one.
3. Quality Fetching: Using the yt-dlp library, the script contacts YouTube to get a list of all available video-only MP4 streams for the given URL.
4. User Selection: The script displays a clean list of resolutions (e.g., 1080p, 720p) and prompts the user to choose one.
5. Downloading: The tool initiates the download for the best video stream and the best audio stream (m4a) that match the selected quality.
6. Progress Tracking: Separate progress bars are shown for the video and audio downloads, indicating the percentage complete, download size, and speed.
7. Merging & Transcoding: Once the separate streams are downloaded, the script automatically uses a local ffmpeg executable (provided by the imageio-ffmpeg library) to merge them into a single, final MP4 file.
8. Cleanup: The temporary video and audio files are automatically deleted, leaving only the final MP4 file in the designated output folder.

Visual Workflow

graph TD
    A[Start] --> B{URL Provided?};
    B -- Yes --> D[Fetch Qualities];
    B -- No --> C[Prompt for URL];
    C --> D;
    D --> E[Display Qualities & Ask for Choice];
    E --> F[User Selects Quality];
    F --> G[Download Video & Audio Streams];
    G --> H{Show Progress Bars};
    H --> I[Merge Streams with FFmpeg];
    I --> J[Create Final MP4 File];
    J --> K[Cleanup Temporary Files];
    K --> L[End];

Loading

Installation

1. Prerequisites: Ensure you have Python 3 and pip3 installed on your system.

2. Clone the Repository (Example):

   git clone <your-repository-url>
   cd yt-scraper

3. Install Dependencies: Install all the required Python libraries from the requirements.txt file.

   pip3 install -r requirements.txt

4. Set Up the Command-Line Alias: To use the tool as a simple scrape command, move the script to a directory in your system's PATH and make it executable. You will likely need sudo for this.

   sudo mv /path/to/your/project/yt-scraper/scrape /usr/local/bin/
   sudo chmod +x /usr/local/bin/scrape

Usage

There are two primary ways to use the tool.

Method 1: Command-Line Argument

Pass the YouTube URL directly as an argument. You must wrap the URL in single quotes (' ') to prevent your shell from interpreting special characters in the URL.

# Download to the default Downloads folder
scrape 'https://www.youtube.com/watch?v=your_video_id'

# Download to a custom folder
scrape 'https://www.youtube.com/watch?v=your_video_id' --output /path/to/your/folder

Method 2: Interactive Prompt

Run the command without any arguments. It will prompt you to enter the URL.

scrape

It will then display:

Please paste the YouTube URL and press Enter: <paste-your-url-here>

Dependencies

• yt-dlp: The core library for downloading video content from YouTube.
• tqdm: Used to create and manage the visual progress bars.
• imageio-ffmpeg: Provides a local, executable version of ffmpeg for merging video and audio streams, avoiding the need for a system-wide installation.