How can I merge chunked Whisper WEBVTT transcriptions into a continuous, accurate SRT file?

Below is an outline of approaches you can take along with some sample code ideas.

---

Approach 1: Avoid Chopping if Possible

If your hardware/resources allow, the simplest solution is to run Whisper on the entire 60‑minute file. This avoids having to merge separate transcripts and eliminates any issues with overlapping segments or misaligned timestamps.

---

Approach 2: Chunking and Merging Transcripts

When you must split the audio (for example to save on memory or processing time), you’ll need to adjust the timestamps from each WEBVTT file so they “line up” correctly in the final .srt file. Here are some key ideas:

1. Include Timestamp Overlaps:
   To reduce the risk of missing words at segment boundaries, consider overlapping each chunk slightly. That way, you have a small region of overlap to help you verify that the transcripts merge well.

2. Adjust Timestamps:
   When processing your 6 WEBVTT files, parse the timestamps and add an offset for each chunk equal to the cumulative length of all previous segments. For example, the first file’s timestamps remain unmodified, the second file’s timestamps get an offset of 10 minutes, and so on.

3. Convert WEBVTT to SRT:
   The two formats are similar but require a couple of changes:

   • Remove WEBVTT header lines.
   • Use numbering (starting from 1) for each subtitle segment.
   • Convert the timestamps from the “HH:MM:SS.mmm” format (if needed, from dot to comma for SRT).

   Using a scripting language (Python in your case) can ease this conversion.

---

Example Python Script

Below is a sketch of how you might accomplish this using Python. (You might need to tweak it based on the exact format of your WEBVTT files.)

PYTHONCopied
import re
from datetime import timedelta

def parse_timestamp(timestamp_str):
    # Convert a timestamp string "HH:MM:SS.mmm" to a timedelta
    h, m, s_ms = timestamp_str.split(':')
    if '.' in s_ms:
        s, ms = s_ms.split('.')
    else:
        s, ms = s_ms, "0"
    return timedelta(hours=int(h), minutes=int(m), seconds=int(s), milliseconds=int(ms))

def format_timestamp(td):
    # Format a timedelta as "HH:MM:SS,mmm"
    total_seconds = int(td.total_seconds())
    ms = int(td.microseconds / 1000)
    hours = total_seconds // 3600
    minutes = (total_seconds % 3600) // 60
    seconds = total_seconds % 60
    return f"{hours:02}:{minutes:02}:{seconds:02},{ms:03}"

def merge_vtt_files(file_paths, output_srt_path):
    srt_entries = []
    index = 1
    time_offset = timedelta(0)
    
    # Regex to capture timestamp lines
    timestamp_regex = re.compile(r'(\d{2}:\d{2}:\d{2}\.\d{3})\s*-->\s*(\d{2}:\d{2}:\d{2}\.\d{3})')
    
    for file_path in file_paths:
        with open(file_path, encoding='utf-8') as f:
            lines = f.readlines()
        
        entry_lines = []
        current_entry = {}
        for line in lines:
            line = line.strip()
            if not line:
                # End of an entry: if timestamps have been captured, add them
                if "start" in current_entry and "end" in current_entry and current_entry.get("text"):
                    # Adjust the timestamps using the accumulated offset
                    start = parse_timestamp(current_entry["start"]) + time_offset
                    end = parse_timestamp(current_entry["end"]) + time_offset
                    # Build the SRT block
                    srt_block = f"{index}\n{format_timestamp(start)} --> {format_timestamp(end)}\n{current_entry['text']}\n"
                    srt_entries.append(srt_block)
                    index += 1
                current_entry = {}
                continue
            
            # Check if line is a timestamp line
            match = timestamp_regex.match(line)
            if match:
                current_entry["start"] = match.group(1)
                current_entry["end"] = match.group(2)
            else:
                # Handle text accumulation
                if "text" in current_entry:
                    current_entry["text"] += "\n" + line
                else:
                    current_entry["text"] = line

        # After processing one file, update the time_offset.
        # Assuming each chunk is exactly 10 minutes
        time_offset += timedelta(minutes=10)
    
    # Write to SRT file
    with open(output_srt_path, "w", encoding="utf-8") as out_file:
        out_file.write("\n".join(srt_entries))

# Example usage:
files = [
    "chunk1.vtt",
    "chunk2.vtt",
    "chunk3.vtt",
    "chunk4.vtt",
    "chunk5.vtt",
    "chunk6.vtt",
]

merge_vtt_files(files, "transcript.srt")

Notes on the script:

• This example assumes that each WEBVTT file is exactly 10 minutes long. If chunks vary in length, you can determine the offset by reading the last timestamp in each file.
• The regular expression and parsing might need adjustments depending on the precise formatting of your VTT files.
• Error checking and handling (e.g., empty files or formatting errors) have been omitted for brevity.

---

Alternative Libraries and Tools

• Python Libraries:
  There are libraries like webvtt-py which can load WEBVTT files. Once loaded, you can iterate over caption objects, adjust timing, and then output SRT using libraries like pysrt.

• NodeJS Libraries:
  If you prefer NodeJS, there are packages (e.g., vtt-to-srt converters) available on npm which can help you process VTT files. You’d similarly adjust the timestamps before merging.

---

Final Recommendations

• Evaluate the Need for Chunking:
  If possible, consider transcribing the entire file at once. This might not only simplify your pipeline but also provide more context for the transcription engine like Whisper.

• Handle Overlaps:
  When chunking, introduce a slight overlap between segments. Then, in the merge process, you can reconcile any repeated content at the boundaries.

• Test and Validate:
  After merging, review the final SRT file for timing consistency and text continuity. Small off-by-a-few-milliseconds errors might require tweaking your offsets or post-processing.

With these approaches, you can automate the merging process, ensuring that your final .srt file has continuous, correctly timed subtitles.