# OpenTranscode v4.4.3 — Production Release

**v4.4.3 fixes the crash + adds a UI toggle for av1an.**

## What changed

### 1. Fixed the AttributeError crash

v4.4.2's launcher script crashed on START with:
```
AttributeError: 'EncoderWorker' object has no attribute 'verbose'
```

The launcher script's `EncoderWorker.__init__` never set `self.verbose`,
so when `_check_disk_space` referenced it, the worker thread crashed.
Fixed — `self.verbose` is now set from `env.av1an_flags["verbose"]` in
both the package and the launcher script.

### 2. UI toggle for av1an

Instead of the `--use-av1an` CLI flag (which was weird to require), the
GUI now has an **"av1an (chunk-parallel)"** checkbox in the options row
next to "Force (skip validation)" and "Delete source after verify".

- **Default OFF** (unchecked) = ffmpeg-only, the reliable default
- **ON** = av1an chunk-parallel, for users with a working VapourSynth setup

The CLI flag `--use-av1an` still works (for scripting), but the UI
toggle takes precedence when set.

### 3. Fixed misleading "single-file script" terminology

The docs called `open-transcode.py` "the single-file script" — but it's
a 6,500-line launcher that mirrors the `opentranscode/` package (16
modules). Calling it "single-file" was misleading. Now consistently
referred to as "the launcher script" throughout docs and comments.

---

# OpenTranscode v4.4.2 — Production Release

**v4.4.2 kills the live tail spam and the confusing "FAIL then OK" double-status.**

## What changed

Two fixes for issues found in v4.4.1 testing:

### 1. Live tail gated behind `--verbose` (launcher script)

The package had the live tail (`│ Encoding: 1373/1376 Frames @ 51.70 fps...`)
gated behind `--verbose` since v4.1.1. The launcher script (which the
tests run against) never got that gate — it was emitting every frame of
SVT-AV1's progress bar to the log in quiet mode. Now gated.

### 2. "FAIL: av1an exit code 1" moved to `_vlog`

When av1an fails but the ffmpeg fallback succeeds, the user was seeing:
```
[22/180] file.mkv — FAIL: av1an exit code 1
[22/180] file.mkv — OK: 1.6MB -> 1.3MB (81%)
```

Two status lines for one file — one FAIL, one OK. Confusing. Now the
av1an failure is logged to `_vlog` (verbose only), and the user only
sees the final outcome:
```
[22/180] file.mkv — OK: 1.6MB -> 1.3MB (81%)
```

If both av1an AND ffmpeg fail, the user sees:
```
[22/180] file.mkv — FAIL: av1an + ffmpeg both failed
```

One line. Clear outcome.

### Note on `--use-av1an`

The log that revealed these issues showed av1an running — which means
`--use-av1an` was passed. The default (ffmpeg-only) doesn't hit the
av1an concat bug at all. If you're seeing av1an failures, drop the
`--use-av1an` flag and use the default ffmpeg path.

---

# OpenTranscode v4.4.1 — Production Release

**v4.4.1 shuts up. Two lines per file: start, finish. Nothing else.**

## What changed

v4.4.0 tried to be helpful with heartbeats and disk-space warnings.
The user told us to shut up. So we did.

**Default log output is now exactly:**

```
Found 180 file(s) to process.
[1/180] filename.mkv
[1/180] filename.mkv — OK: 1.6MB -> 1.3MB (81%)
[2/180] already_done.mkv — SKIP (already av1/opus)
[3/180] next.mkv
[3/180] next.mkv — OK: 2.4MB -> 1.8MB (75%)
...
QUEUE COMPLETE. Success: 178, Failed: 0, Skipped: 2.
```

That's it. Two lines per file that encodes (start + finish). One line
per file that's skipped or fails. No heartbeats. No disk-space
warnings. No per-frame chatter. No "still encoding" messages.

**What's gated behind `--verbose`:**
- Heartbeat (`... 30s elapsed`) — was always-on since v4.3.0, now opt-in
- Disk-space warnings (`WARN: low disk space on output...`) — now opt-in
- Live tail of ffmpeg/av1an stderr — already opt-in since v4.1.1
- CMD: lines, DIAGNOSIS blocks, resolution map — already opt-in since v4.2.1
- Pre-flight validation table — already opt-in since v4.2.1

**If a 10-hour encode looks hung without the heartbeat**, run with
`--verbose` to see it. Or just trust that it's working — ffmpeg
doesn't hang, it just takes a long time.

```bash
opentranscode              # quiet: start + finish only
opentranscode --verbose    # full detail: heartbeats, disk warnings, CMD, etc.
```

---

# OpenTranscode v4.4.0 — Production Release

**v4.4.0 adds massive-file support (30GB+ BluRay rips) and reduces log noise.**

## The v4.4.0 changes

### 1. Massive-file support

Three changes to prevent failures on 30GB+ source files:

**Per-file timeout raised from 2h to 24h (configurable via `--timeout`)**

A 30GB 1080p BluRay rip at SVT-AV1 preset 6 (~5-10 fps) on a 2-hour
movie takes 4-10 hours. The old 7200s (2h) timeout killed massive-file
encodes partway through. New default is 86400s (24h). Configurable:

```bash
opentranscode --timeout 36000    # 10 hours per file
opentranscode --timeout 0        # no timeout (not recommended)
```

**5%-of-source integrity check replaced with absolute 1KB minimum**

The old check rejected outputs smaller than 5% of the source size. For
a 50GB high-bitrate BluRay source, 5% = 2.5GB — but valid AV1 at CRF 32
produces 1-2GB for a 2-hour movie. This caused false "output too small"
failures on valid encodes of massive files.

The new check uses an absolute 1KB minimum (a valid container header
alone is ~1KB; anything below is definitely corrupt). The real
integrity gate is the duration check in `_verify_and_finalize` (output
must be >= 95% of source duration).

**Disk space pre-check (warn, not abort)**

Before encoding each file > 1 GB, the worker checks free space on the
output and temp partitions:

```
[N/180] big_movie.mkv
  WARN: low disk space on output (5.2 GB free, source is 30.0 GB) — encode may fail partway through
  ... 30s elapsed
  ...
```

When scaling, also checks the temp partition (the lossless intermediate
can be 2-3x source size):

```
  WARN: low disk space on temp (20.0 GB free, lossless intermediate may need ~60.0 GB) — consider scaling to a smaller resolution or freeing space
```

This is a WARNING, not an abort — the user might be encoding to a
different partition or know the output will be smaller. If the encode
fails partway due to ENOSPC, the existing error handling catches it.

### 2. Reduced log noise

**Combined status lines** — every file now produces ONE line instead of two:

Before (v4.3.0):
```
[1/180] filename.mkv
  OK: 1.6MB -> 1.3MB (81%)
```

After (v4.4.0):
```
[1/180] filename.mkv — OK: 1.6MB -> 1.3MB (81%)
```

Same for SKIP and FAIL:
```
[2/180] already_done.mkv — SKIP (already av1/opus)
[3/180] broken.mkv — FAIL: not a valid video (ffprobe could not read it)
```

**Disk-space warnings no longer fire for skipped files** — the check
moved after the skip-existing check, so a folder of 175 already-encoded
files produces 175 SKIP lines, not 175 SKIP lines buried in 175 disk-
space warnings.

**Heartbeat stays user-facing** (fixed in v4.3.0) — one line per 30
seconds during long encodes:
```
[1/180] big_movie.mkv
  ... 30s elapsed
  ... 60s elapsed
  ... 90s elapsed
[1/180] big_movie.mkv — OK: 30.2GB -> 2.1GB (7%)
```

The `[N/total] filename` banner appears once at the start of the encode
(so the user knows what's being processed), heartbeats fire every 30s
(so the user knows it's alive), and the final OK/FAIL line repeats the
prefix (so the user can match status to file at a glance).

### What v4.4.0 keeps

- **Skip-existing detection** (v4.3.0) — probes output with ffprobe,
  skips files whose codec matches
- **Live progress tail + heartbeat** (v4.1.1, fixed in v4.3.0) —
  av1an/ffmpeg stderr in `--verbose` mode, heartbeat always on
- **`--use-av1an` opt-in** (v4.2.0) — ffmpeg is the default encode path
- **`--verbose`** (v4.2.1) — full tech detail (CMD:, DIAGNOSIS, etc.)
- **`--force-reencode`** (v4.3.0) — bypass skip-existing
- **`--skip-existing` / `--force-reencode`** (v4.3.0) — defaults to skip

### Usage

```bash
opentranscode                            # default: ffmpeg, skip-existing, 24h timeout
opentranscode --timeout 36000            # 10h per-file timeout
opentranscode --force-reencode           # re-encode even if output exists
opentranscode --use-av1an                # opt into av1an chunk-parallel
opentranscode --verbose                  # full tech detail
```

---

# OpenTranscode v4.3.0 — Production Release

**v4.3.0 adds skip-existing detection + fixes the v4.2.1 heartbeat regression.**

## The v4.3.0 changes

### 1. Skip-existing detection (the big feature)

When you re-run OpenTranscode on a folder you've already encoded, it now
**skips files whose output already exists with a matching codec** instead
of re-encoding them from scratch.

Detection strategy (all must pass):
1. Output file exists at the computed path (`<stem>_archived.<ext>`)
2. ffprobe can read it (not corrupt)
3. Video stream `codec_name` matches the selected encoder (`av1`/`vp9`/`hevc`)
4. Audio stream `codec_name` matches the selected audio profile (`opus`/`vorbis`/`flac`)
5. If scaling was requested, output resolution matches the target

**What is NOT verified** (and why):
- CRF/preset — these are encoder settings, not reliably stored in container
  metadata. To re-encode at a different CRF with the same codec, use
  `--force-reencode`.
- Audio bitrate — varies per file based on loudness normalization.

**CLI flags:**
```bash
opentranscode                    # skip-existing ON (default)
opentranscode --force-reencode   # re-encode everything, even if output exists
```

**Log output in quiet mode:**
```
[1/180] already_encoded.mkv
  SKIP: already encoded (av1/opus)
[2/180] new_file.mkv
  ... 30s elapsed
  OK: 1.6MB -> 1.3MB (81%)
...
QUEUE COMPLETE. Success: 45, Failed: 0, Skipped: 135.
```

Skipped files are counted separately — they're NOT successes (nothing
was encoded) and NOT failures (the output exists and matches).

### 2. Heartbeat regression fix

v4.2.1 gated the 30-second heartbeat behind `--verbose`, which caused
the "hangs on first transcode, forever timer" symptom in quiet mode.
The user saw:

```
[1/180] filename.mkv
```

...and nothing else for the entire encode duration. Looked exactly like
a hang.

**v4.3.0 fix**: the heartbeat is **always user-facing** (quiet or
verbose). The live tail of `frame= 67 fps= 12...` stays gated behind
`--verbose` (that IS noisy), but the heartbeat is the "is it alive?"
signal — one line per 30 seconds, not "absurd amounts of useless info."

Quiet mode now shows:
```
[1/180] filename.mkv
  ... 30s elapsed
  ... 60s elapsed
  ... 90s elapsed
  OK: 1.6MB -> 1.3MB (81%)
```

One line per 30 seconds + the success line. Not noisy. Not silent.

### What the skip detection does NOT cover

The detection is conservative — if in doubt, re-encode:
- No ffprobe available → re-encode (can't verify codec)
- Output file exists but ffprobe can't read it → re-encode (treat as corrupt)
- Video codec matches but audio doesn't → re-encode
- Codec matches but resolution doesn't (when scaling requested) → re-encode

This means the only time a file is skipped is when we're **confident**
the existing output matches your current encoder selection. If you
switch from AV1 to x265, all files re-encode. If you switch from Opus
to Vorbis audio, all files re-encode. If you change the target
resolution, all files re-encode.

---

# OpenTranscode v4.2.1 — Production Release

**v4.2.1 makes the log quiet by default. Tech detail is gated behind
`--verbose`.**

*(Note: v4.3.0 fixes the heartbeat regression introduced here — the
heartbeat is now always user-facing, not gated behind `--verbose`.)*

## The v4.2.1 change

### Why

v4.2.0's log was filling up with thousands of lines of useless info
per file: the `CMD:` banner, ffmpeg's `frame= 67 fps= 12 q=32 ...`
progress chatter, the resolution map, the pre-flight validation table,
the DIAGNOSIS blocks, the 30s heartbeat, the RETRY messages. For a
180-file run that's thousands of lines nobody reads.

The user just needs: did it encode or not?

### What changed

**Default log output is now quiet.** Per file, you see exactly two
lines:

```
[1/180] filename.mkv
  OK: 1.6MB -> 1.3MB (81%)
```

Or on failure:

```
[2/180] next.mkv
  FAIL: ffmpeg exit code 1
```

Final summary stays:

```
QUEUE COMPLETE. Success: 178, Failed: 2.
```

### What's gated behind `--verbose`

- `Found N file(s) to process.` + `Temp dir:` banner
- `─── FILE RESOLUTION MAP ───` + per-file resolution lines
- `─── PRE-FLIGHT VALIDATION ───` + valid/invalid counts
- `  Mode: ffmpeg (...)`, `  Chunking: select ...`
- `  CMD: av1an -i ... --workers 4 --chunk-method select ...`
- `  Source: 1920x1080 -> Output: 1920x1080`
- Live tail of av1an/ffmpeg stderr (`│ frame= 67 fps= 12 ...`)
- `... still encoding (30s elapsed)` heartbeat (every 30s)
- `DIAGNOSIS: ...` blocks (root-cause analysis for av1an failures)
- `  ─── av1an stderr (last 25 lines) ───` dumps
- `  RETRY: ...` / `  RETRY OK: ...` / `  RETRY FAIL: ...` messages
- `  Pre-scale OK (... MB intermediate)`
- `  SUBS:` mux messages
- `CLEANUP: Swept N residual temp file(s)...`
- `  File type: ...` (yt-dlp-download detection)

### What stays user-facing (quiet mode)

- `Found N file(s) to process.` — single banner line
- `[N/total] filename` — per-file header
- `  OK: XMB -> YMB (Z%)` — success
- `  FAIL: <short reason>` — failure (one line, not a multi-line dump)
- `  SKIP: <short reason>` — skipped file (no video stream, too short, etc.)
- `  STOP: skipping remaining files (<reason>)` — when queue auto-aborts
- `CLEANED: Removed N source file(s).` — after delete-source transcodes
- `QUEUE COMPLETE. Success: N, Failed: M.` — final summary
- `ABORT: All N file(s) are invalid. Aborting queue.` — when pre-flight finds zero valid files

### Usage

```bash
opentranscode                    # quiet (default)
opentranscode --verbose          # full tech detail
opentranscode --use-av1an        # opt into av1an chunk-parallel
opentranscode --use-av1an --verbose   # full tech detail + av1an
```

---

# OpenTranscode v4.2.0 — Production Release

**v4.2.0 makes ffmpeg the default encode path. av1an chunk-parallel is
now opt-in via `--use-av1an`.**

## The v4.2.0 change

### Why

v4.1.x tried to make av1an chunk-parallel work reliably. It doesn't,
across distros. Failure modes observed in production:

- **`Unprocessed tokens: --threads`** (v4.1.0/v4.1.1) — `SvtAv1EncApp`
  CLI doesn't accept `--threads`, only `--lp`. Every chunk failed 3x.
- **No av1an output for 270+ seconds** (v4.1.2) — av1an wedged at
  startup or its stderr is buffered and won't flush until exit.
- **y4m pipe breaks** (v4.0.0) — Hybrid chunk method fails on phone-
  recorded MP4s with sparse keyframes.
- **VapourSynth plugin issues** — `lsmash`/`ffms2`/`bestsource` are
  separate packages that most distros don't install by default.

Each fix uncovered a new failure mode. The pattern is clear: av1an is
too fragile to be the default.

### What changed

**Default encode path is now ffmpeg-only.** The av1an pre-flight smoke
test is skipped entirely. `_on_run_clicked` sets `use_ffmpeg_fallback =
True` directly, and `EncoderWorker` runs the `_ffmpeg_fallback_encode`
path for every file.

This means:
- **No VapourSynth dependency** — ffmpeg invokes `libsvtav1` as a
  library, no `SvtAv1EncApp` subprocess, no `libvapoursynth-script.so`.
- **No chunk-method selection** — single-pass ffmpeg per file.
- **No `--threads` CLI quirks** — `-threads` is a valid ffmpeg/libsvtav1
  library option.
- **Immediate progress output** — ffmpeg's progress bar flushes to
  stdout line-by-line, picked up by the live tail (v4.1.1).
- **Slower than chunk-parallel av1an** — single-pass, no scene-split
  parallelism. But it actually completes, which is the only thing that
  matters.

### The `--use-av1an` flag

Users who specifically want av1an chunk-parallel (e.g. they have a
known-good VapourSynth + lsmash/ffms2 setup) can opt in:

```bash
opentranscode --use-av1an
```

When set, the full av1an pre-flight + smoke test runs as before, and
the encode uses the chunk-parallel path. When not set (default), the
smoke test is skipped and ffmpeg is used directly.

### What v4.2.0 keeps

- **Live progress tail** (v4.1.1) — ffmpeg's stdout/stderr emits to the
  GUI log as it arrives, with `\r` progress bar handling.
- **30-second heartbeat** (v4.1.1) — `... still encoding (Xs elapsed)`.
- **`--max-workers` / `--threads-per-worker` CLI flags** (v4.1.0) —
  still wired through `env.av1an_flags`. `--threads-per-worker` is now
  actually useful in the ffmpeg path (libsvtav1 accepts `-threads`).
- **`--chunk-method`** (v4.0.0) — still honored when `--use-av1an` is
  set. No-op in the default ffmpeg path.

### What the ffmpeg command looks like

```
ffmpeg -i input.mkv \
  -c:v libsvtav1 -preset 6 -crf 32 -pix_fmt yuv420p10le -g 240 \
  -c:a libopus -b:a 64k \
  -y output.mkv
```

Simple, reliable, no surprises. The same command works on any distro
with `ffmpeg` compiled against `libsvtav1` (which is the default on
Arch, Debian, Ubuntu, Fedora).

---

# OpenTranscode v4.1.2 — Production Release

**v4.1.2 reverts the `--threads N` injection that broke av1an in v4.1.0/v4.1.1.**

*(Note: v4.2.0 makes ffmpeg the default, so this av1an issue is no
longer reachable unless you opt in with `--use-av1an`.)*

## The v4.1.2 fix (the actual root cause)

### What was broken

v4.1.0 added "intelligent chunking" that injected `--threads N` into
av1an's `--video-params` string, intending to cap each per-chunk encoder
instance to N threads. The theory was sound — without a cap, SVT-AV1's
default `--threads 0` means "use all logical cores," so 13 chunk-parallel
workers × 28 threads = ~364 threads on a 28-thread Xeon → kernel
scheduler drowned → hard lock.

The implementation was wrong. av1an invokes `SvtAv1EncApp` (the
standalone SVT-AV1 CLI binary) per-chunk, not the libsvtav1 library.
**`SvtAv1EncApp` does not accept `--threads`** — it uses `--lp N`
(logical processors) instead. The result, visible in the user's log:

```
Svt[info]: -------------------------------------------
Svt[info]: SVT [version]: SVT-AV1 Encoder Lib v4.2.0
Svt[info]: -------------------------------------------
Unprocessed tokens: --threads
Unprocessed arguments: 6
Error in configuration, could not begin encoding!
```

Every chunk failed 3 times with this error → av1an exited with code 1
→ no output file → OpenTranscode fell back to ffmpeg → ffmpeg succeeded
(because it invokes libsvtav1 as a library, where `-threads` IS valid).
But this happened *per file*, making every encode go through the slow
single-pass ffmpeg fallback path instead of chunk-parallel av1an.

### The fix

**Don't inject `--threads` into av1an's `--video-params` at all.**
Revert `params_fn` to its v4.0.0 signature `(crf, preset) -> str` with
no threads parameter. Thread capping now happens via:

1. **av1an's `--workers` flag** (chunk-parallel count) — this is what
   `_compute_intelligent_worker_count` actually controls. Fewer workers
   = fewer concurrent SVT-AV1 processes = less thread pressure.
2. **`-threads` in the ffmpeg fallback path** — `_svtav1_ffmpeg_args` /
   `_vp9_ffmpeg_args` / `_x265_ffmpeg_args` already accept `-threads`,
   passed to libsvtav1/libvpx/libx265 as library options (where it
   works).

The intelligent worker-count logic from v4.1.0 is **kept** — it still
caps `--workers` to `physical_cores - 1` and reserves 1 logical thread
for OS/UI. But the per-chunk thread cap that broke SVT-AV1 is gone.

### What v4.1.2 keeps from v4.1.0/v4.1.1

- **Intelligent `--workers` count** — `_compute_intelligent_worker_count`
  still caps chunk-parallel workers based on CPU topology.
- **Live progress tail** (v4.1.1) — av1an's stdout/stderr now emits to
  the GUI log as it arrives, with `\r` progress bar handling.
- **30-second heartbeat** (v4.1.1) — `... still encoding (Xs elapsed)`
  so you always know the encode is alive.
- **`--max-workers` / `--threads-per-worker` CLI flags** — still wired
  through `env.av1an_flags`. `--max-workers` controls av1an's
  `--workers`. `--threads-per-worker` is currently a no-op in the
  av1an path (kept for future use if/when SVT-AV1's `--lp` flag is
  wired in correctly).

### What the av1an command looks like now

```
/usr/bin/av1an -i input.mkv --workers 4 --chunk-method select \
  --encoder svt-av1 \
  --video-params --preset 6 --crf 32 --keyint 240 \
  --audio-params -c:a libopus -b:a 64k \
  --concat mkvmerge -o output.mkv
```

No `--threads 6` in `--video-params` — SvtAv1EncApp accepts this cleanly.

---

# OpenTranscode v4.1.1 — Production Release

**v4.1.1 adds live progress tail + heartbeat** so you can see av1an is
working during long encodes. Also bumps threads-per-worker from 4 to 6
for better SVT-AV1 per-chunk throughput.

*(Note: the threads-per-worker bump in v4.1.1 was reverted in v4.1.2 —
see above.)*

## The v4.1.1 fix (in detail)

### The bug

v4.1.0 capped threads per encoder instance at 4 to prevent the
thread-oversubscription hard-lock. That worked (no more hard locks),
but it made each chunk ~7x slower than v4.0.0's "grab all 28 threads"
behavior. The result: file 1 took 10+ minutes, and the user saw
**nothing** in the GUI log the entire time.

The "nothing" was the real killer. The drainer threads in
`_run_with_stop_check` read av1an's stdout/stderr into StringIO
buffers but **only emitted them to the GUI log on process exit**. So
during a 10-minute encode, the user stared at:

```
[1/46] Encoding: video1.mp4
  Source: 1920x1080  ->  Output: 1920x1080
  Chunking: select (av1an default if no override)
  CMD: av1an -i ... --workers 4 --chunk-method select ...
```

...and nothing else for 10 minutes. Looked identical to a wedged
process. User assumed it was "borked" and killed it.

### The fix (three parts)

1. **Live tail** — the drainer now emits each line of av1an's
   stdout/stderr to the GUI log **as it arrives**, prefixed with `│ `
   to distinguish from orchestrator messages. Handles both `\n` (log
   lines like "scenecut: found 8 scene(s)") and `\r` (progress bar
   updates like "Encoding 45%") as line boundaries, so av1an's progress
   bar renders correctly in real-time.

2. **30-second heartbeat** — the poll loop emits
   `... still encoding (Xs elapsed)` every 30 seconds, so even if
   av1an isn't producing line-delimited output (e.g. during a long
   SVT-AV1 encode that only updates a `\r` progress bar), the user
   knows the process is alive.

3. **`IDEAL_THREADS_PER_WORKER` bumped from 4 to 6** — SVT-AV1 with
   only 4 threads was too slow per-chunk. With 6 threads, each chunk
   gets ~50% better throughput while staying under the logical-thread
   budget. On a 28-thread Xeon, this changes the split from 6×4=24
   to 4×6=24 (same total, better per-chunk latency — the first chunk
   completes sooner, so the user sees progress faster).

### What the log looks like now

```
[1/46] Encoding: video1.mp4
  Source: 1920x1080  ->  Output: 1920x1080
  Chunking: select (av1an default if no override)
  CMD: av1an -i ... --workers 4 --chunk-method select ...
  │ INFO encode_file: Input: 1920x1080 @ 29.763 fps, YUVJ420P, SDR
  │ INFO encode_file: scenecut: found 8 scene(s)
  │ DEBUG encode_file: Segmenting video
  │ DEBUG encode_file: Segment done
  │ INFO encode_chunk: Encoding chunk 1
  ... still encoding (30s elapsed)
  │ INFO encode_chunk: Encoding chunk 2
  ... still encoding (60s elapsed)
  │ SUMMARY ------------------------------------------
  │ Average Speed: 4.231 fps
  SUCCESS: video1.mp4 (245.3MB -> 28.7MB, 12%)
```

### Worker-count math, updated for v4.1.1

| Machine                    | budget | workers | threads | active | reserved |
|----------------------------|--------|---------|---------|--------|----------|
| 4-core / 8-thread laptop   | 7      | 1       | 7       | 7      | 1        |
| 8-core / 16-thread desktop | 15     | 2       | 7       | 14     | 1        |
| **14-core / 28-thread Xeon** | 27   | **4**   | **6**   | **24** | **4**    |
| 32-core / 64-thread EPYC   | 63     | 10      | 6       | 60     | 4        |
| 1-core / 2-thread VM       | 1      | 1       | 1       | 1      | 1        |

---

# OpenTranscode v4.1.0 — Production Release

**v4.1.0 adds intelligent chunking** to prevent the thread-oversubscription
hard-lock that v4.0.0 hit on high-core-count machines.

## The v4.1.0 fix (in detail)

### The bug

v4.0.0 introduced the `--chunk-method select` auto-override for phone-
recorded MP4s with sparse keyframes (see the v4.0.0 section below for
that fix). But `select` mode keeps the encode pipeline tighter than
the previous `hybrid` default — chunks warm up faster, more encoder
instances hit full tilt at the same instant.

Meanwhile, `EncoderWorker.run()` was still computing
`worker_count = max(1, physical_cores - 1)` and passing no per-chunk
thread cap to the encoder. SVT-AV1's default `--threads 0` means "use
all logical cores," so each chunk-parallel worker spawned an
SvtAv1EncApp process that grabbed every logical thread.

On a 28-thread Xeon (14 physical cores) with 13 chunk-parallel workers,
the math was: `13 × 28 = ~364 active threads on 28 logical CPUs`. The
kernel scheduler drowned, I/O wait escalated, and the box hard-locked
even though no single process was at fault. The 1-second STOP-button
poll in `_run_with_stop_check` couldn't get scheduled, so even clicking
STOP didn't recover it.

### The fix (three parts)

1. **`_compute_intelligent_worker_count()`** in `EncoderWorker` now
   computes `(worker_count, threads_per_worker)` such that
   `worker_count * threads_per_worker <= logical_threads - 1` (one
   logical thread reserved for OS / UI / av1an orchestrator). The
   budget is split using an ideal `4 threads per worker` — the empirical
   sweet spot for SVT-AV1, x265, and vpxenc. Beyond ~6 threads per
   encoder instance you hit memory-bandwidth contention and diminishing
   returns.

2. **The resolved `threads_per_worker` is passed to `params_fn`** so each
   `_av1_params` / `_vp9_params` / `_x265_params` appends
   `--threads N` to the encoder's `--video-params` string. Every per-
   chunk encoder instance (SvtAv1EncApp / vpxenc / x265) now respects
   its share of the thread budget instead of grabbing all cores.

3. **Two new CLI flags** let the user override the auto math when
   needed:

   ```bash
   opentranscode --max-workers 4                # cap chunk-parallel worker count
   opentranscode --threads-per-worker 6          # per-encoder thread cap
   opentranscode --max-workers 4 --threads-per-worker 6  # full override
   ```

   Both are also visible in `--dry-run` and the GUI env-probe banner.

### Worker-count math, by machine

| Machine                    | budget | workers | threads | active | reserved |
|----------------------------|--------|---------|---------|--------|----------|
| 4-core / 8-thread laptop   | 7      | 1       | 7       | 7      | 1        |
| 8-core / 16-thread desktop | 15     | 3       | 5       | 15     | 1        |
| **14-core / 28-thread Xeon** | 27   | **6**   | **4**   | **24** | **4**    |
| 32-core / 64-thread EPYC   | 63     | 15      | 4       | 60     | 4        |
| 1-core / 2-thread VM       | 1      | 1       | 1       | 1      | 1        |

The Xeon row is the user's box. v4.0.0 hit 13 × 28 = 364 threads →
hard lock. v4.1.0 hits 6 × 4 = 24 threads with 4 reserved for OS/UI.

### Also new in v4.1.0

- `--dry-run` now prints the computed worker math so you can verify
  the thread budget before launching a real encode.
- The GUI env-probe banner shows the same math on startup.
- `EncoderWorker.__init__` reads `env.av1an_flags["max_workers"]` and
  `["threads_per_worker"]` as fallback when the explicit constructor
  args aren't supplied — so the CLI flags reach the GUI-spawned
  worker without `ui_window.py` code changes.

### Backwards compatibility

- `params_fn` accepts an optional third arg `threads=0`. Existing
  callers passing `(crf, preset)` still work because `threads`
  defaults to 0 (the v4.0.0 behavior — no `--threads` flag, encoder
  auto-selects).
- The `_av1_params` / `_vp9_params` / `_x265_params` functions are
  byte-identical to v4.0.0 when `threads=0`.

---

# OpenTranscode v4.0.0 — Production Release

**v4.0.0 resolves the "works up until near the end, never saves chunks into a
full file" bug** that affected phone-recorded MP4s with sparse keyframes.

## The v4.0.0 fix (in detail)

### The bug

When no VapourSynth source plugins are installed (the common case on most
distros — `lsmash`, `ffms2`, `bestsource` are all separate packages),
av1an auto-selects the **Hybrid** chunk method. Hybrid does:

1. `ffmpeg -c copy -f segment` to split the source at scene boundaries
2. Re-decode each segment to y4m via a second ffmpeg invocation
3. Pipe the y4m to the encoder (SvtAv1EncApp / vpxenc / x265)

Phone-recorded MP4s (the `20190707_112725.11b774bacde3.mp4` files in
the production log) only have I-frames every 5–10 seconds. Scene
boundaries detected by av1an's `av_scenechange` rarely align with those
sparse keyframes. The segment muxer can only split on keyframes, so the
segment for scene N actually starts somewhere inside scene N-1's GOP.

The result: the decoder has no I-frame reference → errors with
`[h264 @ 0x...] error while decoding MB 35 25` → the y4m pipe breaks →
the encoder reads EOF mid-frame → `Failed to read y4m frame delimiter.
Read broken. EOF: 1` → every chunk fails after 3 retries → no chunks
to concat → **no output file**.

The previous ffmpeg fallback rescued the file, but it was slow (single-pass,
no chunk-parallel) and the diagnostic was misleading ("concat failure"
when it was actually a chunk-extraction failure).

### The fix (three parts)

1. **`_encode_one` now accepts a `chunk_method` parameter**. When av1an
   fails with the y4m break pattern, it recursively retries with
   `--chunk-method select`. Select uses VapourSynth's `select()` filter
   to extract frames one-by-one — slower than Hybrid but reliable for
   any file VapourSynth can open. This is faster than the ffmpeg
   fallback (chunk-parallel still works) and produces identical-quality
   output (same encoder, same params).

2. **The working chunk_method is cached** in
   `env.av1an_flags["chunk_method_override"]` so subsequent files skip
   the wasted first attempt.

3. **`env_probe` now probes for VapourSynth source plugins** via
   `_probe_vs_source_plugins()`. When NONE are found, it pre-sets
   `chunk_method_override = "select"` to avoid the wasted first attempt
   entirely. The probe checks `~/.local/lib/vapoursynth/`,
   `/usr/lib/vapoursynth/`, `/usr/local/lib/vapoursynth/`, and the
   Debian multiarch path.

### Also fixed

The "SUMMARY block + non-zero exit" diagnostic previously misdiagnosed
y4m break failures as "concat failure" (because SVT-AV1 prints a
SUMMARY block per-chunk before the pipe breaks). The check is now
guarded by `"Failed to read y4m frame delimiter" not in stderr_full` so
it only fires for true concat failures.

### New CLI flag

```bash
opentranscode --chunk-method {auto,select,hybrid,segment,ffms2,lsmash,bestsource,dgdecnv}
```

Lets the user force a specific chunk method. Useful for debugging or
for environments where the probe picks the wrong default. `auto` clears
any override the probe set.

## Quick start

### Option A: Install as a package (recommended)

```bash
cd /path/to/this/directory
pip install -e .                   # editable install (dev)
# OR
pip install -e ".[dev]"            # with pytest + build tools

# Now you can run it three ways:
opentranscode                       # console entry point
python -m opentranscode             # module entry point
python -m opentranscode --version   # → opentranscode 4.4.3
```

### Option B: Run the launcher script (backwards compat)

```bash
python open-transcode.py            # the launcher script still works
```

### Option C: Verify your environment without encoding

```bash
opentranscode --dry-run             # probe env + smoke test, no GUI, no encode
opentranscode --dry-run --chunk-method select  # preview a forced chunk method
opentranscode --dry-run --max-workers 4         # preview worker-count override
opentranscode --verify-only /path/to/existing_output.mkv   # re-verify an output
```

### Option D: Override the intelligent worker math (v4.1.0)

If the auto-computed thread budget still hard-locks the box (rare), or
if you have fast storage and want more parallelism, dial it manually:

```bash
opentranscode --max-workers 4                              # 4 chunks in parallel
opentranscode --threads-per-worker 6                       # 6 threads per encoder
opentranscode --max-workers 8 --threads-per-worker 2       # full override
```

Both flags are stored on `env.av1an_flags` and picked up by the
GUI-spawned worker — no UI changes needed.

## Files

| Path | Description |
|------|-------------|
| `opentranscode/` | **Package** — 16 modules. Importable as `import opentranscode`. |
| `open-transcode.py` | Launcher script (~6,500 lines). Kept for backwards compat + as the test target for the mocked tests. Mirrors the package behavior. |
| `pyproject.toml` | PEP 621 build config. Entry point: `opentranscode = opentranscode.__main__:main`. Ready for `pip install -e .` and `python -m build`. |
| `tests/` | 103 tests across 12 files. All pass in ~10 seconds. |
| `pytest.ini` | pytest config (also in pyproject.toml). |
| `README.md` | This file. |
| `logs/av1an.log.2026-07-13` | The production log that revealed the y4m break bug. Kept for reference. |

## Test suite

```bash
cd /path/to/this/directory
python -m pytest tests/ -v

# 103 tests, ~10 seconds:
#   25 mocked unit tests (smoke test, encoder pipeline, audio loudnorm, etc.)
#   12 real-ffmpeg e2e tests — requires ffmpeg + ffprobe
#   36 package-structure tests
#   13 chunk-method retry tests
#    1 chunk-method e2e recovery test
```

The e2e tests generate a real 2-second test video with ffmpeg, run the
full EncoderWorker pipeline on it (AV1→MKV, x265→MKV, VP9→WebM), and
verify the output file exists, is non-empty, has the correct codec, and
has the expected duration. The chunk-method e2e test additionally
simulates the y4m break pattern and verifies the retry-with-select
produces a valid output file.

## Verification (run these to confirm v4.0.0 works)

```bash
# 1. Package imports cleanly
python -c "import opentranscode; print(opentranscode.__version__)"  # → 4.0.0

# 2. CLI works
python -m opentranscode --version        # → opentranscode 4.0.0
python -m opentranscode --help           # → usage (includes --chunk-method)
python -m opentranscode --dry-run        # → env probe report (shows VS plugins + chunk method)

# 3. All tests pass
python -m pytest tests/ -q               # → 103 passed in ~10s

# 4. Install works
pip install -e .                         # → installs opentranscode + PySide6
opentranscode --version                  # → opentranscode 4.0.0

# 5. Launcher script still works (backwards compat)
python open-transcode.py                 # → launches GUI (if PySide6 + display)
```

## Publishing to PyPI

v4.0.0 is the production release. To publish:

```bash
python -m build               # produces dist/opentranscode-4.0.0.tar.gz + .whl
twine upload dist/*           # publishes to PyPI
```

## The production log that revealed the bug

The file `logs/av1an.log.2026-07-13` is the actual av1an log from the
user's production run that revealed the y4m break bug. Key markers:

```
INFO encode_file: Input: 1920x1080 @ 29.763 fps, YUVJ420P, SDR
INFO encode_file: scenecut: found 8 scene(s) [with extra_splits: 16 scene(s)]
DEBUG encode_file: Segmenting video
DEBUG encode_file: Segment done
INFO encode_file:
        Encoding          Failed to read y4m frame delimiter. Read broken. EOF: 1
        [h264 @ 0x55da365b30c0] error while decoding MB 35 25
WARN encode_chunk: Encoder failed (on chunk 11):
        Encoding          Failed to read y4m frame delimiter. Read broken. EOF: 1
        SUMMARY -----------------------------------------------------------------
        Average Speed:          2.501 fps
ERROR av1an_core::broker: [chunk 4] encoder failed 3 times, shutting down worker
```

The smoke test (line 1 of the log) succeeds because it uses
`chunk_method: Select` — but the real encode (line 4 onward) uses
`chunk_method: Hybrid` (av1an's auto-selection when no VS plugins are
installed) and fails on every chunk.