|
|
||
|---|---|---|
| opentranscode | ||
| tests | ||
| LICENSE | ||
| README.md | ||
| open-transcode.py | ||
| pyproject.toml | ||
| pytest.ini | ||
README.md
OpenTranscode v4.4.3 — Production Release
v4.4.3 fixes the crash + adds a UI toggle for av1an.
What changed
1. Fixed the AttributeError crash
v4.4.2's launcher script crashed on START with:
AttributeError: 'EncoderWorker' object has no attribute 'verbose'
The launcher script's EncoderWorker.__init__ never set self.verbose,
so when _check_disk_space referenced it, the worker thread crashed.
Fixed — self.verbose is now set from env.av1an_flags["verbose"] in
both the package and the launcher script.
2. UI toggle for av1an
Instead of the --use-av1an CLI flag (which was weird to require), the
GUI now has an "av1an (chunk-parallel)" checkbox in the options row
next to "Force (skip validation)" and "Delete source after verify".
- Default OFF (unchecked) = ffmpeg-only, the reliable default
- ON = av1an chunk-parallel, for users with a working VapourSynth setup
The CLI flag --use-av1an still works (for scripting), but the UI
toggle takes precedence when set.
3. Fixed misleading "single-file script" terminology
The docs called open-transcode.py "the single-file script" — but it's
a 6,500-line launcher that mirrors the opentranscode/ package (16
modules). Calling it "single-file" was misleading. Now consistently
referred to as "the launcher script" throughout docs and comments.
OpenTranscode v4.4.2 — Production Release
v4.4.2 kills the live tail spam and the confusing "FAIL then OK" double-status.
What changed
Two fixes for issues found in v4.4.1 testing:
1. Live tail gated behind --verbose (launcher script)
The package had the live tail (│ Encoding: 1373/1376 Frames @ 51.70 fps...)
gated behind --verbose since v4.1.1. The launcher script (which the
tests run against) never got that gate — it was emitting every frame of
SVT-AV1's progress bar to the log in quiet mode. Now gated.
2. "FAIL: av1an exit code 1" moved to _vlog
When av1an fails but the ffmpeg fallback succeeds, the user was seeing:
[22/180] file.mkv — FAIL: av1an exit code 1
[22/180] file.mkv — OK: 1.6MB -> 1.3MB (81%)
Two status lines for one file — one FAIL, one OK. Confusing. Now the
av1an failure is logged to _vlog (verbose only), and the user only
sees the final outcome:
[22/180] file.mkv — OK: 1.6MB -> 1.3MB (81%)
If both av1an AND ffmpeg fail, the user sees:
[22/180] file.mkv — FAIL: av1an + ffmpeg both failed
One line. Clear outcome.
Note on --use-av1an
The log that revealed these issues showed av1an running — which means
--use-av1an was passed. The default (ffmpeg-only) doesn't hit the
av1an concat bug at all. If you're seeing av1an failures, drop the
--use-av1an flag and use the default ffmpeg path.
OpenTranscode v4.4.1 — Production Release
v4.4.1 shuts up. Two lines per file: start, finish. Nothing else.
What changed
v4.4.0 tried to be helpful with heartbeats and disk-space warnings. The user told us to shut up. So we did.
Default log output is now exactly:
Found 180 file(s) to process.
[1/180] filename.mkv
[1/180] filename.mkv — OK: 1.6MB -> 1.3MB (81%)
[2/180] already_done.mkv — SKIP (already av1/opus)
[3/180] next.mkv
[3/180] next.mkv — OK: 2.4MB -> 1.8MB (75%)
...
QUEUE COMPLETE. Success: 178, Failed: 0, Skipped: 2.
That's it. Two lines per file that encodes (start + finish). One line per file that's skipped or fails. No heartbeats. No disk-space warnings. No per-frame chatter. No "still encoding" messages.
What's gated behind --verbose:
- Heartbeat (
... 30s elapsed) — was always-on since v4.3.0, now opt-in - Disk-space warnings (
WARN: low disk space on output...) — now opt-in - Live tail of ffmpeg/av1an stderr — already opt-in since v4.1.1
- CMD: lines, DIAGNOSIS blocks, resolution map — already opt-in since v4.2.1
- Pre-flight validation table — already opt-in since v4.2.1
If a 10-hour encode looks hung without the heartbeat, run with
--verbose to see it. Or just trust that it's working — ffmpeg
doesn't hang, it just takes a long time.
opentranscode # quiet: start + finish only
opentranscode --verbose # full detail: heartbeats, disk warnings, CMD, etc.
OpenTranscode v4.4.0 — Production Release
v4.4.0 adds massive-file support (30GB+ BluRay rips) and reduces log noise.
The v4.4.0 changes
1. Massive-file support
Three changes to prevent failures on 30GB+ source files:
Per-file timeout raised from 2h to 24h (configurable via --timeout)
A 30GB 1080p BluRay rip at SVT-AV1 preset 6 (~5-10 fps) on a 2-hour movie takes 4-10 hours. The old 7200s (2h) timeout killed massive-file encodes partway through. New default is 86400s (24h). Configurable:
opentranscode --timeout 36000 # 10 hours per file
opentranscode --timeout 0 # no timeout (not recommended)
5%-of-source integrity check replaced with absolute 1KB minimum
The old check rejected outputs smaller than 5% of the source size. For a 50GB high-bitrate BluRay source, 5% = 2.5GB — but valid AV1 at CRF 32 produces 1-2GB for a 2-hour movie. This caused false "output too small" failures on valid encodes of massive files.
The new check uses an absolute 1KB minimum (a valid container header
alone is ~1KB; anything below is definitely corrupt). The real
integrity gate is the duration check in _verify_and_finalize (output
must be >= 95% of source duration).
Disk space pre-check (warn, not abort)
Before encoding each file > 1 GB, the worker checks free space on the output and temp partitions:
[N/180] big_movie.mkv
WARN: low disk space on output (5.2 GB free, source is 30.0 GB) — encode may fail partway through
... 30s elapsed
...
When scaling, also checks the temp partition (the lossless intermediate can be 2-3x source size):
WARN: low disk space on temp (20.0 GB free, lossless intermediate may need ~60.0 GB) — consider scaling to a smaller resolution or freeing space
This is a WARNING, not an abort — the user might be encoding to a different partition or know the output will be smaller. If the encode fails partway due to ENOSPC, the existing error handling catches it.
2. Reduced log noise
Combined status lines — every file now produces ONE line instead of two:
Before (v4.3.0):
[1/180] filename.mkv
OK: 1.6MB -> 1.3MB (81%)
After (v4.4.0):
[1/180] filename.mkv — OK: 1.6MB -> 1.3MB (81%)
Same for SKIP and FAIL:
[2/180] already_done.mkv — SKIP (already av1/opus)
[3/180] broken.mkv — FAIL: not a valid video (ffprobe could not read it)
Disk-space warnings no longer fire for skipped files — the check moved after the skip-existing check, so a folder of 175 already-encoded files produces 175 SKIP lines, not 175 SKIP lines buried in 175 disk- space warnings.
Heartbeat stays user-facing (fixed in v4.3.0) — one line per 30 seconds during long encodes:
[1/180] big_movie.mkv
... 30s elapsed
... 60s elapsed
... 90s elapsed
[1/180] big_movie.mkv — OK: 30.2GB -> 2.1GB (7%)
The [N/total] filename banner appears once at the start of the encode
(so the user knows what's being processed), heartbeats fire every 30s
(so the user knows it's alive), and the final OK/FAIL line repeats the
prefix (so the user can match status to file at a glance).
What v4.4.0 keeps
- Skip-existing detection (v4.3.0) — probes output with ffprobe, skips files whose codec matches
- Live progress tail + heartbeat (v4.1.1, fixed in v4.3.0) —
av1an/ffmpeg stderr in
--verbosemode, heartbeat always on --use-av1anopt-in (v4.2.0) — ffmpeg is the default encode path--verbose(v4.2.1) — full tech detail (CMD:, DIAGNOSIS, etc.)--force-reencode(v4.3.0) — bypass skip-existing--skip-existing/--force-reencode(v4.3.0) — defaults to skip
Usage
opentranscode # default: ffmpeg, skip-existing, 24h timeout
opentranscode --timeout 36000 # 10h per-file timeout
opentranscode --force-reencode # re-encode even if output exists
opentranscode --use-av1an # opt into av1an chunk-parallel
opentranscode --verbose # full tech detail
OpenTranscode v4.3.0 — Production Release
v4.3.0 adds skip-existing detection + fixes the v4.2.1 heartbeat regression.
The v4.3.0 changes
1. Skip-existing detection (the big feature)
When you re-run OpenTranscode on a folder you've already encoded, it now skips files whose output already exists with a matching codec instead of re-encoding them from scratch.
Detection strategy (all must pass):
- Output file exists at the computed path (
<stem>_archived.<ext>) - ffprobe can read it (not corrupt)
- Video stream
codec_namematches the selected encoder (av1/vp9/hevc) - Audio stream
codec_namematches the selected audio profile (opus/vorbis/flac) - If scaling was requested, output resolution matches the target
What is NOT verified (and why):
- CRF/preset — these are encoder settings, not reliably stored in container
metadata. To re-encode at a different CRF with the same codec, use
--force-reencode. - Audio bitrate — varies per file based on loudness normalization.
CLI flags:
opentranscode # skip-existing ON (default)
opentranscode --force-reencode # re-encode everything, even if output exists
Log output in quiet mode:
[1/180] already_encoded.mkv
SKIP: already encoded (av1/opus)
[2/180] new_file.mkv
... 30s elapsed
OK: 1.6MB -> 1.3MB (81%)
...
QUEUE COMPLETE. Success: 45, Failed: 0, Skipped: 135.
Skipped files are counted separately — they're NOT successes (nothing was encoded) and NOT failures (the output exists and matches).
2. Heartbeat regression fix
v4.2.1 gated the 30-second heartbeat behind --verbose, which caused
the "hangs on first transcode, forever timer" symptom in quiet mode.
The user saw:
[1/180] filename.mkv
...and nothing else for the entire encode duration. Looked exactly like a hang.
v4.3.0 fix: the heartbeat is always user-facing (quiet or
verbose). The live tail of frame= 67 fps= 12... stays gated behind
--verbose (that IS noisy), but the heartbeat is the "is it alive?"
signal — one line per 30 seconds, not "absurd amounts of useless info."
Quiet mode now shows:
[1/180] filename.mkv
... 30s elapsed
... 60s elapsed
... 90s elapsed
OK: 1.6MB -> 1.3MB (81%)
One line per 30 seconds + the success line. Not noisy. Not silent.
What the skip detection does NOT cover
The detection is conservative — if in doubt, re-encode:
- No ffprobe available → re-encode (can't verify codec)
- Output file exists but ffprobe can't read it → re-encode (treat as corrupt)
- Video codec matches but audio doesn't → re-encode
- Codec matches but resolution doesn't (when scaling requested) → re-encode
This means the only time a file is skipped is when we're confident the existing output matches your current encoder selection. If you switch from AV1 to x265, all files re-encode. If you switch from Opus to Vorbis audio, all files re-encode. If you change the target resolution, all files re-encode.
OpenTranscode v4.2.1 — Production Release
v4.2.1 makes the log quiet by default. Tech detail is gated behind
--verbose.
(Note: v4.3.0 fixes the heartbeat regression introduced here — the
heartbeat is now always user-facing, not gated behind --verbose.)
The v4.2.1 change
Why
v4.2.0's log was filling up with thousands of lines of useless info
per file: the CMD: banner, ffmpeg's frame= 67 fps= 12 q=32 ...
progress chatter, the resolution map, the pre-flight validation table,
the DIAGNOSIS blocks, the 30s heartbeat, the RETRY messages. For a
180-file run that's thousands of lines nobody reads.
The user just needs: did it encode or not?
What changed
Default log output is now quiet. Per file, you see exactly two lines:
[1/180] filename.mkv
OK: 1.6MB -> 1.3MB (81%)
Or on failure:
[2/180] next.mkv
FAIL: ffmpeg exit code 1
Final summary stays:
QUEUE COMPLETE. Success: 178, Failed: 2.
What's gated behind --verbose
Found N file(s) to process.+Temp dir:banner─── FILE RESOLUTION MAP ───+ per-file resolution lines─── PRE-FLIGHT VALIDATION ───+ valid/invalid countsMode: ffmpeg (...),Chunking: select ...CMD: av1an -i ... --workers 4 --chunk-method select ...Source: 1920x1080 -> Output: 1920x1080- Live tail of av1an/ffmpeg stderr (
│ frame= 67 fps= 12 ...) ... still encoding (30s elapsed)heartbeat (every 30s)DIAGNOSIS: ...blocks (root-cause analysis for av1an failures)─── av1an stderr (last 25 lines) ───dumpsRETRY: .../RETRY OK: .../RETRY FAIL: ...messagesPre-scale OK (... MB intermediate)SUBS:mux messagesCLEANUP: Swept N residual temp file(s)...File type: ...(yt-dlp-download detection)
What stays user-facing (quiet mode)
Found N file(s) to process.— single banner line[N/total] filename— per-file headerOK: XMB -> YMB (Z%)— successFAIL: <short reason>— failure (one line, not a multi-line dump)SKIP: <short reason>— skipped file (no video stream, too short, etc.)STOP: skipping remaining files (<reason>)— when queue auto-abortsCLEANED: Removed N source file(s).— after delete-source transcodesQUEUE COMPLETE. Success: N, Failed: M.— final summaryABORT: All N file(s) are invalid. Aborting queue.— when pre-flight finds zero valid files
Usage
opentranscode # quiet (default)
opentranscode --verbose # full tech detail
opentranscode --use-av1an # opt into av1an chunk-parallel
opentranscode --use-av1an --verbose # full tech detail + av1an
OpenTranscode v4.2.0 — Production Release
v4.2.0 makes ffmpeg the default encode path. av1an chunk-parallel is
now opt-in via --use-av1an.
The v4.2.0 change
Why
v4.1.x tried to make av1an chunk-parallel work reliably. It doesn't, across distros. Failure modes observed in production:
Unprocessed tokens: --threads(v4.1.0/v4.1.1) —SvtAv1EncAppCLI doesn't accept--threads, only--lp. Every chunk failed 3x.- No av1an output for 270+ seconds (v4.1.2) — av1an wedged at startup or its stderr is buffered and won't flush until exit.
- y4m pipe breaks (v4.0.0) — Hybrid chunk method fails on phone- recorded MP4s with sparse keyframes.
- VapourSynth plugin issues —
lsmash/ffms2/bestsourceare separate packages that most distros don't install by default.
Each fix uncovered a new failure mode. The pattern is clear: av1an is too fragile to be the default.
What changed
Default encode path is now ffmpeg-only. The av1an pre-flight smoke
test is skipped entirely. _on_run_clicked sets use_ffmpeg_fallback = True directly, and EncoderWorker runs the _ffmpeg_fallback_encode
path for every file.
This means:
- No VapourSynth dependency — ffmpeg invokes
libsvtav1as a library, noSvtAv1EncAppsubprocess, nolibvapoursynth-script.so. - No chunk-method selection — single-pass ffmpeg per file.
- No
--threadsCLI quirks —-threadsis a valid ffmpeg/libsvtav1 library option. - Immediate progress output — ffmpeg's progress bar flushes to stdout line-by-line, picked up by the live tail (v4.1.1).
- Slower than chunk-parallel av1an — single-pass, no scene-split parallelism. But it actually completes, which is the only thing that matters.
The --use-av1an flag
Users who specifically want av1an chunk-parallel (e.g. they have a known-good VapourSynth + lsmash/ffms2 setup) can opt in:
opentranscode --use-av1an
When set, the full av1an pre-flight + smoke test runs as before, and the encode uses the chunk-parallel path. When not set (default), the smoke test is skipped and ffmpeg is used directly.
What v4.2.0 keeps
- Live progress tail (v4.1.1) — ffmpeg's stdout/stderr emits to the
GUI log as it arrives, with
\rprogress bar handling. - 30-second heartbeat (v4.1.1) —
... still encoding (Xs elapsed). --max-workers/--threads-per-workerCLI flags (v4.1.0) — still wired throughenv.av1an_flags.--threads-per-workeris now actually useful in the ffmpeg path (libsvtav1 accepts-threads).--chunk-method(v4.0.0) — still honored when--use-av1anis set. No-op in the default ffmpeg path.
What the ffmpeg command looks like
ffmpeg -i input.mkv \
-c:v libsvtav1 -preset 6 -crf 32 -pix_fmt yuv420p10le -g 240 \
-c:a libopus -b:a 64k \
-y output.mkv
Simple, reliable, no surprises. The same command works on any distro
with ffmpeg compiled against libsvtav1 (which is the default on
Arch, Debian, Ubuntu, Fedora).
OpenTranscode v4.1.2 — Production Release
v4.1.2 reverts the --threads N injection that broke av1an in v4.1.0/v4.1.1.
(Note: v4.2.0 makes ffmpeg the default, so this av1an issue is no
longer reachable unless you opt in with --use-av1an.)
The v4.1.2 fix (the actual root cause)
What was broken
v4.1.0 added "intelligent chunking" that injected --threads N into
av1an's --video-params string, intending to cap each per-chunk encoder
instance to N threads. The theory was sound — without a cap, SVT-AV1's
default --threads 0 means "use all logical cores," so 13 chunk-parallel
workers × 28 threads = ~364 threads on a 28-thread Xeon → kernel
scheduler drowned → hard lock.
The implementation was wrong. av1an invokes SvtAv1EncApp (the
standalone SVT-AV1 CLI binary) per-chunk, not the libsvtav1 library.
SvtAv1EncApp does not accept --threads — it uses --lp N
(logical processors) instead. The result, visible in the user's log:
Svt[info]: -------------------------------------------
Svt[info]: SVT [version]: SVT-AV1 Encoder Lib v4.2.0
Svt[info]: -------------------------------------------
Unprocessed tokens: --threads
Unprocessed arguments: 6
Error in configuration, could not begin encoding!
Every chunk failed 3 times with this error → av1an exited with code 1
→ no output file → OpenTranscode fell back to ffmpeg → ffmpeg succeeded
(because it invokes libsvtav1 as a library, where -threads IS valid).
But this happened per file, making every encode go through the slow
single-pass ffmpeg fallback path instead of chunk-parallel av1an.
The fix
Don't inject --threads into av1an's --video-params at all.
Revert params_fn to its v4.0.0 signature (crf, preset) -> str with
no threads parameter. Thread capping now happens via:
- av1an's
--workersflag (chunk-parallel count) — this is what_compute_intelligent_worker_countactually controls. Fewer workers = fewer concurrent SVT-AV1 processes = less thread pressure. -threadsin the ffmpeg fallback path —_svtav1_ffmpeg_args/_vp9_ffmpeg_args/_x265_ffmpeg_argsalready accept-threads, passed to libsvtav1/libvpx/libx265 as library options (where it works).
The intelligent worker-count logic from v4.1.0 is kept — it still
caps --workers to physical_cores - 1 and reserves 1 logical thread
for OS/UI. But the per-chunk thread cap that broke SVT-AV1 is gone.
What v4.1.2 keeps from v4.1.0/v4.1.1
- Intelligent
--workerscount —_compute_intelligent_worker_countstill caps chunk-parallel workers based on CPU topology. - Live progress tail (v4.1.1) — av1an's stdout/stderr now emits to
the GUI log as it arrives, with
\rprogress bar handling. - 30-second heartbeat (v4.1.1) —
... still encoding (Xs elapsed)so you always know the encode is alive. --max-workers/--threads-per-workerCLI flags — still wired throughenv.av1an_flags.--max-workerscontrols av1an's--workers.--threads-per-workeris currently a no-op in the av1an path (kept for future use if/when SVT-AV1's--lpflag is wired in correctly).
What the av1an command looks like now
/usr/bin/av1an -i input.mkv --workers 4 --chunk-method select \
--encoder svt-av1 \
--video-params --preset 6 --crf 32 --keyint 240 \
--audio-params -c:a libopus -b:a 64k \
--concat mkvmerge -o output.mkv
No --threads 6 in --video-params — SvtAv1EncApp accepts this cleanly.
OpenTranscode v4.1.1 — Production Release
v4.1.1 adds live progress tail + heartbeat so you can see av1an is working during long encodes. Also bumps threads-per-worker from 4 to 6 for better SVT-AV1 per-chunk throughput.
(Note: the threads-per-worker bump in v4.1.1 was reverted in v4.1.2 — see above.)
The v4.1.1 fix (in detail)
The bug
v4.1.0 capped threads per encoder instance at 4 to prevent the thread-oversubscription hard-lock. That worked (no more hard locks), but it made each chunk ~7x slower than v4.0.0's "grab all 28 threads" behavior. The result: file 1 took 10+ minutes, and the user saw nothing in the GUI log the entire time.
The "nothing" was the real killer. The drainer threads in
_run_with_stop_check read av1an's stdout/stderr into StringIO
buffers but only emitted them to the GUI log on process exit. So
during a 10-minute encode, the user stared at:
[1/46] Encoding: video1.mp4
Source: 1920x1080 -> Output: 1920x1080
Chunking: select (av1an default if no override)
CMD: av1an -i ... --workers 4 --chunk-method select ...
...and nothing else for 10 minutes. Looked identical to a wedged process. User assumed it was "borked" and killed it.
The fix (three parts)
-
Live tail — the drainer now emits each line of av1an's stdout/stderr to the GUI log as it arrives, prefixed with
│to distinguish from orchestrator messages. Handles both\n(log lines like "scenecut: found 8 scene(s)") and\r(progress bar updates like "Encoding 45%") as line boundaries, so av1an's progress bar renders correctly in real-time. -
30-second heartbeat — the poll loop emits
... still encoding (Xs elapsed)every 30 seconds, so even if av1an isn't producing line-delimited output (e.g. during a long SVT-AV1 encode that only updates a\rprogress bar), the user knows the process is alive. -
IDEAL_THREADS_PER_WORKERbumped from 4 to 6 — SVT-AV1 with only 4 threads was too slow per-chunk. With 6 threads, each chunk gets ~50% better throughput while staying under the logical-thread budget. On a 28-thread Xeon, this changes the split from 6×4=24 to 4×6=24 (same total, better per-chunk latency — the first chunk completes sooner, so the user sees progress faster).
What the log looks like now
[1/46] Encoding: video1.mp4
Source: 1920x1080 -> Output: 1920x1080
Chunking: select (av1an default if no override)
CMD: av1an -i ... --workers 4 --chunk-method select ...
│ INFO encode_file: Input: 1920x1080 @ 29.763 fps, YUVJ420P, SDR
│ INFO encode_file: scenecut: found 8 scene(s)
│ DEBUG encode_file: Segmenting video
│ DEBUG encode_file: Segment done
│ INFO encode_chunk: Encoding chunk 1
... still encoding (30s elapsed)
│ INFO encode_chunk: Encoding chunk 2
... still encoding (60s elapsed)
│ SUMMARY ------------------------------------------
│ Average Speed: 4.231 fps
SUCCESS: video1.mp4 (245.3MB -> 28.7MB, 12%)
Worker-count math, updated for v4.1.1
| Machine | budget | workers | threads | active | reserved |
|---|---|---|---|---|---|
| 4-core / 8-thread laptop | 7 | 1 | 7 | 7 | 1 |
| 8-core / 16-thread desktop | 15 | 2 | 7 | 14 | 1 |
| 14-core / 28-thread Xeon | 27 | 4 | 6 | 24 | 4 |
| 32-core / 64-thread EPYC | 63 | 10 | 6 | 60 | 4 |
| 1-core / 2-thread VM | 1 | 1 | 1 | 1 | 1 |
OpenTranscode v4.1.0 — Production Release
v4.1.0 adds intelligent chunking to prevent the thread-oversubscription hard-lock that v4.0.0 hit on high-core-count machines.
The v4.1.0 fix (in detail)
The bug
v4.0.0 introduced the --chunk-method select auto-override for phone-
recorded MP4s with sparse keyframes (see the v4.0.0 section below for
that fix). But select mode keeps the encode pipeline tighter than
the previous hybrid default — chunks warm up faster, more encoder
instances hit full tilt at the same instant.
Meanwhile, EncoderWorker.run() was still computing
worker_count = max(1, physical_cores - 1) and passing no per-chunk
thread cap to the encoder. SVT-AV1's default --threads 0 means "use
all logical cores," so each chunk-parallel worker spawned an
SvtAv1EncApp process that grabbed every logical thread.
On a 28-thread Xeon (14 physical cores) with 13 chunk-parallel workers,
the math was: 13 × 28 = ~364 active threads on 28 logical CPUs. The
kernel scheduler drowned, I/O wait escalated, and the box hard-locked
even though no single process was at fault. The 1-second STOP-button
poll in _run_with_stop_check couldn't get scheduled, so even clicking
STOP didn't recover it.
The fix (three parts)
-
_compute_intelligent_worker_count()inEncoderWorkernow computes(worker_count, threads_per_worker)such thatworker_count * threads_per_worker <= logical_threads - 1(one logical thread reserved for OS / UI / av1an orchestrator). The budget is split using an ideal4 threads per worker— the empirical sweet spot for SVT-AV1, x265, and vpxenc. Beyond ~6 threads per encoder instance you hit memory-bandwidth contention and diminishing returns. -
The resolved
threads_per_workeris passed toparams_fnso each_av1_params/_vp9_params/_x265_paramsappends--threads Nto the encoder's--video-paramsstring. Every per- chunk encoder instance (SvtAv1EncApp / vpxenc / x265) now respects its share of the thread budget instead of grabbing all cores. -
Two new CLI flags let the user override the auto math when needed:
opentranscode --max-workers 4 # cap chunk-parallel worker count opentranscode --threads-per-worker 6 # per-encoder thread cap opentranscode --max-workers 4 --threads-per-worker 6 # full overrideBoth are also visible in
--dry-runand the GUI env-probe banner.
Worker-count math, by machine
| Machine | budget | workers | threads | active | reserved |
|---|---|---|---|---|---|
| 4-core / 8-thread laptop | 7 | 1 | 7 | 7 | 1 |
| 8-core / 16-thread desktop | 15 | 3 | 5 | 15 | 1 |
| 14-core / 28-thread Xeon | 27 | 6 | 4 | 24 | 4 |
| 32-core / 64-thread EPYC | 63 | 15 | 4 | 60 | 4 |
| 1-core / 2-thread VM | 1 | 1 | 1 | 1 | 1 |
The Xeon row is the user's box. v4.0.0 hit 13 × 28 = 364 threads → hard lock. v4.1.0 hits 6 × 4 = 24 threads with 4 reserved for OS/UI.
Also new in v4.1.0
--dry-runnow prints the computed worker math so you can verify the thread budget before launching a real encode.- The GUI env-probe banner shows the same math on startup.
EncoderWorker.__init__readsenv.av1an_flags["max_workers"]and["threads_per_worker"]as fallback when the explicit constructor args aren't supplied — so the CLI flags reach the GUI-spawned worker withoutui_window.pycode changes.
Backwards compatibility
params_fnaccepts an optional third argthreads=0. Existing callers passing(crf, preset)still work becausethreadsdefaults to 0 (the v4.0.0 behavior — no--threadsflag, encoder auto-selects).- The
_av1_params/_vp9_params/_x265_paramsfunctions are byte-identical to v4.0.0 whenthreads=0.
OpenTranscode v4.0.0 — Production Release
v4.0.0 resolves the "works up until near the end, never saves chunks into a full file" bug that affected phone-recorded MP4s with sparse keyframes.
The v4.0.0 fix (in detail)
The bug
When no VapourSynth source plugins are installed (the common case on most
distros — lsmash, ffms2, bestsource are all separate packages),
av1an auto-selects the Hybrid chunk method. Hybrid does:
ffmpeg -c copy -f segmentto split the source at scene boundaries- Re-decode each segment to y4m via a second ffmpeg invocation
- Pipe the y4m to the encoder (SvtAv1EncApp / vpxenc / x265)
Phone-recorded MP4s (the 20190707_112725.11b774bacde3.mp4 files in
the production log) only have I-frames every 5–10 seconds. Scene
boundaries detected by av1an's av_scenechange rarely align with those
sparse keyframes. The segment muxer can only split on keyframes, so the
segment for scene N actually starts somewhere inside scene N-1's GOP.
The result: the decoder has no I-frame reference → errors with
[h264 @ 0x...] error while decoding MB 35 25 → the y4m pipe breaks →
the encoder reads EOF mid-frame → Failed to read y4m frame delimiter. Read broken. EOF: 1 → every chunk fails after 3 retries → no chunks
to concat → no output file.
The previous ffmpeg fallback rescued the file, but it was slow (single-pass, no chunk-parallel) and the diagnostic was misleading ("concat failure" when it was actually a chunk-extraction failure).
The fix (three parts)
-
_encode_onenow accepts achunk_methodparameter. When av1an fails with the y4m break pattern, it recursively retries with--chunk-method select. Select uses VapourSynth'sselect()filter to extract frames one-by-one — slower than Hybrid but reliable for any file VapourSynth can open. This is faster than the ffmpeg fallback (chunk-parallel still works) and produces identical-quality output (same encoder, same params). -
The working chunk_method is cached in
env.av1an_flags["chunk_method_override"]so subsequent files skip the wasted first attempt. -
env_probenow probes for VapourSynth source plugins via_probe_vs_source_plugins(). When NONE are found, it pre-setschunk_method_override = "select"to avoid the wasted first attempt entirely. The probe checks~/.local/lib/vapoursynth/,/usr/lib/vapoursynth/,/usr/local/lib/vapoursynth/, and the Debian multiarch path.
Also fixed
The "SUMMARY block + non-zero exit" diagnostic previously misdiagnosed
y4m break failures as "concat failure" (because SVT-AV1 prints a
SUMMARY block per-chunk before the pipe breaks). The check is now
guarded by "Failed to read y4m frame delimiter" not in stderr_full so
it only fires for true concat failures.
New CLI flag
opentranscode --chunk-method {auto,select,hybrid,segment,ffms2,lsmash,bestsource,dgdecnv}
Lets the user force a specific chunk method. Useful for debugging or
for environments where the probe picks the wrong default. auto clears
any override the probe set.
Quick start
Option A: Install as a package (recommended)
cd /path/to/this/directory
pip install -e . # editable install (dev)
# OR
pip install -e ".[dev]" # with pytest + build tools
# Now you can run it three ways:
opentranscode # console entry point
python -m opentranscode # module entry point
python -m opentranscode --version # → opentranscode 4.4.3
Option B: Run the launcher script (backwards compat)
python open-transcode.py # the launcher script still works
Option C: Verify your environment without encoding
opentranscode --dry-run # probe env + smoke test, no GUI, no encode
opentranscode --dry-run --chunk-method select # preview a forced chunk method
opentranscode --dry-run --max-workers 4 # preview worker-count override
opentranscode --verify-only /path/to/existing_output.mkv # re-verify an output
Option D: Override the intelligent worker math (v4.1.0)
If the auto-computed thread budget still hard-locks the box (rare), or if you have fast storage and want more parallelism, dial it manually:
opentranscode --max-workers 4 # 4 chunks in parallel
opentranscode --threads-per-worker 6 # 6 threads per encoder
opentranscode --max-workers 8 --threads-per-worker 2 # full override
Both flags are stored on env.av1an_flags and picked up by the
GUI-spawned worker — no UI changes needed.
Files
| Path | Description |
|---|---|
opentranscode/ |
Package — 16 modules. Importable as import opentranscode. |
open-transcode.py |
Launcher script (~6,500 lines). Kept for backwards compat + as the test target for the mocked tests. Mirrors the package behavior. |
pyproject.toml |
PEP 621 build config. Entry point: opentranscode = opentranscode.__main__:main. Ready for pip install -e . and python -m build. |
tests/ |
103 tests across 12 files. All pass in ~10 seconds. |
pytest.ini |
pytest config (also in pyproject.toml). |
README.md |
This file. |
logs/av1an.log.2026-07-13 |
The production log that revealed the y4m break bug. Kept for reference. |
Test suite
cd /path/to/this/directory
python -m pytest tests/ -v
# 103 tests, ~10 seconds:
# 25 mocked unit tests (smoke test, encoder pipeline, audio loudnorm, etc.)
# 12 real-ffmpeg e2e tests — requires ffmpeg + ffprobe
# 36 package-structure tests
# 13 chunk-method retry tests
# 1 chunk-method e2e recovery test
The e2e tests generate a real 2-second test video with ffmpeg, run the full EncoderWorker pipeline on it (AV1→MKV, x265→MKV, VP9→WebM), and verify the output file exists, is non-empty, has the correct codec, and has the expected duration. The chunk-method e2e test additionally simulates the y4m break pattern and verifies the retry-with-select produces a valid output file.
Verification (run these to confirm v4.0.0 works)
# 1. Package imports cleanly
python -c "import opentranscode; print(opentranscode.__version__)" # → 4.0.0
# 2. CLI works
python -m opentranscode --version # → opentranscode 4.0.0
python -m opentranscode --help # → usage (includes --chunk-method)
python -m opentranscode --dry-run # → env probe report (shows VS plugins + chunk method)
# 3. All tests pass
python -m pytest tests/ -q # → 103 passed in ~10s
# 4. Install works
pip install -e . # → installs opentranscode + PySide6
opentranscode --version # → opentranscode 4.0.0
# 5. Launcher script still works (backwards compat)
python open-transcode.py # → launches GUI (if PySide6 + display)
Publishing to PyPI
v4.0.0 is the production release. To publish:
python -m build # produces dist/opentranscode-4.0.0.tar.gz + .whl
twine upload dist/* # publishes to PyPI
The production log that revealed the bug
The file logs/av1an.log.2026-07-13 is the actual av1an log from the
user's production run that revealed the y4m break bug. Key markers:
INFO encode_file: Input: 1920x1080 @ 29.763 fps, YUVJ420P, SDR
INFO encode_file: scenecut: found 8 scene(s) [with extra_splits: 16 scene(s)]
DEBUG encode_file: Segmenting video
DEBUG encode_file: Segment done
INFO encode_file:
Encoding Failed to read y4m frame delimiter. Read broken. EOF: 1
[h264 @ 0x55da365b30c0] error while decoding MB 35 25
WARN encode_chunk: Encoder failed (on chunk 11):
Encoding Failed to read y4m frame delimiter. Read broken. EOF: 1
SUMMARY -----------------------------------------------------------------
Average Speed: 2.501 fps
ERROR av1an_core::broker: [chunk 4] encoder failed 3 times, shutting down worker
The smoke test (line 1 of the log) succeeds because it uses
chunk_method: Select — but the real encode (line 4 onward) uses
chunk_method: Hybrid (av1an's auto-selection when no VS plugins are
installed) and fails on every chunk.