ctOpus writes exactly the same Opus files as opusenc, byte for byte, and
spreads a single file over all cores. At maximum complexity it is twice as fast as
opusenc at minimum complexity; on 45 hours of music it is 4.4 times as fast as opusenc
with the same settings. Opus is sequential by design: every frame continues the
bitstream, the energy prediction and the filter memories of the frame before it. These
eleven pictures show how, step by step, a task graph on ThinkMeta ConcurrentTasks grew out of it – up
to tasks that compute future frames ahead on speculation.
Picture 1 of 11
The reference: opusenc
opusenc reads the WAV file, resamples it from 44.1 to 48 kHz, encodes
it and writes Ogg pages – all on one thread, one after another. While
it reads or writes, the thread waits for the file. Every stage carries
state from frame to frame: the filter memories, the energy prediction, the
bit reservoir and the bitstream. One core works, the others wait.
These bits are the yardstick: every following picture has
to write exactly the same .opus file, byte for byte. ctOpus starts from a
port of the encoder, function by function: on one thread it is as fast as
opusenc (2.09 s) and bit-identical.
Where the time goes
A profile of opusenc at complexity 10: the resampler takes
34.5 %, the quantization of the frequency bands 25.1 %,
the tonality analysis 8.2 %, the pre-filter 6.6 %, the
transform and the time-frequency decisions together about
9 %. The test music is 44.1 kHz stereo throughout, so the
resampler runs for every file.
Complexity
Opus lets you trade speed for quality, from complexity 0 (fastest)
to 10 (best, the default). On the test file opusenc needs
2.16 s at complexity 10 and 0.97 s at complexity 0. The
goal of ctOpus: complexity 10 faster than opusenc at complexity 0.
Run time · one file, 254 s of music, comp 10
2.16 s
Picture 2 of 11
Reading and writing asynchronously
File input and output move out of the encoder. Read and
Write become tasks on the I/O scheduler; a single task
encodes and does everything opusenc does. Read reads ahead with five overlapping requests of 1 MB into a read buffer, Encode only takes finished blocks
from it. Encode copies its Ogg pages into a write buffer, and Write puts
full blocks into the file, two requests at a time.
While Read or Write wait for the disk, they occupy no thread: the I/O
scheduler runs the other task in the meantime. Measured with the file fresh from the disk, it is as fast or faster – and bit-identical in every check. With a single encoding task both ways take the same time; with the full graph of picture 10 the same step saves 2 to 3 %, because there the encoding task wrote the Ogg pages on the critical path: 16 threads 0.486 → 0.474 s, 4 threads 0.780 → 0.760 s.
Read buffer and write buffer
The read buffer holds 8 blocks of 1 MB, the write buffer 8
blocks of 64 KB. A full buffer stops the task that fills it, an empty
one the task that drains it. opusenc, in contrast, writes every Ogg page with a call
of its own – several thousand times for the test file.
Measured cold
Before every run the input was copied afresh and unbuffered, so it
really came from the disk. One encoding task: 1.891 s synchronous, 1.876 s asynchronous – within the noise; it consumes its input at about 25 MB/s, far less than the disk delivers. Block
size and the number of requests in flight make no difference
(±3 %).
Run time · file fresh from the disk, one encoding task
1.891 → 1.876 s · opusenc 2.083 s
Picture 3 of 11
Resampling in parallel
Distribute unpacks the WAV data into the samples – on the compute scheduler with high priority (bold border), because every resampler waits for its blocks. The resampler computes every output sample
from the input alone, its filter phase follows from the position – so
blocks can be computed independently, on every core, in any order, and give
exactly the same values. Encode takes the blocks from the block ring in
order.
The resampler has left the critical path: 1.86 s at complexity 10,
and 0.70 s instead of 0.97 s at complexity 0 (1.38×). What
remains is almost only the serial encoder.
Samples and block ring
The whole input lies in reserved memory, filled by Distribute and
released as soon as every block has read it. Distribute releases a
block as soon as its input is complete and a slot in the ring is
free; a slot holds 16,384 resampled samples per channel.
Why Distribute computes
Since reading is asynchronous, Distribute no longer waits for the
disk, only for other tasks – and it only computes: converting
samples and splitting them into channels. So it belongs on the
compute scheduler, which is far from saturated. Measured against the
I/O scheduler: 16 threads 0.474 → 0.466 s, 4 threads
0.760 → 0.744 s, bit-identical.
Same sums, same bits
The parallel blocks add up in the same order as opusenc, and at the
end of the file Distribute appends the same extension of the signal:
a continuation of the last 480 samples, then zeros. The encoder sees
exactly the same numbers.
Run time · one file, 254 s of music, comp 10
2.16 → 1.86 s (1.16×)
Picture 4 of 11
The analysis runs ahead
The tonality analysis of a frame depends only on the signal, not on the
decisions of the encoder. It becomes a task of its own that runs ahead of
Encode and puts its result for every frame into a ring. The block ring now
has two readers: a slot is only free again when both have it. Encode still
reads the blocks itself (the arrow on the right).
In between, the encoder itself was ported to C++: that alone took the
run time from 1.86 to 1.73 s.
Time stamps show: Encode now waits neither for blocks (1 ms) nor for
the analysis (2 ms). The 1.5 s that remain are pure serial work,
about 0.67 s of it the quantization of the frequency bands.
A head start with a brake
The analysis replays exactly the same sequence of calls as opusenc
and keeps a history of 512 entries, so it never overwrites what
Encode still reads. It runs at most 200 frames ahead; once that far,
it waits until the gap has shrunk to half – without that brake,
Encode woke it at every frame, which cost 0.13 s with 16
threads.
Run time · one file, 254 s of music, comp 10
1.86 → 1.52 s
Picture 5 of 11
The signal stage joins the analysis
Part of the encoder depends only on the signal as well, not on earlier
decisions: pre-emphasis, tone and transient detection, the pitch search.
The analysis task computes this signal stage ahead and passes signal frames
along with its results.
Encode compares the complete input of that stage – about 16 KB per
frame – and only takes a frame over if it is exactly equal. If the
analysis ever diverges, it gets slower, never wrong. Measured: 12,725 of
12,725 frames taken over. ctOpus now needs 0.62 of the time of opusenc in
the same run, before 0.74.
A new series of measurements
From here on all variants run alternately in the same run, opusenc
included: on some days the machine was 10 to 15 % slower. The
times are therefore not directly comparable with the pictures
before.
When the analysis diverges
After a reset, in speech mode or during a fade, the analysis may no
longer match the encoder. Encode notices it in the comparison and
computes the frame itself.
Run time · the same file, all variants in the same run
1.47 s · opusenc 2.38 s
Picture 6 of 11
The front end is computed ahead
After the signal stage the analysis now also computes the front end of the
next frame: the pre-filter, the transform into the frequency domain, the
band energies and, for frames of 5 ms and more, normalization, bit
allocation and time-frequency decisions. They depend on only a few values
of the serial encoder, and the analysis simply assumes them –
“as at the last sync”.
Encode takes a front end over only if those values match. After a miss it
sends its real values back (dashed), and the analysis recomputes from
there. Hit rate: 12,724 of 12,725 frames. Complexity 10 is now about as
fast as opusenc at complexity 0 (0.97 s in the same run).
Found while building
A result that counted as finished while it was only half written,
and a deadlock between the correction and the wait for blocks
– both found and fixed before the step was done.
Run time · the same file, all variants in the same run
1.21 → 1.00 s
Picture 7 of 11
The front end becomes a task, a helper takes the rounding trial
The front end becomes a task of its own: the analysis no longer has to be
interrupted for a correction, and Encode no longer waits for it.
At high complexity the encoder quantizes every stereo band twice, from the
same state, with the stereo angle rounded down and up, and keeps the
cheaper trial. The second trial now runs on a helper, at
the same time as the first one in Encode. A trial takes only 1 to
2 µs, so both sides first wait actively and only then go to sleep.
Goal reached: complexity 10 is 19 % faster than
opusenc at complexity 0, and 2.7 times as fast as opusenc at complexity
10.
The limit: turbo clock
The quantization of the bands alone takes 0.72 s; with two more
busy cores 0.76 s, with six 0.80 s. Every additional busy
core lowers the turbo clock of all cores, the serial one included.
Run time · the same file, all variants in the same run
0.80 s · opusenc comp 0: 0.99 s
Picture 8 of 11
Taking more work off the encoding task
The graph stays the same; the work moves. The input stage – DC
filter, silence detection, frame energy, stereo width – goes into
the analysis, the top-level stereo angles into the front end. After the
last rounding trial Encode skips work that only served the trials, and it
copies less.
Whatever Encode does not compute itself shortens the serial chain, and every
takeover is tied to an exact comparison. In the same run opusenc needs
1.02 s at complexity 0 and 2.11 s at complexity 10.
What is left
Encode now needs about 0.68 s per file, 0.31 s of it for
its own rounding trial. This serial backbone is the target of the
next step.
Run time · the same file, all variants in the same run
0.80 → 0.70 s
Picture 9 of 11
Speculation: frame N+1 on a copy
Within a frame the bands are serial, but from one frame to the next only two
small things pass: a few bits left over at the end of the frame and the
state of the bitstream coder, which is needed late, if at all. So Encode
copies its state before it quantizes frame N, puts in an estimate for the
leftover bits, and the speculation quantizes frame N+1 on
the copy at the same time, recording every symbol it writes.
When Encode reaches frame N+1, it checks every assumption. If all hold, it
simply replays the recorded log instead of computing the frame; otherwise
the speculation stops at the next band boundary. About 96 % of the
speculated frames are taken over – about 20 % faster. The
speculation has a helper of its own for its rounding trial.
The signal stage becomes a task
With the speculation, the analysis became the bottleneck: frame N+1
was missing in 45 % of the queries. With the signal stage on a
task of its own, it was missing in only about 100 of 12,725.
When not to speculate
Encode and the speculation take turns: if Encode takes frame N+1
over, it computes N+2 itself and speculates on N+3. If the real
result is already there before a speculation could start, there is
none. And below 7 threads there are more busy tasks than threads
– then the speculation costs more than it gains and stays off.
Run time · test file, 8 threads, without → with one speculation stage
0.70 → 0.56 s
Picture 10 of 11
The cascade: three frames ahead
With enough threads the serial encoder works several frames ahead: the copy
of the first speculation builds the copy for the second, and that one the
copy for the third. Up to three speculations compute frames N+1 to N+3
while Encode quantizes frame N, each with a helper of its own from 12
threads on. This is today's graph.
On the test file ctOpus needs 0.50 s at complexity 10 and 0.31 s
at complexity 0 – opusenc 2.21 s and 0.99 s.
ctOpus at maximum complexity is twice as fast as opusenc at
minimum. On 45 hours of music it needs 289.5 seconds instead of
21 minutes, 561 times real time – all 343 files bit-identical.
How many levels pay off
Complexity 10, the test file:
8 threads: 0.71 s without speculation, 0.56 s with one level, 0.53 s with two
10 threads: 0.71 s, 0.56 s, 0.46 s
three levels with their own helpers, 12 threads: 0.455 s
Hence the rule: no speculation below 7 threads, one level at 7,
three from 8, the helpers of the levels only from 12 – below
that they cost more clock than they gain.
Why fibers make the difference
About a dozen stages run at once, and each is written as a plain
loop that waits for its next frame. On ThinkMeta ConcurrentTasks such a wait suspends
only the task; its thread meanwhile runs another stage, so all stages
share one thread per logical processor. A wrong speculation is not
cancelled from outside: the task checks at every band and goes back
to waiting.
Other CPUs
Intel Core i9-11900K, 8 cores: 4.38×
Intel Xeon E3-1275 v6, 4 cores: 4.12×
Intel Core Ultra 7 155H, notebook: 2.52×
On the notebook single tracks reach 3.53×, the album-length files at
the end of the run only 1.89× – most likely its limit under
sustained load. Bit-identical to opusenc on every machine. These
corpus runs were made before reading and writing became asynchronous
(picture 2).
Speed-up over opusenc · 343 files, 45 h of music · still with synchronous reading and writing
4.38×
Picture 11 of 11
The graph in the code
Up to here the graph was spread over the code: all tasks reached into one
shared structure. Now the graph lives in one place in the code, just as it
is drawn: it owns every edge and every node in the order of the data flow,
starts the tasks with their priorities and waits until all of them are
done. Not a single calculation changes.
The single-task path stays as the reference – for input at
48 kHz that has nothing to resample, for example. With asynchronous
reading and writing it is exactly the graph of picture 2.
Structure for free
Measured against the state before, old and new alternately, 11
pairs: all threads 0.438 → 0.440 s, 4 threads 0.747
→ 0.757 s, one thread 1.836 → 1.841 s. The single
pairs scatter from −8 % to +9 % – noise. The
structure costs nothing, and every check stays bit-identical.
Run time · test file, all threads, before → after the rework in the same run
0.438 → 0.440 s
Try it yourself
ctOpus, the reference build of opusenc and the benchmark for your own music are on
GitHub.
Have a workload that should use every core?
Get in touch.
