Subject: Re: [codec] draft-ietf-codec-oggopus and "album" gain

From: Ian Nartowicz <[email-masked]>
Date: Wed, 13 Aug 2014 22:22:01 +0100

After much discussion on IRC, we've come up with some further clarifications to
the use of the output gain field and R128 tags.

In section 5.1:
        Virtually all players and media frameworks should apply it by
-       default.  If a player chooses to apply any volume adjustment or
-       gain modification, such as the R128_TRACK_GAIN, R128_ALBUM_GAIN
-       (see Section 5.2) or a user-facing volume knob, the adjustment
-       MUST be applied in addition to this output gain in order to
-       achieve playback at the desired volume.
+       default.  If a player chooses to apply any gain modification,
+       such as the R128_TRACK_GAIN or R128_ALBUM_GAIN (see Section 5.2),
+       the adjustment MUST be applied in addition to this output
+       gain in order to achieve playback at the normalised volume.

In section 5.2.1:
-   If an encoder wishes to use R128 normalization, and the output gain
-   is not otherwise constrained or specified, the encoder SHOULD write
-   the R128 gain into the 'output gain' field and store a tag containing
-   "R128_TRACK_GAIN=0".  That is, it should assume that by default tools
+   The encoder should assume that by default tools
    will respect the 'output gain' field, and not the comment tag.  If a
    tool modifies the ID header's 'output gain' field, it MUST also
    update or remove the R128_TRACK_GAIN and R128_ALBUM_GAIN comment tags

--ian

---

From: "Timothy B. Terriberry" <[email-masked]>
Date: Thu, 14 Aug 2014 10:24:48 -0700

To capture the reasoning behind these changes...

> +       gain in order to achieve playback at the normalised volume.

Ian felt that including the volume knob example made this text unclear. 
The new text makes it explicit that 'output gain' + R128_TACK_GAIN or 
'output gain' + R128_ALBUM_GAIN gives you _normalized_ volume (as 
defined by EBU R128), which may or may not have any relation to "desired 
volume". I don't think we need to be in the business of telling players 
how to implement a volume knob, so taking the example out seemed best.

>      update or remove the R128_TRACK_GAIN and R128_ALBUM_GAIN comment tags

This was causing confusion because it suggested that an encoder should 
_always_ prefer to put the track gain into the 'output gain' field. In 
reality we used to have text saying that using the album gain for 
'output gain' was preferred, and this was just an alternative (the 
leading "If" clause being the the important part of the sentence, not 
the SHOULD). However, this sentence is entirely redundant with the text 
from the definition of the 'output gain' field: "An encoder SHOULD set 
this field to zero, and instead apply any gain prior to encoding, when 
this is possible and does not conflict with the user's wishes." Removing 
the sentence avoids the confusion.

I've applied both sets of changes locally (using "An encoder" instead of 
"The encoder"), and they'll be included in the next update (after WGLC).

I also moved the "assume by default" sentence to the end of the 
paragraph and removed the preceding paragraph break, as the warning 
about being required to update the tags when the header value changes 
now logically follows from the declaration that these tags are relative 
to that field.

---

From: Ron <[email-masked]>
Date: Sat, 16 Aug 2014 13:31:40 +0930

On Thu, Aug 14, 2014 at 10:24:48AM -0700, Timothy B. Terriberry wrote:
> reality we used to have text saying that using the album gain for 'output
> gain' was preferred, and this was just an alternative (the leading "If"
> clause being the the important part of the sentence, not the SHOULD).
> However, this sentence is entirely redundant with the text from the
> definition of the 'output gain' field: "An encoder SHOULD set this field to
> zero, and instead apply any gain prior to encoding, when this is possible
> and does not conflict with the user's wishes." Removing the sentence avoids
> the confusion.
> 
> I also moved the "assume by default" sentence to the end of the paragraph
> and removed the preceding paragraph break, as the warning about being
> required to update the tags when the header value changes now logically
> follows from the declaration that these tags are relative to that field.

So the patch applied for this part is now:

> - If an encoder wishes to use R128 normalization, and the output gain is not
> - otherwise constrained or specified, the encoder SHOULD write the R128 gain
> - into the 'output gain' field and store a tag containing "R128_TRACK_GAIN=0".
> - That is, it should assume that by default tools will respect the 'output gain'
> - field, and not the comment tag.
>   If a tool modifies the ID header's 'output gain' field, it MUST also update or
>   remove the R128_TRACK_GAIN and R128_ALBUM_GAIN comment tags if present.
> + An encoder should assume that by default tools will respect the 'output gain'
> + field, and not the comment tag.

And I wonder two things about it:

Is that intended to be "An encoder SHOULD assume"?

And if we do want this to be normative, do we need to expand a little on
what 'tools' and 'respect' means there - since that's possibly now less
clear as a statement which stands on its own, rather than a(n intended)
clarification to the text that is being removed.

I agree with the rationale of what's intended by this change, but I'm
less sure if it's now clear to a first time reader whether 'tools'
includes both players and editors, and under what conditions the
assumptions about 'default behaviour' will apply (since if we have the
comment tags, obviously *something* might respect them sometimes).

If we're going to give (normative) guidance there, it possibly wants
to be a little more specific about what exactly it applies to.

  Ron

---

From: Ralph Giles <[email-masked]>
Date: Mon, 25 Aug 2014 16:11:21 -0700

>> - field, and not the comment tag.

I agree this part is confusing. I think it's too terse to offer good
guidance.

>>   remove the R128_TRACK_GAIN and R128_ALBUM_GAIN comment tags if present.

This seems helpful in avoiding broken files.

>> + field, and not the comment tag.

I read this as a non-normative should. Would it be better:

"By default, implementations of this specification MUST respect the
'output gain' field, but MAY NOT respect the comments. Encoder authors
are advised to take this into account. For example, to produce R128
normalized files it's more reliable for post-processing application to
update the 'output gain' field and write a comment 'R128_TRACK_GAIN=0'
than to put the normalized value directly in the comment."

This removes the normative suggestion of 'should' while maintaining the
suggestion and rationale.

 -r

---

From: Ron <[email-masked]>
Date: Tue, 26 Aug 2014 22:36:29 +0930

> This seems helpful in avoiding broken files.

Yes, I agree that part stands as a sensible MUST.  About the only thing
I could niggle about there might be s/update/correct/, but I don't
personally think it's unclear what that indicates you should do or why
(though I'm probably too close to this to be a good judge of that for
other readers).  And if we say 'correct' we might need a few words on,
or a simple example of, what 'correct' actually is.

> I read this as a non-normative should. Would it be better:

I couldn't really decide whether that ought to be normative or not,
mostly because I couldn't draw a useful distinction between "you
SHOULD assume this because the standard says so" and "you should
assume this because it's about the only sensible thing you can".

Either way what we're really saying is "unless you have a good
reason to believe otherwise, this is what you should expect",
which fits with both normative and 'ordinary' use.

(partly that question was in the context of us having a few other
non-normative keywords we need to clean up still, I don't feel
strongly about it going either way here)

> 'output gain' field, but MAY NOT respect the comments.

I'd be inclined to drop the "By default" there, unless we're going
to expand on what the exceptions to the 'default' might be.

> Encoder authors are advised to take this into account.

In which case the normative language probably makes that sentence
redundant.

> For example, to produce R128

Should the example there be ALBUM_GAIN rather than track gain
(consistent with what we recommended before these changes, and
with what is probably preferred in the absence of finer grained
recalibration)?

Maybe s/it's more reliable/it would be more robust/ ?
(the distinction I see there being between what real apps
_actually_ do in practice now and in the future, and what
the standard guarantees you as being baseline support ...
I don't have data on which is really more reliable, but I
can tell you what the standard guarantees is most likely).

But yes, I think that addresses the missing context I saw,
assuming it doesn't also revert whatever it was Ian found
confusing to remove it in the first place :)

  Ron

---

From: "Timothy B. Terriberry" <[email-masked]>
Date: Tue, 26 Aug 2014 13:25:35 -0700

> non-normative keywords we need to clean up still, I don't feel

Now's the time to get those cleaned up. Care to propose resolutions to 
the list for them?

---

From: Gregory Maxwell <[email-masked]>
Date: Tue, 26 Aug 2014 13:26:15 -0700

> with what is probably preferred in the absence of finer grained

Is there a reason we shouldn't just have two examples?

---

From: Ian Nartowicz <[email-masked]>
Date: Wed, 27 Aug 2014 15:30:43 +0100

On Mon, 25 Aug 2014 16:11:21 -0700
Ralph Giles <[email-masked]> wrote:

>>>   If a tool modifies the ID header's 'output gain' field, it MUST also
>>> update or remove the R128_TRACK_GAIN and R128_ALBUM_GAIN comment tags if
>>> present.
>

This seems like a step backwards to me.  That MUST is a requirement
that wasn't present before.  An earlier statement is "Virtually all players
and media frameworks should apply it by default.", which I think is the
appropriate guidance.

>to expand on what the exceptions to the 'default' might be.

I agree with Ron that saying "By default" and "MUST" in the same sentence is
confusing, but I think the solution is not to say MUST.  There are exceptions
to the default.  Trying to specify them all here would seem an impossible feat
of guesswork, but since they exist then MUST is too strong.

>Is there a reason we shouldn't just have two examples?
Possibly, but if you start down that path you need more than two examples.
Either tag or both may be present, with or without a pre-existing output gain
value.

The intention of this change, after much discussion, was specifically not to
constrain how an encoder should split an R128 gain between the output gain
field and the tags.  Hence: advice rather than normative language.  Part of the
reason is quickly apparent in the discussion of whether to place the track gain
or album gain in the output gain field, and the answer was not to specify.

--ian

---

From: Ralph Giles <[email-masked]>
Date: Wed, 27 Aug 2014 08:59:16 -0700

> and media frameworks should apply it by default.", which I think is the

That's another one of those non-normative 'should' clauses we'd like to
resolve. The intention was that players MUST apply the gain; the
exception was for things like transcoders or DAW systems which could
propagate the gain without altering the decoded data. Can you think of
others?

I was trying to clarify that here, while leaving similar wiggle room in
the word 'respect'.

>> Is there a reason we shouldn't just have two examples?

Only brevity. The previous example used album gain because it was
offering guidance on how to record album gain in the absence of an
R128_ALBUM_GAIN tag. I switched to track gain because I thought that was
the more common normalization.

"Implementations of this specification MUST respect the 'output gain'
field, but MAY NOT respect the comments. Encoder authors are advised to
take this into account. For example, it is more robust for a
post-processing application to performing track normalization to update
the 'output gain' field and write a comment 'R128_TRACK_GAIN=0' than to
put the normalization value directly in the comment."

 -r

---

From: Ron <[email-masked]>
Date: Thu, 28 Aug 2014 06:51:14 +0930

On Wed, Aug 27, 2014 at 03:30:43PM +0100, Ian Nartowicz wrote:
> On Mon, 25 Aug 2014 16:11:21 -0700 Ralph Giles <[email-masked]> wrote:
> 

It really always has been intended as a MUST (but we know there are some
VERY RARE exceptions where you won't).

No general purpose player should ever provide a button to disable this.

The floating point version of Opus can ostensibly record samples in the
range of +/- FLT_MAX, and while the reference implementation places some
arbitrary "you've got to be kidding" lower bound on that, it's still
several times greater than the fixed point range, and there is no reason
an alternative implementation might not allow a larger range.

There's at least one test sample that is way below the range of hearing
without the output gain applied, and is only audible after it is, and
there is no reason that real world samples might not exist in the
opposite configuration - where they play at a normal level with the
gain applied, but would blow your speakers and eardrums out without it.

If someone played such a sample, which they thought they knew what it
contained having played it elsewhere, in a player that disables this,
you're potentially going to make them very, very sad.  Or at the very
least, surprised and annoyed.

The output gain is a guarantee to whoever mastered the recording that
"by default" this is the output level that will be seen.  It lets them
take the raw data of their latest rocket launch or atomic test and
'losslessly' adjust the gain to more reasonable listening levels, for
whatever they decide is 'reasonable' for that file, making it suitable
for more general use without needing to re-encode it.

> confusing, but I think the solution is not to say MUST.

My 'objection' to the "By default" was actually that it watered down the
MUST without providing specific guidance as to why most implementers
shouldn't assume they are the special snowflakes the exceptions apply to.

Your confusion in thinking that it's not really a MUST for a general
purpose player, is exactly why I think we need to strengthen and
clarify that to avoid having inconsistent players and sad users :)

> There are exceptions to the default.  Trying to specify them all here
> would seem an impossible feat of guesswork, but since they exist then
> MUST is too strong.

I don't think we need to specify them exhaustively to be able to give
some guidance for what they might be (or perhaps more importantly,
what they definitely are NOT).

In a general sense, really the *only* exception is: "I want to analyse
the raw signal level, in order to further adjust the output_gain and/or
other gain reference tags".

As soon as you say "I want to play this to a listening human", your
exception to the MUST is null and void.  They have a volume knob
which does that job, and it adjusts the level after the output_gain
correction is made (or possibly automatically with guidance from
the comment tags).

Personally, I think that exception is obvious enough in its own right
and well within the existing language elsewhere that we can simply
say MUST with no exceptions here (ie. an encoder that is going to
set the output_gain obviously must be able to measure the raw level
it is going to apply that to -- but that's not the same as playing
it to a listener at that level).

If we need some further language to clarify that though, then yes
we definitely ought to add it.  I can elaborate on the intention,
but we really do need you to tell us what vital clue is missing
that means that isn't clear in your reading of it.  If you think
we should drop the MUST, then clearly something is still not clear
enough here yet :)

> field and the tags.

That's actually never been constrained, and nothing in the language
proposed here should have changed that.  The output_gain has *never*
been constrained to any particular calibration other than "what the
person who made this file thinks is best", and any R128 gain was
*always* to be applied after the output_gain correction.

> Hence: advice rather than normative language.

We're providing advice about how you might use R128 tags and whether
or not a player needs to respect them.  The output_gain always has
been, and MUST be, normative and mandatory for all players at all
times.

> Part of the reason is quickly apparent in the discussion of whether to
> place the track gain or album gain in the output gain field, and the
> answer was not to specify.

I thought the only confusion there was "how do I turn off album gain
if there isn't an explicit album gain tag?" - given that you MUST NOT
disable the output_gain, which may or may not have been calibrated to
R128 album gain levels.

You still MUST NOT turn off output gain, even if R128_ALBUM_GAIN=0
is explicitly specified, since the raw file still could be at "I'm
50 meters from ground zero of an atomic test" levels.  That still
doesn't imply *anything* about what the output_gain is or means.

But we've given you an explicit album gain tag now that you can
switch on or off if it's not zero.  I'm not sure what the remaining
confusion really is, but indeed, let's find it and clear it up.

  Cheers,
  Ron

---

From: Ron <[email-masked]>
Date: Thu, 28 Aug 2014 06:56:55 +0930

> put the normalization value directly in the comment."

s/to performing/performing/ ?  Or maybe "that is performing"?

But yeah, modulo whatever still seems to be misleading Ian,
that seems about right to me.

  Ron

---

From: Ian Nartowicz <[email-masked]>
Date: Wed, 27 Aug 2014 23:54:38 +0100

<snip>

>confusion really is, but indeed, let's find it and clear it up.
And so we're back to square one.

There's no confusion here.  I have pointed up a perfectly reasonable and
widespread requirement to sometimes, at the user's choice, play back music *as
encoded* (and sometimes normalised to R128).  You don't accept that this
is a legitimate requirement, even making up bizarre examples to somehow refute
real things that are happening in real software.

I do understand the desire for "just works" level adjustment in the majority of
software, but I ask for recognition that some specialist software requires more
stringent control of the playback parameters.  I even offered what I believe is
a better all-round solution of the output gain setting the desired playback
level and the two tags each containing the *entire* R128 normalisation
adjustment, to be subtracted out if desired while still leaving the rest of the
output gain in place.

Whenever we get close to a form of words that accepts this requirement as a
possibility, albeit far short of properly supporting it, it slips away again.
I'm out of ideas.

--ian

---

From: "Timothy B. Terriberry" <[email-masked]>
Date: Wed, 27 Aug 2014 16:23:30 -0700

> and media frameworks should apply it by default.", which I think is the

I agree with Ian. The point was to clean up usage of an un-capitalized 
RFC 2119 keyword that we all agree was not meant to be normative. We 
shouldn't be adding NEW normative requirements in its place.

We have not to this point had any problem getting players to apply the 
output gain by default with the current spec language (and Greg 
Maxwell's test files that tell people their player is broken if they 
don't implement this). We've had to cajole a few, but once we have, I 
believe it has been fixed in every case.

-With my individual hat on

---

From: Gregory Maxwell <[email-masked]>
Date: Wed, 27 Aug 2014 16:30:55 -0700

On Wed, Aug 27, 2014 at 4:23 PM, Timothy B. Terriberry
<[email-masked]> wrote:
> I agree with Ian. The point was to clean up usage of an un-capitalized RFC
> 2119 keyword that we all agree was not meant to be normative. We shouldn't
> be adding NEW normative requirements in its place.
>
> output gain by default with the current spec language (and Greg Maxwell's
> test files that tell people their player is broken if they don't implement
> this). We've had to cajole a few, but once we have, I believe it has been
> fixed in every case.
>

Part of the reason for this is that the libopus as of 1.1 (IIRC) makes
it very easy to handle this— you just make a ctl and pass the content
of the tag, and libopus handles the gain internally... so no DSP for
implementers. (And it handles it in a way that expands the dynamic
range for fixed point).

Alas, not even RFC6919 has a "MUST unless you actually know better".

---

From: Jean-Marc Valin <[email-masked]>
Date: Wed, 27 Aug 2014 20:33:20 -0400

> Alas, not even RFC6919 has a "MUST unless you actually know better".

No, but 2119 does and it's spelled S-H-O-U-L-D :-)

	Jean-Marc

---

From: Ron <[email-masked]>
Date: Thu, 28 Aug 2014 13:10:51 +0930

> And so we're back to square one.

Well, if you think that, then maybe it's *you* not making your case
clearly to us :)

> encoded* (and sometimes normalised to R128).

The output_gain is part of the encoding (not a tag that can be freely
stripped away or ignored).  If you want to play the music *as encoded*
that includes the hardware gain.

If it doesn't, and that isn't "guaranteed", then you've basically made
that useless to anyone mastering a recording, and their only option
for robust behaviour in the face of people who insist on making players
that ignore it is to completely re-encode it from scratch.

Which leaves us not only back with the original problem that this was
designed to solve, but with a worse than useless field in the header
that nobody will actually know how it really behaves in the real world.
And you don't get what you want from it either, because the person who
mastered it, irretrievably destroyed that information when they made
their track "guaranteed to work on broken players too".
Everyone loses if you add uncertainty here.

There is one gain knob that is entirely at the discretion of the person
mastering the recording.  We've now given you two that users and players
can fiddle with to their heart's content.

Why do you need yet another one, and why should people making masters
not have one they can rely on to be an integral part of the encoding?

> You don't accept that this is a legitimate requirement, even making up
> bizarre examples to somehow refute real things that are happening in
> real software.

The only requirement that you made clear to me was you wanted a
separate, clearly defined R128_ALBUM_GAIN as well as the track gain.

We've given you that.

If you're going to handwave about other "perfectly reasonable and
widespread requirements", you're going to have to actually give some
examples to make that case.  Especially if it comes at the cost of
the perfectly reasonable requirement for people mastering tracks to
be able to losslessly fix something that was originally (and quite
possibly accidentally) recorded at an entirely unlistenable and
unreasonable level.

We've given you examples of the originally intended use.  You can
brand them bizarre if you like, but badly captured original
recordings that need level adjustment to be actually useable are
hardly something unheard of.  Why force people to re-encode a live
opus capture to make it useable, when they can trivially correct
it instead?

What specific counterexample do you have that is so important it
should make that effectively impossible to do reliably?

  Ron

---

From: [email-masked]
Date: Thu, 28 Aug 2014 10:55:12 -0400

On Monday, August 25 2014, "Ralph Giles" wrote to "[email-masked]" saying:

> are advised to take this into account.

"MAY NOT" isn't an RFC 2119 keyword, because it's ambiguous in English.  I
assume you mean "MAY ignore the comments", not "MUST NOT respect the
comments"?

-- 
Jonathan Lennox
[email-masked]

---

From: Ralph Giles <[email-masked]>
Date: Thu, 28 Aug 2014 09:05:00 -0700

> assume you mean "MAY ignore the comments", not "MUST NOT respect the

Good point! Thanks.

 -r