Subject: Re: [codec] draft-ietf-codec-oggopus and "album" gain

From: Ron <[email-masked]>
Date: Mon, 8 Sep 2014 02:01:26 +0930

On Thu, Aug 28, 2014 at 06:56:55AM +0930, Ron wrote:
> On Wed, Aug 27, 2014 at 08:59:16AM -0700, Ralph Giles wrote:
> > 
> > "Implementations of this specification MUST respect the 'output gain'
> > field, but MAY NOT respect the comments. Encoder authors are advised to
> > take this into account. For example, it is more robust for a
> > post-processing application to performing track normalization to update
> > the 'output gain' field and write a comment 'R128_TRACK_GAIN=0' than to
> > put the normalization value directly in the comment."
> 
> s/to performing/performing/ ?  Or maybe "that is performing"?
> 
> But yeah, modulo whatever still seems to be misleading Ian,
> that seems about right to me.

So I believe this is now the last outstanding question from the last
call comments that we still have open.

There were some concerns raised about this wording, but my questions
about exactly what the problem Ian had with it was have so far gone
unanswered, which makes it a bit tricky to try to address whatever
they really were, and I haven't seen a proposed alternative wording
that covers the problem Ralph and I saw with the initial proposed
edit leaving it without a rationale (which this version tried to fix).

So if someone still thinks this says something that it shouldn't, or
isn't properly clear about what it does say, could you please give
us a sufficient explanation of that so we can understand and fix the
problem, or propose an alternative wording that you think does which
we can discuss.

Otherwise, this is so far the closest to what I think was originally
intended that we have on the table to accept.

If we get this one out of the way now, I think we're nearly done.

  Thanks,
  Ron

---

From: Ron <[email-masked]>
Date: Mon, 8 Sep 2014 06:03:45 +0930

On Sun, Sep 07, 2014 at 06:06:07PM +0100, Ian Nartowicz wrote:
> On Mon, 8 Sep 2014 02:01:26 +0930
> Ron <[email-masked]> wrote:
> 
> Sorry, I didn't realise there were outstanding questions.  Perhaps if I give a
> concrete example, it will show where I'm coming from:
> 
> In this context, a common use will be to encode a given (probably lossless)
> source into Opus.  The requirement here is to (optionally) be able to play back
> the Opus at the same level as the original source, even if there are R128 gain
> tags present.

I'm not sure what the distinction you're making about 'level' here is
or why it's important for a general purpose player?

[ed: Ian, you might want to skip commenting on this first lot unless
 there is something you *really* disagree with.  I've left it here
 because it is important that we are on the same page about this,
 but if you make it past this to the bottom, I *think* I start to
 see what it is that you're *actually* concerned about ... ]

The R128 tags are indeed a mechanism that people can use to normalise
all of their recordings to some consistent level, so they'll all play
back at about the same volume for a given setting that they've dialled
the volume knob on their player to.

They are completely optional, and we don't mandate that a player must
support them (or use them even if it does), but enough people have
said they wanted that for us to define a standard way to do it if you
do.

My understanding is that having both ALBUM_GAIN and TRACK_GAIN meets
the varied requirements that people wanted for that function.

But the output_gain has no such constraint or meaning attached to it.
It's mandatory to apply precisely because it is part of what defines
the "level of the original source" as defined by whoever mastered
that recording.

The *only* reason it exists is where there is no "lossless source",
since it's quite conceivable that people will make devices which
record directly to Opus, and no "lossless master" will ever exist.
We expect most people will never need to use it, but for the case
where you've accidentally recorded something at a ridiculous level
(whether that is loud or quiet), this gives you a way to losslessly
'fix' that, without having to destructively do a lossy to lossy
re-encoding - which would otherwise be your only option.

We don't say anything about what 'level' you should fix this to
if you do that.  It's entirely at the discretion of whoever goes
"damn, I totally screwed this recording up", and whatever they
decide would be a less screwed up level.

The "playback level" of that file, like any other unnormalised
file, is going to be entirely under the control of the user
adjusting their volume control.  This just lets the person who
mastered it get a file that's way out there, back in the ballpark.

There should be no reason for an 'end user' to ever need or want
to "turn that off" - since at best it will do almost nothing for
them, and at worst it could do Something Terrible.  If they want
a "different" level, they turn their volume knob.  If they want
a "standard" level, they put the file through a tool that adds
R128 tags.

This isn't a "normalisation mechanism" so much as a "tool of last
resort" for salvaging a botched recording that there is no other
good way to save.  If people can't rely on that tool working,
then it's useless to them and they'll just destructively re-encode
in the cases where they really would have needed it and could have
used it.

Does that make sense?  Because if it doesn't, we need to talk
about that more or explain it better in the draft - and if you
think it means something different to that, then we're going to
have a really hard time converging on understanding for anything
else here.

> This requirement will be met, one way or another for good or
> for bad, my me and probably by other authors, but I'd prefer
> to do it in a way that complies with the specification.

Sorry if I'm appearing to be thick here, but I still don't understand
what "This requirement" is?  You've told me "what" you want, but not
*why* that is actually needed or can't already be done with what we
have.

Your example above seems to hinge on there being some prior lossless
source, but that's not the case where the output_gain is useful.

If you have other lossless source, then generally you can simply just
re-encode the file from scratch if you somehow botched the gain on
the transcode, you don't need the output_gain knob to fix that.

> Clearly this is not possible in all cases because there is no explicit
> identification of the original source level or the total amount of gain
> attributable to the R128 tags, but so long as the spec has enough flexibility
> to allow a use case then I can work with it.  Applying normative language to
> placing one of the R128 values into the output gain and then requiring that it
> be applied in all cases did not seen sufficiently flexible.

Hmm.  So your concern is actually with the part that suggests people
MAY wish to put the base normalisation in output_gain and set R128=0?

Not actually with the fact we say you MUST always apply output_gain?

It's not a normative requirement that they do it that way (as opposed
to instead setting output_gain=0 and R128=X) -- it was really just a
clarification that the R128 tags are Optional, and that if you *do*
want the "original level" to be normalised, then using output_gain
would guarantee that "robustly".

But yes, I think I can see how that advice may steer people toward
creating files where a lossless master existed, and wasn't originally
normalised but the Opus encoding is, with no certain way for people
to be able to reliably "unnormalise it" again.

You might for some files be able to deduce that moving the output_gain
value to an R128 tag and setting output_gain=0 *could* do that, but
there is no way to be certain that's true for every file (purely because
the file might be of the sort I described above, but rather than pick
some random level that makes it "not ridiculous" anymore, they did
actually normalise it to R128 when they fixed it, and the original
might still be 'dangerously' useless - though you could again apply
some heuristic based on the amount of gain adjustment seen ...)

That does seem to be an unintended consequence of the "For example"
advice.  Which although not normative in itself, might not actually
be the preferred default for people to use.

> The original
> language of my patch left it entirely up to the encoder and subsequent tools
> where to place the R128 gain values, allowing for them to be placed entirely in
> the tags and therefore identified.  Or not, but that's at least under the
> control of the user.

My main problem with that one was it removed all the context explaining
when and why people might use this and left it too vague for there to
be some reasonable consistency among how people interpret it (since if
everyone does something different, we'll just have a complete mess,
worse that if we allowed no gain tags at all).

But yes, the intent was certainly not for there to be information loss
somewhere as a result of doing this.

So in that light, I think this part should stand:

> field, but MAY NOT respect the comments.

Because it's a simple statement that reaffirms tags are always optional
and the header gain is an intrinsic part of the "original recording".

But this part:

> Encoder authors are advised to

Possibly needs to give some slightly different advice, at least for the
case where R128 normalisation is something that someone retrofits to
a recording that was previously mastered to some acceptable level, to
avoid actually losing useful information about the original in that case.

Does that seem like it would cover what you're worried about?

Do you have some proposed language for what you'd like us to recommend
people do if they want to retrofit R128 normalisation to recordings?

  Cheers,
  Ron

---

From: "Timothy B. Terriberry" <[email-masked]>
Date: Mon, 08 Sep 2014 07:00:30 -0700

> call comments that we still have open.

I think there were a few more non-normative "shoulds" that needed to be 
cleaned up. Did you and Ralph have some suggestions for those? I didn't 
see any on the list.

---

From: Ian Nartowicz <[email-masked]>
Date: Mon, 8 Sep 2014 16:46:02 +0100

>people do if they want to retrofit R128 normalisation to recordings?

I hadn't considered language that goes any further than leaving the issue
unspecified.  Actively encouraging encoders not to put R128 gains into the
output gain field seems to go directly against a key design goal for Opus.
Although I'd be happy to see this particular case recognised, do we want to to
second-guessing users and filling the spec with exceptions?

In an ideal world I'd still like to see a solution that produces normalised
output by default for all players, with sufficient information for more
advanced players to "de-normalise" out a specific R128 gain when needed.

--ian

---

From: Ron <[email-masked]>
Date: Tue, 9 Sep 2014 02:14:40 +0930

On Mon, Sep 08, 2014 at 07:00:30AM -0700, Timothy B. Terriberry wrote:
> cleaned up. Did you and Ralph have some suggestions for those? I didn't see
> any on the list.

Ralph went through those and pushed changes to replace the stray
lowercase shoulds with alternative language, and those looked ok
to me and shouldn't be controversial.  Several of them read better
without the shoulds anyway and I don't think any of them change
the intended meaning where they happened.

There may still be a couple of MAYs that we're using in the natural
language sense which might not quite fit the BCP14 sense of an
"optional item", but that's possibly a question for better language
lawyers than me to give an opinion on.

That said, they indeed haven't been discussed or vetted here yet,
and I agree they should be.  Given the number of "minor editorial
changes" that have occurred in response to various comments, I
guess we should probably wrap up whatever language is needed to
meet Ian's concern then push out another draft that people can
review in its entirety?

I'd certainly welcome people reviewing those things in git, but
it might be a bit much to expect them to do so that way.

If there's a better plan than that, I'm open to it though.

  Ron

---

From: Ron <[email-masked]>
Date: Tue, 9 Sep 2014 03:27:41 +0930

On Mon, Sep 08, 2014 at 04:46:02PM +0100, Ian Nartowicz wrote:
> unspecified.

I don't think we should be leaving it entirely unspecified, that just
guarantees that nobody will get what they wanted from adding the R128
tags in the first place, since everybody will do something different
and in ways that subtly conflict with each other or produce surprising
behaviour in some players.

If we were to go that way, we should just say the header gain is
strictly mandatory and simply remove all reference to any sort of
"normalisation" tags.  But some people wanted us to specify those
tags, precisely to end the random state of madness that currently
exists with them elsewhere.  So if we do that, we should specify
a way that people agree works, and doesn't have obvious problems.

> Actively encouraging encoders not to put R128 gains into the

No, I don't think that's the case.  I explained what the header gain
field was designed for, and it has nothing to do with "normalisation",
it's purely to allow *arbitrary* lossless correction where the only
other alternative might be a destructive re-encoding.

The only reason you'd really want to put an R128 correction in there is
if you really *always* wanted that to be the "original level" of the
file.  In a case where that was simply an advisory thing, and preserving
the "original level" of some previous master was also important, then
putting that into the advisory R128 tags seems perfectly correct to me.

I don't really see how that has anything to do with the "design goals"
of Opus, let alone be in conflict with them.

Unless you're going to tell me that I'm still completely failing to
understand what your actual concern is, the only problem I'm seeing
is that we've accidentally entangled an example of how the mandatory
and optional gains interact, in a way that might lead some people to
encode R128 normalisation suboptimally for some cases.

Which is indeed unintentional, and if so we should fix that.

Probably by splitting the example of how gains interact from the
advice we give about how optional R128 normalisation should be
indicated.

> Although I'd be happy to see this particular case recognised,
> do we want to to second-guessing users and filling the spec
> with exceptions?

I don't see where we'd be second guessing anybody, or needing to
make any exceptions at all to clarify this?

The basic rules are simple.

 - Mandatory gain is mandatory.  It's in the header.
 - Optional gain is optional.  It's in the comment tags.
 - Optional gain is always in addition to mandatory gain.

Where we might have made a bit of a mess that needs to be cleaned
up further, is that previously people thought we would only need
one R128 gain tag for TRACK_GAIN, and that "for simplicity" if
people wanted ALBUM_GAIN, they would either set that level while
encoding, or tweak the header gain to retrofit it.

Ultimately, other people made a convincing argument for also
having a separate R128_ALBUM_GAIN tag.  Which really does free
up the header gain to again *only* be used for its original
purpose of "arbitrary lossless correction as a tool of last
resort".

But it appears some of the language from prior to that hasn't
yet completely been untangled, and we're hinting to people that
they might prefer to use the mandatory gain, because it is
guaranteed to be applied and is therefore "more robust", but
we're using R128 gain as an example for that -- when the people
who Really Care about R128 normalisation actually do want that
to remain optional.

It was never the intent here that R128 normalisation should be
in any way mandatory.  The example was just intended to show
clearly how mandatory and optional gain are combined.

What I'm getting from your comments is we need an example that
looks less like advice for how to add R128 normalisation, and
perhaps some better advice for what the best way to add that
really is - something more like "If it's optional, *always*
prefer to put it in the tags, not the header gain".

Am I somewhere close to the mark there?  I *thought* I understood
what you wanted before the first change, but then we accidentally
made you sad trying to clarify that for other concerns too.

> advanced players to "de-normalise" out a specific R128 gain when needed.

Well that should be 'easy', you just need to convince the player
authors to support the R128 tags, and turn them on by default
(assuming that doesn't cause their own users to revolt).

Our job here is just to be clear about how you can encode a file
to make that possible, when it is what you desire for that file.

We need the mandatory gain for lossless correction, but that's
an entirely separate issue to providing good advice for how to
make the best use of the optional gain tags for normalisation.

Only specialised editors should be "ignoring" the mandatory gain,
but it seems quite reasonable for an ordinary end user player to
be able to toggle R128 gain at the user's preference.

That's always been the intent.  If that's not what the words say
then we should fix the words so they do.

  Ron

---

From: Ian Nartowicz <[email-masked]>
Date: Mon, 8 Sep 2014 19:19:32 +0100

On Tue, 9 Sep 2014 03:27:41 +0930

>then we should fix the words so they do.

I would be entirely happy if it was recommended that R128 gain values were
stored in their tags and not normally in the output gain field, but I've felt
resistance to that idea before.  The spec itself advised the opposite and
explained the reason: everyone will apply the output gain, not everyone will
apply the tag gain.

Do we agree about this?  Might be best to wait for some other comments.

--ian

---

From: Ron <[email-masked]>
Date: Tue, 9 Sep 2014 06:06:04 +0930

On Mon, Sep 08, 2014 at 07:19:32PM +0100, Ian Nartowicz wrote:
> resistance to that idea before.

The original push-back as I recall it was against the idea of needing
a separate R128_ALBUM_GAIN tag at all -- since the original thinking
was header gain could be used for that, if it wasn't already done
during the initial encode, so only a separate R128_TRACK_GAIN was
needed for "alternative normalisation".

But now that we've also added R128_ALBUM_GAIN, it would be silly not
to recommend using it in the way that the people who wanted it, wanted
to be able to use it.

I think any remaining language that indicates otherwise is mostly an
oversight, partly because the people who proposed the changes to the
text weren't the people who pushed to have this added, and partly
because it was done by making minimal changes to the existing text.

"We knew what we meant", but if it doesn't actually say that to others,
then yes we ought to fix that.  This is the problem with being too
close to your own writing :/

> The spec itself advised the opposite and explained the reason:
> everyone will apply the output gain, not everyone will apply the
> tag gain.

Right, but that's only a reason to do it that way if it's important
to you that *everyone* always applies the gain you set.  Which is an
important use case, but not the one you've been concerned about.

In the case where the gain you are indicating really is advisory,
as you've said you really want the R128 normalisation to be (and
which it makes sense to me that in many cases they should be),
then it's absolutely appropriate to only use the tags for that.

The spec doesn't currently say you *can't* do that, or even that
you SHOULD NOT do it, but I agree the example given might mislead
some people into doing that when it's probably not what they really
wanted, or what's best for the case you describe.

> Do we agree about this?  Might be best to wait for some other comments.

I think Ralph now also sees this in a similar way, and I think I
can fairly safely say without fear of contradiction that it was
never intended for R128 normalisation to be mandatory.

It just took us a while to convert the Solution you were proposing
into an actual Problem we could wrap our heads around, since we
were reading the text as already allowing you to do this, and blind
to the example's urging that maybe you shouldn't - so we couldn't
see what your objection to the header gain MUST really was.

I think mostly now someone just needs to proofread the troublesome
bits again, and propose some text that more plainly says what we
mean, what is required and why, and gives some better guidance as
to what Best Practice for optional normalisation tags should be.

When we can agree on that, we'll know we're really all on the same
page about all of this :)

  Thanks!
  Ron