Subject: [Tools-implementation] Reflections

From: Russ Housley <[email-masked]>
Date: Thu, 27 Aug 2020 12:17:34 -0400

I think we made an mistake in the way that we went about deploying CodiMD.  The result was that we had to keep warning WG Chairs about changes.  I think we will do better with the the approach that we are taking with Matrix, but there is one aspect that I want to highlight now.  While were are figuring out how to deploy a new piece of infrastructure, putting it in experiment.ietf.org (or similar) is an important label.  Then, we we think it it ready for production, we need to let the Secretariat review what has been done.  They are the ones that need to tackle trouble tickets and calls, so they should have an opportunity to review the configuration before it gets deployed.  More that ten years ago, when we were moving critical applications from tools.ietf.org to the Secretariat-supported servers, this was an important step in the process, and we did not seem to have an explicit handover for CodiMD.

Russ

---

From: Robert Sparks <[email-masked]>
Date: Thu, 27 Aug 2020 11:26:43 -0500

On 8/27/20 11:17 AM, Russ Housley wrote:
> I think we made an mistake in the way that we went about deploying CodiMD.  The result was that we had to keep warning WG Chairs about changes.

Elaborate on this please.

I can't see a path in retrospect where warning WG chairs about changes 
was avoidable - it would have happened even if we had jumped into 
running it at the secretariat from the beginning.

>   I think we will do better with the the approach that we are taking with Matrix, but there is one aspect that I want to highlight now.  While were are figuring out how to deploy a new piece of infrastructure, putting it in experiment.ietf.org (or similar) is an important label.  Then, we we think it it ready for production, we need to let the Secretariat review what has been done.  They are the ones that need to tackle trouble tickets and calls, so they should have an opportunity to review the configuration before it gets deployed.  More that ten years ago, when we were moving critical applications from tools.ietf.org to the Secretariat-supported servers, this was an important step in the process, and we did not seem to have an explicit handover for CodiMD.
>
> Russ
>

---

From: Russ Housley <[email-masked]>
Date: Thu, 27 Aug 2020 12:32:40 -0400

> On Aug 27, 2020, at 12:26 PM, Robert Sparks <[email-masked]> wrote:
> 
> 
> I can't see a path in retrospect where warning WG chairs about changes was avoidable - it would have happened even if we had jumped into running it at the secretariat from the beginning.

Yes, we need to do the warnings, regardless of where it is deployed, but codimd.experiment.ietf.org <http://codimd.experiment.ietf.org/> and codimd.ietf.org <http://codimd.ietf.org/> give a very different impression from the start.

> -- 
> Tools-implementation mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tools-implementation

---

From: Glen <[email-masked]>
Date: Thu, 27 Aug 2020 13:20:57 -0700

Russ -

On Thu, Aug 27, 2020 at 9:32 AM Russ Housley <[email-masked]>
wrote:> On Aug 27, 2020, at 12:26 PM, Robert Sparks
<[email-masked]> wrote:
> I think we made a mistake in the way that we went about deploying CodiMD.
> The result was that we had to keep warning WG Chairs about changes.

Another concern: As we move to containers, it is important that the
operations staff have insight into the way that each container is
built. This is important so that we can debug in the middle of the
night. We need documentation and source control. Ideally, we would
have a way to roll back to previous release versions as we do with the
datatracker.

> Yes, we need to do the warnings, regardless of where it is deployed, but codimd.experiment.ietf.org and codimd.ietf.org give a very different impression from the start.
> I think we will do better with the the approach that we are taking with Matrix,

Yes, I agree the new naming scheme/expectations will be better.

> but there is one aspect that I want to highlight now. While were are figuring out how to deploy a new piece of infrastructure, putting it in experiment.ietf.org (or similar) is an important label. Then, we we think it it ready for production, we need to let the Secretariat review what has been done. They are the ones that need to tackle trouble tickets and calls, so they should have an opportunity to review the configuration before it gets deployed. More that ten years ago, when we were moving critical applications from tools.ietf.org to the Secretariat-supported servers, this was an important step in the process,

I believe that the Secretariat should set up the production instance.
That way we will know what is going on. Even in the case of a
bespoke/custom piece of code, but *especially* in the case of an
open-source package, we need to know the configuration. We can't just
"review" a piece of open-source code, especially one that has been
patched. We have to *know* what is going on, know the setup, and have
complete documentation for it. This is important for us, as the people
responsible for operating the IETF's IT services (not designing, just
operating).

I am really glad that we're not repeating our approach with these new
messaging services, and I think we will do better on this approach in
this aspect of things as well.

> and we did not seem to have an explicit handover for CodiMD.

We still do not have an explicit handover for CodiMD, and it would be
good to do that in a formal way.

My intention is to set up a new instance of CodiMD at the same time I
set up Matrix, Zulip, and IETF-FTP. I've submitted a request to Robert
for details and documentation about his setup, any customizations and
patches, and I will apply those to the new CodiMD setup when I do that
work. This will enable us properly be the operations staff of CodiMD.

Glen