Subject: [TOOLS-DEVELOPMENT] Search across the web site, mail archive, and datatracker

From: Russ Housley <[email-masked]>
Date: Fri, 25 May 2018 14:44:36 -0400

We have been asked to look into a search feature that is homed at www.ietf.org that will search across the web site, mail archive, and datatracker.

The way I use the IETF website, I know when I am searching for meeting-related material, I-D-related material, or mail-archive-related material.  I rarely want to mix these things, so I am not the right person to make suggestions about the way forward.

That said, others, especially folks that are new to the IETF, come to www.ietf.org looking for information about IETF work or a particular standard, and the current search does not help them enough.  They probably don’t even know about datatracker. For example, they would expect to get results about a current work item, say TLS 1.3, that includes information that we know is in the datatracker, especially the most recent Internet-Draft.

My first thought was to see if we could leverage Google search.  The SoW for the recent website revamp says that core functions must not require javascript. According to the documentation from Google (https://developers.google.com/custom-search/docs/overview) custom search does require javascript.  Since search ought to be a core function, the Google custom search is not a way forward.

Does anyone have thoughts about how to approach this search feature?

Russ

---

From: Robert Sparks <[email-masked]>
Date: Fri, 25 May 2018 14:12:40 -0500

My first thought (which assumes that the search function already on 
www.ietf.org is sufficient for searching the website - is that a correct 
assumption?) is to provide an api from the datatracker and the 
mailarchive that would take the search term and return what's needed 
(json objects, I think) to allow the www website to nicely show the 
results grouped by type. We could make a view there that doesn't use js 
(waits for results before displaying everything, then displays 
_everything), but could provide a better experience for folks that are 
willing to let js run (starts showing results as they come in, might 
provide a way to collapse the results, in a tree perhaps).

RjS

On 5/25/18 1:44 PM, Russ Housley wrote:
> _______________________________________________
> TOOLS-DEVELOPMENT mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tools-development

---

From: Bob Hinden <[email-masked]>
Date: Fri, 25 May 2018 13:41:59 -0700

Russ,

> On May 25, 2018, at 11:44 AM, Russ Housley <[email-masked]> wrote:
> 

> That said, others, especially folks that are new to the IETF, come to www.ietf.org looking for information about IETF work or a particular standard, and the current search does not help them enough…

I am not convinced that an IETF wide search function like you describe will be very useful   I worry it will either come up with too many hits, or the wrong ones.  For example, I think a search for “IPv6 neighbor discovery” will return thousands of emails, hundreds of drafts, and tens of RFCs, and a lot of IPv6 and 6man w.g. agendas.  Somewhere in the vast amount of returns one might figure out that 6man is doing an update to IPv6 Node Requirements that includes Neighbor Discovery items.  Is this useful?

In the case you describe, we would have to formally define what an “work item” means, and have a formal list.   Otherwise the search tool is know to have to know what the user is intending to search for, that seems difficult to me.

Bob

---

From: Russ Housley <[email-masked]>
Date: Fri, 25 May 2018 17:49:24 -0400

> I am not convinced that an IETF wide search function like you describe will be very useful   I worry it will either come up with too many hits, or the wrong ones.  For example, I think a search for…

I made a similar comment, and the ranking of the search hits will certainly be vital to the tools being useful.

> In the case you describe, we would have to formally define what an “work item” means, and have a formal list.   Otherwise the search tool is know to have to know what the user is intending to searc…

Interesting thought.  I wonder if it would be hard to process the charter text or charter milestones to help create a set of active work items.

Russ

---

From: Ryan Cross <[email-masked]>
Date: Fri, 25 May 2018 15:04:14 -0700

I imagine it would be possible to create an Elasticsearch index for each type of “document”, draft, email, webpage, etc.  You could filter by type and indicate the type in search results, using an icon for example.

Ryan

> On May 25, 2018, at 12:12 PM, Robert Sparks <[email-masked]> wrote:
> 
> My first thought (which assumes that the search function already on www.ietf.org is sufficient for searching the website - is that a correct assumption?) is to provide an api from the datatracker and the mailarchive that would take the search term and return what's needed (json objects, I think) to allow the www website to nicely show the results grouped by type. We could make a view there that doesn't use js (waits for results before displaying everything, then displays _everything), but could provide a better experience for folks that are willing to let js run (starts showing results as they come in, might provide a way to collapse the results, in a tree perhaps).
> 
> RjS
>