Subject: [Tools-discuss] Difficulty using DocumentInfo

From: Adam Roach <[email-masked]>
Date: Thu, 05 Apr 2012 10:39:10 -0500

At the most recent code sprint, I ran into some difficulty with the 
DocumentInfo class (from ietf/doc/models.py).

As originally envisaged, this model looked well designed. It contained 
all the information you might want to know about a document. In 
particular, if one makes certain assumptions about what is meant by 
field names, it contained both a local filesystem path to the document 
(from "get_file_path") and a handy external URL that could presumably be 
used to access the document from anywhere on the Internet (the field 
"external_url").

The latter is very important for generating links to documents once one 
has retrieved them using whatever query is appropriate for the situation.

However, what has ended up being populated in the "external_url" field 
is emphatically not a URL, and is highly frustrating to use.

For example, if I pull the agenda document for the MMUSIC meeting in 
Paris and check the "external_url" field, I get 
"/proceedings/83/agenda/agenda-83-mmusic.htm". I'm not as bothered that 
it's missing a scheme and an authority as I am that I can't get to a 
document at that path on datatracker. For example, If I do an 
href="/proceedings/83/agenda/agenda-83-mmusic.htm" (which the web 
browser ends up resolving to 
http://datatracker.ietf.org/proceedings/83/agenda/agenda-83-mmusic.htm), 
I get a 404 error.

We really need to fix this up. Either we need to add URL patterns to the 
datatracker tree that successfully the URLs in this field (which would 
require a new pattern for each document type), or we need to repopulate 
the database with values that correctly point to values that can be 
resolved on the datatracker.

Alternately, I note that Ole has been hand crufting URLs together inside 
the templates (e.g. http://datatracker.ietf.org/meeting/{{ meeting.num 
}}/agenda/{{ session.info.acronym }}-agenda/) which most decidedly won't 
work in the general case (e.g., I have a collection of mixed documents 
associated with a meeting session, and want to properly link to each of 
them without a slew of document-type-specific logic). If we move this 
hand-crufting inside the DocumentInfo class in one big switch statement, 
at least it's consolidated all in one place and (from an 
external-to-the-class perspective) handled in a consistent fashion. It 
does mean that one has to open the DocumentInfo class for every new 
document type and add a new case to the switch statement -- but we're 
already going to have to that for the "get_file_path " method, which 
already has such a switch statement.

I guess what I'm asking is: what would people prefer?

(a) Fixing the URL patterns so that the values in 
DocumentInfo.external_url work
(b) Fixing the values in DocumentInfo.external_url so they match the 
existing URL patterns, or
(c) Adding a new method to DocumentInfo that returns a value that can 
always be used in an href (and similar constructs); e.g. get_href()

I can do (a) or (c) myself -- (b) would require that Henrik (or someone 
with similar access to the database) perform the fix-up.

/a

---

From: Adam Roach <[email-masked]>
Date: Thu, 05 Apr 2012 12:50:09 -0500

On 4/5/12 12:11 PM, Ryan Cross wrote:
> The method get_absolute_url() should return the information you are looking for.
>

Actually, I *did* manage to find that method (even though it's on 
Document instead of DocumentInfo). The problem is that it does not work 
-- the resulting URLs don't resolve, at least not for agendas. 
get_absolute_url uses the following model for building agenda path names:

                 url = '%s/proceedings/%s/%s/%s' % 
(settings.MEDIA_URL,meeting.number,self.type_id,filename)

It would appear that the proper form for agendas is:

                 url = '%s/meeting/%s/agenda/%s-agenda/' % 
(settings.MEDIA_URL,meeting.number,session.group.acronym)

And then there are different forms for minutes and slides.

So we're back to the question about whether we should fix 
get_absolute_url() to match ietf/meeting/urls.py or fix 
ietf/meetings/urls.py to match get_absolute_url().

/a

---

From: Henrik Levkowetz <[email-masked]>
Date: Thu, 05 Apr 2012 23:18:47 +0200

Hi Adam,

On 2012-04-05 17:39 Adam Roach said the following:

<snip very complete presentation of problem>

The thing is, the 'external_url' attribute was not intended to be used
as a way to get the url of a document like a draft, agenda, charter,
etc.; which all reside at well-known paths in the server file-system;
it was meant as a way to record the url at which an _external_ document
(not residing on the datatracker server) could be found.

The conversion code has not adhered to this; which means that I failed
to communicate my intentions for this field.

It also means that both (b) and (c) below should be carried out -- for
(b) this means blanking out this field for docs with known locations.

> with similar access to the database) perform the fix-up.

For (c) I would suggest adding a dictionary to settings.py which
took the document prefix (the result of document.name.split('-',1)[0])
as a key, and returned a path format string, which would use a dictionary
of values relevant to the document to fill in the variable spots.

I'd use PEP 3101 format strings rather than the default %-formatting
in order to be able to express such things as zero-filled 2-digit
revision numbers (see http://www.python.org/dev/peps/pep-3101/)

You could have for instance

DOC_PATHS = {
	...
	"draft": "http://www.ietf.org/id/{doc.name}-{doc.rev:02}.txt",
	...
}

to pick a simple example.

The get_href() would then pick the appropriate format string from the
DOC_PATHS setting, and use .format(doc) on it to fill in the blanks
-- *or*, if there were no pattern for the document prefix, return the
doc.external_url if set.

Some utility functions would probably be needed in order to make it
easy to get things such as meeting and group given a document.

(We probably should have another method to get the path of a document
on the local file system.  That could probably be defined in terms of
get_url() together with an url lookup (or it could be done the other
way around, of course)).

Best regards,

	Henrik