Template lookup order is kind of limiting

[richtera]

I traced through the layout lookup code and I found the following for lookup. The upper case values are the actual values in the post and it's folder name and output type.

[0]:"TYPE/LAYOUT.LANG.OUTPUT_TYPE.html"
[1]:"TYPE/SECTION.LANG.OUTPUT_TYPE.html"
[2]:"TYPE/section.LANG.OUTPUT_TYPE.html"
[3]:"TYPE/list.LANG.OUTPUT_TYPE.html"
[4]:"TYPE/LAYOUT.OUTPUT_TYPE.html"
[5]:"TYPE/SECTION.OUTPUT_TYPE.html"
[6]:"TYPE/section.OUTPUT_TYPE.html"
[7]:"TYPE/list.OUTPUT_TYPE.html"
[8]:"TYPE/LAYOUT.LANG.html"
[9]:"TYPE/SECTION.LANG.html"
[10]:"TYPE/section.LANG.html"
[11]:"TYPE/list.LANG.html"
[12]:"TYPE/LAYOUT.html"
[13]:"TYPE/SECTION.html"
[14]:"TYPE/section.html"
[15]:"TYPE/list.html"
[16]:"SECTION/LAYOUT.LANG.OUTPUT_TYPE.html"
[17]:"SECTION/SECTION.LANG.OUTPUT_TYPE.html"
[18]:"SECTION/section.LANG.OUTPUT_TYPE.html"
[19]:"SECTION/list.LANG.OUTPUT_TYPE.html"
[20]:"SECTION/LAYOUT.OUTPUT_TYPE.html"
[21]:"SECTION/SECTION.OUTPUT_TYPE.html"
[22]:"SECTION/section.OUTPUT_TYPE.html"
[23]:"SECTION/list.OUTPUT_TYPE.html"
[24]:"SECTION/LAYOUT.LANG.html"
[25]:"SECTION/SECTION.LANG.html"
[26]:"SECTION/section.LANG.html"
[27]:"SECTION/list.LANG.html"
[28]:"SECTION/LAYOUT.html"
[29]:"SECTION/SECTION.html"
[30]:"SECTION/section.html"
[31]:"SECTION/list.html"
[32]:"section/LAYOUT.LANG.OUTPUT_TYPE.html"
[33]:"section/SECTION.LANG.OUTPUT_TYPE.html"
[34]:"section/section.LANG.OUTPUT_TYPE.html"
[35]:"section/list.LANG.OUTPUT_TYPE.html"
[36]:"section/LAYOUT.OUTPUT_TYPE.html"
[37]:"section/SECTION.OUTPUT_TYPE.html"
[38]:"section/section.OUTPUT_TYPE.html"
[39]:"section/list.OUTPUT_TYPE.html"
[40]:"section/LAYOUT.LANG.html"
[41]:"section/SECTION.LANG.html"
[42]:"section/section.LANG.html"
[43]:"section/list.LANG.html"
[44]:"section/LAYOUT.html"
[45]:"section/SECTION.html"
[46]:"section/section.html"
[47]:"section/list.html"
[48]:"_default/LAYOUT.LANG.OUTPUT_TYPE.html"
[49]:"_default/SECTION.LANG.OUTPUT_TYPE.html"
[50]:"_default/section.LANG.OUTPUT_TYPE.html"
[51]:"_default/list.LANG.OUTPUT_TYPE.html"
[52]:"_default/LAYOUT.OUTPUT_TYPE.html"
[53]:"_default/SECTION.OUTPUT_TYPE.html"
[54]:"_default/section.OUTPUT_TYPE.html"
[55]:"_default/list.OUTPUT_TYPE.html"
[56]:"_default/LAYOUT.LANG.html"
[57]:"_default/SECTION.LANG.html"
[58]:"_default/section.LANG.html"
[59]:"_default/list.LANG.html"
[60]:"_default/LAYOUT.html"
[61]:"_default/SECTION.html"
[62]:"_default/section.html"
[63]:"_default/list.html"

And for the baseof lookup.

[0]:"TYPE/LAYOUT-baseof.LANG.OUTPUT_TYPE.html"
[1]:"TYPE/SECTION-baseof.LANG.OUTPUT_TYPE.html"
[2]:"TYPE/section-baseof.LANG.OUTPUT_TYPE.html"
[3]:"TYPE/list-baseof.LANG.OUTPUT_TYPE.html"
[4]:"TYPE/baseof.LANG.OUTPUT_TYPE.html"
[5]:"TYPE/LAYOUT-baseof.OUTPUT_TYPE.html"
[6]:"TYPE/SECTION-baseof.OUTPUT_TYPE.html"
[7]:"TYPE/section-baseof.OUTPUT_TYPE.html"
[8]:"TYPE/list-baseof.OUTPUT_TYPE.html"
[9]:"TYPE/baseof.OUTPUT_TYPE.html"
[10]:"TYPE/LAYOUT-baseof.LANG.html"
[11]:"TYPE/SECTION-baseof.LANG.html"
[12]:"TYPE/section-baseof.LANG.html"
[13]:"TYPE/list-baseof.LANG.html"
[14]:"TYPE/baseof.LANG.html"
[15]:"TYPE/LAYOUT-baseof.html"
[16]:"TYPE/SECTION-baseof.html"
[17]:"TYPE/section-baseof.html"
[18]:"TYPE/list-baseof.html"
[19]:"TYPE/baseof.html"
[20]:"SECTION/LAYOUT-baseof.LANG.OUTPUT_TYPE.html"
[21]:"SECTION/SECTION-baseof.LANG.OUTPUT_TYPE.html"
[22]:"SECTION/section-baseof.LANG.OUTPUT_TYPE.html"
[23]:"SECTION/list-baseof.LANG.OUTPUT_TYPE.html"
[24]:"SECTION/baseof.LANG.OUTPUT_TYPE.html"
[25]:"SECTION/LAYOUT-baseof.OUTPUT_TYPE.html"
[26]:"SECTION/SECTION-baseof.OUTPUT_TYPE.html"
[27]:"SECTION/section-baseof.OUTPUT_TYPE.html"
[28]:"SECTION/list-baseof.OUTPUT_TYPE.html"
[29]:"SECTION/baseof.OUTPUT_TYPE.html"
[30]:"SECTION/LAYOUT-baseof.LANG.html"
[31]:"SECTION/SECTION-baseof.LANG.html"
[32]:"SECTION/section-baseof.LANG.html"
[33]:"SECTION/list-baseof.LANG.html"
[34]:"SECTION/baseof.LANG.html"
[35]:"SECTION/LAYOUT-baseof.html"
[36]:"SECTION/SECTION-baseof.html"
[37]:"SECTION/section-baseof.html"
[38]:"SECTION/list-baseof.html"
[39]:"SECTION/baseof.html"
[40]:"section/LAYOUT-baseof.LANG.OUTPUT_TYPE.html"
[41]:"section/SECTION-baseof.LANG.OUTPUT_TYPE.html"
[42]:"section/section-baseof.LANG.OUTPUT_TYPE.html"
[43]:"section/list-baseof.LANG.OUTPUT_TYPE.html"
[44]:"section/baseof.LANG.OUTPUT_TYPE.html"
[45]:"section/LAYOUT-baseof.OUTPUT_TYPE.html"
[46]:"section/SECTION-baseof.OUTPUT_TYPE.html"
[47]:"section/section-baseof.OUTPUT_TYPE.html"
[48]:"section/list-baseof.OUTPUT_TYPE.html"
[49]:"section/baseof.OUTPUT_TYPE.html"
[50]:"section/LAYOUT-baseof.LANG.html"
[51]:"section/SECTION-baseof.LANG.html"
[52]:"section/section-baseof.LANG.html"
[53]:"section/list-baseof.LANG.html"
[54]:"section/baseof.LANG.html"
[55]:"section/LAYOUT-baseof.html"
[56]:"section/SECTION-baseof.html"
[57]:"section/section-baseof.html"
[58]:"section/list-baseof.html"
[59]:"section/baseof.html"
[60]:"_default/LAYOUT-baseof.LANG.OUTPUT_TYPE.html"
[61]:"_default/SECTION-baseof.LANG.OUTPUT_TYPE.html"
[62]:"_default/section-baseof.LANG.OUTPUT_TYPE.html"
[63]:"_default/list-baseof.LANG.OUTPUT_TYPE.html"

Although this is fairly flexible. I was expecting the sequence to be from more specific to the least specific.

For example. If I have a template

_default/LAYOUT.OUTPUT_TYPE.html

it currently does not override

section/LAYOUT.html

I would agree that this couldn't be easily "changed" without making all kinds of sites fail, but maybe there is a way to enable a new lookup mode. Ideally, it would look for the most specific layout (i.e. kind, layout, language, type, output type) in all folders and then the next least one.

[bep]

I would agree that this couldn't be easily "changed" without making all kinds of sites fail,

You are right about that.

The current lookup rules is from more specific to the least specific, but as you have figured out, specificity is in the eye of the beholder.

[richtera]

My main problem is that overriding a layout of a certain output type needs to be done many times because it looks for layouts without that output type before looking at the default. But I could see an argument to consider section, type and so on the same way. Personally I had expected the lookup to follow your order except consider each filename across the folder sequence first. Therefore the more specific file inside of default would be more important than the less specific file in the posts or section folder.
Would it be possible to add a programmed lookup sequence in the config maybe?

[moorereason]

@richtera,
This would be a major breaking change, so I have a few questions:

How did you work around this issue in your current use case? Or can you? Do you consider this a curious implementation detail or a fundamental flaw? How serious is the problem for you?

[richtera]

I have to make multiple copies of the same file to override it for each view type. It just kind of goes counter to overriding views but does work. I feel that there might be a better way to do the lookups and it might be possible to have a lookup mode type switch for each theme module.

[moorereason]

it might be possible to have a lookup mode type switch for each theme module.

Why is that desirable? Are we trying to build an obstacle course? 😉

[richtera]

My thought was that if we are looking for specific templates before less specific ones, then it should really allow a more specific template inside of _default to override a less specific template inside of the posts or view folder. For example I have a lesson.tmpl.html layout which is a template type output (it's just a document fragment I usually load using fetch from the browser) for a lesson which is now being overridden by lesson.html even though that layout is a full layout for a whole page. So unless I repeat lesson.tmpl.html in every folder I have lesson.html then it's not doing the right thing even if the tmpl.html file is the same in all cases.

[richtera]

I think the only thing that's really hurting is the fact that for baseof lookups in _default doesn't contain a non LANG versions in it.
[60]:"_default/LAYOUT-baseof.LANG.OUTPUT_TYPE.html"
[61]:"_default/SECTION-baseof.LANG.OUTPUT_TYPE.html"
[62]:"_default/section-baseof.LANG.OUTPUT_TYPE.html"
[63]:"_default/list-baseof.LANG.OUTPUT_TYPE.html"
If I specify both LANG and OUTPUT_TYPE on each file it works pretty well, but it maybe nicer to be able to just specify OUTPUT_TYPE. Most of the other lookups are not that necessary in that case. NOTE: This whole resolution list was generated by breaking inside of hugo when resolving a layout for a file that had an OUTPUT_TYPE other than html.

[moorereason]

I would recommend crafting a test case to match your scenario instead of manually creating a lookup list. The largest list we currently test is about 36 entries long. See

hugo/output/layout_test.go

Line 28 in b5d906e

func TestLayout(t *testing.T) {

[richtera]

Ok, will check. I think the main problem I have is that if I want to specify the most specific templates using OUTPUT_TYPE I also have to provide a variation for each language since there is no default lookup for baseof with OUTPUT_TYPE but no language. I have worked around the problem by always specifying OUTPUT_TYPE on all my templates which then will override the particular _default types and they are early in the lookup order for each folder. Like post/layout.en.html.html. I am not quite sure if it makes sense to include non OUTPUT_TYPE lookups in the sequence, but that's very subjective. I didn't expect hugo to consider posts/layout.json before it would consider _default/layout.rss.json when the output type is rss but specifying all fields mostly works.

[moorereason]

@richtera, I submitted a PR that adds some additional scenarios to the tests configuration. We max out at 80 lookups!

Hoping those help this discussion.