Allow more characters when creating various nodes

[domenic]

Stop using XML grammar productions for validating element, attribute, and doctype names. These were overly-restrictive, as it is possible to create nodes with names that don't match those productions using the HTML parser. After this change, each construct has its own custom name validation algorithm.

The only remaining dependency on XML naming rules is for processing instructions, which are uncommon and cannot be created with the HTML parser anyway.

Closes #769. Closes #849. Closes #1373.

• At least two implementers are interested (and none opposed):
  • Gecko
  • Chromium
• Tests are written and can be reviewed and commented upon at:
  • Allow more characters in element/attribute names and prefixes web-platform-tests/wpt#38503
  • https://chromium-review.googlesource.com/c/chromium/src/+/6570951
• Implementation bugs are filed:
  • Chrome: https://bugs.chromium.org/p/chromium/issues/detail?id=1334640
  • Firefox: https://bugzilla.mozilla.org/show_bug.cgi?id=1773312
  • Safari: https://bugs.webkit.org/show_bug.cgi?id=241419

(See WHATWG Working Mode: Changes for more details.)

---

Original points for discussion, discussed and concluded on in following comments. These do not reflect the currently proposed change.

• I did not disallow = inside attribute local names. Both the parser and DOM APIs currently disallow them, except the parser allows it for the first character. I'm happy to change this if people prefer; I started with the simpler version.
• This does not disallow lone surrogates, the Unicode replacement character U+FFFD, single quotes, or < in any position, because the HTML parser allows introducing those already and it seems nicer to align.
• I did not change validation for createProcessingInstruction() or createDocumentType(). We could try to simplify those too, perhaps after investigating parser behavior. But they didn't seem to be causing any real web developer pain, unlike elements and local names, so I thought it'd be better to just leave them as-is.

---

Preview | Diff

[annevk]

Do we want to call out the = difference in a note? At least a comment would be good I think.

I'm not a big fan of adding "DOM API" to the naming. That makes more sense if this was defined outside of the DOM Standard itself. I think dropping it would still make everything work.

[annevk]

Thanks, this looks good to me, modulo a small oversight. Anyone else that should review this? @mfreed7 perhaps?

[mfreed7]

Looks good!

[domenic]

Discussed equals sign at HTML triage meeting. Conclusion: disallow it in attributes everywhere. (Even though the parser allows it in the first-character position.)

[domenic]

I think this is ready for re-review.

Potential issue: XML's definition of Char seems nonsensical (it excludes various Unicode characters below U+0020). And, its definition of the [^#x00#x09#x0A#0x0Cx0D#x20/>] syntax depends on that definition. Hmm.

[domenic]

Refined to no longer use EBNF.

[annevk]

This does not seem equivalent to the sorta-EBNF from before. In particular if the first code point is from BeyondHTMLParserName the second code point was more limited.

[domenic]

In particular if the first code point is from BeyondHTMLParserName the second code point was more limited.

I'm not sure exactly what you mean. Recall that it's a union of both. The second+ code point is from HTMLParserCompatibleName, which had [^#x00#x09#x0A#0x0Cx0D#x20/>]* for that position. Which is exactly what the current draft says, right?

[annevk]

I don't think the EBNF allows for the second code point to be U+0001 when the first is :, for instance. At least the intent was to prevent that. Does EBNF work completely differently from ABNF in that | doesn't signify OR but instead "union"?

(I didn't see "An equivalent EBNF is the following" initially and I don't think what it states is correct.)

[domenic]

I see, I did not capture that this was a branching scenario depending on the behavior of the first code point. And you addressed what harms names like that might hypothetically cause in #849 (comment) .

I'll revise.

[domenic]

I think that is done. The other way I could write this is by looping over the characters individually, which is what a performant implementation would do (instead of using lots of O(n) "contains" operations). But I think this is relatively clear.

(Edit: well, a performant implementation would be looping over code units, since that's JS's native string format... which feels ickier to spec.)

[annevk]

Thanks, this looks accurate to me.

[domenic]

OK, this (and whatwg/html#7991) is just waiting on someone to write web platform tests. Then we can close a ~5 year old recurring pain point on the web platform!

For fun, these are all the references to this I can find:

• Make element and attribute names behave like the HTML parser #449
• A way for createAttribute() to bypass XML Name validation in HTML documents #769
• Attribute DOM representation and parsing is inconsistent html#4275
• Inconsistency in getAttribute and setAttribute regarding invalid attribute names html#3733
• https://www.google.com/search?q=createElement+OR+setAttribute+%252BInvalidCharacterError

I suspect there are more GitHub issues from earlier, because why would I have posted #449 if not because of some other issue someone filed? But I couldn't find them.

[annevk]

@josepharhar would you be interested in finishing this?

[josepharhar]

Yes, I have started a WPT here: web-platform-tests/wpt#38503

[annevk]

\o/ I suspect that once you implement this and do a try run you'll find a lot of existing WPT tests that can be adjusted. There's probably no need for a new file, but maybe.

[cdumez]

Any progress on this?

[josepharhar]

Not recently, I have unfortunately been focused on other stuff.

[josepharhar]

Here is a new WPT: web-platform-tests/wpt#38503

And here is another patch in progress to update existing WPTs, which doesn't have a WPT PR yet because we stopped generating those until the code review gets approved: https://chromium-review.googlesource.com/c/chromium/src/+/6570951