Insufficient indentation for nested lists

[pcraig3]

Hello! I've been using the markdownify library in my app for a long time and it's been working great, but recently I updated and I noticed that the the interpretation for nested lists had changed and it messed up my workflow.

My workflow looks like this:

• import content as HTML
• converting the HTML to markdown for editing (using markdownify)
• editors can edit the content
• once editing is finished, convert markdown back to HTML for publishing (using Markdown)

I noticed that since this commit, markdownify changed from using tabs for list indentation (/t) to using spaces, which looks like 2 spaces for unordered lists and 3 spaces for ordered lists.

Since Markdown expects nested lists to be indented by 4 spaces, the nested lists output by markdownify aren't currently interpreted by Markdown.

Here's an example of what I mean:

Nested unordered list

original HTML	markdownify output	final HTML (using Markdown)
original list has 4 levels of indentation	only 2 spaces used for nested list indentation	final list has 2 levels of indentation

Nested ordered list

original HTML	markdownify output	final HTML (using Markdown)
original list has 4 levels of indentation	only 3 spaces used for nested list indentation	final list has 3 levels of indentation

I recently created a PR in my app to resolve this by overriding convert_li. If you are interested, I could do a PR on this repo as well.

On the other hand, if there's some configuration option that I am missing that resolves this or Markdown is too-strictly interpreting the spec and you don't believe this is a real issue, that's fine too.

[chrispy-snps]

@pcraig3 - the updated list rendering follows CommonMark guidelines for indentation, as described here:

CommonMark Spec > 5.2 List items

The relevant text is:

The most important thing to notice is that the position of the text after the list marker determines how much indentation is needed in subsequent blocks in the list item. If the list marker takes up two spaces of indentation, and there are three spaces between the list marker and the next character other than a space or tab, then blocks must be indented five spaces in order to fall under the list item.

Are you open to filing an issue for MarkDown and using your workaround branch in the meantime? My project might also need to use MarkDown soon, and I would also like to see their list indenting be supported more comprehensively.

[chrispy-snps]

(@pcraig3 - by the way, I appreciate the cleanness and cleverness with which you illustrated the indenting examples with images organized within a table.)

[pcraig3]

Hi @chrispy-snps, I spent some time digging through the issues in the Markdown library and the maintainer is pretty explicit that they are not changing this. They consider nested lists to require 4 spaces, full stop.

Here're a couple of relevant issues:

• Nested List and Inline markdown issues Python-Markdown/markdown#1500
• Not handling nested lists Python-Markdown/markdown#1378

The "4 spaces rule" is in their documentation (see "Indentation/Tab length"):

The syntax rules clearly state that when a list item consists of multiple paragraphs, “each subsequent paragraph in a list item must be indented by either 4 spaces or one tab” (emphasis added). However, many implementations do not enforce this rule and allow less than 4 spaces of indentation. The implementers of Python-Markdown consider it a bug to not enforce this rule.

Furthermore, the maintainers are explicit on this point:

It should be noted that Python Markdown is not meant to parse every Markdown convention out in the wild. Python Markdown is also an old-school Markdown parser, not a CommonMark parser.

One solution they suggest is to adjust the tab_length var to be less than 4.

Given all that, it doesn't seem like there is any point of raising this issue in the Markdown library, as they have addressed it.

Where that leaves us is that markdownify and Markdown are following different rules about what nested lists should look like.

People who are using both libs (like me) have to decide which one to patch if nested lists are relevant for your use case.

What should we do with this issue? Close it? It doesn't seem like markdownify is going to change this behaviour either.

[chrispy-snps]

@pcraig3 - I completely understand the frustration of your situation.

Thinking out loud - if markdownify implemented an indent-width configuration option (or something like that), does Markdown process them correctly?

1.  Item 1
2.  Item 2
    -    Subitem 1
         A subitem 1 paragraph.
    An item 2 paragraph.
3.  Item 3

While we're talking about indenting, does Markdown have any indenting requirements for block quoting?

[chrispy-snps]

@pcraig3 - in your local installation of markdownify, could you try the following code edit in markdownify/__init__.py and see if Markdown processes the resulting list items correctly?

    def convert_li(self, el, text, parent_tags):
        # ...omitted...
        bullet_width = min(len(bullet), 4)
                       ^^^^^^^^^^^^^^^^^^^
        # ...omitted...

If it does, I'll submit a pull request to implement this as an option.

I added some thoughts on Markdown's indentation requirements here.