Support Markdown rendering

[khalidabuhakmeh]

Allow folks to take a markdown file and render it to the console.

https://github.com/charmbracelet/glow

[mcon]

I'm going to have a shot at this, looks like it's not too tricky to use https://github.com/xoofx/markdig (I'll be working here: https://github.com/mcon/spectre.console/tree/markdown - should have something of decent quality shortly)

[mcon]

PR here if anyone's interested: #204

[enginexon]

Is this still interesting topic for community? Was it already implemented?

[patriksvensson]

@enginexon A spike PR was opened at #204, but nothing was completed. It's up for grabs.

[frankhaugen]

I like the general concept, but I think the implementation should be a "Document", that is a full width panel, and have some internals for paging etc., but leaves the content up te "extension" through a DocumentContent -type that is inheritable, or something, at least it should allow one to override the raw content and then when it hits the renderer it is in "markup", so the internet at large could then just introduce their own nuget named something like: Spectre.Console.Document.Markdown, and you will get the community to provide the extension, (Markdown might be internally made, since someone already made some code for that in above mentioned PR).

It feels like a natural addition to have some sort of "document viewer", that have in the case of markdown 1 file per page, but being able to construct a "Pagable Document Viewer" would be awesome, but it needs some actual design an forthought to make it "fit" into the whole of spectre

[FrankRay78]

Should this really be a core part of spectre.console, or would it best be located in some form of contrib project? (like what Cake does, here: https://github.com/cake-contrib)

I often wonder the same thing each time I consider making a spectre.console widge for asciichart.

[frankhaugen]

Conceptually I like it, but "contrib" named nugets are usually a "red flag" for many, (at least two different teams I've worked with have had bad experience with "anything goes" contrib packages), so should probably brand it "Spectre.Console.Extensions" or "Spectre.Console.Community", and then try to motivate the community to use that as a prefix.

Personally, I like granualar nugets, so I would be urked by needing to get asciichart if I just want Markdown, so a singular "contrib" project should regardless aim for granularity, (makes handling bugs easier aswell)

[crwsolutions]

Hi all, time to dust this one off? With all those AI chat responses in markup, I think this feature is more important now then it was the past. I think/ hope I have time to update the PR to the latest version. Is that something desirable?

[FrankRay78]

@crwsolutions I'm interested in this idea. Do you have specific use cases in mind?

[crwsolutions]

@FrankRay78 I am fiddling around with AI. It has a main loop to process chat responses. Like so (I do not render the stream, thats a nice to have for me):

    while (true)
    {
        // Get user prompt and add to chat history
        AnsiConsole.Markup("[green]Your prompt: [/]");
        var userPrompt = Console.ReadLine();

        if (userPrompt == "bye")
        {
            break;
        }

        chatHistory.Add(new ChatMessage(ChatRole.User, userPrompt));

        // Stream the AI response and add to chat history
        var response = "";
        await AnsiConsole
            .Status()
            .Spinner(Spinner.Known.Dots)
            .SpinnerStyle(new Style(foreground: Color.Green))
            .StartAsync(
                "Processing...",
                async ctx =>
                {
                    await foreach (
                        var item in _chatClient.GetStreamingResponseAsync(
                            chatHistory,
                            chatOptions
                        )
                    )
                    {
                        ctx.Status("Receiving...");
                        response += item.Text;
                    }
                }
            );
        MdConsole.Render(response);
        chatHistory.Add(new ChatMessage(ChatRole.Assistant, response));
        Console.WriteLine();
    }
    AnsiConsole.MarkupLine("[red]Bye, Bye! 💖[/]");
}

the main markup in my case is headings, bold, lists and code blocks. The code blocks are C#, Typescript, json and SQL. My current output:

[phil-scott-78]

I gave this a go a little while ago in terms of the AI chat stuff. It gets hard quick because I think the expectation would be that the markdown would be rendered as it is streamed to the app from the LLM. Rendering incomplete markdown and then adjusting it on the fly isn't trivial, things like lists, qutoes and block code blocks all have complicated logic that needs to be accounted for as it streams, adjusting as new content and the markdown AST changes all for content that could be thousands of characters long.

I think the amount of edge cases, moving parts and scope of this would be large enough to warrant being an entirely different project with someone with a need for this being the maintainer. It's gonna be a big chunk of code to maintain with its own set of issues. I think there is some stuff potentially Spectre has internal that might need to be made public to make it happen.

[crwsolutions]

@phil-scott-78 Streaming would be very cool, but a nice to have as far as I am concerned. I am not able to decode what they are saying in this issue where they discuss this matter, but maybe it is helpful for you? xoofx/markdig#858

[phil-scott-78]

I dug up my old code. Probably a good starting point if someone wants to give it a go.

class MarkdownRenderable(string markdown) : Renderable
{
    protected override IEnumerable<Segment> Render(RenderOptions options, int maxWidth)
    {
        var ast = Markdown.Parse(markdown);
        foreach (var block in ast)
        {
            var r = RenderBlock(block).Render(options, maxWidth);
            foreach (var segment in r)
            {
                yield return segment;
            }

            yield return Segment.LineBreak;
        }
    }

    private IRenderable RenderBlock(Block block)
    {
        return block switch
        {
            ParagraphBlock paragraph => CreateTextBlock(paragraph),
            HeadingBlock heading => BuildingHeadingBlock(heading),
            ListBlock list => BuildListItemControl(list),
            QuoteBlock quoteBlock => BuildQuoteControl(quoteBlock),
            FencedCodeBlock codeBlock => BuildCodeControl(codeBlock),
            CodeBlock codeBlock => BuildCodeControl(codeBlock),
            ThematicBreakBlock => new Grid() { Width = 120 }.AddColumn().AddRow(new Rule()),
            _ => new Paragraph()
        };
    }

    private IRenderable BuildingHeadingBlock(HeadingBlock heading)
    {
        var sb = new StringBuilder();
        if (heading.Inline != null)
        {
            AddInlineContent(sb, heading.Inline);
        }

        var content = sb.ToString().EscapeMarkup();

        switch (heading.Level)
        {
            case 1:
                return new FigletText(content);
            case > 4:
                return new Text(content, Color.Yellow);
        }

        var rowChar = heading.Level == 2 ? '=' : '-';

        var headerRow = new Text(content, Color.Yellow);
        var rowRow = new Text(new string(rowChar, content.Length), Color.Yellow);

        return new Rows(headerRow, rowRow);
    }

    private IRenderable CreateTextBlock(LeafBlock textBlock)
    {
        var sb = new StringBuilder();
        if (textBlock.Inline != null)
        {
            AddInlineContent(sb, textBlock.Inline);
        }

        var grid = new Grid { Width = 120, }
            .AddColumn()
            .AddRow(new Markup(sb.ToString()));

        return grid;
    }

    private IRenderable BuildCodeControl(CodeBlock codeBlock)
    {
        var sb = new StringBuilder();
        var lines = codeBlock.Lines.Lines;

        foreach (var line in lines)
        {
            sb.AppendLine(line.ToString());
        }

        var panel = new Panel(sb.ToString().EscapeMarkup().Trim())
        {
            Border = new LeftBorder(),
            BorderStyle = Color.Blue,
            Padding = new Padding(1, 0, 0, 0),
        };

        if (codeBlock is FencedCodeBlock fencedCodeBlock && !string.IsNullOrEmpty(fencedCodeBlock.Info))
        {
            panel.Header = new PanelHeader(fencedCodeBlock.Info.EscapeMarkup());
        }

        return panel;
    }

    private IRenderable BuildQuoteControl(QuoteBlock quoteBlock)
    {
        var sb = new StringBuilder();

        foreach (var block in quoteBlock)
        {
            if (block is ParagraphBlock { Inline: not null } paragraphBlock)
            {
                AddInlineContent(sb, paragraphBlock.Inline);
            }
            else
            {
                var renderable = RenderBlock(block);
                sb.AppendLine(renderable.ToString());
            }
        }

        var panel = new Panel(sb.ToString().TrimEnd())
        {
            Border = new LeftBorder(),
            BorderStyle = new Style(Color.Green),
            Padding = new Padding(1, 1)
        };

        return panel;
    }

    private IRenderable BuildListItemControl(ListBlock listBlock)
    {
        var table = new Table().HideHeaders().Border(TableBorder.None);

        table.AddColumn(new TableColumn("Marker").Width(3));
        table.AddColumn(new TableColumn("Content"));

        var index = 1;

        foreach (var item in listBlock)
        {
            if (item is ListItemBlock listItem)
            {
                string marker = listBlock.IsOrdered ? $"{index}." : "\u25cb";
                index++;

                // Create a list of renderables for all blocks in the list item
                var contentRenderables = new List<IRenderable>();

                foreach (var block in listItem)
                {
                    // Recursively render each block and add it to the list
                    var blockRenderable = RenderBlock(block);
                    contentRenderables.Add(blockRenderable);
                }

                // Use Rows to combine all the renderables
                var contentRows = new Rows(contentRenderables);

                // Add the marker and content to the table
                table.AddRow(new Text(marker, Color.Green), contentRows);
            }
        }

        return table;
    }

    private void AddInlineContent(StringBuilder sb, ContainerInline textBlockInline)
    {
        foreach (var inline in textBlockInline)
        {
            switch (inline)
            {
                case CodeInline code:
                    sb.Append("[fuchsia]`");
                    sb.Append(code.Content.EscapeMarkup());
                    sb.Append("`[/]");
                    break;
                case LiteralInline literal:
                    sb.Append(literal.Content.ToString().EscapeMarkup());
                    break;

                case EmphasisInline emphasis:
                    switch (emphasis.DelimiterCount)
                    {
                        case 1:
                            sb.Append("[yellow]");
                            break;
                        case 2:
                            sb.Append("[blue]");
                            break;
                    }

                    AddInlineContent(sb, emphasis);
                    sb.Append("[/]");
                    break;

                case LineBreakInline:
                    sb.Append(" ");
                    break;

                default:
                    if (inline is ContainerInline containerInline)
                    {
                        AddInlineContent(sb, containerInline);
                    }

                    break;
            }
        }
    }
}

class LeftBorder : BoxBorder
{
    public override string GetPart(BoxBorderPart part)
    {
        if (part is BoxBorderPart.Left)
        {
            return "\u2502";
        }

        return " ";
    }
}

[FrankRay78]

Remind me please - is there an allowable NuGet prefix that community extensions can use? Or is 'Spectre.Console.*' entirely reserved?

There is an open issue about creating a page on the website listing some useful extensions, and clarifying how to go about it.

It seems this is timely to clarify.

[patriksvensson]

I think Spectre.Console.Community is open for all. If not, tell me and I'll file an issue with NuGet.

[crwsolutions]

Hi @FrankRay78 , I've worked a little bit more on the subject and taken the code of @mcon and @phil-scott-78 and split it. I've reused the json superpowers library and written an xml and csharp myself to markup these code blocks. It is still POC - ish. But you can find it here: https://github.com/crwsolutions/Spectre.Console.Extensions.Markup

It contains 4 projects:

• Spectre.Console.Xml, markup for xml, works exactly as the json one. No dependencies.
• Spectre.Console.CSharp, markup for c#, works also as the json one. Depends on Microsoft.CodeAnalysis.*
• Spectre.Console.Extensions.Markup, parses the markup. Contains the code of @mcon and @phil-scott-78
• Spectre.Console.Extensions.Markup.Example. Shows the markup, uses the example of @mcon

Is the Spectre.Console team interested in taking over this code or parts of it? I am not attached to it at all and would be happy to contribute it (and change things if you like).

[crwsolutions]

I also added a "superpowers-" project for javascript and sql:

javascript:

sql:

[crwsolutions]

I have been working on the subject for the past few days. I can imagine you don't want to have all these code renderers in your code base, because the list is endless and keeps moving. What you might consider is to manage only the markup, json and xml renderers and outsource the rest to the community.

The main part with the minimal implementation is this then:

public sealed class MarkdownRenderable(string markdown) : Renderable
{
    public readonly Dictionary<string, Func<string, JustInTimeRenderable>> CodeblockRenderables =
        new(StringComparer.OrdinalIgnoreCase)
        {
            { "json", code => new JsonText(code) },
            { "xml", code => new XmlText(code) },
        };

Users can then add their own renderers to the list.

I’m curious to hear what you think of that idea.

[SadE54]

It seems nice ! Does it support table rendering ?

[crwsolutions]

@SadE54 They were ignored, but I added it this morning:

[SadE54]

nice, Thanks ! 👌