Skip to content

Allow markdown or custom formatting for code blocks #1493

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jun 30, 2025

Conversation

aymeric-roucher
Copy link
Collaborator

@aymeric-roucher aymeric-roucher commented Jun 27, 2025

This follows up on #1442 and supersedes and closes #1491:

  • In Change code tags to xml #1442, I changed code block fencing blocks from markdown to XML to align with general usage of XML tags in chat templates
  • While this does not negatively impact most models, it still degraded performance for Gemma-3-12B-IT, and maybe other models, we hypothese that this is due to presence of markdown-formatted blocks in training corpora. Thank you @peabody124 for raising the issue!
  • This PR introduces the possibility to customize code block tags
    • It also adds a check for the presence of markdown blocks as a fallback in any case.
    • It also adapts the system prompt to whichever code block format is used. In particular, in the system prompts, it wraps the tool descriptions within these code tags, so as to further nudge the model towards using the proper tags.
      cc @albertvillanova

@aymeric-roucher aymeric-roucher changed the title Allow markdown formatting for code blocks Allow markdown or custom formatting for code blocks Jun 27, 2025
Copy link
Member

@albertvillanova albertvillanova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Good fix! Also, it was nice to remove the remaining <end_code> tags...

Comment on lines 177 to 178
initial_pattern = rf"{code_block_tags[0]}(.*?){code_block_tags[1]}"
matches = re.findall(initial_pattern, text, re.DOTALL)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why you renamed it to initial_pattern instead of original pattern?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a residue of implementation change, reverting this

aymeric-roucher and others added 2 commits June 27, 2025 17:36
@peabody124
Copy link

I can confirm using this branch with code_block_tags="markdown" reproduces the benefits of #1491 . Closing my PR.

@aymeric-roucher aymeric-roucher merged commit 31f7a6d into main Jun 30, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants