Weird syntax highlighting in Rust

[A-Walrus]

Summary

Looking at some code from rust-analyzer:config.rs.
Results in some strange syntax highlighting within the config_data! macro:

Not sure if it counts as a bug since the code inside the macro is not actually valid rust syntax, but thought I'd file this anyway just in case 🤷‍♂️.

For reference this is what it looks like in vscode:

Also wrong, but in a different way 😆

Reproduction Steps

I tried this:

1. hx config.rs

I expected this to happen:
Text inside quotes is colored as such.
Instead, this happened:
Some text is colored incorrectly

Helix log

No response

Platform

Linux

Terminal Emulator

kgx (Gnome Console)

Helix Version

helix 23.03

[pascalkuthe]

Very strange. You don't show it in your screenshot but the highlighting just stops working after a certain point altogether. This looks a lot like #6509 but I tried increasing that limit and event entirely removing it from the codebase and it doesn't change anything. Some parts of this are due to a bug in the rust grammar I suspect but some things here a super weird.

for example for "0" the 0 gets highlighted as 0 event tough even as a lexical token it's a string literal. However tree-sitter-scopes and tree-sitter-subtree actually say it's a string_literal. There are some suspicious token_tree captures showing up tough so maybe they are taking precedence somehow? Another weird thing is that the " get their own token trees. I think they probably need to specially handle string literals inside the scanner to avoid that?

Trying it out a similar issue seems to occur in nvim s oit's probably an upstream issue of some sort

[CalebLarsen]

@A-Walrus Sorry for the ping after a long time, I think this issue has been fixed. Can you confirm?

[A-Walrus]

I can still reproduce with helix 25.01.1 (e7ac2fcd)

This is the file if anyone wants to reproduce https://github.com/rust-lang/rust-analyzer/blob/2400b36a2ed40f68a26473f69ac208ba10d98af9/crates/rust-analyzer/src/config.rs

[CalebLarsen]

Ok, here is what I have learned.

The issue is caused by creating a tree-sitter injection when we invoke a macro.
If we disable it, we get the correct highlighting in this instance, but it breaks on other instances such as nested macros (ie vec![vec![]]).

I did a sanity test to see if other tree-sitter clients had this problem, but to my surprise, both neovim and tree-sitter highlight highlighted both correctly.

I spent some time yesterday looking into how this is happening, but I didn't make any progress on it.
However, I did want to acknowledge that the issue persists and that it is solvable.

Reproduction case:

a! { struct A {
    b: bool = "false",
}}