[flake8-comprehensions]: Handle trailing comma in C403 fix

[ayushbaweja]

Summary

Resolves #16099 based on #15929

Test Plan

Added test case s = set([x for x in range(3)],) and updated snapshot.

[github-actions]

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

✅ ecosystem check detected no linter changes.

[dhruvmanila]

I think we should use checker.tokens().in_range(...) here as we already have access to the parsed tokens. I think the range would be TextRange::new(argument.end(), call.end()). You can refer to other references of in_range method to check how it's being used.

[dylwil3]

Good call @dhruvmanila ! Do you (@ayushbaweja) want to also make the same changes to unnecessary_generator_list.rs and unnecessary_generator_set.rs? Here's the patch for the first of those, for example:

diff --git a/crates/ruff_linter/src/rules/flake8_comprehensions/rules/unnecessary_generator_list.rs b/crates/ruff_linter/src/rules/flake8_comprehensions/rules/unnecessary_generator_list.rs
index cc389e1f4..2e3b7fbc9 100644
--- a/crates/ruff_linter/src/rules/flake8_comprehensions/rules/unnecessary_generator_list.rs
+++ b/crates/ruff_linter/src/rules/flake8_comprehensions/rules/unnecessary_generator_list.rs
@@ -4,7 +4,7 @@ use ruff_python_ast as ast;
 use ruff_python_ast::comparable::ComparableExpr;
 use ruff_python_ast::parenthesize::parenthesized_range;
 use ruff_python_ast::ExprGenerator;
-use ruff_python_trivia::{SimpleTokenKind, SimpleTokenizer};
+use ruff_python_parser::TokenKind;
 use ruff_text_size::{Ranged, TextRange, TextSize};
 
 use crate::checkers::ast::Checker;
@@ -125,10 +125,12 @@ pub(crate) fn unnecessary_generator_list(checker: &Checker, call: &ast::ExprCall
 
         // Replace `)` with `]`.
         // Place `]` at argument's end or at trailing comma if present
-        let mut tokenizer =
-            SimpleTokenizer::new(checker.source(), TextRange::new(argument.end(), call.end()));
-        let right_bracket_loc = tokenizer
-            .find(|token| token.kind == SimpleTokenKind::Comma)
+        let after_arg_tokens = checker
+            .tokens()
+            .in_range(TextRange::new(argument.end(), call.end()));
+        let right_bracket_loc = after_arg_tokens
+            .iter()
+            .find(|token| token.kind() == TokenKind::Comma)
             .map_or(call.arguments.end(), |comma| comma.end())
             - TextSize::from(1);
         let call_end = Edit::replacement("]".to_string(), right_bracket_loc, call.end());

[ayushbaweja]

Sure! Will make the change and also update the unnecessary_generator files

[dylwil3]

Thanks so much, this is great!