Fix outputExtension option in unplugin, enabling .svelte.ts/.svelte.js for Svelte Runes reactive compatibility

[adam2am]

Summary

• Expand handling of .civet files to support output extensions like .svelte.ts or .svelte.js.
• Update isCivetTranspiled regex to match .civet.svelte.ts and .civet.svelte.js.
• Update cleanCivetId() to correctly strip new suffixes (.svelte.ts, .svelte.js).
• This enables .civet files to output TypeScript-compatible .svelte files, which are needed for Svelte's new Runes API.
• No breaking changes - .civet.jsx / .civet.tsx are still supported as before.

Motivation

• Svelte's new Runes system.
• Civet users can now fully integrate with workflows by outputting .svelte.ts from .civet files.
  (even tho for now IDE will not recognize syntax of it out of the box, have to manually do .d.ts)

Thanks for reading! 🚀 Happy to adjust anything if needed

[edemaine]

Thanks for this PR, and sorry for the delay. Do you think it would be possible to use outputExtension instead of hard-coding Svelte support? That would be better, I think, but it may require a bit more restructuring.

[adam2am]

Thanks for this PR, and sorry for the delay. Do you think it would be possible to use outputExtension instead of hard-coding Svelte support? That would be better, I think, but it may require a bit more restructuring.

You mean something like this?

  outExt :=
    options.outputExtension ?? (ts is "preserve" ? ".tsx" : ".jsx")
  cleanOutExt := outExt.replace(/\./g, '\\.')
  cleanOutputExtRE := new RegExp(`${cleanOutExt}$`)
  isCivetTranspiled := new RegExp(`(\\.civet)(?:${cleanOutExt})?([?#].*)?$`)

  function cleanCivetId(id: string): {id: string, postfix: string}
    let postfix = ''
    id = id
    .replace postfixRE, (match) =>
      postfix = match
      ''
    .replace cleanOutputExtRE, ''
    {id, postfix}

Also while experimenting I noticed it's adding it to the end, so it's file.civet.ts / file.civet.svelte.ts without stripping
Do you think would it be a cool idea to optionally add something like

   stripCivet  := options.stripCivetExtension ?? false

and some extra logic to make it possible > file.ts / file.svelte.ts / file.tsx or there is no point for doing that?

[edemaine]

If we stick to regular expressions, you'll need to escape all active regexp characters, not just .. (Can't wait for RegExp.escape...) Something like

cleanOutExt := outExt.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')

But I wonder if we could instead just remove do the postfixRE removal, then detect whether the filename endsWIth(outExt), strip that, and then check for /\.civet$/, instead of building a big regular expression.

Do you think would it be a cool idea to optionally add something like

   stripCivet  := options.stripCivetExtension ?? false

I don't think this can work in the unplugin, because we wouldn't be able to detect transformed files, because they no longer have .civet in them. But I'm also not quite sure why it really matters: I don't think the unplugin ever emits a file with the output extension in it, does it? (The CLI already supports writing .ts files instead of .civet.ts files.) I think the unplugin's filenames are all for internal processing purposes. The point of this PR is to allow postprocessing triggered by extensions other than .tsx, including .svelte.ts.

[adam2am]

instead of building a big regular expression.

Hey, sorry for the delay, could you review my code in latest commit is it overcoooked or a nice one

Basically
1 - helper functions instead of regex for validation
2 - splitIdPostfix - only removes ? or # part
3 - validateAndExtractCivetBase -

    - removes postfix
    - checks if id ends with outputextension
    - checks if id before ends with .civet
        true > returns base .civet path, 
        otherwise null

4 - resolveId now uses splitIdPostfix to only remove postfix = path without striping outext, then it appends everything
5 - loadinclude = direct, clear validation (is not null)
6 - load - uses validateAndExtractCivetBase to both validate and extract civet file

[edemaine]

I really like the new approach! This is exactly what I was thinking.

I posted a few comments, the most important of which is about handling missing outputExtension. The rest are mostly about Civet code style.

Once implemented, I'll do some thorough testing, and then we can finally merge. Sorry for the long delay!

[edemaine]

This looks wrong (too many \s) but also no longer needed; remove.

[adam2am]

Ok, then I replaced further

if match := filePath.match civetExtension
  filePath = filePath[0...-match[0]#]

with

if filePath.endsWith civetSuffix
  filePath = filePath[0...-civetSuffix#]

[edemaine]

This comment should be removed or moved, given that you removed the regular expression.

But I'm worried that a missing .jsx/.tsx (what this comment is about) won't be handled correctly anymore. Before the (\.[jt]sx)? did the work. I've added a comment below where I think it needs to be fixed.

[adam2am]

I just deleted this comment.
.jsx/.tsx extensions still present if you mention it. When I change .svelte.ts to .tsx

civetVitePlugin({
	outputExtension: '.tsx',

it would throw the error:

1:18:48 PM [vite] (client) ✨ optimized dependencies changed. reloading
Svelte error: rune_outside_svelte
The $state rune is only available inside .svelte and .svelte.js/ts files
https://svelte.dev/e/rune_outside_svelte
at <instance_members_initializer> (/home/user/Documents/repos/sv-civet-proof-of-concept/src/routes/class.civet.tsx:20:10)

[edemaine]

Suggested change

	unless cleanedId.endsWith outputExtension return null
	return unless cleanedId.endsWith outputExtension

Actually, to handle missing added outputExtension, I think this should be more like

Suggested change

	unless cleanedId.endsWith outputExtension return null
	if cleanedId.endsWith outputExtension

(and inside the if block, maybe update cleanedId like the next line)

We'll still return undefined if there isn't then a .civet extension.

[adam2am]

I went with this

function extractCivetFilename(id: string, outputExtension: string): string?
  cleanedId := normalizePath(id.replace postfixRE, '')
  lCleanedId := cleanedId.toLowerCase()
  lOutputExtension := outputExtension.toLowerCase()
  if lCleanedId.endsWith lOutputExtension
    base := cleanedId[<-outputExtension.length]
    lBase := base.toLowerCase()
    return base if lBase.endsWith civetSuffix

as extra covering edge-cases with paths and uppercase

[edemaine]

Thanks for the edits! I did some more thorough testing, and found and fixed some issues:

1. extractCivetFilename still isn't correct: if the cleaned id doesn't end with outputExtension, we still want to check for .civet extension, as explained in the comment you deleted.
2. We also need to remove the outputExtension in resolveId (at least in Vite, because of HTML <script> tag rewriting), but your changes removed this stripping. So I think extractCivetFilename and splitIdPostfix should go back to being unified.
3. I don't see why we should lower-case when testing for outputExtension, because it's always something we're adding manually, so case should not have changed. Arguably we should allow for .CiVeT, but we don't currently, so let's leave that for another time.

I fixed these and did general cleanup/rewrite in https://github.com/DanielXMoore/Civet/tree/unplugin-extension branch. Unfortunately, because you PR'd off your main branch, I can't push to it. (Please PR off a non-main branch in the future.) Can you pull from https://github.com/DanielXMoore/Civet/tree/unplugin-extension ?

With my commit, the integration/unplugin-examples seem to work, as does a real-world Vite example.

[adam2am]

3. lower-case when testing for outputExtension

Was just thinking about a few weird edge cases, same thing with normalizePath. But yeah, overall it's looking solid now, your code’s way cleaner to read too, holy amazing. Totally forgot to test it with integrations before

[edemaine]

Thanks! Are you able to pull my changes into your branch? If not, we can make a new PR on my branch.

[adam2am]

Thanks! Are you able to pull my changes into your branch? If not, we can make a new PR on my branch.

My bad, got distracted before. Now your code is merged into the branch

[edemaine]

Thanks!