-
-
Notifications
You must be signed in to change notification settings - Fork 1.4k
fix(coverage): exclude injected functions without mapping #7192
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
hi-ogawa
wants to merge
11
commits into
vitest-dev:main
from
hi-ogawa:fix-coverage-injected-function
Closed
Changes from 5 commits
Commits
Show all changes
11 commits
Select commit
Hold shift + click to select a range
6db00c2
fix(coverage): exclude injected functions without mapping
hi-ogawa 6889649
refactor: minor
hi-ogawa ed02a0a
refactor: offsetToPosition
hi-ogawa bb6f209
chore: type
hi-ogawa 2eea05c
test: tweak
hi-ogawa f5c0d9d
test: update istanbul
hi-ogawa b829b5a
test: update
hi-ogawa 4f63f61
Merge branch 'main' into fix-coverage-injected-function
hi-ogawa 9f1bc80
wip: copy repro
hi-ogawa cf257b1
wip: repro with 18983
hi-ogawa 47d0ece
fix: remove VITE_EXPORTS_LINE_PATTERN
hi-ogawa File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,6 +1,7 @@ | ||
export { | ||
lineSplitRE, | ||
offsetToLineNumber, | ||
offsetToPosition, | ||
positionToOffset, | ||
} from '@vitest/utils' | ||
export { | ||
|
21 changes: 21 additions & 0 deletions
21
test/coverage-test/fixtures/configs/vitest.config.injected-functions.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
import MagicString from 'magic-string' | ||
import { defineConfig } from 'vitest/config' | ||
|
||
export default defineConfig({ | ||
plugins: [ | ||
{ | ||
name: 'repro', | ||
transform(code, id, _options) { | ||
if (id.endsWith('injected-functions.ts')) { | ||
const output = new MagicString(code) | ||
output.prepend(`;function prepended(){};`) | ||
output.append(`;function appended(){};`) | ||
return { | ||
code: output.toString(), | ||
map: output.generateMap({ hires: 'boundary' }), | ||
} | ||
} | ||
}, | ||
}, | ||
], | ||
}) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
function original() { | ||
console.log("hello") | ||
} | ||
original() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
import { test } from 'vitest' | ||
import "../src/injected-functions"; | ||
|
||
test('basic', () => {}) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,81 @@ | ||
import { expect } from 'vitest' | ||
import { isV8Provider, readCoverageMap, runVitest, test } from '../utils' | ||
|
||
test('filter out functions injected by plugin', async () => { | ||
const { stderr } = await runVitest({ | ||
include: ['fixtures/test/injected-functions.test.ts'], | ||
coverage: { | ||
reporter: ['json', 'html'], | ||
include: ['fixtures/src/injected-functions.ts'], | ||
}, | ||
config: 'fixtures/configs/vitest.config.injected-functions.ts', | ||
}) | ||
expect(stderr).toBe('') | ||
|
||
const coverageMap = await readCoverageMap() | ||
const fileCoverage = coverageMap.fileCoverageFor('<process-cwd>/fixtures/src/injected-functions.ts') | ||
if (isV8Provider()) { | ||
expect(fileCoverage.fnMap).toMatchInlineSnapshot(` | ||
{ | ||
"0": { | ||
"decl": { | ||
"end": { | ||
"column": 1, | ||
"line": 3, | ||
}, | ||
"start": { | ||
"column": 0, | ||
"line": 1, | ||
}, | ||
}, | ||
"line": 1, | ||
"loc": { | ||
"end": { | ||
"column": 1, | ||
"line": 3, | ||
}, | ||
"start": { | ||
"column": 0, | ||
"line": 1, | ||
}, | ||
}, | ||
"name": "original", | ||
}, | ||
} | ||
`) | ||
} | ||
else { | ||
expect(fileCoverage.fnMap).toMatchInlineSnapshot(` | ||
{ | ||
"0": { | ||
"decl": { | ||
"end": { | ||
"column": 20, | ||
"line": 1, | ||
}, | ||
"start": { | ||
"column": 9, | ||
"line": 1, | ||
}, | ||
}, | ||
"loc": { | ||
"end": { | ||
"column": null, | ||
"line": 3, | ||
}, | ||
"start": { | ||
"column": 20, | ||
"line": 1, | ||
}, | ||
}, | ||
"name": "original", | ||
}, | ||
} | ||
`) | ||
} | ||
expect(fileCoverage.f).toMatchInlineSnapshot(` | ||
{ | ||
"0": 1, | ||
} | ||
`) | ||
}) |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All code that is not present on source maps should already be excluded from final coverage report. Bundlers tend to inject plenty of helpers, generators and polyfills that are not present on source maps - these should be automatically be excluded.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Was it the case even before having this patch #5457? I agree that should be the expected behavior of v8-to-istanbul in any cases, but there's still an edge case even after the patch when the injected code's line overlaps with original code as in the reproduction #7130 (comment):
Also it turned out this is also the case for istanbul provider too as seen in the new snapshot
test/coverage-test/test/injected-functions.test.ts
, which is not fixed in this PR and haven't investigated yet.This looks to me a bug, which is fixable either Vitest side or v8-to-istanbul eventually. I'm wondering how you see this issue in general. We want to land ssr transform fix vitejs/vite#18983, which can cause this edge case more often, so I would like to know your opinions on whether it's fine to land or wait until we get a better understanding of the issue.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I did some manual testing with this and see now why this work-around is required. To me this looks like a bug in
v8-to-istanbul
. Ideally we should file a bug report there with minimal repro and PR later on. Or even better, not usev8-to-istanbul
at all and roll our own V8 coverage processing.The #5457 is only about line coverage and does not affect function coverage at all. The
v8-to-istanbul
has some special handing for line coverage that is not done for function coverage.I see that
VITE_EXPORTS_LINE_PATTERN
is now conflicting when using Vite version from the PR. Can we remove that and be backwards compatible?Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Right, I think it's worth reporting on istanbul side and it would also help isolating the issue properly.
istanbul-lib-source-maps
probably has a similar issue with functions overlapped in a single line (one with source map and other without), so I'll try the repro for that too.What do you mean by conflicting? In current Vitest,
VITE_EXPORTS_LINE_PATTERN
seems to have no effect on coverage as I tested in #7132. Also the issue #7130, which is fixed by this PR, happens on user land plugin doing a similar injection and not only caused by new SSR transform.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When using
"vite": "https://pkg.pr.new/vite@18983"
with this PR, I get:And when removing the
VITE_EXPORTS_LINE_PATTERN
in the same setup, I get correct results:There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, that's not what I expected and I haven't actually tested it 😅 I need to debug this thing again. Thanks for checking 👍
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, okay, I remembered this one. Currently it's dropping an entire
Object.defineProperty(__vite_ssr_exports__
line which includes originalfunction sum
, so this PR alone doesn't fix vitejs/vite#18983.I'm not sure what's the good way to proceed. From what I understand,
VITE_EXPORTS_LINE_PATTERN
is not necessary, so we can apply #7132 anytime. What kind of compatibility check needed there?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Before this PR it used to be necessary. See here how end of line 4 is included in sourcemaps: https://evanw.github.io/source-map-visualization/#NTY5AGZ1bmN0aW.... That's from
main
branch. When I remove theVITE_EXPORTS_LINE_PATTERN
inmain
, the function count incorrectly raises to 2.So for
vite@18983
we need to removeVITE_EXPORTS_LINE_PATTERN
like shown in #7192 (comment).And once
VITE_EXPORTS_LINE_PATTERN
is removed, we need to make sure older Vite versions like 5 still keep working as expected.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, I just noticed #7132 first commit passed CI, then 2nd one failed. Sorry, I didn't even look at that. That's another thing unexpected.