Implementing code coverage #208

suhaani-agarwal · 2025-06-19T10:43:12Z

Implementing code coverage for animint2 using codecov. (using covr for R side and Chrome devtools coverage + v8-to-istanbul for JS side)

Closes #122

suhaani-agarwal · 2025-06-25T13:05:20Z

This PR adds R and JavaScript code coverage tracking:

For compiler tests we use the covr package to track which R code lines are executed during testing. The coverage data is automatically uploaded to Codecov after successful test runs via covr::codecov().
For renderer tests, we leverage Chrome DevTools Protocol through Chromote to capture precise JS coverage data as the animint.js code executes in the browser. The raw coverage data is converted to Istanbul/lcov format using puppeteer-to-istanbul and uploaded to Codecov

Once request for installation of Codecov is approved for animint, and the CODECOV_TOKEN is added to GitHub Secrets, the coverage tracking will be uploaded to Codecov . we can further refine the coverage reporting and verify if this works once codecov token is uploaded.

codecov · 2025-07-01T11:27:20Z

Welcome to Codecov 🎉

Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.

Thanks for integrating Codecov - We've got you covered ☂️

suhaani-agarwal · 2025-07-04T10:55:22Z

Update : the R side code coverage is correctly being uploaded to codecov for every commit. see here : https://app.codecov.io/gh/animint/animint2/tree/coverage/R
Here we can also look at the R code being covered by the tests and the parts of R code not being covered by tests. for example : https://app.codecov.io/gh/animint/animint2/blob/coverage/R%2Ffacet-.r

About the JS code coverage, I have managed to collect the JS code coverage from chrome devtools while the renderer tests run. This coverage is currently being stored in a raw json format. to upload it to codecov, we require it to be in a different (istanbul) format. For this, I am currently facing difficulty downloading the node package pupeteer-to-istanbul (responsible for converting raw json js coverage data to codecov accepted format) correctly in Github Actions workflow. As soon as I am able to convert it correctly, The js coverage will also be uploaded to codecov along with the R coverage.

suhaani-agarwal · 2025-07-08T19:26:45Z

code coverage is now being uploaded for both JS and R codes to codecov . see here : https://app.codecov.io/github/animint/animint2/commit/1747fde5868452bd9651a8637022bb3b75b84836/tree?dropdown=coverage

tdhock · 2025-07-08T19:32:24Z

this is really excellent work, thanks!
I see files under vendor are used to compute code coverage for this report https://app.codecov.io/github/animint/animint2/commit/1747fde5868452bd9651a8637022bb3b75b84836/tree/inst/htmljs?dropdown=coverage

Can you please tell codecov to ignore the files under vendor/ ?

DESCRIPTION

.gitignore

tdhock · 2025-07-08T19:37:18Z

there is an issue with the CRAN test suite, which currently has a WARNING but is not failing (green build).
Can you please fix the github action so that the current code results in a failed CRAN test suite? (red build because of warning below)

* checking DESCRIPTION meta-information ... NOTE
Package listed in more than one of Depends, Imports, Suggests, Enhances:
  ‘covr’
A package should be listed in only one of these fields.
* checking top-level files ... NOTE
Non-standard files/directories found at top level:
  ‘package-lock.json’ ‘package.json’ ‘v8-to-istanbul.js’
* checking for left-over files ... OK
* checking index information ... OK
* checking package subdirectories ... OK
* checking code files for non-ASCII characters ... OK
* checking R files for syntax errors ... OK
* checking whether the package can be loaded ... OK
* checking whether the package can be loaded with stated dependencies ... OK
* checking whether the package can be unloaded cleanly ... OK
* checking whether the namespace can be loaded with stated dependencies ... OK
* checking whether the namespace can be unloaded cleanly ... OK
* checking loading without being on the library search path ... OK
* checking whether startup messages can be suppressed ... OK
* checking use of S3 registration ... OK
* checking dependencies in R code ... NOTE
Namespace in Imports field not imported from: ‘covr’
  All declared Imports should be used.
* checking S3 generic/method consistency ... OK
* checking replacement functions ... OK
* checking foreign function calls ... OK
* checking R code for possible problems ... [19s/15s] OK
* checking Rd files ... OK
* checking Rd metadata ... OK
* checking Rd line widths ... OK
* checking Rd cross-references ... OK
* checking for missing documentation entries ... OK
* checking for code/documentation mismatches ... OK
* checking Rd \usage sections ... OK
* checking Rd contents ... OK
* checking for unstated dependencies in examples ... OK
* checking contents of ‘data’ directory ... OK
* checking data for non-ASCII characters ... OK
* checking LazyData ... OK
* checking data for ASCII and uncompressed saves ... OK
* checking examples ... [48s/44s] OK
* checking examples with --run-donttest ... [93s/83s] OK
* checking for unstated dependencies in ‘tests’ ... WARNING
Warning: 'library' or 'require' call not declared from: ‘jsonlite’
* checking tests ...
  Running ‘namespace.R’
  Running ‘testthat.R’ [36s/30s]
 [38s/32s] OK

suhaani-agarwal · 2025-07-08T20:12:01Z

there is an issue with the CRAN test suite, which currently has a WARNING but is not failing (green build). Can you please fix the github action so that the current code results in a failed CRAN test suite? (red build because of warning below)

That warning was appearing earlier because of a file I had created for a previous implementation, which is no longer needed. I've removed that file now, and the warning no longer appears.

tdhock · 2025-07-08T20:42:09Z

In CRAN test suite I still see

* checking top-level files ... NOTE
Non-standard file/directory found at top level:
  ‘v8-to-istanbul.js’

the fix is to add this file to Rbuildignore. But before doing that, can you please modify the CI so that the CRAN test suite will fail when there is a WARNING or NOTE?
so then we should get a red/failed build next.
and then you can push the fix after that?

suhaani-agarwal · 2025-07-09T09:28:25Z

In CRAN test suite I still see
* checking top-level files ... NOTE
Non-standard file/directory found at top level:
  ‘v8-to-istanbul.js’
the fix is to add this file to Rbuildignore. But before doing that, can you please modify the CI so that the CRAN test suite will fail when there is a WARNING or NOTE? so then we should get a red/failed build next. and then you can push the fix after that?

thanks for the clarification! I have now modified the CI so that the CRAN test suite will fail when there is a WARNING or NOTE: https://github.com/animint/animint2/actions/runs/16154589438/job/45594000311
and after that I fixed the error by adding the file v8-to-istanbul.js to Rbuildignore.

suhaani-agarwal · 2025-07-09T09:32:02Z

update : currently, one of the test files, test-renderer2-PredictedPeaks is causing chromote timeout issue which isnt getting resolved even after increasing the chromote timeout. So I am trying to figure out why that is happening.

suhaani-agarwal · 2025-07-10T09:41:18Z

The original test-renderer2-PredictedPeaks was a little heavy and was slowing down the browser, especially after we added JavaScript coverage collection through Chrome DevTools. I’ve now refactored the test to make it lighter by:

Subsetting the dataset to a minimal size,
Removing unnecessary visualization components not relevant to the test logic.

With this update, both JavaScript and R code coverage are now correctly collected and uploaded to Codecov:
https://app.codecov.io/github/animint/animint2/commit/9952830992b8e8f216647c4e174665234cd46bee/tree?dropdown=coverage

tests/testthat/test-renderer2-PredictedPeaks.R

tdhock · 2025-07-10T10:49:38Z

tests/testthat/test-renderer2-PredictedPeaks.R

-              showSelected=c("dotID", "chrom"),
-              clickSelects="peak.name",
-              size=11,
-              data=PredictedPeaks$chromCounts)+
-    scale_y_discrete("cell type", drop=FALSE),
-  chroms=ggplot()+
-    theme_bw()+
-    theme_animint(width=1500, height=330)+
-    scale_y_discrete("chromosome", drop=FALSE)+ 
-    scale_x_continuous("position on chromosome (mega bases)")+
-    geom_text(aes(0, chrom, label=paste0(peaks, "_")),
-              clickSelects="chrom",
-              showSelected="dotID",
-              hjust=1,


it is unusual that these were not used at all in the test.
Instead of deleting them, I would think it should be preferable to include them in the test somehow.

It seems that there are several significant changes to this test, and it is not clear which one is responsible for the fix, can you please clarify?

deleting two plots

making smaller data set

modifying timeout

Timeout has not been modified ( I have again set it to 120 as it was before)

making smaller data set is the change responsible for the fix.

I have now restored the 2 deleted plots and also included them in a small test case. Is that okay?

…out)

…plots

suhaani-agarwal · 2025-07-10T18:44:04Z

Apologies for any confusion caused. While updating NEWS.md, I accidentally ran git rebase instead of merging master into coverage, which resulted in some duplicate commits and a rewritten history due to a force push.

I've verified that all intended changes ,including test improvements and coverage additions are intact. The branch is now clean, up-to-date with master, and ready for review.
Please let me know if any further changes are needed

tdhock · 2025-07-10T18:50:54Z

Can you please provide a link to a more recent coverage report which shows R/JS code coverage? (but not vendor files)

Like this: https://app.codecov.io/github/animint/animint2/commit/1747fde5868452bd9651a8637022bb3b75b84836/tree/inst/htmljs?dropdown=coverage

For some reason I can't find a newer report on codecov. What do I need to click on?

suhaani-agarwal · 2025-07-10T19:00:30Z

Can you please provide a link to a more recent coverage report which shows R/JS code coverage? (but not vendor files)

Like this: https://app.codecov.io/github/animint/animint2/commit/1747fde5868452bd9651a8637022bb3b75b84836/tree/inst/htmljs?dropdown=coverage

coverage report for the most recent commit : https://app.codecov.io/github/animint/animint2/commit/1065d7f9c082536e4db9e0e90dc7abd0717883ba/tree

Please click on "inst" -> "htmljs" -> "animint.js" to view the JS code coverage : https://app.codecov.io/github/animint/animint2/commit/1065d7f9c082536e4db9e0e90dc7abd0717883ba/blob/inst/htmljs/animint.js?dropdown=coverage
and on "R" -> Rcode_file.r to view the R code coverage for that file

For some reason I can't find a newer report on codecov. What do I need to click on?

So the link to the homepage for the coverage reports for animint2 is : https://app.codecov.io/github/animint/animint2
here, after selecting the branch "coverage" , you will be able to access the most recent coverage report for different files (after scrolling down) : https://app.codecov.io/github/animint/animint2/tree/coverage

In order to view the coverage reports for different commits, you can click on "commits" in the navbar and then click on the commit you want to view the coverage report for.

Shall I make a video demonstrating the usage of this dashboard?

tdhock · 2025-07-10T19:36:41Z

ok, I had to click File Explorer.
Thanks!
Video would be nice for your blog.
Great work!!

suhaani-agarwal marked this pull request as ready for review July 8, 2025 19:25

tdhock reviewed Jul 8, 2025

View reviewed changes

DESCRIPTION Outdated Show resolved Hide resolved

tdhock reviewed Jul 8, 2025

View reviewed changes

.gitignore Show resolved Hide resolved

tdhock reviewed Jul 10, 2025

View reviewed changes

tests/testthat/test-renderer2-PredictedPeaks.R Outdated Show resolved Hide resolved

tdhock reviewed Jul 10, 2025

View reviewed changes

tests/testthat/test-renderer2-PredictedPeaks.R Outdated Show resolved Hide resolved

tdhock reviewed Jul 10, 2025

View reviewed changes

suhaani-agarwal added 13 commits July 10, 2025 21:42

initial implementation for code coverage

407162c

added covr package

117dc25

checking only js coverage

8004af8

updated tests.yaml

94049e2

updated tests.yaml

30ebdb4

testing only r coverage

b94919c

trying out a simpler approach

d82f2cd

updated tests.yaml

d3dc471

deleted unnecessary file

2805a59

trying out another method

cdf6b9c

updated tests.yaml

3359121

removed syntax error

e8fe132

added coverage

208c16c

suhaani-agarwal added 23 commits July 10, 2025 21:42

replaced npx

b684b6c

using /usr/bin/sudo

9aaf93d

installing node with different approach

731ee38

installing node

4c48e59

updated installing node

6f962e8

manually converting raw coverage to lcov

7ff84ae

updated conversion function for all js files

070d92b

added test to check if the coverage conversion works correctly

0ac7a38

updated conversion function

e4d8ac8

using v8-to-istanbul.js

f15d318

removed unnecessary changes and ignoring files under vendor/

9449b09

added Node.js related files to .Rbuildignore

ca8ebb1

bringing the PredictedPeaks test back (caused problem - chromote time…

4ed642a

…out)

cran check will now fail for WARNING and NOTE

50ef480

updated build.sh to add cran check fail for warnings and notes

9618971

added v8-to-istanbul.js to .Rbuildignore

2e54580

added echo in build.sh and increased chromote timeout

aa40873

made PredictedPeaks test lighter

3bf89cf

updated repo name in README and removed addition of empty spaces

a4333bd

added copy of dataset, improved indentations, brought back 2 deleted …

1e46b2e

…plots

added test case for chroms and oneChrom for 2nd viz

54eeac5

updated NEWS.md

94d0b0e

updated NEWS.md

cba32ec

suhaani-agarwal force-pushed the coverage branch from 2fce574 to cba32ec Compare July 10, 2025 16:36

version++

1065d7f

tdhock merged commit 33f5a80 into master Jul 10, 2025
5 checks passed

Implementing code coverage #208

Implementing code coverage #208

Uh oh!

Conversation

suhaani-agarwal commented Jun 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

suhaani-agarwal commented Jun 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Jul 1, 2025

Welcome to Codecov 🎉

Uh oh!

suhaani-agarwal commented Jul 4, 2025

Uh oh!

suhaani-agarwal commented Jul 8, 2025

Uh oh!

tdhock commented Jul 8, 2025

Uh oh!

Uh oh!

Uh oh!

tdhock commented Jul 8, 2025

Uh oh!

suhaani-agarwal commented Jul 8, 2025

Uh oh!

tdhock commented Jul 8, 2025

Uh oh!

suhaani-agarwal commented Jul 9, 2025

Uh oh!

suhaani-agarwal commented Jul 9, 2025

Uh oh!

suhaani-agarwal commented Jul 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tdhock Jul 10, 2025

Choose a reason for hiding this comment

Uh oh!

suhaani-agarwal Jul 10, 2025

Choose a reason for hiding this comment

Uh oh!

suhaani-agarwal commented Jul 10, 2025

Uh oh!

tdhock commented Jul 10, 2025

Uh oh!

suhaani-agarwal commented Jul 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tdhock commented Jul 10, 2025

Uh oh!

Uh oh!

Uh oh!

suhaani-agarwal commented Jun 19, 2025 •

edited

Loading

suhaani-agarwal commented Jun 25, 2025 •

edited

Loading

suhaani-agarwal commented Jul 10, 2025 •

edited

Loading

suhaani-agarwal commented Jul 10, 2025 •

edited

Loading