JOSS MS comments

[shazanfar]

Related to openjournals/joss-reviews#2440

Thanks for writing this software, I think the starfish & its associated documentation represents a huge amount of work and the authors as well as all involved should be commended for their efforts!

MS:

Since other data modalities are also measured (e.g. protein), it's worth using more general wording than "gene expression", perhaps "abundance" is more relevant.

It's difficult to assess from the JOSS MS and the software documentation who exactly is the target audience. While indeed anyone with Python + some basic knowledge + starfish installed can run the Quick Start, I think the learning curve becomes quite steep once you start going through other examples and tutorials. It seems to me that either starfish is more suited to practitioners with at least some image analysis experience (or knowledge of the concepts behind), or starfish requires some more automated quality metrics/troubleshooting to be available & described.

Documentation/MS:

It's quite nice how various technologies' data formats are listed and examples shown + used. This being said, surely a number of technologies input formats are still under active development, and likely to change in future. How do you ensure that various file format updates for technologies are dealt with? Is it within scope of starfish developers to provide scripts for different versions of input files? I think answers to these should be mentioned in the Tutorials and Examples area.

I don't think seaborn is listed as a dependency, but is called at the end of the Quick start tutorial, it's worth either including as a dependency or mentioning it's needed at the beginning of the Quick start tutorial.

It's unclear from the User Guide how one could extract all relevant information into an external file format, to be used in other environments (e.g. R/Bioconductor). This is actually given in "Working with Starfish outputs" under "Help & Reference", but IMO could appear in the Tutorials and Examples area for greater visibility.

I can't see clear examples in the User Guide + Tutorials showcasing the scalability of starfish, I'd suggest adding a tutorial/how-to for starfish in such a context. Again, it's worth just making the "Help & Reference" section more visible, esp the 'Processing with AWS' section.

Cheers,
Shila

[neuromusic]

Thanks for writing this software, I think the starfish & its associated documentation represents a huge amount of work and the authors as well as all involved should be commended for their efforts!

Thank you! It's been a big effort from multiple folks!!

Since other data modalities are also measured (e.g. protein), it's worth using more general wording than "gene expression", perhaps "abundance" is more relevant.

starfish is designed to localize and decode gene expression from mRNA spots and does not currently support quantification of protein abundance.

It's difficult to assess from the JOSS MS and the software documentation who exactly is the target audience.

This is a good point. I've added clarifications in the manuscript and user guide to clarify expectations of our user's knowledge.

How do you ensure that various file format updates for technologies are dealt with? Is it within scope of starfish developers to provide scripts for different versions of input files? I think answers to these should be mentioned in the Tutorials and Examples area.

Most of the file formats produced by the various SpaceTx labs are not standard, but are ad hoc implementations. We do not commit to supporting these file formats and the formatting examples are only provided as examples of how to convert data to the spacetx format, using starfish's TileFetcher to perform the conversion. I've added notes to the Tutorials and Examples to help clarify.

mentioning [seaborn is] needed at the beginning of the Quick start tutorial

nice catch. done.

This is actually given in "Working with Starfish outputs" under "Help & Reference", but IMO could appear in the Tutorials and Examples area for greater visibility.

it's worth just making the "Help & Reference" section more visible, esp the 'Processing with AWS' section.

Hmm... yeah, part of this is purely a pragmatic limitation of sphinx: Everything in Examples and Tutorials is a python script which can be downloaded and run by the use, while the "output" examples and AWS tutorial are non-executable. Nonetheless, I've moved these resources into the user guide and added sections highlighting these resources.

Thank you @shazanfar for your detailed review!!