[REVIEW]: Lightshow: a Python package for generating computational x-ray absorption spectroscopy input files

[editorialbot]

Submitting author: @matthewcarbone (Matthew Carbone)
Repository: https://github.com/AI-multimodal/Lightshow
Branch with paper.md (empty if default branch): joss-manuscript
Version: v0.1.3
Editor: @ppxasjsm
Reviewers: @maurov, @larsenkg
Archive: 10.5281/zenodo.8118592

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/a9cabcd7f4b85a926a797997c6622b43"><img src="https://joss.theoj.org/papers/a9cabcd7f4b85a926a797997c6622b43/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/a9cabcd7f4b85a926a797997c6622b43/status.svg)](https://joss.theoj.org/papers/a9cabcd7f4b85a926a797997c6622b43)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@maurov, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review.
First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @ppxasjsm know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @maurov

📝 Checklist for @larsenkg

[editorialbot]

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf

[editorialbot]

Software report:

github.com/AlDanial/cloc v 1.88  T=0.09 s (454.9 files/s, 70133.6 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                          18            679           1280           2573
reStructuredText                12            201            224            201
Markdown                         1             15              0            158
YAML                             3             10             15            126
TeX                              1             10              0             97
TOML                             1              6              1             82
Bourne Shell                     1             25              6             80
Jupyter Notebook                 1              0            431             44
DOS Batch                        1              8              1             27
make                             1              4              6             10
JSON                             1              0              0              1
-------------------------------------------------------------------------------
SUM:                            41            958           1964           3399
-------------------------------------------------------------------------------


gitinspector failed to run statistical information for the repository

[editorialbot]

Wordcount for paper.md is 1200

[editorialbot]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[editorialbot]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- None

MISSING DOIs

- 10.1039/b926434e may be a valid DOI for title: Parameter-free calculations of X-ray spectra with FEFF9
- 10.1103/physrevb.83.115106 may be a valid DOI for title: Bethe-Salpeter equation calculations of core excitation spectra
- 10.1088/2516-1075/ab3123 may be a valid DOI for title: Bethe-Salpeter equation for absorption and scattering spectroscopy: implementation in the exciting code

INVALID DOIs

- None

[ppxasjsm]

@matthewcarbone could you please fix the missing DOIs please and generate the pdf again using the editorial bot?

[matthewcarbone]

@editorialbot generate pdf

[editorialbot]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[matthewcarbone]

@ppxasjsm I've added the DOIs explicitly for those manuscripts. Should I add every DOI? Seems like it's imbalanced now. Certain references have DOIs and some don't!

[ppxasjsm]

Yes please add the DOI if you don't mind.

[maurov]

Review checklist for @maurov

Conflict of interest

• I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the https://github.com/AI-multimodal/Lightshow?
• License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• Contribution and authorship: Has the submitting author (@matthewcarbone) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
• Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
• Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
• Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
• State of the field: Do the authors describe how this software compares to other commonly-used packages?
• Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

[maurov]

@matthewcarbone I have started the review and my main concern is that I do not understand why you have decided to create a new software from scratch instead contributing to existing well established open-source Python projects in the field, namely, Pymatgen and ASE?

Since you are closer to Pymatgen, why not simply extending pymatgen.io with OCEAN, EXCITING and Xspectra?

Furthermore, in the statement of need is written that Lightshow is a tool for XAS simulation and analysis. In the current status, I see only a (possibly redundant) tool for generating XAS input files, but I do not see any tool helpful for the analysis of the XAS simulations. As part of the XAS community myself, I would expect much more on the analysis side. For example: a tool for extracting local coordination environment from the input structures or modify the local environment (e.g. angles, coordination, distances); perform convergence tests versus a given structural parameter; handle the generated spectra (e.g. plots and validation/comparison with experimental data).

I think that all this should be clearly stated out in the software paper and in the documentation.

The current documentation is minimalist and does not show a single case of the full workflow: from the input structure down to a XAS spectrum. I think this should go into the quickstart or refer to a more complete notebook (just generating the input file is not enough). In my opinion, the first/minimal thing a generic user would like to do with Lightshow is: 1) load a CIF file, 2) generate an input file for a given calculator, 3) run the calculator, 4) plot the XAS spectrum. Points 3 and 4 are very important in the workflow.

I do think there is a need in the scientific community for such a tool and your research group has the expertise for it, but with the current contribution you are sharing only the tip of the iceberg. So, please, add more detailed notebooks with real cases examples.

I would also describe in the documentation how to extend Lightshow with other calculators or add at least input file generation for FDMNES, which is well known in the XAS community.

These are my general comments. I will add more technical aspects later on or directly link to Github issues.

[matthewcarbone]

@maurov thanks for the initial feedback. I can try to address some of your concerns here, and I'll get in touch with the team to handle the more technical comments regarding e.g. notebooks.

I do not understand why you have decided to create a new software from scratch instead contributing to existing well established open-source Python projects in the field, namely, Pymatgen and ASE?
Since you are closer to Pymatgen, why not simply extending pymatgen.io with OCEAN, EXCITING and Xspectra?

In a perfect world, this is the correct open-source solution, but unfortunately there are fundamental limitations we are bound by at my institution. For one, this work was funded by members of BNL and therefore we are tied down by certain requirements. For example, a full "project," if you will, is required to be licensed in a certain way. Making these contributions upstream to ase or pymatgen was unfortunately just not an option for us. That said, now that our code is published, it is certainly permissible for the maintainers of these packages (or honestly anyone, including us) to copy/paste Lightshow code upstream as long as long as they copy/paste the license with it!

Furthermore, in the statement of need is written that Lightshow is a tool for XAS simulation and analysis. In the current status, I see only a (possibly redundant) tool for generating XAS input files, but I do not see any tool helpful for the analysis of the XAS simulations.

I don't think it's redundant. It is slightly redundant for FEFF but we've not found a tool that does what we want for the other codes (Pymatgen can handle VASP to some extent but it's very confusing, at least to me). For benchmarking specifically we offer some functionality that is uniquely useful (matching unique sites between different codes). That said, analysis tools are absolutely the next step and are definitely going to be a very useful addition for us and others.

As part of the XAS community myself, I would expect much more on the analysis side. For example: a tool for extracting local coordination environment from the input structures or modify the local environment (e.g. angles, coordination, distances); perform convergence tests versus a given structural parameter; handle the generated spectra (e.g. plots and validation/comparison with experimental data).

All great ideas that I agree with. We just felt it was out-of-scope for the "v1" release.

In my opinion, the first/minimal thing a generic user would like to do with Lightshow is: 1) load a CIF file, 2) generate an input file for a given calculator, 3) run the calculator, 4) plot the XAS spectrum. Points 3 and 4 are very important in the workflow.

At its current stage, Lightshow is not a workflow code. The subtleties in developing workflows for e.g. different computer architectures was just not something we wanted to address right now. Even ase often just requires an executable in the PATH, and then just calls that executable from a lightweight Python function. We figured at this stage that part was not really required for what we want to do.

Lightshow's intention at this stage is to write the input files for the user. Running the code is a whole different can of worms that we agreed was not in scope for the first release. That said, I think one thing we can do at least is to make a tutorial of how you'd run the codes using a simple bash script on some HPC, or something like this.

I do think there is a need in the scientific community for such a tool and your research group has the expertise for it, but with the current contribution you are sharing only the tip of the iceberg. So, please, add more detailed notebooks with real cases examples.

Will do!

I would also describe in the documentation how to extend Lightshow with other calculators or add at least input file generation for FDMNES, which is well known in the XAS community.

FDMNES is actually at the top of our list for the next kind of code to add. We definitely are seeing it all over the place and agree it makes sense to add to Lightshow.

[Kevin-Mattheus-Moerman]

@ppxasjsm thanks for your help editing this submission. Can you check in and see where this review is at? Perhaps the reviewers could use a reminder.

[ppxasjsm]

@maurov Just following up on @Kevin-Mattheus-Moerman suggestion and @matthewcarbone's responses to your initial feedback. Do you think you will be able to proceed with the review, or are there too many substantial concerns?

[maurov]

@ppxasjsm @Kevin-Mattheus-Moerman thank you very much indeed for the reminder, but I was waiting some more activity on the repository after @matthewcarbone reply to my comments. In particular, I was expecting more notebooks with real cases examples and a tutorial showing how to use the code in the whole workflow. Once this is done, the paper could be accepted in my opinion. There is a need in the community and Lightshow goes in the right direction. I was expecting more for a first release, but I understand authors' concerns too. Anyway, I will complete the review soon.

[matthewcarbone]

Yup sorry for the delays. It's proposal writing season right now and things are really busy. However, my team and I are working on these notebooks and will get the merged in as soon as possible. Thanks, all!

[ppxasjsm]

Thanks for the update @matthewcarbone!

[ppxasjsm]

@editorialbot add @larsenkg to reviewers

[editorialbot]

@larsenkg added to the reviewers list!

[ppxasjsm]

@larsenkg, I think you said you'd be back from holidays about now? You can get started by generating your reviewer checklist using editorialbot with:
@editorialbot generate my checklist

Please let me know if you have an issues.

[larsenkg]

Review checklist for @larsenkg

Conflict of interest

• I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the https://github.com/AI-multimodal/Lightshow?
• License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• Contribution and authorship: Has the submitting author (@matthewcarbone) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
• Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
• Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
• Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
• State of the field: Do the authors describe how this software compares to other commonly-used packages?
• Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

[maurov]

@matthewcarbone regarding the software paper, I think overall reads well. I recommend adding a couple of sentences about the fact that this code is part of a (much) larger workflow consisting in running the ab-initio codes, generating the simulated spectra and comparing/benchmarking with experimental spectra.

Please, could you consider the following points ?

• correct citation syntax for Meng et al.
• add citation for Pymatgen
• personally, I would add a citation for FEFF10/Corvus

[editorialbot]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[matthewcarbone]

@editorialbot check references

[editorialbot]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1063/1.4812323 is OK
- 10.1016/j.commatsci.2011.02.023 is OK
- 10.1039/B926434E is OK
- 10.1103/PhysRevB.80.075102 is OK
- 10.1103/PhysRevB.87.205105 is OK
- 10.1103/PhysRevB.83.115106 is OK
- 10.1039/D2CP01030E is OK
- 10.1103/PhysRevB.98.235205 is OK
- 10.1088/2516-1075/ab3123 is OK
- 10.48550/arXiv.2303.17089 is OK
- 10.1107/S1600577521008614 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[Kevin-Mattheus-Moerman]

@matthewcarbone On the community guidelines, apologies, I seem to have missed it. Thanks, what you have there is simple/concise, but sufficient.
Thanks also for implementing those DOI changes, note this one still seems missing from the paper:

• First-principles calculations of x-ray absorption in a scheme based on ultrasoft pseudopotentials : https://doi.org/10.1103/PhysRevB.80.075102

[matthewcarbone]

@Kevin-Mattheus-Moerman No problem. Note, I think we might be looking at different drafts. This is what I'm seeing after downloading from this post by the bot:

DOI looks correct.

[Kevin-Mattheus-Moerman]

@editorialbot generate pdf

[editorialbot]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[Kevin-Mattheus-Moerman]

@matthewcarbone currently your paper.bib file says:

Hence the above updated paper draft still lacks the DOI:

Perhaps you were working in a different branch? Can you update your bib file and the paper in the joss-manuscript branch?

[matthewcarbone]

@Kevin-Mattheus-Moerman ah that's a different reference. Ok all is fixed now.

[matthewcarbone]

@editorialbot generate pdf

[editorialbot]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[Kevin-Mattheus-Moerman]

@matthewcarbone I think all looks good now. So we can proceed.

[Kevin-Mattheus-Moerman]

@larsenkg thanks for your review here. Since the community guidelines are there (see CONTRIBUTING.md on repo), and the references have been fixed. We will proceed now with acceptance, but for completeness it would be great if you could tick those last boxes ☝️ .

[Kevin-Mattheus-Moerman]

@editorialbot recommend accept

[editorialbot]

I'm sorry human, I don't understand that. You can see what commands I support by typing:

@editorialbot commands

[Kevin-Mattheus-Moerman]

@editorialbot accept

[editorialbot]

Doing it live! Attempting automated processing of paper acceptance...

[editorialbot]

Ensure proper citation by uploading a plain text CITATION.cff file to the default branch of your repository.

If using GitHub, a Cite this repository menu will appear in the About section, containing both APA and BibTeX formats. When exported to Zotero using a browser plugin, Zotero will automatically create an entry using the information contained in the .cff file.

You can copy the contents for your CITATION.cff file here:

CITATION.cff

cff-version: "1.2.0"
authors:
- family-names: Carbone
  given-names: Matthew R.
  orcid: "https://orcid.org/0000-0002-5181-9513"
- family-names: Meng
  given-names: Fanchen
  orcid: "https://orcid.org/0000-0001-9869-9005"
- family-names: Vorwerk
  given-names: Christian
  orcid: "https://orcid.org/0000-0002-2516-9553"
- family-names: Maurer
  given-names: Benedikt
  orcid: "https://orcid.org/0000-0001-9152-7390"
- family-names: Peschel
  given-names: Fabian
  orcid: "https://orcid.org/0000-0003-0619-6713"
- family-names: Qu
  given-names: Xiaohui
  orcid: "https://orcid.org/0000-0001-5651-8405"
- family-names: Stavitski
  given-names: Eli
  orcid: "https://orcid.org/0000-0002-3337-2930"
- family-names: Draxl
  given-names: Claudia
  orcid: "https://orcid.org/0000-0003-3523-6657"
- family-names: Vinson
  given-names: John
  orcid: "https://orcid.org/0000-0002-7619-7060"
- family-names: Lu
  given-names: Deyu
  orcid: "https://orcid.org/0000-0003-4351-6085"
contact:
- family-names: Carbone
  given-names: Matthew R.
  orcid: "https://orcid.org/0000-0002-5181-9513"
- family-names: Meng
  given-names: Fanchen
  orcid: "https://orcid.org/0000-0001-9869-9005"
doi: 10.5281/zenodo.8118592
message: If you use this software, please cite our article in the
  Journal of Open Source Software.
preferred-citation:
  authors:
  - family-names: Carbone
    given-names: Matthew R.
    orcid: "https://orcid.org/0000-0002-5181-9513"
  - family-names: Meng
    given-names: Fanchen
    orcid: "https://orcid.org/0000-0001-9869-9005"
  - family-names: Vorwerk
    given-names: Christian
    orcid: "https://orcid.org/0000-0002-2516-9553"
  - family-names: Maurer
    given-names: Benedikt
    orcid: "https://orcid.org/0000-0001-9152-7390"
  - family-names: Peschel
    given-names: Fabian
    orcid: "https://orcid.org/0000-0003-0619-6713"
  - family-names: Qu
    given-names: Xiaohui
    orcid: "https://orcid.org/0000-0001-5651-8405"
  - family-names: Stavitski
    given-names: Eli
    orcid: "https://orcid.org/0000-0002-3337-2930"
  - family-names: Draxl
    given-names: Claudia
    orcid: "https://orcid.org/0000-0003-3523-6657"
  - family-names: Vinson
    given-names: John
    orcid: "https://orcid.org/0000-0002-7619-7060"
  - family-names: Lu
    given-names: Deyu
    orcid: "https://orcid.org/0000-0003-4351-6085"
  date-published: 2023-07-29
  doi: 10.21105/joss.05182
  issn: 2475-9066
  issue: 87
  journal: Journal of Open Source Software
  publisher:
    name: Open Journals
  start: 5182
  title: "Lightshow: a Python package for generating computational x-ray
    absorption spectroscopy input files"
  type: article
  url: "https://joss.theoj.org/papers/10.21105/joss.05182"
  volume: 8
title: "Lightshow: a Python package for generating computational x-ray
  absorption spectroscopy input files"

If the repository is not hosted on GitHub, a .cff file can still be uploaded to set your preferred citation. Users will be able to manually copy and paste the citation.

Find more information on .cff files here and here.

[editorialbot]

🐘🐘🐘 👉 Toot for this paper 👈 🐘🐘🐘

[editorialbot]

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨

Here's what you must now do:

0. Check final PDF and Crossref metadata that was deposited 👉 Creating pull request for 10.21105.joss.05182 joss-papers#4436
1. Wait a couple of minutes, then verify that the paper DOI resolves https://doi.org/10.21105/joss.05182
2. If everything looks good, then close this review issue.
3. Party like you just published a paper! 🎉🌈🦄💃👻🤘

Any issues? Notify your editorial technical team...

[Kevin-Mattheus-Moerman]

@matthewcarbone congratulations on this JOSS publication!

@ppxasjsm thanks for editing!

And a special thanks to the reviewers: @maurov, @larsenkg

[editorialbot]

🎉🎉🎉 Congratulations on your paper acceptance! 🎉🎉🎉

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](https://joss.theoj.org/papers/10.21105/joss.05182/status.svg)](https://doi.org/10.21105/joss.05182)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.05182">
  <img src="https://joss.theoj.org/papers/10.21105/joss.05182/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: https://joss.theoj.org/papers/10.21105/joss.05182/status.svg
   :target: https://doi.org/10.21105/joss.05182

This is how it will look in your documentation:

We need your help!

The Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

• Volunteering to review for us sometime in the future. You can add your name to the reviewer list here: https://reviewers.joss.theoj.org/join
• Making a small donation to support our running costs here: https://numfocus.org/donate-to-joss