Skip to content

[REVIEW]: Lepton #1005

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
13 of 36 tasks
whedon opened this issue Oct 8, 2018 · 116 comments
Closed
13 of 36 tasks

[REVIEW]: Lepton #1005

whedon opened this issue Oct 8, 2018 · 116 comments
Assignees
Labels
accepted published Papers published in JOSS recommend-accept Papers recommended for acceptance in JOSS. review

Comments

@whedon
Copy link

whedon commented Oct 8, 2018

Submitting author: @slithiaote (Sébastien Li-Thiao-Té)
Repository: https://github.com/slithiaote/lepton
Version: v1.2
Editor: @kyleniemeyer
Reviewers: @tpetricek, @rljacobson
Archive: 10.5281/zenodo.3492221

Status

status

Status badge code:

HTML: <a href="http://joss.theoj.org/papers/5c3cb173a0e6dc2a25a775b5d133b027"><img src="http://joss.theoj.org/papers/5c3cb173a0e6dc2a25a775b5d133b027/status.svg"></a>
Markdown: [![status](http://joss.theoj.org/papers/5c3cb173a0e6dc2a25a775b5d133b027/status.svg)](http://joss.theoj.org/papers/5c3cb173a0e6dc2a25a775b5d133b027)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@tpetricek & @rljacobson, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

  1. Make sure you're logged in to your GitHub account
  2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.theoj.org/about#reviewer_guidelines. Any questions/concerns please let @yochannah know.

Please try and complete your review in the next two weeks

Review checklist for @tpetricek

Conflict of interest

Code of Conduct

General checks

  • Repository: Is the source code for this software available at the repository url?
  • License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
  • Version: v1.2
  • Authorship: Has the submitting author (@slithiaote) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

  • Installation: Does installation proceed as outlined in the documentation?
  • Functionality: Have the functional claims of the software been confirmed?
  • Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

  • A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
  • Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
  • Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
  • Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
  • Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
  • Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

  • Authors: Does the paper.md file include a list of authors with their affiliations?
  • A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
  • References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

Review checklist for @rljacobson

Conflict of interest

Code of Conduct

General checks

  • Repository: Is the source code for this software available at the repository url?
  • License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
  • Version: v1.2
  • Authorship: Has the submitting author (@slithiaote) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

  • Installation: Does installation proceed as outlined in the documentation?
  • Functionality: Have the functional claims of the software been confirmed?
  • Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

  • A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
  • Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
  • Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
  • Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
  • Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
  • Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

  • Authors: Does the paper.md file include a list of authors with their affiliations?
  • A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
  • References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?
@whedon
Copy link
Author

whedon commented Oct 8, 2018

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @adammichaelwood, it looks like you're currently assigned as the reviewer for this paper 🎉.

⭐ Important ⭐

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

  1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

watching

  1. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

notifications

For a list of things I can do to help you, just type:

@whedon commands

@whedon
Copy link
Author

whedon commented Oct 8, 2018

Attempting PDF compilation. Reticulating splines etc...

@whedon
Copy link
Author

whedon commented Oct 8, 2018

@yochannah
Copy link

👋 Gentle nudge - @tpetricek, @adammichaelwood, have you had a chance to start looking at this yet?

@tpetricek
Copy link

Thanks for the nudge! I was at a conference last week, but will post my initial comments today or on Thursday.

@adammichaelwood
Copy link

Sorry - I've been a bit swamped in all areas.
I can make no promises, but I will try to carve out time for this (and my other review I volunteered for)
as soon as I can.

@tpetricek
Copy link

[Better late than never! Below is my review. I was not able to tick all the boxes above, so the review explains what problems I had and what I think needs to be corrected. I opened one issue for Lepton tracking the most important question, i.e. how to actually compile it]

The project is a literate programming package that allows users to write code in a classic literate programming style as proposed by Knuth. In case of Lepton, this means using LaTeX syntax with embedded chunks of code that can be combined together to define the program (this also means you can refer to a code block defined later in the file). Lepton is mostly an implementation of the original idea by Knuth, without adding much new (although the documentation is not very clear about that, so there might be new things I might have missed). However, that seems reasonable - a good implementation of the idea for OCaml would be a useful contribution.

Lepton is "implemented in itself" and my understanding is that to build it, you first need to run an existing version (to bootstrap it) and to obtain OCaml code which you can then compile to actually compile Lepton. However, I was not able to do this from scratch (see issue slithiaote/lepton#2). I think the repository needs a Makefile of some sorts and clear instructions about how to bootstrap things (possibly for different OSes, if this cannot be universal).

Not being able to bootstrap Lepton meant that I was not able to tick the following review boxes:

  • Installation - as discussed above, I'm not sure how to bootstrap the build process
  • Functionality - consequently, I was not able to run Lepton and see what it does

Out of the documentation required for JOSS, I think the following are missing or not clear:

  • Installation instructions - I understand that this is tricky given the nature of the project (and the fact that it is written using itself), but there should be clear instructions how to compile it (for some supported platform) and run it. I think this should also use OCaml source code rather than binary for bootstrapping.

  • Example usage - There are some instructions in lepton_manual.pdf, but it is not entirely clear to me how to go from nw file to an executable (how to go from nw file to LaTeX file is however quite clear).

  • Automated tests - As far as I can see, there are no tests

  • Community guidelines - Besides the standard license file, which sets the basic rules, there is not much to guide community contributions. Having a standard CONTRIBUTING file would be useful.

@yochannah
Copy link

@slithiaote let me know if/when you've addressed the relevant issues and are ready to move forward with this

@slithiaote
Copy link

@yochannah I replied to the issue/report on Nov 7th. At least part of the problem seems to be Windows, which is not a supported platform. I have asked for more input, but @tpetricek did not reply and I am hesitant to ping.

@tpetricek
Copy link

tpetricek commented Dec 5, 2018

@slithiaote I missed the reply. Sorry. I just added a comment on the issue.

That said, I think there are other issues (see my comment above) that need to be addressed before this can be accepted as a JOSS publication - there are a few things that the JOSS checklist requires that I think are not yet satisfied by lepton.

[EDIT] Also, if Windows is not supported platform (not even through the Ubuntu bash), then I don't think I can review this contribution.

@yochannah
Copy link

@tpetricek Sorry, I missed the edit note. I can look for another reviewer - it took me a while to find people interested in literate programming - you don't have any suggestions of possible interested colleagues or communities to approach?

@tpetricek
Copy link

@yochannah I'm happy to stay involved if the author is interested in supporting Windows, at least through the Ubuntu Bash shell - I don't see why that should be impossible and I think it would be a good thing for the project in general.

No matter what the supported platforms are, I think the project should use some form of pre-generated OCaml source code for bootstrapping rather than a checked-in executable where nobody knows what the executable might be doing. I think making that change (and adding a makefile) would likely make this work on Windows anyway.

That said, if the project can be accepted with a binary file in the repository, then it needs to be reviewed by someone who is able (and brave enough) to run it, so in that case, I won't be able to help. I sadly don't have any good alternative reviewer ideas.

@slithiaote
Copy link

@tpetricek Lepton compiles on Windows, but is mostly unusable because it requires the Unix library and Unix tools. Your comment made me realize that Windows Subsystem for Linux can provide those, although I will not be able to test it myself. I have been working on a source-based distribution for the past few days. I should be able to update this week.

@yochannah Have you tried the "reproducible research" community ?

@yochannah
Copy link

Okay - this is good - it sounds like we may be able to keep @tpetricek onboard if we can get this on Windows! 👍

I do think I generally agree about @tpetricek's comment re pre-generated OCaml source code.

@slithiaote
Copy link

I just committed to Github and tagged v1.1

@yochannah
Copy link

Hey all - just checking the status of this review. @tpetricek, has @slithiaote's update managed to resolve your issues? @adammichaelwood, do you think you might have a chance to complete your review soon? Thanks all!!

@tpetricek
Copy link

Thanks for the ping. I'm still trying to get it to build. Just added a comment there: slithiaote/lepton#2

In addition to that, the project still does not satisfy some of the JOSS items from the check-list above, so adding those (assuming they are strictly required for acceptance) would be another good thing to do in the meantime (especially, tests, contributor guidelines)

@yochannah
Copy link

yochannah commented Feb 13, 2019

Quick note - the readme on this repo was updated recently! :) I've added some comments about build issues in slithiaote/lepton#2

@slithiaote
Copy link

Yes, I updated the readme recently, but I did not have the time to comment as well. As far as I understand, @tpetricek expects to find installation instructions in README.md instead of lepton.pdf, so I included a copy of these instructions in README.md, as well as contributor guidelines similar to what is present in a few recent accepted JOSS papers.

The project contains two test files: an embedded lepton_manual.nw to produce the usage manual and lepton.nw itself. These two files use most of the functionality of Lepton. I believe that bootstrapping Lepton constitutes a fairly complete test of the software. Incidentally, boostrapping will automatically attempt to process lepton_manual.nw.

@yochannah
Copy link

@slithiaote I think I generally do agree - clear getting started docs in the readme is important for all open source software. I've added some comments myself in slithiaote/lepton#2 as I also had trouble getting things to run.

@labarba
Copy link
Member

labarba commented Mar 16, 2019

👋 @slithiaote — Have you been able to make progress on this submission in the past month? What's your status?

@slithiaote
Copy link

Not much has evolved since my last reply to comments last month. I am unsure what the issue is. The software is written in OCaml using only the standard library, so compilation should be uneventful. I anticipate that bootstrapping may not work out-of-the-box on all systems, but I did not receive error outputs on that topic.

@yochannah
Copy link

👋 @adammichaelwood, @tpetricek - would y'all have a chance to revisit this soon & let us know where things are? Thanks!!

@yochannah
Copy link

@whedon commands

@whedon
Copy link
Author

whedon commented Mar 23, 2019

Here are some things you can ask me to do:

# List all of Whedon's capabilities
@whedon commands

# Assign a GitHub user as the sole reviewer of this submission
@whedon assign @username as reviewer

# Add a GitHub user to the reviewers of this submission
@whedon add @username as reviewer

# Remove a GitHub user from the reviewers of this submission
@whedon remove @username as reviewer

# List of editor GitHub usernames
@whedon list editors

# List of reviewers together with programming language preferences and domain expertise
@whedon list reviewers

# Change editorial assignment
@whedon assign @username as editor

# Set the software archive DOI at the top of the issue e.g.
@whedon set 10.0000/zenodo.00000 as archive

# Set the software version at the top of the issue e.g.
@whedon set v1.0.1 as version

# Open the review issue
@whedon start review

EDITORIAL TASKS

# Compile the paper
@whedon generate pdf

# Compile the paper from alternative branch
@whedon generate pdf from branch custom-branch-name

# Remind an author or reviewer to return to a review after a
# certain period of time (supported units days and weeks)
@whedon remind @reviewer in 2 weeks

# Ask Whedon to accept the paper and deposit with Crossref
@whedon accept

# Ask Whedon to check the references for missing DOIs
@whedon check references

@yochannah
Copy link

/ooo today until april 1

@ooo
Copy link

ooo bot commented Mar 23, 2019

@whedon
Copy link
Author

whedon commented Oct 18, 2019

PDF failed to compile for issue #1005 with the following error:

E, [2019-10-18 20:18:05#375] ERROR -- : Failed to parse BibTeX on value "This" (NAME) ["@", "Comment"]
bibtex.y:138:in on_error': Failed to parse BibTeX on value "This" (NAME) ["@", "Comment"] (BibTeX::ParseError) from /app/vendor/ruby-2.4.4/lib/ruby/2.4.0/racc/parser.rb:259:in _racc_do_parse_c'
from /app/vendor/ruby-2.4.4/lib/ruby/2.4.0/racc/parser.rb:259:in do_parse' from bibtex.y:111:in parse'
from /app/vendor/bundle/ruby/2.4.0/gems/bibtex-ruby-5.0.0/lib/bibtex/bibliography.rb:67:in parse' from /app/vendor/bundle/ruby/2.4.0/gems/bibtex-ruby-5.0.0/lib/bibtex/bibliography.rb:50:in open'
from /app/vendor/bundle/ruby/2.4.0/gems/bibtex-ruby-5.0.0/lib/bibtex/utilities.rb:25:in open' from /app/vendor/bundle/ruby/2.4.0/bundler/gems/whedon-1e4ee47b240d/lib/whedon/bibtex_parser.rb:36:in generate_citations'
from /app/vendor/bundle/ruby/2.4.0/bundler/gems/whedon-1e4ee47b240d/lib/whedon/compilers.rb:222:in crossref_from_markdown' from /app/vendor/bundle/ruby/2.4.0/bundler/gems/whedon-1e4ee47b240d/lib/whedon/compilers.rb:20:in generate_crossref'
from /app/vendor/bundle/ruby/2.4.0/bundler/gems/whedon-1e4ee47b240d/lib/whedon/processor.rb:95:in compile' from /app/vendor/bundle/ruby/2.4.0/bundler/gems/whedon-1e4ee47b240d/bin/whedon:79:in compile'
from /app/vendor/bundle/ruby/2.4.0/gems/thor-0.20.3/lib/thor/command.rb:27:in run' from /app/vendor/bundle/ruby/2.4.0/gems/thor-0.20.3/lib/thor/invocation.rb:126:in invoke_command'
from /app/vendor/bundle/ruby/2.4.0/gems/thor-0.20.3/lib/thor.rb:387:in dispatch' from /app/vendor/bundle/ruby/2.4.0/gems/thor-0.20.3/lib/thor/base.rb:466:in start'
from /app/vendor/bundle/ruby/2.4.0/bundler/gems/whedon-1e4ee47b240d/bin/whedon:116:in <top (required)>' from /app/vendor/bundle/ruby/2.4.0/bin/whedon:23:in load'
from /app/vendor/bundle/ruby/2.4.0/bin/whedon:23:in `

'

@Kevin-Mattheus-Moerman
Copy link
Member

@openjournals/dev @arfon ☝️ do you know what is going on here?

@slithiaote
Copy link

Is it perhaps related to the comment in the first line of my bibtex file ?
@Comment This file was generated from lepton.nw by lepton.bin

@arfon
Copy link
Member

arfon commented Oct 18, 2019

Yeah, any chance you could remove that line please @slithiaote?

@Kevin-Mattheus-Moerman
Copy link
Member

Kevin-Mattheus-Moerman commented Oct 21, 2019

@slithiaote can you merge this PR slithiaote/lepton#5 to fix the above. Once you do that can you run @whedon generate pdf and tag @openjournals/joss-eics so we can resume the process of accepting your work. Thanks

@slithiaote
Copy link

I would like to keep the Bibtex comment because it makes it clear that users should not edit the bib file directly. I can remove @COMMENT and leave the text dangling outside the Bibtex entries. Both methods are official ways to add comments in Bibtex files.

Do I need to tag a new release on zenodo with tag @openjournals/joss-eics ?

@arfon
Copy link
Member

arfon commented Oct 22, 2019

I can remove @comment and leave the text dangling outside the Bibtex entries. Both methods are official ways to add comments in Bibtex files.

That probably works. No new release is necessary.

@slithiaote
Copy link

@whedon generate pdf

@whedon
Copy link
Author

whedon commented Oct 22, 2019

Attempting PDF compilation. Reticulating splines etc...

@whedon
Copy link
Author

whedon commented Oct 22, 2019

@arfon
Copy link
Member

arfon commented Oct 22, 2019

@whedon accept

@whedon
Copy link
Author

whedon commented Oct 22, 2019

Attempting dry run of processing paper acceptance...

@whedon
Copy link
Author

whedon commented Oct 22, 2019


OK DOIs

- 10.1093/comjnl/27.2.97 is OK
- 10.1016/j.procs.2012.04.190 is OK
- 10.1016/j.procs.2012.04.047 is OK
- 10.1109/52.311070 is OK
- 10.5281/zenodo.1311588 is OK

MISSING DOIs

- None

INVALID DOIs

- None

@whedon
Copy link
Author

whedon commented Oct 22, 2019

Check final proof 👉 openjournals/joss-papers#1039

If the paper PDF and Crossref deposit XML look good in openjournals/joss-papers#1039, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.

@whedon accept deposit=true

@whedon
Copy link
Author

whedon commented Oct 22, 2019


OK DOIs

- 10.1093/comjnl/27.2.97 is OK
- 10.1016/j.procs.2012.04.190 is OK
- 10.1016/j.procs.2012.04.047 is OK
- 10.1109/52.311070 is OK
- 10.5281/zenodo.1311588 is OK

MISSING DOIs

- None

INVALID DOIs

- None

1 similar comment
@whedon
Copy link
Author

whedon commented Oct 23, 2019


OK DOIs

- 10.1093/comjnl/27.2.97 is OK
- 10.1016/j.procs.2012.04.190 is OK
- 10.1016/j.procs.2012.04.047 is OK
- 10.1109/52.311070 is OK
- 10.5281/zenodo.1311588 is OK

MISSING DOIs

- None

INVALID DOIs

- None

@Kevin-Mattheus-Moerman
Copy link
Member

@whedon accept deposit=true

@whedon
Copy link
Author

whedon commented Oct 24, 2019

Doing it live! Attempting automated processing of paper acceptance...

@whedon
Copy link
Author

whedon commented Oct 24, 2019

🐦🐦🐦 👉 Tweet for this paper 👈 🐦🐦🐦

@whedon
Copy link
Author

whedon commented Oct 24, 2019

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨

Here's what you must now do:

  1. Check final PDF and Crossref metadata that was deposited 👉 Creating pull request for 10.21105.joss.01005 joss-papers#1046
  2. Wait a couple of minutes to verify that the paper DOI resolves https://doi.org/10.21105/joss.01005
  3. If everything looks good, then close this review issue.
  4. Party like you just published a paper! 🎉🌈🦄💃👻🤘

Any issues? notify your editorial technical team...

@Kevin-Mattheus-Moerman
Copy link
Member

@openjournals/dev this DOI is not resolving yet

@arfon
Copy link
Member

arfon commented Oct 24, 2019

@openjournals/dev this DOI is not resolving yet

Looks like the Crossref queue is backed up. It should register in the next few hours.

@kyleniemeyer
Copy link

@Kevin-Mattheus-Moerman @arfon looks good now!

@kyleniemeyer
Copy link

Congrats @slithiaote on your article's publication in JOSS—thanks for being patient over this long process. Many thanks to @tpetricek and @rljacobson for reviewing this submission!

@whedon
Copy link
Author

whedon commented Oct 24, 2019

🎉🎉🎉 Congratulations on your paper acceptance! 🎉🎉🎉

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](https://joss.theoj.org/papers/10.21105/joss.01005/status.svg)](https://doi.org/10.21105/joss.01005)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.01005">
  <img src="https://joss.theoj.org/papers/10.21105/joss.01005/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: https://joss.theoj.org/papers/10.21105/joss.01005/status.svg
   :target: https://doi.org/10.21105/joss.01005

This is how it will look in your documentation:

DOI

We need your help!

Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

@slithiaote
Copy link

Thanks to the editors and reviewers from JOSS for their dedication and perseverance.

@whedon whedon added published Papers published in JOSS recommend-accept Papers recommended for acceptance in JOSS. labels Mar 2, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
accepted published Papers published in JOSS recommend-accept Papers recommended for acceptance in JOSS. review
Projects
None yet
Development

No branches or pull requests