[PRE REVIEW]: Lepton

[whedon]

Submitting author: @slithiaote (Sébastien Li-Thiao-Té)
Repository: https://github.com/slithiaote/lepton
Version: v1.0
Editor: @yochannah
Reviewers: @adammichaelwood, @tpetricek

Author instructions

Thanks for submitting your paper to JOSS @slithiaote. The JOSS editor (shown at the top of this issue) will work with you on this issue to find a reviewer for your submission before creating the main review issue.

@slithiaote if you have any suggestions for potential reviewers then please mention them here in this thread. In addition, this list of people have already agreed to review for JOSS and may be suitable for this submission.

Editor instructions

The JOSS submission bot @whedon is here to help you find and assign reviewers and start the main review. To find out what @whedon can do for you type:

@whedon commands

[whedon]

Hello human, I'm @whedon. I'm here to help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@whedon commands

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

--> Check article proof 📄 <--

[jedbrown]

Thanks for your submission. I have some questions and concerns:

• I don't see any source code in the repository, just an executable. Is that intentional?

• The CeCILL-B license is not OSI-approved (for the same reason 4-clause BSD is not).

https://joss.readthedocs.io/en/latest/submitting.html#submission-requirements

• Can you explain how you believe this software fits the research criteria in the submission requirements?

[slithiaote]

Thank you for your comments.

• It is intentional not to have source code in a separate file from documentation :
  -- In the literate programming paradigm, source code is presented in a human readable format, regardless of the constraints of the programming language. Consequently, it is usually embedded in a file in documentation format. Here, the source code is embedded in the plain text lepton.nw file, while humans should read the generated lepton.pdf file.
  -- The actual machine-readable source code is intended to be extracted by Lepton from the lepton.nw file, following the instructions in Section 3 (Compilation). As Lepton generates both the pdf and the executable from these instructions, humans should not have to deal with the intermediate Ocaml .ml files.

• As far as I understand, the CeCILL licenses are listed on the OSI website as open source. Do you mean that the B version is not OSI-approved ? https://opensource.org/licenses/CECILL-2.1

• I believe that Lepton fits the research criteria :
  -- This software is useful for research. In our mathematics lab at Paris 13 University, it has been used to produce data analysis reports on the amounts of oxydative stress in rat eyes in response to the injection of different chemicals. Another example is that we use it to develop image analysis software for detecting ulcers in colonoscopy videos. Using Lepton guarantees that the data analysis results are produced by the source code described in the same document.
  -- This software is research in itself. In the last decade, several solutions have been proposed to improve the reproducibility of research results and in particular data analysis. The current trend is "notebooks" such as Jupyter or Matlab notebooks, i.e. systems that enable interactive/online code execution and commenting of the code blocks. This topic is still debated in the community, and there are dissenting opinions on "reproducible research" itself such as https://doi.org/10.1080/0952813X.2017.1413140. Lepton supports / implements a proposal for reproducible research papers that is focused on plain text for long term readability and universality (not dedicated to a specific programming language). The two cited papers in ICCS are about Lepton. In addition, Lepton is designed to be easily re-implemented for long term maintainability ; publishing the documented implementation details provides a reference implementation and preserve the research.

Can I improve the paper.md file after submission following your comments ? How should I do it ?

[jedbrown]

1. How was lepton.bin bootstrapped? Is it possible to recreate without executing the existing Linux/x86-only binary?

2. Correct, CeCILL-B (which is very different from CeCILL-2.1) is not OSI-approved.

3. This is more "meta" than typical, but there have been accepted papers on tooling to facilitate reproducible research and I think it would be in scope according to that precedent.

[slithiaote]

1. How was lepton.bin bootstrapped? Is it possible to recreate without executing the existing Linux/x86-only binary?
   -- From the point of view of the computer, lepton.bin is compiled from main.ml (and several modules such as lepton.ml, lexer.ml, interpreters.ml, see chunk 26 on page 8 of lepton.pdf)
   -- It is possible to recreate the source files from lepton.nw, with just copy-and-paste. The executable does that for you, and executes the instructions in chunk 16 for lexer.ml and chunk 26 for compiling, but everything is manageable by hand. It is a feature of Lepton that the source code is not too large so that it should be easy to recreate by hand, and sufficiently documented so that it should be easy to re-implement for long-term maintainability.
   -- Bootstrapping starts with a program that assembles and writes chunks to disk from a .nw file. As soon as you have that, you can stop writing source code directly, and write lepton.nw instead.
   -- The second stage is command execution via the shell interpreter, which enables lepton to generate itself by typing lepton.bin lepton.nw
   -- A third stage is to encapsulate the source code in an executable program. Lepton does this partially. As lepton can process input on stdin, it is usable as a command interpreter via the #! mechanism. On a suitably configured system, the command ./lepton.nw in a terminal will generate the lepton.bin executable and the manual.
2. Correct, CeCILL-B (which is very different from CeCILL-2.1) is not OSI-approved.
   -- I understand. I will switch to CeCILL-2.1 then.

I have updated the github repository and improved the paper.md file following your comments. Was that the correct way to proceed ?

[jedbrown]

1. Thanks for the explanation. Please consider the review criteria.

2. You could consider an OSI-approved permissive license. Changing to CeCILL-2.1 is far more consequential.

3. Yes, updating the repository to improve paper.md and/or address reviewer comments is the intended process.

[slithiaote]

Following your comments, I have modified the "Compilation" section into broader and more detailed instructions for "Obtaining the software".
I have re-read and I hope that this should satisfy the review criteria. If not, could you be more specific as to what you feel needs improvements ?

[arfon]

👋 @yochannah - would you be willing to edit this submission for JOSS?

[yochannah]

@whedon assign @yochannah as editor

[whedon]

OK, the editor is @yochannah

[yochannah]

I know of one possible reviewer who uses OCaml - just checking with them now. @slithiaote do you have any suggestions for reviewers?

[yochannah]

@kit-ty-kate 👋 would you be interested in reviewing this?

[kit-ty-kate]

I'm not sure I'm well suited to review this.
I tried to compile the project, there were no instructions to compile anything and the only thing I could try (i.e. ./lepton.bin lepton.nw) ended up in an infinite loop with some error messages and some OCaml files being generated.
The generated code itself is not very readable, almost no indentation and I don't know what to do with it. Also it uses old-style OCaml constructs but that's minor. I couldn't find any instructions to build the generated OCaml files but I managed to have something with ocamlbuild -package str,unix main.byte in the end (but it produced some maybe relevant and some irrelevant warnings)

Modern OCaml code have at least an opam file and it's not the case here, but I guess it's maybe not the goal of the tool as it seems to have its own build format (e.g. <<shell_compile -exec shell>>= in lepton.nw)
I'm certainly not an expert in literate programming and don't know much about it so I can't really comment on that I guess apart from the fact that it is very different from how OCaml projects are usually handled.

[yochannah]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

👉 Check article proof 📄 👈

[slithiaote]

Thank you for your comments.

I agree that this is very different from how OCaml projects are usually handled. In literate programming, the file lepton.nw containing the documentation is intended to also generate the source code. In the case of Lepton, you should not have to compile the generated OCaml files yourself because running ./lepton.bin lepton.nw -o lepton.tex should produce a new lepton.bin. Similarly, the generated code is not intended to be read directly, but rather from the lepton.pdf file.

The ocamlopt compilation command is in Section 3 of lepton.pdf. Did you download the Zenodo package ? Following Jed Brown's comments, I rewrote that section on Github and I feel that the instructions are now clearer.

The "infinite loop" is a LaTeX compilation issue, at least on my system. When encountering a serious error such as a missing package, LaTeX stops and asks the user what to do, which puts everything in a waiting state. I removed the unnecessary packages, selected nonstopmode and inserted a warning in the manual.

The article proof does not render the item list, whereas the github preview does.

[arfon]

The article proof does not render the item list, whereas the github preview does.

@slithiaote - this PR should fix the rendering issue in the PDF: slithiaote/lepton#1

[arfon]

@whedon generate pdf

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

👉 Check article proof 📄 👈

[yochannah]

@slithiaote thanks for the edits you've made. I think there are still some formatting issues in your pdf around the quotes - it looks like maybe there are two single apostrophes '' rather than a single double quote " around Lepton, which makes the proof look a little strange.

I'm still hunting for reviewers & will update when I find someone.

[adammichaelwood]

I'm available for review. Let me know how I can best help.

[yochannah]

@adammichaelwood Hooray, and thank you for volunteering!

I would like to have a second reviewer for this paper. I'll kick off the official reviewer process once I find a second (if you can suggest anyone who might be able to review this, that would be awesome!). In the meantime feel free to look at the package and look at the review criteria for JOSS.

[yochannah]

Still hunting for a second reviewer at the minute. I've sent a few more emails & will update as soon as we find someone.

[slithiaote]

Thanks

[adammichaelwood]

I will mention this in the Write the Docs slack. It seems likely there are some other people into literate programming in that community.

[tpetricek]

Hi all, @yochannah asked me to be the second reviewer for this - I'm happy to accept. I'll go ahead and put together a review following the guidelines in the next few days. Is there anything else that I need to do?

[yochannah]

Hey @tpetricek and thanks for volunteering to review! I'll kick the review process off now.

[yochannah]

@whedon assign @adammichaelwood as reviewer

[whedon]

OK, the reviewer is @adammichaelwood

[yochannah]

@whedon add @tpetricek as reviewer

[whedon]

OK, @tpetricek is now a reviewer

[yochannah]

@whedon start review

[whedon]

OK, I've started the review over in #1005. Feel free to close this issue now!

[yochannah]

Okies - @adammichaelwood @tpetricek, you can start the review over in #1005. @slithiaote thanks for your patience while we were finding reviewers!