[REVIEW]: Augmentor: An Image Augmentation Library for Machine Learning

[whedon]

Submitting author: @mdbloice (Marcus Bloice)
Repository: https://github.com/mdbloice/Augmentor
Version: v0.1.8
Editor: @arfon
Reviewer: @sealhuang
Archive: 10.5281/zenodo.1041946

Status

Status badge code:

HTML: <a href="http://joss.theoj.org/papers/269f2a501216b676385c8d745b856764"><img src="http://joss.theoj.org/papers/269f2a501216b676385c8d745b856764/status.svg"></a>
Markdown: [![status](http://joss.theoj.org/papers/269f2a501216b676385c8d745b856764/status.svg)](http://joss.theoj.org/papers/269f2a501216b676385c8d745b856764)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer questions

@sealhuang, please carry out your review in this issue by updating the checklist below (please make sure you're logged in to GitHub). The reviewer guidelines are available here: http://joss.theoj.org/about#reviewer_guidelines. Any questions/concerns please let @arfon know.

Conflict of interest

• As the reviewer I confirm that I have read the JOSS conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the repository url?
• License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• Version: Does the release version given match the GitHub release (v0.1.5)?
• Authorship: Has the submitting author (@mdbloice) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• Authors: Does the paper.md file include a list of authors with their affiliations?
• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

[whedon]

Hello human, I'm @whedon. I'm here to help you with some common editorial tasks for JOSS. @sealhuang it looks like you're currently assigned as the reviewer for this paper 🎉.

⭐ Important ⭐

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As as reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all JOSS reviews 😿

To fix this do the following two things:

1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

2. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

For a list of things I can do to help you, just type:

@whedon commands

[arfon]

@sealhuang - please carry out your review in this issue by updating the checklist above and giving feedback in this issue. The reviewer guidelines are available here: http://joss.theoj.org/about#reviewer_guidelines

Any questions/concerns please let me know.

[sealhuang]

I have completed the review, @arfon and @mdbloice.

The Augmentor package provides a convenient and flexible approach to generate new image samples via various operations. The package is particularly useful for generating batch of sample while training a deep learning model.

Overall I think this is a submission suited for the JOSS, and it meets the requirements for acceptance with a few exceptions. I've detailed the items below

• Version: The latest stable version is 0.1.18, not 0.1.5

• Authors: There are 3 authors listed in the paper.md (minor point: no affiliation information for the 2nd author), however, the commit logs show some other people also contribute codes to the project. If they are not listed as authors, I think they should be referenced/acknowledged somewhere in the docs as contributors explicitly.

• Automated tests: Although the test code is included in the repo, however, the document about how to use the test code is still lacking (i.e. how to use pytest to verify the software). In addition, the tests code is not installed via pip installation approach. I think it would be better that moving the tests directory into Augmentor, and installing the core code and tests code together. And a test API (something like Augmentor.test()) may be more user-friendly.

This is my first review with JOSS, so please tell me if I am doing something incorrectly. @arfon

[arfon]

@sealhuang - many thanks for the review and your thoughtful comments ✨ . @mdbloice - over to you for your responses.

[mdbloice]

Hi @sealhuang @arfon many thanks for the review and organisation!

I will try to address each point here:

• Version: Since submitting to JOSS I made a few bug fixes, which were quite important for a few features of the Augmentor. As they were quite crucial I wanted the changes to be on the PyPI, hence the version number discrepancy. These changes were quite minor, and feature-wise there is very little difference between v0.1.5 and v0.1.8.
• Authors: Yes, author 2 does not have an academic affiliation so this was purposely left as it is. As for the contributors, is not not enough that they are listed in the Contributors tab on the repo homepage? I would guess this list will change fairly often, so I think it would be wasted effort to maintain my own list somewhere such as in the README would it not?
• Automated tests: I would prefer not to include the tests in the pip installation, however, I have now added a section to the README which describes how one can perform the tests locally by cloning the repository. I hope that this compromise is OK with you @sealhuang ?

If anything requires clarification please let me know and I will respond as quickly as I can, and may I also apologise for the lateness of my reply, I have been inundated at work of late :)

Again many thanks to both for the review and efforts,

Marcus.

[arfon]

Authors: Yes, author 2 does not have an academic affiliation so this was purposely left as it is. As for the contributors, is not not enough that they are listed in the Contributors tab on the repo homepage? I would guess this list will change fairly often, so I think it would be wasted effort to maintain my own list somewhere such as in the README would it not?

@mdbloice - in this review we faced a similar issue and the submitting author opened an issue on their repository inviting the other contributors to be authors (most declined). Regardless, it would probably be a good idea to include a short sentence in the paper somewhere acknowledging the other contributors and linking to the GitHub contributors page.

[mdbloice]

Hi @arfon certainly I can do that, and I have now updated the paper.md file to contain an acknowledgement and a link to the contributors tab of the repository.

[sealhuang]

@mdbloice Thanks for your work. Given you have described how to test the code, I think it's ok for the compromise on automated tests. So you have addressed all my concerns, and I'm ready to accept this and recommend it for publication. I think the package would be very helpful for the field. Congrats!
@arfon

[mdbloice]

Great @sealhuang , thanks a lot, I appreciate the review and your encouraging words!

[mdbloice]

By the way, @arfon is there anything I need to do or change regarding the version number mismatch?

[arfon]

By the way, @arfon is there anything I need to do or change regarding the version number mismatch?

I can fix that.

@mdbloice - At this point could you make an archive of the reviewed software in Zenodo/figshare/other service and update this thread with the DOI of the archive? I can then move forward with accepting the submission.

[mdbloice]

@whedon set 10.5281/zenodo.1041946 as archive

[whedon]

I'm sorry @mdbloice, I'm afraid I can't do that. That's something only JOSS editors are allowed to do.

[mdbloice]

Hi @arfon thanks for the info, I have done that for version 0.1.5 using Zenodo and as you can see I tried to set the archive using whedon, but was not allowed :)
Anyway, the DOI is above.
Thanks again!

[arfon]

@whedon set 10.5281/zenodo.1041946 as archive

[whedon]

OK. 10.5281/zenodo.1041946 is the archive.

[arfon]

@sealhuang - many thanks for your review ✨

@mdbloice - your paper is now accepted into JOSS and your DOI is https://doi.org/10.21105/joss.00432 ⚡️ 🚀 💥

[mdbloice]

@arfon @sealhuang that's fantastic news, many many thanks for the efforts!

[mdbloice]

@arfon - not sure if this is the right place to point this out, but one thing I just noticed is that the link to this review thread is broken on the JOSS page for the paper where it says "View review issue".

[arfon]

@arfon - not sure if this is the right place to point this out, but one thing I just noticed is that the link to this review thread is broken on the JOSS page for the paper where it says "View review issue".

Good catch! This should be fixed now.