Skip to content

Adding --recurse / --no-recurse option #283

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Nov 16, 2021

Conversation

mayuriesha
Copy link
Contributor

To help us get this pull request reviewed and merged quickly, please be sure to include the following items:

  • Tests (if applicable)
  • Documentation (if applicable)
  • Changelog entry
  • A full explanation here in the PR description of the work done

PR Type

What kind of change does this PR introduce?

  • Bugfix
  • Feature
  • Code style update (formatting, local variables)
  • Refactoring (no functional changes, no api changes)
  • Build related changes
  • CI related changes
  • Documentation content changes
  • Tests
  • Other

Backward Compatibility

Is this change backward compatible with the most recently released version? Does it introduce changes which might change the user experience in any way? Does it alter the API in any way?

  • Yes (backward compatible)
  • No (breaking changes)

Issue Linking

Closes #268

What's new?

  • Introducing new flag --recurse/--no-recurse to scan entire directory or just the root directory. It is defaulted to recurse to maintain the current behavior of scan-folder.

Copy link
Contributor

@rbailey-godaddy rbailey-godaddy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you don't mind a little more tweaking...

It would be more consistent with existing behavior to look at the value of global_options.recurse instead of extracting it in the caller and then passing recurse as a separate parameter. Additionally, you can then just check self.global_options.recurse within the class instead of needing to add self.recurse.

Also, the more I look at this, the more I can't see where the recurse parameter to scan_folder.main() is coming from. The crazy thing is that I play-tested it and it obviously works, so I guess I'm good with things as-is if nobody else has complaints -- I just can't understand what is happening!

@mayuriesha
Copy link
Contributor Author

I believe self.global_options are for the flags that are defined in cli.py which are tartufo command line options. Where as the recurse is an option that goes with sub_command scan-folder. So I am not sure if we can use self.global_options.recurse.

About the parameter recurse for scan_folder->main, as far as I understand the click.command is handling the arguments/options that are defined and passes it to the method definition below.
https://github.com/pallets/click/blob/5d759e0ab383136f9869cdb5e14f00c78b38a9c9/docs/options.rst

README.md Outdated
@@ -159,6 +159,7 @@ Commands:
pre-commit Scan staged changes in a pre-commit hook.
scan-local-repo Scan a repository already cloned to your local system.
scan-remote-repo Automatically clone and scan a remote git repository.
scan-folder Scan a folder.
Copy link
Contributor

@sushantmimani sushantmimani Nov 12, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit

Suggested change
scan-folder Scan a folder.
scan-folder Scan a folder on your local system.

Comment on lines 29 to 38
self.assertEqual(2, len(issues))
self.assertEqual("KQ0I97OBuPlGB9yPRxoSxnX52zE=", issues[0].matched_string)
self.assertEqual(IssueType.Entropy, issues[0].issue_type)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need a similar assertions for the new issue.

CHANGELOG.md Outdated
@@ -20,6 +20,8 @@ Features:
`--scan-filenames/--no-scan-filenames` flag which allows users to enable or disable file name scanning.
* [#254](https://github.com/godaddy/tartufo/pull/260) - Changes the default value of
`--regex/--no-regex` to True.
* [#268](https://github.com/godaddy/tartufo/issues/268) - Adds a new
`--recurse / --no-recurse` flag which allows users scan entire directory or just the root directory
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
`--recurse / --no-recurse` flag which allows users scan entire directory or just the root directory
`--recurse / --no-recurse` flag which allows users to recursively scan the entire directory or just the root directory

Copy link
Contributor

@sushantmimani sushantmimani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🦅

Copy link
Contributor

@tarkatronic tarkatronic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No hard blockers here; overall it looks great! Just a few suggestions to improve the tests with some Python-isms. 😄

Thank you!

Comment on lines 30 to 32
actual_issues = []
for issue in issues:
actual_issues.append(issue.matched_string)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a blocker, but cases like this are ideal for list comprehensions 😄

Suggested change
actual_issues = []
for issue in issues:
actual_issues.append(issue.matched_string)
actual_issues = [issue.matched_string for issue in issues]


with patch("pathlib.Path.open", side_effect=OSError()):
with self.assertRaises(click.FileError):
list(test_scanner.scan())

def test_scan_all_the_files_recursively(self):
folder_path = pathlib.Path(__file__).parent / "data/scan_folder"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All path separators should be outside the string, so that we can be absolutely certain they are OS-independent.

Suggested change
folder_path = pathlib.Path(__file__).parent / "data/scan_folder"
folder_path = pathlib.Path(__file__).parent / "data" / "scan_folder"

Comment on lines 64 to 66
actual_issues = []
for issue in issues:
actual_issues.append(issue.matched_string)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same list comprehension trick can be applied here

Suggested change
actual_issues = []
for issue in issues:
actual_issues.append(issue.matched_string)
actual_issues = [issue.matched_string for issue in issues]

)

def test_scan_only_root_level_files(self):
folder_path = pathlib.Path(__file__).parent / "data/scan_folder"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
folder_path = pathlib.Path(__file__).parent / "data/scan_folder"
folder_path = pathlib.Path(__file__).parent / "data" / "scan_folder"

Comment on lines 85 to 87
actual_issues = []
for issue in issues:
actual_issues.append(issue.matched_string)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
actual_issues = []
for issue in issues:
actual_issues.append(issue.matched_string)
actual_issues = [issue.matched_string for issue in issues]

@mayuriesha mayuriesha merged commit 2daef6e into godaddy:v3.x Nov 16, 2021
@mayuriesha mayuriesha linked an issue Nov 16, 2021 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add a --recurse / --no-recurse flag to scan-folder
4 participants