Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new feature Delete Artifacts with pattern #84

Open
wants to merge 111 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
111 commits
Select commit Hold shift + click to select a range
7fb64bb
Add new feature Delete Artifacts matching patterns
Oct 10, 2023
a93e9ea
Replace 'touch' with 'writeFile' and 'mkdir' with 'dir'
Jul 24, 2023
3bcaa86
Add a test 'testRemoveLastSlash_NoSlash()' to check archiveRootPath f…
Jul 24, 2023
acbbd07
Refactor: Extract method from private static final class to public st…
Jul 24, 2023
ac83d8e
Remove unused else if statement for directory handling
Jul 24, 2023
55dd7f7
Add unit tests for getInclude & getExclude methods for code coverage
Jul 25, 2023
9396187
Add annotation to exclude checkRoles method from code coverage
Jul 25, 2023
54a777f
the logic for deleting parent directory has been moved to a separate …
Jul 25, 2023
ce8d152
Extract method from private static final class to public static
Jul 25, 2023
5cafa92
Test: Add one unit test for 'isDirEmpty' method to improve code coverage
Jul 25, 2023
3eb31f6
Remove commented out code
Jul 25, 2023
4910b77
Add annotation to exclude getInclude&Exclude, setInclude&Exclude meth…
Jul 25, 2023
0ef4dc2
Fixed 'Assignment Branch Condition too high' issue by refactoring
Jul 27, 2023
e88f034
Extract two lines of code into deleteFileAtPath() method
Jul 27, 2023
b2e7791
Extracted the logic that decides whether to delete a directory into a…
Jul 27, 2023
c55c64a
Add a test for isDirEmpty method for code coverage
Jul 27, 2023
4d22638
Extract code block into a separate method called 'deleteFileOrLogError'
Jul 28, 2023
32a06f9
Remove '@Generated' annotation
Jul 31, 2023
32b604c
Add unit tests for getInclude and getExclude methods for code coverage
Jul 31, 2023
96ae7c0
Minor correction: Add a comment and IOExeption
Jul 31, 2023
28c1cff
Add tests for isDirEmpty method and checkRoles mehtod
Aug 3, 2023
011f4bd
update access modifier
Aug 3, 2023
dbac7b9
Add unit test for deleteParentDirectories method
Aug 4, 2023
ae84063
Formatting the test code
Aug 4, 2023
624e895
Fixed code defect
Aug 4, 2023
aa3a550
Refactoring the code to make it cleaner and easier to understand
Aug 4, 2023
dbbc5a0
Remove '// TODO Auto-generated method stub'
Aug 8, 2023
45ce8f8
Refactor deleteParentDirectories method for easy to write unit tests
Aug 8, 2023
69f3fbc
Add unit tests for shouldDelete method and deleteDirecotry method
Aug 8, 2023
ec2cf6e
Refactor: merge two unit tests
Aug 8, 2023
d291e2d
Move deleteFileOrLogError method inside Delete class for writing unit…
Aug 8, 2023
4833d65
Added tests for deleteFileOrLogError method
Aug 9, 2023
76cde80
Add a test for shouldDeleteDirectory method for code coverage
Aug 9, 2023
95bef37
Fix uncvovered line for code coverage
Aug 9, 2023
8835036
Fixed partially covered lines for code coverage
Aug 10, 2023
3f158e4
Covered one missing branch issue for code coverage
Aug 10, 2023
82e63cb
Fixing partialy covered statement
Aug 10, 2023
7d6d452
Create a parent directory that is not equal to archiveRootPath
Aug 10, 2023
e04ddab
Refacto shouldDeleteDirectory method for code coverage
Aug 10, 2023
fd94659
Rewrite a test for shouldDeleteDirectory method for code coverage
Aug 10, 2023
6d45cbd
Refactor: Extract the common code lines and organize the test methods…
Aug 10, 2023
0116e54
remove .vscode/settings.json
Aug 29, 2023
8179064
remove 'pom copy.xml'
Aug 29, 2023
79c6ae0
Add .vscode/settings.json to .gitignore list
Aug 30, 2023
5ebf960
update .gitignore list
Aug 31, 2023
69acfd9
Refactor: Rename 'LOGGER' to 'LOG' for naming convension
Sep 28, 2023
3d0a43f
Removed 'LOGGER' and added new line at the end of file for consistency
Sep 28, 2023
0eb7ade
Removed 'serialVersionUID'
Sep 28, 2023
5ca1edf
Refactor: Rename method from 'isInvalidDirectory' to 'isValidDirector…
Sep 29, 2023
cb5d393
Test: Update assertion to test 'isValidDirecotry' method
Sep 29, 2023
64f8c5f
Refactor: Rename method 'isDirEmpty' to 'isDirectoryEmpty' for improv…
Sep 29, 2023
59dc567
Refactor: update the test for 'isDirectoryEmpty' after renaming
Sep 29, 2023
6dea4b0
Fix: Add space between ')' and '{' for consistent code style
Sep 29, 2023
b634e5f
Refactor: Reorganize the constructor, renamed 'vFile', added a commen…
Sep 29, 2023
36841e3
Update: updated the 'removeLastSeparator' method to ensure compatibil…
Sep 29, 2023
6a745a1
Refactor: Rename 'vFile' to 'file' in 'invoke' method
Sep 29, 2023
ddccd42
Refactor: change access modifier to package-private for some methods
Sep 29, 2023
1634138
Test: Added a test for 'removeLastSlash' method for code coverage
Sep 29, 2023
d9d9dd6
Test: Add a test for 'removeLastSlash' method with backslash for code…
Sep 29, 2023
419199f
Refactor: Reorganize the 'constructor' to match the structure of othe…
Sep 29, 2023
b81478c
Refactor: Rename 'removeLastSlash' method to 'removeLastFileSeparator…
Oct 2, 2023
6c770af
Refator: Removed 'removeLastFileSeparator' method and replaced 'archi…
Oct 3, 2023
a835a0c
Removed 'checkRoles' method
Oct 3, 2023
ddb53e2
Removed unit tests associated with the 'removeLastFileSeparator' method
Oct 3, 2023
d806f3e
Removed unit test associated with the 'checkRoles' method
Oct 3, 2023
f569791
Replaced 'archiveRootPath' with 'archiveRootFile' of type File in uni…
Oct 3, 2023
3ff8e6e
Refactor: Removed 'incldue' and 'exclude' fields and their associated…
Oct 3, 2023
3bedd1a
Fix: Corrected typo 'archiveRooFile' to 'archiveRootFile' and updated…
Oct 4, 2023
f553893
Added back 'checkRoles' method with documentation
Oct 4, 2023
237fbf3
Added back the unit test for 'checkRoles' method
Oct 4, 2023
470eb42
Removed redundant test methods
Oct 4, 2023
ebc8038
Refactor: renamed 'include/exclude' to 'includePatterns/excludePatterns'
Oct 4, 2023
8349e7c
Renamed 'include/exclude' to 'includePatterns/excludePatterns' in con…
Oct 4, 2023
b63907f
Renamed 'include/exclude' to 'includePatterns/excludePatterns' in jen…
Oct 4, 2023
4eee00c
Renamed 'DeleteArtifactsWithPattern' to 'DeleteArtifactsMatchingPatte…
Oct 4, 2023
6343df5
Renamed all 'DeleteArtifactsWithPattern' to 'DeleteArtifactsMatchingP…
Oct 4, 2023
0be8337
Refacotr: Removed redundent code and organized the import statements
Oct 5, 2023
dfd1641
Refactor: Reorganize Import Statements
Oct 5, 2023
47d3685
Replaced 'delete-artifacts-with-pattern-configuration.png' with 'dele…
Oct 5, 2023
e6f1c1e
Updated 'README' file in response to attribute name changes
Oct 5, 2023
3c6c139
Refactor: Avoid using intermediate assignment to fix 'Assignment Bran…
Oct 5, 2023
f20872c
Refactor: Simplify 'isDirectoryEmpty' method and updated unit tests
Oct 9, 2023
e87e078
Refactor: simplify directory comparison logic
Oct 17, 2023
c639f99
A new serialVersionUID is auto-generated
Oct 17, 2023
3db4cad
Refactor error handling in 'deleteDirectory' method
Oct 19, 2023
b416844
Added/updated unit tests for 'deleteDirectory' method for code coverage
Oct 19, 2023
e471acb
Addressed various warnings highlighted by IntelliJ IDEA
Oct 19, 2023
1cbee8f
Refactor: Inline 'inValidDirectory' method and remove unnecessary tests
Oct 31, 2023
d47dcf8
Refactor: Inline 'shouldDelete' and 'shouldDeleteDirectory' methods a…
Nov 2, 2023
2658aa7
docs: Update comment for artifacts deletion matching patterns
Nov 2, 2023
a9b7d7e
chore: Add newline at the end of file
Nov 2, 2023
4cfe033
Test: Add coverage for null directory scenario
Nov 3, 2023
353fed8
Update src/main/java/pl/damianszczepanik/jenkins/buildhistorymanager/…
damianszczepanik Nov 4, 2023
45c5610
Added new line at the end of file
Nov 5, 2023
3b47a72
Suggestion applied: Rename 'Delete' methed to 'DeleteFileCallable'
Nov 6, 2023
41cef69
Renamed 'deleteInstance' to 'deleteFileCallableInstance'
Nov 6, 2023
42d0cf4
Refactor: Removed 'Set<File> directories' and the loop, simplified th…
Nov 6, 2023
4708128
Refactor: Delete the file first then handle empty directories; Rename…
Nov 6, 2023
ec2d1dd
Removed serialVersionUID as it is not needed in the static class
Nov 14, 2023
d32bfac
Refactor: Renamed 'archiveRootFile' to 'archiveRootDirectory'
Nov 15, 2023
ac49b59
Refactor: Renamed 'archiveRootDir' to 'archiveRootDirectory'
Nov 15, 2023
39ad85b
Refactor: Renamed 'vRoot' to 'virtualRoot'
Nov 15, 2023
3b20420
Refactor: Moved 'isDirectoryEmpty' method into 'DeleteFileCallable' c…
Nov 15, 2023
e03a4c2
Added 'serialVersionUID' since 'FileCallable' class is connected to S…
Nov 15, 2023
5cf758f
Refactor: Renamed 'hasValidParent' method to 'isArchiveRootDirectory'…
Nov 15, 2023
6d8605b
Refactor: Removed 'directory != null' condition from while loop and u…
Nov 16, 2023
7379491
Added comments for 'deleteEmptyDirectoryAndParent' method for better …
Nov 16, 2023
02c810c
Refactor 'isArchiveRootDirectory' method for improved readability and…
Nov 16, 2023
0628225
Remove redundant file deletion logging
Dec 5, 2023
9798873
Added a comment for 'System.setProperty()'
Dec 5, 2023
9de4746
Refactor: Rename the method names for asserting a file exists or not
Dec 5, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
/.idea
**/*.iml
target/
work/
.vscode/
39 changes: 39 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -215,6 +215,45 @@ pipeline {
}
```

#### Delete build artifacts matching patterns
The following configuration has a single rule that utilizes the `Match every build` condition and deletes specific build artifacts based on Ant-style glob patterns defined in the `Include Patterns/Exclude Patterns`.

Here's an example using the include pattern `"**"`, exclude pattern `"**/*.log"` to delete all files except log files:

```groovy
properties(
[
buildDiscarder(BuildHistoryManager(
[
[
actions: [DeleteArtifactsMatchingPatterns(excludePatterns: '**/*.log', includePatterns: '**')],
conditions: [MatchEveryBuild()]
]
]
)
)
]
)

node {

sh '''
rm -rf *
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is not clear to me what is the intention of this code snipped in readme file

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a shell command that removes all files and directories in the current directory.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know what it does but I don't understand why did you add this code here? What is the goal? Are you expecting that someone will copy&paste that code into his pipeline ?

touch test.xml
touch testLog.log
touch testTxt.txt
mkdir -p testFolder
touch testFolder/test1.xml
touch testFolder/testLog1.log
touch testFolder/testTxt1.txt
'''
archiveArtifacts artifacts: '**', followSymlinks: false
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

... I guess this is valuable

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is the changes in wiki, because Github does not allow to send pull request to the wiki. https://github.com/nurgul212/build-history-manager-plugin-wiki/compare/artifacts-pattern?expand=1

In wiki, Delete-artifacts-matching-patterns-action.md , there is an example to present build history before and after running the configuration with this part.

}
```

#### Configuration for delete artifacts matching include and exclude patterns
![feature overview page](./.README/delete-artifacts-matching-patterns-configuration.png)

## Wiki
Please refer to the [Wiki](https://github.com/jenkinsci/build-history-manager-plugin/wiki)
for more detailed information.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
package pl.damianszczepanik.jenkins.buildhistorymanager.descriptors.actions;

import hudson.Extension;
import hudson.model.Descriptor;
import org.jenkinsci.Symbol;
import pl.damianszczepanik.jenkins.buildhistorymanager.model.actions.Action;
import pl.damianszczepanik.jenkins.buildhistorymanager.model.actions.DeleteArtifactsMatchingPatternsAction;

/**
* Descriptor implementation needed to render UI for {@link DeleteArtifactsMatchingPatternsAction}.
*/
@Extension
@Symbol("DeleteArtifactsMatchingPatterns")
public class DeleteArtifactsMatchingPatternsActionDescriptor extends Descriptor<Action>{

public DeleteArtifactsMatchingPatternsActionDescriptor() {
super(DeleteArtifactsMatchingPatternsAction.class);
}

@Override
public String getDisplayName() {
return "Delete artifacts matching patterns";
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
package pl.damianszczepanik.jenkins.buildhistorymanager.model.actions;

import java.io.File;
import java.io.IOException;
import java.nio.file.DirectoryStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.Collection;
import java.util.logging.Level;
import java.util.logging.Logger;

import org.jenkinsci.remoting.RoleChecker;
import org.kohsuke.stapler.DataBoundConstructor;
import org.kohsuke.stapler.DataBoundSetter;

import hudson.FilePath;
import hudson.FilePath.FileCallable;
import hudson.Util;
import hudson.model.Run;
import hudson.remoting.VirtualChannel;
import jenkins.util.VirtualFile;

/**
* Deletes the artifacts matching patterns.
*/
public class DeleteArtifactsMatchingPatternsAction extends Action {
private static final Logger LOG = Logger.getLogger(DeleteArtifactsMatchingPatternsAction.class.getName());

private String includePatterns;
private String excludePatterns;

@DataBoundConstructor
public DeleteArtifactsMatchingPatternsAction() {
// Jenkins stapler requires to have public constructor with @DataBoundConstructor
}

public String getIncludePatterns() {
return includePatterns;
}

@DataBoundSetter
public void setIncludePatterns(String includePatterns) {
this.includePatterns = includePatterns;
}

public String getExcludePatterns() {
return excludePatterns;
}

@DataBoundSetter
public void setExcludePatterns(String excludePatterns) {
this.excludePatterns = excludePatterns;
}

// if 'file' is on a different node, this FileCallable will be transferred to that node and executed there.
public static final class DeleteFileCallable implements FileCallable<Void> {
private static final long serialVersionUID = 2469669731898204125L;
private final File archiveRootDirectory;

public DeleteFileCallable(File archiveRootDirectory) {
this.archiveRootDirectory = archiveRootDirectory;
}

@Override
public Void invoke(File file, VirtualChannel channel) throws IOException {
deleteFileOrLogError(file);
return null;
}

/**
* Deletes the specified directory if it is empty and its parent is the archive root directory.
* Additionally, deletes the parent directories recursively.
*
* @param directory The directory to be deleted.
* @throws IOException If an I/O error occurs.
*/
void deleteEmptyDirectoryAndParent(File directory) throws IOException {
if (isDirectoryEmpty(directory.toPath()) && isArchiveRootDirectory(directory.getParentFile())) {
Util.deleteFile(directory);
deleteParentDirectories(directory.getParentFile());
}
}

boolean isDirectoryEmpty(Path path) throws IOException {
try (DirectoryStream<Path> directoryStream = Files.newDirectoryStream(path)) {
return !directoryStream.iterator().hasNext();
}
}

boolean isArchiveRootDirectory(File directory) {
return this.archiveRootDirectory.equals(directory);
}

void deleteParentDirectories(File directory) {
while (!isArchiveRootDirectory(directory)) {
deleteDirectory(directory);
directory = directory.getParentFile();
}
}

void deleteDirectory(File directory) {
boolean deleteSuccess = directory.delete();
if(!deleteSuccess) {
LOG.log(Level.FINE, "Deletion of directory failed: " + directory.getAbsolutePath());
}
}

void deleteFileOrLogError(File file) throws IOException {
if (file.isFile()) {
Util.deleteFile(file);
deleteEmptyDirectoryAndParent(file.getParentFile());
return;
}
LOG.log(Level.FINE, file + " is neither a directory nor a regular file.");
}

/**
* Overrides the 'checkRoles' method to fulfill the requirement of implementing RoleSensitive.
*
* @param checker The RoleChecker used for access control and security purposes.
* @throws SecurityException If there are security-related issues.
*/
@Override
public void checkRoles(RoleChecker checker) throws SecurityException {
//Nothing is being done here even though a call to RoleChecker#check(…​) is expected.
}
}

@Override
public void perform(Run<?, ?> run) throws IOException, InterruptedException {
VirtualFile virtualRoot = run.getArtifactManager().root();
Collection<String> files = virtualRoot.list(includePatterns, excludePatterns, false);
LOG.log(Level.FINE, "Include Pattern Files: " + files);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did you check in logs what does it print? probably not what you expected :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ping


for (String path : files) {
deleteFileAtPath(virtualRoot, path);
}
}

public void deleteFileAtPath(VirtualFile virtualRoot, String path) throws IOException, InterruptedException {
VirtualFile virtualFile = virtualRoot.child(path);
File file = new File(virtualFile.toURI().getPath());
FilePath filePath = new FilePath(file);
DeleteFileCallable deleteFileCallableInstance = new DeleteFileCallable(new File(virtualRoot.toURI()));
filePath.act(deleteFileCallableInstance);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
<?jelly escape-by-default='true'?>
<j:jelly xmlns:j="jelly:core" xmlns:f="/lib/form">
<f:entry title="Patterns to include" field="includePatterns">
<f:textbox />
</f:entry>
<f:entry title="Patterns to exclude" field="excludePatterns">
<f:textbox />
</f:entry>
</j:jelly>
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
<p><b>Description</b></p>
<p>Deletes build artifacts that match the Ant-style glob patterns provided.</p>
<p><b>Include Patterns</b></p>
<p>The include pattern is a comma-separated list of Ant-style globs that follows a specific format.
The path separator for this format is the forward slash (/) and an empty string is used in the include pattern to indicate that there are no matches.
If the goal is to match everything except for certain files (defined as "Exclude Patterns"), then the recommended approach is to use the value <code>**</code> here and call out exclusions below.</p>
<p><b>Exclude Patterns</b></p>
<p>The exclude pattern is an optional input that should be in the same format as the include pattern.</p>
<p><b>Use cases</b></p>
<p>To delete artifacts with a specific file extension, for example use the include pattern "**/*.log" to delete all the log files.</p>
<p>To keep artifacts with a specific file extension, for example use the exclude pattern "**/*.log" to delete everything except the log files.</p>

<p><b>Warning!</b></p>
<p>It is important to note that this feature should be used with caution as it can be permanently
delete artifacts that may be important for later reference or analysis. It is recommended that users
carefully review the pattern and build history before executing the deletion.</p>
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
package pl.damianszczepanik.jenkins.buildhistorymanager.descriptors.actions;

import static org.assertj.core.api.Assertions.assertThat;

import hudson.model.AbstractDescribableImpl;
import hudson.model.Descriptor;
import org.jenkinsci.plugins.structs.SymbolLookup;
import org.junit.Rule;
import org.junit.Test;
import org.jvnet.hudson.test.JenkinsRule;

public class DeleteArtifactsMatchingPatternsActionDescriptorTest {

@Rule
public JenkinsRule j = new JenkinsRule();

@Test
public void getDisplayName_ReturnsDescriptorName() {

// given
Descriptor descriptor = SymbolLookup.get().findDescriptor(AbstractDescribableImpl.class, "DeleteArtifactsMatchingPatterns");

// when
String displayName = descriptor.getDisplayName();

// then
assertThat(displayName).isEqualTo("Delete artifacts matching patterns");
}
}
Loading