Update RTS For DSpace 8.0

[nwoodward]

This PR is a work in progress to update the Replication Task Suite so that it's compatible with DSpace 8.0.

[nwoodward]

Added ability to specify content store in the configuration rather than only using the primary. Fixes #73.

[nwoodward]

Now using the DSpace timestamp when creating BagIt bags so that MD5 checksums of identical bags from DSpace objects will match. Fixes #70.

This PR is now dependent on duraspace/bagit-support#51 in the bagit-support library.

[nwoodward]

Added catch for exception during initialization. Should fix #75.

[amgciadev]

@nwoodward happy to test this PR if helpful. I won't be able to start testing until the New Year, when we start work on DSpace 8.1 in out dev instance.

[nwoodward]

@amgciadev Thank you! I'm going to come back to this PR and check on any remaining issues next month once DSpace 8.1 is out. I think that would be a good time to create a new release.

[amgciadev]

@nwoodward now that DSpace 8.1 is out we've started work on upgrading our dev servers. Just let me know when this PR is ready so we can help testing it.

[nwoodward]

Hi @amgciadev. I would appreciate it if you could give this PR a test. The vast majority of it is formatting and dependencies, so it's mostly testing that the functionality remains the same in DSpace 8.1. Unfortunately, I had to remove the refactoring to set the last modified time on BagIt packages, which would have resolved the checksum mismatches. I'm not sure when the severe bug in that library will get fixed. But this PR will at least allow us to continue using the RTS with DSpace 8.

[amgciadev]

Hi @nwoodward, thanks so much for all the work on this! As we use this toolkit in our prod environments it was a bit of a blocker for our upgrade to 8.1! I'll start testing today and let you know how it goes.

PS It's a pity that the checksum mismatches fix cannot be in yet as you are waiting on the other library dependency - hopefully they review your PR soon!

Ta!

[amgciadev]

Hi @nwoodward, I have now deployed and tested this PR and I can report that all the operations I've performed have run successfully and as expected. Here is a summary of what I've tested via curation tasks:

1. Transmit all AIPs
2. Check that AIPs are present (I've also deleted one and then checked is was missing)
3. Estimate size
4. Read odometer

[nwoodward]

Hi @amgciadev. Thank you for your help! I'm glad the testing is going well so far. I'm going to take a look at DSpace/DSpace#10499.

[amgciadev]

Hi @amgciadev. Thank you for your help! I'm glad the testing is going well so far. I'm going to take a look at DSpace/DSpace#10499.

I have a small fix within curation.java (I believe the only task affected is restoresinglefromaip):

// Skip this check if the curation task being performed is restoresinglefromaip
                // This task is part of the RTS (dspace-replicate) to restore a missing object from AIP
                // Relevant class: org.dspace.ctask.replicate.METSRestoreFromAIP
                if (dso == null && !this.task.equalsIgnoreCase("restoresinglefromaip")) {
                    super.handler.logError("Id must be specified: a valid dso handle or 'all'; " + this.id + " could " +
                                           "not be resolved to valid dso handle");
                    throw new IllegalArgumentException(
                        "Id must be specified: a valid dso handle or 'all'; " + this.id + " could " +
                        "not be resolved to valid dso handle");
                }

But please do have a look as you may have a more elegant idea

[amgciadev]

@nwoodward I can also confirm that Automatic Sync Changes (via Queue) and running the replication q also work as expected, so I am a +1 for this PR

[nwoodward]

@amgciadev Thank you for testing! I'll confer with Tim about cutting a release and updating the Wiki.

[amgciadev]

@nwoodward @tdonohue do you know when the release will be made available?

[nwoodward]

Hi @amgciadev. Please bear with me as I figure out the Sonatype publishing issues with my signing key. I'll try to get this pushed out as soon as possible.