Support for meta groups for checksums

[andrewpatto]

As the number of checksums increases - it may be painful to specify them all by hand on each CLI invoke.

It might be useful to have some metagroups.

So (without any thought).. aws-common might mean aws-etag-8mib, aws-etag-16mib, aws-etag-5mib.

cloud-common might be all the checksums that are useful across GCP and AWS and Azure.
sha might mean all the sha family.
default might be a sensible set that we would recommend.

But this ticket is more about putting that mechanism in place - rather than deciding on all the groups (just put in a couple of examples for this ticket).

Maybe --checksum-group for the CLI? (happy for other thoughts)

[mmalenic]

Could also just use the same --checksum mechanism for this, as long as there is no overlap in values. E.g. --checksum sha,aws could be equivalent to --checksum sha1,sha256,aws-etag-8,aws-etag-5,aws-etag-16.

[mmalenic]

One general thing to consider is how many checksums to actually support - there's quite a few. A lot of them have the same/similar interfaces so could all be supported with a few lines of code if we wanted. E.g. see https://github.com/RustCrypto/hashes.

[andrewpatto]

Yeah it'll be trivial to add checksums.. so limiting the scope is probably the harder job. I think if we view the main job is to support cloud object operations - we should be starting with checksums that are used natively by object stores.

[andrewpatto]

See docs/README - where I have started documenting the default AWS part sizes for various tools. I think we need an 'aws-common' (need better name) that adds in checksum/part sizes that cover those common sizes