Onestop-client refactor

[erinreeves]

Determine how to address these issues:

• SqsHandlers.delete - should this instead take a single record not a list of records? It's only operating on the first record.
• verify smart_open fixes this: We read the entire file locally to a variable from an s3 bucket https://github.com/cedardevs/onestop-clients/blob/master/onestop-python-client/onestop/util/S3Utils.py#L261
• Is it necessary to have try/catch around code in methods that shouldn't be waiting on an asynchronous prior run method? Ex: S3Utils.upload_s3. Then a user has to know to check the run, and what put that upload call in a while loop? What if it will never pass 'cuz of the problem that the file doesn't exist?
• Should WebPublisher have onestop search methods? If we wish to have that functionality perhaps make a class more suited for it.
• Verify search_onestop in WebPublishing is respecting params. Lead to removing semi-dup unused get_granules_onestop.
• How to allow a user to use our application without credentials.
• Hope is to have in helm values. Perhaps 2 helm values, one with credentials that will be listed in gitignore and other with unconfidential information that can be checked in.
• Move any e2e type of scripts to tests/e2e directory. Perhaps pull apart into smaller chunks? Only move ...?
• CsbExtractor rename to something more generic?

Random fixes:

• Make the avro schema variables that are optional actually optional, currently not inside the S3MessageAdapter.transform method but are in the schemas.

• Any spot upload_archive is called need to check the response is not None.

• Move test_message in scripts to after where config dict is initialized.

• add try/except Client error to scripts get_uuid_metadata( and read_bytes?)

• Create a parent class, KafkaWrapper?, to hold register_client, connect (shared methods that are the same in both classes)

• WebPublisher.publish_registry shouldn't take "method" and "metadata_type" params as a string, but instead an enum. At very least needs else statement to catch if they mistype the type or used lowercase.

• Replaced prints with log statements and change any using logging to use ClientLogger.

• Change logging. to self.logger

• Move wait_for method in WebPublisher unit test file to tests.utils

• Removed unused methods (collection_id_map)

• Rename method def get_csv_s3(self, boto_client, bucket, key):
  https://github.com/cedardevs/onestop-clients/blob/master/onestop-python-client/onestop/util/S3Utils.py#L99
  so directly below we have read_bytes_s3 ...
  so maybe we have get_s3_bytes
  get_s3_file
  read_bytes_s3 -> get_s3_bytes
  get_csv_s3 -> get_s3_file

• In launch_e2e.py uncomment out commented out postgres chunk. requires having a container running locally that was being problematic.

• Upgrade pyyaml to at least 5.4 (vulnerability https://github.com/cedardevs/onestop-clients/security/dependabot/onestop-python-client/requirements.txt/PyYAML/open)