Skip to content

Modifications needed for the migration to cozy #899

Open
@shepilov

Description

@shepilov

Results of the migration:

  • List of the migration by user:
    For each migration, we need to have:
    • Number of the migrated files
    • Total number of files for the user
    • Number of the migrated files
    • Number of errors
      For every error, we need to have the type of error(if we can determine)
      • stacktrace
      • input data that caused an error (user, doc id, file id, etc)

List of the errors that we know and need to deal with:

  • DNS resolution of the user's hostname
  • Unknown errors with the empty response from the cozy stack
  • Files don't exist in S3
  • Metadata is corrupted(undefined 'id' field)

To run several migrations in parallel, we need to split the user list into batches.
To be simple and not use a distributed lock on the user, we can continue to consume users from command params, but we need to pass a lot of emails as well as it's not so comfortable to do with params, so let's also introduce env variable for the users list for the migrations.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions