Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[World is on Fire] Have a contingency plan when web-bugs repo disappears. #3118

Closed
karlcow opened this issue Jan 2, 2020 · 9 comments
Closed
Assignees

Comments

@karlcow
Copy link
Member

karlcow commented Jan 2, 2020

There is right now an incident. The web-bugs repo is gone.

webcompat-web-bugs-repo

We need a contingency plan and tools.

  1. Missing a web page saying that the site is down currently.
  2. Having a warning as early as possible on https://twitter.com/webcompat
    Ex: https://twitter.com/webcompat/status/1212880610089299968
  3. Have a backup of everything (at least comments and issues) so we can send a static version instead of the nothingness we currently have. [meta] Old issues/comments sedimentation #2449
  4. Consolidate our accounts information

Capture d’écran 2020-01-03 à 08 51 17

@karlcow
Copy link
Member Author

karlcow commented Jan 2, 2020

As for the January 2020 incident.

  • Detected by @ksy36
  • 01:57+09:00 Reported by @miketaylr to GitHub
  • 02:22+09:00 Reported by @miketaylr to the internal mailing-list of Mozilla webcompat team
  • 07:15+09:00 Only webcompat-bot issues are gone from the repo
  • 07:27+09:00 Notice from Github that the repo has been disabled.
  • 08:33+09:00 The full repo is gone.
  • 08:39+09:00 Tweeted by @karlcow on webcompat account.

@miketaylr
Copy link
Member

We could revive some of the code from #2733 for a landing "page is down" page.

@miketaylr
Copy link
Member

I've sent some messages via LinkedIn (one to @clarkbw... hopefully GitHub @support can reverse this soon, and help us understand why it happened without any kind of heads up).

@karlcow
Copy link
Member Author

karlcow commented Jan 3, 2020

When the form can't communicate with github web-bugs.
At the end of the process for users this is what happens. Probably another issue to solve in case of non communications with GitHub.

Capture d’écran 2020-01-03 à 10 23 32

miketaylr pushed a commit that referenced this issue Jan 3, 2020
karlcow pushed a commit that referenced this issue Jan 3, 2020
Issue #3118 - Display maintenance page for /issues/new route
@karlcow
Copy link
Member Author

karlcow commented Jan 3, 2020

@miketaylr and I created an incident report with more details that we are updating once the details are coming in.

@miketaylr also created a PR #3119 to have a landing page for the route /issues/new It was deployed today on staging and production.

@miketaylr
Copy link
Member

miketaylr commented Jan 3, 2020

TL;DR of the issue: someone anonymously reported illegal content, and we failed to moderate it (😢). As a result, GitHub suspended @webcompat-bot and the webcompat/web-bugs repo.

Update: @github support un-suspended the web-bugs repo, but @webcompat-bot's reports are still hidden. I've proposed the following plan (only listing technical details, there will be some social things to fix too):

  1. Remove the option for anonymous reporting from webcompat.com today (Friday, Jan 3, 2020). This will allow GitHub authenticated users to continue to report issues. Currently all reporting is disabled; we’re serving a “maintenance.html” page: Disable anonymous reporting #3121
  2. Come up with a plan for moving towards a private-by-default anonymous reporting flow, with issues being made public once triaged. I don’t suspect this will be a ton of work (more than a few weeks), but the devil is always in the details: Create plan for private-by-default anonymous reporting flow #3124
  3. Make changes to our application to link screenshots by default, rather than display them inline: Remove inline screenshot images #3122
  4. Make changes to our application to prevent .onion URLs from being reported: Prevent .onion URLs from being reported #3123
  5. Come up with a better plan for automated methods of tagging NSFW content long-term.
  6. Once we have a private-by-default anonymous reporting workflow in place, I will be in touch to request that the issues reported by @webcompat-bot be made public again. I've asked if we can audit the remaining issues to ensure all harmful content is deleted.

This is the short-term plan. We will likely have a lot of discussions in Berlin about medium- to long-term plans about hosting choices, the GitHub platform, anonymous reporting, illegal content, etc.

@karlcow
Copy link
Member Author

karlcow commented Jan 5, 2020

@miketaylr In complement to #3118 (comment), I created Webcompat Belt On project and the Remove illegal content issue.

@karlcow
Copy link
Member Author

karlcow commented Jan 10, 2020

@miketaylr should we close this one? As we set things in mmotion in the project itself.

@miketaylr
Copy link
Member

I think it makes sense, yes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants