Enable code coverage during testing

[sharwell]

Currently the test script does not run the tests with code coverage enabled. Several steps are required to enable code coverage collection:

Ensure Windows PDBs are available for OpenCover

1. Extract embedded PDBs to portable PDBs (where applicable)
2. Convert portable PDBs to Windows PDBs
3. Delete the original portable PDBs
4. Copy the dll or exe to the directory containing symbols

The following sequence applies this to all assemblies in a folder (e.g. the folder containing a unit test output):
https://github.com/DotNetAnalyzers/StyleCopAnalyzers/blob/c7254023938e4e6cc5bb03caf6da0ba814845203/build/opencover-report.ps1#L54-L95

Wrap the unit test call with OpenCover.Console

https://github.com/DotNetAnalyzers/StyleCopAnalyzers/blob/c7254023938e4e6cc5bb03caf6da0ba814845203/build/opencover-report.ps1#L111-L121

Prepare the results

For local builds, run ReportGenerator on the output:
https://github.com/DotNetAnalyzers/StyleCopAnalyzers/blob/c7254023938e4e6cc5bb03caf6da0ba814845203/build/opencover-report.ps1#L147-L148

For CI builds, send the collection to codecov.io:
https://github.com/DotNetAnalyzers/StyleCopAnalyzers/blob/c7254023938e4e6cc5bb03caf6da0ba814845203/appveyor.yml#L14

[sharwell]

@tmat This is my first priority this week. At this point I have a very good understanding of how the pieces fit together (portable/embedded PDB + xunit console + OpenCover console + reporting), but very little understanding of the way roslyn-tools invokes the xunit console for testing purposes. I'm hoping you can help me incorporate each of the necessary steps above into the testing harness we use.

[tmat]

Why is this needed now? If anything it would be better to invest our time in helping OpenCover to support Portable PDBs.

[sharwell]

@tmat That could be beneficial as well. I found in the proof of concept that the PDB conversion step wasn't quite as complex as mentioned above because the built uses embedded PDB for all binaries.

[tmat]

I'd love to get automatic code coverage reports. But I'm not sure if it's worth the complexity at this point. The effort spent on implementing a workaround in our repos won't help our customers. Effort spent on helping OpenCover to adopt Portable PDBs faster will help everyone.

[sharwell]

@tmat fortunately, of the issues involved with getting this working, the PDBs seem like a solved problem. The toolset already included everything to make it work seamlessly. We can take a look at the couple remaining small items tomorrow. 😁

[tmat]

I don't think I'll have time this week to look into this.

[Evangelink]

I think the ticket is no longer relevant as we can see coverage on PRs (ping @sharwell ).