Add code coverage

[djusv]

I would love to see the ability to add code coverage to these views.

[JakeWharton]

How would you envision it looking? Wouldn't the code coverage be almost exactly the same for all devices?

[djusv]

You're correct if all tests are a successful the coverage will be the same, but I'm interested in what if everything went OK how much did i cover with my tests?

Maybe something like graphs showing off in percentages which methods from which classes have been hit or not?

[ducrohet]

Couldn't code coverage be different if there are tests that are specific to different form factors (phone vs. tablet)?

[RichardGuion]

In our case the code coverage would be different for phone vs tablet.

[RichardGuion]

Does using the Spoon test runner mean that we cannot use a code coverage tool? From other examples I've seen, it seems if you want to get code coverage from Instrumentation tests running on an Android device, you have to use com.jayway.maven.plugins.android.generation2 to run the tests and the pull the coverage files and generate then report.

[JakeWharton]

Spoon doesn't do anything differently than any other instrumentation test runner. We simply end up invoking am instrument on the device through 'ddmlib'.

I'm happy to pull whatever reports are generated into the output folder for later analysis by other tools. That seems very useful at the least.

[RichardGuion]

I see what you are saying about invoking that command. My question was more directed to the the Spoon plugin for Maven.

I was able to get code coverage working with android-maven-plugin (see below). I specified arguments for coverage, createReport, and coverageFile -- these all get passed as parameters to am instrument. You would be planning on adding similar paramters to spoon-maven-plugin to accomplish the same thing? Basically turn on coverage through am, tell it where to put the ec file, then pull it off and stick it somewhere on my hard drive when it is done.

                <plugin>
                    <groupId>com.jayway.maven.plugins.android.generation2</groupId>
                    <artifactId>android-maven-plugin</artifactId>
                    <configuration>
                        <test>
                            <!-- Run test with flag "-w coverage true" this is need for generate coverage.ec file, result file-->
                            <coverage>true</coverage>
                            <createReport>true</createReport>
                            <coverageFile>/mnt/sdcard/tmp/com.mycompany/files/coverage.ec</coverageFile>
                        </test>
                    </configuration>
                    <extensions>true</extensions>
                    <!-- need for pull coverage.ec file and move to tested project-->
                    <executions>
                        <execution>
                            <id>pull-coverage</id>
                            <phase>post-integration-test</phase>
                            <goals>
                                <goal>pull</goal>
                            </goals>
                            <configuration>
                                <pullSource>/data/data/com.mycompany/files/coverage.ec</pullSource>
                                <pullDestination>${project.basedir}/target/emma/coverage.ec</pullDestination>
                            </configuration>
                        </execution>
                    </executions>

[JakeWharton]

Yeah I think we can definitely support that. Like I said I'm not sure an actual code coverage display itself belongs in the Spoon output but we definitely should be supporting the generation of the files for external tools.

[JakeWharton]

I have this working on a branch. The current problem is that the file generated on the device has stupid permissions so you can't just pull it off. There's tricks around this, but I was too lazy to do them now. I'll hopefully circle back in the next week or two and get this finalized.

[RichardGuion]

I ran into that problem with emma regarding the permissions on the device and not being able to pull it off the folder with restricted access. I solved it by changing the location where the .ec file is generated:

                    <configuration>
                        <test>
                            <!-- Run test with flag "-w coverage true" this is need for generate coverage.ec file, result file-->
                            <coverage>true</coverage>
                            <createReport>true</createReport>
                            <coverageFile>/mnt/sdcard/tmp/com.mycompany/files/coverage.ec</coverageFile>
                        </test>
                    </configuration>

and then later in the execution, the pull source matches this location to get the ec file, which works because this folder isn't restricted:

                        <execution>
                            <id>pull-coverage</id>
                            <phase>post-integration-test</phase>
                            <goals>
                                <goal>pull</goal>
                            </goals>
                            <configuration>
                                <pullSource>/mnt/sdcard/tmp/com.mycompany/files/coverage.ec</pullSource>
                                <pullDestination>${project.basedir}/target/emma/coverage.ec</pullDestination>
                            </configuration>
                        </execution>

[JakeWharton]

Not all of our devices have an SD card and I don't want to require it. It's not that hard to work around, just unfortunate.

[JDFind]

This seems fixed way back in 44c52c9.
Is this issue still necessary?