Code Coverage For PHP Windows Extensions

How to get code coverage for PHP on Windows:

  1. You need to have a version of Microsoft Visual Studio with Code Coverage support (this means "Team Studio" or whatever name it has this month) This code coverage is NOT available in the Express versions or standard versions of Visual Studio (sorry)
  2. Compile PHP with the same VSTS version you want to use to get code coverage. In addition you'll need to add the /profile switch to the compiler. At some point this will be integrated as a switch for the PHP configure system on windows. For now you can just add it to the LD_FLAGS stuff.
  3. Instrument the dll - You'll find the code coverage tools for visual studio in "C:\Program Files\Microsoft Visual Studio 9.0\Team Tools\Performance Tools" or somewhere similiar vsinstr.exe /coverage {$dll name here} will do the instrumentation for you
  4. Start the code coverage reporter VSPerfCmd.exe -start:coverage -output:($filenamehere}.coverage
  5. Run your PHP tests that you want code coverage reported on - remember to set your PHP_TEST_EXECUTABLE and then php.exe run-test.php {$any args here} {$path to test files here}
  6. Stop the code coverage reporter VSPerfCmd.exe -shutdown
  7. Open the file in visual studio to see the code coverage results

As you can see, this could be a nice automated task.

Figuring out this path to madness took a lot of googling to find some information about running code coverage from the command line.

However the binary .coverage format isn't very useful for anything but browsing it yourself. You can choose to export as xml it in the IDE, but it's much more efficient to do this kind of thing automatically. After all you can collect the data automatically, why not get it in a useful format as well? As you seen from the above blog post there is a .NET assembly that should be able to get you the coverage data in xml format.

In order to use .NET assemblies through cscript (with Wscript.createObject) you have to first register them with regasm.exe. but attempting to register it gives you this

RegAsm error: Failed to load 'C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE\PrivateAssemblie\Microsoft.VisualStudio.Coverage.Analysis.dll' because it is not a valid .NET assembly

Must not have been compiled with the 'INTEROPERABILITY' stuff on.. useful that, so you can't use regasm on it properly and therefore can't use it with cscript or PHP... in fact you can only use it with something written in C#, VB.NET or similiar languages and compile it down. Not useful. Why this kind of coverage to xml tool wasn't included is beyond me.

So I quick wrote up a very small tool to do the conversion. It needs to have the Microsoft.VisualStudio.Coverage.Analysis.dll from your VSTS copied into the same directory wherever you decide to place the tool. Then you can invoke it from the command line

codecoverage2xml.exe $pathtocoveragefile [$optionalpathtoxmloutputfile]

The code for it is available in the download if you want to tinker with it. Notice that you'll have to use the source and recompile it if you use Visual Studio 2005 not 2008 as the Microsoft.VisualStudio.Coverage.Analysis.dll is different between the two. (you don't have to change the code, just add a reference to the right assembly dll and compile it)

Result? A nice xml page of the information you can then manipulate however you'd like. And with the exe you can completely automate the testing and coverage.


Elizabeth Smith’s Blog: Code Coverage For PHP Windows Extensions : Dragonfly Networks

[...] Smith has a new blog post today showing the steps you’ll need to get code coverage testing up and running on your [...]

2008-10-11 1:25 am

Post a Reply