query re: code coverage

[SignpostMarv]

Having a bit of a problem at the dayjob; have got this psalm module integrated with a plugin for tests, but getting 0% coverage despite tests passing.

Any ideas where I might go looking?

[weirdan]

Can you elaborate on what you're trying to achieve?

[SignpostMarv]

using code coverage to make sure I've got a gherkin snippet for all the test cases I've written in the plugin.

[SignpostMarv]

note for reference: psalm/plugin-phpunit uses this module but doesn't have coverage enabled so I don't have a reference point to figure out what I'm doing wrong :s

[weirdan]

Hm, ok. I don't think it would be easy, if possible at all. What this module does, roughly, is this:

1. Extract the snippet and combine it with preamble
2. Put it into a file
3. Run Psalm on that file and collect the issues it reports
4. Let you make assertions on the collected issues

Since Psalm is run as an external binary, it's not imediately clear to me how to enable code coverage collection during its run, and how to get coverage data out of it.

[weirdan]

Possibly it could be done by running Psalm like this:

PSALM_ALLOW_XDEBUG=1 php -dauto_prepend_file=start-coverage.php -dauto_append_file=stop-coverage.php $(which psalm) --threads=1

Where start-coverage.php and stop-coverage.php would contain something along the lines of https://github.com/sebastianbergmann/php-code-coverage#usage

Alternatively, a modified copy of c3 (https://github.com/Codeception/c3) could be injected into Psalm environment. As there won't be any webserver running, it would have to be changed to communicate with Codeception by some non-HTTP means.

[weirdan]

The latter approach seem technically feasible:

• It should be possible to include c3 in an auto_prepend file
• It should be possible to control it through env vars, e.g.

  HTTP_X_CODECEPTION_CODECOVERAGE=1 PSALM_ALLOW_XDEBUG=1 php -dauto_prepend_file=c3.php $(which psalm) --threads=1 ...
  REQUEST_URI=/c3/phpunit $(which psalm)
  REQUEST_URI=/c3/clear $(which psalm)
  

[SignpostMarv]

@weirdan do you think it'd be more practical to re-open vimeo/psalm#3977 so one could just use phpunit ?

[weirdan]

This module's dependents may still benefit from having code coverage, so I plan to look into this anyway.