Everything

php

Test coverage

php.internals

Vinicius Dias

1 year ago

Hello, folks. If this is not the place to ask, I'm sorry.

I would like to know how or where I can find the code coverage for
php-src to see if there is any uncovered code I might be able to
contribute with new tests.

Since gcov.php.net doesn't exist anymore, I was not able to find
anything related to it anywhere. I even tried generating a coverage
report locally with `lcov --capture --directory . --output-file
coverage.info` after running `make test`, but I got no luck with it.

Thank you in advance for your help.

hakre

1 year ago

On Thursday 07 August 2025 17:20:03 (+02:00), Vinicius Dias wrote:

> Hello, folks. If this is not the place to ask, I'm sorry.
>
> I would like to know how or where I can find the code coverage for
> php-src to see if there is any uncovered code I might be able to
> contribute with new tests.
>
> Since gcov.php.net doesn't exist anymore, I was not able to find
> anything related to it anywhere. I even tried generating a coverage
> report locally with `lcov --capture --directory . --output-file
> coverage.info` after running `make test`, but I got no luck with it.
>
> Thank you in advance for your help.
>

I don't know, but I found the .github folder quite well organized when I'm
exploring in such directions and perhaps you can gather something in there
lcov related.

Otherwise a quick server over the PHP repositories on Github is also a good
starting point if some documentation / scripts etc. is in a separate
repository.

Let me know if you spot something, I'm also interested in the testing
documentation and the phpqa docs recently went into php-src (docs) in
April, this sounds as if this could belong together.

Best

-- hakre

Ilija Tovilo

1 year ago

Hi Vinicius

On Thu, Aug 7, 2025 at 5:20 PM Vinicius Dias <carlosv775@gmail.com> wrote:

>
> I would like to know how or where I can find the code coverage for
> php-src to see if there is any uncovered code I might be able to
> contribute with new tests.

It lives here:

https://app.codecov.io/github/php/php-src

However, it seems it's currently borked. The nightly job succeeded, so
I'm not immediately sure what's wrong. I'll put it on my list, unless
somebody beats me to it.

Ilija

Thomas Krüger

1 year ago

Hi Vinicius,

as Ilija mentioned you can see here an overall view on the current 55%
line coverage in PHP. I guess its "line coverage" (a less strong code
coverage type in comparison to types like "branch coverage", "MC/DC" or
wonderful "path coverage"):
https://app.codecov.io/github/php/php-src?search=&trend=all%20time

More important than high line coverage would be to collect the
requirements of features in a project, which should be tested plus
negative variants of the reuirements. So, high code coverage is fine,
but high test coverage would be more helpful. :)

Here some sloppy explanations:

Code coverage:
Percentage of how many lines of code are covered by automated tests.

Test coverage:
Percentage of how many defined test cases covered real requirements.

Requirement coverage:
Percentage of how many reuirements are well defined and how many are
missing.

Be ware of:
100% line coverage does not mean anything other that 100% of the lines
of a project a "covered" by "any" automated tests, Your project still
includes bugs, missing features, security problems and crashes. So
additional coverage types like input value coverage, path coverage,
MC/DC and other helps to have more confidents in the correctness of the
project. :)

Best regards,
Thomas

Am 07.08.2025 17:44 schrieb Ilija Tovilo:

Ilija Tovilo

1 year ago

Hi Vinicius

On Thu, Aug 7, 2025 at 5:44 PM Ilija Tovilo <tovilo.ilija@gmail.com> wrote:

>
> Hi Vinicius
>
> On Thu, Aug 7, 2025 at 5:20 PM Vinicius Dias <carlosv775@gmail.com> wrote:
> >
> > I would like to know how or where I can find the code coverage for
> > php-src to see if there is any uncovered code I might be able to
> > contribute with new tests.
>
> It lives here:
>
> https://app.codecov.io/github/php/php-src
>
> However, it seems it's currently borked. The nightly job succeeded, so
> I'm not immediately sure what's wrong. I'll put it on my list, unless
> somebody beats me to it.

Looks like whatever issue there was fixed itself. I'm happy to review
any patches improving coverage.

There are some inaccuracies with tracking, i.e. some executed lines
are marked as red. This happens often for macros that insert branches
that are never taken (and aren't intended to be taken).

We also don't care about the failure paths of the Z_PARAM_ macros.
These are tested sufficiently, so we don't need to repeat the same
test for every function.

Thanks!
Ilija