Computing code coverage on Azure Pipelines

php.internals

Nikita Popov

6 years ago
Hi, gcov.php.net used to provide valgrind reports and code coverage for php-src. It no longer works with 7.4/8.0, because the OS is very old, and building new versions of PHP there gets very hard. The valgrind reports have essentially been subsumed by asan/ubsan on Azure Pipelines, but we're still missing a replacement for code coverage. I think it should be possible to set up an Azure Pipelines job that runs tests with coverage and publishes it, preferably on Azure Pipelines itself, or if that doesn't work something like codecov.io. Maybe someone is interested in giving this a try? Regards, Nikita

Gabriel Caruso

6 years ago
Hello Nikita, Em dom, 15 de set de 2019 às 19:29, Nikita Popov <nikita.ppv@gmail.com> escreveu:
> Hi, > > gcov.php.net used to provide valgrind reports and code coverage for > php-src. It no longer works with 7.4/8.0, because the OS is very old, and > building new versions of PHP there gets very hard. The valgrind reports > have essentially been subsumed by asan/ubsan on Azure Pipelines, but we're > still missing a replacement for code coverage. > > I think it should be possible to set up an Azure Pipelines job that runs > tests with coverage and publishes it, preferably on Azure Pipelines itself, > or if that doesn't work something like codecov.io. > > Maybe someone is interested in giving this a try? > > Regards, > Nikita >
I'm, as I've also reported this a while ago: https://bugs.php.net/bug.php?id=78288. Do you have something in mind how/what to set up? Best regards, -- Gabriel Caruso

Nikita Popov

6 years ago
On Sun, Sep 15, 2019 at 7:39 PM Gabriel Caruso <carusogabriel34@gmail.com> wrote:
> Hello Nikita, > > Em dom, 15 de set de 2019 às 19:29, Nikita Popov <nikita.ppv@gmail.com> > escreveu: > >> Hi, >> >> gcov.php.net used to provide valgrind reports and code coverage for >> php-src. It no longer works with 7.4/8.0, because the OS is very old, and >> building new versions of PHP there gets very hard. The valgrind reports >> have essentially been subsumed by asan/ubsan on Azure Pipelines, but we're >> still missing a replacement for code coverage. >> >> I think it should be possible to set up an Azure Pipelines job that runs >> tests with coverage and publishes it, preferably on Azure Pipelines >> itself, >> or if that doesn't work something like codecov.io. >> >> Maybe someone is interested in giving this a try? >> >> Regards, >> Nikita >> > > I'm, as I've also reported this a while ago: > https://bugs.php.net/bug.php?id=78288. > > Do you have something in mind how/what to set up? >
The first step is probably to get code coverage working locally. We have a ./configure --enable-gcov option and there's some extra make targets in build/Makefile.gcov. Maybe taking a look at what https://github.com/php/web-gcov/blob/master/cron/cron.sh does would help. Second step is to add a new job in azure by copying https://github.com/php/php-src/blob/master/azure/job.yml and adding the necessary gcov/lcov magic there. The hard part if how to get those results published on Azure under the "Code coverage" tab. There is a PublishCodeCoverageResults task for this purpose, but I don't know how one would use that with lcov in particular. It seems to me that part of it requires a coverage overview in a specific format and part is just a directory with HTML, so maybe the result from ltp genhtml can just be used there? Figuring out how to make this work is likely going to be "fun" ;) Nikita