CI: Include testing in All_Debug CI job

[rcorsi]

Checking if it would be possible to run testing in All_Debug CI job

Follow up to #6577

[rcorsi]

All_Debug Tests passed successfully with the inclusion of some fixes that are not yet merged.

https://github.com/LadybirdBrowser/ladybird/actions/runs/18811632170/job/53673640024?pr=6581

Defining TESTS_ONLY=1 environment variable because benchmarks must be excluded as LibRegex has some heavy benchmarks with 10 million character test variable and 100k for loops.

Also currently LibWeb is excluded as it generates way too much output and ctest process gets very large (>500GB of output). There appears to be flags on ctest that can limit the output, but so far have not found the magical values that will make it not crush a system.

[rcorsi]

Got a failure in TestLibCoreFileWatcher with Sanitizer which I've seen before while verifying the threadSanitizer PR (#6034) I created and is due to timings in timers that might not be generous enough in some cases. Could be fixed by setting Tests/LibCore/TestLibCoreFileWatcher.cpp:21 value other than macOS to be the same as line 19..... Another issue to be created.

https://github.com/LadybirdBrowser/ladybird/actions/runs/18825647743/job/53707812045?pr=6581

        Start  99: TestLibCoreFileWatcher
 99/166 Test  #99: TestLibCoreFileWatcher .........................***Exception: Illegal  2.46 sec
Running test 'file_watcher_child_events'.
Completed test 'file_watcher_child_events' in 152ms
Running test 'contents_changed'.
VERIFICATION FAILED: false at /home/runner/_work/ladybird/ladybird/Tests/LibCore/TestLibCoreFileWatcher.cpp:112
Running 3 cases out of 3.
Stack trace (most recent call first):
#0  (inlined)          in operator() at /home/runner/_work/ladybird/ladybird/Tests/LibCore/TestLibCoreFileWatcher.cpp:112:9
#1  0x000055f2ffef524a in call at /home/runner/_work/ladybird/ladybird/AK/Function.h:225:30
#2  (inlined)          in AK::Function<void ()>::operator()() const at /home/runner/_work/ladybird/ladybird/AK/Function.h:148:29
#3  0x00007f8e7ee9cb88 in Core::Timer::timer_event(Core::TimerEvent&) at /home/runner/_work/ladybird/ladybird/Libraries/LibCore/Timer.cpp:94:19
#4  0x00007f8e7ee8015e in Core::ThreadEventQueue::process() at /home/runner/_work/ladybird/ladybird/Libraries/LibCore/ThreadEventQueue.cpp:121:37
#5  (inlined)          in Core::EventLoopImplementationUnix::pump(Core::EventLoopImplementation::PumpMode) at /home/runner/_work/ladybird/ladybird/Libraries/LibCore/EventLoopImplementationUnix.cpp:318:47
#6  0x00007f8e7ef12a97 in Core::EventLoopImplementationUnix::exec() at /home/runner/_work/ladybird/ladybird/Libraries/LibCore/EventLoopImplementationUnix.cpp:310:13
#7  0x00007f8e7ede37c4 in Core::EventLoop::exec() at /home/runner/_work/ladybird/ladybird/Libraries/LibCore/EventLoop.cpp:90:24
#8  0x000055f2fff173f1 in __test_contents_changed at /home/runner/_work/ladybird/ladybird/Tests/LibCore/TestLibCoreFileWatcher.cpp:116:20
#9  0x00007f8e7fb7b7a3 in AK::Function<void ()>::operator()() const at /home/runner/_work/ladybird/ladybird/AK/Function.h:148:29
Errors while running CTest
#10 0x00007f8e7fb3f788 in Test::TestSuite::run(AK::Vector<AK::NonnullRefPtr<Test::TestCase>, 0ul, (AK::FastLastAccess)0> const&) at /home/runner/_work/ladybird/ladybird/Libraries/LibTest/TestSuite.cpp:195:22
#11 0x00007f8e7fb3921c in Test::TestSuite::main(AK::ByteString const&, AK::Span<AK::StringView>) at /home/runner/_work/ladybird/ladybird/Libraries/LibTest/TestSuite.cpp:147:15
#12 0x000055f2ffef448c in main at /home/runner/_work/ladybird/ladybird/Libraries/LibTest/TestMain.cpp:26:44
#13 0x00007f8e7c22a1c9 at /lib/x86_64-linux-gnu/libc.so.6
#14 0x00007f8e7c22a28a in __libc_start_main at /lib/x86_64-linux-gnu/libc.so.6
#15 0x000055f2ffef4c44 in _start at /home/runner/_work/ladybird/ladybird/Build/bin/TestLibCoreFileWatcher

[rcorsi]

Hi @trflynn89, is this followup PR to the PR you merged earlier #6577 something valuable or no chance of merging it? Thanks for your feedback.

[trflynn89]

IMO there's not a lot of value in running tests with all flags enabled, I think most developers are turning on single flags at a time (even often just in a single file).

[rcorsi]

@trflynn89 I got the same impression from Andrew as he said that on the Serenity project only build was used with All_Debug, because test took too long.

And your right, most developers would not use this feature, its certainly is a CI thing. I find it convenient to have it in the Meta/ladybird.py script, but if that would be a blocking reason I would remove it without issue.

Still I went ahead with testing with All_Debug just for learning purposes and I did find two issues, one which you already merged in #6264 and one still not completed in #6263. I mean not critical issues, but if someone wants to use the DEBUG macros and then they get interrupted by some unexpected issue, would be annoying.

Any way, regarding Andrew's statement that Serenity All_Debug testing took too long, here in Ladybird, so far with LibWeb excluded, it is taking ~160 seconds to run it in CI. So not so bad.

For LibWeb, ctest process is taking all available RAM, haven't figured out how to prevent that yet. Don't know if it is normal or a bug in ctest on my system or something in Ladybird or something in LibWeb test. Would like to get to the bottom of that.

Anyways, thanks for your candid feedback.