Skip to content

Change tst/testall.g to quit GAP with exit status indicating outcome#31

Open
fingolfin wants to merge 1 commit into
masterfrom
mh/XGT_Test
Open

Change tst/testall.g to quit GAP with exit status indicating outcome#31
fingolfin wants to merge 1 commit into
masterfrom
mh/XGT_Test

Conversation

@fingolfin
Copy link
Copy Markdown
Member

This is what we want for e.g. tests in the package distro

This is what we want for e.g. tests in the package distro
@codecov
Copy link
Copy Markdown

codecov Bot commented Feb 7, 2024

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.44%. Comparing base (77840f6) to head (d5915b1).
⚠️ Report is 23 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master      #31      +/-   ##
==========================================
- Coverage   93.71%   93.44%   -0.28%     
==========================================
  Files           3        3              
  Lines         175      183       +8     
==========================================
+ Hits          164      171       +7     
- Misses         11       12       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@fingolfin
Copy link
Copy Markdown
Member Author

This causes CI to pass but only because we now quit GAP with an exit code in tst/testall.g. But the second GAP process that gets launched from the primary GAP still runs into an error; it just doesn't get reported back to the top level GAP ...

I don't like papering over the issue, so I'll not merge this for now

@fingolfin
Copy link
Copy Markdown
Member Author

Ah, the failures are not new: #22

@fingolfin fingolfin closed this Nov 6, 2024
@fingolfin fingolfin reopened this Nov 6, 2024
@fingolfin
Copy link
Copy Markdown
Member Author

The test file is not very useful right now either (other than showing that the test setup is broken). @RussWoodroofe added in PR #16 (thanks you!), but right now all it does is run all tests in .tst files -- of which there are none.

But more worryingly, this test setup relies on the GAP function InputOutputLocalProcess. As described here this function is quite limited and among other problems does not allow querying the exit status of the subprocess. So there is fundamentally no way to do that as a check.

Right now the only way for a complete solution I can think of is to replace this test setup by a different one that is not driven by GAP, but rather by python/perl/a shell script/whatever.

That said, perhaps we can come up with a way to at least patch this enough to be useful?

@fingolfin fingolfin closed this Sep 14, 2025
@fingolfin fingolfin reopened this Sep 14, 2025
@RussWoodroofe
Copy link
Copy Markdown
Contributor

I'd understood the brief for GAP testing to be to run some code, and compare its output with a standard output. The routine I wrote makes this possible: running a child gap instance in package mode, and put the output to the standard terminal.

So, the kinds of tests that one can do with this are for conflicts between other packages and package mode and/or the xgap library.

I'd be happy to do some extra plumbing to return exit codes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants