Skip to content

Conversation

@MasonProtter
Copy link
Member

@MasonProtter MasonProtter commented Nov 10, 2025

This adds another possible env variable JULIA_REFERENCETESTS_UPDATE_AND_ERROR that acts like JULIA_REFERENCETESTS_UPDATE except it will still emit an error when it updates the file.

I want this for a CI workflow where I want to set this variable to true so that during the CI run, failed tests still show as failed tests, but then I can use a github action to collect all the changed files and automatically open a PR with the option to update the files.

There's a demo of the workflow here: MasonProtter/reference-test-test#1

@oxinabox
Copy link
Member

This workflow is a very cool idea.

But do we need this new option for your workflow?
Can't we just use the normal error settings
and then your workflow can copy the files from ENV["REFERENCE_TESTS_STAGING_PATH"] ?
That path is what we already have set up so that failures can be uploaded as artifacts to be looked at.

@MasonProtter
Copy link
Member Author

Ah, I didn't know about ENV["REFERENCE_TESTS_STAGING_PATH"]!

@oxinabox
Copy link
Member

we should document it better and make it clear how to use it to upload artefacts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants