Use logstash-versions.yml for observabilitySRE stack versions #18375
Conversation
🤖 GitHub commentsExpand to view the GitHub comments
Just comment with:
|
|
This pull request does not have a backport label. Could you fix it @donoghuc? 🙏
|
donoghuc
added
backport-8.19
ci/observabilitySREsmoke_tests.sh
Outdated
| export FILEBEAT_IMAGE_VERSION="${FILEBEAT_IMAGE_VERSION:-$QUALIFIED_VERSION}" | ||
| # Look up corresponding LOSTASH_STREAM to test against. MAKE SURE TO UDPATE this per branch for example the 9.2 branch in logstash would | ||
| # correspond to the '9.current' key in the logstash-versions.yml file | ||
| LOGSTASH_STREAM=main |
There was a problem hiding this comment.
With this initial step no concern and for the long term it would be great if we can dynamically resolve artifact version (eg: if branch 9.1, consider values of 9.prev or 9.current if matches 9.1)
There was a problem hiding this comment.
I think ultimately this is going to be a "moving target". IE just recently we had to update 9.1 -> 9.previous 9.2 -> 9.current so computing from branch name wont be possible without an external input. The other thought I had was to centralize this to look up programatically. Perhaps a new section in logstash-versions.yml:
logstash-streams:
"8.19": "8.previous"
"9.1": "9.previous"
"9.2": "9.current"
"main": "9.next"
There was a problem hiding this comment.
computing from branch name wont be possible without an external input
Agree! In the past I had a python script in elastic_integration plugin to check if stream value/artifact-version contains the branch we are running from, if it works for you. Then we moved to matrix because for the elastic_integration case it can be 1:N mapping.
There was a problem hiding this comment.
Thinking about this more, i actually think we want to track the stream information in the logstash repo (this would only require a single update when a new branch is created I added a commit 1985c4b showing an option for that. The main reason is that we should be able to run the scripts for executing the tests from any branch, there is not really a clean way to determine what the intended target stream is for the current branch. For example, if i raise a PR from a branch called patch-n there is not really any information to reliable determine what target branch/stream of logstash that should be targeted at.
versions.yml
Outdated
| logstash-core: 9.3.0 | ||
| logstash-core-plugin-api: 2.1.16 | ||
| # Logstash stream corresponding to keys in https://github.com/logstash-plugins/.ci/blob/1.x/logstash-versions.yml | ||
| logstash-stream: main |
There was a problem hiding this comment.
If this is approved/agreen upon I will update https://github.com/elastic/logstash/blob/main/.github/workflows/bump-logstash.yml to manage this along with logstash version.
Previously the version of logstash checked in to the logstash repo was used to determine stack version for smoke and acceptance testing of the observabilitySRE artifact. Given the ES/Filebeat containers are subject to a unified release the delay between a logstash version bump and those artifacts being available was causing failures when the smoke/acceptance tests were run. In plugin testing the way we deal with this is to ensure the logstash-versions.yml file is updated when stack components are ready. This commit introduces the use of that pattern to the observabilitySRE artifact testing. The important step will be to set the STACK_VERSION to correspond to the desired stack version. For example the 9.2 logstash branch should look up `9.current`.
donoghuc
force-pushed
the
use-logstash-versions-as-source-of-truth-fips
branch
from
1985c4b to
3001241
Compare
💚 Build Succeeded
History
|
3 participants
Previously the version of logstash checked in to the logstash repo was used to determine stack version for smoke and acceptance testing of the observabilitySRE artifact. Given the ES/Filebeat containers are subject to a unified release the delay between a logstash version bump and those artifacts being available was causing failures when the smoke/acceptance tests were run. In plugin testing the way we deal with this is to ensure the logstash-versions.yml file is updated when stack components are ready. This commit introduces the use of that pattern to the observabilitySRE artifact testing. The important step will be to set the STACK_VERSION to correspond to the desired stack version. For example the 9.2 logstash branch should look up
9.current.closes #18358