add support for custom informative comments in the bot

[smoors]

it would be nice if we can let the bot write custom informative comments in a PR, for example easybuild error messages:

• easystack file could not be parsed
• unknown easybuild option
• missing easyconfig for dependency

a simple implementation could be that the bot checks for the existence of a file after running bot/build.sh, say bot.comments, and writes the contents of the file as a comment.

those comments should of course be handled with care, you don't want to accidentally write secret info in the comments.

[hvelab]

Hi Sam!

We commented this on our support meetings and this would be a very nice-to-have development, so if you plan to develop it by yourselves we can give you guidance on it and help with any review or testing you might need. One of points we discussed is that with only reporting the error code from EB should be fine, as now they are more extensive: https://github.com/easybuilders/easybuild-framework/blob/develop/easybuild/tools/build_log.py#L74

[smoors]

hi Helena! :)

yes, reporting the error code would already be a nice improvement, but i’d like to support other comments as well.
for example, we have a policy in our cluster that we only install software from the 'a' toolchains. it would be nice if the bot can say: "hey, you’re trying to install in 2023b, that’s not supported".

i don’t have plans to work in it yet, feel free to take it up if you want. i’ll add a note here once i find time to get started.

[ocaisa]

For me, the best place to implement policy like that is in the EasyBuild hooks. What might be interesting would be for the bot to support something like BOT_EXPOSE ... when analysing the output file so that you'd have a method to pass a message back to where the bot is reporting. It currently already checks for certain regex, this would be extending that a bit. That way you could have custom policy that is bot-aware and something that the bot can hook in to.

Right now, the way we implement hooks is not easily extensible I think, we way want to think about how to design things to make that possible.

[smoors]

For me, the best place to implement policy like that is in the EasyBuild hooks.

indeed, that's what we do currently.

What might be interesting would be for the bot to support something like BOT_EXPOSE ... when analysing the output file so that you'd have a method to pass a message back to where the bot is reporting. It currently already checks for certain regex, this would be extending that a bit. That way you could have custom policy that is bot-aware and something that the bot can hook in to.

that's a good alternative solution, and doesn't require writing another file.

Right now, the way we implement hooks is not easily extensible I think, we way want to think about how to design things to make that possible.

not sure what you mean exactly with extensible here?

[ocaisa]

Right now, the way we implement hooks is not easily extensible I think, we way want to think about how to design things to make that possible.

not sure what you mean exactly with extensible here?

Right now, there is no simple way for someone to add local customisation or additional hooks to the eb_hooks.py that we currently ship in EESSI. Ideally, you would people to have the ability to customise on top of what we ship.