[BRAINSTORMING] Bountysource for testing? Is it possible?

[Yehonal]

Hi guys,

Everyone wants that a PR must fix an issue, but it must be also properly tested. Who places bounties on AC is not always able to test that the fix is properly working and they would like an assurance included within the price of the bounty itself. So I was wondering: would it be possible to send bounties for the testing phase?

The short answer is Yes, we can just ask the developer to provide video evidence that his own work is properly working by asking him to follow a certain procedure as an acceptance criteria for the PR itself.

The long answer is It's not always constructive: The developer could not have time or the possibility to provide such detailed
information and he is willing to share the bounty with a tester who can properly do it.

Ideally, good testing should be done in the following way:

1. Provide a good quality video that shows a .server info output within the game and/or the console of the worldserver to assure that it's using the correct commit.
2. Record and follow the "how to test" steps provided by the developer (it must work 100% as intended)
3. Test the edge-cases and try to push the system to an extreme condition where it could fail, thus pondering if it requires more work or not (certain edge cases not directly caused by the PR changes themselves can still pass, but we must be aware of them)

There is a limitation though: bountysource doesn't support the possibility to split the bounty with multiple people (as far as I know, I could be wrong).

So, what would be the workaround/solution for it in your opinion?

My workaround idea:

The such called "backer" (who places the bounty) can open a second issue (connected to the actual one) to provide a bounty specifically for the test itself.

[FrancescoBorzi]

@Yehonal I completely agree. Giving rewards, as well as recognition, to testers is a topic that I care much about. I've even [suggested to GitHub that we should have a way to keep track of their contributions], just like we do with people who write code.

Apparently, Bountysource doesn't have a proper way to place a separate bounty for testing and one for development, so the only "workaround" I can think of is the same as you suggested. It's not ideal, but unfortunately, I cannot think of something better.

What I would like to emphasise, though, is that the person claiming the bounty IS responsible to get the PR properly tested by either: testing it him/herself or by getting help from other users. I don't mind helping in that sense and keep pinging people to test PRs, but I would like to make it clear that it is the responsibility of the bounty hunter to make sure the PR is properly tested.

[Yehonal]

I completely agree. The bounty should not be acquired without providing proof of tests by the developer itself or a tester (it must be part of the acceptance criteria)