Formed in 2009, the Archive Team (not to be confused with the archive.org Archive-It Team) is a rogue archivist collective dedicated to saving copies of rapidly dying or deleted websites for the sake of history and digital heritage. The group is 100% composed of volunteers and interested parties, and has expanded into a large amount of related projects for saving online and digital history.
[CI] Run hourly to look for head changes #1607
Conversation
Codecov Report
@@ Coverage Diff @@
## master #1607 +/- ##
=========================================
Coverage 58.73% 58.74%
Complexity 92 92
=========================================
Files 399 399
Lines 17864 17864
Branches 2475 2475
=========================================
+ Hits 10493 10494 +1
Misses 6646 6646
+ Partials 725 724 -1
Continue to review full report at Codecov.
|
|
| Test | Results |
|---|---|
| Failed | 0 |
| Passed | 1685 |
| Skipped | 12 |
| Total | 1697 |
💚 Flaky test report
Tests succeeded.
Expand to view the summary
Test stats 🧪
| Test | Results |
|---|---|
| Failed | 0 |
| Passed | 1685 |
| Skipped | 12 |
| Total | 1697 |
|
I worry a bit about this behavior. Unless it's very clear what is happening, I can foresee a scenario like this:
At this point, if a developer goes to try to reproduce this problem, it seems natural to me that they would try to replicate it given the state of their branch when it was submitted, instead of reproducing what is actually being tests, which is their changes merged with HEAD on the branch. If we can make it absolutely clear that the test failure happened against the changes which were merged with the head of the branch and give a clear indication to the developer against the actual test environment (their changes + HEAD), then they can replicate that state locally. Otherwise, we may be setting developers up for a very painful debugging experience. This goes doubly for OSS contributors who might get behavior they were not expecting. Hopefully I explained this concern well, but if not I'm happy to jump on a call. Also, if the Java devs are OK with this then it seems perfectly fine and I have no objection at all. I just wanted to specifically call out the behavior and what could happen if we aren't careful just so there aren't any misunderstandings. Thanks! EDIT: I would almost rather see a second test (or even pipeline) that was "forward-looking" and had this aggressive merge behavior so that the developers could both see their changes tested against the branch as-submitted, and their changes tested against the head of the branch for which the PR is intended to be merged. In an ideal world, it would only run where the two differ. |
|
Follow-up. @v1v explained to me that this behavior is actually on by default (which I wasn't aware of) and that this change simply makes the period between checks more aggressive. That makes this change much more desirable and I think we should merge it. |
|
LGTM, we will consume 30-50 API calls with the scan is not too much |
What does this PR do?
Automatically trigger any builds for any PRs that got a change in their head target from once a week to every hour if required
This will not update the PR but the workspace where the build happens in the CI. In other words, the build merges the PR with its Target branch before running anything.
It has been configured to run hourly to avoid any issues with the GitHub API quota, we could stress a bit more to run every 30 minutes, but let's see if this approach is somehow interesting to you.
Why
This will ensure PRs are always ready to merge, even if they have not been updated with the latest changes.