The following discussion is an archived debate. Please do not modify it. To request review of this BRFA, please start a new section at Wikipedia:Bots/Noticeboard. The result of the discussion was Approved.
Function details: Based on pre-configured information about bot tasks (name of bot account, what edit summaries it uses, what pages/namespaces it edits, how many edits are expected in the last x days, etc), it identifies bots and bot tasks which have stopped working. Stalled bot tasks can be identified even if the bot account is still running other tasks. Bots which perform actions other than editing (deletions/blocks/patrols etc) can also be monitored. A bot status table would be generated and posted to WP:Bot activity monitor.
If configured, this bot can also issue alerts to the operator to let them know that their bot tasks are not running. Alerts can be sent via talk page or email or least intrusively, via a ping from a central page.
I expect anyone should be able to set up a bot for tracking (to be included in status table), but of course only the operator(s) should set up alerts for themselves.
The configuration parameters for describing bot tasks are given at WP:Bot activity monitor. However, I'm a bit confused on how and where should people set up these configurations. My initial thought was to have a central JSON page: Wikipedia:Bot activity monitor/config.json but the problems with JSON are (i) it requires regexes to be weirdly escaped (eg. \d needs to be written as \\d, though it will show up as \d while viewing the page) and (ii) it looks so clumsy, especially when there would be 100s of tasks. It seems using template markup is better to describe the configurations – but should they go on a central page or be decentralized on bot user pages? The drawback of the latter is that it discourages users from setting up tracking for others' bots. – SD0001 (talk) 15:35, 12 November 2020 (UTC)[reply]
I'm super happy to see this; thanks for your work on it, SD0001! I don't have the expertise to comment on the technical questions, but as far as monitoring goes, my sense is that many bots that stop working have retired operators, so it would be good for there to be notifications not just to the talk page of the operator. Looking forward to seeing this in operation! {{u|Sdkb}}talk 15:58, 12 November 2020 (UTC)[reply]
Okay, so am I reading it correctly that this is an opt in situation, only "checking up" on whichever specific bots are listed? Also, why do we need a second process when Wikipedia:Bots/Requests for approval/MajavahBot 3 exists? Primefac (talk) 14:12, 13 November 2020 (UTC)[reply]
@Primefac: This is a lot more advanced than MajavahBot 3. See User:SD0001/Bot_activity_monitor/Report for the kind of output it produces – that's based on the data for a cherry-picked set of bots at Wikipedia:Bot activity monitor/config.json. And as mentioned it also supports sending notifications to botops. Because all of this requires data about bot tasks in a machine-readable form, it necessarily has to be "opt-in" (through folks can opt-in others' bots). – SD0001 (talk) 19:07, 13 November 2020 (UTC)[reply]
Fair enough. Per the general precedent, there's no issue with creating a database-style report for these bots (i.e. "only edits one page") but when it starts getting towards notifications there come more questions. Speaking as a bot operator, I don't really care if someone keeps tabs on my bot, but I don't want any sort of automated notice if I happen to decide not to run one of my "active" tasks for some period of time, and I'd rather not find out after receiving a notification that someone's added my name to the "notify" list. Primefac (talk) 20:15, 13 November 2020 (UTC)[reply]
Agreed. I myself wouldn't want these notifications – I've implemented error handling in my own bot tasks so that whenever an error occurs, I get an email with the stack trace of the error – which would be more useful than a generic message from a third-party bot which says the task didn't run. This is why I say above notifications would (should) only be enabled by botops themselves. But I think we can just let this be a convention rather than try to restrict it at the technical level, and hope that people won't be jerks? Remember that it's technically also possible for a random guy to subscribe you to random wikiprojects' newsletters – but this doesn't seem to happen in practise. – SD0001 (talk) 11:33, 14 November 2020 (UTC)[reply]
I'll drop a note at WP:BON about this bot. One point is worth noting, just in case it isn't obvious, is that the "monitoring" is intended for bots that run fully automatically – with zero human intervention. It wouldn't make sense to track bots that are one-time or on-demand, or the even the ones which require any level of operator intervention to run. The intent is to "catch" bot stoppages which the operator may not be aware of, typically occurring due to unforeseen issues such as the ones Anomie mentioned at this discussion, quoting:
Something changes that causes the bot to fail unless its code or configuration is updated ...
A software update by the hosting provider breaks the bot's code, again requiring a code update.
The bot's process stops running or locks up, and the operator isn't paying attention to notice and restart it.
The hosting provider where it bot is being run closes the account (or closes entirely).
I think this is helpful and not particularly problematic. Of course, operators are not obligated to run tasks, but many times the downtime is accidental not intentional. For example, when my task 3 stops we lose a day of Main Page history. It did lock up once after some maintenance from my host, so Template:2020 Main Page history is missing November 9b and 10. Setting up good app/server monitoring is not what most bots do. Note I haven't looked too closely at the implementation yet to say if I have any concerns with that part. ProcrastinatingReader (talk) 15:21, 14 November 2020 (UTC)[reply]
Sounds like a great idea. I would prefer JSON where each bot monitor is its own object with fields for summary regexes, expected run times, number of runs per day, an array of pages that are expected to have been edited, etc. JSON has the added benefit of being highly extensible since it can contain other objects allowing for more complex configurations. That said it may not be widely accessible to less-technical botops, and the regex escape problem is always a nuisance. Either way, sounds great and I look forward to seeing it in operation! — Wug·a·po·des 03:25, 15 November 2020 (UTC)[reply]
BAG question: is the notification system up and running? Primefac (talk) 10:52, 16 November 2020 (UTC)[reply]
Not yet, the notifications code as presently written will keep spamming the botop every half an hour until their bot comes back up again! I'll probably have to further use SQLite to keep track of notifications sent to avoid repetitions. – SD0001 (talk) 15:12, 16 November 2020 (UTC)[reply]
If you can have the dbase/check running without the notifications enabled, feel free to start running that part while the rest gets hammered out. Primefac (talk) 17:40, 16 November 2020 (UTC)[reply]
@SD0001: Happy new year! Gently checking in on the status of this. — The Earwigtalk 05:10, 12 January 2021 (UTC)[reply]
A bit caught up at the moment with other projects. Will try to take a look at this soon. – SD0001 (talk) 13:58, 26 January 2021 (UTC)[reply]
Apologies to everyone for the long delay. I've changed the config file from JSON to the more tractable wikitext format: it's at Wikipedia:Bot activity monitor/config, powered by Module:Bot task. We've presently got a small cherry-picked set of bots. Requesting folks to add other automatic bots here that you know of. Maybe this can also be mentioned in the next Bots newsletter whenever it goes out. Also in special:diff/1008068560 I've hinted at an automated way of generating a database of active bot tasks, in case anyone wishes to explore that. The work on alerts is not over yet. – SD0001 (talk) 18:22, 22 February 2021 (UTC)[reply]
I can add mine (notifying Draft creators of impending G13 timescale) if you wish? ƒirefly ( t · c ) 18:55, 22 February 2021 (UTC)[reply]
Yep. Feel free to edit that page! – SD0001 (talk) 20:05, 22 February 2021 (UTC)[reply]
Done! ƒirefly ( t · c ) 21:43, 22 February 2021 (UTC)[reply]
Where are you planning to run this bot from? If it's on Toolforge and then Toolforge goes down, it wouldn't be able to alert people. On the other hand, if Toolforge is down, the notifications probably aren't that useful since most bot operators can't do much. In any case, I used to pay for an external monitoring service, so this would be nice. Legoktm (talk) 01:36, 23 February 2021 (UTC)[reply]
Toolforge. I think the platform itself is quite stable – I don't recall it having gone down anytime in the past year. ToolsDB and Redis which I think are less reliable, are not being used. The bot only needs a SQLite db for caching. If it does happen though, as you say, there's little point in notifying people about failures. The status report update task can still run – I suppose I'll take a look at GitHub Actions for that, which is also free. – SD0001 (talk) 11:38, 23 February 2021 (UTC)[reply]
I wasn't aware of that. Will probably later look into replacing SQLite with redis as it's used for caching. – SD0001 (talk) 21:10, 2 March 2021 (UTC)[reply]
Here is a list of bots that have edited the wiki over the last day or so. It also tries to show what the bot did – the first 3 words of the edit summary after skipping words like "bot" or "task" and any wikilinks, punctuation and numbers. Fun fact: automatic userpage moves by global renamers also get marked as bot edits! – SD0001 (talk) 12:09, 26 February 2021 (UTC)[reply]
Approved for trial (30 days). Please provide a link to the relevant contributions and/or diffs when the trial is complete. as this bot has limited pages to edit / actions to perform - no big deal letting it officially be in a trial. — xaosfluxTalk 10:13, 28 April 2021 (UTC)[reply]
I've gotten around to this finally to implement the alerts function. The template at WP:Bot activity monitor/Notification is used as the text. The redundancy prevention system is basically similar to what AnomieBOT does at User talk:AnomieBOT. Once a notification is sent, it won't be sent again as long as that section remains on the page (determined by looking at the section header). If/when the botop fixes the issue, they can change the section header (e.g. append " - Fixed") or remove or archive that section. Then only would SDZeroBot notify them again if their bot is still broken or re-broken.
I've tested this personally with some mock configs and am happy with the results. But for BRFA purposes, if notifications are to be trialled, I'd need a few volunteers. You can add the |notify=User talk:YOURUSERNAME parameter to the bot task config at WP:Bot activity monitor/Configurations. – SD0001 (talk) 16:11, 11 May 2021 (UTC)[reply]
Trial complete. The report at Wikipedia:Bot activity monitor/Report (and formerly at User:SD0001/Bot activity monitor/Report) has been updated for more than a month. As for the talk page notifications, no one volunteered to be a test case, which is fine – I've tested the code personally with some mock configurations (eg. [1]) and am happy with the results. – SD0001 (talk) 17:08, 31 May 2021 (UTC)[reply]
The above discussion is preserved as an archive of the debate. Please do not modify it. To request review of this BRFA, please start a new section at Wikipedia:Bots/Noticeboard.
Approved.
Approved for trial (30 days). Please provide a link to the relevant contributions and/or diffs when the trial is complete. as this bot has limited pages to edit / actions to perform - no big deal letting it officially be in a trial. — xaosflux Talk 10:13, 28 April 2021 (UTC)[reply]
Trial complete. The report at Wikipedia:Bot activity monitor/Report (and formerly at User:SD0001/Bot activity monitor/Report) has been updated for more than a month. As for the talk page notifications, no one volunteered to be a test case, which is fine – I've tested the code personally with some mock configurations (eg. [1]) and am happy with the results. – SD0001 (talk) 17:08, 31 May 2021 (UTC)[reply]
