Set a baseline before you turn on alerts

MaintenanceBeginner~10 min

Before you start

Plugin version 1.9.0 or newer for alerts, and at least a week of agent activity in the log. The Activity Log read requires the manage-options capability, so run this from an administrator connection.

Version 1.9.0 can tell you when something important happens: an app being approved, a high-risk tool running, a burst of denied calls, or a safety switch being flipped. All of it is off until you choose. Turning alerts on takes a minute. Knowing which ones would help and which would be noise takes a sense of what a normal week looks like on your site, and your own Activity Log has that. This prompt reads a week of it and hands back the numbers. It reads and reports, and changes nothing.

The prompt
I am about to choose which alerts to turn on for this site, and I want a
baseline first. Read only, change nothing.
Read the Activity Log newest first, 200 rows a page, and keep paging until
created_at (it is UTC) is older than seven days ago. There is no date filter on
this read, so that is the only way to bound it. If it takes more than five
pages, stop and tell me how far back you got.
Set aside rows that are not tool calls: ability "(transport)", an ability
starting with "oauth:", "aafm/write-outcome", "aafm/activity-log-cleared",
"aafm/read-only-mode", "aafm/high-risk-abilities-unlocked",
"aafm/review-publish", "aafm/review-discard", any ability that does not start
with "aafm/" or "aafm-bridge/", any row whose detail starts with "Enabled ",
"Disabled " or "Blocked " (someone switching an ability on or off in wp-admin),
and any denied row with no acting user.
Count those on one line, with two exceptions that I want counted
separately: rows named "oauth:authorize" with status success (each is an app
being approved to connect), and rows named "aafm/read-only-mode" or
"aafm/high-risk-abilities-unlocked" (each is a safety switch being changed).
Then, for the tool calls, give me:
1. Calls per day, with how many succeeded, errored and were denied.
2. The ten busiest abilities and their counts.
3. The busiest five-minute stretch for denied calls, in UTC, and how many
denials fell in it.
4. How many calls ran to an ability whose name starts with "aafm/delete-", and
which ones. This is a name match only.
5. The number of apps approved in the week.
6. The safety-switch rows, with timestamp and detail.
Put the numbers first. Then say in two sentences what a "normal" day looks like
here, and say plainly if the week was too quiet to tell.

Why this is safe to run

It is a single read, and it reports history. Nothing in your settings, your alerts or your connections is reachable from it.

It also cannot see the alert settings, on purpose. An agent holding an administrator credential could otherwise learn how loud your alerts are, so those rows are left out of what the read returns. The choice stays with you.

Turning the numbers into a choice

Open Settings, then Notifications, and pick how you want to be told: an admin notice only, an email right away, or one daily email. Then tick the events. The guide to getting an email when an agent does something risky walks through each setting.

Read the baseline against them:

  • Burst of denied calls. The alert fires on the tenth denied call within five minutes (each denial restarts the clock), and also when a connection hits the rate limit. If your busiest five minutes holds three denials, it will stay quiet. If it holds ten or more, it would have fired.
  • A new app or Application Password. If you approved one app all week, an alert on each is a useful tripwire. If you connect clients daily, it will wear thin.
  • High-risk or permanent-delete runs. If the count of delete-style calls was zero, any alert on this is a surprise worth reading. If an agent routinely cleans up, the daily email suits better than the immediate one.
  • Read-only or high-risk lock switches. Each change is a row in the log, so the prompt counts them. If you rarely flip them, an alert on each change will be easy to read.

Each type of event sends at most one email every 15 minutes, and mail goes through WordPress’s own mail, so on a host where mail is unreliable the admin notice is the dependable choice. The daily email needs WP-Cron.

Rerun the prompt a week after you change your choices. The numbers show whether the alerts you picked would have been quiet or loud. When one does fire for denials, triage a burst of denied calls finds the cause, and the weekly change report is the same read in report form.

Already connected? Skip the paste. From an admin connection, ask your agent for the “set a baseline before you turn on alerts” recipe, or just ask what a normal week looks like in the log, and it runs the same read on your own site.

How it works

  1. Read a week of the Activity Log newest first, paging back to the start of the window.
  2. Set aside the rows that are not tool calls, then count tool calls per day by status.
  3. Find the busiest five minutes for denials, since a burst alert fires at ten denied calls in five minutes.
  4. Count apps approved, delete-style calls and safety-switch changes in the week.
  5. Report the numbers plainly and leave the choice of alerts to you.

Questions

Why not just turn every alert on?
You can, and they are off by default for a reason: an alert you learn to ignore is worse than none. Knowing how many calls and denials a normal week holds tells you whether an email on a denial burst would fire daily or once a quarter, and whether a daily digest suits you better than an immediate message.
Can the agent read or change my alert settings?
No. There is no ability for alerts, and rows about your notification settings or failed alert sends are left out of what the log read returns. You choose them yourself under Settings, Notifications.
Does this tell me what a high-risk run would look like on my site?
Only roughly. The prompt counts calls to abilities whose names start with delete, which is a name match and not the plugin's own high-risk classification, because the log read does not return that. It tells you whether destructive calls are rare enough that an alert on each one would be useful.
What do the alerts email me?
What happened and a link to the Activity Log. Never the content of a call, a user name or an IP address, so you open the tab to see who.