Profile.next_nag_date tracks when the next hourly/daily reminder
should be sent. Normally, sendalerts sets this field when
a check goes down, and sendreports clears it out whenever
it is about to send a reminder but realizes all checks are up.
The problem: sendalerts can set next_nag_date to a non-null
value, but it does not clear it out when all checks are up.
This can result in a hourly/daily reminder being sent out
at the wrong time. Specific example, assuming hourly reminders:
13:00: Check A goes down. next_nag_date gets set to 14:00.
13:05: Check A goes up. next_nag_date remains set to 14:00.
13:55: Check B goes down. next_nag_date remains set to 14:00.
14:00: Healthchecks sends a hourly reminder, just 5 minutes
after Check B going down. It should have sent the reminder
at 13:55 + 1 hour = 14:55
The fix: sendalerts can now both set and clear the next_nag_date
field. The main changes are in Project.update_next_nag_dates()
and in Profile.update_next_nag_date(). With the fix:
13:00: Check A goes down. next_nag_date gets set to 14:00.
13:05: Check A goes up. next_nag_date gets set to null.
13:55: Check B goes down. next_nag_date gets set to 14:55.
14:55: Healthchecks sends a hourly reminder.
If the notification does not exist, or is more than a hour
old, return HTTP 200 (instead of 400 or 404) so the other
party doesn't retry over and over again.
To send notifications, sendalerts calls Flip.send_alerts().
I updated Flip.send_alerts() to be a generator, and to yield
a (channel, error, send_time_in_seconds) triple per sent
notification.
The initial implementation was just calling signal-cli directly
using `subprocess.run`.
Going with DBus makes it easier to shield signal-cli from the
rest of the system. It also makes sure the signal-cli daemon is
running in the background and receiving messages. This is important
when a recipient does the "Reset secure connection" from the app. We
must receive their new keys, otherwise our future messages will
appear as "bad encrypted message" for them.
Normally, when a webhook call fails (timeout, connection
error, non-2xx response), the HTTP request is retried up to two
times (so up to 3 times total). This is useful when sending
actual notifications, in case the webhook target has a temporary
glitch.
When interactively testing a webhook integration
("Send Test Notification" in the
"Integrations" page), we would prefer to see any errors ASAP
on the screen instead of retrying and so possibly swallowing them.
One specific use case is webhook targets that take long time to
generate a response. "Send Test Notification" is synchronous,
meaning that the user could be stuck for
5 x 3 = 15 seconds waiting for the test HTTP request to time out
three times.
