- IE hosts were namespaced off from sysops' view.  For any host that was deployed to legacy Icinga and labeled as owned by IE, the permissions were set in the webpanel so that Icinga users belonging to the SysOps IDM group would not be able to see the IE host in Icinga, thus preventing the majority of ShiftOps from monitoring and responding in kind to any issues. I reached out a handful of times to inquire why this was the case and was told there were not yet any documented procedures for how SysOps was to respond to alerts from those hosts, so it was simpler to keep them hidden from our view.
- IE previously implemented a check_priority_services.sh service that would determine if a process state had changed in the last hour.  This was essentially a bash wrapper that was applied to many hosts manually, and was not brought into our monitoring-plugins repo until November 2019.  This wrapper, however, was redundant, as SysOps had moved away from the legacy check_procs plugin, instead implementing go-check for processes, which already has the functionality of alerting should a process be running but have an age below a determined time threshold. Implementing this as an additional status for a proc check was a matter of adding a conditional flag to the already existing service, rather than creating a duplicate service with bash wrapper around another check.
- In February, IE began adding MegaCLI checks to a number of their baremetal hosts.  This was done by pulling in a check_megacli python script from the Icinga depot which wound up spamming the alerts channels with python tracebacks for several hours due to an dependency error.  I reached out to explain that we had already implemented MegaCLI checks into our go-check, and that out monitoring-plugins-dss RPM pulls in the requisite dependencies automatically, and that the service config was already in Icinga and could be applied without client-side configuration.  The same day, a redundant ipmi_psu_status alert was applied, despite us already supporting PSU checks via go-check, and spammed the same channel with pernissions errors as sudoers was only configured for the go-check and not the new IPMI script.