# EDGI Remote Project Manager - Application

## A problem solved coordinating a remote team

For many years, I championed the creation and operation of the stori.es SaaS platform for Consumer Reports.  This user-generated content management system enables non-profits to collect qualitative and quantitative data from their supporters: real stories from real people.  That data is exceptionally valuable both in bulk and at the level of individual experiences reported by faithful and verified observers.

Activity on many data collection platforms comes in bursts driven by events or directed by outreach efforts including email blasts.  Consumer Reports is a large non-profit, and their trustworthiness and broad mission enabled the acquisition of a large email house file greater than one million people.  Our SaaS infrastructure was hosted on Amazon Web Services and supported auto-scaling, but it could take a few minutes to boot up new EC2 instances once scaling thresholds were reached.  Worse, we had yet to address a bottleneck where requests converged on a single RDS MySQL database instance.  We had learned to avoid scaling issues by bumping up the EC2 instance quantities and RDS instance size in advance of anticipated high-traffic events.

I was attending the O'Reilly OSCON open source conference when I received a message via Google Hangouts from our remote Project Manager and primary point of contact at our contracted software development firm.  Our Nagios website uptime monitoring system had faithfully reported delayed requests and failed form submissions due to a high-traffic event.  We had been through this rodeo before when our stakeholders hadn't thought to notify us of an email blast to the full house list.  I quickly verified this was occurring via Hangouts and phone calls with the Consumer Reports advocacy outreach team.  I reassured the stakeholders that we were on the issue, had a fix, and would report back promptly once the incident was handled.

In this circumstance, the small RDS database instance was croaking so I directed the Project Manager who also handled operations to scale the instance.  At a comparative lull in the incoming traffic observed in realtime via Google Analytics, the instance was scaled and database connection pool refreshed.  We also expanded the maximum number of EC2 instances to ensure no requests would be dropped.  Once we observed the data flowing into the stori.es web application infrastructure smoothly and at great speed, I contacted the stakeholders to let them know they could stop worrying.

Missed opportunities like this cost organizations a great deal in terms of goodwill and confidence.  While we minimized the number of failed requests and submissions by acting swiftly, the advocacy outreach team sent out an "Oops!" message hours later to try to regain some of that trust and give people a second chance to share their story.  Somewhat counter-intuitively, email subject lines that admit failure tend to generate greater clickthroughs, and our system handled the even bigger response with ease.