Categories
eCommerce Performance Engineering Site Reliability

Always Be Improving

(Decorist : 8/18-12/19)

Challenge

Leading-by-example to own and improve systems as sole ENGR having SRE/DevOps/Frontend/Backend experience.

Action

Mar 2019

Watching our AWS costs rise ~8% monthly…

Costs Rising

I learned about and subscribed to Reserved Instances to realize costs savings for our hosting spend:

Dec 2019

Though not leading to cost savings or revenue generation, part of my responsbilities have been database administration, jumping in when the production DB would spike like below, figuring out if a runaway process needed to be terminated, if a slow query was bringing it to its knees, if a cron job was introducing load, or whatever needed to be done to keep the site up.

Or when bots would crawl the site, bringing it down, necessitating an IP block:

Or when digging into the logs to find that a route was 500ing and had to be fixed:

Mar 2020

Using Cloudcraft, I diagrammed our AWS infrastructure, identifying and deleting 1000 unused SQS instances.

Also identified and deleted numerous unused RDS snapshots:

All changes led to a yet another 37% reduction in MoM AWS costs:

Results

  • Saved company 115% of my salary in 2019 through process improvements.
Categories
Process

Env Consistency

(Decorist : 8/19-8/19)

Challenge

Since the beginning of the company, six disparate environments had organically come into being, each powered by their own DB – unnecessarily complicating testing.

Action

Introduced daily process for prod data replication with scrubbed PII propagating to other environments.

Results

  • Homogenized environments, empowering staff to use prod login on any env, greatly simplifying testing and