Given a steady stream of revenue, fundamentally alter the underlying business model towards CPC/CPA.
Action
Phase 1
In March 2020 pre-COVID, did a quick dive on Skimlinks documentation and put together a quick explanation/overview to demonstrate how easy it would be to include. Business climate wasn’t right so didn’t pursue.
Phase 2
Sep 2020, post-CEO-stepping-down and several months after Phase 1, explored use of Viglink (Sovrn) only to discover that business had previously been CPA-based before fulfillment was brought in-house ~2017.
Digging around, found some old Skimlinks code, then leveraged updated documentation to prove click-tracking with custom params could still work.
Phase 3
The business, being unsure whether to use one or several affiliate networks, needed partnership to figure out the best implementation path forward. For four networks, I kludge’d scripts into production and verified clicks, evaluating custom parameters as they’d flow through the lifecycle and eventually be reported through APIs.
When the business decided to focus on Commission Junction and Skimlinks, I led development to integrate JS libs (monkey-patching CJ to play-well on the same page as Skimlinks) and verify clicks / custom params.
Result
Overcame fits-and-starts to deliver affiliate model.
A newly-added Custom Dimension VWO wasn’t tracking.
Action
We had just introduced VWO into the stack for A/B testing when one of the experiements wasn’t reporting correctly.
I jumped in with PROD to debug and worked with VWO to resolve and figured out there was a race condition with our Angular code preventing the beaconing of the Custom Dimension.
I fixed the issue and deployed to prod.
Results
Collaborated w/Product Manager to get VWO experimentation working.
The business wasn’t confident in ROI on FB marketing spend because FB pixel wasn’t firing (and no one internally was a Subject Matter Expert.)
Action
Worked with a SF-based, non-technical marketing consulting and delegated to a Delhi-based junior frontend engineer to get familiar with FB Pixel.
We went back and forth, encumbered by the 12.4h time difference and lack of holistic understanding about how to best run campaign and test the efficacy of the pixel.
Finally, after weeks of simply not being able to get to “Done,” I jumped in and found the FB Pixel Helper:
With the problem solved, we were able to finally turn up our marketing spend.
After reviewing the company’s developer docs and Platform Roadmap, I addressed low-hanging fruit around improving readability and structure while working with key stakeholders (Head of Platform, VP PROD) to organize growth priorities regarding platform adoption over timeframes of 1, 1-3, 3-6, and 6-12 months.
Upon further iterations around growth KPIs while learning about constraints of the platform engineering team, I advocated for the creation of a separate Developer Relations team of a minimum of a Lead, Partner Engineer, Developer Advocate, and a Product Manager (and provided the basic job descriptions for each.)
Also, upon discovering that the Hello World experience of integrating with the platform was less-than-intuitive, crafted the following How-To in one take without a script and using QuickTime:
I took the basics of use cases that had been articulated, in spreadsheet and Pivotal forms, and refined them with an intent to GTD while keeping our eyes-on-the-prize. Also, I introduced feature and bug templates in Pivotal to make it easy for team members to follow a quality process…
PRD, original
… as well as the use of Koan for retrospective-like reflection.
Growth
Survey
After our first broadcast event, I sent out a simple survey gathering NPS and other key data.
To help everyone keep their eyes on the prize, I threw together a quick spreadsheet tracking KPIs. At first I was tracking daily but switched to weekly.
Landing Page
To facilitiate a better understanding for customers, both demand- and supply-side, I put together a first, simple version of a Landing Page.
Design Improvement
In addition to all other improvements I’ve already mentioned, I applied minimalist quick improvements to take the general UX from this on the PLP
… to this (w/hand-rolled JS and CSS ‘filters’ added in the header)
… and this on the Activity Feed
… to this
Being Social
In a bid to help the supply-side of the market promote their brand, learned how user metadata is managed in WordPress in order to modify the registration form in order to capture salient social media handles.
regisration form, after
Once submitted, the handles were captured in the DB and then rendered in three locations.
After I started, I saw an opportunity to improve code DRYess and consolidated the rendering code into a single place.
Refined Strategy
In addition to being the technical contributions as Head of Engineering, I helped refine strategy:
incl. for the pitch deck
introduced structured approach to product management
budgeted ops costs in prep for pre-seed/seed funding
introduced process around password credential and software license management
Results
Led Growth efforts with Landing Pages, NPS Surveys, social virality, and KPI tracking.
In January, peers in different areas of the company came together in a cross-functional guild with the goal to beat company plan around yearly supply-side user acquisition. Over the following months, we ideated and executed on a few ideas to make that happen.
Where I saw opportunity to realize the most gains for the least amount of (engineering) effort, I acted as Senior Engineering Representative and spoke to the likely costs of implementation, overseeing the ideas we as a guild prioritized.
Re-consign
To realize an idea around “re-consigning” – that is, enabling demand-side to re-sell an item they purchased from supply-side, I threw-together a “reconsign” link with an Javascript alert and beaconing usage via Segment, worked with PROD and DSGN to flesh out the feature in-full, and then managed a remote team to to get the feature (fully-baked) into production.
Friendbuy
Long championed by the CFO, and by consensus of the guild, we scoped out and delivered an integration with Friendbuy; which wound up being beneficial for user acqusition on both sides of the marketplace.
Funnel Optimization
The guild sanctioned a prioritized idea around optimizing the supply-side user funnel. DSGN built out wireframes and I managed the team to realized the new acquisition funnel in Phoenix.
Results
Led best 2017 revenue-gen (adding $3M to the bottom-line) and best 2017 lead-gen features (boosting leads by 50%.)
Did not actually implement social signup, merely checked for clicks on buttons (using Optimizely to show/hide based on whichever variation user was assigned.)
With social signup options above:
With social signup options below:
Results
Tested conversion hypotheses with social signup buttons.
Following is a representative control version of the landing page
I whiteboarded several concepts gaining buy-in from the chief product stakeholder and then implemented same, using Optimizely to set the parameter to determine which variation a user saw; below you will see the iterations.
Unfortunately, none performed better than control.
Results
Tracked performance of inbound traffic from Facebook, Twitter, Google AdWords, and StumbleUpon and designed/implemented various landing pages to improve signup rates.
Contributors, as they are called, are the +5M people around the world who do work on CrowdFlower’s platform. The application that enables them to do work is one of the company’s heavily trafficked as well as most complicated – blending a Rails backend with MooTools, jQuery, and RequireJS in the frontend.
The application’s UX
…had largely stayed the same for the last five years. In Q1/2014, we decided to enhance it by making it more interactive and towards engaging our users more and conveying the just how much work there is in our system.
Working with the Product Manager and an external Designer, we came up with the following high-resolution mock
Because the application is so heavily used, we knew we couldn’t merely throw the switch on a new design overnight; both from a community management standpoint as well as application performance. Instead, we chose a strategy of introducing a first at the company: use of A/B Testing to determine a design that would perform as well as if not better than the original.
Our key metric for performance in that regard had to do with contributor’s performance after being exposed to the new UX, particular the messaging around our forthcoming gamification and introduction of Levels. In the beginning, we did not have the infrastructure to determine the value for that metric so we simply settled on ‘clicks’ as a (conversion) proxy to understand if the new design was having an impact.
Infrastructure
Without an A/B Testing framework in place, I needed to choose one. As requirements were not concrete for such, I did some due diligence in vetting several options, coming up with a review of A/B testing frameworks for Rails.
It became obvious that Vanity was best suited to our needs. (Since it doesn’t yet have the ability to throttle a percentage of the traffic receiving experiments, I augmented it with Flipper.)
Once that was in place, we could begin iterating on the design, knowing with confidence how we were impacting the user experience.
Server-side
We knew we wanted the experience to be snappy, but completely replacing the existing experience with a Rich Internet Application was far out of the scope for the first month, particularly as there were infrastructure changes to be made to retrofit the stack with A/B Testing. We decided to make progress iteratively over several sprints.
In our first test, we pitted the control (original) against a bare-bones implementation version of the high-resolution mock as the new design.
original
The new version out-performed control (in terms of clicks) 21.3% vs 20.3% (at 95% confidence) so I continued to iterate on the implementation, coming up with the following
To calculate the overall satisfaction by other contributors for a task (denoted by the stars) proved to be too inefficient in this iteration; it wound up losing.
Client-side
On the assumption that we needed to make the experience snappier in order to drive engagement, it was obvious that we would need to have more (and faster) interaction and therefore, an interactive client-side implementation.
As what was essentially a completely parallel product, leveraging only some of the infrastructure that the server-side rendition was utiziling, I begin to flesh out the following
Further refinement (an actual data) was necessary to get it looking more like the high-res mock (and like its server-side-rendered peer)
At this point, we implemented and integrated with our own homemade badging solution, beginning to display badges in the following iteration
The new version out-performed control (in terms of clicks) 21.3% vs 20.3% (at 95% confidence) so I continued to iterate on the implementation, coming up with the following
Testing the impact of particular messaging was also of interest, so we added a Guiders variation as well. At this time we also leveraged Google Analytics Events on the Guider buttons to track how the far the user got in our messaging.
Letting the experiments run a few days with sufficent traffic, we found that client-side-rendered version peformed no worse than the server-side-rendered version (23.9% vs 22.9%) and that having guiders also performed not significantly worse (23.1% vs 23.7%) so we decided to keep both.
By that time, the new version was out-performing control (the original design) 22.2% vs 20.7% (at 99% confidence) so a decision was made to move forward rolling out the new experience to 100% of contributors, doing some polishing (copy/styling) work before finally settling on the following
Results
Used A/B testing to upgrade company’s most highly-trafficked page (5+M views/month,) increasing user engagement by 5% and saving $2K/month (in Bunchball costs) by rolling own simple badging solution.
