Project: SoFi

SoFi thumbnail 1
SoFi thumbnail 2
SoFi thumbnail 3
SoFi thumbnail 4
SoFi main image

Experimentation Platform Accelerator

SoFi

Web ApplicationServerless

A lightweight experimentation platform that enables rapid testing and validation of new features through A/B testing and data-driven insights.

Completion DateMarch 20, 2025
Project Day4 of 10
Source CodeCorrespondence

Project Demo

Project Metrics

7
Hours Spent
8,933
Lines of Code
2,194
Lines of Markdown
$45.27
LLM Cost

Project Details

The Experimentation Platform Accelerator accelerates innovation by providing a structured approach to testing and validation, reducing the technical overhead of implementing experiments. It helps SoFi rapidly test and validate new features to maintain competitive advantage.

Business Value

  • Data-Driven Decisions
  • Innovation Acceleration

Key Features

Experiment Management Interface

Create and configure A/B tests with multiple variants, define experiment parameters and success metrics, and view active and completed experiments.

Sample Page with Live Experiments

Demonstrates experiments in action on a sample checkout page, shows different button colors based on experiment variant, and tracks user interactions with experimental elements.

Event Collection System

Records user interactions with experimental elements, captures conversion events for statistical analysis, and stores events in DynamoDB with experiment context.

Report Generation Pipeline

Processes collected event data using AWS Batch, calculates conversion rates and statistical significance, and generates comprehensive JSON reports.

Technologies Used

Frontend

ReactTypeScriptCSS ModulesD3.jsTanStack Router

Backend

Node.jsExpressJavaDagger 2Apache Commons Math

DevOps

AWS LambdaAPI GatewayDynamoDBS3AWS BatchAWS CDK

Other

A/B TestingStatistical Analysis

Challenges & Solutions

AWS Batch connectivity issues

Added public IP assignment to Fargate tasks, created security group with outbound internet access, and granted ECR pull permissions to task execution role.

CloudFront 403 Forbidden errors

Removed website configuration from S3 bucket, added S3 managed encryption, simplified CloudFront distribution configuration, and added handling for 403 errors.

DynamoDB query expressions in Java

Fixed the ExperimentEvent class by moving annotations, updated query approach to use QueryConditional.sortBetween(), and ensured consistent AWS region settings.

Key Learnings

  • NX Monorepo Structure: The NX monorepo structure provides a clean separation of concerns while enabling code sharing, though it requires careful configuration to avoid nested directories.
  • TypeScript/Java Integration: Maintaining consistent data models between TypeScript and Java components requires careful planning, including mirrored class structures and proper serialization.
  • AWS Integration: AWS services provide powerful capabilities for building cloud-native applications, but require careful configuration and error handling for reliable operation.
  • API Robustness: Implementing comprehensive validation, logging, and error handling in the API is essential for building a maintainable application.