Tools: SQL, Python, Power BI, LLM

Dashboard Link: Live Dashboard Here

Python (A/B and Hypothesis Test): Python Notebook

View Data Schema: Data Schema

GitHub Link: Check SQL Queries, and more.

Note: After clicking on Live Dashboard, please select the “Fit to Page” or “Full Screen Mode” option on the live screen so that the dashboard can adjust according to your desktop or laptop system.

Overview:

This is an end-to-end Sign-up Rate Improvement solution. The focus is to design the test, build data schema, run test, validate results, and perform statistical analysis.

Situation

The startup is facing low signup rates. Lower signup rates are creating difficulties for the startup to grow the way they were expecting.

Task

The task was to identify the reason for lower signups and build solutions for that.

Action

On analyzing data, the maximum drop-off in the signup funnel was at the user info page. I suggested the signup page simplification.

I collaborated with product and UX teams to simplify the signup page.

Then I built an A/B test design (variant control groups, sample size, success metrics, downstream effect analysis), ran the test, and validated results via statistical analysis, including confounding variables and robustness checks, and developed a dashboard in Power BI.

Result

We are 95% confident that the new, simpler signup page improved the signup rate by atleast 10%.

Some Quantifiable Results:

• Defined success metric (sign-up rate) for A/B test, selected traffic percentage, event type, exposure type, and more to get a relevant, unbiased comparison between control and variant groups.
• Conducted end-to-end hypothesis testing, confirming that variant groups have better sign-up rates (10-19%) statistically.
• Connected SQL with Python using SQL Alchemy and psycopg2, reducing data access time by 40%.
• Improved decision-making by 35% by creating a Power BI dashboard, conveying insights in a simple yet effective manner.
• Performed a robustness check to confirm the statistical significance, increasing confidence in the result.