{
 "nbformat": 4,
 "nbformat_minor": 5,
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.10.0"
  }
 },
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Notebook 3 — Building Synthetic Users with StrataSynth\n",
    "\n",
    "**What you'll learn:**\n",
    "- Generate psychologically defined synthetic users (personas) using PsycheGraph\n",
    "- Build specific user archetypes: angry customer, confused user, manipulative negotiator\n",
    "- Reuse personas across multiple datasets for consistent product testing\n",
    "- Use synthetic users to stress-test conversational AI before production\n",
    "\n",
    "**Why this matters:**  \n",
    "Real users don't behave uniformly. Your AI system will face fragile trust, escalating emotions,\n",
    "persuasion attempts, and contradictions. Synthetic users let you simulate these scenarios\n",
    "systematically — before your product reaches production.\n",
    "\n",
    "**Requirements:**\n",
    "```\n",
    "pip install requests pandas\n",
    "```\n",
    "\n",
    "---\n",
    "> **API endpoint:** `https://api.stratasynth.com`  \n",
    "> Get your API key at [app.stratasynth.com](https://app.stratasynth.com)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests\n",
    "import json\n",
    "import time\n",
    "import pandas as pd\n",
    "\n",
    "API_KEY  = \"ss_live_...\"\n",
    "BASE_URL = \"https://api.stratasynth.com\"\n",
    "\n",
    "TOKEN = requests.post(\n",
    "    f\"{BASE_URL}/auth/token\",\n",
    "    json={\"api_key\": API_KEY}\n",
    ").json()[\"data\"][\"token\"]\n",
    "HEADERS = {\"Authorization\": f\"Bearer {TOKEN}\"}\n",
    "print(\"Authenticated.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Understanding PsycheGraph personas\n",
    "\n",
    "A PsycheGraph profile is a structured psychological model of a synthetic person. It includes:\n",
    "\n",
    "| Layer | What it defines |\n",
    "|---|---|\n",
    "| `archetype` | The base personality type (24 available) |\n",
    "| `attachment_style` | How they relate to others: secure, anxious, avoidant, disorganized |\n",
    "| `core_fear` | What drives their defensive behavior |\n",
    "| `defense_mechanisms` | How they protect themselves psychologically |\n",
    "| `cognitive_biases` | Systematic errors in their thinking |\n",
    "| `communication_style` | Directness, emotional expressiveness, vocabulary |\n",
    "| `voice_print` | Sentence length, hesitation patterns, formality |\n",
    "| `current_state` | Active stressors and current mood |\n",
    "| `beliefs` | 12 core beliefs about self, others, and future (0–1 scale) |\n",
    "\n",
    "**Personas are reusable.** Generate once, store the `persona_id`, use across multiple datasets."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Available archetypes\n",
    "\n",
    "```\n",
    "working_mother          caregiver_adult_child    estranged_parent\n",
    "romantic_partner        grieving_spouse          young_professional\n",
    "career_crisis_adult     burnt_out_professional   late_career_pivot\n",
    "chronic_illness_adult   recovery_journey         midlife_reexamination\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 1 — Generate specific user types\n",
    "\n",
    "### User type A: The angry customer (burnt_out_professional)\n",
    "\n",
    "High stress, low patience, dismissive attachment, catastrophizing bias.  \n",
    "Use for: customer support training, complaint resolution agents."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def generate_persona(archetype, complexity=3, language=\"en\"):\n",
    "    \"\"\"Generate a PsycheGraph persona and wait until READY.\"\"\"\n",
    "    resp = requests.post(\n",
    "        f\"{BASE_URL}/personas/generate\",\n",
    "        headers=HEADERS,\n",
    "        json={\"archetype\": archetype, \"complexity\": complexity, \"language\": language}\n",
    "    ).json()\n",
    "    persona_id = resp[\"data\"][\"personaId\"]\n",
    "    print(f\"  Generating {archetype} → {persona_id}\")\n",
    "\n",
    "    # Poll until READY\n",
    "    while True:\n",
    "        status_resp = requests.get(\n",
    "            f\"{BASE_URL}/personas/{persona_id}\", headers=HEADERS\n",
    "        ).json()[\"data\"]\n",
    "        if status_resp[\"status\"] == \"READY\":\n",
    "            return status_resp\n",
    "        elif status_resp[\"status\"] == \"FAILED\":\n",
    "            raise RuntimeError(f\"Persona generation failed\")\n",
    "        time.sleep(10)\n",
    "\n",
    "# Generate three distinct user types\n",
    "angry_customer   = generate_persona(\"burnt_out_professional\", complexity=4)\n",
    "confused_user    = generate_persona(\"career_crisis_adult\",    complexity=3)\n",
    "manipulative_user = generate_persona(\"estranged_parent\",      complexity=4)\n",
    "\n",
    "print(\"\\nAll personas ready.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Inspect the angry customer profile\n",
    "profile = angry_customer[\"profile\"]\n",
    "\n",
    "print(\"=== Angry Customer Profile ===\")\n",
    "print(f\"Archetype:        {profile['archetype']}\")\n",
    "print(f\"Attachment style: {profile['psychological_core']['attachment_style']}\")\n",
    "print(f\"Core fear:        {profile['psychological_core']['core_fear']}\")\n",
    "print(f\"Defense mechanisms: {profile['psychological_core']['defense_mechanisms']}\")\n",
    "print(f\"Cognitive biases: {profile['psychological_core']['cognitive_biases']}\")\n",
    "print(f\"Current stressors: {profile['current_state']['stressors']}\")\n",
    "print(f\"Current mood:     {profile['current_state']['mood']}\")\n",
    "print()\n",
    "print(\"Communication style:\")\n",
    "cs = profile.get(\"communication_style\", {})\n",
    "for k, v in cs.items():\n",
    "    print(f\"  {k}: {v}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Compare belief states across the three user types\n",
    "BELIEF_KEYS = [\n",
    "    \"self_worth\", \"world_is_safe\", \"future_positive\",\n",
    "    \"partner_supportive\", \"own_agency\", \"relationships_stable\"\n",
    "]\n",
    "\n",
    "def extract_beliefs(persona_data):\n",
    "    beliefs = persona_data[\"profile\"].get(\"beliefs\", {})\n",
    "    return {k: beliefs.get(k, {}).get(\"value\", None) for k in BELIEF_KEYS}\n",
    "\n",
    "df_beliefs = pd.DataFrame({\n",
    "    \"Angry Customer (burnt_out)\": extract_beliefs(angry_customer),\n",
    "    \"Confused User (crisis)\": extract_beliefs(confused_user),\n",
    "    \"Manipulative User (estranged)\": extract_beliefs(manipulative_user),\n",
    "}).T\n",
    "\n",
    "df_beliefs.plot(\n",
    "    kind=\"bar\",\n",
    "    figsize=(12, 5),\n",
    "    title=\"Belief profiles by synthetic user type\",\n",
    "    colormap=\"Set2\",\n",
    "    ylim=(0, 1)\n",
    ")\n",
    "plt.axhline(0.5, color=\"gray\", linestyle=\"--\", alpha=0.4, label=\"Neutral (0.5)\")\n",
    "plt.xticks(rotation=15)\n",
    "plt.ylabel(\"Belief value (0=negative, 1=positive)\")\n",
    "plt.legend(loc=\"upper right\")\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 2 — Generate conversations starring these users\n",
    "\n",
    "Now we use the persona IDs to generate datasets where these specific users appear.\n",
    "This guarantees the psychological profile is carried through the entire conversation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def run_job(scenario_id, persona_a_id, count=5, label=\"\"):\n",
    "    \"\"\"Create and wait for a dataset generation job.\"\"\"\n",
    "    resp = requests.post(\n",
    "        f\"{BASE_URL}/jobs/dataset\",\n",
    "        headers=HEADERS,\n",
    "        json={\n",
    "            \"scenario_id\": scenario_id,\n",
    "            \"persona_a_id\": persona_a_id,\n",
    "            \"conversation_count\": count,\n",
    "            \"complexity\": 4,\n",
    "            \"language\": \"en\",\n",
    "            \"adapter\": \"flat_jsonl\",\n",
    "        }\n",
    "    ).json()\n",
    "    job_id = resp[\"data\"][\"jobId\"]\n",
    "    print(f\"  {label} → job {job_id}\")\n",
    "\n",
    "    while True:\n",
    "        result = requests.get(f\"{BASE_URL}/jobs/{job_id}\", headers=HEADERS).json()[\"data\"]\n",
    "        if result[\"status\"] == \"COMPLETED\":\n",
    "            return job_id, result[\"download_url\"]\n",
    "        elif result[\"status\"] == \"FAILED\":\n",
    "            raise RuntimeError(f\"Job {job_id} failed\")\n",
    "        time.sleep(15)\n",
    "\n",
    "# Scenario PRO-01: performance review — perfect stress test for difficult users\n",
    "jobs = {\n",
    "    \"angry_customer\":    run_job(\"PRO-01\", angry_customer[\"personaId\"],    label=\"Angry customer in performance review\"),\n",
    "    \"confused_user\":     run_job(\"PRO-03\", confused_user[\"personaId\"],     label=\"Confused user in career pivot scenario\"),\n",
    "    \"manipulative_user\": run_job(\"FAM-03\", manipulative_user[\"personaId\"], label=\"Manipulative user in boundary-setting scenario\"),\n",
    "}\n",
    "\n",
    "print(\"\\nAll jobs completed.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 3 — Inspect what makes each user distinct"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import urllib.request\n",
    "\n",
    "def load_dataset(download_url):\n",
    "    path, _ = urllib.request.urlretrieve(download_url)\n",
    "    conversations = []\n",
    "    with open(path) as f:\n",
    "        for line in f:\n",
    "            conversations.append(json.loads(line))\n",
    "    return conversations\n",
    "\n",
    "datasets = {name: load_dataset(url) for name, (_, url) in jobs.items()}\n",
    "\n",
    "# Compare communication act distributions per user type\n",
    "act_counts = {}\n",
    "for name, convs in datasets.items():\n",
    "    acts = []\n",
    "    for conv in convs:\n",
    "        for turn in conv[\"turns\"]:\n",
    "            if turn[\"speaker\"] == \"A\" and turn.get(\"communication_act\"):\n",
    "                acts.append(turn[\"communication_act\"])\n",
    "    act_counts[name] = pd.Series(acts).value_counts(normalize=True)\n",
    "\n",
    "df_acts = pd.DataFrame(act_counts).fillna(0)\n",
    "df_acts.plot(\n",
    "    kind=\"bar\", figsize=(14, 6),\n",
    "    title=\"Communication act distribution by synthetic user type (speaker A)\",\n",
    "    colormap=\"Set1\"\n",
    ")\n",
    "plt.xticks(rotation=45, ha=\"right\")\n",
    "plt.ylabel(\"Proportion of turns\")\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Show example turns from the angry customer\n",
    "print(\"=== Angry customer — example turns ===\")\n",
    "for conv in datasets[\"angry_customer\"][:1]:\n",
    "    for turn in conv[\"turns\"][:6]:\n",
    "        if turn[\"speaker\"] == \"A\":\n",
    "            print(f\"[{turn['communication_act']:20s}] {turn['text'][:120]}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Show example turns from the manipulative user\n",
    "print(\"=== Manipulative user — example turns ===\")\n",
    "for conv in datasets[\"manipulative_user\"][:1]:\n",
    "    for turn in conv[\"turns\"][:6]:\n",
    "        if turn[\"speaker\"] == \"A\":\n",
    "            print(f\"[{turn['communication_act']:20s}] {turn['text'][:120]}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 4 — Persona reuse: same user across multiple scenarios\n",
    "\n",
    "Persona IDs are stored for 7 days. This means you can run the **same user** through multiple scenarios\n",
    "and build a library of consistent synthetic users for your test suite."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Save persona IDs for reuse\n",
    "persona_library = {\n",
    "    \"angry_customer_v1\":    angry_customer[\"personaId\"],\n",
    "    \"confused_user_v1\":     confused_user[\"personaId\"],\n",
    "    \"manipulative_user_v1\": manipulative_user[\"personaId\"],\n",
    "}\n",
    "\n",
    "# Save to disk — reuse in future runs without re-generating\n",
    "with open(\"persona_library.json\", \"w\") as f:\n",
    "    json.dump(persona_library, f, indent=2)\n",
    "\n",
    "print(\"Persona library saved:\")\n",
    "for name, pid in persona_library.items():\n",
    "    print(f\"  {name}: {pid}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Reload and reuse in a new job (no re-generation cost)\n",
    "with open(\"persona_library.json\") as f:\n",
    "    saved_library = json.load(f)\n",
    "\n",
    "# Run the angry customer through a DIFFERENT scenario (ROM-02: breakup negotiation)\n",
    "print(\"Running angry customer through ROM-02...\")\n",
    "job_id_new, url_new = run_job(\n",
    "    scenario_id=\"ROM-02\",\n",
    "    persona_a_id=saved_library[\"angry_customer_v1\"],\n",
    "    count=3,\n",
    "    label=\"Angry customer in breakup scenario\"\n",
    ")\n",
    "print(f\"\\nNew dataset ready: {job_id_new}\")\n",
    "print(\"Same persona, different scenario — consistent psychological profile guaranteed.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 5 — Trust and tension dynamics by user type"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "fig, axes = plt.subplots(1, 3, figsize=(16, 5), sharey=True)\n",
    "titles = [\"Angry Customer\", \"Confused User\", \"Manipulative User\"]\n",
    "user_keys = [\"angry_customer\", \"confused_user\", \"manipulative_user\"]\n",
    "\n",
    "for ax, title, key in zip(axes, titles, user_keys):\n",
    "    convs = datasets[key]\n",
    "    for conv in convs[:5]:  # first 5 conversations\n",
    "        turns = conv[\"turns\"]\n",
    "        trust   = [t.get(\"relationship_state\", {}).get(\"trust\")   for t in turns]\n",
    "        tension = [t.get(\"relationship_state\", {}).get(\"tension\") for t in turns]\n",
    "        indices = list(range(len(turns)))\n",
    "        ax.plot(indices, trust,   color=\"#22c55e\", alpha=0.4, linewidth=1)\n",
    "        ax.plot(indices, tension, color=\"#ef4444\", alpha=0.4, linewidth=1)\n",
    "\n",
    "    ax.set_title(title, fontsize=12, fontweight=\"bold\")\n",
    "    ax.set_xlabel(\"Turn\")\n",
    "    ax.set_ylim(0, 1)\n",
    "    ax.axhline(0.5, color=\"gray\", linestyle=\"--\", alpha=0.3)\n",
    "\n",
    "axes[0].set_ylabel(\"Score (green=trust, red=tension)\")\n",
    "\n",
    "from matplotlib.lines import Line2D\n",
    "legend_elements = [\n",
    "    Line2D([0], [0], color=\"#22c55e\", linewidth=2, label=\"Trust\"),\n",
    "    Line2D([0], [0], color=\"#ef4444\", linewidth=2, label=\"Tension\"),\n",
    "]\n",
    "fig.legend(handles=legend_elements, loc=\"upper right\")\n",
    "fig.suptitle(\"Trust & Tension dynamics by synthetic user type\", fontsize=14, fontweight=\"bold\")\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Summary\n",
    "\n",
    "You now have a library of synthetic users you can deploy at scale:\n",
    "\n",
    "| User type | Archetype | Best for testing |\n",
    "|---|---|---|\n",
    "| Angry customer | `burnt_out_professional` | Complaint handling, de-escalation, emotional regulation |\n",
    "| Confused user | `career_crisis_adult` | Clarity of instructions, help flow, disambiguation |\n",
    "| Manipulative user | `estranged_parent` | Boundary enforcement, refusal handling, manipulation detection |\n",
    "\n",
    "**The key advantage:** these users behave consistently because they are driven by **belief state and relationship state** — not random LLM improvisation. The same persona ID always produces the same psychological baseline.\n",
    "\n",
    "**Next steps:**\n",
    "- Add these scenarios to your CI/CD pipeline with `adapter=\"openai_finetuning\"` to build regression test sets\n",
    "- Evaluate your agent's responses against StrataSynth ground truth using Notebook 2\n",
    "- Publish your synthetic user datasets to Hugging Face for reproducible benchmarking"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Bonus — V2 dimensional personas + free-text persona brief\n",
    "\n",
    "Two capabilities added in 2026:\n",
    "\n",
    "1. **`mode: \"dimensional\"`** — the persona's Big Five / Schwartz / attachment vector is sampled\n",
    "   from published country-calibrated norms (8 countries), stratified by age and gender, and exposed\n",
    "   back in a `psychometric` layer. Cultural conditioning (Hofstede + World Bank WGI) shapes how the\n",
    "   persona relates to institutions and pressure.\n",
    "2. **`overrides._persona_description`** — a free-text persona brief. Unambiguous demographic signals\n",
    "   (gender, age, profession) are extracted deterministically and the brief reaches the identity\n",
    "   generation prompt verbatim: *\"a 45-year-old plumber from Sevilla\"* produces exactly that person.\n"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "resp = requests.post(\n",
    "    f\"{HUMANS_URL}/humans/generate\",\n",
    "    headers=HEADERS,\n",
    "    json={\n",
    "        \"language\": \"en\",\n",
    "        \"mode\": \"dimensional\",\n",
    "        \"demographics\": {\"country\": \"GB\"},\n",
    "        \"overrides\": {\n",
    "            \"_persona_description\": (\n",
    "                \"A 52-year-old man from London, taxi driver for 20 years, \"\n",
    "                \"proud and stubborn, distrustful of apps taking a cut of his fares\"\n",
    "            )\n",
    "        },\n",
    "        \"seed\": 7,\n",
    "    },\n",
    ").json()\n",
    "human_id = resp[\"data\"][\"human_id\"]\n",
    "\n",
    "# Poll until READY, then inspect the psychometric layer\n",
    "import time\n",
    "while True:\n",
    "    h = requests.get(f\"{HUMANS_URL}/humans/{human_id}\", headers=HEADERS).json()[\"data\"]\n",
    "    if h[\"status\"] in (\"READY\", \"FAILED\"):\n",
    "        break\n",
    "    time.sleep(6)\n",
    "\n",
    "profile = h[\"profile\"]\n",
    "print(profile[\"demographics\"])                      # gender=male, age=52, profession='taxi driver'\n",
    "print(profile[\"psychegraph\"][\"psychometric\"][\"vector\"][\"big_five\"])  # sampled GB norms\n"
   ]
  }
 ]
}