# ChatGPT Survey Empirical Results Replication

This repository reproduces the empirical tables and figures for:

Can AI Help Reduce Prejudice? Evaluating the Effectiveness of AI-Powered Personalized Persuasion on Support for Transgender Rights

Authors: Charles Crabtree, John B. Holbein, Mitchell Bosley, Semra Sevi

The package includes de-identified analysis inputs, Python and R replication scripts, static figure assets, and optional LLM annotation code paths.

## Quick Start

```bash
chmod +x run_replication.sh
./run_replication.sh
```

By default, this creates the Python environment with `uv sync`, stages bundled
annotation inputs under `output/`, reruns the Python analyses, and runs the R
pipeline automatically when `Rscript` is available and `inputs/r/` is present.

## Paper-Facing Results

Use the results mode to regenerate the empirical table and figure bundle for the
paper main text and appendix:

```bash
BOOT_REPS=1000 N_CORES=1 ./run_replication.sh --results --run-r
```

or:

```bash
BOOT_REPS=1000 N_CORES=1 ./scripts/run_results.sh
```

This produces:

- `output/paper_outputs/` for main-text and appendix tables/figures, named in paper order;
- `output/results_artifacts/` for the shareable table/figure bundle and checksum manifest;
- `output/release/chatgpt-alignment-results-artifacts.zip` for the zip file to share.

## Smoke Testing And Bootstrap Runs

Some R outputs use bootstrap-based uncertainty estimates. Use low replication counts only for quick checks:

```bash
BOOT_REPS=10 N_CORES=1 ./run_replication.sh --results --run-r
```

For release-quality empirical outputs, use:

```bash
BOOT_REPS=1000 N_CORES=1 ./run_replication.sh --results --run-r
```

The R driver warns when `BOOT_REPS < 200`.

## Inputs

Bundled inputs live under `inputs/`:

- `inputs/analysis_dataset.csv`: minimal numeric analysis dataset with anonymized respondent IDs;
- `inputs/alignment_results_full.json`: bundled LLM annotation outputs keyed by anonymized respondent IDs;
- `inputs/alignment_results_postNov_runs1_openai.json`: optional OpenAI annotation replicate;
- `inputs/INPUTS_MANIFEST_SHA256.txt`: checksum manifest for bundled inputs;
- `inputs/r/`: de-identified R inputs used by `analysis/r/00_main_from_inputs.R`.

## Repository Layout

- `inputs/`: de-identified bundled inputs.
- `analysis/`: Python analysis scripts.
- `analysis/r/`: R analysis scripts.
- `scripts/`: release/export/smoke-test utilities.
- `static/figures/`: static figure assets copied into generated outputs.
- `docs/`: anonymization and release-gate notes.
- `output/`: generated files; safe to delete and regenerate.

## Optional Annotation Regeneration

The empirical outputs can be reproduced with the bundled annotation JSON files. To regenerate annotation JSONs with Gemini or OpenAI, place the required local transcript export at `data/dec_3_2024_wave_1.csv`, set the relevant API key, and run:

```bash
export GEMINI_API_KEY=...
./run_replication.sh --annotate gemini
```

or:

```bash
export OPENAI_API_KEY=...
./run_replication.sh --annotate openai
```

## Outputs

Generated artifacts land under `output/`. The useful release surfaces are:

- `output/paper_outputs/`: readable generated tables and figures.
- `output/results_artifacts/`: the same paper-facing outputs plus checksum manifests.
- `output/release/chatgpt-alignment-results-artifacts.zip`: the shareable zip.
