Experiment Planner

Initializing WebR... Please wait.

WDD Counts Planning

Estimate reduction precision for place-based interventions. Use Advanced Settings to set the confidence interval level, and whether to use historical pre-data and/or unit time estimates.

Advanced Settings
TimeEstimateSELowerUpper

IRR Counts Planning

Estimate IRR precision for comparing count rates. If pre-time is not zero, it changes the estimator to a Difference-in-Differences relative changes estimate, which is useful when control and treated have different baseline counts.

Note: If Pre-time is 0, Control Count is used as the baseline for both groups.

TimeIRR EstSE (log)LowerUpper

Proportion Events Planning

Plan for binary outcomes like clearance rates. Sample sizes are the number of events observed in a period; for example, 100 crimes in the control area and 50 in the treated area.

TimeEstimateSELowerUpper

Simulation Proportion Events

Simulate power with varying baseline rates across categories. The Odds Ratio target represents a consistent proportional increase (e.g., Odds Ratio of 1.5 increases 10% to 15% and 6% to 9%).

Note: Baseline is the distribution of crimes in the sample (e.g. violent crime=0.2, property crime=0.8); these can add to more than 1 as they will be normalized. Proportion is the baseline outcome rate for that category (e.g. 10% clearance for property, 50% for violent crimes).

Total NORR TargetSim ORRBiasSE

Help & Methodology

Citation References

If you use these tools or the WDD statistic in your research, please use the following citations:

For a mathematical write up of the methods, as well as how I used AI to help create this application, see the github repo.

WDD Counts Planning

The Weighted Displacement Difference (WDD) tool is designed for planning experiments involving crime counts. It helps estimate the precision (Standard Error) of crime reduction estimates over time.

  • Baseline Counts: The average number of crimes per unit time (e.g., crimes per week) expected in both areas.
  • Treated Reduction: The absolute decrease in crimes you expect the treatment to cause per unit time.
  • Pre-time Periods: The number of historical time periods used for the baseline variance calculation. If set to 0, it assumes no historical data is available.
  • Unit Time: If checked, results are shown as rates per period. If unchecked, results show the cumulative reduction across the study duration.
  • Video Demo: Watch WDD Demo

IRR Counts Planning

The Incident Rate Ratio (IRR) tool uses the Wilson log IRR estimator. It is suitable for comparing crime rates between two areas, with or without historical (pre) data. This tool is valid if you expect a percentage reduction, but the control and treated areas have very different total crime counts (for the DiD estimate). For the continuous monitoring over time though with no pre-treated, treated and control should have similar crime counts.

  • Control/Treated Count: The baseline number of crimes per unit time in each area.
  • IRR Target: The expected ratio of the treated rate to the control rate (e.g., 0.8 for a 20% reduction).
  • Pre-time Periods: If greater than 0, the tool calculates a Difference-in-Differences (DiD) log IRR. If 0, it performs a simple post-period comparison.
  • Video Demo: Watch IRR Demo

Proportion Events Planning

This tool is used for planning experiments where the outcome is a proportion, such as arrest rates per crime incident. It is useful if the events have a consistent rate, like use of force, across treated and control.

  • Control/Treated Proportion: The expected probability of an event in each group (e.g., 0.4 for a 40% arrest rate).
  • Sample Size per Period: The number of crime incidents occurring in each arm of the experiment per time unit. This is used to determine the total N as time progresses.
  • Alpha Level: The significance level (default 0.05 for 95% confidence intervals).
  • Video Demo: Watch Proportion Demo

Simulation Proportion Events

A simulation-based approach using logistic regression to plan for experiments with multiple crime types and varying baseline solution rates. This is useful if the baseline events have different rates (like clearances for different crimes), but you expect the treatment effect to be constant in terms of an Odds Ratio.

The Odds Ratio applied is a specific increase across all sub-categories. For example, if you set an ORR of 1.5, and the baseline clearance rate for one category is 10% (0.1) and another is 6% (0.06), the treatment will increase the clearance rates to approximately 15% and 9% respectively. The simulation estimates a logistic regression equation of the form:

logit(P(Event)) = β0 + Σ βi·CrimeTypei + βtreat·Treat

Where:

  • Fixed Effects: The βi terms are fixed effects for each crime type, accounting for different baseline solve rates.
  • Treatment Effect: The βtreat term represents the consistent log-odds ratio reduction (or increase) applied across all crime types (calculated as ln(Target IRR)).
  • Bias & SE: By replicating this model many times, the tool estimates the potential bias and the standard error of the treatment effect at various sample sizes.

General Notes

This application runs entirely in your browser using R-WASM (WebR). Calculations are performed locally on your machine. Standard errors and confidence intervals are calculated based on the formulas described in the Crime De-Coder blog posts.

References & Further Reading