How we check our results
You have to defend every number to a supervisor, an examiner or a journal. So every statistic in Researchite is checked against established software — mostly R — on the same data, and must give the same answer.
687 automated checks in 11 areas · all passing when last run on 2 November 2026
The rule
For each calculation, we run the same data through standard software and record its answer. Researchite’s answer must match it — usually to within 0.01% of R’s value. The checks run again whenever that part of Researchite changes, before it’s released. If one fails, the change doesn’t go out.
What is checked, and against what
| Area | Compared with | Data | Checks |
|---|---|---|---|
| Sample size (20 methods) | R’s power.t.test, standard t-tables and textbook worked examples (e.g. Fleiss) | Worked examples | 178 |
| Tests and tables (chi-square, Fisher, t-test, Mann–Whitney, ANOVA, Kruskal–Wallis, correlation) | R: chisq.test, t.test, wilcox.test, kruskal.test, cor.test; published reference values | A made-up survey | 57 |
| Regression (logistic, linear, Poisson, negative binomial, modified Poisson, multinomial, ordinal) | R: glm, lm, MASS::glm.nb, the HC0 sandwich, nnet::multinom, MASS::polr, brant | A made-up dataset | 103 |
| Survey design (weights, clusters, strata) | R’s survey package 4.5: svymean, svyciprop, svychisq, svyttest, svyglm, regTermTest | A made-up two-stage survey | 59 |
| Multilevel models (ICC, MOR, PCV) | R’s lme4 2.0: glmer (Laplace) and lmer (REML) | The same two-stage survey | 25 |
| Survival analysis | R’s survival package: survfit, survdiff, coxph (Efron ties), cox.zph; poisson.test | A made-up follow-up study | 26 |
| Scale validation (EFA, CFA, SEM) | R 4.4: psych, GPArotation, factanal, lavaan, semTools | Published data: Holzinger & Swineford (1939); Bollen (1989) | 34 |
| Meta-analysis, prediction-model and diagnostic-accuracy reviews | R: metafor 4.8; mada 0.5.12 (bivariate/HSROC); bayesmeta 3.5 (Bayesian random effects) | Ten made-up studies; the published AUDIT-C data | 91 |
| Intercoder agreement (Cohen’s kappa) | R’s irr 0.84 (kappa2) | Two made-up coded transcripts | 21 |
| Stata, R and SPSS commands shown with results | The R commands are run in R and must give the same estimates and p-values | The example survey | 31 |
| Reading Stata and SPSS files | Files written by R’s haven (Stata 8, 13, 14, 15; SPSS .sav and .zsav) | Test files | 62 |
| All areas | 687 | ||
“Made-up” data are small invented datasets built to exercise each method, including awkward cases: missing values, zero cells, ties and tiny groups. Where a classic published dataset exists, we use it, so anyone can repeat the comparison.
Examples from the checks
A few of the values Researchite must reproduce. The numbers are R’s; Researchite’s answer has to match within the tolerance shown.
| Calculation | R gives | Must match |
|---|---|---|
| Meta-analysis: random effects (REML), odds ratio, 10 studies | metafor: pooled log OR 0.5160 (SE 0.1405), τ² 0.0832, I² 46.5% | within 0.001% |
| Bayesian meta-analysis: half-normal(0.5) prior on τ, odds ratio, 10 studies | bayesmeta: posterior median log OR 0.5033 (95% CrI 0.2258–0.8577), τ 0.2874 | within 0.005% |
| Diagnostic accuracy: bivariate model, AUDIT-C, 14 studies | mada: sensitivity 0.891 (0.808–0.941), Λ 3.262, β −0.610 | within 0.001% (the model’s estimates) |
| Survey design: weighted percentage with ANC4+ | survey: 53.42% (95% CI 46.91–59.82%), design effect 1.910 | within 0.01% |
| Survival: log-rank test across three groups | survival: χ² 24.896, 2 df, p 3.9 × 10⁻⁶ | within 0.001% (χ²), 0.1% (p) |
| Multilevel logistic: random intercept for 40 clusters | lme4: cluster variance 0.2758, ICC 7.73%, MOR 1.650 | within 0.3% (see below) |
| Logistic regression: model fit | glm: likelihood-ratio p 1.15 × 10⁻⁵, Nagelkerke R² 0.0944 | within 0.01–0.1% |
| Intercoder agreement: pooled over all codes | irr: κ = 0.5973 | to 9 decimal places |
| Scale validation: sampling adequacy, Holzinger & Swineford | psych: KMO 0.752 | to 6 decimal places |
Where small differences can come from
- Iterative models stop at a tolerance. R’s glmer stops searching a little earlier than Researchite does, so multilevel logistic results agree within 0.3% rather than 0.01%. For a few REML meta-analyses with very small between-study variance, we compared with metafor at a stricter setting than its default; likewise bayesmeta, whose default grid is coarser than Researchite’s (about 0.5% on the prediction interval).
- Choices that software makes differently. For example, adding 0.5 to zero cells (we follow metafor and mada), or a continuity correction in a sample size formula. Researchite shows its choice — beside the result, or in the methods text it writes — so your report can say exactly what was done.
- Rounding in the display. Tables show two or three decimals; the calculation keeps full precision.
Check it yourself
- Results in Data and analysis come with the Stata, R and SPSS commands that give them. Run those commands on your data in your own software and compare.
- The sample size calculator shows the formula and the reference for each method, so the number can be worked by hand. Our guides work several examples step by step.
- Meta-analyses state the method and the estimator, so the same data can be pooled in R’s metafor or in RevMan.
What isn’t checked this way
Some parts of Researchite aren’t calculations, so they can’t be checked against other software:
- Transcription and translation are done by AI. Accuracy depends on the audio, accents and people speaking over each other. Before analysis, read a sample of each transcript against your recording. Researchite deletes the audio after transcribing it, so keep your own copy. We don’t yet publish accuracy figures by language.
- AI suggestions — codes, drafts, screening decisions, translations — are only ever suggestions. Nothing enters your study until you accept it, and the methods text says where AI assisted.
- References are built from each paper’s PubMed, Europe PMC or Crossref record, never written by AI.
Found a difference?
If a result in Researchite doesn’t match your own software, tell us at support@researchite.com with the analysis and the software you compared with. We’ll look into it straight away, and if we’re wrong, we’ll fix it and add the case to the checks.