# Word sources and selection rationale This document explains **which words live in the AGS Vocab bank, where they came from, and why we chose those sources**. It is the authoritative defence of the bank's composition — written so that schools, parents, and academic reviewers can verify that the vocabulary tested is appropriate, well-grounded, and not arbitrary. > **TL;DR.** The bank combines (1) a *hand-curated* core covering grade-band > milestones, (2) imports from peer-reviewed corpus-frequency lists for > breadth, and (3) culturally relevant additions (Singlish). Every word row > records the source it came from (`words.source` column), so any claim > about coverage can be traced back to a citation. --- ## 1. Why this matters A vocabulary assessment is only as defensible as its bank. If a school administrator is told their Grade-5 cohort scored a Vocabile of 580, the score's meaning depends entirely on what was in the test. Two failure modes to avoid: - **Arbitrary selection** — "the developer's favourite words" → score reflects whether learners share the developer's idiolect, not actual vocabulary breadth. - **Mono-source bias** — one wordlist's quirks (e.g. British vs American register, academic vs everyday) over-represent themselves in the score. We address both by pulling from multiple, independently-validated sources and tracking provenance per row. --- ## 2. Sources used The bank pulls from five complementary sources. Each row in `words` carries a `source` column identifying which one (or `null` if hand-curated). Rank within the source is in `source_rank` so admins can query e.g. "show me the top 500 of NGSL that we haven't yet seeded definitions for." ### 2.1 Curated core (`source = null`) **Size**: ~1,700 entries. **Coverage**: K through BEYOND, spanning every Vocabile grade band. **Defence**: Hand-written by the project authors against a fixed schema ([POS / grade band / difficulty tier / definition]). Each entry carries an intentional level assignment based on: - Standard English-curriculum readability benchmarks for the grade - Anchor difficulty tier (`common` → `rare`) calibrated against the IRT difficulty offset (`word_difficulty_levels.theta_anchor`) - Plain-English definitions targeted at the learner's grade level (so a G3 learner reading a definition of a G3 word is not blocked by vocabulary in the gloss itself) This is the *primary* source for the Diagnostic test — its IRT items have been authored carefully and definitions verified against reading level. ### 2.2 NGSL — New General Service List **Citation**: Browne, C., Culligan, B. & Phillips, J. (2013). *The New General Service List.* https://www.newgeneralservicelist.com/ **Size**: 2,801 word families covering ~92% of running text in modern written English. **Corpus**: 273-million-word *Cambridge English Corpus*. **License**: Freely distributable for educational and research use, with attribution. **Why we use it**: NGSL is the modern successor to West's (1953) General Service List — the canonical "high-coverage everyday vocabulary" list in English-language pedagogy. Items are ranked by both frequency AND *dispersion* (how widely distributed across genres), avoiding the trap of including words that are very frequent in one register but unknown in others. **Defence for our use**: We import NGSL to ensure the Diagnostic doesn't miss high-frequency words that the bulk hand-curation might overlook. A learner who *cannot* recognise NGSL #500 is meaningfully different from one who can — that distinction matters for ability discrimination at the mid-band. ### 2.3 AVL — Academic Vocabulary List **Citation**: Gardner, D. & Davies, M. (2014). *A New Academic Vocabulary List.* Applied Linguistics 35(3): 305-327. **Size**: 3,015 lemmas of "academic words" — defined as words used substantially more often in academic prose than in general English. **Corpus**: 120-million-word academic sub-corpus of COCA (Corpus of Contemporary American English). **License**: Freely available; methodology open-published. **Why we use it**: NGSL covers everyday speech and prose; AVL covers the *Tier 2* academic vocabulary that secondary and tertiary-bound learners must master — words like `analyze`, `evaluate`, `hypothesise`, `derive`. **Defence for our use**: Our target population includes upper-secondary learners (Grade 9-12). AVL discriminates ability among learners reading academic material — exactly the right tail of the Vocabile scale. ### 2.4 Dale-Chall (3000 familiar words) **Citation**: Dale, E. & Chall, J. S. (1948 / revised 1995). *The Dale-Chall List of Familiar Words.* **Size**: ~3,000 words known by 80% or more of US 4th-grade students. **Corpus**: Empirically validated against thousands of students; this list is the basis of the Dale-Chall readability formula still used today. **License**: The list itself is widely reproduced under fair use; the formula is uncopyrightable as a mathematical procedure. **Why we use it**: Anchors the *bottom* of the bank — words a typical 9-year-old already knows. Critical for Diagnostic items that target the K-G4 range, where IRT requires very-low-difficulty items to confirm a floor. **Defence for our use**: This is the most empirically validated low-tier list available. Imported with `--default-level=G3 --default-difficulty=common`. ### 2.5 Singlish (`source = curated`, tagged `is_singlish=true`) **Size**: 18 entries. **Why it's here**: AGS Vocab is a Singapore-headquartered product. Local learners encounter Singlish daily (`kiasu`, `shiok`, `makan`, `paiseh`); excluding it would mis-measure their actual vocabulary. Admins who deploy in non-Singapore contexts disable it via the `show_singlish` config toggle, in which case Singlish items are filtered out of every test. --- ## 3. Why not other sources? Sources we *considered and rejected*, with reasons: - **GSL (West, 1953)** — superseded by NGSL; the underlying corpus is pre-television-era and dramatically misses modern usage. - **COCA top-N raw frequency** — raw frequency favours function words (`the`, `of`, `and`) that don't make useful vocab test items. NGSL is COCA-equivalent but already deduplicated for function words. - **Oxford 3000 / 5000** — excellent quality, but the list and its definitions are Oxford's intellectual property; redistribution restrictions make it unsuitable for a self-hosted school deployment. - **Roget's Thesaurus** — organised by semantic field, not by frequency or grade level; doesn't map cleanly to our level-banded IRT pool. - **Wikipedia article titles** — dominated by proper nouns and domain-specific jargon; not vocabulary in the assessment sense. --- ## 4. Composition (current) Run: ```sql SELECT COALESCE(source, '(curated)') AS src, COUNT(*) AS n FROM words GROUP BY src ORDER BY n DESC; ``` (Filament admins can also see this at `/admin/words` filtered by `source`.) The bank composition is intentionally **curated-heavy at the upper grade bands** (G9-BEYOND) where definition quality matters most for fairness, and **import-heavy at the everyday tier** where validated frequency rankings are more authoritative than hand-curation could be at scale. --- ## 5. The import pipeline ### 5.1 Run an import ```pwsh php artisan vocab:import path/to/source.csv \ --source=ngsl \ --default-level=G5 \ --default-difficulty=familiar ``` Flags: - `--source=` — the canonical code to write into `words.source`. - `--default-level=` — Vocabile level code (K, G1, …, G12, BEYOND) used when a row doesn't specify its own. - `--default-difficulty=` — common/familiar/academic/advanced/sophisticated/rare, used when a row doesn't specify. - `--overwrite-source` — by default, importing does NOT overwrite the `source` column on rows already tagged as curated (so we preserve the authority of hand-written items even when an external list re-introduces the same lemma). Pass this flag to force overwrite. - `--dry-run` — read and validate without writing. ### 5.2 CSV format ``` lemma,pos,vocabile_level,difficulty,source_rank,definition ephemeral,adj,G11,sophisticated,15,Lasting only for a very short time ``` The header row is required. All columns except `lemma` are optional; missing values fall back to the `--default-*` flags. Lines starting with `#` are ignored, so you can comment your imports inline. ### 5.3 Idempotence Imports key on `lemma` (unique). Re-running an import: - Updates POS / level / difficulty / source_rank / definition (if non-null in the CSV). - Preserves the `source` of curated rows unless `--overwrite-source` is passed. - Never duplicates rows. ### 5.4 Where to get the actual full lists - **NGSL**: https://www.newgeneralservicelist.com/ — download the spreadsheet, save the lemma + rank columns as CSV with our header, run `--source=ngsl`. - **AVL**: https://www.academicvocabulary.info/ — same conversion, run `--source=avl --default-level=G9 --default-difficulty=academic`. - **Dale-Chall**: widely reproduced online; pick a copy you trust. `--source=dale_chall --default-level=G3 --default-difficulty=common`. These full imports add an estimated ~5,500 unique lemmas net of overlap with the curated core. After all three, the bank covers roughly the working recognition vocabulary of a native English speaker through age 12-14, with the hand-curated upper tier covering ages 15-18. --- ## 6. What we are NOT claiming Honest scope of the bank, so the score isn't over-interpreted: 1. **Not a full lexical inventory.** Native English speakers at age 18 recognise 40,000-60,000 word families. The bank holds ~2,000-10,000. The Vocabile score discriminates ABILITY via adaptive sampling — it does not measure "how many words you know" as a raw count. 2. **Not multilingual.** Other than the explicit Singlish set, items are English-only. Bilingual learners' L1 vocabulary is not represented. 3. **Not domain-specific.** A learner who reads constant football news has rich football vocabulary that the bank ignores. We test the *generalist* register. 4. **Not balanced for cultural register.** US, UK, Australian, and Singaporean English are all present, but distribution is uneven; an item that uses British register may be slightly harder for US-school cohorts at the margin. Calibration over real responses will reduce this bias over time. --- ## 7. Updating this document Whenever you import a new source, add a sub-section under §2 with: - Full citation (author, year, work, URL) - Size + corpus - License - Rationale for inclusion Whenever you reject a candidate source, add it to §3 with a one-sentence reason — so future maintainers don't re-litigate the same decision.