Mandarin Tones is a self-hosted course for adults learning Mandarin Chinese, at https://mandarin.503mail.com. It covers the 149 words of HSK 1 (the first official level) in 15 lessons, each with a short two-person conversation and a listening quiz. You speak into the microphone and it checks both the word and each tone. This post is the complete rebuild guide, and every block has a Copy button.
What it does
- Words: each lesson has 10 words. Each word card shows the characters, pinyin with tone marks, pinyin with tone numbers (ni3 hao3, which reads clearly with a screen reader) and the meaning. Buttons: Listen, Listen slowly, Say it, and "hear my pitch / the teacher's pitch as a hum", so you can hear the shape of a tone without seeing a graph.
- Conversation: 6 to 8 lines between 王方 Wáng Fāng (female voice) and 李明 Lǐ Míng (male voice), built from the lesson's words. Listen to the whole conversation, or take one part: the other person's lines play by themselves, and you say yours. There's an optional challenge that hides the Chinese and shows only the English.
- Quiz: 10 listening questions per lesson: what does it mean, and which pinyin has the right tones.
- Tone setup: you say a high "mā" and a low "mǎ", so the checker knows your voice range.
- Tone drill: hear a sound and pick its tone, or say a sound with a given tone.
- A high beep means "speak now", a low beep means it stopped listening. Every button is labeled, and results are read out through a live region, so it works with a screen reader.
How it works
- The browser page (plain HTML and JavaScript, no build step) plays teacher clips. These are MP3 files recorded ahead of time, so nothing is generated while you learn.
- "Say it" records from the microphone until you've been quiet for about 0.9 seconds.
- Tone check, in the browser: the YIN pitch tracker measures your pitch every 10 ms and splits the recording into syllables. A small model then decides tone 1, 2, 3 or 4 from the shape of each syllable, measured against your own range from the tone setup. Takes well under 0.1 s.
- Word check, on the server: the same recording goes to
/api/recognizeon CT 309 as 16 kHz audio. Vosk (an offline speech recognizer) only listens for the course's own words, so a wrong word lands on another course word instead of being "corrected". About 0.2 to 0.5 s. The audio is never saved or logged. - Conversation lines are compared by sound (pinyin without tones), so sound-alike characters such as 他 and 她 (both "tā") don't count as wrong. A line passes when 80% of its sounds are heard in order.
- Progress is saved in the browser only (localStorage). There are no accounts.
Containers involved
| Container | Role | LAN IP |
|---|---|---|
| CT 108 (Caddy reverse proxy) | HTTPS for mandarin.503mail.com, passes everything to CT 309 | x.x.0.122 |
| CT 309 (mandarin) | Local Caddy (static site + /api/*), Vosk word check on port 8000, voice rendering tools |
x.x.0.184 (DHCP) |
CT 309 is Debian 12, unprivileged, with nesting, 2 cores, 2 GB RAM and an 8 GB disk. It uses about 1.5 GB of that: the Python venv, the 42 MB Vosk model, two 63 MB Piper voices and about 2,400 MP3 clips. DNS needs nothing new: *.503mail.com already points at home.
The teacher voices
Every course word is recorded in 6 voices, at normal and slow speed:
- Microsoft (edge-tts): Xiaoxiao (female), Xiaoyi (female), Yunxi (male), Yunyang (male). The edge-tts tool sends each word's text to Microsoft's online read-aloud service once, when it's recorded. The MP3s are then stored on CT 309, and lessons never contact Microsoft. Your voice never goes to Microsoft.
- Piper (fully offline): chaowen (male) and xiao_ya (female). These two take pinyin directly, so the script gives them the exact tones.
Measured on the teacher clips themselves, the Microsoft voices make real tones: "má" rises about 5.5 steps and "mà" falls about 5. The Piper voices are almost flat on single syllables: their "mà" falls less than 1 step. So the default is Xiaoxiao and Yunxi taking turns, and the Piper voices and Yunyang are labeled "tones are weaker" in Settings. Conversation lines exist only in their speaker's voice (王方 = Xiaoxiao, 李明 = Yunxi).
Where to run each command
- Proxmox host shell: in the Proxmox web UI click pve → Shell (or SSH to the host as root).
- Inside CT 309 (mandarin): Proxmox web UI → 309 (mandarin) → Console, or
pct enter 309from the host shell. - Inside CT 108 (Caddy reverse proxy): Proxmox web UI → 108 (caddy) → Console.
Rebuild, step by step
1. Make a working folder on the host
Proxmox host shell:
mkdir -p /root/mandarincd /root/mandarin2. Create every file
Paste each block into the Proxmox host shell, from inside /root/mandarin. Each one writes one file.
create-ct.sh: creates and starts CT 309.
cat > /root/mandarin/create-ct.sh <<'EOF'
#!/bin/bash
# Creates CT 309 "mandarin" (Debian 12): Caddy web + Vosk Mandarin speech check + voice rendering.
set -e
pct create 309 local:vztmpl/debian-12-standard_12.12-1_amd64.tar.zst \
--hostname mandarin --cores 2 --memory 2048 --swap 512 \
--rootfs local-lvm:8 --net0 name=eth0,bridge=vmbr0,ip=dhcp \
--features nesting=1 --unprivileged 1 --onboot 1 --ostype debian \
--timezone America/New_York
pct start 309
sleep 8
pct exec 309 -- ip -4 -o addr show eth0
EOFsetup-ct.sh: runs inside CT 309: installs Caddy, Vosk, edge-tts and Piper, plus the Mandarin model and the two Piper voices.
cat > /root/mandarin/setup-ct.sh <<'EOF'
#!/bin/bash
# Runs INSIDE CT 309. Installs Caddy, a Python venv with vosk + edge-tts + piper (with its Chinese
# add-on), the Vosk Mandarin model and the two Piper voices that take pinyin (chaowen, xiao_ya).
set -e
export DEBIAN_FRONTEND=noninteractive
apt-get update
apt-get install -y python3-venv python3-pip curl unzip caddy lame ffmpeg
mkdir -p /opt/mandarin /var/www/mandarin
python3 -m venv /opt/mandarin/venv
/opt/mandarin/venv/bin/pip install --upgrade pip
/opt/mandarin/venv/bin/pip install vosk edge-tts "piper-tts[zh]" unicode_rbnf
cd /opt/mandarin
if [ ! -d model-cn ]; then
curl -sSLo model.zip https://alphacephei.com/vosk/models/vosk-model-small-cn-0.22.zip
unzip -q model.zip
mv vosk-model-small-cn-0.22 model-cn
unlink model.zip
fi
mkdir -p voices
cd voices
for v in chaowen xiao_ya; do
for f in zh_CN-$v-medium.onnx zh_CN-$v-medium.onnx.json; do
[ -f "$f" ] || curl -sSLo "$f" "https://huggingface.co/rhasspy/piper-voices/resolve/main/zh/zh_CN/$v/medium/$f"
done
done
ls -la /opt/mandarin /opt/mandarin/voices
echo SETUP-DONE
EOFCaddyfile: CT 309's own Caddy: /api/* goes to the word check, everything else is the static site.
cat > /root/mandarin/Caddyfile <<'EOF'
# CT 309 local Caddy (behind the CT 108 edge). Static site + /api/* -> speech check on :8000.
:80 {
handle /api/* {
reverse_proxy 127.0.0.1:8000
}
handle {
root * /var/www/mandarin
file_server
}
}
EOFmandarin_speech.py: the word check server (Vosk, limited to the course's words).
cat > /root/mandarin/mandarin_speech.py <<'EOF'
#!/opt/mandarin/venv/bin/python3
# Mandarin word check (CT 309). The app records the learner saying one word and posts it here.
# POST /api/recognize?word=你好 body = 16 kHz mono 16-bit little-endian PCM
# -> {"heard": "你好", "ok": true}
# POST /api/recognize?line=你好！ (conversation line) -> {"heard": "你好", ...}; the app scores it
# Vosk only listens for the course's own words (+ [unk]), so a wrong word lands on another course
# word (or nothing) instead of being "corrected". Tones are checked in the browser, not here.
# Audio is never written to disk or logged.
import json, os, urllib.parse
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from vosk import Model, KaldiRecognizer, SetLogLevel
SetLogLevel(-1)
MODEL = Model(os.environ.get('VOSK_MODEL', '/opt/mandarin/model-cn'))
RATE = 16000
MAX_BYTES = RATE * 2 * 15 # 15 seconds
def phrase(word):
"""Grammar entry for a word: the word itself if the model knows it, else its characters."""
if MODEL.vosk_model_find_word(word) != -1:
return word
return ' '.join(c for c in word if MODEL.vosk_model_find_word(c) != -1)
with open(os.environ.get('COURSE', '/var/www/mandarin/course.json'), encoding='utf-8') as f:
WORDS = sorted({w['hz'] for L in json.load(f)['lessons'] for w in L['words']})
PHRASES = {w: phrase(w) for w in WORDS}
BACK = {p: w for w, p in PHRASES.items() if p}
GRAMMAR = json.dumps(sorted(set(BACK)) + ['[unk]'], ensure_ascii=False)
class Handler(BaseHTTPRequestHandler):
def do_GET(self):
if urllib.parse.urlparse(self.path).path in ('/api/health', '/health'):
self.reply(200, {'ok': True, 'words': len(BACK)})
else:
self.reply(404, {'error': 'not found'})
def do_POST(self):
url = urllib.parse.urlparse(self.path)
if url.path not in ('/api/recognize', '/recognize'):
return self.reply(404, {'error': 'not found'})
size = int(self.headers.get('Content-Length') or 0)
if size <= 0 or size > MAX_BYTES:
return self.reply(413, {'error': 'audio missing or too long'})
pcm = self.rfile.read(size)
query = urllib.parse.parse_qs(url.query)
want = query.get('word', [''])[0].strip()
line = query.get('line', [''])[0].strip() # a conversation line: also listen for its single characters
grammar = GRAMMAR
extra = set()
if want and want not in PHRASES and phrase(want): # a word outside the course list
extra.add(phrase(want))
extra.update(p for p in (phrase(c) for c in line if '一' <= c <= '鿿') if p)
if extra:
grammar = json.dumps(sorted(set(BACK) | extra) + ['[unk]'], ensure_ascii=False)
rec = KaldiRecognizer(MODEL, RATE, grammar)
rec.AcceptWaveform(pcm)
text = json.loads(rec.FinalResult()).get('text', '').replace('[unk]', '').strip()
heard = BACK.get(text, text.replace(' ', ''))
self.reply(200, {'heard': heard, 'ok': bool(want) and heard.replace(' ', '') == want})
def reply(self, code, obj):
body = json.dumps(obj, ensure_ascii=False).encode()
self.send_response(code)
self.send_header('Content-Type', 'application/json; charset=utf-8')
self.send_header('Cache-Control', 'no-store')
self.send_header('Content-Length', str(len(body)))
self.end_headers()
self.wfile.write(body)
def log_message(self, fmt, *args):
pass # no request logs: these are people's voices
if __name__ == '__main__':
ThreadingHTTPServer(('127.0.0.1', 8000), Handler).serve_forever()
EOFmandarin-speech.service: runs it as a service (as the nobody user).
cat > /root/mandarin/mandarin-speech.service <<'EOF'
[Unit]
Description=Mandarin word check (Vosk)
After=network.target
[Service]
ExecStart=/opt/mandarin/venv/bin/python3 /opt/mandarin/mandarin_speech.py
Restart=always
User=nobody
[Install]
WantedBy=multi-user.target
EOFrender-course.py: records every word, tone-drill sound and conversation line in all the teacher voices. It skips clips that already exist, so you can run it again after adding words.
cat > /root/mandarin/render-course.py <<'EOF'
#!/opt/mandarin/venv/bin/python3
# Records every course word + tone-drill syllable with 4 Microsoft voices (edge-tts, from the
# characters; "say" in course.json overrides the text) and the two pinyin Piper voices (pinyin goes
# straight to the voice with spoken tones: 3+3 -> 2+3, 不 before tone 4 -> bu2).
# /var/www/mandarin/audio/<voice>/<speed>/<id>.mp3 speed n / s
# voice x Xiaoxiao, i Xiaoyi, y Yunxi, g Yunyang (Microsoft); f xiao_ya, m chaowen (Piper)
# id = hex of the UTF-8 text: the characters for words, "py:ma1" for drill syllables.
import asyncio, json, os, re, subprocess, sys, wave
import numpy as np
from piper import PiperVoice
from piper.config import SynthesisConfig
from piper.phonemize_chinese import _split_initial_final_tone
WEB = '/var/www/mandarin'
VOICES = {'m': 'chaowen', 'f': 'xiao_ya'}
SPEEDS = {'n': 1.0, 's': 1.6}
# Microsoft voices (edge-tts): text goes to Microsoft once per clip; the MP3s are kept here.
# They read characters (they apply tone changes themselves), so drill syllables use a character.
EDGE = {'x': 'zh-CN-XiaoxiaoNeural', 'i': 'zh-CN-XiaoyiNeural', 'y': 'zh-CN-YunxiNeural', 'g': 'zh-CN-YunyangNeural'}
EDGE_RATE = {'n': '-10%', 's': '-40%'}
DRILL_HZ = {'ma': '妈麻马骂', 'yi': '衣姨椅意', 'ba': '八拔把爸', 'tang': '汤糖躺烫',
'shi': '诗十使是', 'wu': '屋无五雾', 'qi': '七骑起气', 'fan': '翻烦反饭'}
DRILL = [f'{s}{t}' for s in DRILL_HZ for t in (1, 2, 3, 4)]
def spoken(word):
"""Citation pinyin -> the tones actually said (same rules as app.js spokenTones)."""
syl = word['py'].split()
tones = [int(s[-1]) for s in syl]
for i in range(len(syl) - 1):
if word['hz'][i:i + 1] == '不' and tones[i + 1] == 4:
tones[i] = 2
elif tones[i] == 3 and tones[i + 1] == 3:
tones[i] = 2
return [s[:-1] + str(t) for s, t in zip(syl, tones)]
def phonemes(sylls):
out = []
for syl in sylls:
syl = syl.replace('ü', 'v')
m = re.match(r'^([a-z]+[^e])r([1-5])$', syl) # erhua: nar3 -> na3 + er5
parts = [m.group(1) + m.group(2), 'er5'] if m and syl != 'er' else [syl]
for p in parts:
ini, fin, tone = _split_initial_final_tone(p)
if not fin:
sys.exit(f'bad pinyin syllable: {p}')
out += [ini or 'Ø', fin, tone]
return out + ['。']
def render(voice, sylls, speed, mp3):
ids = voice.phonemes_to_ids(phonemes(sylls))
audio = voice.phoneme_ids_to_audio(ids, SynthesisConfig(length_scale=SPEEDS[speed]))
pcm = (np.clip(audio, -1, 1) * 32767).astype(np.int16)
wav = mp3[:-4] + '.wav'
with wave.open(wav, 'wb') as w:
w.setnchannels(1)
w.setsampwidth(2)
w.setframerate(voice.config.sample_rate)
w.writeframes(pcm.tobytes())
subprocess.run(['lame', '--quiet', '-b', '64', wav, mp3], check=True)
os.unlink(wav)
async def render_edge(items):
import edge_tts
sem = asyncio.Semaphore(4)
made = 0
async def one(voice, rate, text, mp3):
nonlocal made
async with sem:
for attempt in range(4):
try:
await edge_tts.Communicate(text, voice, rate=rate).save(mp3 + '.tmp')
os.replace(mp3 + '.tmp', mp3)
made += 1
return
except Exception as e:
print('retry', text, voice, e)
await asyncio.sleep(2 + attempt * 3)
print('FAILED', text, voice)
jobs = []
for vk, key, text in items: # items: (voice key, clip key, text)
for sk, rate in EDGE_RATE.items():
folder = f'{WEB}/audio/{vk}/{sk}'
os.makedirs(folder, exist_ok=True)
mp3 = f"{folder}/{key.encode('utf-8').hex()}.mp3"
if not os.path.exists(mp3) or '--force' in sys.argv:
jobs.append(one(EDGE[vk], rate, text, mp3))
await asyncio.gather(*jobs)
return made
course = json.load(open(f'{WEB}/course.json', encoding='utf-8'))
words = [w for L in course['lessons'] for w in L['words']]
texts = [(w['hz'], w.get('say', w['hz']) + '。') for w in words]
texts += [('py:' + d, DRILL_HZ[d[:-1]][int(d[-1]) - 1] + '。') for d in DRILL]
edge_items = [(vk, key, text) for vk in EDGE for key, text in texts]
# conversation lines: only in the speaker's own voice (key = the line's characters)
people = course.get('people', {})
edge_items += [(people[who]['voice'], hz, hz) for L in course['lessons'] for who, hz, _, _ in L.get('talk', [])]
edge_made = asyncio.run(render_edge(edge_items))
print(f'EDGE-DONE made {edge_made} clips')
items = [(w['hz'], spoken(w)) for w in words]
items += [('py:' + d, [d]) for d in DRILL]
made = 0
for vk, vname in VOICES.items():
voice = PiperVoice.load(f'/opt/mandarin/voices/zh_CN-{vname}-medium.onnx')
for sk in SPEEDS:
folder = f'{WEB}/audio/{vk}/{sk}'
os.makedirs(folder, exist_ok=True)
for key, sylls in items:
mp3 = f"{folder}/{key.encode('utf-8').hex()}.mp3"
if os.path.exists(mp3) and '--force' not in sys.argv:
continue
render(voice, sylls, sk, mp3)
made += 1
print(f'RENDER-DONE made {made} clips for {len(items)} items')
EOFinstall.sh: copies the site and server into CT 309, then restarts them. Pass render to also record the audio.
cat > /root/mandarin/install.sh <<'EOF'
#!/bin/bash
# Runs on the Proxmox HOST after deploy.ps1 copied files to /root/mandarin/.
# bash /root/mandarin/install.sh [render]
# Site files go into CT 309 /var/www/mandarin (audio/ and samples/ are kept), speech server restarted.
set -e
S=/root/mandarin
pct push 309 $S/site.tar /root/site.tar
pct exec 309 -- tar -xf /root/site.tar -C /var/www/mandarin --no-same-owner
pct exec 309 -- chmod -R a+rX /var/www/mandarin
pct push 309 $S/mandarin_speech.py /opt/mandarin/mandarin_speech.py
pct push 309 $S/render-course.py /opt/mandarin/render-course.py
pct push 309 $S/mandarin-speech.service /etc/systemd/system/mandarin-speech.service
pct push 309 $S/Caddyfile /etc/caddy/Caddyfile
pct exec 309 -- systemctl daemon-reload
pct exec 309 -- systemctl enable mandarin-speech
pct exec 309 -- systemctl restart mandarin-speech
pct exec 309 -- systemctl reload caddy
if [ "$1" = "render" ]; then
pct exec 309 -- /opt/mandarin/venv/bin/python3 /opt/mandarin/render-course.py
pct exec 309 -- chmod -R a+rX /var/www/mandarin/audio
fi
sleep 3
pct exec 309 -- curl -s http://127.0.0.1/api/health
echo
echo INSTALL-DONE
EOFedge-add.sh: runs inside CT 108: adds the mandarin.503mail.com site block, validates the config, and restores the backup if validation fails.
cat > /root/mandarin/edge-add.sh <<'EOF_OUTER1'
#!/bin/bash
# Runs INSIDE CT 108 (edge Caddy): adds mandarin.503mail.com -> CT 309 (once), validates, reloads.
# bash edge-add-mandarin.sh <CT 309 LAN IP>
set -e
IP="$1"
[ -n "$IP" ] || { echo "usage: $0 <CT 309 LAN IP>"; exit 1; }
CF=/etc/caddy/Caddyfile
if grep -q '^mandarin.503mail.com' "$CF"; then echo "already there"; exit 0; fi
cp "$CF" "$CF.bak-mandarin"
cat >> "$CF" <<EOF
# Mandarin tone trainer (CT 309, separate project from the creator platform).
mandarin.503mail.com {
request_body {
max_size 5MB
}
reverse_proxy $IP:80
}
EOF
if caddy validate --config "$CF" --adapter caddyfile; then
systemctl reload caddy
echo EDGE-OK
else
cp "$CF.bak-mandarin" "$CF"
echo "EDGE-FAILED, restored backup"
exit 1
fi
EOF_OUTER1course.json: all the course content: 15 lessons, each with 10 words and a conversation. Each word has characters (hz), numbered pinyin (py) and English (en). A word can also have say, which is the text the Microsoft voices read instead (都 is read as 兜 so it comes out "dōu", not "dū"). Each conversation line is [speaker, characters, pinyin, English].
cat > /root/mandarin/course.json <<'EOF'
{
"name": "HSK 1",
"people": {
"A": { "hz": "王方", "py": "wang2 fang1", "voice": "x", "about": "female voice" },
"B": { "hz": "李明", "py": "li3 ming2", "voice": "y", "about": "male voice" }
},
"lessons": [
{ "title": "Hello and polite words", "words": [
{ "hz": "你", "py": "ni3", "en": "you" },
{ "hz": "好", "py": "hao3", "en": "good, well" },
{ "hz": "我", "py": "wo3", "en": "I, me" },
{ "hz": "是", "py": "shi4", "en": "to be (am, is, are)" },
{ "hz": "谢谢", "py": "xie4 xie5", "en": "thank you" },
{ "hz": "不客气", "py": "bu4 ke4 qi5", "en": "you're welcome" },
{ "hz": "对不起", "py": "dui4 bu5 qi3", "en": "sorry" },
{ "hz": "没关系", "py": "mei2 guan1 xi5", "en": "it doesn't matter, that's OK" },
{ "hz": "再见", "py": "zai4 jian4", "en": "goodbye" },
{ "hz": "请", "py": "qing3", "en": "please; to invite" }
], "talk": [
["A", "对不起！", "dui4 bu5 qi3", "Sorry!"],
["B", "没关系。", "mei2 guan1 xi5", "That's OK."],
["A", "你好！", "ni3 hao3", "Hello!"],
["B", "你好！", "ni3 hao3", "Hello!"],
["A", "谢谢你！", "xie4 xie5 ni3", "Thank you!"],
["B", "不客气。", "bu4 ke4 qi5", "You're welcome."],
["A", "再见！", "zai4 jian4", "Goodbye!"],
["B", "再见！", "zai4 jian4", "Goodbye!"]
]},
{ "title": "People and family", "words": [
{ "hz": "他", "py": "ta1", "en": "he, him" },
{ "hz": "她", "py": "ta1", "en": "she, her" },
{ "hz": "我们", "py": "wo3 men5", "en": "we, us" },
{ "hz": "人", "py": "ren2", "en": "person, people" },
{ "hz": "爸爸", "py": "ba4 ba5", "en": "dad" },
{ "hz": "妈妈", "py": "ma1 ma5", "en": "mom" },
{ "hz": "儿子", "py": "er2 zi5", "en": "son" },
{ "hz": "女儿", "py": "nv3 er2", "en": "daughter" },
{ "hz": "朋友", "py": "peng2 you5", "en": "friend" },
{ "hz": "同学", "py": "tong2 xue2", "en": "classmate" }
], "talk": [
["A", "他是你爸爸吗？", "ta1 shi4 ni3 ba4 ba5 ma5", "Is he your dad?"],
["B", "是，他是我爸爸。", "shi4 ta1 shi4 wo3 ba4 ba5", "Yes, he's my dad."],
["A", "她是你妈妈吗？", "ta1 shi4 ni3 ma1 ma5 ma5", "Is she your mom?"],
["B", "不是，她是我朋友。", "bu4 shi4 ta1 shi4 wo3 peng2 you5", "No, she's my friend."],
["A", "你们是同学吗？", "ni3 men5 shi4 tong2 xue2 ma5", "Are you two classmates?"],
["B", "是，我们是同学。", "shi4 wo3 men5 shi4 tong2 xue2", "Yes, we're classmates."]
]},
{ "title": "Numbers one to ten", "words": [
{ "hz": "一", "py": "yi1", "en": "one" },
{ "hz": "二", "py": "er4", "en": "two" },
{ "hz": "三", "py": "san1", "en": "three" },
{ "hz": "四", "py": "si4", "en": "four" },
{ "hz": "五", "py": "wu3", "en": "five" },
{ "hz": "六", "py": "liu4", "en": "six" },
{ "hz": "七", "py": "qi1", "en": "seven" },
{ "hz": "八", "py": "ba1", "en": "eight" },
{ "hz": "九", "py": "jiu3", "en": "nine" },
{ "hz": "十", "py": "shi2", "en": "ten" }
], "talk": [
["A", "你女儿几岁？", "ni3 nv3 er2 ji3 sui4", "How old is your daughter?"],
["B", "她七岁。", "ta1 qi1 sui4", "She's seven."],
["A", "你儿子呢？", "ni3 er2 zi5 ne5", "And your son?"],
["B", "我儿子十岁。", "wo3 er2 zi5 shi2 sui4", "My son is ten."],
["A", "我有三个儿子。", "wo3 you3 san1 ge5 er2 zi5", "I have three sons."],
["B", "三个？太好了！", "san1 ge5 tai4 hao3 le5", "Three? That's great!"]
]},
{ "title": "Question words", "words": [
{ "hz": "什么", "py": "shen2 me5", "en": "what" },
{ "hz": "谁", "py": "shei2", "en": "who" },
{ "hz": "哪", "py": "na3", "en": "which" },
{ "hz": "哪儿", "py": "nar3", "en": "where" },
{ "hz": "吗", "py": "ma5", "en": "question word: turns a sentence into a yes or no question" },
{ "hz": "呢", "py": "ne5", "en": "question word: and you? what about?" },
{ "hz": "几", "py": "ji3", "en": "how many (small numbers)" },
{ "hz": "多少", "py": "duo1 shao5", "en": "how many, how much" },
{ "hz": "怎么", "py": "zen3 me5", "en": "how" },
{ "hz": "怎么样", "py": "zen3 me5 yang4", "en": "how is it? how about?" }
], "talk": [
["A", "这是什么？", "zhe4 shi4 shen2 me5", "What is this?"],
["B", "这是茶。", "zhe4 shi4 cha2", "This is tea."],
["A", "他是谁？", "ta1 shi4 shei2", "Who is he?"],
["B", "他是我同学。", "ta1 shi4 wo3 tong2 xue2", "He's my classmate."],
["A", "你朋友在哪儿？", "ni3 peng2 you5 zai4 nar3", "Where is your friend?"],
["B", "她在家。", "ta1 zai4 jia1", "She's at home."],
["A", "你怎么样？", "ni3 zen3 me5 yang4", "How are you?"],
["B", "我很好，谢谢！", "wo3 hen3 hao3 xie4 xie5", "I'm very well, thanks!"]
]},
{ "title": "Names and jobs", "words": [
{ "hz": "叫", "py": "jiao4", "en": "to be called" },
{ "hz": "名字", "py": "ming2 zi5", "en": "name" },
{ "hz": "老师", "py": "lao3 shi1", "en": "teacher" },
{ "hz": "学生", "py": "xue2 sheng5", "en": "student" },
{ "hz": "医生", "py": "yi1 sheng1", "en": "doctor" },
{ "hz": "先生", "py": "xian1 sheng5", "en": "Mister, sir" },
{ "hz": "小姐", "py": "xiao3 jie3", "en": "Miss, young lady" },
{ "hz": "中国", "py": "zhong1 guo2", "en": "China" },
{ "hz": "北京", "py": "bei3 jing1", "en": "Beijing" },
{ "hz": "汉语", "py": "han4 yu3", "en": "Chinese language" }
], "talk": [
["A", "你叫什么名字？", "ni3 jiao4 shen2 me5 ming2 zi5", "What's your name?"],
["B", "我叫李明。你呢？", "wo3 jiao4 li3 ming2 ni3 ne5", "My name is Li Ming. And you?"],
["A", "我叫王方。我是老师。", "wo3 jiao4 wang2 fang1 wo3 shi4 lao3 shi1", "My name is Wang Fang. I'm a teacher."],
["B", "我是医生。", "wo3 shi4 yi1 sheng1", "I'm a doctor."],
["A", "你是中国人吗？", "ni3 shi4 zhong1 guo2 ren2 ma5", "Are you Chinese?"],
["B", "是，我是北京人。", "shi4 wo3 shi4 bei3 jing1 ren2", "Yes, I'm from Beijing."]
]},
{ "title": "Food and drink", "words": [
{ "hz": "吃", "py": "chi1", "en": "to eat" },
{ "hz": "喝", "py": "he1", "en": "to drink" },
{ "hz": "茶", "py": "cha2", "en": "tea" },
{ "hz": "水", "py": "shui3", "en": "water" },
{ "hz": "米饭", "py": "mi3 fan4", "en": "cooked rice" },
{ "hz": "菜", "py": "cai4", "en": "dish of food; vegetable" },
{ "hz": "苹果", "py": "ping2 guo3", "en": "apple" },
{ "hz": "水果", "py": "shui3 guo3", "en": "fruit" },
{ "hz": "杯子", "py": "bei1 zi5", "en": "cup, glass" },
{ "hz": "饭店", "py": "fan4 dian4", "en": "restaurant, hotel" }
], "talk": [
["A", "你想喝什么？", "ni3 xiang3 he1 shen2 me5", "What would you like to drink?"],
["B", "我想喝茶。", "wo3 xiang3 he1 cha2", "I'd like some tea."],
["A", "你吃米饭吗？", "ni3 chi1 mi3 fan4 ma5", "Would you like some rice?"],
["B", "吃。我很喜欢中国菜。", "chi1 wo3 hen3 xi3 huan5 zhong1 guo2 cai4", "Yes. I really like Chinese food."],
["A", "苹果多少钱？", "ping2 guo3 duo1 shao5 qian2", "How much are the apples?"],
["B", "十块。", "shi2 kuai4", "Ten yuan."]
]},
{ "title": "Days and dates", "words": [
{ "hz": "今天", "py": "jin1 tian1", "en": "today" },
{ "hz": "明天", "py": "ming2 tian1", "en": "tomorrow" },
{ "hz": "昨天", "py": "zuo2 tian1", "en": "yesterday" },
{ "hz": "现在", "py": "xian4 zai4", "en": "now" },
{ "hz": "年", "py": "nian2", "en": "year" },
{ "hz": "月", "py": "yue4", "en": "month; moon" },
{ "hz": "号", "py": "hao4", "en": "day of the month; number" },
{ "hz": "星期", "py": "xing1 qi1", "en": "week" },
{ "hz": "点", "py": "dian3", "en": "o'clock" },
{ "hz": "分钟", "py": "fen1 zhong1", "en": "minute" }
], "talk": [
["A", "今天几号？", "jin1 tian1 ji3 hao4", "What's the date today?"],
["B", "今天十月九号。", "jin1 tian1 shi2 yue4 jiu3 hao4", "Today is October 9th."],
["A", "今天星期几？", "jin1 tian1 xing1 qi1 ji3", "What day of the week is it?"],
["B", "星期三。", "xing1 qi1 san1", "Wednesday."],
["A", "现在几点？", "xian4 zai4 ji3 dian3", "What time is it now?"],
["B", "现在八点十分。", "xian4 zai4 ba1 dian3 shi2 fen1", "It's ten past eight."]
]},
{ "title": "Time of day and weather", "words": [
{ "hz": "上午", "py": "shang4 wu3", "en": "morning" },
{ "hz": "中午", "py": "zhong1 wu3", "en": "noon" },
{ "hz": "下午", "py": "xia4 wu3", "en": "afternoon" },
{ "hz": "时候", "py": "shi2 hou5", "en": "time, when" },
{ "hz": "天气", "py": "tian1 qi4", "en": "weather" },
{ "hz": "冷", "py": "leng3", "en": "cold" },
{ "hz": "热", "py": "re4", "en": "hot" },
{ "hz": "下雨", "py": "xia4 yu3", "en": "to rain" },
{ "hz": "岁", "py": "sui4", "en": "years old" },
{ "hz": "睡觉", "py": "shui4 jiao4", "en": "to sleep" }
], "talk": [
["A", "今天天气怎么样？", "jin1 tian1 tian1 qi4 zen3 me5 yang4", "What's the weather like today?"],
["B", "今天很冷。", "jin1 tian1 hen3 leng3", "It's very cold today."],
["A", "明天呢？", "ming2 tian1 ne5", "And tomorrow?"],
["B", "明天下雨。", "ming2 tian1 xia4 yu3", "It's going to rain tomorrow."],
["A", "你下午做什么？", "ni3 xia4 wu3 zuo4 shen2 me5", "What are you doing this afternoon?"],
["B", "我在家睡觉。", "wo3 zai4 jia1 shui4 jiao4", "I'm staying home to sleep."]
]},
{ "title": "Places and where things are", "words": [
{ "hz": "家", "py": "jia1", "en": "home, family" },
{ "hz": "学校", "py": "xue2 xiao4", "en": "school" },
{ "hz": "医院", "py": "yi1 yuan4", "en": "hospital" },
{ "hz": "商店", "py": "shang1 dian4", "en": "shop, store" },
{ "hz": "里", "py": "li3", "en": "inside" },
{ "hz": "上", "py": "shang4", "en": "on, above; to go up" },
{ "hz": "下", "py": "xia4", "en": "under, below; to go down" },
{ "hz": "前面", "py": "qian2 mian4", "en": "in front" },
{ "hz": "后面", "py": "hou4 mian4", "en": "behind" },
{ "hz": "在", "py": "zai4", "en": "at, in; to be (somewhere)" }
], "talk": [
["A", "你住在哪儿？", "ni3 zhu4 zai4 nar3", "Where do you live?"],
["B", "我住在学校后面。", "wo3 zhu4 zai4 xue2 xiao4 hou4 mian4", "I live behind the school."],
["A", "医院在哪儿？", "yi1 yuan4 zai4 nar3", "Where is the hospital?"],
["B", "医院在商店前面。", "yi1 yuan4 zai4 shang1 dian4 qian2 mian4", "The hospital is in front of the shop."],
["A", "你的书在哪儿？", "ni3 de5 shu1 zai4 nar3", "Where is your book?"],
["B", "在桌子上。", "zai4 zhuo1 zi5 shang4", "On the table."]
]},
{ "title": "Coming and going", "words": [
{ "hz": "去", "py": "qu4", "en": "to go" },
{ "hz": "来", "py": "lai2", "en": "to come" },
{ "hz": "回", "py": "hui2", "en": "to return" },
{ "hz": "坐", "py": "zuo4", "en": "to sit; to take (a bus, a plane)" },
{ "hz": "住", "py": "zhu4", "en": "to live (somewhere)" },
{ "hz": "开", "py": "kai1", "en": "to open; to drive" },
{ "hz": "出租车", "py": "chu1 zu1 che1", "en": "taxi" },
{ "hz": "飞机", "py": "fei1 ji1", "en": "airplane" },
{ "hz": "买", "py": "mai3", "en": "to buy" },
{ "hz": "钱", "py": "qian2", "en": "money" }
], "talk": [
["A", "你明天去哪儿？", "ni3 ming2 tian1 qu4 nar3", "Where are you going tomorrow?"],
["B", "我去北京。", "wo3 qu4 bei3 jing1", "I'm going to Beijing."],
["A", "你怎么去？", "ni3 zen3 me5 qu4", "How are you getting there?"],
["B", "我坐飞机去。", "wo3 zuo4 fei1 ji1 qu4", "I'm flying."],
["A", "你什么时候回来？", "ni3 shen2 me5 shi2 hou5 hui2 lai2", "When are you coming back?"],
["B", "星期五回来。", "xing1 qi1 wu3 hui2 lai2", "On Friday."]
]},
{ "title": "Things we do", "words": [
{ "hz": "看", "py": "kan4", "en": "to look, watch, read" },
{ "hz": "看见", "py": "kan4 jian4", "en": "to see" },
{ "hz": "听", "py": "ting1", "en": "to listen" },
{ "hz": "说", "py": "shuo1", "en": "to speak, say" },
{ "hz": "读", "py": "du2", "en": "to read aloud" },
{ "hz": "写", "py": "xie3", "en": "to write" },
{ "hz": "做", "py": "zuo4", "en": "to do, make" },
{ "hz": "工作", "py": "gong1 zuo4", "en": "work, job; to work" },
{ "hz": "学习", "py": "xue2 xi2", "en": "to study" },
{ "hz": "打电话", "py": "da3 dian4 hua4", "en": "to make a phone call" }
], "talk": [
["A", "你在做什么？", "ni3 zai4 zuo4 shen2 me5", "What are you doing?"],
["B", "我在学习汉语。", "wo3 zai4 xue2 xi2 han4 yu3", "I'm studying Chinese."],
["A", "你会写汉字吗？", "ni3 hui4 xie3 han4 zi4 ma5", "Can you write Chinese characters?"],
["B", "我会写一些。", "wo3 hui4 xie3 yi1 xie1", "I can write some."],
["A", "你喜欢看电影吗？", "ni3 xi3 huan5 kan4 dian4 ying3 ma5", "Do you like watching movies?"],
["B", "喜欢。我们明天去看电影。", "xi3 huan5 wo3 men5 ming2 tian1 qu4 kan4 dian4 ying3", "Yes. Let's go see a movie tomorrow."]
]},
{ "title": "Things around the house", "words": [
{ "hz": "东西", "py": "dong1 xi5", "en": "thing, stuff" },
{ "hz": "书", "py": "shu1", "en": "book" },
{ "hz": "衣服", "py": "yi1 fu5", "en": "clothes" },
{ "hz": "桌子", "py": "zhuo1 zi5", "en": "table, desk" },
{ "hz": "椅子", "py": "yi3 zi5", "en": "chair" },
{ "hz": "电脑", "py": "dian4 nao3", "en": "computer" },
{ "hz": "电视", "py": "dian4 shi4", "en": "television" },
{ "hz": "电影", "py": "dian4 ying3", "en": "movie" },
{ "hz": "字", "py": "zi4", "en": "Chinese character, written word" },
{ "hz": "本", "py": "ben3", "en": "counting word for books" }
], "talk": [
["A", "这是你的电脑吗？", "zhe4 shi4 ni3 de5 dian4 nao3 ma5", "Is this your computer?"],
["B", "不是，是我儿子的。", "bu4 shi4 shi4 wo3 er2 zi5 de5", "No, it's my son's."],
["A", "那些是什么东西？", "na4 xie1 shi4 shen2 me5 dong1 xi5", "What are those things?"],
["B", "那些是我女儿的衣服。", "na4 xie1 shi4 wo3 nv3 er2 de5 yi1 fu5", "Those are my daughter's clothes."],
["A", "桌子上有几本书？", "zhuo1 zi5 shang4 you3 ji3 ben3 shu1", "How many books are on the table?"],
["B", "有三本。", "you3 san1 ben3", "There are three."]
]},
{ "title": "Feelings and describing", "words": [
{ "hz": "爱", "py": "ai4", "en": "to love" },
{ "hz": "喜欢", "py": "xi3 huan5", "en": "to like" },
{ "hz": "想", "py": "xiang3", "en": "to want; to think; to miss" },
{ "hz": "高兴", "py": "gao1 xing4", "en": "happy, glad" },
{ "hz": "漂亮", "py": "piao4 liang5", "en": "pretty, beautiful" },
{ "hz": "大", "py": "da4", "en": "big" },
{ "hz": "小", "py": "xiao3", "en": "small" },
{ "hz": "多", "py": "duo1", "en": "many, much" },
{ "hz": "少", "py": "shao3", "en": "few, little" },
{ "hz": "很", "py": "hen3", "en": "very" }
], "talk": [
["A", "你喜欢这个杯子吗？", "ni3 xi3 huan5 zhe4 ge5 bei1 zi5 ma5", "Do you like this cup?"],
["B", "很喜欢，太漂亮了！", "hen3 xi3 huan5 tai4 piao4 liang5 le5", "I love it, it's so pretty!"],
["A", "你想买大的吗？", "ni3 xiang3 mai3 da4 de5 ma5", "Do you want to buy a big one?"],
["B", "不，我想买小的。", "bu4 wo3 xiang3 mai3 xiao3 de5", "No, I want to buy a small one."],
["A", "我今天很高兴！", "wo3 jin1 tian1 hen3 gao1 xing4", "I'm very happy today!"],
["B", "我们都很高兴。", "wo3 men5 dou1 hen3 gao1 xing4", "We're all very happy."]
]},
{ "title": "Animals and small words", "words": [
{ "hz": "狗", "py": "gou3", "en": "dog" },
{ "hz": "猫", "py": "mao1", "en": "cat" },
{ "hz": "太", "py": "tai4", "en": "too (much)" },
{ "hz": "都", "py": "dou1", "en": "all, both", "say": "兜" },
{ "hz": "和", "py": "he2", "en": "and, with" },
{ "hz": "的", "py": "de5", "en": "shows belonging, like 's in English" },
{ "hz": "了", "py": "le5", "en": "shows a change or a finished action" },
{ "hz": "个", "py": "ge4", "en": "general counting word" },
{ "hz": "些", "py": "xie1", "en": "some" },
{ "hz": "块", "py": "kuai4", "en": "piece; yuan (money)" }
], "talk": [
["A", "你家有狗吗？", "ni3 jia1 you3 gou3 ma5", "Does your family have a dog?"],
["B", "没有狗，有猫。", "mei2 you3 gou3 you3 mao1", "No dog, but we have a cat."],
["A", "你的猫多大了？", "ni3 de5 mao1 duo1 da4 le5", "How old is your cat?"],
["B", "我的猫三岁了。", "wo3 de5 mao1 san1 sui4 le5", "My cat is three."],
["A", "这个杯子多少钱？", "zhe4 ge5 bei1 zi5 duo1 shao5 qian2", "How much is this cup?"],
["B", "八块。", "ba1 kuai4", "Eight yuan."],
["A", "太好了，我买了。", "tai4 hao3 le5 wo3 mai3 le5", "Great, I'll take it."]
]},
{ "title": "Have, can, this and that", "words": [
{ "hz": "不", "py": "bu4", "en": "not, no" },
{ "hz": "有", "py": "you3", "en": "to have; there is" },
{ "hz": "没有", "py": "mei2 you3", "en": "to not have; did not" },
{ "hz": "会", "py": "hui4", "en": "can (a learned skill); will" },
{ "hz": "能", "py": "neng2", "en": "can, be able to" },
{ "hz": "认识", "py": "ren4 shi5", "en": "to know (a person)" },
{ "hz": "这", "py": "zhe4", "en": "this" },
{ "hz": "那", "py": "na4", "en": "that" },
{ "hz": "喂", "py": "wei4", "en": "hello (on the phone)", "note": "On the phone most people say it with tone 2: wéi." }
], "talk": [
["A", "喂，你好！", "wei2 ni3 hao3", "Hello? (answering the phone)"],
["B", "你好！你认识张老师吗？", "ni3 hao3 ni3 ren4 shi5 zhang1 lao3 shi1 ma5", "Hello! Do you know Teacher Zhang?"],
["A", "认识，她是我朋友。", "ren4 shi5 ta1 shi4 wo3 peng2 you5", "Yes, she's my friend."],
["B", "她会说汉语吗？", "ta1 hui4 shuo1 han4 yu3 ma5", "Can she speak Chinese?"],
["A", "会，她是汉语老师。", "hui4 ta1 shi4 han4 yu3 lao3 shi1", "Yes, she's a Chinese teacher."],
["B", "你明天能来吗？", "ni3 ming2 tian1 neng2 lai2 ma5", "Can you come tomorrow?"],
["A", "能，明天见！", "neng2 ming2 tian1 jian4", "Yes, see you tomorrow!"]
]}
]
}
EOFindex.html: the page shell.
cat > /root/mandarin/index.html <<'EOF'
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<title>Mandarin Tones</title>
<meta name="description" content="Learn HSK 1 Mandarin words with a tone checker that listens to your voice.">
<link rel="stylesheet" href="style.css?v=4">
</head>
<body>
<a class="skip" href="#main">Skip to content</a>
<header><a href="#home" class="brand">Mandarin Tones</a> <span lang="zh-CN" aria-hidden="true">声调</span></header>
<main id="main" tabindex="-1"></main>
<div id="say" class="sr" aria-live="assertive"></div>
<script src="tone.js?v=5"></script>
<script src="app.js?v=6"></script>
</body>
</html>
EOFstyle.css: dark theme, tone colors (1 yellow, 2 green, 3 blue, 4 red).
cat > /root/mandarin/style.css <<'EOF'
:root {
--bg: #111; --fg: #eee; --muted: #aaa; --line: #6a6a6a; --box: #1b1b1b;
--good: #6fd38a; --bad: #ff8a80; --accent: #8ab4ff; --t1: #ffd166; --t2: #6fd38a; --t3: #8ab4ff; --t4: #ff8a80;
color-scheme: dark;
}
* { box-sizing: border-box; }
body { background: var(--bg); color: var(--fg); font: 18px/1.5 system-ui, sans-serif; margin: 0 auto; max-width: 760px; padding: 0 16px 48px; }
a { color: var(--accent); }
header { padding: 14px 0; border-bottom: 1px solid var(--line); margin-bottom: 8px; display: flex; gap: 10px; align-items: baseline; }
.brand { font-weight: 700; font-size: 1.2em; text-decoration: none; color: var(--fg); }
.skip { position: absolute; left: -999px; } .skip:focus { left: 16px; top: 8px; background: #000; padding: 6px; }
.sr { position: absolute; width: 1px; height: 1px; overflow: hidden; clip: rect(0 0 0 0); }
h1 { font-size: 1.5em; margin: 16px 0 8px; } h2 { font-size: 1.2em; margin: 20px 0 8px; }
h1:focus, main:focus { outline: none; }
.box { border: 1px solid var(--line); border-radius: 8px; padding: 12px 16px; margin: 12px 0; background: var(--box); }
button, select { font: inherit; color: var(--fg); background: #262626; border: 1px solid var(--line); border-radius: 8px; padding: 10px 14px; margin: 4px 6px 4px 0; cursor: pointer; min-height: 44px; }
button:hover { background: #333; } button:focus-visible, select:focus-visible, a:focus-visible { outline: 3px solid var(--accent); outline-offset: 2px; }
button.main { background: #1f3a66; border-color: var(--accent); font-weight: 600; }
button.rec { background: #5c1f1f; border-color: var(--bad); }
button[disabled] { opacity: .5; cursor: default; }
ul.lessons { list-style: none; padding: 0; margin: 0; }
ul.lessons li { margin: 8px 0; }
ul.lessons h3 { margin: 4px 0; font-size: 1.1em; }
ul.lessons p { margin: 4px 0; }
.talk-line { border-left: 4px solid var(--line); padding: 6px 12px; margin: 10px 0; }
.talk-line.A { border-color: var(--t1); } .talk-line.B { border-color: var(--t3); }
.talk-line.now { background: #222; }
.talk-line .who { font-weight: 600; } .talk-line .zh { font-size: 1.5em; }
ul.lessons a { display: inline-block; padding: 6px 2px; font-size: 1.05em; }
.prog { color: var(--muted); font-size: .9em; }
.card { text-align: center; }
.hz { font-size: 4.2em; line-height: 1.2; margin: 6px 0; }
.py { font-size: 1.6em; } .num { color: var(--muted); } .en { font-size: 1.15em; margin-top: 4px; }
.t1 { color: var(--t1); } .t2 { color: var(--t2); } .t3 { color: var(--t3); } .t4 { color: var(--t4); } .t5 { color: var(--muted); }
.note { color: var(--muted); font-size: .95em; }
.row { display: flex; flex-wrap: wrap; justify-content: center; }
.result { min-height: 3em; }
.result .ok { color: var(--good); } .result .no { color: var(--bad); }
.result ul { margin: 6px 0; padding-left: 22px; text-align: left; }
canvas { width: 100%; height: 160px; display: block; margin-top: 8px; background: #0b0b0b; border: 1px solid #333; border-radius: 6px; }
.meter { height: 8px; background: #222; border-radius: 4px; overflow: hidden; margin: 8px auto; max-width: 300px; }
.meter i { display: block; height: 100%; width: 0; background: var(--good); transition: width .05s; }
.choices button { min-width: 120px; }
label { display: inline-block; margin: 6px 12px 6px 0; }
.small { font-size: .9em; color: var(--muted); }
EOFtone.js: the pitch tracker, syllable splitting, the tone model, the hum playback and the microphone recorder.
cat > /root/mandarin/tone.js <<'EOF'
// tone.js - pitch tracking and Mandarin tone grading, all in the browser (no audio leaves the page
// for this part). Pitch: YIN on 16 kHz audio, 10 ms steps. Tones are judged on semitones relative
// to the speaker's own range (set by the tone setup: a high tone-1 "ma" and a low tone-3 "ma").
'use strict';
const Tone = (() => {
// 40 ms window: low male voices (~85 Hz) need 3+ cycles in the window to track reliably
const SR = 16000, HOP = 160, WIN = 640, TAU_MIN = 32, TAU_MAX = 290; // 55-500 Hz
function resample(x, from) {
if (from === SR) return Float32Array.from(x);
const ratio = from / SR, n = Math.floor(x.length / ratio), out = new Float32Array(n);
for (let i = 0; i < n; i++) { // box average over the source span (cheap low-pass)
const a = Math.floor(i * ratio), b = Math.max(a + 1, Math.floor((i + 1) * ratio));
let s = 0;
for (let j = a; j < b && j < x.length; j++) s += x[j];
out[i] = s / (b - a);
}
return out;
}
// Frames: {f0 (Hz, 0 = unvoiced), rms}
function track(x) {
const frames = [], d = new Float32Array(TAU_MAX + 1);
for (let i = 0; i + WIN + TAU_MAX < x.length; i += HOP) {
let e = 0;
for (let j = 0; j < WIN; j++) e += x[i + j] * x[i + j];
const rms = Math.sqrt(e / WIN);
d[0] = 1;
let sum = 0;
for (let tau = 1; tau <= TAU_MAX; tau++) {
let s = 0;
for (let j = 0; j < WIN; j++) { const v = x[i + j] - x[i + j + tau]; s += v * v; }
sum += s;
d[tau] = sum > 0 ? s * tau / sum : 1;
}
let tau = -1;
for (let t = TAU_MIN; t < TAU_MAX; t++) {
if (d[t] < 0.15) { while (t + 1 < TAU_MAX && d[t + 1] < d[t]) t++; tau = t; break; }
}
if (tau < 0) { // no clear dip: take the best one if it is decent
let best = TAU_MIN;
for (let t = TAU_MIN; t < TAU_MAX; t++) if (d[t] < d[best]) best = t;
if (d[best] < 0.3) tau = best;
}
let f0 = 0;
if (tau > 0) {
const a = d[tau - 1], b = d[tau], c = d[tau + 1] ?? b, den = a - 2 * b + c;
f0 = SR / (tau + (den ? (a - c) / (2 * den) : 0));
}
frames.push({ f0, rms });
}
// drop quiet frames, fix octave jumps against the median, smooth with a 5-frame median
const maxRms = Math.max(...frames.map(f => f.rms), 1e-9);
for (const f of frames) if (f.rms < maxRms * 0.08) f.f0 = 0;
const voiced = frames.filter(f => f.f0).map(f => f.f0).sort((a, b) => a - b);
const med = voiced.length ? voiced[voiced.length >> 1] : 0;
for (const f of frames) {
if (!f.f0) continue;
if (f.f0 > med * 1.8) f.f0 /= 2;
else if (f.f0 < med * 0.56) f.f0 *= 2;
}
const raw = frames.map(f => f.f0);
for (let i = 0; i < frames.length; i++) {
if (!raw[i]) continue;
const w = raw.slice(Math.max(0, i - 2), i + 3).filter(Boolean).sort((a, b) => a - b);
frames[i].f0 = w[w.length >> 1];
}
return frames;
}
// Split the voiced frames into n syllables: [{a, b}] frame ranges (b exclusive).
function segment(frames, n) {
let runs = [], start = -1, gap = 0;
for (let i = 0; i <= frames.length; i++) {
const v = i < frames.length && frames[i].f0 > 0;
if (v) { if (start < 0) start = i; gap = 0; }
else if (start >= 0 && (++gap > 2 || i === frames.length)) { runs.push({ a: start, b: i - gap + 1 }); start = -1; gap = 0; }
}
const long = runs.filter(r => r.b - r.a >= 5);
if (long.length) runs = long;
if (!runs.length) return [];
while (runs.length > n) { // join the two runs with the smallest gap
let k = 0;
for (let i = 1; i < runs.length - 1; i++) if (runs[i + 1].a - runs[i].b < runs[k + 1].a - runs[k].b) k = i;
runs.splice(k, 2, { a: runs[k].a, b: runs[k + 1].b });
}
while (runs.length < n) { // split the longest run at its quietest point
let k = 0;
runs.forEach((r, i) => { if (r.b - r.a > runs[k].b - runs[k].a) k = i; });
const r = runs[k], len = r.b - r.a;
if (len < 8) break;
let cut = r.a + Math.floor(len / 2), low = Infinity;
for (let i = r.a + Math.floor(len * 0.25); i < r.a + Math.ceil(len * 0.75); i++) {
const e = (frames[i - 1].rms + frames[i].rms + frames[i + 1].rms);
if (e < low) { low = e; cut = i; }
}
runs.splice(k, 1, { a: r.a, b: cut }, { a: cut, b: r.b });
}
if (runs.length !== n) { // fallback: share the voiced frames out evenly
const v = [];
frames.forEach((f, i) => { if (f.f0) v.push(i); });
if (v.length < n * 3) return runs;
runs = [];
for (let k = 0; k < n; k++) {
const a = v[Math.floor(k * v.length / n)], b = v[Math.floor((k + 1) * v.length / n) - 1] + 1;
runs.push({ a, b });
}
}
return runs;
}
const st = (hz, ref) => 12 * Math.log2(hz / ref);
const mean = a => a.reduce((s, v) => s + v, 0) / a.length;
// profile = {top, bottom} in Hz (tone-1 level and tone-3 low point of this speaker)
function defaultProfile(frames) {
const v = frames.filter(f => f.f0).map(f => f.f0).sort((a, b) => a - b);
const med = v.length ? v[v.length >> 1] : 150;
const mid = med < 165 ? 125 : 215; // typical low/high voice
return { top: mid * 2 ** (3 / 12), bottom: mid * 2 ** (-4 / 12), guessed: true };
}
// One syllable -> features. Semitones are relative to the profile bottom.
function features(frames, seg, prof, multi) {
let pts = frames.slice(seg.a, seg.b).map(f => f.f0).filter(Boolean);
const cut0 = Math.floor(pts.length * 0.15), cut1 = Math.floor(pts.length * (multi ? 0.1 : 0.05));
if (pts.length - cut0 - cut1 >= 4) pts = pts.slice(cut0, pts.length - cut1);
const s = pts.map(f => st(f, prof.bottom));
const sm = s.map((v, i) => mean(s.slice(Math.max(0, i - 1), i + 2)));
const q = Math.max(1, Math.floor(sm.length / 4));
const span = Math.max(3, st(prof.top, prof.bottom));
const start = mean(sm.slice(0, q)), end = mean(sm.slice(-q));
let imin = 0;
sm.forEach((v, i) => { if (v < sm[imin]) imin = i; });
return {
n: sm.length, curve: sm, start, end, min: sm[imin], at: sm.length > 1 ? imin / (sm.length - 1) : 0.5,
level: mean(sm), span, ms: (seg.b - seg.a) * 10,
pos: v => v / span, // 0 = speaker's low, 1 = speaker's high tone 1
};
}
// Feature vector for the tone model: the pitch curve at 5 points (in units of the speaker's
// range), its rise, low point, where the low point is, word position, length.
function featVec(F, n, i) {
const c = F.curve, sp = F.span;
const P = k => { const x = k * (c.length - 1), a = Math.floor(x), b = Math.min(c.length - 1, a + 1); return (c[a] + (c[b] - c[a]) * (x - a)) / sp; };
const p = [0, 0.25, 0.5, 0.75, 1].map(P), multi = n > 1 ? 1 : 0, fin = i === n - 1 ? 1 : 0;
return [1, ...p, p[4] - p[0], F.min / sp, F.at, multi, fin, multi * (p[4] - p[0]), multi * p[2], Math.min(F.ms, 600) / 300];
}
// Tone model: softmax over featVec, trained 2026-10-13 on the Microsoft Xiaoxiao, Xiaoyi and
// Yunxi teacher clips (check-tones.html: collect, train). Rows = tones 1-4.
const W = [[-0.37, -0.354, -0.038, 0.45, 0.767, -1.34, -0.987, 2.386, -0.274, 0.067, 0.145, 1.427, 0.34, 0.528],
[0.746, -0.848, -1.406, -1.024, 0.737, 1.31, 2.158, -0.192, -0.859, 0.534, -0.466, -0.211, -0.564, 0.102],
[0.88, 0.61, -1.135, -1.028, -0.165, 0.612, 0.002, -1.229, -0.553, -0.307, 0.308, -1.39, -0.203, 0.612],
[-1.257, 0.592, 2.58, 1.603, -1.339, -0.581, -1.173, -0.965, 1.686, -0.294, 0.014, 0.173, 0.428, -1.242]];
// Returns {tone: 1-4 or 0 (could not tell), sure: bool, p: probabilities, desc}
function classify(F, n = 1, i = 0) {
if (!F || F.n < 3) return { tone: 0, sure: false, desc: 'too short or too quiet to hear the pitch' };
const x = featVec(F, n, i), z = W.map(w => w.reduce((s, v, j) => s + v * x[j], 0));
const m = Math.max(...z), e = z.map(v => Math.exp(v - m)), S = e.reduce((a, b) => a + b), p = e.map(v => v / S);
const best = p.indexOf(Math.max(...p));
return { tone: best + 1, sure: p[best] >= 0.6, p, desc: describe(F) };
}
function describe(F) {
const p = F.pos(F.level), where = p >= 0.6 ? 'high' : p < 0.33 ? 'low' : 'middle';
const delta = Math.round(F.end - F.start), dipBy = Math.round(Math.min(F.start, F.end) - F.min);
if (dipBy >= 2 && F.at > 0.2 && F.at < 0.8) return `dipped ${dipBy} steps, then rose ${Math.round(F.end - F.min)}`;
if (delta >= 2) return `${where}, rose ${delta} steps`;
if (delta <= -2) return `${where}, fell ${-delta} steps`;
return `${where} and level`;
}
// Full analysis of one recording (Float32Array at 16 kHz) for a word with n syllables.
function analyse(x16, n, prof) {
const frames = track(x16);
prof = prof && prof.top ? prof : defaultProfile(frames);
const segs = segment(frames, n);
const multi = n > 1;
const syl = [];
for (let i = 0; i < n; i++) {
const F = segs.length === n ? features(frames, segs[i], prof, multi) : null;
syl.push({ F, ...classify(F, n, i) });
}
return { frames, segs, syl, prof, voiced: frames.some(f => f.f0) };
}
// Level of a held vowel (for the tone setup): median Hz of the longest voiced run, or 0.
function heldPitch(x16) {
const frames = track(x16), segs = segment(frames, 1);
if (!segs.length) return { hz: 0, low: 0 };
const v = frames.slice(segs[0].a, segs[0].b).map(f => f.f0).filter(Boolean).sort((a, b) => a - b);
return { hz: v[v.length >> 1] || 0, low: v[Math.floor(v.length * 0.1)] || 0 };
}
// Play a pitch curve as a hum (frames at 10 ms; unvoiced frames are silent).
function hum(ctx, frames) {
const t0 = ctx.currentTime + 0.05, osc = ctx.createOscillator(), g = ctx.createGain();
osc.type = 'triangle';
g.gain.setValueAtTime(0, t0);
osc.connect(g).connect(ctx.destination);
let on = false, last = 0;
frames.forEach((f, i) => {
const t = t0 + i * 0.01;
if (f.f0) {
osc.frequency.setValueAtTime(f.f0, t);
if (!on) { g.gain.setTargetAtTime(0.25, t, 0.01); on = true; }
last = t;
} else if (on) { g.gain.setTargetAtTime(0, t, 0.015); on = false; }
});
const end = Math.max(last, t0) + 0.1;
g.gain.setTargetAtTime(0, end - 0.08, 0.02);
osc.start(t0);
osc.stop(end + 0.1);
return (end - ctx.currentTime) * 1000;
}
async function decode(ctx, url) {
const buf = await ctx.decodeAudioData(await (await fetch(url)).arrayBuffer());
return resample(buf.getChannelData(0), buf.sampleRate);
}
function toPcm16(x16) {
const out = new Int16Array(x16.length);
for (let i = 0; i < x16.length; i++) out[i] = Math.max(-1, Math.min(1, x16[i])) * 32767;
return out;
}
return { SR, resample, track, segment, analyse, classify, featVec, heldPitch, hum, decode, toPcm16, st };
})();
// Microphone recorder: stops by itself ~0.9 s after you stop talking (or after 6 s, or stop()).
class Recorder {
constructor(ctx) { this.ctx = ctx; }
async start(onLevel) {
if (!this.stream) {
this.stream = await navigator.mediaDevices.getUserMedia({
audio: { echoCancellation: false, noiseSuppression: false, autoGainControl: false, channelCount: 1 },
});
}
const ctx = this.ctx, src = ctx.createMediaStreamSource(this.stream);
const node = ctx.createScriptProcessor(2048, 1, 1), chunks = [];
let heard = false, quiet = 0, noise = 0, n = 0, total = 0;
const blockMs = 2048 / ctx.sampleRate * 1000;
return new Promise(resolve => {
const finish = () => {
node.disconnect(); src.disconnect();
this.finish = null;
const all = new Float32Array(total);
let o = 0;
for (const c of chunks) { all.set(c, o); o += c.length; }
resolve({ audio: Tone.resample(all, ctx.sampleRate), heard });
};
this.finish = finish;
node.onaudioprocess = e => {
const x = e.inputBuffer.getChannelData(0);
chunks.push(Float32Array.from(x));
total += x.length;
let s = 0;
for (let i = 0; i < x.length; i++) s += x[i] * x[i];
const rms = Math.sqrt(s / x.length);
n++;
if (n <= 3) noise = Math.max(noise, rms); // first ~0.13 s = room noise
const thr = Math.max(0.012, noise * 3);
if (onLevel) onLevel(rms);
if (rms > thr) { heard = true; quiet = 0; } else if (heard) quiet += blockMs;
if ((heard && quiet > 900) || n * blockMs > 6000) finish();
};
src.connect(node);
node.connect(ctx.destination);
});
}
stop() { if (this.finish) this.finish(); }
}
EOFapp.js: all the screens: start page, tone setup, drill, word cards, conversations and quiz.
cat > /root/mandarin/app.js <<'EOF'
// Mandarin Tones - HSK 1 words with a tone checker. Plain JS, no build step.
// Views (by URL hash): #home, #setup, #drill, #lesson/N[/W], #talk/N[/A|B], #quiz/N.
'use strict';
const $main = document.getElementById('main');
const $say = document.getElementById('say');
const KEY = 'mandarin';
let course, ctx, rec;
let S = load();
function load() {
let s = {};
try { s = JSON.parse(localStorage.getItem(KEY)) || {}; } catch (e) { }
return Object.assign({ voice: 'mix', speed: 'n', prof: null, words: {}, quiz: {}, talk: {}, hideText: false, flip: 0 }, s);
}
function save() { try { localStorage.setItem(KEY, JSON.stringify(S)); } catch (e) { } }
function announce(text) { $say.textContent = ''; setTimeout(() => { $say.textContent = text; }, 60); }
const esc = s => String(s).replace(/[&<>"]/g, c => ({ '&': '&', '<': '<', '>': '>', '"': '"' }[c]));
const sleep = ms => new Promise(r => setTimeout(r, ms));
const audioCtx = () => (ctx = ctx || new (window.AudioContext || window.webkitAudioContext)());
// ---------- pinyin ----------
const MARKS = { a: 'āáǎà', e: 'ēéěè', i: 'īíǐì', o: 'ōóǒò', u: 'ūúǔù', v: 'ǖǘǚǜ' };
function mark(syl) { // "hao3" -> "hǎo", "nv3" -> "nǚ"
const t = +syl.slice(-1), base = syl.slice(0, -1);
if (!(t >= 1 && t <= 4)) return base.replace(/v/g, 'ü');
let i = base.search(/[ae]/);
if (i < 0) i = base.indexOf('ou');
if (i < 0) { for (let k = base.length - 1; k >= 0; k--) if ('iouv'.includes(base[k])) { i = k; break; } }
if (i < 0) return base;
return (base.slice(0, i) + MARKS[base[i]][t - 1] + base.slice(i + 1)).replace(/v/g, 'ü');
}
const sylls = w => w.py.split(' ');
const HAN = /[一-鿿]/;
// Character for each syllable + whether punctuation comes before it ("哪儿" = one syllable nar3).
function align(hz, syl) {
const out = [];
let k = 0, brk = false;
const chars = [...(hz || '')];
for (let c = 0; c < chars.length && k < syl.length; c++) {
if (!HAN.test(chars[c])) { brk = brk || /[，。！？、,.!?]/.test(chars[c]); continue; }
out.push({ ch: chars[c], brk });
brk = false;
if (/[^e]r\d$/.test(syl[k]) && chars[c + 1] === '儿') c++;
k++;
}
while (out.length < syl.length) out.push({ ch: '', brk: false });
return out;
}
// Tones as actually said (same rules as render-course.py for words): 不 before tone 4 -> 2,
// 一 before tone 4 -> 2 and before 1-3 -> 4, tone 3 before tone 3 -> 2. Not across punctuation.
function spoken(w) {
const s = sylls(w), t = s.map(x => +x.slice(-1)), a = align(w.hz, s);
for (let i = 0; i < s.length - 1; i++) {
if (a[i + 1].brk) continue;
if (a[i].ch === '不' && t[i + 1] === 4) t[i] = 2;
else if (a[i].ch === '一' && t[i] === 1 && t[i + 1] >= 1 && t[i + 1] <= 4) t[i] = t[i + 1] === 4 ? 2 : 4;
else if (t[i] === 3 && t[i + 1] === 3) t[i] = 2;
}
return s.map((x, i) => x.slice(0, -1) + t[i]);
}
const TONE_NAME = { 1: 'tone 1, high and level', 2: 'tone 2, rising', 3: 'tone 3, low or dipping', 4: 'tone 4, falling', 5: 'light tone' };
const pyHtml = s => s.map(x => `<span class="t${x.slice(-1)}">${esc(mark(x))}</span>`).join(' ');
const hex = s => [...new TextEncoder().encode(s)].map(b => b.toString(16).padStart(2, '0')).join('');
// ---------- audio ----------
// Teacher voices. Mixes take turns word by word.
const VOICES = [
['mix', 'Microsoft, female and male taking turns (Xiaoxiao and Yunxi)', ['x', 'y']],
['x', 'Microsoft Xiaoxiao, female'], ['i', 'Microsoft Xiaoyi, female'],
['y', 'Microsoft Yunxi, male'],
['all3', 'Microsoft Xiaoxiao, Yunxi and Xiaoyi taking turns', ['x', 'y', 'i']],
['g', 'Microsoft Yunyang, male (tones are weaker)'],
['both', 'Piper, female and male taking turns (xiao_ya and chaowen)', ['f', 'm']],
['f', 'Piper xiao_ya, female (tones are weaker)'], ['m', 'Piper chaowen, male (tones are weaker)'],
];
function voiceFor(n) {
const v = VOICES.find(x => x[0] === S.voice) || VOICES[0];
return v[2] ? v[2][n % v[2].length] : v[0];
}
const clipUrl = (key, v, speed) => `audio/${v}/${speed || S.speed}/${hex(key)}.mp3`;
let player;
function play(url) {
if (player) player.pause();
player = new Audio(url);
return new Promise(res => { player.onended = res; player.onerror = res; player.play().catch(res); });
}
function beep(freq, ms) {
const c = audioCtx(), o = c.createOscillator(), g = c.createGain(), t = c.currentTime;
o.frequency.value = freq; g.gain.setValueAtTime(0.15, t); g.gain.setTargetAtTime(0, t + ms / 1000, 0.02);
o.connect(g).connect(c.destination); o.start(t); o.stop(t + ms / 1000 + 0.15);
return sleep(ms + 200);
}
async function record(onLevel) {
const c = audioCtx();
if (c.state === 'suspended') await c.resume();
rec = rec || new Recorder(c);
await beep(880, 120);
const out = await rec.start(onLevel);
beep(440, 100);
return out;
}
// Teacher voices' own ranges (from their ma1 / ma3 clips) so their curves are judged the same way.
const teacherProf = {};
async function profFor(v) {
if (!teacherProf[v]) {
const c = audioCtx();
const hi = Tone.heldPitch(await Tone.decode(c, clipUrl('py:ma1', v, 'n')));
const lo = Tone.heldPitch(await Tone.decode(c, clipUrl('py:ma3', v, 'n')));
teacherProf[v] = { top: hi.hz, bottom: lo.low };
}
return teacherProf[v];
}
async function teacherCurve(key, n, v) {
const x = await Tone.decode(audioCtx(), clipUrl(key, v, 'n'));
return Tone.analyse(x, n, await profFor(v));
}
async function checkWord(x16, word, isLine) {
try {
const ctl = new AbortController(), timer = setTimeout(() => ctl.abort(), 8000);
const r = await fetch(`api/recognize?${isLine ? 'line' : 'word'}=` + encodeURIComponent(word), {
method: 'POST', body: Tone.toPcm16(x16).buffer, signal: ctl.signal,
headers: { 'Content-Type': 'application/octet-stream' },
});
clearTimeout(timer);
return r.ok ? await r.json() : null;
} catch (e) { return null; }
}
function mikeError(e) {
return e && e.name === 'NotAllowedError'
? 'The microphone is blocked. Allow microphone access for this site in your browser settings, then try again.'
: 'Could not start the microphone: ' + (e && e.message || e);
}
// ---------- drawing ----------
function draw(canvas, curves) { // curves: [{a: analysis, color, label}]
const r = canvas.getBoundingClientRect(), dpr = window.devicePixelRatio || 1;
canvas.width = r.width * dpr; canvas.height = r.height * dpr;
const g = canvas.getContext('2d'); g.scale(dpr, dpr);
const W = r.width, H = r.height;
g.clearRect(0, 0, W, H);
g.font = '13px system-ui';
curves.forEach(({ a, color, label }, k) => {
if (!a || !a.syl.length) return;
const n = a.syl.length, lo = -3, hi = Math.max(10, Tone.st(a.prof.top, a.prof.bottom) + 4);
g.strokeStyle = color; g.fillStyle = color; g.lineWidth = 3;
g.fillText(label, 8, 16 + k * 16);
a.syl.forEach((s, i) => {
if (!s.F) return;
const x0 = W * (i + 0.08) / n, x1 = W * (i + 0.92) / n;
g.beginPath();
s.F.curve.forEach((v, j) => {
const x = x0 + (x1 - x0) * j / Math.max(1, s.F.curve.length - 1);
const y = H - 6 - (H - 12) * (v - lo) / (hi - lo);
j ? g.lineTo(x, y) : g.moveTo(x, y);
});
g.stroke();
});
});
}
// ---------- views ----------
function view(html, title) {
$main.innerHTML = html;
document.title = title ? `${title} - Mandarin Tones` : 'Mandarin Tones';
const h = $main.querySelector('h1');
if (h) { h.tabIndex = -1; h.focus(); }
window.scrollTo(0, 0);
}
const allWords = () => course.lessons.flatMap(L => L.words);
function lessonProgress(i) {
const L = course.lessons[i], passed = L.words.filter(w => (S.words[w.hz] || {}).pass).length;
return { passed, total: L.words.length, quiz: S.quiz[i], talk: S.talk[i] };
}
function home() {
const items = course.lessons.map((L, i) => {
const p = lessonProgress(i), n = i + 1;
const q = p.quiz != null ? `Quiz best ${p.quiz} of 10.` : 'Quiz not taken yet.';
const t = p.talk != null ? `Conversation best ${p.talk} of ${Math.ceil((L.talk || []).length / 2)} lines.` : 'Conversation not practiced yet.';
return `<li class="box"><h3>Lesson ${n}: ${esc(L.title)}</h3>
<p class="prog">${p.passed} of ${p.total} words said right. ${t} ${q}</p>
<p><a href="#lesson/${n}">Words</a> · <a href="#talk/${n}">Conversation</a> · <a href="#quiz/${n}">Quiz</a></p></li>`;
}).join('');
view(`<h1>Mandarin Tones</h1>
<p>Learn the ${allWords().length} words of HSK 1, the first official level of Mandarin Chinese. Each lesson has three parts: learn the words, practice a short conversation where you take one of the two parts, and a listening quiz. The app listens to you and checks the words and each tone.</p>
<div class="box"><h2>1. Tone setup</h2>
<p>${S.prof ? 'Done. The tone checker knows your voice range.' : 'Not done yet. Takes about a minute: you say two notes so the checker knows how high and low your voice goes.'}</p>
<a href="#setup">${S.prof ? 'Redo the tone setup' : 'Start the tone setup'}</a></div>
<div class="box"><h2>2. Tone drill</h2><p>Practice hearing and saying the four tones on single sounds like ma and ba.</p><a href="#drill">Open the tone drill</a></div>
<h2>3. Lessons</h2><ul class="lessons">${items}</ul>
<div class="box"><h2>Settings</h2>
<label>Teacher voice <select id="voice">${VOICES.map(([k, name]) => `<option value="${k}">${esc(name)}</option>`).join('')}</select></label>
<label>Speed <select id="speed"><option value="n">Normal</option><option value="s">Slow</option></select></label>
<p class="small">Headphones help: the checker then hears only you. The tone check runs in your browser. For the word check your recording is sent to this site's server, checked, and thrown away; nothing is saved. Progress is kept in this browser only.</p></div>`);
const v = document.getElementById('voice'), sp = document.getElementById('speed');
v.value = S.voice; sp.value = S.speed;
v.onchange = () => { S.voice = v.value; save(); };
sp.onchange = () => { S.speed = sp.value; save(); };
}
function setup() {
let step = 0, top = 0;
const steps = [
{ say: 'mā', key: 'py:ma1', text: 'Say <b>mā</b> (ma1): tone 1, high and level, like singing one steady note. Hold it for about a second.' },
{ say: 'mǎ', key: 'py:ma3', text: 'Say <b>mǎ</b> (ma3): tone 3, go down low, as low as is comfortable, like a sigh.' },
];
function show(msg) {
const st = steps[step];
view(`<h1>Tone setup, step ${step + 1} of 2</h1>
<p>${st.text}</p>
<p class="small">After you press "Say it" you will hear a high beep. Speak after the beep. A low beep means it stopped listening.</p>
<div class="row"><button id="hear">Hear the example</button><button id="go" class="main">Say it</button></div>
<div class="meter" aria-hidden="true"><i id="lvl"></i></div>
<div id="res" class="result" role="status">${msg || ''}</div>
<p><a href="#home">Back to the start page</a></p>`, 'Tone setup');
document.getElementById('hear').onclick = () => play(clipUrl(st.key, voiceFor(S.flip++)));
document.getElementById('go').onclick = go;
}
async function go() {
const $r = document.getElementById('res'), lvl = document.getElementById('lvl');
let out;
try { out = await record(v => { lvl.style.width = Math.min(100, v * 600) + '%'; }); } catch (e) { $r.textContent = mikeError(e); return; }
const p = Tone.heldPitch(out.audio);
if (!p.hz) { $r.textContent = 'I could not hear a voice. Try again, a little louder or closer to the microphone.'; announce($r.textContent); return; }
if (step === 0) {
top = p.hz; step = 1;
show(`Got it: your high note is about ${Math.round(top)} hertz. Now step 2.`);
announce(`Got it, ${Math.round(top)} hertz. Step 2.`);
} else {
const bottom = p.low, gap = Tone.st(top, bottom);
if (gap < 3) {
step = 0;
show(`Those two were only ${gap.toFixed(1)} steps apart. Let's redo it: make the first one higher and the second one lower.`);
announce('Too close together. Starting again at step 1.');
return;
}
S.prof = { top, bottom }; save();
view(`<h1>Tone setup done</h1><p>Your range: ${Math.round(bottom)} to ${Math.round(top)} hertz, about ${Math.round(gap)} musical half steps. The checker will judge your tones inside this range.</p>
<p><a href="#drill">Go to the tone drill</a></p><p><a href="#lesson/1">Start lesson 1</a></p><p><a href="#home">Back to the start page</a></p>`, 'Tone setup done');
}
}
show();
}
// Grade one recording for a word: tones (browser) + word (server). Returns {html, text, pass, a}
async function grade(x16, w) {
const want = spoken(w), n = want.length;
const [a, word] = await Promise.all([Promise.resolve(Tone.analyse(x16, n, S.prof)), w.hz ? checkWord(x16, w.hz) : null]);
if (!a.voiced) return { html: '<p class="no">I could not hear a voice. Try again, a little louder.</p>', text: 'I could not hear a voice. Try again.', pass: false, a };
const lines = [], said = [];
let checks = 0, good = 0;
if (w.hz) {
if (!word) lines.push(['', 'Word check is not available right now; tones only.']);
else if (word.ok) { lines.push(['ok', `Word: right, I heard ${word.heard}.`]); checks++; good++; }
else { lines.push(['no', word.heard ? `Word: I heard ${word.heard} instead.` : 'Word: I could not make out the word.']); checks++; }
}
want.forEach((syl, i) => {
const t = +syl.slice(-1), s = a.syl[i], name = mark(syl);
if (t === 5) { lines.push(['', `${name}: light tone, not graded.`]); return; }
checks++;
if (!s.tone) { lines.push(['no', `${name}: could not hear the pitch clearly.`]); said.push('?'); return; }
const ok = s.tone === t;
if (ok) good++;
said.push(s.tone);
lines.push([ok ? 'ok' : 'no', `${name}: wanted ${TONE_NAME[t]}; heard tone ${s.tone}${s.sure ? '' : ' (not sure)'}: ${s.desc}. ${ok ? 'Right.' : 'Try again.'}`]);
});
const pass = checks > 0 && good === checks;
const head = pass ? 'All right!' : `${good} of ${checks} right.`;
return {
html: `<p class="${pass ? 'ok' : 'no'}"><b>${head}</b></p><ul>${lines.map(([c, t]) => `<li class="${c}">${esc(t)}</li>`).join('')}</ul>`,
text: head + ' ' + lines.map(l => l[1]).join(' '), pass, score: checks ? good / checks : 0, a,
};
}
// Character -> sound without tone ("他" and "她" are both "ta"), from the course's own pinyin.
// Lines are compared by sound so sound-alike characters the recognizer picks don't count as wrong.
let SOUND = null;
const plain = syl => syl.replace(/\d$/, '').replace(/([^e])r$/, '$1');
function soundMap() {
if (SOUND) return SOUND;
SOUND = {};
const add = (hz, py) => align(hz, py.split(' ')).forEach((a, i) => { if (a.ch && !SOUND[a.ch]) SOUND[a.ch] = plain(py.split(' ')[i]); });
course.lessons.forEach(L => { L.words.forEach(w => add(w.hz, w.py)); (L.talk || []).forEach(t => add(t[1], t[2])); });
Object.values(course.people).forEach(p => add(p.hz, p.py));
return SOUND;
}
function heardSounds(text) {
const m = soundMap(), out = [];
[...text].filter(c => HAN.test(c)).forEach((c, i, all) => {
if (c === '儿' && i && '哪那这'.includes(all[i - 1])) return; // nǎr: 儿 is part of the syllable
out.push(m[c] || c);
});
return out;
}
// Longest common subsequence length (characters said in the right order).
function lcs(a, b) {
const d = Array(b.length + 1).fill(0);
for (const x of a) { let prev = 0; for (let j = 1; j <= b.length; j++) { const t = d[j]; d[j] = x === b[j - 1] ? prev + 1 : Math.max(d[j], d[j - 1]); prev = t; } }
return d[b.length];
}
// Grade one conversation line {hz, py}: passes when 80% of the sounds are heard in order (compared
// without tones, so sound-alike characters count). Tones are only a guide here: a tone counts if it
// matches the tone as said OR as written (tone changes are optional in fast speech).
async function gradeLine(x16, line) {
const want = spoken(line), cite = sylls(line), n = want.length;
const [a, word] = await Promise.all([Promise.resolve(Tone.analyse(x16, n, S.prof)), checkWord(x16, line.hz, true)]);
if (!a.voiced) return { html: '<p class="no">I could not hear a voice. Try again, a little louder.</p>', text: 'I could not hear a voice. Try again.', pass: false, a };
const exp = cite.map(plain), out = [];
let wordsOk = null;
if (word) {
const heardText = [...(word.heard || '')].filter(c => HAN.test(c)).join(''), got = lcs(exp, heardSounds(heardText));
wordsOk = got / exp.length;
out.push([wordsOk >= 0.8 ? 'ok' : 'no', heardText ? `Words: I heard ${heardText}, ${got} of ${exp.length} sounds right.` : 'Words: I could not make out the words.']);
} else out.push(['', 'Word check is not available right now; tones only.']);
let graded = 0, right = 0;
const wrong = [];
want.forEach((syl, i) => {
const t = +syl.slice(-1), c = +cite[i].slice(-1), s = a.syl[i];
if (t === 5) return;
graded++;
if (s.tone === t || s.tone === c) right++;
else wrong.push(`${mark(syl)}: wanted tone ${t}, heard ${s.tone ? 'tone ' + s.tone + ' (' + s.desc + ')' : 'no clear pitch'}`);
});
// Tone guessing in running speech is rough (about 60% on the teacher voices), so in
// conversations tones are a hint only; the strict tone check is on the single-word cards.
out.push(['', `Tone guide, rough in full sentences: ${right} of ${graded} tones matched.`]);
wrong.forEach(x => out.push(['', 'Maybe check: ' + x]));
const pass = wordsOk != null && wordsOk >= 0.8;
const head = wordsOk == null ? 'The word check is not available right now.' : pass ? 'Good, that line works!' : 'Not quite yet. Listen again and try once more.';
return {
html: `<p class="${pass ? 'ok' : 'no'}"><b>${head}</b></p><ul>${out.map(([c, t]) => `<li class="${c}">${esc(t)}</li>`).join('')}</ul>`,
text: head + ' ' + out.map(l => l[1]).join('. '), pass, a,
};
}
// Conversation: listen to all of it, or take part A or B. #talk/N[/A|B]
function talk(li, role) {
const L = course.lessons[li];
if (!L || !L.talk) return home();
const P = course.people, lines = L.talk.map(([who, hz, py, en]) => ({ who, hz, py, en }));
const name = who => `${P[who].hz} (${sylls(P[who]).map(mark).join(' ')}, ${P[who].about})`;
const lineHtml = (l, i, hide) => `<div class="talk-line ${l.who}" id="ln${i}">
<span class="who">${esc(P[l.who].hz)} <span class="small">${esc(sylls(P[l.who]).map(mark).join(' '))}</span>:</span>
${hide ? `<div><i>Your line. In English: ${esc(l.en)}</i></div>` : `<div class="zh" lang="zh-CN">${esc(l.hz)}</div>
<div class="py">${pyHtml(sylls(l))}</div><div class="num">${esc(l.py)}</div><div class="en">${esc(l.en)}</div>`}
</div>`;
const clip = (l, sp) => clipUrl(l.hz, P[l.who].voice, sp);
if (!role) { // overview + listen to everything
view(`<h1>Lesson ${li + 1} conversation: ${esc(L.title)}</h1>
<p>Two people talk: ${esc(name('A'))} and ${esc(name('B'))}. Listen first, then take one part and say those lines yourself.</p>
<div class="row"><button id="all" class="main">Listen to the whole conversation</button><button id="stop">Stop</button></div>
<div class="row"><a href="#talk/${li + 1}/A">Practice as ${esc(P.A.hz)} (${esc(sylls(P.A).map(mark).join(' '))})</a> <a href="#talk/${li + 1}/B">Practice as ${esc(P.B.hz)} (${esc(sylls(P.B).map(mark).join(' '))})</a></div>
<div id="lines">${lines.map((l, i) => lineHtml(l, i) + `<button data-i="${i}" class="one">Play line ${i + 1}</button>`).join('')}</div>
<p><a href="#lesson/${li + 1}">Words for this lesson</a> · <a href="#quiz/${li + 1}">Quiz</a> · <a href="#home">All lessons</a></p>`, `Conversation ${li + 1}`);
let stopped = false;
document.getElementById('stop').onclick = () => { stopped = true; if (player) player.pause(); };
document.getElementById('all').onclick = async () => {
stopped = false;
for (let i = 0; i < lines.length && !stopped; i++) {
document.querySelectorAll('.talk-line').forEach(e => e.classList.toggle('now', e.id === 'ln' + i));
await play(clip(lines[i]));
await sleep(500);
}
document.querySelectorAll('.talk-line').forEach(e => e.classList.remove('now'));
};
$main.querySelectorAll('button.one').forEach(b => { b.onclick = () => play(clip(lines[+b.dataset.i])); });
return;
}
const mine = lines.map((l, i) => l.who === role ? i : -1).filter(i => i >= 0);
let k = 0, passed = 0;
const results = {};
async function step() {
if (k >= lines.length) return finish();
const l = lines[k], my = l.who === role, myNum = mine.indexOf(k) + 1;
view(`<h1>Conversation ${li + 1}, you are ${esc(P[role].hz)}: line ${k + 1} of ${lines.length}</h1>
<label><input type="checkbox" id="hide" ${S.hideText ? 'checked' : ''}> Challenge: hide the Chinese for my lines (show only the English)</label>
${lineHtml(l, k, my && S.hideText)}
${my ? `<p>Your turn (your line ${myNum} of ${mine.length}). Say it after the high beep.</p>
<div class="row"><button id="go" class="rec">Say it</button><button id="hear">Hear it first</button><button id="slow">Hear it slowly</button><button id="skip">Skip</button></div>
<div class="meter" aria-hidden="true"><i id="lvl"></i></div><div id="res" class="result"></div>
<div class="row"><button id="next" class="main" disabled>Next line</button></div>`
: `<p>${esc(P[l.who].hz)} says this line.</p><div class="row"><button id="again">Play it again</button><button id="next" class="main">Next line</button></div>`}
<p><a href="#talk/${li + 1}">Back to the whole conversation</a></p>`, `Conversation ${li + 1}`);
document.getElementById('hide').onchange = e => { S.hideText = e.target.checked; save(); step(); };
const next = document.getElementById('next');
next.onclick = () => { k++; step(); };
if (!my) {
document.getElementById('again').onclick = () => play(clip(l));
play(clip(l)).then(() => next.focus());
return;
}
const $r = document.getElementById('res'), lvl = document.getElementById('lvl');
document.getElementById('hear').onclick = () => play(clip(l));
document.getElementById('slow').onclick = () => play(clip(l, 's'));
document.getElementById('skip').onclick = () => { k++; step(); };
document.getElementById('go').onclick = async () => {
const btn = document.getElementById('go');
btn.disabled = true;
let out;
try { out = await record(x => { lvl.style.width = Math.min(100, x * 600) + '%'; }); }
catch (e) { btn.disabled = false; $r.textContent = mikeError(e); announce($r.textContent); return; }
lvl.style.width = '0';
$r.textContent = 'Checking...';
const g = await gradeLine(out.audio, l);
$r.innerHTML = g.html;
announce(g.text);
if (g.pass && !results[k]) { results[k] = true; passed++; }
btn.disabled = false;
next.disabled = false;
};
}
function finish() {
S.talk[li] = Math.max(S.talk[li] || 0, passed); save();
const other = role === 'A' ? 'B' : 'A';
view(`<h1>Conversation done</h1><p>You said ${passed} of ${mine.length} of ${esc(P[role].hz)}'s lines well.</p>
<p><a href="#talk/${li + 1}/${role}">Practice as ${esc(P[role].hz)} again</a></p>
<p><a href="#talk/${li + 1}/${other}">Now switch: practice as ${esc(P[other].hz)}</a></p>
<p><a href="#quiz/${li + 1}">Take the lesson ${li + 1} quiz</a></p>
<p><a href="#home">All lessons</a></p>`, 'Conversation done');
}
step();
}
function lesson(li, wi) {
const L = course.lessons[li];
if (!L) return home();
const w = L.words[wi], key = w.hz, want = spoken(w), cite = sylls(w);
const v = voiceFor(wi);
const changed = want.join(' ') !== cite.join(' ');
const rec0 = S.words[w.hz];
view(`<h1>Lesson ${li + 1}: ${esc(L.title)}</h1>
<p>Word ${wi + 1} of ${L.words.length}${rec0 && rec0.pass ? ', already said right' : ''}</p>
<div class="box card">
<p class="sr">Word: <span lang="zh-CN">${esc(w.hz)}</span>. Pinyin ${esc(cite.join(' '))}. Means ${esc(w.en)}.</p>
<div class="hz" lang="zh-CN" aria-hidden="true">${esc(w.hz)}</div>
<div class="py" aria-hidden="true">${pyHtml(cite)}</div>
<div class="num" aria-hidden="true">${esc(cite.join(' '))}</div>
<div class="en" aria-hidden="true">${esc(w.en)}</div>
<p class="note">Tones: ${cite.map(s => TONE_NAME[s.slice(-1)]).join('; ')}.</p>
${changed ? `<p class="note">Said as ${esc(want.join(' '))}: ${cite.some((s, i) => s === 'bu4' && want[i] === 'bu2') ? '不 (bu) changes to tone 2 before a tone 4.' : 'a tone 3 before another tone 3 changes to tone 2.'} The checker expects the changed tone.</p>` : ''}
${w.note ? `<p class="note">${esc(w.note)}</p>` : ''}
<div class="row"><button id="hear" class="main">Listen</button><button id="slow">Listen slowly</button><button id="go" class="rec">Say it</button></div>
<div class="meter" aria-hidden="true"><i id="lvl"></i></div>
<div id="res" class="result"></div>
<div class="row"><button id="mine" disabled>Hear my pitch as a hum</button><button id="theirs">Hear the teacher's pitch as a hum</button></div>
<canvas id="cv" aria-hidden="true"></canvas>
</div>
<div class="row"><button id="prev" ${wi ? '' : 'disabled'}>Previous word</button><button id="next">${wi + 1 < L.words.length ? 'Next word' : 'Go to the conversation'}</button></div>
<p><a href="#home">Back to all lessons</a></p>`, `Lesson ${li + 1}`);
let mine = null, theirs = null;
const $r = document.getElementById('res'), cv = document.getElementById('cv'), lvl = document.getElementById('lvl');
const teacher = () => theirs || teacherCurve(key, want.length, v).then(a => (theirs = a));
document.getElementById('hear').onclick = () => play(clipUrl(key, v));
document.getElementById('slow').onclick = () => play(clipUrl(key, v, 's'));
document.getElementById('theirs').onclick = async () => Tone.hum(audioCtx(), (await teacher()).frames);
document.getElementById('mine').onclick = () => mine && Tone.hum(audioCtx(), mine.frames);
document.getElementById('prev').onclick = () => { location.hash = `#lesson/${li + 1}/${wi}`; };
document.getElementById('next').onclick = () => { location.hash = wi + 1 < L.words.length ? `#lesson/${li + 1}/${wi + 2}` : `#talk/${li + 1}`; };
document.getElementById('go').onclick = async () => {
const btn = document.getElementById('go');
btn.disabled = true;
$r.innerHTML = '';
let out;
try { out = await record(x => { lvl.style.width = Math.min(100, x * 600) + '%'; }); }
catch (e) { btn.disabled = false; $r.textContent = mikeError(e); announce($r.textContent); return; }
lvl.style.width = '0';
$r.textContent = 'Checking...';
const g = await grade(out.audio, w);
mine = g.a;
document.getElementById('mine').disabled = false;
$r.innerHTML = g.html + (S.prof ? '' : '<p class="small">Tip: do the tone setup on the start page so the checker knows your voice range.</p>');
announce(g.text);
const r = S.words[w.hz] || { tries: 0, best: 0 };
r.tries++; r.best = Math.max(r.best, g.score || 0); if (g.pass) r.pass = true;
S.words[w.hz] = r; save();
btn.disabled = false;
draw(cv, [{ a: await teacher().catch(() => null), color: '#8ab4ff', label: 'Teacher (blue)' }, { a: mine, color: '#ffd166', label: 'You (yellow)' }]);
};
teacher().then(a => draw(cv, [{ a, color: '#8ab4ff', label: 'Teacher (blue)' }])).catch(() => { });
}
function shuffle(a) { for (let i = a.length - 1; i > 0; i--) { const j = Math.floor(Math.random() * (i + 1)); [a[i], a[j]] = [a[j], a[i]]; } return a; }
function quiz(li) {
const L = course.lessons[li];
if (!L) return home();
const qs = shuffle(L.words.slice()).slice(0, 10).map((w, i) => ({ w, kind: i % 2 && sylls(w).some(s => !s.endsWith('5')) ? 'tones' : 'meaning' }));
let k = 0, score = 0;
function ask(msg) {
if (k >= qs.length) return done();
const { w, kind } = qs[k], v = voiceFor(k);
let opts;
if (kind === 'meaning') {
opts = shuffle([w, ...shuffle(L.words.filter(x => x !== w && x.en !== w.en)).slice(0, 3)]).map(x => ({ label: x.en, ok: x === w }));
} else {
const right = sylls(w).join(' '), set = new Set([right]);
let guard = 0;
while (set.size < 4 && guard++ < 50) {
set.add(sylls(w).map(s => { const t = +s.slice(-1); return s.slice(0, -1) + (t === 5 || Math.random() < 0.5 ? t : 1 + Math.floor(Math.random() * 4)); }).join(' '));
}
opts = shuffle([...set]).map(p => ({ label: `${p.split(' ').map(mark).join(' ')} (${p})`, ok: p === right }));
}
view(`<h1>Lesson ${li + 1} quiz, question ${k + 1} of ${qs.length}</h1>
${msg ? `<p role="status">${esc(msg)}</p>` : ''}
<p>${kind === 'meaning' ? 'Listen. What does it mean?' : 'Listen. Which pinyin has the right tones?'}</p>
<div class="row"><button id="hear" class="main">Play the word</button></div>
<div class="row choices">${opts.map((o, i) => `<button data-i="${i}">${esc(o.label)}</button>`).join('')}</div>
<p><a href="#lesson/${li + 1}">Back to the lesson</a></p>`, 'Quiz');
const go = () => play(clipUrl(w.hz, v));
document.getElementById('hear').onclick = go;
$main.querySelectorAll('.choices button').forEach(b => {
b.onclick = () => {
const o = opts[+b.dataset.i];
if (o.ok) score++;
const fb = `${o.ok ? 'Right' : 'Not quite'}: it was ${w.hz}, ${sylls(w).join(' ')}, ${w.en}. Score ${score} of ${k + 1}.`;
k++;
ask(fb);
};
});
setTimeout(go, 400);
}
function done() {
S.quiz[li] = Math.max(S.quiz[li] || 0, score); save();
view(`<h1>Quiz done: ${score} of ${qs.length}</h1>
<p>${score === qs.length ? 'Perfect!' : 'Try it again any time to beat your score.'}</p>
<p><a href="#quiz/${li + 1}">Take the quiz again</a></p>
${course.lessons[li + 1] ? `<p><a href="#lesson/${li + 2}">Go to lesson ${li + 2}: ${esc(course.lessons[li + 1].title)}</a></p>` : ''}
<p><a href="#home">Back to all lessons</a></p>`, 'Quiz done');
}
ask();
}
function drill() {
const BASES = ['ma', 'yi', 'ba', 'tang', 'shi', 'wu', 'qi', 'fan'];
let heard = { right: 0, all: 0 }, said = { right: 0, all: 0 }, cur = null, n = 0;
const pick = () => { const b = BASES[Math.floor(Math.random() * BASES.length)], t = 1 + Math.floor(Math.random() * 4); return { b, t, key: `py:${b}${t}`, v: voiceFor(n++) }; };
view(`<h1>Tone drill</h1>
<div class="box"><h2>Hear the tone</h2>
<p>Play a sound, then pick the tone you heard.</p>
<div class="row"><button id="hplay" class="main">Play a sound</button></div>
<div class="row choices" id="hpick">${[1, 2, 3, 4].map(t => `<button data-t="${t}" disabled>Tone ${t}</button>`).join('')}</div>
<p id="hres" role="status"></p></div>
<div class="box"><h2>Say the tone</h2>
<p id="sq">Press "New sound" to get a sound and a tone to say.</p>
<div class="row"><button id="snew" class="main">New sound</button><button id="shear" disabled>Hear it</button><button id="sgo" class="rec" disabled>Say it</button></div>
<div class="meter" aria-hidden="true"><i id="lvl"></i></div>
<div id="sres" class="result"></div>
<canvas id="cv" aria-hidden="true"></canvas></div>
<p><a href="#home">Back to the start page</a></p>`, 'Tone drill');
let hcur = null;
const $hres = document.getElementById('hres'), picks = [...document.querySelectorAll('#hpick button')];
document.getElementById('hplay').onclick = () => {
if (!hcur) { hcur = pick(); $hres.textContent = ''; }
picks.forEach(b => { b.disabled = false; });
play(clipUrl(hcur.key, hcur.v));
};
picks.forEach(b => {
b.onclick = () => {
if (!hcur) return;
const ok = +b.dataset.t === hcur.t;
heard.all++; if (ok) heard.right++;
$hres.textContent = `${ok ? 'Right' : 'Not quite'}: that was ${mark(hcur.b + hcur.t)}, ${TONE_NAME[hcur.t]}. Score ${heard.right} of ${heard.all}. Press Play a sound for the next one.`;
hcur = null;
picks.forEach(x => { x.disabled = true; });
document.getElementById('hplay').focus();
};
});
const sq = document.getElementById('sq'), sres = document.getElementById('sres'), lvl = document.getElementById('lvl');
document.getElementById('snew').onclick = () => {
cur = pick();
sq.innerHTML = `Say <b>${esc(mark(cur.b + cur.t))}</b> (${cur.b}${cur.t}): ${TONE_NAME[cur.t]}.`;
sres.innerHTML = '';
document.getElementById('shear').disabled = false;
document.getElementById('sgo').disabled = false;
announce(`Say ${cur.b} ${cur.t}, ${TONE_NAME[cur.t]}.`);
};
document.getElementById('shear').onclick = () => cur && play(clipUrl(cur.key, cur.v));
document.getElementById('sgo').onclick = async () => {
if (!cur) return;
let out;
try { out = await record(x => { lvl.style.width = Math.min(100, x * 600) + '%'; }); }
catch (e) { sres.textContent = mikeError(e); announce(sres.textContent); return; }
lvl.style.width = '0';
const g = await grade(out.audio, { hz: '', py: cur.b + cur.t });
said.all++; if (g.pass) said.right++;
sres.innerHTML = g.html + `<p>Score ${said.right} of ${said.all}.</p>`;
announce(g.text + ` Score ${said.right} of ${said.all}.`);
const t = await teacherCurve(cur.key, 1, cur.v).catch(() => null);
draw(document.getElementById('cv'), [{ a: t, color: '#8ab4ff', label: 'Teacher (blue)' }, { a: g.a, color: '#ffd166', label: 'You (yellow)' }]);
};
}
function route() {
if (rec) rec.stop();
const [name, a, b] = location.hash.replace(/^#/, '').split('/');
if (name === 'setup') setup();
else if (name === 'drill') drill();
else if (name === 'lesson') lesson(+a - 1, Math.max(0, (+b || 1) - 1));
else if (name === 'quiz') quiz(+a - 1);
else if (name === 'talk') talk(+a - 1, b === 'A' || b === 'B' ? b : null);
else home();
}
fetch('course.json?v=3').then(r => r.json()).then(c => { course = c; window.addEventListener('hashchange', route); route(); })
.catch(() => { $main.innerHTML = '<h1>Could not load the course</h1><p>Please reload the page.</p>'; });
EOFcheck-tones.html: a test page that runs the tone checker over every teacher clip (the teacher's tones are right, so every mismatch is a checker mistake). It's also where the tone model gets retrained (see Notes).
cat > /root/mandarin/check-tones.html <<'EOF'
<!doctype html>
<html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1">
<title>Tone checker test</title><link rel="stylesheet" href="style.css?v=1"></head>
<body><main>
<h1>Tone checker test</h1>
<p>Runs the tone checker on every teacher clip (both voices, normal and slow). The teacher's tones are right, so each mismatch is a checker mistake.</p>
<button id="go" class="main">Run the test</button>
<p id="sum" role="status"></p>
<ul id="bad"></ul>
</main>
<script src="tone.js?v=4"></script>
<script>
'use strict';
const hex = s => [...new TextEncoder().encode(s)].map(b => b.toString(16).padStart(2, '0')).join('');
function spoken(w) {
const s = w.py.split(' '), t = s.map(x => +x.slice(-1));
for (let i = 0; i < s.length - 1; i++) {
if ((w.hz || '')[i] === '不' && t[i + 1] === 4) t[i] = 2;
else if (t[i] === 3 && t[i + 1] === 3) t[i] = 2;
}
return t;
}
// --- model training helpers (run from the console): collect(voices) -> FEAT, then train/pred ---
window.collect = async (voices, speeds = ['n', 's']) => {
window.FEAT = []; window.FDONE = false;
const ctx = new AudioContext(), course = await (await fetch('course.json')).json();
const items = course.lessons.flatMap(L => L.words).map(w => ({ key: w.hz, w }));
for (const b of ['ma', 'yi', 'ba', 'tang', 'shi', 'wu', 'qi', 'fan']) for (let t = 1; t <= 4; t++) items.push({ key: `py:${b}${t}`, w: { py: b + t } });
for (const v of voices) {
const url = (k, s) => `audio/${v}/${s}/${hex(k)}.mp3`;
const hi = Tone.heldPitch(await Tone.decode(ctx, url('py:ma1', 'n'))), lo = Tone.heldPitch(await Tone.decode(ctx, url('py:ma3', 'n')));
const prof = { top: hi.hz, bottom: lo.low };
for (const sp of speeds) for (const it of items) {
const t = spoken(it.w), a = Tone.analyse(await Tone.decode(ctx, url(it.key, sp)), t.length, prof);
t.forEach((tone, i) => {
const F = a.syl[i].F;
if (tone === 5) return;
FEAT.push({ v, sp, key: it.key, i, n: t.length, want: tone, got: a.syl[i].tone, segs: a.segs.length, prof, F });
});
}
}
FDONE = true;
};
window.train = (rows, iters = 1500, lr = 0.3, l2 = 0.001) => {
const X = rows.map(r => Tone.featVec(r.F, r.n, r.i)), Y = rows.map(r => r.want - 1), d = X[0].length;
const W = [0, 1, 2, 3].map(() => new Array(d).fill(0));
for (let it = 0; it < iters; it++) {
const G = W.map(w => w.map(() => 0));
X.forEach((x, n) => {
const z = W.map(w => w.reduce((s, v, j) => s + v * x[j], 0)), m = Math.max(...z), e = z.map(v => Math.exp(v - m)), S = e.reduce((a, b) => a + b);
e.forEach((v, k) => { const g = v / S - (k === Y[n] ? 1 : 0); for (let j = 0; j < d; j++) G[k][j] += g * x[j]; });
});
W.forEach((w, k) => w.forEach((v, j) => { w[j] -= lr * (G[k][j] / X.length + l2 * v); }));
}
return W;
};
window.pred = (W, r) => { const x = Tone.featVec(r.F, r.n, r.i), z = W.map(w => w.reduce((s, v, j) => s + v * x[j], 0)); return z.indexOf(Math.max(...z)) + 1; };
window.heldOut = (voices) => {
const rows = FEAT.filter(f => f.F && f.F.curve.length >= 3), out = [];
for (const hold of voices) {
const W = train(rows.filter(r => r.v !== hold)), test = rows.filter(r => r.v === hold);
const M = [1, 2, 3, 4].map(() => [0, 0, 0, 0]);
test.forEach(r => M[r.want - 1][pred(W, r) - 1]++);
const ok = M.reduce((s, r, i) => s + r[i], 0);
out.push(`held-out ${hold}: ${ok}/${test.length} = ${(100 * ok / test.length).toFixed(1)}% ` + M.map((r, i) => `T${i + 1}[${r.join(',')}]`).join(' '));
}
return out.join('\n');
};
document.getElementById('go').onclick = async () => {
const ctx = new AudioContext(), course = await (await fetch('course.json')).json();
const items = course.lessons.flatMap(L => L.words).map(w => ({ key: w.hz, w }));
for (const b of ['ma', 'yi', 'ba', 'tang', 'shi', 'wu', 'qi', 'fan']) for (let t = 1; t <= 4; t++) items.push({ key: `py:${b}${t}`, w: { py: b + t } });
const res = { total: 0, right: 0, byTone: {} }, bad = [], sum = document.getElementById('sum');
window.RESULTS = { res, bad };
const only = new URLSearchParams(location.search).get('v');
for (const v of only ? only.split(',') : ['x', 'i', 'y', 'g', 'f', 'm']) {
res.byVoice = res.byVoice || {};
const vr = res.byVoice[v] = { total: 0, right: 0 };
const url = (k, s) => `audio/${v}/${s}/${hex(k)}.mp3`;
const hi = Tone.heldPitch(await Tone.decode(ctx, url('py:ma1', 'n'))), lo = Tone.heldPitch(await Tone.decode(ctx, url('py:ma3', 'n')));
const prof = { top: hi.hz, bottom: lo.low };
for (const sp of ['n', 's']) {
for (const it of items) {
const want = spoken(it.w), a = Tone.analyse(await Tone.decode(ctx, url(it.key, sp)), want.length, prof);
want.forEach((t, i) => {
if (t === 5) return;
const s = a.syl[i], bt = res.byTone[t] = res.byTone[t] || { total: 0, right: 0 };
res.total++; bt.total++; vr.total++;
if (s.tone === t) { res.right++; bt.right++; vr.right++; }
else bad.push(`${v}/${sp} ${it.key} ${it.w.py} syl ${i + 1}: want ${t} got ${s.tone} (${s.desc}) segs=${a.segs.length}` +
(s.F ? ` start=${s.F.start.toFixed(1)} end=${s.F.end.toFixed(1)} min=${s.F.min.toFixed(1)} at=${s.F.at.toFixed(2)} span=${s.F.span.toFixed(1)}` : ''));
});
sum.textContent = `${res.right} of ${res.total} right so far`;
}
}
}
sum.textContent = `Done: ${res.right} of ${res.total} right (${(100 * res.right / res.total).toFixed(1)}%). ` +
Object.entries(res.byTone).map(([t, b]) => `Tone ${t}: ${b.right}/${b.total}`).join(', ') + '. Voices: ' +
Object.entries(res.byVoice).map(([v, b]) => `${v} ${b.right}/${b.total}`).join(', ');
document.getElementById('bad').innerHTML = bad.map(x => `<li>${x}</li>`).join('');
};
</script></body></html>
EOF3. Pack the site files
Proxmox host shell:
tar -cf site.tar index.html style.css tone.js app.js course.json check-tones.html4. Create the container
Proxmox host shell. Takes about 30 seconds and prints the new container's IP at the end.
bash create-ct.sh5. Install the software inside CT 309
Proxmox host shell. Takes about 2 minutes (downloads about 250 MB).
pct push 309 setup-ct.sh /root/setup-ct.shpct exec 309 -- bash /root/setup-ct.shIt should end with SETUP-DONE.
6. Install the site and record the voices
Proxmox host shell. Recording takes about 8 to 10 minutes in total: 1,640 Microsoft clips (4 at a time) and 724 Piper clips.
bash install.sh renderIt should end with EDGE-DONE made ... clips, RENDER-DONE ..., then {"ok": true, "words": 149} and INSTALL-DONE. If a few Microsoft clips print FAILED, run the same command again: it only records what's missing.
7. Add the site to the edge Caddy (CT 108)
Proxmox host shell:
pct push 108 edge-add.sh /root/edge-add-mandarin.shThis passes CT 309's current IP to the script:
pct exec 108 -- bash /root/edge-add-mandarin.sh $(pct exec 309 -- hostname -I | awk '{print $1}')It should print Valid configuration and EDGE-OK. Open https://mandarin.503mail.com. Caddy gets the certificate on the first visit, so the very first load can take a few seconds.
8. Check it
Open https://mandarin.503mail.com/check-tones.html and press Run the test. With all six voices it takes about 4 minutes. The 13 October 2026 results:
- Xiaoxiao 91% right
- Xiaoyi 77%
- Piper xiao_ya 65%
- Yunxi 58%
- Piper chaowen 55%
- Yunyang 33%
A big drop from these numbers means something is wrong with the audio or tone.js.
Updating the site later
Edit the files in /root/mandarin, rebuild site.tar (step 3), then run one of these in the Proxmox host shell. Use the first after changing only page files, and the second after adding words or conversation lines to course.json (it records only the new clips):
bash install.shbash install.sh renderAfter changing app.js, tone.js or style.css, raise the ?v= numbers in index.html so browsers load the new copy.
Troubleshooting: where the logs are
Word check (Vosk)
- Where: systemd journal for
mandarin-speech, in CT 309 (mandarin) - Open: Proxmox host shell
pct exec 309 -- journalctl -u mandarin-speech -n 50 --no-pager- Look for: nothing at all is normal. It logs no requests on purpose (people's voices).
TracebackorErrno 98 Address already in useare real errors. On the page, a failed word check shows "Word check is not available right now; tones only."
Word check health
pct exec 309 -- curl -s http://127.0.0.1/api/health- Look for:
{"ok": true, "words": 149}.502means the service is down:pct exec 309 -- systemctl restart mandarin-speech.
Caddy inside CT 309
pct exec 309 -- journalctl -u caddy -n 50 --no-pager- Look for:
serving initial configurationis healthy.dial tcp 127.0.0.1:8000: connect: connection refusedmeans the word check is down.
Edge Caddy (CT 108)
pct exec 108 -- journalctl -u caddy -n 100 --no-pager- Look for:
certificate obtained successfullyfor mandarin.503mail.com.dial tcp ... :80: connect: no route to hostmeans CT 309 is stopped or its IP changed (DHCP). If the IP changed, fix thereverse_proxyline in/etc/caddy/Caddyfileinside CT 108, thenpct exec 108 -- systemctl reload caddy.
Recording problems
retry ... 403orFAILEDfrom edge-tts: Microsoft refused or rate-limited that request. Runbash install.sh renderagain later. It only records what's missing.- Harmless noise from Piper:
pthread_setaffinity_np failed ... Invalid argument(onnxruntime inside a container). It still works.
In the browser
- "The microphone is blocked": allow the microphone for mandarin.503mail.com in the browser's site settings.
- "I could not hear a voice": the recording was silent. Check that the right microphone is selected.
Quick health check
pct exec 309 -- systemctl is-active caddy mandarin-speechNotes and gotchas from the build
- Uptime Kuma: no monitor yet.
- A voice that sounds fine can still have flat tones. All three Piper voices sounded fine by ear, but measuring them showed almost no pitch movement on single syllables (their "mà" fell less than 1 step, and chaowen's "má" even went down). I checked the pitch tracker on a synthetic glide first, and it was accurate, so the flat tones really are in the voices. Measure before trusting a teacher voice.
- Piper huayan was dropped. It reads characters through eSpeak, which guessed wrong (哪儿 came out as "na three", and some tones were off). chaowen and xiao_ya take pinyin phonemes directly. render-course.py builds the phoneme IDs from the course pinyin itself, applying the tone changes 3+3 → 2+3 and 不 before tone 4 → bú.
- chaowen and xiao_ya need Piper's Chinese add-on. Without it they crash with
No module named 'unicode_rbnf', which shows up only aswave.Error: # channels not specified. Fix:pip install "piper-tts[zh]" unicode_rbnf. - Low male voices need a longer pitch window. With a 25 ms window, a voice around 90 Hz only fits about two cycles, and the tracker falls apart. tone.js uses 40 ms.
- The tone model: my first hand-written rules scored 56%. The current model was trained on the Xiaoxiao, Xiaoyi and Yunxi clips. Trained on two of the female voices and tested on the third, it scored 76 to 82%. For male voices it's about 50%, so lower voices get rougher results. To retrain, open check-tones.html, then in the browser console run
collect(['x','i','y']). WhenFDONEis true, runheldOut(['x','i','y'])to see the scores, thenJSON.stringify(train(FEAT.filter(f => f.F && f.F.curve.length >= 3)))and paste the result overWin tone.js. - Tones in full sentences are much harder (62% on the female teacher voice, 40% on the male). So conversation lines pass or fail on the words only, and tones show as a rough guide. The strict tone check is on the single-word cards and in the drill.
- Never run these UTF-8 files through Windows PowerShell 5
Get-Content/Set-Contentwithout-Encoding UTF8. It garbled every Chinese character and tone mark in index.html and app.js. Edit them with a real editor. - The word check grammar is the 149 course words, plus the characters of a conversation line when checking a line. Sound-alike characters (他/她, 好/号) are expected. That's why lines are compared by sound.