Reading and writing MediaWiki pages from a script: action=raw and the two-token login
Reading: skip the HTML, ask for raw wikitext
Fetching an article URL with a normal client and parsing the HTML is where most scripts start, and it goes
wrong in a way that is easy to miss: a plain GET of the page URL can land you on Recent Changes
instead of the article — a redirect that a browser resolves invisibly but that leaves your parser looking
at completely different content and “working”.
Ask for the source directly:
https://wiki.example.com/index.php?title=Release:MyProject&action=raw
That returns clean wikitext, no skin, no navigation, no redirect games. For anything programmatic — diffing a deployed-versions table, checking whether a section exists — this is the endpoint you want.
Writing: login token, then CSRF token, then edit
Editing needs two tokens, fetched in this order, on the same session (cookies matter):
import requests
API = "https://wiki.example.com/api.php"
s = requests.Session()
# 1. login token
lt = s.get(API, params={"action": "query", "meta": "tokens",
"type": "login", "format": "json"}).json()
login_token = lt["query"]["tokens"]["logintoken"]
# 2. log in with a BOT password (Special:BotPasswords), not your account password
r = s.post(API, data={"action": "login", "lgname": USER, "lgpassword": PASSWORD,
"lgtoken": login_token, "format": "json"}).json()
assert r["login"]["result"] == "Success", r
# 3. CSRF token — only valid after a successful login
ct = s.get(API, params={"action": "query", "meta": "tokens", "format": "json"}).json()
csrf = ct["query"]["tokens"]["csrftoken"]
# 4. edit
e = s.post(API, data={"action": "edit", "title": PAGE, "text": new_wikitext,
"summary": "automated update", "bot": 1,
"token": csrf, "format": "json"}).json()
assert "edit" in e and e["edit"]["result"] == "Success", e
Things that bite here:
- Fetch the CSRF token after logging in. A token fetched before login belongs to the anonymous session
and the edit fails with
badtoken— a confusing error, because the token looks fine. - Use a bot password (
Special:BotPasswords), which gives you auser@botnamelogin and a separate 32-character secret with a restricted grant set. Your interactive account password may be blocked by the wiki configuration and will not survive a 2FA rollout. - Assert on the response body, not the status code. MediaWiki cheerfully returns
200 OKwith{"error": {...}}. A script that only checksr.okreports success while nothing was written.
The trap that actually cost me the time: the credential file
The credentials lived in a per-service file in my credential store, and I wrote the obvious loader: first line is the user, second the password. It failed with a login error, and I spent the next while debugging the token dance — which was correct all along.
The file was a free-form note, not user:pass. It contained a sentence, the user@botname login and
the 32-character secret embedded in prose. So parse by shape, not by position:
import re
text = open(cred_path).read()
login = re.search(r'\b[\w.-]+@[\w.-]+\b', text).group(0) # user@botname
secret = re.search(r'\b[A-Za-z0-9]{24,}\b', text).group(0) # long alnum token
Two lessons worth generalising:
- Verify the credential you loaded before blaming the protocol. One
print(repr(login))would have ended it immediately. When auth fails, dump what you actually sent (masked) before touching the flow. - If you keep notes in credential files, keep the machine-readable part in a fixed
key: valueform and the prose below it. Free-form is fine for humans and a landmine for the next script.