Setup for 2023-12-31~2024-04-16:
- proxy https://phabricator.wikimedia.org/robots.txt + custom text via #wrangler
Setup as of 2024-04-16: Load extension via P49
Possible reinforcement:
1. [Pray to Upstream to implement proper robots.txt management](https://we.phorge.it/T15670#16591)
2. Host robots.txt file elsewhere and proxy it via cloudflare worker
1 is probably the way to go but too lazy to file a task.