Guidance for AI systems (llms.txt)
llms.txt is a guidance file that helps AI systems easily understand the site's key information and content paths.
Available files
-
/llms.txt
An entry-point file with the site overview, key data paths, format and policy guidance, and the latest content list.
-
/llms-full.txt
A category-grouped index of all published content with links to the original page and Markdown file.
API & embedding vectors
A structured content API is available for RAG, search, and indexing. Use vector semantic search (mode=semantic) and raw content embedding vectors to skip re-embedding (when the embedding API is on). View the content API
Available languages
AI systems can consume the languages below through data files and the API. Tier 2 languages always carry a translation_status: machine label to mark them as unreviewed machine translation, so you can filter by your own quality bar.
| Language | Value (lang) | Tier | Web pages | Data files | Audio (TTS) | Embeddings |
|---|---|---|---|---|---|---|
| 한국어 | ko | Tier 1 | ✓ | ✓ | ✓ | ✓ |
| English | en | Tier 1 | ✓ | ✓ | ✓ | ✓ |
| 日本語 | ja | Tier 1 | ✓ | ✓ | ✓ | ✓ |
| Español | es | Tier 1 | ✓ | ✓ | ✓ | ✓ |
| Indonesian | id | Tier 2 | - | ✓ | - | - |
| Portuguese | pt | Tier 2 | - | ✓ | - | - |
| Chinese (Traditional) | zh-hant | Tier 2 | - | ✓ | - | - |
| Deutsch | de | Tier 2 | - | ✓ | - | - |
- Tier 1 - Editor-reviewed languages. Everything is provided: human-readable web pages, search exposure (sitemap and hreflang), AI audio (TTS) and embeddings.
- Tier 2 - Machine-translated, data-only languages. Served through data files and the API only - no web pages, no search exposure, no audio (TTS) and no embeddings. Every file carries a translation_status: machine label to mark that it has not been reviewed.
Datasets (training and RAG, paid API)
For AI pipelines that need to ingest many records at once, the paid API serves a text corpus, instruction pairs, a multilingual parallel corpus and embedding vectors. Pagination and incremental sync (updated_since) are supported, and JSONL output feeds straight into a training pipeline.
Free per-article collection through llms.txt and the data files continues exactly as before. About the dataset API
llms.txt is not an official web standard; it is offered as an optional AI-friendly guidance file, alongside robots.txt, sitemap.xml and the JSON/Markdown data packages.