App/Category/Overview

XOWA currently has two category systems. This page will explain the rationale / details behind both.

An overview follows below.

For more information about V2 setup see App/Category/Building

For more information about V2 internals see App/Category/Internals

Version 1

Version 1 is a simplistic category system.

• It relies only on page content inside the xml file. It does not use any of the category*.sql dumps.

Note the following limitations:

• Does not work with large categories. It gets linearly worse with more members (do not use it to load a category with over 10,000 members)
• Does not support paging. If a category has 1,000 members, it will load title information on all 1,000 (instead of just the first 200)
• Does not use sortkey. For example, Jimmy Wales will alphabetize under J (for Jimmy Wales) instead of W (for Wales, Jimmy)
• Does not accurately reflect page membership in categories.

For example, most hidden categories are added to a template which is then included in a page.

Specifically, a page called "File:GNU.png" may belong to "All free media". However, the "File:GNU.png" page doesn't have the [[Category:All_free_media]] but instead embeds a template {{All_non_free_media}} which has the [[Category:All_free_media]]

Since a full parse (with templates) of the entire xml file would take many hours, this membership data is omitted.

V1 should be considered obsolete. No signficant changes will be made to it, as V2 is the official category system.

However, because V1 is faster to setup than V2, it still remains the default (with a strong recommendation to upgrade to V2 when time permits)

Version 2

Version 2 is an accurate category system.

• It uses the Wikimedia dump files: categorylinks.sql, page_props.sql

It addresses each of the limitations of version 1, including

• Works with large categories
• Supports paging
• Uses sortkey
• Accurately includes all members of a category

It has a few limitations:

• It requires additional dump files (as mentioned above).
• It takes longer to setup. A separate .sql file must be parsed. For English Wikipedia this process takes about another hour.
• It takes more disk space. The v2 system stores sortkeys individually per entry (just like Wikipedia). However this text data greatly increases the overall file size. English Wikipedia will have about 10.0 GB of extra data.

V2 is the official category system and should generate Category pages just like Wikipedia.