Dev/Command-line/Site meta

From XOWA: the free, open-source, offline wiki application

XOWA can download the metadata for the Wikimedia wikis


Wikimedia exposes an API for accessing the meta-data for a given wiki. For example, for English Wikipedia, the following will return most of the meta-data around the wiki installation.|namespaces|statistics|interwikimap|namespacealiases|specialpagealiases|libraries|extensions|skins|magicwords|functionhooks|showhooks|extensiontags|protocols|defaultoptions|languages

XOWA can call this API to download metadata for each wiki and save them in a database for data-processing. XOWA uses this info to resolve namespaces, but it will also incorporate other metadata from this API in future releases.


Assuming you are on a Windows system with XOWA installed at C:\xowa

  • Create a plain text-file called "C:\xowa\build_site_meta.gfs"
  • Save the following text to the file:
app.bldr.cmds {

  // NOTE: wiki doesn't matter; just use any wiki name that is on your system
  add('', 'util.site_meta') {

    // path of the database to generate; default is C:\xowa\bin\any\xowa\cfg\wiki\site_meta.sqlite3
    db_url = 'C:\xowa\site_meta__enwiki.sqlite3';

    // skip any wikis which have been downloaded after this time. default is now() - 1 day
    // the purpose of this argument is to avoid recalling the api if it's already been called recently.
    // for example, if the script runs for 800 wikis and fails for 3 wikis,
    // you can rerun the script again and it will only download the 3 failed ones; not all 800
    cutoff_time = '2015-07-01';

    // list of wikis to download; note that each wiki must be separated by a new-line. default is all wikis listed in [[Dashboard/Import/Online]]
    wikis  = 
  • Run the file with the following:
java -jar xowa_windows.jar --app_mode cmd --cmd_file C:\xowa\build_site_meta.gfs
  • Open C:\xowa\site_meta__enwiki.sqlite3 in a sqlite shell and run the following:
SELECT * FROM site_statistic;



Getting started