App/Import/Image databases/Pruning

XOWA's image databases will accumulate old images and take up unnecessary disk space. This page outlines the process to prune these old images

Background

XOWA releases monthly updates of English Wikipedia images. However, every month, English Wikipedia removes many old images. Over time, the removed images will cause XOWA's image databases to grow larger than necessary.

As of 2014-07-07, XOWA's image database for English Wikipedia was 92 GB. It should be 68 GB.

If you want to reduce your image database size for English Wikipedia, you can try either of the two options below.

Option 1: Download the updated set

The simplest approach is to download the latest set. Note that this will require downloading approximately 62 GB of compressed data. Users who have bandwidth issues (caps / low connectivity) may want to look at the next option.

Option 2: Prune the images manually

Note the following:

• This is an advanced step that requires command-line usage
• It will take anywhere between 2 and 10 hours depending on your machine's speed
• It works by runing about 200 .sql files on over 90 GB of databases
• It is the exact same process I used to generate the update set above.

The following steps are required:

• Download the Xowa_enwiki_2014-07-07_prune.7z file at https://archive.org/details/Xowa_enwiki_latest Note that the .7z file is 3 MB.
• Unzip them to your XOWA root directory. When you are done, you will have a file called /xowa/file/en.wikipedia.org/tmp_sql/en.wikipedia.org-000-atr.sql as well as many others
• Create a xowa_prune.gfs file in your xowa root with the following text. Note that the file must have UNIX style new lines (Windows users should use a text editor like Notepad++)

app.bldr.pause_at_end_('n');
app.scripts.run_file_by_type('xowa_cfg_app');
app.bldr.cmds {
  add     ('en.wikipedia.org' , 'file.diff_regy.exec');
}
app.bldr.run;

• Open up the command line and run the following (change xowa_windows.jar as per your Operating System):

java -Xmx256m -jar xowa_windows.jar --cmd_file xowa_prune.gfs --app_mode cmd

If you run into problems, feel free to contact me as per Help/Feedback