Commons:Montage

Montage is a photo evaluation tool for and by Wiki Loves competitions. It offers a flexible round-based workflow that is configurable to contests of all sizes, and is suited for evaluation of 10-1000s images. You can find more information about the tool below.

Request access

Description of the tool

Montage helps with photo evaluation, by pulling the images on Commons into the tool, based on lists of files, or one or more Commons categories. Montage will create tasks in proportion to the quorum and the number of images. The total number of tasks is then distributed among the number of judges of the round after activation of the round. The tool provides two main views for coordinators (creating/editing/closing jury rounds) and jurors (participating in rounds).

Coordinator view

Coordinators can see a list of the campaigns they are associated with. Each campaign links to a page which lists all the jury rounds designed for that campaign. Coordinators can see a list of the rounds associated with a campaign they are associated with, and rounds are divided among active and inactive. Each round shows the type of voting, the number of photos, and the number of jury members. Active rounds show the percentage complete. Inactive rounds show the total number of selected photos.

Coordinators can see the details for a selected round when creating or editing. The list includes:

• Name of the round
• Status (active, inactive, or closed)
• Type (Yes/no votes, rating, ranking)
• The quorum for each image, for up/down and rating votes
• Images (imported from a Commons category, a previous round, or a CSV of filenames)
• Jury members (the list of Wikimedia usernames)

Note: Due to the way votes are assigned, the jury membership is only editable when a round is set as inactive.

Help, my jury member is not available anymore!

When you have already activated your round and one (or more) of your jury members becomes unavailable do not simply delete the individual jury member. As stated above, Montage will create tasks in proportion to the quorum and the number of images. If your take out a jury member and activate the round again, the remaining tasks will be redistributed among the remaining jury members. The best option here, therefore, is to replace the jury member by pausing the campaign, deleting the old one and adding a new member, and then re-activating the campaign. All open tasks will then be assigned to this new account. You are able to assign different jurors per round, so the next round you can start fresh.

As long as not all tasks within the round are completed, it is not possible to finalize and move to the next round.

Round closing

Coordinators can close out a round to select images. The round closing interface allows the coordinators to choose how the round will close, by either specifying the number of images to select or the cutoff for selection. The page will show the number of images and number of authors/photographers who will go to the next round. Once a round is marked as complete, there will be an option to download a list, create a follow up round, or see more details on the result.

Round vote details

Coordinators can see a list of all of each vote in a round in a campaign they are associated with. The votes are also downloadable as a CSV.

Import dialog

When creating a round, coordinators can choose how to import files. They can provide a list of commons categories, including an optional filter based on the resolution of an image. Before finalizing the import, it will show the number of images that will be imported.

Juror view

Campaign overview

Campaign overview

Jurors can see a list of the campaigns with rounds they are associated with. Each campaign links to the round overview page.

Round overview

Jurors can see a list of rounds they are associated with. Each active round links to the voting dashboard for that round. The round displays their progress, and the round's due date.

Voting

There are three different voting possibilities: this short video explains them for the jurors.

Jurors can see the next task in a round they are associated with. For yes/no and rating type rounds, the interface includes a high-resolution version of the image, along with limited metadata (the image's resolution), and the juror can select yes/no or a star rating. The juror also has the option of skipping a task, and getting another task from their queue.

For ranking type rounds, the interface shows a rank-able list of images with limited metadata (the image's resolution). The juror can arrange the photos in an order and then submit the round. The ranking round has a 50 image limit.

• 
  Rating view
• 
  Ranking view

General progress

The juror can also see their overall progress and the due date.

More roles

Montage has a simple permission scheme tailored to the tasks of contest organization and judging.

• Maintainers - Creators/operators/maintainers of Montage

• Add Organizers. (Note: Maintainers can technically do anything, as they have database access to the system, however they are intended to only add organizers.)

• Organizers

• Create campaigns
• Add coordinators to campaigns they created
• All actions available to coordinators.

• Coordinators

Each participating country/team will choose one coordinator who will have the following privileges in the tool:

• Create/cancel/close a round
• Add/remove jurors
• Mark jurors active/inactive
• Initiate a task reassignment
• Download results and audit logs

• Jurors

• Rate and rank photos to which they are assigned
• See their own progress

Manual for coordinators

You should take the following steps to set up your campaign for your jury to start the voting process:

• Sign your country up for using Montage in the talk page. Let us know what the user ID of your country's coordinator is.
• The developer team will create a campaign for your country and will contact you on the talk page when your campaign is ready. Note that the majority of the countries will have only one campaign for the contest.

Note: If you want us to create any other campaign for your country (for example, if you have regional contests with different set of jury members and jury processes), please let us know about this when you sign up to use Montage.

How to check if your campaign is ready?

Commons:Montage campaign ready to start

• You see a done-mark in the table in the talk-page and you know your campaign is read.
• Go to the coordinator panel in Montage (please note the change in hyperlink from montage-beta to montage) and find your campaign.

Step 1. Check your campaign settings

Make sure the campaign settings dates for your campaign are correct. This is a very important step.

Click Edit Campaign and make sure:

• Start time and date of your campaign are set correctly (to the minute).
• Close Time of your campaign is the date and time you want to import your photos in round 1. Make sure these values are not the actual end time of your campaign. If you go with the actual end time, any photo that has been edited (even by users other than the initial uploader) after the end time of your campaign will be excluded from the jury process and this is not what you want. We recommend using the time that you want to create your round 1 as the end time of your campaign. If in doubt, please ask us.

Step 2. Create round 1

• Create the first round of your jury process in this campaign. Note that more rounds can be added only after the previous round is closed.

Round Settings

Round File Settings in Edit Round mode

These are some ways you can have control over what photos get imported in your round 1 and some of the features your jury will see or not. Choose each box carefully, do not go with the option of all boxes checked unless you mean it. :)

"Disqualify by resolutions" allows you to exclude photos under a certain resolution value from entering round 1.

"Disqualify by upload date" disqualifies photos that are uploaded or updated outside of your campaign start and end time.

"Disqualify jurors" disqualifies photos uploaded by your jurors.

"Disqualify coordinators" disqualifies photos uploaded by your coordinators. This features is not currently working.

"Disqualify organizers" disqualifies photos uploaded by your organizers. This features is not currently working.

"Show filename" makes the filename of the file uploaded to Commons visible to your jury.

"Show link" does not do anything at the moment.

"Show resolution" makes the resolution of the photo visible to the juror at the time of voting. (The juror can always see this information by going to Commons but this option makes this information more easily accessible.)

Importing photos

Screenshot of the activate button in Montage

You can either use a Category, an URL to a physical file, or copy/paste, to import photos. To do this, you should take the following steps and make sure your file follows a specific format.

1. Select Category on Wikimedia Commons -- For the majority of campaigns, the photos for the first round will need to be imported from Category:Images from Wiki Loves Monuments 2017 in XX where XX is the name of the country behind your campaign. If you are using a different category, please make sure you know what you are doing and why you are doing so or you may exclude photos of participants by mistake. We don't check what category you're using and it's your responsibility to make sure you are using the correct one. All images for the campaign need to be in this single category to be correctly imported. Make sure to use the internal name on Commons, not an external link from your web browser, and click on the desired category from the drop down that will be shown to you.
2. Select File List URL -- You can import a CSV file containing a list of image files. The file should begin with a header line "img_name", and then list one file per line. Files should not have the "File:" prefix. Lines starting with "#" are ignored. You can create the list of files as a gist, and then paste a link to the raw file in Montage.
   • See this demo file.
   • Montage can also accept these columns in a "full" CSV: ['img_name', 'img_major_mime', 'img_minor_mime', 'img_width', 'img_height', 'img_user', 'img_user_text', 'img_timestamp']. If any of these are missing, it will just look for an "img_name" column and look up the rest of the data.
3. Select File List -- Paste a list of UTF-8 encoded image file names. Omit "File:" and replace spaces by underscores. You can build the list with the AutoWikiBrowser (AWB).

The import is finalized once the coordinator clicks the 'activate' button at the bottom of the campaign, and the tasks for the jurors will be created.

If your contest has many photos, be patient. :) Depending on the size of the contest you may have to wait between a few seconds to a few minutes before your round 1 is created/populated.

Let your jury know that they can start the voting process.

If you run into any issue in the above steps, please ping us in Montage's talk-page. We are in UTC+2 and UTC-7 and we will try to get back to you as soon as possible.

File format tips

When using the file interface:

• Generate an UTF-8 Linux style flat TXT file (no DOS CR/LF, so use an FTP program like FileZilla to convert when uploading)
• Put it on a plain FTP/HTTP server (do not use Google Drive, nor OneDrive, since they generate structured content while downloading that is not correctly handled by Montage)
• The file should be public readable (Montage is not using credentials to access it; execute access on the directory is encouraged for security reasons)
• When in doubt you could use wget on Linux to verify the format of the download
• Only have a single column containing the JPG file names
• Replace the spaces in the file names by _ (underscore)
• Do not use double quotes
• When using the above format commas in file names are accepted

Downloading the result

The best way to see your final result after downloading is viewing them in either Libre Office (Libre Calc) or Google Doc (Google Spreadsheets). Excel 2019 does not function optimally here, but a work around is to open the results in Google Doc, and downloading them again in a xlxs format which can be opened in Excel. If this is not an option for you, please put up a request for your results on the talk page.

You can view the jury comments (if any) through the 'see jury comments' button. It may require some smart copypasting to use it effectively.

Status

Montage is under active maintenance by a team of volunteers, with the latest release being an upgrade from Python 2 to 3 in 2024. There will be bugs, but we are dedicated to enabling a better judging process, and protecting the integrity of contest data.

• Please see our Montage Status Page for latest release and issue resolution information.
• Issues can be reported on GitHub

We ask that Montage users add this page to their watchlist. Montage is hosted on Toolforge, which provides the servers and databases, as well as database backups. Keep an eye on the Toolforge status page as well.

To maximize stability, we maintain a separate development instance for our own usage, as well as coordinator and organizer experiments. If you would like access, simply ask on the Talk page.

Development

Montage is developed by MahmoudHashemi, Slaporte, and Yarl, and with support from YuviPanda. The development is focused on creating a tool based on WLM jury tool requirements. The source code for the tool is available.