Commons:WMF support for Commons/GLAM Metrics Needs

Welcome to the “GLAM Metrics Needs” page! This page aims to create a space to collect, document, and discuss the requirements for reliable, maintained tools for reporting GLAM impact.

We(a small team of people at the Foundation who are responding to community needs for a revitalized Wikimedia Commons.) invite all members of the GLAM-Wiki community to share their needs and ideas in this space. We'd also love to understand how GLAM metrics are used through specific examples and use cases. Whether you are a Wikimedia contributor interested in reporting on the impact of GLAM partnerships or a GLAM institution that wants to measure the impact of its own activities, please join us to help shape this project. We want to ensure that the needs of all users are met and that we create a space that is both useful and inclusive.

We look forward to hearing your thoughts and ideas!

Point of information

This page was created in February 2023, likely inspired by discussion and meetings that took place that month started by the Wikimedians in Residence Exchange Network, and with the Wikimedia Foundation and Wikimedia Sweden/Content Partnerships Hub. For context, here are the links to relevant conversations and documents.

• WREN mailing list message - "The problems with Wikimedia metrics", 8 Feb 2023
• meta:Wikimedians in Residence Exchange Network/GLAM manifesto 2023

GLAM Wiki Dashboard

We (WMB) have dozens of GLAM-Wiki partnerships and have been sharing metrics with them and the community through our This Month in GLAM Brazil reports and most recently, also monthly, trying to use the GLAM Wiki Dashboard platform generated report. I want to share a list of quantitative metrics we share (and want to share) with our partners:

Metric	Available at/through
Total number of files	GLAMWikiDashboard
Number of views in the month	GLAMWikiDashboard
Number of views since the begining of the partnership (category creation)	GLAMWikiDashboard
Number of usages throughout the projects	GLAMWikiDashboard
Number of projects using media files from the partnership	GLAMWikiDashboard
Link to the Wikimedia Commons category of the GLAM partnership	GLAMWikiDashboard
Most viewed media files in the month and mean views per day	GLAMWikiDashboard
Link to the Wikipedia page of the GLAM partnership	Wikidata item
Number of items on Wikidata about the items of their collections	Wikidata Query Service
Timeline chart comparing total views and content upload	Wikimedia Commons Query Service?
Number of editors of items and media files	Wikimedia API
Bubble chart of topics depicted on the media files (SDC)	Wikimedia Commons Query Service?

Unfortunatelly, we don't have all our GLAM-Wiki partnerships on the GLAM Wiki Dashboard platform, so the metrics we share are limited by what we can get with other tools (mainly Glamorgan) with notes explaining each metric. EPorto (WMB) (talk) 17:26, 22 February 2023 (UTC)

Previous studies

2022 GLAM tool survey

There was a recent survey by WMF on tools that GLAMs use.Here's the survey results on meta. There was also a blog post summarizing the findings: "In the category of statistics tools, the popularity of tools varies. The very flexible Pageviews Analysis is by far the most popular; various more specific dashboard tools receive similar appreciation and seem at a similar level with each other in popularity. In the category of batch contribution and upload tools, two tools focused on structured data and Wikidata contribution rank most highly: QuickStatements and OpenRefine. Pattypan follows (with a small distance) as the third preferred tool, receiving relatively more appreciation from Wikimedians (and a bit less from cultural institution workers)."

2013 report which is still relevant

Report on requirements for usage and reuse statistics for GLAM content (2013)

I'm adding a few headers with distinct (but interrelated) remarks and suggestions so that they are easier to have individual conversations about (if relevant). I'm contributing these with my volunteer hat on!

In 2013, as part of the development process for the GLAMwiki Toolset, a report of GLAM metrics needs was already written, and it may still contain very relevant data/input. Let's not ignore this, and potentially revisit: Report on requirements for usage and reuse statistics for GLAM content.pdf. Spinster (talk) 08:32, 24 February 2023 (UTC)

Anecdata

As the Wikipedian-in-Residence at the Brigham Young University Library, metrics on image views for images I've uploaded are and important way that I communicate the impact of my work to library administration and my colleagues. Of course, it's not the only way, but I would like to know what the tools I use are actually measuring and be able to reliably access data from them. GLAM metrics are also important for institutions who may not have a dedicated part-time (or full-time) position. The metrics can help them to decide to continue to fund a position, or to help them apply for grants to fund those positions. Personally, I was able to apply for an internal grant for a student copyediting position based on the pageviews of Wikipedia pages we've edited. Accurate metrics help to build trust with stakeholders who control that kind of money. Rachel Helps (BYU) (talk) 19:49, 24 February 2023 (UTC)

Proposal: conduct a new/updated user research project

This page is created to collect needs, but may only result in anecdotal data points. In my opinion, it would be incredibly helpful if, today, a new similar project of end user research (including an international, multilingual questionnaire, and user interviews with international GLAM staff) would be held, to update the above insights fromt the 2013 research and provide a reliable and comprehensive international overview of needs. I can imagine WMF would be in a good place to take the lead on this / do this research. Spinster (talk) 08:32, 24 February 2023 (UTC)

New needs arising from long-term engagement with Wikimedia projects, and from structured data (Wikidata and SDC)

In conversations with GLAMs, I am very often hearing needs around metrics that arise from the institution's longer-term engagement with Wikimedia projects (volunteers have worked on the GLAM's Wikimedia assets/information for a long time; what has happened there?), and related to the new developments of Wikidata and Structured Data on Wikimedia Commons. These are new developments compared to the 2013 report I linked above, and I think deserve some special attention. I will randomly list some questions that I hear GLAMs ask a lot:

• How many files on Wikimedia Commons / items on Wikidata / Wikipedia articles around my collections have been updated by volunteers, which ones, and in which ways? 'Updating' can mean many things: (for Commons) digital restoration, categorization, addition of structured data, correction of metadata, overwriting files... In most cases, the GLAMs are not only interested in the mere numbers, but also in getting back the exact files/data and changes so that some type of feedback/roundtripping can happen. The longer GLAM partnerships have taken place, the more relevant this becomes.
• GLAMs are interested to know about usage of 'their' collections data that was added to Wikidata. How often are their Wikidata items displayed in some way in Wikipedia articles, for instance? I have understood that this is already not trivial to measure for re-use inside Wikimedia wikis; ideally, GLAMs would be very motivated to also see metrics about re-use of Wikidata items outside of Wikimedia projects.
• GLAMs are generally interested to learn if adding structured data to their files on Commons does result in better re-use and discoverability of the files. How does the addition of SDC help with this? This is a complex question which also depends on external conditions, e.g. how well Wikimedia Commons search leverages SDC, how much of the (multilingual) SDC is exposed to external search engines (Google and other generic image search engines)...

All of the above examples are not only relevant for metrics in the pure sense, but also touch upon the hot topic of data roundtripping which may become more important for Wikimedia content partners in the future: giving partners easy-to-use tools to feed back Wikimedia data to their own databases. Spinster (talk) 08:47, 24 February 2023 (UTC)

Just for GLAMs? As an individual Wikimedia editor, I would love to see these stats about my individual contributions too!

GLAM (or more broadly: Wikimedia content partnerships) statistics haven't gotten enough product attention and sustained maintenance, possibly also because "GLAM partnerships" are seen as an edge case or a specialist area of work in the Wikimedia movement (with individual contributors and readers as the main demographics deserving the majority of product development investment). I want to counteract this a bit. As a Wikimedia volunteer contributor, it would be extremely empowering for me to also have access to a dashboard-ish thing with all of the above metrics. Hell, I would very much want to know how often my own personal Commons uploads are viewed, who has enriched them, what they have been enriched with, where my Wikidata items are being used, and how often! IMO such stats can be a huge motivator and help with editor retention. Just putting this here, if it can help with prioritizing this area of work at all :-)) Spinster (talk) 09:04, 24 February 2023 (UTC)

Updates

Hello @Udehb-WMF, I hope this message finds you well! I was revisiting this page, and was wondering if we could have an update about all the metrics needs pointed in here by @Rachel Helps (BYU), @Spinster and myself (pinging @Fuzheado here, as he also interacted with the page). Cheers, EPorto (WMB) (talk) 17:25, 4 August 2023 (UTC)

I'm on the DPLA Wikimedia working group and image metrics are really important for their work too (both for convincing hubs to donate images to Wikimedia Commons and for showing them afterwards that it's worthwhile). Rachel Helps (BYU) (talk) 17:28, 4 August 2023 (UTC)

The Update

Hello User:EPorto (WMB), User:Rachel Helps (BYU), User:Spinster, User:Fuzheado and all,

Thank you for your patience and for reaching out for an update regarding the metrics needs shared on this page. I'm pleased to inform you that we have taken note of the invaluable feedback and requirements shared by the GLAM community here. We are actively working towards addressing these needs through the Commons Impact Metrics initiative.

For detailed information about our ongoing efforts and the objectives of this project, please visit the Commons Impact Metrics project page.

Your insights and contributions have been instrumental in shaping this initiative, and we look forward to continued collaboration.

Once again, thank you Udehb-WMF