Commons:Datos estructurados/Etiquetado asistido por computadora

This page is presently inactive and retained primarily for historical interest. This feature is disabled and will be removed in the future.

El etiquetado asistido por computadora es una tecnología nueva. Algunas etiquetas sugeridas pueden ser equivocadas o inapropiadas. Esto es un comportamiento esperado.

La herramienta de etiquetado asistido por computadora es una característic desarollada por el equipo de Datos Estructurados en Commons para ayudar a los miembros de la comunidad a identificar y etiquetar declaraciones de representación para archivos de Commons. Hay decenas de millones de archivos cuidadosamente curados en Commons, pero la herramienta de datos estructurados es nueva. Con esta nueva característica, los archivos existentes pueden tener su contenido descrito de manera fácil, rápida y precisa. Para contribuir, los editores no necesitarán saber cómo funciona Wikidata ni hablar un idioma en particular. Esta nueva característica solicita a los usuarios sugerencias para etiquetas, utilizando un modelo de visión por computadora, para revisión humana. Las y los usuarios de Commons podrán visitar una página especial en Commons y ver las etiquetas de representaciones sugeridas, que se pueden seleccionar para confirmar o ignorar. Las etiquetas nunca se agregarán automáticamente sin participación humana.

Computer-aided tagging helps populate files with structured data, in turn these files can be found using general search terms in Special:MediaSearch in a manner that was previously not possible. This helps users easily find media that otherwise isn't easy to come across using the old search, which often relies on specific information in file descriptions or category placement to find files. If specific information is lacking, it can be hard-to-impossible to find a lot of media on Commons through the standard search. For example, Peter_iredale_sunset_edited1.jpg shows up in a search for "beach" using Special:MediaSearch thanks to the "beach" depicts statement added by computer-aided tagging; it does not show up at all in a search for "beach" using regular search.

El etiquetado asistido por computadora es una extensión de MediaWiki independiente y no es una parte nucelar de Commons en sí misma, y se vincula a Commons usando Special: SuggestedTags. En el back-end, la herramienta utilizará Google Cloud Vision para obtener sugerencias. Wikimedia ya usa el servicio Google Cloud Vision en el reconocimiento óptico de caracteres de Wikisource , y esto funcionará de manera similar. 'Esta herramienta está habilitada para usuarios registrados y confirmados automáticamente' . No está activada de manera predeterminada para ningún grupo de usuarios y no está disponible para usuarios nuevos y no registrados.

Updates on CAT/SuggestedTags usage, September 2020

To date (Updated on 14 February 2022):

1. 5,809 total users have made edits via the Computer-Aided Tagging tool

   • 962 of these users did so via mobile web

2. 341,957 total files have had edits made via Computer-Aided Tagging

   • 41,563 of those files have Computer-Aided Tagging edit on mobile web

3. 72% of files with CAT edits had those edits done by the same user who uploaded the file
4. Approximately 10,000 files edited by CAT so far were purely manual edits
5. We’re averaging about 20 new users a week currently

Charts for this data are updated every Monday on the CAT usage report analytics page

CAT specificity

We’re working on possible techniques for improving the tool’s ability to accurately identify specific elements of photos, but it’s important to keep in mind that the Google Vision algorithm already does fairly well in many topic spaces.

Upcoming tweaks to the queue for general images

Although most usage of the Computer Aided Tagging system comes from users editing their own uploads (72%), there is a separate queue for “popular” images. Based on recent feedback from the Commons community, we’re exploring ways to prioritize this queue differently. Particularly, we’re considering a system that would focus more on files that do not have curated categories yet.

Google Cloud Vision

Toda la información que pasa a través de Google Cloud Vision también será pública. Los volcados estarán disponibles de datos completamente anónimos que enumeran el archivo Commons, sus etiquetas sugeridas y qué etiquetas fueron aceptadas. Google Cloud Vision está completamente aislado de Wikimedia Commons, la función está separada de la experiencia central de Commons'.

Although there are open source computer vision platforms available to start from, any such package would require resources or specialized expertise to provide an industry-standard experience with computer vision that the Wikimedia Foundation is unable to itself provide at this time. The team recognizes that Google Cloud Vision is not open source software. There will not be any non-free or proprietary code written by the Foundation for this project; all contributions will remain open source.^{[clarification needed]} Google will not have access to any private, non-public, personal information, there will be no direct communication between users and Google's service.

Architecture and workflow

Design of information flow in computer-assisted image tagging. The "machine vision" provider on the far right requests and sends potential tags for images; there is no personal information exchanged and the provider is isolated from the rest of the system and Commons.

Registered, auto-confirmed users will be able to opt-in through their preferences or uploading files. After some time has passed, the user will be contacted through their notifications that their uploads are ready for tagging at Special:SuggestedTags. Users who have opted-in can visit Special:SuggestedTags at any time to view files ready for tag processing. Anonymous users, new users, and users who have not opted-in will not be able to access Special:SuggestedTags.

The concepts that are available for tagging are ones that translate from Google Knowledge Graph IDs to Wikidata IDs. At 2.1 million triplets, the list is too long to catalog here, but is available for download as freebase-wikidata mappings.

Development stage

All originally planned features for the tool are now deployed and available for use. The development team will continue with tweaks, and possible new features in the future.

• Usability study
• Tracking task for building the tool

Implementation and usage notes

• No personal information is sent to the computer vision platform provider. At launch, this new feature will only use the Google Cloud Vision system, which will be accessed via a middleware layer that hides all user data. Commons images are sent to Google servers from Wikimedia Foundation servers. There will be no direct communication between the user and external services. No personal information (IP, username, etc.) is sent to Google servers. The middleware that contacts Google servers is a Wikimedia project and is open source. No part of Google's service or code will be part of Wikimedia infrastructure.
• Suggestions from the computer vision will not be added to an image file’s structured data until a user has verified them: This service is provided as a means to augment human activity, not replace it. All suggestions from the computer vision service are stored in a separate, specialized database. Suggestions are not saved as structured data on the Commons file until a human user confirms them.
• Users can opt in to receive notifications alerting them that their recent uploads have suggested tags. In the last step of the UploadWizard upload process, users have an option to enable notifications that will inform them when recently uploaded files have passed the waiting period and have tags available for confirmation. This option can also be found in User Preferences under Notifications.
• User contributions that confirm suggested depicts tags are licensed as CC0. This data is equivalent to adding Wikidata to an image, and as such must be contributed under the same CC0 license that Wikidata uses. Clear license notices will inform users that all contributions made via the computer vision tool will be licensed under CC0.
• Analysis of images on Commons: The feature will analyze only images, and provide suggested “depicts” tags based on the content of those images.
• Certain types of images will be excluded: Some types of imagery on Commons are not well-suited for this type of system. Small images (less than 100px wide), artworks (identified via the Artwork template), book page scans, and other files will not be included.
• Newly uploaded files will be analyzed, but not during upload: Commons users continuously monitor new files for vandalism, copyright violations, and relevance to the project. Files that don’t meet the criteria are marked for deletion. The new computer vision feature will only analyze new files after a waiting period has passed, and will not analyze files marked for deletion.
• All tag confirmations show up as regular structured data edits with an edit summary tag that identifies their origin from the computer vision tool: This enables all the usual curation and moderation workflows so changes can be improved, edited, or reverted. It also helps us measure the revert rate and ensure that edits made using CAT are not more frequently reverted than the average edit.
• Problematic tags can be blocked from being suggested: There is a blocklist of tags that will not be suggested by the tagging tool. The official blocklist currently exists within the configuration file for Commons and cannot be edited directly by the community, but suggestions can be made on the blocklist talk page.

Userbox

You can use this userbox in your user page.

{{User Computer-aided tagging}} links talk view

This user uses Computer-aided tagging tool for tagging images.

This was a failed project

As early as 13 February 2020, experienced Commons users were complaining that the bulk of tags added using this tool were, as one put it, "way too vague, irrelevant or even detrimental". After numerous such complaints over the next several years, on 16 June 2023 the Sr. Director in the WMF Product department acknowledged that "We understand that the accuracy and utility of the tags generated by this tool have been called into question." After some study, on 14 September 2023 they announced, "we will be deactivating the tool on September 20, 2023, after completing the necessary code changes."