Commons:Village pump/Technical

From Wikimedia Commons, the free media repository
Jump to navigation Jump to search

Shortcuts: COM:VP/T • COM:VPT

Welcome to the Village pump technical section
Technical discussion
Village pump/Technical
 Bug reports
 Code review
Tools
 Tools/Directory
 Idea Lab



This page is used for technical questions relating to the tools, gadgets, or other technical issues about Commons; it is distinguished from the main Village pump, which handles community-wide discussion of all kinds. The page may also be used to advertise significant discussions taking place elsewhere, such as on the talk page of a Commons policy. Recent sections with no replies for 30 days and sections tagged with {{Section resolved|1=--~~~~}} may be archived; for old discussions, see the archives; recent archives: /Archive/2024/10 /Archive/2024/11.

Please note
 
SpBot archives all sections tagged with {{Section resolved|1=~~~~}} after 1 day and sections whose most recent comment is older than 30 days.

How to search fields of files' Information template?

[edit]

For example, how could one search file description for a term like "Kathmandu" (as asked about by another user here) like on can search with intitle. description:"Kathmandu" does show some results but I don't know what it does and the results don't have that word in their description. I could not find info on this at mw:Help:CirrusSearch either. Info how to search specified fields of {{Information}} should be added here.

One could also use this to infer categories (such as by reading the date field and then adding it to a category by date like "Videos of {year}") as proposed here. This may also be needed for a date range filter, see phab:T329961. I'd like to search the date field but there is no information on how to do that at Help:Searching but I think it's already possible if I remember correctly. For example, I found that many files in deepcategory:"NASA videos from unidentified year" deepcategory:"Videos of 2020‎" have been miscategorized into Videos of 2020 (and thus should not be copied into "NASA videos in 2020" from there) where they have the correct date in the date field which is why I'd like to use that to correct that as well as copy them to their year category in Category:Videos from NASA by year. Prototyperspective (talk) 19:03, 6 October 2024 (UTC)[reply]

Prototyperspective (talk) 12:16, 10 October 2024 (UTC)[reply]
Here EBernhardson (WMF) said Unfortunately, the image description is simply an argument to a template. CirrusSearch doesn't do anything at that level and can't be that specific..
I think the best workaround currently would be to use the insource search operator with the field name first so for example I searched for insource:"|source=[https://soundcloud.com to identify files for Category:Audio files from Soundcloud.com. I think easily searching fields of the File pages' Information template could be enabled by
  1. Developing some regex that searches for any content after e.g. |source=
  2. Creating some alias for it so instead of writing some complex regex query every time one can simply enter e.g. info-source:"soundcloud.com"
Please comment what you think about this proposed way to make this possible and if you have any info on what would be needed for that. Prototyperspective (talk) 13:06, 23 October 2024 (UTC)[reply]
Would be great if somebody could develop such (a) regex(es) if there is no better way to search specific fields of the Information template. It's great that files have that structured metadata but it could be much more useful if it was searchable. Prototyperspective (talk) 13:17, 17 November 2024 (UTC)[reply]
[edit]

Example: An edit to a file changed its [en] caption to "28th September 2024 Sunday", but the preview in its diff and permalink displays not that but another text: "An introduction to persistent identifiers as part of a FAIR data landscape", which is the caption from the latest revision (made in diff). Previews in diffs and permalinks should display their specified revision, not the latest. -- Wotheina (talk) 05:05, 7 October 2024 (UTC)[reply]

My guess is that it shows always the latest caption? Another note, the caption box is not visible with OS X and Safari browser when I am looking the old revisions of the file pages. --Zache (talk) 07:07, 7 October 2024 (UTC)[reply]
I think Wotheina's bug has confused me before too. Is there a Phabricator: task for it? I did a quick search but I am not the best at finding bugs there. Also the Safari bug should be filed too. Commander Keane (talk) 21:45, 23 October 2024 (UTC)[reply]

W3C validator has been basically down for months

[edit]

Where I and these folks are from, we cannot use the link provided by {{Valid SVG}}. The validator always says "429: Too many requests". The page loads, but the "external validation service" doesn't. Aaron Liu (talk) 20:47, 19 October 2024 (UTC)[reply]

The link may fail for many reasons. I'm often run into the problem of unable to validate because the DOCTYPE is not known; I just select SVG 1.1 and ignore the minor errors.
Chealer reports failure for File:Netscape_icon.svg. That file did not have {{Valid SVG}} or {{Igen}}, so I added them and {{Invalid SVG}}.
Igen created a working URL that said the file was valid
{{Valid SVG}} gave the message that validity was not checked. That suggests a broken template.
{{Invalid SVG}} said the file was invalid (that's its job), but the URL said it was valid.
That link is the same as above. However, a second click complained "Sorry! This document cannot be checked." The problem was the external checker was not available.
Note: I can do a control-Reload and sometimes get a good result.
The previous succeeding checks used SVG 1.1 + xHTML + MathML, but the failing clicks did not recognize a Document type. However, I could select "SVG 1.1" and revalidate resulting in the URL
The new URL reports 46 errors and 1 warning. The inkscape and rdf namespaces are not treated as warnings.
Somebody may be limiting the requests or there may be a race condition that the Nu validator does not respond quickly enough.
First, I think it is poor practice for the templates to use uri=https%3A%2F%2Fcommons.wikimedia.org%2Fwiki%2FSpecial%3AFilepath%2FNetscape_icon.svg.
The more direct method would use
But that validation fails for files without a DOCTYPE. Supply an explicit SVG 1.1:
Which, of course, does not recognize inkscape or rdf namespaces because they are not in the SVG 1.1 DTD and reports dozens of errors.
Perhaps the doctype parameter was used to encourage switching to the Nu validator. Here's the URL without the parameter.
That URL fails with the external checker not being available.
Second, the problem seems to be the validator wants to hand off the request to the Nu validator. It may do this when the SVG file does not have a DOCTYPE.
The GitHub issue used the W3C Nu validator, https://validator.w3.org/nu/ , directly. The Special URL was not an issue.
Instead of using that, go directly to Nu. Chealer offers
That old version of the file has a DOCTYPE, so the old W3C validator does a reasonable validation:
The current version of File:Flag of Cuba.svg does not have an XML processing instruction or a DOCTYPE. Force SVG 1.1
Conceivably, SVG 1.1 could be forced all the time, but that raises numerous inkscape and rdf errors.
Conclusion. I would use the Nu validator without the schema parameter.
Glrx (talk) 17:36, 22 October 2024 (UTC)[reply]

This just materialized on a file I was editing, so I quickly turned the red link blue and found that it's popped up on 800+ files. What is this? Why is it here? —Justin (koavf)TCM 16:05, 24 October 2024 (UTC)[reply]

I noticed this too. I got as far as figuring out that {{#invoke:ISOdate|ISOdate|1=2006-07-24}} adds the category. I'm not too versed in Lua modules, so can't really continue the hunt. Cryptic-waveform (talk) 19:02, 24 October 2024 (UTC)[reply]
And so does {{#invoke:DateI18n|Date|year=2000|month=1|day=1}}. Cryptic-waveform (talk) 19:06, 24 October 2024 (UTC)[reply]
May be related to https://github.com/wikimedia/mediawiki-extensions-JsonConfig/commit/db05e3251393d734605d20c98a63dbd7f93d9426. Cryptic-waveform (talk) 19:13, 24 October 2024 (UTC)[reply]
@Cscott: Is this related to your change? And is it a desired outcome? Cryptic-waveform (talk) 19:15, 24 October 2024 (UTC)[reply]
@Koavf, @Cryptic-waveform & @Cscott - FYI, I used the CropTool today and the extracted photo file has the this same hidden category as the the original photo file here: File:Zheng Jianbang in 2023 (cropped).jpg.
  • 1. Does this category still apply to cropped images extracted from files within this new category?
  • 2. Was a there any "notice" or other advance communication about this small, but highly noticable "hidden" category? -- Ooligan (talk) 22:11, 24 October 2024 (UTC)[reply]
  1. I guess? ¯\_(ツ)_/¯
  2. No.
Justin (koavf)TCM 22:34, 24 October 2024 (UTC)[reply]

Related: https://en.wikipedia.org/wiki/Wikipedia:Categories_for_discussion/Log/2024_October_24#Category:Pages_using_the_JsonConfig_extension. Cryptic-waveform (talk) 19:21, 24 October 2024 (UTC)[reply]

See also mw:Extension:JsonConfig. No idea why this is something that needs to be tracked. —Justin (koavf)TCM 19:43, 24 October 2024 (UTC)[reply]

Requested fix

[edit]

Commons:Administrators'_noticeboard#Interface_admin_request. —Justin (koavf)TCM 22:38, 24 October 2024 (UTC)[reply]

The category is empty now. Good to delete? Regards, Aafi (talk) 13:46, 5 November 2024 (UTC)[reply]
Perhaps also good to keep a look at phab:T378352. Regards, Aafi (talk) 13:47, 5 November 2024 (UTC)[reply]
Emptying was predicted to complete at the end of december. File File:Chrysalis5504.jpg had Category:Pages using the JsonConfig extension and it was red, whereas the cat itself was empty. No change after "null edit". Fixed after real edit. Now nazi file File:Deutsches_Reichsgesetzblatt_33T1_126_0846.jpg has that cat (see above). Strange. Taylor 49 (talk) 19:29, 6 November 2024 (UTC)[reply]
File:Deutsches_Reichsgesetzblatt_33T1_126_0908.jpg pretends to be in "Category:Pages using the JsonConfig extension" (red and empty, should NOT be there) and in "Category:Nazi symbols" (OK), OTOH it does NOT show "Category:Nazi symbols status", but when looking into that cat, it is listed there. Commons is severely broken today. Taylor 49 (talk) 19:34, 6 November 2024 (UTC)[reply]
Taylor 49, last time I checked Nazi symbols status had approximately 27,000 files and presumably all of them had Category:Pages using the JsonConfig extension. Currently, there are 14,829 files and it looks like all of them have Category:Pages using the JsonConfig extension. I don't know exactly what happened to the rest of the files.
Deutsches Reichsgesetzblatt 33T1 126 0846.jpg and Deutsches Reichsgesetzblatt 33T1 126 0908.jpg are not longer in Category:Pages using the JsonConfig extension and are normally categorised. Is it safe to assume for the rest of the files? Also Category:Nazi symbols and Category:Nazi symbols status are also correctly behaving now. Ratekreel (talk) 04:17, 7 November 2024 (UTC)[reply]
A few files have "fixed themselves" since yesterday, but there is still the discrepancy in both directions:
It's ultimately a BUG, and a new one. The legacy flaw of all WMF wikis is that sometimes a file claims to be in an existing cat, but when looking on that cat the file is not there. What we can observe here is new and worse than that. Taylor 49 (talk) 14:40, 7 November 2024 (UTC)[reply]

Should Template:Compressed version be used with Lossy WebP?

[edit]

It's unclear to me what the role of Template:Archival version is now in 2024. Although it makes a lot of sense in a world where GIF, PNG, and JPEG were the "Big Three" commonly supported image formats on the web (ignoring BMP, which I think did enjoy an era of support most associated with the early web), it seems a bit technical whether or not the Template:Archival version & Template:Compressed version templates should still be used on new uploads today with the advent of WebP.

I'm not an expert, but I think modernly, Mediawiki is happy to take PNG and shunt it to WebP automatically, as a display format that saves bandwidth without conscious change of workflow on part of the user. The transparent nature of these bandwidth savings is less applicable to JPEG (which can also be automatically converted to WebP for bandwidth savings), but when that's done, the converted WebP still must deal with the limitations of the legacy JPEG format. Namely: lack of support for alpha transparency, and converting lossy-to-lossy (necessarily degrading the image, versus if the user had just embraced lossy WebP to begin with).

I'm new to Wikimedia, and trying to determine the best way to upload my images so that:

  1. Pages that include my images continue to load fast
  2. My images are stored at the highest resolution possible and in the best quality possible

To satisfy both of these goals, I thought I should hijack Template:Archival version and Template:Compressed version as an "existing structure" and seemingly "best practice" put into place during an era when it was expected that the Template:Compressed version would be a JPEG. Lending legitimacy to this, the Template has a (deprecated) field named "Filetype", implying that at least in the past, it was appropriate to use the template with more than just JPEG images.

So I went ahead and tried it:

However, there are currently a few problems with this.

  1. Although Template:Compressed version claims filetype can be omitted and then defaults to an "intelligent guess", it seems that in reality, it has not been able to detect usage of the template with WebP, and instead defaults to the text to "This image is a JPEG version [...]" when used with WebP. If this is a valid usecase of the template, the text can be changed to support detection of WebP easily enough.
  2. Although I hoped that by using WebP for the compressed thumbnail version instead of JPEG I would gain access to alpha transparency (improving dark mode support on Wikipedia), it seems that the thumbnail generated at w:Connection Machine filled the transparency with the colour white, which is especially unfortunate for this primarily black object where it is hard to discern detail against a white background. Although in this specific case, I may want to consider a permanently dark background to set this object against instead of transparency, it can be imagined that there are cases where it would be nice for thumbnails to adapt to Light Mode and Dark Mode versions of Wikipedia.
    I am told that the lack of support for transparent thumbnails of WebP images is caused by "Phab:T283646 which leads to this ImageMagick issue" but I wonder if it might also have been done on purpose, to avoid issues where people have uploaded transparent images in the era of Wikipedia not supporting dark mode, not expecting them to be set against anything other than a white background...
  3. I'm not completely sure that any of this makes sense for me to worry about, and perhaps it is actually fine to upload only a 60 MB PNG and let Mediawiki worry about bandwidth efficiency. I need to check in with people more familiar with the technology involved, which is what I'm doing right now here at the Village Pump.
I would note that Mediawiki doesn't seem to be doing a great job with my 60 MB PNG, still serving it as a 4 MB thumbnail image at least in File: namespace, while the WebP I uploaded is 2 MB at full resolution (barely discernible loss of quality), is converted to a 1.13 MB PNG for display in File: namespace, a 75% savings.
I'd note it is a little frustrating that it converted my WebP to PNG, but I suppose this was done to ensure compatibility. https://caniuse.com/webp claims WebP support is up to 95.69% among browsers that are currently used. The remaining 4.3% is still a sizable portion to support, especially in the context of Wikipedia, but it seems like it would at least be possible to read my useragent and determine that like almost all browsers released in the past 5 years, my browser supports WebP and can be served a smaller file.

In summary, please leave your thoughts on if using Lossy WebP in Template:Compressed version is valid and useful, or if there is some better technical way going forward, such as uploading only a Lossless WebP (which unlike PNG, does properly support photo metadata). —Hubcapp(talk) 18:37, 25 October 2024 (UTC)[reply]

1. {{Archival version}} is to indicate "don't mess with this, so that others can mess with the same reference at a later time". Now we could say that this is somewhat dated, as we already don't allow most people to change the version of an image, asking people to upload them separately.
2. Storing VERY high resolution images, means that the thumbnail has to take a huge leap from large to small. And it needs to do this with settings that are 'OK' for most of the images. This at times can result in suboptimal results. For that reason, it may be better to make your own slightly smaller version, applying self selected settings, and then uploading that smaller version of the file, for 'general use' in the Wikipedia, where often images get downscaled to something like 220px or smaller. However, in MOST cases, the images aren't used that widely and that critical that this is warranted, and it can always be done later.
3. Compressed version: This template indeed was not handling webp. I have updated its dependency to provide support for this.
4. In general, you should not worry about data and bandwidth usage. Where required and useful, as well as permitting time and money, the engineers will use newer technologies as they become ubiquitously available. In many cases having a simpler more 'wasteful' flow, makes up by being way faster and easier to manage and understand in the short term. (At the same time, this is not a license for uploading multi gigabyte files with little value and/or use). —TheDJ (talkcontribs) 15:44, 29 October 2024 (UTC)[reply]
Re 2.: There exists Template:No scaled down dupes, which implies it is improper to upload a lower resolution version of an image. Do you mean it's actually OK to upload a "smaller" version (in terms of resolution), or by "smaller" did you mean "more optimally compressed but still at full resolution"? And to clarify, you're saying that the Archival/Compressed versions dichotomy is in fact still useful for new uploads (but generally only for very high traffic images where it matters more)?
Re 3.: Thank you for clarifying it's OK to use WebP in Template:Compressed version and providing support for it!
Re 4.: Is it perhaps a good idea to upload *only* Lossless WebP? In my case, the resolution of my camera is roughly 6000 by 4000, so I'm not in multi-gig territory. However "Very Large Image" seems to be both a matter of opinion, and a definition that has changed as technology has advanced.
Thanks, —Hubcapp(talk) 02:34, 30 October 2024 (UTC)[reply]

Wikimaps Warper edits not working

[edit]

Somehow Special:OAuthListConsumers/view/d3308b00209d49e5a1d764aee019f83c stopped working for me. The tool sets the bounding box for maps depending on definitions of control points at https://warper.wmflabs.org . It's unclear who currently maintains the tool. @Chippyy: seems inactive.

For several, including Here, I had to complete the infobox fields manually. It doesn't seem to be a problem of the other content on file description pages, as this didn't get updated either.

It does work for @Kognos at [1] and a few other contributors [2].
 ∞∞ Enhancing999 (talk) 10:19, 27 October 2024 (UTC)[reply]

Also, it doesn't appear to depend on the absence of SDC statements: [3] failed too.
It doesn't seem to work for other users any more either: [4] by @Groupsixty.
 ∞∞ Enhancing999 (talk) 11:25, 27 October 2024 (UTC)[reply]
I'm afraid I have no idea why it's not working for others. I have had problems with the warper in the past, but it has been working OK, if sometimes a bit slow, recently. As you have noticed, I use the warper quite a lot. I set the template to Map, then click on the "Georeference..." button. If I am not currently logged on to the warper I get a logon page, select log on through Commons, then allow Oauth. I then georeference the map, and the bounds are automatically updated. I guess this is no different from what others are trying to do. I agree that the lack of any clear path to support is frustrating. I have been quite unclear in the past as to where any requests for help should be posted, and have had little success in getting support. The Wikimaps Warper category is useful, but there is still no indication as to who actually owns the project. Any enlightenment appreciated! Kognos (talk) 18:09, 28 October 2024 (UTC)[reply]
Maybe it's a browser issue, It used to work for me last year. The category is at Category:Wikimaps Warper.
 ∞∞ Enhancing999 (talk) 18:39, 28 October 2024 (UTC)[reply]
@Susannaanas do you know who is maintaining wikimaps warper? --Zache (talk) 18:27, 28 October 2024 (UTC)[reply]

Can category content be viewed by date?

[edit]

Hi. Category content is always sorted by alphabet as the default. This can be unconvenient when parts of series become scattered all over the list. Is it possible or could it be possible to overview the contents by date at the push of a button? This would be pretty neat. I will give two examples : 1 sorted by alphabet and 2 sorted by date, (more coherent). As you can see this link is very long and can be hard to put into a short working link (such as a text button). I would like (the option) to keep the layout ot example 1 in favor of listed as in 2, because it shows images next to eachother and gives overall a better overview per viewed segment of pages, like as if looking trough a category. This project idea would make the category contents available, to more people in a more appealing format, to have series more or less grouped by upload-date, and not scattered by alphabet. This would be more ergonomic and visually intuitive. Can this be done? Thanks. Peli (talk) 14:26, 28 October 2024 (UTC)[reply]

Currently if you enable Help:Gadget-DeepcatSearch it lets you 1 click to a deepcat search of the cat you are viewing. with a few more clicks you can get to a sorted-by-upload-date list.
i have considered expanding its functionality to include more direct links to oldest-first, newest-first... lists.
it's also not difficult to make some user js to make those links. RoyZuo (talk) 14:48, 28 October 2024 (UTC)[reply]
Honestly, having a one-click sort by date would be super appreciated. I could only wish to have the ability to make custom js. Huntster (t @ c) 17:46, 28 October 2024 (UTC)[reply]

Ah cool. But it would not be available to all visitors unless they install it? I wanted to do a post script of previous request. :: Earlier trials to install it on that page failed, but now the link is on the page. This fixes my issue, and keeps this as wanted extention to all cat pages if possible in any way. To me this sorting method means a incredible release of the pressure to rush to categorize everything (over 100k items) just to be able to overlook some parts of this material as more coherent series. Especially now the tool Cat-a-lot has become a much too slow tool for this scale projects. Thanks. I had that gadget enabled and just now explored the sort options, seems to work well to find other points of departure to explore this huge 'unsorted' category. Peli (talk) 14:26, 28 October 2024 (UTC)[reply]

  • I think it would be great if that gadget would be enabled by default. However, for that to happen the deepcategory problems (at least phab:T376440) would need to be solved.
  • The link currently shows items directly in the category as well as in subcategories – if you only want to see items directly in the category, change "deepcategory" to "incategory".
  • Another issue is that one can only sort by upload date, but not by date of what's in the media file (the date field in the file's {{Information}}) – see e.g. phab:T329961 and the things linked there. Cat-a-lot has recently been improved so should be faster again.
Prototyperspective (talk) 21:43, 28 October 2024 (UTC)[reply]
If you want to sort by date, ppl need to get on the CommonsData bandwagon, because that is the only way that will ever become a reality. —TheDJ (talkcontribs) 15:18, 29 October 2024 (UTC)[reply]
Disagree. What would be needed is to make use of the Information template – see #How to search fields of files' Information template?. I don't care whether the data from that template is 1. read and written to structured data and 2. kept in sync with structured data or whether it can be searched another way, I just disagree that this is required and also see an issue where this data would need to be written to over 100 million files and thus may not be feasible while one can already search data in the file information template via insource which could readily be improved upon. (If it doesn't become a reality otherwise, that is because of ideology and/or having invested much into a somewhat redundant techcomponent that makes it hard for some to see or consider, possibly–likely better, alternatives.) Prototyperspective (talk) 16:44, 29 October 2024 (UTC)[reply]
Is that (whatever it means or implies) really needed, just to find out the upload-date? This can be done already in special search. Pelikana (talk) 15:29, 29 October 2024 (UTC)[reply]
No, the comment was about the date the media was recorded. I think I overread "upload-date" and the title was unclear on whether you mean upload-date or date the file was taken but it's also relevant in general so I thought it was relevant enough to mention. Prototyperspective (talk) 16:50, 29 October 2024 (UTC)[reply]
@TheDJ do you mean sdc or...? RoyZuo (talk) 19:06, 1 November 2024 (UTC)[reply]
Ok. tx Any way to get around alphabetical order can be helpful in some huge unsorted categories. On smaller cats and sorted cats by crafted sortkeys the option would be needless and to be avoided. Maybe it could be just an optional piece of code to put in the header of the most relevant huge unsorted categories. I have another topic to ask attention for, it is on the [talk page of visual file change]. Open question and request about scrolling by keyboard. Peli (talk) 10:21, 31 October 2024 (UTC)[reply]
Requested more than a decade ago (phab:T71417) and triaged as "Low" priority. 😒 — Draceane talkcontrib. 20:16, 1 November 2024 (UTC)[reply]
The API also supports by date added to the category (Not neccessarily the same thing as date created) - https://commons.wikimedia.org/w/api.php?action=query&list=categorymembers&cmtitle=Category:Uncategorized_images_of_the_Rijksmuseum&cmsort=timestamp&cmlimit=max . In theory it would be easier to add support for sorting categories in that way than other forms of sort by "date". Someone would still have to do it though. Bawolff (talk) 06:31, 4 November 2024 (UTC)[reply]
That would also be very useful. Maybe that should be added to that linked phab issue? I don't think it would be as useful as sorting by date taken however. Prototyperspective (talk) 10:54, 4 November 2024 (UTC)[reply]

Upload Wizard very slow

[edit]

For the past few days, Upload Wizard has been very slow. Uploads that used to take secs now take mins. The behaviour I see: when I start uploads, it runs at 1-2 MB/sec for a few seconds. Then it slows down to < 1KB/sec for long periods. Intermittently, there is a burst of 1-2 MB/sec for a few seconds.
When I use chunked upload for a new version of a file, with chuck size 4 MB, the chunks upload fast, but then finishing and posting takes several minutes.
Others have reported similar issues @廣九直通車 and MPF: . Would appreciate any suggestions and help. Tagooty (talk) 14:51, 28 October 2024 (UTC)[reply]

I just uploaded 3 images with Upload Wizard, each with around 10MB size. It took me almost 9 minutes to upload, compose and queue the files, and after populating the files with descriptions and categories, it takes me another 3 minutes to finish the whole upload procedure. Moreover, during the first 9 minutes, I remember Upload Wizard reported errors with server connection. What's the problem? Should I send the matter to Phabricator?廣九直通車 (talk) 08:04, 29 October 2024 (UTC)[reply]
Dito, also have errors when uploading larger files --PantheraLeo1359531 😺 (talk) 08:08, 29 October 2024 (UTC)[reply]
Interestingly, I also tried direct upload from Flickr. Everything works perfectly fine in this case. Interesting...廣九直通車 (talk) 10:08, 29 October 2024 (UTC)[reply]
Today, I'm unable to upload any picture with Upload Wizard. I have opened a new thread on a help page about it. -- Jakubhal 17:33, 29 October 2024 (UTC)[reply]
Hi all, I would suggest you to open a Phabricator ticket about the problems you're having, and then kindly link it back to me, so that I could solicitate some action from the devs. Is it possible? Sannita (WMF) (talk) 18:43, 29 October 2024 (UTC)[reply]
I guess this is phab:T378276 Bawolff (talk) 02:50, 31 October 2024 (UTC)[reply]
Well, as of 31 October, it appears that the problem is largely solved. Just uploaded a total of 55MB of files, and the process is smooth. This thread may be kept in case things break again, however.廣九直通車 (talk) 08:29, 31 October 2024 (UTC)[reply]
Seems like the underlying cause is phab:T378385. Basically, the upload wizard (chunked upload) jobs share infrastructure with other jobs that only get triggered rarely. All of a sudden flagged revisions extension started making a lot of a type of job that normally does not get triggered very often. This meant that the UploadWizard jobs had to wait for those to complete first, slowing things down. It seems like that situation resolved, so upload wizard should be back to normal now. Bawolff (talk) 21:27, 31 October 2024 (UTC)[reply]
Sounds like weird planning when batch jobs meant to do stuff in off-hours to reduce load for day-to-day operations in one wiki sink regular operations in another wiki.
 ∞∞ Enhancing999 (talk) 07:13, 1 November 2024 (UTC)[reply]
Similiarly: phab:T378385#10282871
 ∞∞ Enhancing999 (talk) 22:33, 1 November 2024 (UTC)[reply]
@Bawolff Not sure if phab:T378276 is resolved and the root cause properly addressed as task phab:T379035 is titled "consider".
@Sannita (WMF): is there any progress on your side?
 ∞∞ Enhancing999 (talk) 12:53, 7 November 2024 (UTC)[reply]
The immediate issue is resolved - its no longer slow. The other task is a follow up to try and reduce the risk of a similar thing happening in the future. Bawolff (talk) 13:59, 7 November 2024 (UTC)[reply]
Well, it's no longer slow each time a user aborts an upload, but that doesn't mean the problem was resolved.
Given the current configuration that has Commons uploaders wait while MediaWiki performance maintenance at Wikipedia, it's likely to happen again. We don't want to open and close tickets merely because a user closed their browser.
 ∞∞ Enhancing999 (talk) 14:06, 7 November 2024 (UTC)[reply]
As a general principle, tasks are supposed to represent something to do. The phab:T379035 is supposed to represent the long term fix, where the other one is supposed to represent making things work right now & figuring out what happened. Since the long term fixes stuff is encapsulated in the other task, it makes no sense to keep the T378385 open since there is nothing to do on that task that isn't covered by T379035 [That said, the issue is possibly reoccurring right now]. Bawolff (talk) 10:24, 9 November 2024 (UTC)[reply]
phab:T378276 is about an existing, severe problem that needs to be solved.
phab:T379035 about an investigation that may or may not fix the problem. For casual readers, it might not even clear that is concerns Commons and that it could fix a severe issue at Commons.
 ∞∞ Enhancing999 (talk) 11:34, 9 November 2024 (UTC)[reply]
Thanks to those working on these issues. For the record, I recently posted to the Help Desk here: Help_desk#Upload Wizard broken for me. RobbieIanMorrison (talk) 20:11, 9 November 2024 (UTC)[reply]

Good news: phab:T379035#10318390 notes the deployment, so category updates at Wikipedia will no longer have Commons uploaders wait while uploading. Also, if I understood the fix correctly, uploads by url wont impact upload wizard performance any more. Thanks for everyone involved in finally investigating and fixing this.

It would be good to have links to the reports allowing to check those upload queues for saturation.
 ∞∞ Enhancing999 (talk) 14:04, 14 November 2024 (UTC)[reply]

Tech News: 2024-44

[edit]

MediaWiki message delivery 20:52, 28 October 2024 (UTC)[reply]

Russian txt in English field

[edit]

Is it possible to automatically switch all the descriptions created by Aleksandr Patil from English to Russian? He apparently is not aware that he is using the English field. Hjart (talk) 21:09, 1 November 2024 (UTC)[reply]

I guess we need a friendly bot owner. Ymblanter (talk) 21:55, 1 November 2024 (UTC)[reply]

How to copy texts from Wikipedia articles via browser automatically?

[edit]
Illustration of process I'm looking to automate more

Does somebody know how to automatically fetch Wikipedia article content into local files? The caveat here is that the API probably can't be used since the content is modified via CSS based on the CSS classes of the text content. Scraping the full HTML files and then converting these to text locally after the CSS modifications would probably be too difficult – I think automating the browser would be easier and maybe there are further methods.
The content I'm looking to copy is the article contents up to the See also/Notes/References section. It would be best if all tables removed via hiding their CSS classes mark the file somehow so depending on the table one can insert the table in list format manually (this is under development, it would be best if it automatically converted tables or added a note that 'Table XYZ is not included here'). Prototyperspective (talk) 12:27, 3 November 2024 (UTC)[reply]

Template:LicenseReview, where to find the dropdown menu messages?

[edit]

Hello,

as a license reviewer, I'll have the option to click either "[license +]" or "[license -]". On "license -", I get a dropdown choice of different options, like "all rights reserved" or "non-commercial". Where are these text strings located? Where is a good place to ask for additions to them (maybe here?)? I'd like to have something like License review failed -> Reason: "Source link dead & archived versions unusable." to document the reality. Regards, Grand-Duc (talk) 22:01, 3 November 2024 (UTC)[reply]

Hi Grand-Duc! If the file was uploaded long ago you might want to read/comment Commons:Village_pump#Almost_400k_files_need_license_review. Asking for new options could be placed at MediaWiki talk:Gadget-LicenseReview.js. --MGA73 (talk) 13:20, 7 November 2024 (UTC)[reply]

Tech News: 2024-45

[edit]

MediaWiki message delivery 20:46, 4 November 2024 (UTC)[reply]

template:information undisplayable name

[edit]

I found the error. even if ・・・・ inputs template to them name but {{Author missing}}. thanks. --eien20 (talk) 22:28, 9 November 2024 (UTC)[reply]

Garbled characters in SVG image

[edit]
Fiji ....

See File:Fiji Province.svg. How can I fix it?--BigBullfrog (talk) 14:54, 10 November 2024 (UTC)[reply]

@BigBullfrog:
The file was using the Fantasy font. On Commons, that apparently turns Latin characters into special characters (which is not UTF-8). I overrode that choice with a style block that specifies text elements should use sans-serif.
The file also has Arabic in it (with display="none"), but it should use translation units rather than translation planes.
Glrx (talk) 15:30, 10 November 2024 (UTC)[reply]
@Glrx: It doesn't work in some Wikipedias (e.g. en wiki, de wiki). --BigBullfrog (talk) 18:36, 11 November 2024 (UTC)[reply]
Those links work for me. You may need to refresh the page without using the browser cache. Glrx (talk) 20:25, 11 November 2024 (UTC)[reply]

Problem with thumbnails

[edit]

File:Yiannis Boutaris Drasi.jpg is doing a problem with thumbnails, it can't be used in Wikipedia, Regards!! --Ezarateesteban 22:03, 10 November 2024 (UTC)[reply]

Creation of new pages logged as performed unknown action "create/create"

[edit]

It seems that the creation of new pages currently shows up in the logs as [User] performed unknown action "create/create". Just reporting this for the technically minded... Gestumblindi (talk) 00:12, 11 November 2024 (UTC)[reply]

Yikes. filed —TheDJ (talkcontribs) 09:58, 11 November 2024 (UTC)[reply]

Please help me clean up this template

[edit]

I created {{AlphaFold}} to better address a frequent source of AI images that I was seeing, but I'm not good enough with templates to set up proper langswitching, etcetera. Could someone take a look and make sure I didn't break anything? Thanks! The Squirrel Conspiracy (talk) 04:48, 11 November 2024 (UTC)[reply]

Tech News: 2024-46

[edit]

MediaWiki message delivery 00:03, 12 November 2024 (UTC)[reply]

NamespaceInfo and subpages

[edit]

Special:NamespaceInfo lists namespaces that can have subpages. Sample: "User talk" can have subpages, "Files" can't.

Currently the following namespaces have subpages enabled, but I'm not sure if they should:

  • Category
  • Creator
  • Institution


 ∞∞ Enhancing999 (talk) 10:32, 12 November 2024 (UTC)[reply]

Problem with Commons categories in Multilingual sites in Wikidata, consequences for Commons

[edit]

In Wikidata there are Wikidata items that have a label that starts with "Category", have a statement "instance of" and have a link to "category's main topic" (and vice versa in the linked Wikidata item about the subject with a "topic's main category"); here referred to as Wikimedia category and subject Wikidata item.

I always thought that such Wikidata items in Commons are automatically put through to for example the Wikidata Infobox and corresponding Wikipedia's. Example: d:Q9097136 is such a Wikimedia category; its 'Multilingual sites' contains a link to the Commons category Category:Spijkenisse and it has no image, that is in the corresponding subject Wikidata item d:Q488545. The Wikidata Infobox in Category:Spijkenisse shows the image and other information from the subject Wikidata item and it shows even the link to the subject Wikidata item, but this subject Wikidata item does not have any 'Multilingual sites'. And the EN-WP article w:en:Spijkenisse has a link to the correct Commons category (under "Tools"), though there is no link in the Wikimedia category to the EN-WP article. So my conclusion is that there is an automatically procedure that links both Wikidata items to each other and their content to all the places where they are needed.
 Question Is this correct?

Problem: But now there is a discussion on Commons:Categories for discussion/2024/09/Category:Redirects connected to a Wikidata item where someone pointed out that there are situations in which this automatism does not work and so people invent work-arounds that causes other problems (in this case often an empty gallery page is created with a redirect to the Commons category, and this empty gallery page is used in the 'Multilingual sites' of the subject Wikidata item, see all pages in c:Category:Redirects connected to a Wikidata item for examples). One problem is a link in a Commons category to a Wikidata "Family tree" template:

  • Open the c:Category:Carlo_Buonaparte and the "Family tree" template. On the side of the sons you can see "Joseph Bonaparte" does not have any link, just a "[d]" you can click to enter his d:Q7726 Wikidata entry: There you can see the 'Multilingual sites' field has no entries, as the Category:Joseph Bonaparte is instead linked to d:Q31993664. If you are looking for images you have to do a long scroll to the Property:P373 in order to jump back to Commons. So even if the Commons category is present in the property "Commons category" of d:Q7726 it makes no direct link.

 Question Can this problem be solved? Is this the right place to ask for such a solution? I already have asked the same question on d:Wikidata:Project_chat/Archive/2024/11#Problem_with_Commons_categories_in_Multilingual_sites but I did not get an answer. JopkeB (talk) 05:39, 14 November 2024 (UTC)[reply]

I think it's fairly easy to fix: One should just add Commons sitelinks to the item with the articles they describe, whenever possible.
 ∞∞ Enhancing999 (talk) 12:42, 16 November 2024 (UTC)[reply]
Thanks Enhancing999 for your reaction. What do you exactly mean? For the example of Category:Carlo_Buonaparte: what should be done to get the correct links to Joseph Bonaparte? JopkeB (talk) 04:49, 18 November 2024 (UTC)[reply]
Ask Wikidata to link Category:Joseph Bonaparte from d:Q7726.
 ∞∞ Enhancing999 (talk) 10:45, 18 November 2024 (UTC)[reply]
✓ Done Anyone can do that. Why did you say "Ask Wikidata"? Yann (talk) 11:13, 18 November 2024 (UTC)[reply]
Because, it's likely that your edit will be undone shortly (not by me).
 ∞∞ Enhancing999 (talk) 11:19, 18 November 2024 (UTC)[reply]
I don't see why. I have done hundreds such edits, and they were very rarely contested. I think the links were automatically and erroneously added when a Wikipedia category exists. Commons categories should be linked to Wikipedia articles, not to Wikipedia categories. Is there anyone opposing that? Yann (talk) 11:37, 18 November 2024 (UTC)[reply]
@Yann: I am afraid Enhancing999 is right: Pi bot undid your change and the problem has not yet been solved. JopkeB (talk) 15:32, 18 November 2024 (UTC)[reply]
Ah. :( @Mike Peel: Hi, Please stop your bot from linking to the wrong items, i.e. [15]. There is quite a consensus that Wikipedia articles should be linked to Commons categories. Thanks, Yann (talk) 15:36, 18 November 2024 (UTC)[reply]
AFAIK Mike Peel has decided that he has authority over what Commons users want. :((( Now I have opened a discussion on Wikidata:Project chat. You may want to give your opinions there. Yann (talk) 18:24, 18 November 2024 (UTC)[reply]
Yann: do you have diffs to support your statement "I have done hundreds such edits"?
 ∞∞ Enhancing999 (talk) 14:22, 19 November 2024 (UTC)[reply]
OK, so this is actually an issue with {{Wikidata/FamilyTree}}, @Pyb: can you help? The Lua code that looks for the Commons link should automatically follow topic's main category (P910) values to the category item and get the Commons sitelink from there, but it looks like it currently doesn't. You could use the 'getCommonsLink' function from Wikidata Infobox to do this. I'll also raise this at Template talk:Wikidata/FamilyTree. Thanks. Mike Peel (talk) 07:23, 19 November 2024 (UTC)[reply]
@Mike Peel: No, it is not. This issue is not only for people, but for any subject.
Wikidata links to the wrong items. Demonstration (and please note that this is valid for any Wikimedia project, not only Commons): Say that a project has a page X linked to a WD item #1. Then, as the subject becomes sufficiently important, a subpage X1 is created. Will WD change the link to another item?
If there is no gallery, a Commons category is linked to the corresponding WD item. Then why changing the link if a gallery is created? Yann (talk) 07:48, 19 November 2024 (UTC)[reply]
Subpages are rare on Wikipedias. Category items still exist for Wikipedia categories, though - in the named case, you broke the Commons link at de:Kategorie:Joseph Bonaparte with your edits. For templates, the solution is to follow topic's main category (P910)/category's main topic (P301), which is what the Wikidata Infobox does, and could also be applied here. It's not optimal, and and I would like to see things improved in the long term - but it's what works practically right now. If you want to have a better solution, get a clear consensus somewhere, and go ask WMDE to make changes to Wikidata. Mike Peel (talk) 08:22, 19 November 2024 (UTC)[reply]
According to [16], things are not set in stone. Martin says that links could be changed if there is a consensus on Commons for that. Yann (talk) 16:48, 19 November 2024 (UTC)[reply]

Templates in description field bug

[edit]

Continuing Commons:Village pump/Technical/Archive/2024/10#Templates in description field cause empty caption in Media Viewer: I write to inform that I reported the problem as https://phabricator.wikimedia.org/T380190. You can add more details if you wish, or interact in some other ways – I'm not familiar with Phabricator savoir vivre and politics.

I would also like to add that I feel demotivated and hopeless by the way the now archived thread linked above was handled. I asked for help and no one even bothered to answer "no, I won't help, do it yourself". The thread was left to die, without anyone giving a helping hand to bring the topic closer to some solution. I don't think that ignoring a problem makes it disappear. This was an unpleasant experience for me to collaborate this way. I hope someone reads this feedback and makes some use of it. Derbeth talk 16:15, 18 November 2024 (UTC)[reply]

Maybe User:Sannita (WMF) can help? It's a problem I was wondering how to get around it before.
 ∞∞ Enhancing999 (talk) 14:25, 19 November 2024 (UTC)[reply]
I'll flag it to the devs and see if they can do something about it. I'm sorry that @Derbeth had this unpleasant experience. I'll keep you posted about it, but please be aware that it might take some time. Sannita (WMF) (talk) 14:51, 19 November 2024 (UTC)[reply]

Thank you Sannita. I fully understand fixing bugs takes time. I just wish discussions here were more like "exchanging different opinions -> agreeing what to do -> doing", instead of "dissolving" without any conclusion, as I felt was the case with the topic I started in October. --Derbeth talk 16:51, 19 November 2024 (UTC)[reply]

Yeah, I see. As I said, I'll try this time to get you a solution for it, instead of just disappearing. If you don't hear from me, you're free to ping me on my talk page about this bug. Sannita (WMF) (talk) 17:21, 19 November 2024 (UTC)[reply]
I wish that too. I think people did not engage exchanging different opinions -> because it's clear this needs to be fixed and people share the opinion that it needs to be fixed with it not being useful in regards to getting it fixed to write that. -> agreeing what to do for bugs people generally agree that it should be reported and fixed. Somebody should have asked to create a phab issue. -> doing this part is the key problem. Even major bugs like all interactive charts on Wikipedia being broken or Commons categories not showing on mobile are not getting fixed after timescales of 5 years to decades. I don't think complaining about this is enough anymore so I made concrete suggestions on how to fix this here. They could readily be discussed and implemented which would result in all technical issues being addressed more often and more quickly:
Please increase MediaWiki development capacity further.
Prototyperspective (talk) 17:37, 19 November 2024 (UTC)[reply]
The commons category thing is not a case of insufficient resources. The graph thing is partially a resource problem, but that is hardly the only issue there and it would be a gross oversimplification to boil it down to just that. Some problems are resource problems, i would even say many problems on commons are resource problems, but not all problems are resource problems. Bawolff (talk) 21:09, 19 November 2024 (UTC)[reply]
Those were just examples. Meant to add a note that the hiding of categories on mobile doesn't even really require much resources but dropped it; this just makes it even worse. Add them already, what gives WMF the permission to just hide away this core useful data I wonder. It's not a simplification to say things are technical development capacity problems, it would be solved much faster with more thinking/development capacity or are we waiting for a new Internet protocol for it to be invented before it can be addressed? I was not describing "resource" problems, I was describing a lack of technical development...obviously increasing technical development would result in more technical issues getting fixed and faster. Prototyperspective (talk) 21:15, 19 November 2024 (UTC)[reply]
Sure, but i also think some of these problems are deciding what to do problems. No amount of technical resources will fix a problem if the underlying issue is we simply decided not to do it. Like the category thing in particular is pretty clearly not a case of WMF being too busy - its a trivial change. Its a product failure - there is a mismatch between what commons thinks is important and what that particular team thinks is important. If we want to fix that we need better communication not more people fixing bugs (there are other things where more people would be helpful). Bawolff (talk) 09:11, 20 November 2024 (UTC)[reply]
These were examples so ignore this example if it's a bad example. we simply decided not to do it we did not. its a trivial change which is one reason why I don't donate to WMF and think they're doing a bad job what that particular team thinks is important what team? we need better communication there have been Wishlist proposals and phabricator issues and it's the most-supported request in the technical needs survey. All WMF needs to do is listen a bit and/or engage in any of these venues. not more people fixing bugs I see a tiny fraction of usually quite useful proposals in meta:Category:Community Wishlist Survey results have been implemented so far and on phabricator it's an even more desolate situation. We need far more people fixing bugs. Prototyperspective (talk) 19:28, 20 November 2024 (UTC)[reply]

Tech News: 2024-47

[edit]

MediaWiki message delivery 01:57, 19 November 2024 (UTC)[reply]

"Content"-pages on Special:NamespaceInfo

[edit]

Special:NamespaceInfo marks some namespaces as "content"-namespaces. Presumably at Commons that is "File:"-namespace. You might want to compare it with Wikipedia:Special:NamespaceInfo and Wikidata:Special:NamespaceInfo.

It's not clear what the exact meaning or definition of "content"-namespace is. Maybe it's merely that they are counted that way for Special:Statistics? Weirdly it shows more "uploaded files" than content pages.

Possibly we should also add "Data:"-namespace and "TimeText:"-namespace to the group.

Not sure about "Category"- and "Gallery"-namespace. I'd probably either add or remove both.
 ∞∞ Enhancing999 (talk) 14:32, 19 November 2024 (UTC)[reply]

Content namespaces are usually default namespaces to search, namespaces used forspecial:statistics, used in a bunch of query special pages, and a bunch of other places in mediawiki. Its basically anything that should be considered real content as opposed to something like the Commons: namespace which hosts meta content. You can request the list of content namespaces be changed at phabricator. Bawolff (talk) 20:54, 19 November 2024 (UTC)[reply]

"Use this file" no reaction

[edit]

File:Dunluce Castle. County Antrim, Ireland-LCCN2002717364.jpg clicking the gadget has no response for me. is it the same for you? RoyZuo (talk) 15:12, 20 November 2024 (UTC)[reply]

Date bug?

[edit]

in the mediaviewer it says "Created: 1500 Uploaded: 17 June 2014". i dont know where 1500 comes from. RoyZuo (talk) 15:13, 20 November 2024 (UTC)[reply]

There is some hidden text in the template output that says: between 1890 and 1900<div style="display: none;">date QS:P571,+1500-00-00T00:00:00Z/6,P1319,+1890-00-00T00:00:00Z/9,P1326,+1900-00-00T00:00:00Z/9</div>. Possibly this comes from Module:Complex date but i am not sure. Bawolff (talk) 17:06, 20 November 2024 (UTC)[reply]

Category and disambiguation

[edit]

Hi everybody. I hope it's the right place to my question about categories and disambiguation. I started to move, but now I stopped it, categories of Category:Municipalities in Spain having ambiguos titles to leave space for the disambiguation page. For example I moved Category:Lena to Category:Lena (Asturias), Category:Orbita to Category:Orbita, Ávila, Category:María to Category:María, Almería and so on, around 20 or maybe 30 movings.

N.B. In other situations, when the topic was clearly the main one, I just created a disambiguation page, for example in Category:Ávila I added a link to Category:Ávila (disambiguation), the same in Category:Segovia and Category:Segovia (disambiguation).

Anyway I was asked to revert some edits for some categories, Category:Pego, Category:Llerena, Category:Tormos, Category:Caso,Tibi, Ibi, Navia and some others (but not clear exactly which ones) with the motivation that "are all municipialities and therefore primary topics".

My question is about it: is it always the municipality the primary topic? I do not think so, I can't find a rule about it and I think that every situation is a specific one. I start to check other examples in Category:Disambiguation categories and it seems that the municipality is NOT always the main topic, including when municipality it's olny one. Just some examples in alphabetical order. Category:Aba, Nigeria and Category:Aba, Category:Acushnet and Category:Acushnet, Massachusetts, Category:Adria and Category:Adria (Italy), Category:Adstock and Category:Adstock, Quebec.

I'm ready to revert and/or fix my edits if they are wrong, but I would to know if there are any rules about it or it's better to analize every single situation one by one, and use the talk's page in the category to discuss it. Thank you. MrKeefeJohn (talk) 07:48, 21 November 2024 (UTC)[reply]