Commons talk:Video2commons/Archive 1

This is an archive of past discussions. Do not edit the contents of this page. If you wish to start a new discussion or revive an old one, please do so on the current talk page.

Very slow

I'm trying to upload a YouTube video using Video2commons. About 40 minutes after starting it, it's still at 6 percent. Is this the usual speed or does this mean it isn't working? SarahSV ^(talk) 15:06, 10 December 2016 (UTC)

Finally done, so that has answered the question. SarahSV ^(talk) 16:36, 10 December 2016 (UTC)

@SlimVirgin: Yeah that's kind of usual for a 1,920 × 1,080 pixels (1080p / 2K) video with 20m duration. At this resolution, the transcoding time is usually a few times larger than the video duration to preserve the quality. --Zhuyifei1999 (talk) 04:04, 11 December 2016 (UTC)

@Zhuyifei1999: thanks for the reply. It was the first time I'd done it, so I didn't know what to expect, but the next one I uploaded was faster. I know now just to let it run. Thanks again, SarahSV ^(talk) 15:45, 11 December 2016 (UTC)

This section is resolved and can be archived. If you disagree, replace this template with your comment. --Roy17 (talk) 20:01, 22 July 2019 (UTC)

Video formats

A question, what video formats supports the tool? Bye, --Elisardojm (talk) 22:40, 24 January 2017 (UTC)

Before transcoding? I honestly don't know. FFmpeg's list shows a lot. (Sorry not pasting the list because it is too huge) --Zhuyifei1999 (talk) 07:41, 25 January 2017 (UTC)

Oh, I only wanted to know if it supported commons formats, I didn't know that exists so many formats! :) Thanks Zhuyifei1999. Bye, --Elisardojm (talk) 21:41, 25 January 2017 (UTC)

This section is resolved and can be archived. If you disagree, replace this template with your comment. --Roy17 (talk) 20:01, 22 July 2019 (UTC)

Translation of "Links"

Hi, at translation tool of this help page, it don't allow to translate the header "Links" to other languages, could somebody fix that? Bye, --Elisardojm (talk) 08:32, 22 February 2017 (UTC)

Should be

Done --Zhuyifei1999 (talk) 10:38, 22 February 2017 (UTC)

Yes!, thanks! --Elisardojm (talk) 10:41, 22 February 2017 (UTC)

This section is resolved and can be archived. If you disagree, replace this template with your comment. --Roy17 (talk) 20:01, 22 July 2019 (UTC)

Conversion of wav and mp3

I find this very useful to upload and convert wav and mp3 files from my mobile. However the uploaded files have an extra timed-text tab which can be suppressed for these formats. Shyamal L. (talk) 09:02, 7 April 2018 (UTC)

The tab is added by MediaWiki software. The tool can't suppress it --Zhuyifei1999 (talk) 15:40, 7 April 2018 (UTC)

This section is resolved and can be archived. If you disagree, replace this template with your comment. --Roy17 (talk) 20:01, 22 July 2019 (UTC)

Translation marks

Steinsplitter, could you add the translation marks to the new content? I have tryed but it seems that don't worked. Bye, --Elisardojm (talk) 20:08, 25 April 2018 (UTC)

Thanks Brackenheim! Bye, --Elisardojm (talk) 07:47, 30 April 2018 (UTC)

You're welcome ;-) --Brackenheim (talk) 08:15, 30 April 2018 (UTC)

This section is resolved and can be archived. If you disagree, replace this template with your comment. --Roy17 (talk) 20:01, 22 July 2019 (UTC)

Error message when I upload from Youtube to commons

I am getting an error message whenever I wanted to get public domain content from Youtube to commons. One can see the message here: https://pastebin.com/sx7a3taA. --Pavan santhosh.s (talk) 09:47, 21 October 2018 (UTC)

@Pavan santhosh.s: I just upgraded youtube_dl version. Could you check if it fixed the issue? --Zhuyifei1999 (talk) 16:37, 21 October 2018 (UTC)

This section is resolved and can be archived. If you disagree, replace this template with your comment. --Roy17 (talk) 20:01, 22 July 2019 (UTC)

Error when I try to convert a Youtube video

There is this problem :
"Error: An exception occurred: DownloadError: ERROR: FHQgXqK3M28: "token" parameter not in video info for unknown reason; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see https://yt-dl.org/update on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output."

--ComputerHotline (talk) 18:35, 24 April 2019 (UTC)

I just upgraded youtube-dl. Could you try again? --Zhuyifei1999 (talk) 20:52, 24 April 2019 (UTC)

It works. Thanks. --ComputerHotline (talk) 05:43, 25 April 2019 (UTC)

This section is resolved and can be archived. If you disagree, replace this template with your comment. --Roy17 (talk) 20:01, 22 July 2019 (UTC)

Signature extraction failed

Hi, Any idea this fails to import? I get a quite long error message starting with Error: An exception occurred: DownloadError: ERROR: Signature extraction failed: Traceback (most recent call last): File "/data/project/video2commons/www/python/venv/local/lib/python2.7/site-packages/youtube_dl/extractor/youtube.py", line 1236. Regards, Yann (talk) 12:58, 3 July 2019 (UTC)

I just upgraded youtube-dl. Does that fix it? --Zhuyifei1999 (talk) 15:11, 3 July 2019 (UTC)

Yes, it works! Thanks a lot. Yann (talk) 16:05, 3 July 2019 (UTC)

This section is resolved and can be archived. If you disagree, replace this template with your comment. --Roy17 (talk) 20:01, 22 July 2019 (UTC)

Upload a new version of this file

I uploaded this video fragment with black stripes. Then I removed this black border at my computer.

Thus, I want upload to Commons new cropped version of this video fragment. How I can upload a new version of this file using vodeo2commons tool? --Andrew Krizhanovsky (talk) 19:16, 21 February 2019 (UTC)

Rename it --Zhuyifei1999 (talk) 20:05, 21 February 2019 (UTC)

OK. I uploaded new version of video (without black lines): File:Nikita Liyan-Kun Hubilai 1938 v2.webm. But I cannot move File:Nikita Liyan-Kun Hubilai 1938 v2.webm to File:Nikita Liyan-Kun Hubilai 1938.webm in order to replace the second file. What I should to do? Should I request the deletion of first file? --Andrew Krizhanovsky (talk) 20:50, 24 February 2019 (UTC)

Yeah, that's one way. You could also request a history merge which I've just done. Or you could ping an admin --Zhuyifei1999 (talk) 23:04, 24 February 2019 (UTC)

Great! Thank you! --Andrew Krizhanovsky (talk) 04:09, 25 February 2019 (UTC)

This section is resolved and can be archived. If you disagree, replace this template with your comment. --Roy17 (talk) 09:47, 4 August 2019 (UTC)

Errno 116

When I try to upload a video now, it says "Error: An exception occurred: IOError: [Errno 116] Stale file handle" before even skipping the first window. PseudoSkull (talk) 00:13, 30 May 2019 (UTC)

It's still happening, with every video attempt, even from multiple different attempted YT videos. Please help!!! @Zhuyifei1999: PseudoSkull (talk) 01:02, 30 May 2019 (UTC)

Restarted. Does that help? --Zhuyifei1999 (talk) 03:51, 30 May 2019 (UTC)

This section is resolved and can be archived. If you disagree, replace this template with your comment. --Roy17 (talk) 09:47, 4 August 2019 (UTC)

Not importing subtitles from youtube

I've used this for two youtube videos, which I am sure had subtitles, but they were not imported.

File:The Speaker Explains - Keeping Order in the House of Commons.webm from [1], subtitles in English (United Kingdom)
File:林鄭月娥 - 老公英戶籍是旅遊證件不會放棄居英權.webm from [2], subtitles in Chinese (Hong Kong) --Roy17 (talk) 18:25, 15 June 2019 (UTC)

I'll look into it this weekend hopefully --Zhuyifei1999 (talk) 15:10, 3 July 2019 (UTC)

In case you need something to run tests: https://www.youtube.com/results?search_query=TED&sp=EgQoATAB .--Roy17 (talk) 17:25, 3 July 2019 (UTC)

It seems to be failing https://github.com/toolforge/video2commons/blob/master/video2commons/backend/subtitles/__init__.py#L47. pycountry has no idea about 'en-gb'... --Zhuyifei1999 (talk) 01:16, 24 July 2019 (UTC)

Looks like https://github.com/LuminosoInsight/langcodes shall do. I'll get the patch once the server load is down. --Zhuyifei1999 (talk) 01:26, 24 July 2019 (UTC)

https://github.com/toolforge/video2commons/commit/616a7c5c18b52bd040876b56d0954050ff939f2c It works. TimedText:Zhuyifei1999_test_20190725.webm.en-gb.srt --Zhuyifei1999 (talk) 19:40, 25 July 2019 (UTC)

Thanks a lot! I've copied your test srt to where it should belong.--Roy17 (talk) 20:22, 25 July 2019 (UTC)

This section is resolved and can be archived. If you disagree, replace this template with your comment. --Roy17 (talk) 09:47, 4 August 2019 (UTC)

[BUG] file page was not created??

Tracked in Phabricator
Task T229605
RESOLVED

File:參議院亞太小組主席加德納批評北京侵蝕港人自由.webm, file uploaded but page gone?--Roy17 (talk) 17:48, 1 August 2019 (UTC)

@Zhuyifei1999: urgent! Not just me but also several others:

@Puramyun31 and Nederlandse Leeuw: please create the description pages manually.--Roy17 (talk) 18:07, 1 August 2019 (UTC)

MediaWiki should always create the page when the file is uploaded. If it's not then it is a MediaWiki issue. Can you report on phab? --Zhuyifei1999 (talk) 18:11, 1 August 2019 (UTC)

Manually creating file pages returns an error.--Roy17 (talk) 18:26, 1 August 2019 (UTC)

Done, thanks for the information and request. Nederlandse Leeuw (talk) 18:42, 1 August 2019 (UTC)

This section is resolved and can be archived. If you disagree, replace this template with your comment. --Roy17 (talk) 09:47, 4 August 2019 (UTC)

Is Twitter supported?

I tried https://twitter.com/VOAHK/status/970153299415482368 but it returned An exception occurred: DownloadError: ERROR: no suitable InfoExtractor for URL. Is this an upstream problem of youtube-dl, a problem of twitter, or of v2c?--Roy17 (talk) 13:26, 2 August 2019 (UTC)

v2c issue. I'll look into it this weekend. --Zhuyifei1999 (talk) 15:08, 2 August 2019 (UTC)

@Roy17: Should be fixed / workaround-ed. Try again? --Zhuyifei1999 (talk) 23:08, 3 August 2019 (UTC)

Many thanks for the quick debug!--Roy17 (talk) 09:47, 4 August 2019 (UTC)

This section is resolved and can be archived. If you disagree, replace this template with your comment. --Roy17 (talk) 09:47, 4 August 2019 (UTC)

It doesn't work

Video2commons doesn't work today. --ComputerHotline (talk) 06:02, 6 April 2019 (UTC)

Same here. I've tried several times since 5 April. Keeps pending. Aborting not possible either.--Roy17 (talk) 22:44, 6 April 2019 (UTC)

We have 3 conversion hosts, but apparently:

Host 1 was the operational host and started malfunctioning since April 4th, most likely related to NFS issues. I am unable to access the host and I've asked for a hard reboot
Host 2 left the pool since mid-Feburary, no idea why. I've remotely soft-rebooted the host and see if it'll repool itself.
Host 3 was shutoff since mid Feburary, no idea why either.

--Zhuyifei1999 (talk) 23:10, 6 April 2019 (UTC)

The second host is up, Hopefully pending tasks will be rescheduled to it within an hour or two, those that are stuck in 'Converting' is probably unrecoverable. :( --Zhuyifei1999 (talk) 23:13, 6 April 2019 (UTC)

Today (April 7th), it always shutdown :-( --ComputerHotline (talk) 04:55, 7 April 2019 (UTC)

This section is resolved and can be archived. If you disagree, replace this template with your comment. Roy17 (talk) 16:29, 16 December 2019 (UTC)

Replace pipe | with | instead of |

I have this suggestion, because the curly bracket method renders wrong attribution, see File:Desh and Parwin - A Punjabi Wedding in Singapore.webm. (It had been causing problems with LR script because LR template taking | would be malformed too, but that was fixed by changing the script.)--Roy17 (talk) 16:29, 16 December 2019 (UTC)

The relevant code is urlextract.py#L207. Would you create a PR? --Zhuyifei1999 (talk) 17:02, 16 December 2019 (UTC)

This section is resolved and can be archived. If you disagree, replace this template with your comment. Roy17 (talk) 15:19, 21 December 2019 (UTC)

Does the automatic upload summary contain private information?

For example, File:Ferrata Triglav.webm is an upload from personal devices, so the upload summary contains a code. Does this code contain any personal information such as IP or device model?--Roy17 (talk) 15:19, 21 December 2019 (UTC)

You mean "uploads:d3ff299e-23e6-11ea-bdb8-0242c0a88a0b"? It's a ~~randomly-generated~~ date-time (of the initial chunk upload processing) + MAC address (of the server) en:UUID that identifies the file upload session. --Zhuyifei1999 (talk) 19:34, 21 December 2019 (UTC)

This section is resolved and can be archived. If you disagree, replace this template with your comment. Roy17 (talk) 01:12, 11 February 2020 (UTC)

Language

I translated Video2commons to Arabic on translatewiki.net but why is there no option to change the language as in flickr2commons --بدارين-bdareen (talk) 17:39, 29 August 2016 (UTC)

@بدارين: It is using your language settings from the wiki. Best --Steinsplitter (talk) 17:50, 29 August 2016 (UTC)

Thank you

This tool is fantastic. Thank you! Legoktm (talk) 07:24, 28 January 2017 (UTC)

You're welcome ;) --Zhuyifei1999 (talk) 09:53, 28 January 2017 (UTC)

DownloadError: ERROR: content too short

Hi, On a few occasions I've been getting the following error: "An exception occurred: DownloadError: ERROR: content too short (expected 113850282 bytes and served 109899264)"
Not sure if it's something my end or a technical issue with the tool,
Thanks, –Davey2010^Talk 17:34, 10 March 2017 (UTC)

This is an long-standing issue upstream. Idk what to do about it; suggestions welcome. For now, retrying a few times usually can get the download completed. --Zhuyifei1999 (talk) 04:50, 11 March 2017 (UTC)

Hi Zhuyifei1999, Oops sorry I never gave it a thought to look at Github, Ah sorry I wouldn't know the first thing about software etc etc, I only need to restart once and it works no problem so it's not really an issue or anything but as I said I wasn't sure if it was something my end, Anyway thanks for replying, Have a great weekend. –Davey2010^Talk 13:10, 11 March 2017 (UTC)

TaskError: pywikibot.Error

Hi, I'm trying to upload a file, I retry three times and I always obtained the same problem: "An exception occurred: TaskError: pywikibot.Error: UploadWarning: exists-normalized: File exists with different extension as "00005.MTS.Still001.jpg"". Could somebody help me? Bye, --Elisardojm (talk) 18:32, 3 April 2017 (UTC)

@Elisardojm: use a different file name. The warning basically means there are a file named File:00005.MTS.Still001.jpg and MediaWiki gives an upload warning. The current coding of v2c is that any upload warning halts the entire upload. Not sure about a better way to resolve this :/ --Zhuyifei1999 (talk) 08:02, 4 April 2017 (UTC)

Thanks! Bye, --Elisardojm (talk) 23:03, 17 April 2017 (UTC)

Can you add a possibility to edit the file name to upload to in the interface? I don't have my source file anymore and get this error so I cannot upload it again from scratch. Or if you just could help me one time backend with "Organizing Translation Workshops" that would be helpful. Ainali (talk) 18:41, 23 August 2019 (UTC)

@Ainali: https://tools.wmflabs.org/video2commons/static/uploads/85d4b0ce-c57f-11e9-ad4b-0242c0a8b505 --Zhuyifei1999 (talk) 21:07, 23 August 2019 (UTC)

Thanks! it took a few tries to manage to download the file, but now it's up on Commons! Ainali (talk) 08:48, 28 August 2019 (UTC)

Quality options

Hi, I have a question, I have uploaded the file File:Punta Socastro, Viveiro.webm, the file in Commons is a 8 min 55 s, 1.920 × 1.080 (2,18 GB) webm file, but it's quality it's worse than the original file. The original file has same time and resolution, but it only has 1,02 GB in an MTS file. I don't understand how a greater file has worse quality than an smaller file. Could it be a problem of the options that I choose at upload time? Bye, --Elisardojm (talk) 19:09, 7 May 2017 (UTC)

There are two issues there:

The file bloat up in size -- The video is interlaced. Trying to encode the interlacing as if the interlace does not exist require huge amount of data, when v2c is trying to preserve the quality of original video. The video should be deinterlaced before re-encoding.
Apparent decrease in quality -- What you see on the player in File:Punta Socastro, Viveiro.webm is transcoded again by WMF videoscalers, which will show the scaled videos with smaller resolution when the higher resolution ones has not finished transcoding on WMF videoscalers. Do you see any loss in quality in the original v2c upload? I personally can't find any.

In any case, I'll deinterlace the video to File:Punta Socastro, Viveiro (deinterlaced).webm --Zhuyifei1999 (talk) 06:39, 8 May 2017 (UTC)

Yes, I have seen the original v2c upload now, and I compared my original file with the uploaded file and it has loss of quality. The original file is an 16181kbps file, and I have seen that the uploaded only has 5,52 Mbps. Could be that the problem? Bye, --Elisardojm (talk) 20:39, 9 May 2017 (UTC)

Not numerically, but if the quality loss is visible. Anyhow, if the video is interlaced, v2c will have a hard time transcoding. Please deinterlace them first. --Zhuyifei1999 (talk) 05:53, 10 May 2017 (UTC)

Zhuyifei1999, if I deinterlace the video, will it increase the quality? How can I deinterlace the videos? Bye, --Elisardojm (talk) 11:25, 16 May 2017 (UTC)

@Elisardojm: No, all of interlacing, deinterlacing, and transcoding is lossy and lose quality. Deinterlacing will make a fast-moving video more watchable (File:Punta Socastro, Viveiro (deinterlaced).webm vs File:Punta Socastro, Viveiro.webm) and reduce the complexity on transcoding, which may or may not increase the quality of transcoded video. As for how to deinterlace, I think Google is better than I am to answer this question. I personally use FFmpeg's yadif filter --Zhuyifei1999 (talk) 11:37, 19 May 2017 (UTC)

And Zhuyifei1999, is there any other method for upload videos with more quality to Commons? Bye, --Elisardojm (talk) 11:43, 4 June 2017 (UTC)

Well, you're always welcome to do the transcoding yourself with max quality settings. Beware that you may end up with very gigantic files --Zhuyifei1999 (talk) 11:45, 4 June 2017 (UTC)

And this great tool ;)..., couldn't has it an option to select the quality? O:) Bye, --Elisardojm (talk) 11:58, 4 June 2017 (UTC)

I honestly don't know how that option will be useful to most people. It's like "pick a number between 0 and 63", if you choose 0 (highest quality) the file will probably be too large to be uploaded and task insanely long time to transcode, and if you choose 63 (lowest quality) the video is unwatchable. --Zhuyifei1999 (talk) 12:22, 4 June 2017 (UTC)

Anyways, I still don't see any visible quality loss during v2c's transcoding process --Zhuyifei1999 (talk) 12:24, 4 June 2017 (UTC)

I'm comparing the video with my original file in MTS extension, and it has more quality than those, and it has 1,2GB vs 1,27 GB... Bye, --Elisardojm (talk) 13:32, 4 June 2017 (UTC)

Is it possible to give the tool more than one URL to do batch uploads

Hi

I really like the tool, is it possible to use it to copy several files across at a time? Something like Flickr2Commons can do. I realise this may be quite slow if the files are very large but would be very useful for transfering smaller files like audio.

Thanks

--John Cummings (talk) 14:21, 12 July 2017 (UTC)

@John Cummings: The relevant ticket is #48. It's not too hard to implement this, but some stuffs to consider:

Anyone can overload the servers to an extreme extent. The servers can only reliably process around 48 tasks in parallel without overloading at maximum capacity. The queue is shared (and I don't see a way to do per-user isolated queue), and tasks submitted when the servers get overloaded make the running tasks even slower, until the servers give up (when running between 48 and 96 tasks in parallel at maximum capacity) and the new tasks start to pop in queue. What's worse is that transcodes often take hours. Someone once started, IIRC, around 200 tasks and I was forced to abort them to let new tasks from other users to start to be processed.
Perhaps I should just AGF? The backend can perhaps try to download the uploadable-and-highres version (usually from YouTube) when available instead of highres-but-must-be-transcode version, but I never figured out how to explain "uploadable formats" to youtube-dl (the library that does the selection and downloading).
The current workflow is have the user to enter a url and allow them to edit the filename and file description before upload. In batched uploads there could be dozens or hundreds of files. How should the user be able to access / edit their filenames and file descriptions? Or should there be an alternative workflow?

If these are resolved I think it'll be straightforward to implement batch uploads.

--Zhuyifei1999 (talk) 19:29, 12 July 2017 (UTC)

BTW: The tool can prefill the source url if the tool is accessed with something like https://tools.wmflabs.org/video2commons/#!https://www.youtube.com/watch?v=2h7wR5ScQaM. If you are doing mass imports you can create a sandbox page with all the urls and open each one in new tab for prefilling, but it's quite a waste of browser resources :/. See issue #82 --Zhuyifei1999 (talk) 19:42, 12 July 2017 (UTC)

@Zhuyifei1999: thanks very much for your thoughtful replies, very glad to know you're on top of things :) --John Cummings (talk) 09:57, 13 July 2017 (UTC)

Well, I'd really like it if these issues can be resolved --Zhuyifei1999 (talk) 17:34, 13 July 2017 (UTC)

blacklist channel

Hi, Is there a blacklist for YouTube channels that upload content under a CC-license that isn't theirs? this channel would need to be added. --Vera ^(talk) 15:55, 30 October 2017 (UTC)

We don't have a blacklist now, nor do we prevent people from processing videos/audios that aren't explicitly labeled CC-BY. Please see User_talk:Zhuyifei1999/Archive_27#video2commons --Zhuyifei1999 (talk) 19:25, 30 October 2017 (UTC)

the UploadWizard has a blacklist of flickr accounts that upload pictures under a Creative Commons license that definitely isn't theirs. User:FlickreviewR/bad-authors Vera ^(talk) 20:45, 30 October 2017 (UTC)

You're free to submit a pull request. I personally do not have the time nor the willingness to implement this, for the reasons explained in the link above. --Zhuyifei1999 (talk) 21:25, 30 October 2017 (UTC)

WebM (VP9/Opus)

Hello.

Why this parameter is "experimental" ?

It works correctly.

--ComputerHotline (talk) 07:02, 9 March 2018 (UTC)

It sometimes fails due to issue #15 --Zhuyifei1999 (talk) 17:27, 9 March 2018 (UTC)

I've found why there is this error : video2commons use these commands lines to convert a video file :


ffmpeg -y -i v.mkv -threads 0 -skip_threshold 0 -bufsize 6000k -rc_init_occupancy 4000 -qmin 19 -qmax 19 -vcodec libvpx-vp9 -tile-columns 4 -f webm -ss 0 -an -pass 1 -passlogfile tmp\log.log tmp\null

ffmpeg -y -i v.mkv -threads 0 -skip_threshold 0 -bufsize 6000k -rc_init_occupancy 4000 -qmin 19 -qmax 19 -vcodec libvpx-vp9 -tile-columns 4 -f webm -ss 0 -acodec copy -pass 2 -passlogfile tmp\log.log v.webm

Or, in the audio codec, Opus may be used :


ffmpeg -y -i v.mkv -threads 0 -skip_threshold 0 -bufsize 6000k -rc_init_occupancy 4000 -qmin 19 -qmax 19 -vcodec libvpx-vp9 -tile-columns 4 -f webm -ss 0 -an -pass 1 -passlogfile tmp\log.log tmp\null

ffmpeg -y -i v.mkv -threads 0 -skip_threshold 0 -bufsize 6000k -rc_init_occupancy 4000 -qmin 19 -qmax 19 -vcodec libvpx-vp9 -tile-columns 4 -f webm -ss 0 -acodec libopus -pass 2 -passlogfile tmp\log.log v.webm

--ComputerHotline (talk) 18:56, 13 March 2018 (UTC)

I wish it is that simple. Messiest code ever and I just never get the interest to rewrite it. Feel free to submit a pull request, however. --Zhuyifei1999 (talk) 15:08, 14 March 2018 (UTC)

I don't want to register on github. I think someone could make it. --ComputerHotline (talk) 18:51, 14 March 2018 (UTC)

Support for Google Photos

I know that YouTube is supported, however I would like to upload videos directly from Google Photos so that I don't have to download the video and upload it. I am not sure what format Google Photos uses, however I am pretty sure it is MP4. — Preceding unsigned comment added by Atomicdragon136 (talk • contribs) 00:09, 13 April 2018 (UTC)

@Atomicdragon136: Do you have an example link? --Zhuyifei1999 (talk) 22:05, 25 April 2018 (UTC)

Here's an example video that is being stored using Google Photos Atomicdragon136 (talk) 22:20, 26 April 2018 (UTC)

I see the relevant upstream support request: https://github.com/rg3/youtube-dl/issues/13889. Gotta wait till it's supported upstream. --Zhuyifei1999 (talk) 16:18, 27 April 2018 (UTC)

Is there a way to pre-fill the description of the file?

I am organizing a video section in WLM Romania this year and I was thinking about sending people to this tool so that they don't need to worry about formats and so on. I was wondering if there is a way to pre-fill the {{Information}} template like you can do it for the UploadWizard. Thanks.--Strainu (talk) 21:29, 18 August 2018 (UTC)

Right now no. How is the UW prefilled? --Zhuyifei1999 (talk) 04:49, 19 August 2018 (UTC)

Zhuyifei1999: by query parameters, here is an example: [3].--Strainu (talk) 22:28, 19 August 2018 (UTC)

Interesting. Right now it's only able to prefill the URL. For v2c to prefill the {{Information}} template the entire file description page has to be provided. Is that okay? --Zhuyifei1999 (talk) 04:59, 20 August 2018 (UTC)

Works for me, just as long as it's a get request (I'm calling it from wiki).--Strainu (talk) 14:18, 20 August 2018 (UTC)

Done Eg. https://tools.wmflabs.org/video2commons/?url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DC0DPdy98e4c&filename=Filename&filedesc=%7B%7BInformation%7D%7D&format=webm+(VP9%2FOpus%2C+experimental) The query string will take precedence over whatever is provided by the backend during the course of the form (so until the job is submitted). --Zhuyifei1999 (talk) 08:09, 21 August 2018 (UTC)

Thanks Zhuyifei1999, I've implemented this for WLMRO, e.g. ro:Lista_monumentelor_istorice_din_județul_Neamț. I have one more small request if possible: can you recover the data after authentication? If I am logged in everything works as intended, but if I'm not, after login the task is not resumed and I have to go back to the wiki to restart the process. Thanks a lot!--Strainu (talk) 21:16, 27 August 2018 (UTC)

Done. (With lots of trial and error) --Zhuyifei1999 (talk) 13:59, 28 August 2018 (UTC)

Bugs & feature requests from WLMRO

I'm going to gather here all the feedback I receive from users participating in WLM Romania. If you would rather have these in github, I can do that as well.

4K videos are downsized to 1080p.
if the uploader uses an already used filename, an error is recieved, but the user does not have the possibility to edit the name/description before retrying. This is probably true for any MediaWiki error.--Strainu (talk) 18:13, 2 September 2018 (UTC)

@Strainu: Sorry for not seeing this.

I don't think we intentionally downsize 4Ks to 1080p. Do you have a test case for this?
For any MediaWiki warnings, the tool just gives up. There are plans (#68 & #81) to make this somewhat better user experience, but they are so hard to write :(

--Zhuyifei1999 (talk) 16:29, 21 October 2018 (UTC)

I don't have any 4K videos and I know the user was uploading from his own computer, but you might try with an YouTube video perhaps?--Strainu (talk) 17:19, 21 October 2018 (UTC)

Well, I know it used to work. I don't have any 4k CC-BY videos off of my head right now. --Zhuyifei1999 (talk) 22:37, 21 October 2018 (UTC)

Panoramic video is messed up

Please take a look at File:Coriolis on the Playground (360VR).webm and at its source. I am unsure what is messed up exactly, video2commons or Commons as a whole, but reporting to the most specific group, hopefully this is the correct approach. ℺ Gone Postal (〠 ✉ • ✍ ⏿) 15:02, 5 November 2018 (UTC)

I think there is probably some lack of support for 360 videos somewhere. Most likely YouTube player has its own plugin to render 360 videos, but Commons don't, and v2c don't have special treatment for 360s during transcoding. --Zhuyifei1999 (talk) 07:28, 14 November 2018 (UTC)

Multiple selection

Hi, Is there a way you can select multiple files so you can upload like 50 files at once instead of one file ?, Thanks, –Davey2010^Talk 21:07, 13 November 2018 (UTC)

It would be nice to be able to upload whole channel. ℺ Gone Postal (〠 ✉ • ✍ ⏿) 05:03, 14 November 2018 (UTC)
Umm... #Is_it_possible_to_give_the_tool_more_than_one_URL_to_do_batch_uploads & Issue #48 --Zhuyifei1999 (talk) 07:31, 14 November 2018 (UTC)

Hi Zhuyifei1999, Funny enough I did think servers/overload were or could be an issue,

There's currently a thread at [[Commons:Village_pump#[Update]_Free_Music_Archive]] inregards to transferring music files hence this question really

Okie dokie not a problem , Thanks for your help. –Davey2010^Talk 14:51, 14 November 2018 (UTC)

Error uploading file

Hi, I'm trying to upload a file but I always get the same error: "TaskError: pywikibot.Error: APIError: chunk-too-small: Minimum chunk size is 1,024 bytes for non-final chunks." Sombody knows whats the problem? Bye, Elisardojm (talk) 12:18, 2 December 2018 (UTC)

Old bug in pywikibot, no one knows how to fix: phab:T132676 --Zhuyifei1999 (talk) 16:43, 4 December 2018 (UTC)

Then Zhuyifei1999, how could I upload the file? Cutting it a little and trying again? Bye, --Elisardojm (talk) 18:33, 4 December 2018 (UTC)

That should work --Zhuyifei1999 (talk) 05:10, 5 December 2018 (UTC)

Uploading errors--I need help

I tried uploading 116th United States Congress House Floor - 2019-01-10.webm and now I am getting the error message "An exception occurred: TaskError: File already exists. Please choose another name." I have several more files to upload and I fear I will get this over and over again. Can someone help me, please? —Justin (koavf)❤T☮C☺M☯ 17:40, 29 January 2019 (UTC)

@Koavf: Don't create file description pages manually. video2commons currently refuse to reupload if page exists previously, and this might even apply even if the page is deleted. Could you change the file name on v2c and run again? I can move the files over existing description pages if you want. --Zhuyifei1999 (talk) 19:21, 29 January 2019 (UTC)

Okay, I don't think I can rename on v2c's end. I am presently uploading 116th United States Congress House Floor - 2019-01-16.webm now and 116th United States Congress House Floor - 2019-01-10.webm is still stuck. Can you please overwrite? And can you also delete 116th United States Congress House Floor - 2019-01-17.webm, 116th United States Congress House Floor - 2019-01-24.webm, and 116th United States Congress House Floor - 2019-01-25.webm if they are just going to fail as uploads? —Justin (koavf)❤T☮C☺M☯ 19:23, 29 January 2019 (UTC)

There's a title for you to change on the filename input. I cannot change the task data for you (too complicated) --Zhuyifei1999 (talk) 20:08, 29 January 2019 (UTC)

I don't see any such option. —Justin (koavf)❤T☮C☺M☯ 22:08, 29 January 2019 (UTC)

@Koavf: In the first page, you should have 4 inputs, one 'URL' text input and 3 'Kepp...' checkboxes. In the second page, you should have 3 inputs, two for 'Filename', a text input and a dropdown for the format, and a textbox for the file description page. In the third page, you get a confirmation page with 0 inputs. The input I was talking about is on the second page, how did you miss it? --Zhuyifei1999 (talk) 22:31, 29 January 2019 (UTC)

I have no clue what you are talking about. I go to the URI https://tools.wmflabs.org/video2commons/ and I see

Your tasks:

116th United States Congress House Floor - 2019-01-10 An exception occurred: TaskError: File already exists. Please choose another name. [Remove]

116th United States Congress House Floor - 2019-01-16 VP8 File too large to upload directly! You may want to request a server-side upload. [Remove]

116th United States Congress House Floor - 2019-01-10 An exception occurred: TaskError: File already exists. Please choose another name. [Restart][Remove]

There is no fourth page or dropdown anywhere. Where are you describing these resources as being on the Web? —Justin (koavf)❤T☮C☺M☯ 23:06, 29 January 2019 (UTC)

@Koavf: There should be a modal window when you create the task. Once the task is created the arguments cannot be modified. (There should ideally be a way to modify that after submission but it's so complex I gave up after several coding attempts) --Zhuyifei1999 (talk) 00:35, 30 January 2019 (UTC)

Yes, there were options initially but not now. Can you assist me getting these files moved from the server onto Commons proper? —Justin (koavf)❤T☮C☺M☯ 00:39, 30 January 2019 (UTC)

Could you clarify the assistance you need? I could, for example, manually, fetch the file description or the file you uploaded as long as the task is still visible. The transcoded file itself is removed as soon as the task fails, unless it's kept for server-side upload, in which case it's kept for 60 more days.

The easiest way I'd say is to start a new task changing the initial parameters. --Zhuyifei1999 (talk) 00:46, 30 January 2019 (UTC)

I'm sorry that this is somehow unclear but: 1.) I uploaded this file to the wmflab server using v2c. 2.) I want it to be on Commons at the name I suggested. If just deleting the file description page will allow me to do that, then please delete the file description page. I cannot start a new task, as I no longer have the original file--I deleted it once it was uploaded. This is all a very drawn-out and iterative process that frankly seems like it has way too much overhead. I'm editing the page to point out the problem with being unable to upload if a description page exists but that is really helpful information to have before you spend hours downloading, converting, and then uploading a file. —Justin (koavf)❤T☮C☺M☯ 19:11, 30 January 2019 (UTC)

I can fetch the files that was uploaded to v2c from the storage as long as the task is visible. I'm working on it. --Zhuyifei1999 (talk) 20:35, 30 January 2019 (UTC)

Thank you very much for your effort. —Justin (koavf)❤T☮C☺M☯ 20:48, 30 January 2019 (UTC)

116th United States Congress House Floor - 2019-01-10 is on https://tools.wmflabs.org/video2commons/static/uploads/71b206f0-235e-11e9-b509-0242c0a8f603. You can use that directly as a source url. The given file description is

=={{int:filedesc}}==
{{Information
|description={{en|1=116th United States Congress House Floor, 2019-01-10}}
|date=2019-01-10
|source=http://houselive.gov/ViewPublisher.php?view_id=14
|author={{Creator:116th United States Congress}}
|permission=
|other_versions=
|other_fields=
}}

=={{int:license-header}}==
{{PD-USGov}}

[[Category:Videos of the 116th United States Congress]]
[[Category:Videos in English]]
[[Category:United States House of Representatives]]
[[Category:Videos from the House Floor]]
[[Category:2019-01-10]]

I don't think deleting the file description page can help; if you need this exact name, you can upload and move over it. I'm sorry if the workflow is not as smooth as it can be. It's part of a very old issue that is so badly difficult to fix --Zhuyifei1999 (talk) 20:50, 30 January 2019 (UTC)

Exitcode: 137

I get return code 137 on transcoding several different files after a few hours in the pass 2. For example:

An exception occurred: TaskError: /mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/a7bf1e154486d334/dl.unknown_video -threads 16 -row-mt 1 -qmin 19 -qmax 19 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -ab 128000 -ar 48000 -acodec libopus -pass 2 -passlogfile /srv/v2c/output/a7bf1e154486d334/dl.unknown_video.vp9.webm.log /srv/v2c/output/a7bf1e154486d334/dl.unknown_video.vp9.webm Exitcode: 137

Is there any quota on the time/resources which can cause ffmpeg process to be killed? -- Meisam (talk) 08:15, 24 April 2019 (UTC)

Exitcode 137 is SIGKILL, which normally is an out of memory condition, but I checked the logs and do not see a log entry regarding out of memory. The limits are defined in https://github.com/toolforge/video2commons/blob/master/video2commons/backend/encode/globals.py --Zhuyifei1999 (talk) 20:52, 24 April 2019 (UTC)

@Zhuyifei1999: I don't think that I'm hitting these limits. The files are 0.75-1.5GB each. Can you please check any of these logs to see what is happening here? All of them exit with the same Exitcode: 137

/srv/v2c/output/e52152d54a0f0293/dl.unknown_video.vp9.webm.log
/srv/v2c/output/1ff5fdf911179b4b/dl.unknown_video.vp9.webm.log 
/srv/v2c/output/9742ef267a12e893/dl.unknown_video.vp9.webm.log
/srv/v2c/output/78062dc7c4183098/dl.unknown_video.vp9.webm.log 
/srv/v2c/output/81c6bae4c627c0bb/dl.unknown_video.vp9.webm.log

Many thanks! -- Meisam (talk) 21:09, 24 April 2019 (UTC)

The Logs doesn't have anything useful.

v2ccelery:

[2019-04-24 00:52:57,703: WARNING/PoolWorker-10] 38: Running cmd: /mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/e52152d54a0f0293/dl.unknown_vide
o -threads 16 -row-mt 1 -qmin 19 -qmax 19 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -ab 128000 -ar 48000 -acodec libopus -pass 2 -passlogfile /s
rv/v2c/output/e52152d54a0f0293/dl.unknown_video.vp9.webm.log /srv/v2c/output/e52152d54a0f0293/dl.unknown_video.vp9.webm
[2019-04-24 00:59:22,841: WARNING/PoolWorker-6] 62: Running cmd: /mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/1ff5fdf911179b4b/dl.unknown_video
 -threads 16 -row-mt 1 -qmin 19 -qmax 19 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -ab 128000 -ar 48000 -acodec libopus -pass 2 -passlogfile /sr
v/v2c/output/1ff5fdf911179b4b/dl.unknown_video.vp9.webm.log /srv/v2c/output/1ff5fdf911179b4b/dl.unknown_video.vp9.webm
[2019-04-24 01:05:39,433: WARNING/PoolWorker-6] 63: Running cmd: /mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/1ff5fdf911179b4b/dl.unknown_video
 -threads 16 -row-mt 1 -qmin 19 -qmax 19 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -ab 128000 -ar 48000 -acodec libopus -pass 2 -passlogfile /sr
v/v2c/output/1ff5fdf911179b4b/dl.unknown_video.vp9.webm.log /srv/v2c/output/1ff5fdf911179b4b/dl.unknown_video.vp9.webm
[2019-04-24 01:06:26,447: WARNING/PoolWorker-10] -1: Cleaning up...
[2019-04-24 01:06:26,628: ERROR/PoolWorker-10] Task video2commons.backend.worker.main[c31cf24c-53a6-4386-a9c7-79e7ab45a4ce] raised unexpected: TaskError('/mnt/nfs/labstore-secondary-pro
ject/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/e52152d54a0f0293/dl.unknown_video -threads 16 -row-mt 1 -qmin 19 -qmax 19 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag
-in-frames 25 -f webm -ss 0 -ab 128000 -ar 48000 -acodec libopus -pass 2 -passlogfile /srv/v2c/output/e52152d54a0f0293/dl.unknown_video.vp9.webm.log /srv/v2c/output/e52152d54a0f0293/dl.
unknown_video.vp9.webm\nExitcode: 137',)
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/celery/app/trace.py", line 367, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/celery/app/trace.py", line 622, in __protected_call__
    return self.run(*args, **kwargs)
  File "/srv/v2c/video2commons/backend/worker.py", line 123, in main
    file = encode.encode(file, convertkey, statuscallback, errorcallback)
  File "/srv/v2c/video2commons/backend/encode/__init__.py", line 51, in encode
    return target if job.run() else None
  File "/srv/v2c/video2commons/backend/encode/transcodejob.py", line 181, in run
    self.set_error(status, transcode_key)
  File "/srv/v2c/video2commons/backend/encode/transcodejob.py", line 115, in set_error
    self.errorcallback(error)
  File "/srv/v2c/video2commons/backend/worker.py", line 107, in errorcallback
    raise TaskError(text)
TaskError: /mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/e52152d54a0f0293/dl.unknown_video -threads 16 -row-mt 1 -qmin 19 -qmax 19 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -f webm -ss 0 -ab 128000 -ar 48000 -acodec libopus -pass 2 -passlogfile /srv/v2c/output/e52152d54a0f0293/dl.unknown_video.vp9.webm.log /srv/v2c/output/e52152d54a0f0293/dl.unknown_video.vp9.webm
Exitcode: 137

journal:

Extended content

Apr 24 00:52:07 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:52:07 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:52:11 encoding01 sudo[3623]: root : TTY=unknown ; PWD=/ ; USER=puppet ; COMMAND=list /bin/cat /var/lib/puppet/state/last_run_summary.yaml
Apr 24 00:52:11 encoding01 sudo[3624]: root : TTY=unknown ; PWD=/ ; USER=puppet ; COMMAND=/bin/cat /var/lib/puppet/state/last_run_summary.yaml
Apr 24 00:52:11 encoding01 sudo[3624]: pam_unix(sudo:session): session opened for user puppet by (uid=0)
Apr 24 00:52:11 encoding01 sudo[3624]: pam_unix(sudo:session): session closed for user puppet
Apr 24 00:52:17 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:52:17 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:52:27 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:52:27 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:52:37 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:52:37 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:52:47 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:52:47 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:52:57 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:52:57 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:53:07 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:53:07 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:53:11 encoding01 sudo[3633]: root : TTY=unknown ; PWD=/ ; USER=puppet ; COMMAND=list /bin/cat /var/lib/puppet/state/last_run_summary.yaml
Apr 24 00:53:11 encoding01 sudo[3634]: root : TTY=unknown ; PWD=/ ; USER=puppet ; COMMAND=/bin/cat /var/lib/puppet/state/last_run_summary.yaml
Apr 24 00:53:11 encoding01 sudo[3634]: pam_unix(sudo:session): session opened for user puppet by (uid=0)
Apr 24 00:53:11 encoding01 sudo[3634]: pam_unix(sudo:session): session closed for user puppet
Apr 24 00:53:17 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:53:17 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:53:27 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:53:27 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:53:37 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:53:37 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:53:47 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:53:47 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:53:57 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:53:57 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:54:07 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:54:07 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:54:11 encoding01 sudo[3643]: root : TTY=unknown ; PWD=/ ; USER=puppet ; COMMAND=list /bin/cat /var/lib/puppet/state/last_run_summary.yaml
Apr 24 00:54:11 encoding01 sudo[3644]: root : TTY=unknown ; PWD=/ ; USER=puppet ; COMMAND=/bin/cat /var/lib/puppet/state/last_run_summary.yaml
Apr 24 00:54:11 encoding01 sudo[3644]: pam_unix(sudo:session): session opened for user puppet by (uid=0)
Apr 24 00:54:11 encoding01 sudo[3644]: pam_unix(sudo:session): session closed for user puppet
Apr 24 00:54:17 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:54:17 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:54:27 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:54:27 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:54:37 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:54:37 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:54:47 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:54:47 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:54:57 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:54:57 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:55:01 encoding01 CRON[3652]: pam_unix(cron:session): session opened for user root by (uid=0)
Apr 24 00:55:01 encoding01 CRON[3653]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Apr 24 00:55:01 encoding01 CRON[3652]: pam_unix(cron:session): session closed for user root
Apr 24 00:55:06 encoding01 systemd-timesyncd[431]: interval/delta/delay/jitter/drift 2048s/+0.001s/0.002s/0.002s/+10ppm
Apr 24 00:55:07 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:55:07 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:55:11 encoding01 sudo[3656]: root : TTY=unknown ; PWD=/ ; USER=puppet ; COMMAND=list /bin/cat /var/lib/puppet/state/last_run_summary.yaml
Apr 24 00:55:11 encoding01 sudo[3657]: root : TTY=unknown ; PWD=/ ; USER=puppet ; COMMAND=/bin/cat /var/lib/puppet/state/last_run_summary.yaml
Apr 24 00:55:11 encoding01 sudo[3657]: pam_unix(sudo:session): session opened for user puppet by (uid=0)
Apr 24 00:55:11 encoding01 sudo[3657]: pam_unix(sudo:session): session closed for user puppet
Apr 24 00:55:17 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:55:17 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:55:27 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:55:27 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:55:37 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:55:37 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:55:47 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:55:47 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:55:57 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:55:57 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:56:07 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:56:07 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:56:11 encoding01 sudo[3666]: root : TTY=unknown ; PWD=/ ; USER=puppet ; COMMAND=list /bin/cat /var/lib/puppet/state/last_run_summary.yaml
Apr 24 00:56:11 encoding01 sudo[3667]: root : TTY=unknown ; PWD=/ ; USER=puppet ; COMMAND=/bin/cat /var/lib/puppet/state/last_run_summary.yaml
Apr 24 00:56:11 encoding01 sudo[3667]: pam_unix(sudo:session): session opened for user puppet by (uid=0)
Apr 24 00:56:11 encoding01 sudo[3667]: pam_unix(sudo:session): session closed for user puppet
Apr 24 00:56:17 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:56:17 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:56:27 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:56:27 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:56:37 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:56:37 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:56:47 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:56:47 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:56:57 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:56:57 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:57:07 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:57:07 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:57:11 encoding01 sudo[3676]: root : TTY=unknown ; PWD=/ ; USER=puppet ; COMMAND=list /bin/cat /var/lib/puppet/state/last_run_summary.yaml
Apr 24 00:57:11 encoding01 sudo[3677]: root : TTY=unknown ; PWD=/ ; USER=puppet ; COMMAND=/bin/cat /var/lib/puppet/state/last_run_summary.yaml
Apr 24 00:57:11 encoding01 sudo[3677]: pam_unix(sudo:session): session opened for user puppet by (uid=0)
Apr 24 00:57:11 encoding01 sudo[3677]: pam_unix(sudo:session): session closed for user puppet
Apr 24 00:57:17 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:57:17 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:57:27 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:57:27 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:57:37 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:57:37 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:57:47 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:57:47 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:57:57 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:57:57 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:58:07 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:58:07 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:58:11 encoding01 sudo[3688]: root : TTY=unknown ; PWD=/ ; USER=puppet ; COMMAND=list /bin/cat /var/lib/puppet/state/last_run_summary.yaml
Apr 24 00:58:11 encoding01 sudo[3689]: root : TTY=unknown ; PWD=/ ; USER=puppet ; COMMAND=/bin/cat /var/lib/puppet/state/last_run_summary.yaml
Apr 24 00:58:11 encoding01 sudo[3689]: pam_unix(sudo:session): session opened for user puppet by (uid=0)
Apr 24 00:58:11 encoding01 sudo[3689]: pam_unix(sudo:session): session closed for user puppet
Apr 24 00:58:18 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:58:18 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:58:28 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:58:28 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:58:38 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:58:38 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:58:48 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:58:48 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:58:58 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:58:58 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:59:08 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:59:08 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:59:11 encoding01 sudo[3698]: root : TTY=unknown ; PWD=/ ; USER=puppet ; COMMAND=list /bin/cat /var/lib/puppet/state/last_run_summary.yaml
Apr 24 00:59:11 encoding01 sudo[3699]: root : TTY=unknown ; PWD=/ ; USER=puppet ; COMMAND=/bin/cat /var/lib/puppet/state/last_run_summary.yaml
Apr 24 00:59:11 encoding01 sudo[3699]: pam_unix(sudo:session): session opened for user puppet by (uid=0)
Apr 24 00:59:11 encoding01 sudo[3699]: pam_unix(sudo:session): session closed for user puppet
Apr 24 00:59:18 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:59:18 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:59:28 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:59:28 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:59:38 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:59:38 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:59:48 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:59:48 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:59:58 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 00:59:58 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:00:08 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:00:08 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:00:11 encoding01 sudo[3708]: root : TTY=unknown ; PWD=/ ; USER=puppet ; COMMAND=list /bin/cat /var/lib/puppet/state/last_run_summary.yaml
Apr 24 01:00:11 encoding01 sudo[3709]: root : TTY=unknown ; PWD=/ ; USER=puppet ; COMMAND=/bin/cat /var/lib/puppet/state/last_run_summary.yaml
Apr 24 01:00:11 encoding01 sudo[3709]: pam_unix(sudo:session): session opened for user puppet by (uid=0)
Apr 24 01:00:11 encoding01 sudo[3709]: pam_unix(sudo:session): session closed for user puppet
Apr 24 01:00:18 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:00:18 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:00:28 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:00:28 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:00:38 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:00:38 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:00:48 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:00:48 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:00:58 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:00:58 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:01:08 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:01:08 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:01:11 encoding01 sudo[3718]: root : TTY=unknown ; PWD=/ ; USER=puppet ; COMMAND=list /bin/cat /var/lib/puppet/state/last_run_summary.yaml
Apr 24 01:01:11 encoding01 sudo[3719]: root : TTY=unknown ; PWD=/ ; USER=puppet ; COMMAND=/bin/cat /var/lib/puppet/state/last_run_summary.yaml
Apr 24 01:01:11 encoding01 sudo[3719]: pam_unix(sudo:session): session opened for user puppet by (uid=0)
Apr 24 01:01:11 encoding01 sudo[3719]: pam_unix(sudo:session): session closed for user puppet
Apr 24 01:01:18 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:01:18 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:01:28 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:01:28 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:01:38 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:01:38 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:01:48 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:01:48 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:01:50 encoding01 systemd[1]: Starting Collect apt metrics for prometheus-node-exporter...
Apr 24 01:01:51 encoding01 systemd[1]: Started Collect apt metrics for prometheus-node-exporter.
Apr 24 01:01:58 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:01:58 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:02:08 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:02:08 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:02:11 encoding01 sudo[3742]: root : TTY=unknown ; PWD=/ ; USER=puppet ; COMMAND=list /bin/cat /var/lib/puppet/state/last_run_summary.yaml
Apr 24 01:02:11 encoding01 sudo[3743]: root : TTY=unknown ; PWD=/ ; USER=puppet ; COMMAND=/bin/cat /var/lib/puppet/state/last_run_summary.yaml
Apr 24 01:02:11 encoding01 sudo[3743]: pam_unix(sudo:session): session opened for user puppet by (uid=0)
Apr 24 01:02:11 encoding01 sudo[3743]: pam_unix(sudo:session): session closed for user puppet
Apr 24 01:02:18 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:02:18 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:02:28 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:02:28 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:02:38 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:02:38 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:02:48 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:02:48 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:02:58 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:02:58 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:03:08 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:03:08 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:03:11 encoding01 sudo[3752]: root : TTY=unknown ; PWD=/ ; USER=puppet ; COMMAND=list /bin/cat /var/lib/puppet/state/last_run_summary.yaml
Apr 24 01:03:11 encoding01 sudo[3753]: root : TTY=unknown ; PWD=/ ; USER=puppet ; COMMAND=/bin/cat /var/lib/puppet/state/last_run_summary.yaml
Apr 24 01:03:11 encoding01 sudo[3753]: pam_unix(sudo:session): session opened for user puppet by (uid=0)
Apr 24 01:03:11 encoding01 sudo[3753]: pam_unix(sudo:session): session closed for user puppet
Apr 24 01:03:18 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:03:18 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:03:28 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:03:28 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:03:38 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:03:38 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:03:48 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:03:48 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:03:58 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:03:58 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:04:08 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:04:08 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:04:11 encoding01 sudo[3762]: root : TTY=unknown ; PWD=/ ; USER=puppet ; COMMAND=list /bin/cat /var/lib/puppet/state/last_run_summary.yaml
Apr 24 01:04:11 encoding01 sudo[3763]: root : TTY=unknown ; PWD=/ ; USER=puppet ; COMMAND=/bin/cat /var/lib/puppet/state/last_run_summary.yaml
Apr 24 01:04:11 encoding01 sudo[3763]: pam_unix(sudo:session): session opened for user puppet by (uid=0)
Apr 24 01:04:11 encoding01 sudo[3763]: pam_unix(sudo:session): session closed for user puppet
Apr 24 01:04:18 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:04:18 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:04:28 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:04:28 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:04:38 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:04:38 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:04:48 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:04:48 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:04:58 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:04:58 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:05:01 encoding01 CRON[3771]: pam_unix(cron:session): session opened for user root by (uid=0)
Apr 24 01:05:01 encoding01 CRON[3772]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Apr 24 01:05:01 encoding01 CRON[3771]: pam_unix(cron:session): session closed for user root
Apr 24 01:05:08 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:05:08 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:05:11 encoding01 sudo[3775]: root : TTY=unknown ; PWD=/ ; USER=puppet ; COMMAND=list /bin/cat /var/lib/puppet/state/last_run_summary.yaml
Apr 24 01:05:11 encoding01 sudo[3776]: root : TTY=unknown ; PWD=/ ; USER=puppet ; COMMAND=/bin/cat /var/lib/puppet/state/last_run_summary.yaml
Apr 24 01:05:11 encoding01 sudo[3776]: pam_unix(sudo:session): session opened for user puppet by (uid=0)
Apr 24 01:05:11 encoding01 sudo[3776]: pam_unix(sudo:session): session closed for user puppet
Apr 24 01:05:18 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:05:18 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:05:28 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:05:28 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:05:38 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:05:38 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:05:48 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:05:48 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:05:58 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:05:58 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:06:08 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:06:08 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:06:11 encoding01 sudo[3785]: root : TTY=unknown ; PWD=/ ; USER=puppet ; COMMAND=list /bin/cat /var/lib/puppet/state/last_run_summary.yaml
Apr 24 01:06:11 encoding01 sudo[3786]: root : TTY=unknown ; PWD=/ ; USER=puppet ; COMMAND=/bin/cat /var/lib/puppet/state/last_run_summary.yaml
Apr 24 01:06:11 encoding01 sudo[3786]: pam_unix(sudo:session): session opened for user puppet by (uid=0)
Apr 24 01:06:11 encoding01 sudo[3786]: pam_unix(sudo:session): session closed for user puppet
Apr 24 01:06:18 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:06:18 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:06:28 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:06:28 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:06:38 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:06:38 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:06:48 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:06:48 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:06:58 encoding01 rc.local[733]: Error: Could not request certificate: getaddrinfo: Name or service not known
Apr 24 01:06:58 encoding01 puppet-agent[1173]: Could not request certificate: getaddrinfo: Name or service not known

dmesg:

[Apr23 06:24] Process accounting resumed
[Apr24 06:24] Process accounting resumed

--Zhuyifei1999 (talk) 21:19, 24 April 2019 (UTC)

@Zhuyifei1999: I think I’ve found what is wrong here. The ulimit is putting 2 days limit on the cpu time (not the elapsed real time!) In my case, the ffmpeg is running with 16 threads which means the runtime will be limited to much less than 2 days (exact ratio will depend on the thread bindings, IO time, parallel efficiency of the code, etc.). Can you please change the https://github.com/toolforge/video2commons/blob/master/video2commons/backend/encode/globals.py and multiply the “background_time_limit” by “ffmpeg_threads”? Many thanks! -- Meisam (talk) 19:39, 25 April 2019 (UTC)

@Meisam: Sorry, missed your message. (The ping adeed didn't work). I changed it to use wall clock time. Could you see if it works? --Zhuyifei1999 (talk) 18:59, 30 April 2019 (UTC)

@Zhuyifei1999: Thank you! This problem seems to be fixed. For now I’m struggling with other upload issues. I will start a new discussion if I cannot manage to solve it. -- Meisam (talk) 15:38, 1 May 2019 (UTC)

Error: An exception occurred: IOError: [Errno 30] Read-only file system

Trying a few different files. This error pops up in red while initially uploading: Error: An exception occurred: IOError: [Errno 30] Read-only file system: '/data/project/video2commons/video2commons/frontend/static/uploads/0e0... (longer string). It proceeds to finish the upload, but instead of going to the next screen it just gives me the upload button/text box again. — Rhododendrites ^talk | 00:50, 2 June 2019 (UTC)

Nevermind. Working last night (but the text is displaying in French now?). — Rhododendrites ^talk | 12:42, 2 June 2019 (UTC)

That EROFS error was related to a faulty puppet parameter that has been fixed on the hosts a few days ago, but the fix might not have been applied to the 'containers' that runs the webservice. I just restarted the webservice, can you see if it works now? --Zhuyifei1999 (talk) 21:02, 2 June 2019 (UTC)

An exception occurred: DownloadError: ERROR: unable to download video data

I have not been using video2commons for quite some time because there were many issues with incorrect encodings for me. Today I have decided to give it a try with this video, but I am now receiving an error stating that even the video data itself is not downloadable. ℺ Gone Postal (〠 ✉ • ✍ ⏿) 08:57, 4 July 2019 (UTC)

Can't reproduce. Is this error during the form submission or during the job running after the task is submitted? --Zhuyifei1999 (talk) 23:28, 4 July 2019 (UTC)

The error is after the task actually submits. It sits there for a few seconds and then does that. ℺ Gone Postal (〠 ✉ • ✍ ⏿) 23:54, 4 July 2019 (UTC)

I can't reproduce. I got to the transcoding phase with no problems. Both currently-running backends run the same version (2019.7.2) of youtube-dl so it should not be a version issue. ~~Would you mind keeping the task displayed in the task list so I can get the task ID and check relevant logs?~~ --Zhuyifei1999 (talk) 02:07, 5 July 2019 (UTC)

Never mind, I see the task. --Zhuyifei1999 (talk) 02:08, 5 July 2019 (UTC)

This is all its logs:

[2019-07-04 08:53:56,936: WARNING/PoolWorker-7] -1: Downloading...
[2019-07-04 08:53:56,938: WARNING/PoolWorker-7] -1: Creating YoutubeDL instance
[2019-07-04 08:53:57,041: WARNING/PoolWorker-7] -1: Preprocessing...
[2019-07-04 08:53:57,891: WARNING/PoolWorker-7] video2commons.backend.worker.main[e4baaf99-8c0a-4dbd-95aa-cc80a7410fb0]: video doesn't have subtitles
[2019-07-04 08:53:57,980: WARNING/PoolWorker-7] video2commons.backend.worker.main[e4baaf99-8c0a-4dbd-95aa-cc80a7410fb0]: "id" field is not a string - forcing string conversion, there is an error in extractor
[2019-07-04 08:53:58,331: ERROR/PoolWorker-7] video2commons.backend.worker.main[e4baaf99-8c0a-4dbd-95aa-cc80a7410fb0]: ERROR: unable to download video data: ''
[2019-07-04 08:53:58,333: WARNING/PoolWorker-7] -1: Cleaning up...
[2019-07-04 08:53:58,335: VERBOSE/PoolWorker-7] Found 1 commons:commons processes running, including this one.
[2019-07-04 08:53:58,336: VERBOSE/PoolWorker-7] Dropped throttle(s).
[2019-07-04 08:53:58,343: ERROR/PoolWorker-7] Task video2commons.backend.worker.main[e4baaf99-8c0a-4dbd-95aa-cc80a7410fb0] raised unexpected: DownloadError(u"ERROR: unable to download video data: ''",)
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/celery/app/trace.py", line 367, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/celery/app/trace.py", line 622, in __protected_call__
    return self.run(*args, **kwargs)
  File "/srv/v2c/video2commons/backend/worker.py", line 113, in main
    outputdir, statuscallback, errorcallback
  File "/srv/v2c/video2commons/backend/download/__init__.py", line 96, in download
    info = dl.extract_info(url, download=True, ie_key=ie_key)
  File "/usr/local/lib/python2.7/dist-packages/youtube_dl/YoutubeDL.py", line 807, in extract_info
    return self.process_ie_result(ie_result, download, extra_info)
  File "/usr/local/lib/python2.7/dist-packages/youtube_dl/YoutubeDL.py", line 861, in process_ie_result
    return self.process_video_result(ie_result, download=download)
  File "/usr/local/lib/python2.7/dist-packages/youtube_dl/YoutubeDL.py", line 1642, in process_video_result
    self.process_info(new_info)
  File "/usr/local/lib/python2.7/dist-packages/youtube_dl/YoutubeDL.py", line 1917, in process_info
    self.report_error('unable to download video data: %s' % error_to_compat_str(err))
  File "/usr/local/lib/python2.7/dist-packages/youtube_dl/YoutubeDL.py", line 624, in report_error
    self.trouble(error_message, tb)
  File "/usr/local/lib/python2.7/dist-packages/youtube_dl/YoutubeDL.py", line 594, in trouble
    raise DownloadError(message, exc_info)
DownloadError: ERROR: unable to download video data: ''

Redownloading on the same host does not produce the error. I'm guessing that this was a temporary issue of some sort. Have you retried? --Zhuyifei1999 (talk) 02:15, 5 July 2019 (UTC)

Sound uploads

For various reasons, I find it extremely convenient to convert and upload sound recordings from my mobile phone into Commons via Video2Commons with the video checkbox turned off. Perhaps the resulting file could automatically be given an oga extension instead of ogg? Shyamal L. (talk) 04:56, 3 August 2019 (UTC)

'ogg', instead of 'oga' seems to be the officially the preferred file extension for audio files of the Ogg container format. Could you explain what you are trying to achieve with 'oga'? --Zhuyifei1999 (talk) 23:15, 3 August 2019 (UTC)

I see. I thought that would prevent the timed text tab from appearing on commons. Shyamal L. (talk) 13:10, 14 August 2019 (UTC)

An exception occurred: <...> Target filename is invalid.

Tool doesn't want to work. "An exception occurred: TaskError: pywikibot.Error: UploadWarning: badfilename: Target filename is invalid." Target filenames I've used: "From Kvarsa station to Bolgury passing loop" and "From Quarsa to Bolgury". --Wadorgurt (talk) 01:18, 20 August 2019 (UTC)

I am unable to get the task ID. was it removed? --Zhuyifei1999 (talk) 18:22, 20 August 2019 (UTC)

Yes. --Wadorgurt (talk) 15:59, 21 August 2019 (UTC)

I can't find the logs without a task ID. Would you mind rerunning? --Zhuyifei1999 (talk) 18:15, 21 August 2019 (UTC)

Done. --Wadorgurt (talk) 17:07, 22 August 2019 (UTC)

Thanks, I got the backtrace and task parameters. The task parameter show no hidden characters or anything unusual, and the backtrace is:

[2019-08-22 16:32:37,963: ERROR/PoolWorker-16] Task video2commons.backend.worker.main[6775cc6d-ffcd-49a0-a76a-a08aca41d88c] raised unexpected: TaskError('pywikibot.Error: UploadWarning:
 badfilename: Target filename is invalid.',)
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/celery/app/trace.py", line 367, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/celery/app/trace.py", line 622, in __protected_call__
    return self.run(*args, **kwargs)
  File "/srv/v2c/video2commons/backend/worker.py", line 138, in main
    filedesc, username, statuscallback, errorcallback
  File "/srv/v2c/video2commons/backend/upload/__init__.py", line 48, in upload
    size, statuscallback, errorcallback
  File "/srv/v2c/video2commons/backend/upload/__init__.py", line 91, in upload_pwb
    chunk_size=chunked, async=bool(chunked)  # , ignore_warnings=['exists-normalized']
  File "/usr/local/lib/python2.7/dist-packages/pywikibot/tools/__init__.py", line 1447, in wrapper
    return obj(*__args, **__kw)
  File "/usr/local/lib/python2.7/dist-packages/pywikibot/site.py", line 6250, in upload
    offset=result.get('offset', False))
TaskError: pywikibot.Error: UploadWarning: badfilename: Target filename is invalid.

Which makes no sense to me. I'll look deeper over the weekend. --Zhuyifei1999 (talk) 06:41, 23 August 2019 (UTC)

I just looked at the description of 'badfilename'. It means that the filename it will be stored under is different than what is requested. However, I cannot reproduce this upload warning in Special:Upload, and it might have something to do with chunked uploads. Could you try without 'From' and/or 'Bolgury' in the title? I can move the file after it uploads. --Zhuyifei1999 (talk) 21:04, 25 August 2019 (UTC)

I tried with the default filename, and video was successfully uploaded. Please move it to "From Quarsa to Bolgury". --Wadorgurt (talk) 07:37, 29 August 2019 (UTC)

Done --Zhuyifei1999 (talk) 07:21, 30 August 2019 (UTC)

Note that you cannot upload a file if there is an existing description page on Commons by that name.

Anything we can do about this? I mean, you can upload a new version of a file on Commons, so why wouldn't this just be handled the same way? Alternatively, if more caution is called for, maybe check to see the uploader is also the uploader of the existing file? Or check for certain user rights? Something? :/ — Rhododendrites ^talk | 02:48, 9 October 2019 (UTC)

It generates an uploadwarning, which is on the backend (where the download / transcode / upload happens). The backend receives the warning and stores the condition, which is displayed on the web interface. At this point, the code can show that an upload warning occurred and display a warning, and a user can choose to ignore the warning... but then what? The interface has no possible way to route this 'ignore warning' request to the same host that was running the task...

I could make it set a ignore warnings flag and rerun the task, but it could take a long time and the circumstances cause the set of warning changed, and we would ignore any new warnings. At that point... one could have rerun with a renamed filename. --Zhuyifei1999 (talk) 21:46, 9 October 2019 (UTC)

TaskError: pywikibot.Error: APIError: blocked

I'm trying to upload a file from Youtube this file but getting the following error message:

An exception occurred: TaskError: pywikibot.Error: APIError: blocked: You have been blocked from editing. [blockinfo:{u'blockedtimestamp': u'2020-01-07T16:16:20Z', u'blockid': 378404, u'blockedby': u'Elcobbola', u'blockreason': u'Abusing multiple accounts: {{Checkuserblock}}', u'blockedbyid': 223409, u'blockexpiry': u'infinite'}; help:See https://commons.wikimedia.org/w/api.php for API usage. Subscribe to the mediawiki-api-announce mailing list at <https://lists.wikimedia.org/mailman/listinfo/mediawiki-api-announce> for notice of API deprecations and breaking changes.]

Any idea? -- Geagea (talk) 10:00, 9 January 2020 (UTC)

Already was fixed meanwhile. -- Geagea (talk) 10:31, 9 January 2020 (UTC)

Isn't WMCS on the autoblock whitelist? (forgot where that is) --Zhuyifei1999 (talk) 16:53, 9 January 2020 (UTC)

I added a 4.5min video from voanews.com yesterday but it is still in progress

Are there server issues?--Roy17 (talk) 01:12, 11 February 2020 (UTC)

Overload. https://tools.wmflabs.org/nagf/?project=video Pooling 3rd server now. --Zhuyifei1999 (talk) 01:28, 11 February 2020 (UTC)

CC @Eatcha: --Zhuyifei1999 (talk) 01:47, 11 February 2020 (UTC)

For how bad the load is, on the worst one, encoding05, with a current 1 minute load average of 114, with the instance virtual core count of 16, that's having a bit more than 7 instances under full load --Zhuyifei1999 (talk) 01:51, 11 February 2020 (UTC)

Roy17 Zhuyifei Sorry, I stopped the batch uploading of all videos from 2 channels. The load was, afaik was generated by these batch uploads. Is there any time(in UTC) when the load is minimum ? I will schedule batch tasks in the least busy slot of the day. Infact some of these batch uploads exited with 137. I made a mistake, I am sorry for it. -- Eatcha (talk) 04:02, 11 February 2020 (UTC)

I never did statistics, but you can check NAGF data for yourself. I don't mind if you batch upload, but please don't run more than 8 tasks at once unless you give me a ping, in which case I'll do manual task priorities.

For future (when I finally get to rewriting much of v2c) reference: I need to fix process killing on abortion, set a task retry on load threshold. --Zhuyifei1999 (talk) 04:11, 11 February 2020 (UTC)

Import from uploads

How did v2c usually download videos from Youtube? In what video format/resolution? How should one correctly download youtube video for the following importing it "from uploads"? --Wadorgurt (talk) 15:22, 20 February 2020 (UTC)

How did v2c usually download videos from Youtube?: When it worked, youtube-dl
In what video format/resolution?: In most common configuration, format specification bestvideo[vcodec=vp9][filesize<4294967296]+bestaudio[acodec=opus]/bestvideo[ext=webm][filesize<4294967296]+bestaudio[ext=webm]/bestvideo+bestaudio/best to youtube-dl
How should one correctly download youtube video for the following importing it "from uploads"?: Don't understand.

--Zhuyifei1999 (talk) 17:16, 20 February 2020 (UTC)

There is a problem on V2C website

Good evening.

The website https://tools.wmflabs.org/video2commons/ doesn't work at this time.

--ComputerHotline (talk) 18:16, 6 April 2020 (UTC)

I'm having this issue as well. It looks like most/all of the app's JS/CSS files on tools-static.wmflabs.org are failing to load with a 403 error from Cloudflare. –IagoQnsi (talk) 21:15, 6 April 2020 (UTC)

Fails to convert video

Hi. I have a mp4 file of the size 247 MB. I tried to upload it to commons using this tool. However, I got this error. Could you please help me with this?
acagastya 04:27, 12 May 2020 (UTC)

Fixed by Zhuyifei1999 on GitHub in this patch.
acagastya 16:28, 13 May 2020 (UTC)

I tried to convert a video from YouTube today and I got an error. I downloaded the file to my computer then uploaded it to Video2commons and it processed, but that was extra steps. Excuse me for not getting the error code. @Quilt Phase: you said you had the same issue today, is that right? Blue Rasberry (talk) 00:06, 17 May 2020 (UTC)

@Bluerasberry: That's true. video2commons code hinges on the youtube-dl command line utility. From time to time, youtube-dl has to be upgraded to keep track of changes in the way YouTube works. The error message I get when attempting to use video2commons with a YouTube URL:

Error: An exception occurred: DownloadError: ERROR: Unable to download webpage: HTTP Error 429: Too Many Requests (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see https://yt-dl.org/update on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

is exactly the same I sometimes get when not using the latest version of youtube-dl in a command-line utility on my computer. Quilt Phase (talk) 00:39, 17 May 2020 (UTC)

HTTP Error 429: Too Many Requests is not because of older version of youtube-dl BUT google/YouTube blocked the public IP indefinitely. According to YouTube/Google, Downloading videos from YouTube fails to comply with their TOS. Converting videos doesn't involve youtube-dl. Unrelated : The developer of the tool resigned and retired.

Following is a part of their huge TOS.

The following restrictions apply to your use of the Service. You are not allowed to:

1) access, reproduce, download, distribute, transmit, broadcast, display, sell, license, alter, modify or otherwise use any part of the Service or any Content except:
 (a) as expressly authorized by the Service; or 
 (b) with prior written permission from YouTube and, if applicable, the respective rights holders; 

3) access the Service using any automated means (such as robots, botnets or scrapers) except 
 (a) in the case of public search engines, in accordance with YouTube’s robots.txt file; or 
 (b) with YouTube’s prior written permission;

// Eatcha (talk) 10:31, 17 May 2020 (UTC)

There is a problem on V2C website (2)

Good afternoon.

The website https://tools.wmflabs.org/video2commons/ doesn't work at this time.

--ComputerHotline (talk) 12:01, 12 June 2020 (UTC)

it's blocking on "converting..." --ComputerHotline (talk) 19:42, 12 June 2020 (UTC)

Yep, I'm seeing that too. — Huntster (t @ c) 12:03, 13 June 2020 (UTC)

Same here stuck while conversion (Converting...). Based on YouTubeReviewBot Logs imo it's related to NFS maintenance (read about the maintenance on cloud announce mailing list, was scheduled at 11th of June , 1600 UTC). I have no idea how to fix this or who's the best person to contact (Zhuyifei1999 has retired, and maybe busy). // Eatcha (talk) 17:41, 13 June 2020 (UTC)

@Matanya: could you please help perhaps?--Roy17 (talk) 20:25, 14 June 2020 (UTC)

Working on it. matanya • talk 08:51, 15 June 2020 (UTC)

Should be OK now, sorry for the inconvenience. matanya • talk 09:59, 15 June 2020 (UTC)

Thank you very much, Matanya, for working on this. I've finished one task and it's processing a second now, so all seems good. Question, though: is it normal to abort a task (as I did with two that got hung up due to the error) and the status never move past "Your task is being aborted..." with no Remove option? Just concerned there may be some stuck processes or something. Cheers! — Huntster (t @ c) 22:31, 15 June 2020 (UTC)

Must delete via redis. @Acagastya: It's yours now. You can figure this out. --Zhuyifei1999 (talk) 23:37, 15 June 2020 (UTC)

v2c on Windows for Dummies

As yall know, it's been a while since v2c cannot convert youtube videos coz ip is blocked. I have recently finally figured out how to simulate the v2c process on my Windows PC.

First, you need youtube-dl. You also need ffmpeg or libav (I use ffmpeg).

Get youtube-dl.exe from http://ytdl-org.github.io/youtube-dl/download.html .
Get ffmpeg from https://ffmpeg.zeranoe.com/builds/ .

Put them into a root folder so that you have

C:\rootpath\youtube-dl.exe
C:\rootpath\ffmpeg\bin\ffmpeg.exe

Take note, the rootpath must not contain a space, otherwise an error will occur in ytdl.

This is all set up. The download process is as follows.

Open cmd.exe (shortcut: press Windows+R, type cmd, click OK).
Enter cd C:\rootpath
If you want to extract both video and audio, enter
youtube-dl -f "bestvideo[vcodec=vp9][filesize<4294967296]+bestaudio[acodec=opus]/bestvideo[ext=webm][filesize<4294967296]+bestaudio[ext=webm]/bestvideo+bestaudio/best" --write-sub --write-auto-sub --sub-format "srt/ttml/best" --sub-lang en --ffmpeg-location %CD%\ffmpeg\bin -k URL

(This code is adapted from special:diff/396496704.)
If you only want the video (which is quite common when you dont want the copyrighted music soundtrack), enter
youtube-dl -f webm --write-sub --write-auto-sub --sub-format "srt/ttml/best" --sub-lang en URL

Explanation:

Replace URL with the URL of the youtube video you want.
--write-sub lets you download the human contributed subtitles.
--write-auto-sub lets you download the youtube autogenerated subtitles.
--sub-lang en lets you download only the English subtitles. Change en to other langcodes of your choice. Separate multiple ones with a comma. For example, --sub-lang en,zh-Hans,fr downloads English, simplified Chinese and French subtitles. These codes are case sensitive.
How to check what langcodes are available for your video? Enter youtube-dl --list-subs URL .
If you want to get all the available human contributed subtitles, replace --write-sub --write-auto-sub --sub-format "srt/ttml/best" --sub-lang en with --all-subs --sub-format "srt/ttml/best" .
If there are both human sub and autogen sub available in a language, using --write-sub --write-auto-sub will get the human sub.
In the method of downloading both video and audio, ytdl actually downloads them separately and merges them. -k lets you keep the separate video and audio files. Removing -k, you will get the merged video only.

With this you get the video (in webm or mkv) and a bunch of subtitles.

Commons only accepts srt subtitle files, so if the subs are in ttml format, I am using https://gotranscript.com/subtitle-converter to convert ttml to srt.

If your video is webm, go straight to Special:UploadWizard or Special:Upload. If it's mkv, go to v2c.

After uploading the video, you can add the srt files this way:

Go to your video. Click TimedText on top.
Choose the correct langcode and click Go.
Open your srt in Microsoft Notepad. Copy everything.
Paste it in the TimedText page and publish.

This guide is written with elements from https://www.reddit.com/r/learnprogramming/comments/44nhzp/how_to_use_youtubedl_and_ffmpeg_to_download/ and https://github.com/ytdl-org/youtube-dl/blob/master/README.md .

Feel free to suggest better and smoother workarounds, as well as a non commercial alternative to the ttml2srt converter.--Roy17 (talk) 20:25, 14 June 2020 (UTC)

File exists with different extension error

The script does not want to convert my files pointing out that files exist with a different extension. Truly there are images of the same name, but WMC commons allows to have a same name for files with different files extension. So there could be file:dog.jpg and file:dog.ogv. So what's the problem? The error message is for example: An exception occurred: TaskError: pywikibot.Error: UploadWarning: exists-normalized: File exists with different extension as "Modré_loubí_(001).jpg". --Juandev (talk) 10:52, 23 June 2020 (UTC)

Agreed, that's a very frustrating "feature". — Huntster (t @ c) 19:06, 23 June 2020 (UTC)

Seems to think my file is a webpage? (youtube-dl problem)

Tried a couple times to upload a video file. It gets past the upload and my confirmation of the description information, then gives this error:

An exception occurred: DownloadError: ERROR: Unable to download webpage: HTTP Error 308: Permanent Redirect (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see https://yt-dl.org/update on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

— Rhododendrites ^talk | 14:35, 11 July 2020 (UTC)

V2C doesn't works

Hello.

[4] doesn't works :-(

--ComputerHotline (talk) 18:52, 11 July 2020 (UTC)

Ok. It works ;-) --ComputerHotline (talk) 19:02, 11 July 2020 (UTC)

Not working

I'm getting this error:

An exception occurred: DownloadError: ERROR: Unable to download webpage: HTTP Error 308: Permanent Redirect (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see https://yt-dl.org/update on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

--TaronjaSatsuma (talk) 14:44, 12 July 2020 (UTC)

Same here. Buidhe (talk) 08:02, 13 July 2020 (UTC)
I'm getting errors too, and did not before. It seems like the process has changed slightly? After you first upload a video there is now a prompt on whether to use the uploaded file or not. Also, I don't recall "Creating YoutubeDL instance" as part of the process before.--Pharos (talk) 16:10, 21 July 2020 (UTC)

It's working for me now.--Pharos (talk) 13:20, 2 August 2020 (UTC)

googlevideo.com - downloading URL has been blacklisted

Hello. I'm trying to upload a video from YouTube to Commons using this tool. Because it doesn't support direct downloading from YouTube, I've got a direct link to the video using special browser plugins. Each YouTube video is hosted on the web server with links like r1---sn-8xgn5uxa-quhl.googlevideo.com/videoplayback?expire=11111111&ei=xxxxxx ... However, when I pasted this link to video2commons, added description and launched uploading, I've got an error «An exception occurred: TaskError: Your downloading URL has been blacklisted.». Where is this black list and is it possible to exclude googlevideo.com from it to allow downloading videos from YouTube and uploading them to Commons? Maybe there are some third party websites, which don't generate direct link to googlevideo, but download videos to their own servers and then generate temporary download link, which will work with V2C? For example, I've found a website https://notube.net which works in such manner and generates temporary link which exists 15 minutes (enough time to upload video through V2C), but it doesn't work with long videos (longer than 10-15 minutes) and 4K videos. Xenotron (talk) 17:48, 28 July 2020 (UTC)

DownloadError

Hi, Trying to upload a video via URL and it keeps saying:

Error: An exception occurred: DownloadError: ERROR: Unable to download webpage: HTTP Error 429: Too Many Requests (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see https://yt-dl.org/update on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

Thanks, –Davey2010^Talk 00:52, 12 August 2020 (UTC)

Exact : I have this error too. When I want to convert a video. --ComputerHotline (talk) 06:40, 16 August 2020 (UTC)

(Same issue for me too) Reported the issue at #124. Kind regards, — Tulsi Bhagat ^{[ contribs | talk ]} 04:26, 8 September 2020 (UTC)

Looks like it already reported here phab:T236446. — Tulsi Bhagat ^{[ contribs | talk ]} 05:31, 8 September 2020 (UTC)

Many thanks Tulsi Bhagat for the update :), Thanks, –Davey2010^Talk 18:07, 28 October 2020 (UTC)

video2commons is down

On this page, I see an error message:

Something went terribly wrong, and we failed to find the cause automatically. Please file an issue in GitHub.

There's a discussion about this issue on GitHub. Jarble (talk) 13:42, 21 August 2020 (UTC)

Your task is pending...

Hello.

V2C freeze and shows this message "Your task is pending...".

--ComputerHotline (talk) 18:46, 26 August 2020 (UTC)

V2C doesn't works. It shows : "An exception occurred: TaskError: /mnt/nfs/labstore-secondary-project/gentoo-prefix/usr/bin/ffmpeg -y -i /srv/v2c/output/bb4ba32fe4671093/dl.mp4 -max_muxing_queue_size 4096 -threads 16 -row-mt 1 -qmin 19 -qmax 19 -vcodec libvpx-vp9 -tile-columns 4 -auto-alt-ref 1 -lag-in-frames 25 -speed 4 -f webm -an -pass 1 -passlogfile /srv/v2c/output/bb4ba32fe4671093/dl.mp4.vp9.webm.log /dev/null Exitcode: 1" --ComputerHotline (talk) 12:40, 27 August 2020 (UTC)

Same. It doesn't work. MiguelAlanCS (talk) 11:04, 13 September 2020 (UTC)

Error: Something went wrong while uploading

I've used v2c with no problems before, at the moment it fails during uploading with the statement: "Error: Something went wrong while uploading… try again?". Tried about 5 times and never got more than 42% uploaded before failure. The file is not that large (144MB). I also tried uploading a file that worked with v2c before and got same error. Thanks for the tool, hope it can be made functional again. Dennis the Peasant (talk) 14:17, 22 September 2020 (UTC)

I tried again about 2 hours later - without changing anything on my end - and the upload went through with no problems. Dennis the Peasant (talk) 16:56, 22 September 2020 (UTC)

Safe YouTube alternative: VLC

PC Mag is a British computing guide and has an article that has a guide to using VLC media player for retrieving YouTube videos. Simply extracting the original data when it is commons licensed should abide with the YouTube TOS, and VLC seems like a safe alternative while V2C can't do that. Kingsif (talk) 04:37, 19 October 2020 (UTC)

Thanks! MiguelAlanCS (talk) 16:31, 28 October 2020 (UTC)

YouTube produces error

Hey, I have been trying to pull a youtube file, and it gives me this error: Error: An exception occurred: DownloadError: ERROR: Unable to download webpage: HTTP Error 429: Too Many Requests (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see https://yt-dl.org/update on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

Cheeers, Sadads (talk) 10:34, 19 October 2020 (UTC)

Video page tagged as having no licence info

I have just uploaded File:Onion grading at the Centre for Agroecology, Water and Resilience_(CAWR),_Coventry University.webm using this tool. The resultant page was tagged with {{No license since}} rather than the expected {{YouTube}} (which I have now applied). Is this a bug? Andy Mabbett (Pigsonthewing); Talk to Andy; Andy's edits 13:04, 3 November 2020 (UTC)

[Feature request] test file name before confirm

it would be best if the tool will test whether a file with the same name as what you fill in the form already exists, and display a link to that file. this way it's easy to check if a video has already been imported in its name. RZuo (talk) 12:27, 22 April 2022 (UTC)

Error message

Each time I try to upload I get an error message saying "no such file". The video is from my phone, where it was recorded. Is there something I can do to fix this? Rubbish computer Ping me or leave a message on my talk page 18:29, 20 May 2022 (UTC)

ERROR

Hello.

V2C generate an error :

An exception occurred: Exception: b'<class \'UnicodeDecodeError\'>([\'utf-8\', \'b"Input #0, mov,mp4,m4a,3gp,3g2,mj2, from \\\'/srv/v2c/output/84c30be95a8bcbd5/dl.mp4\\\':\\\\n Metadata:\\\\n major_brand : isom\\\\n minor_version : 512\\\\n compatible_brands: isomiso2avc1mp41\\\\n title : 2022-06-07_meeting-NUPES-Belfort\\\\n artist : Thomas Bresson\\\\n date : 2022-06-07\\\\n encoder : vsdc\\\\n comment : Dans le cadre des \\\\xc3\\\\xa9lections l\\\\xc3\\\\xa9gislatives des 12 et 19 juin 2022 : r\\\\xc3\\\\xa9union publique de la NUPES (Nouvelle Union populaire \\\\xc3\\\\xa9cologique et sociale) en pr\\\\xc3\\\\xa9sence des candidats Florian Chauche, G\\\\xc3\\\\xa9rald Loridat et de la d\\\\xc3\\\\xa9put\\\\xc3\\\\xa9e europ\\\\xc3\\\\xa9enne Le\\\\xc3\\\\xafla Chaibi, \\\\xc3\\\\n copyright : Thomas Bresson\\\\n Duration: 01:31:57.81"\', 616, 617, \'invalid continuation byte\'])'

--ComputerHotline (talk) 13:21, 8 June 2022 (UTC)

The maintainer of v2c has left the project. Therefore this will most likely not be fixed. It looks like your file contains in the metadata a text string with ISO8859 encoded letters and this is interpreted by v2c as unicode and rejected. Remove the metadata and your upload will probably work. v2c would remove the metadata, if it could read them. --C.Suthorn (talk) 20:31, 8 June 2022 (UTC)

An exception occurred, 10-06-2022

Hi, I got the following message while importing [5]:

An exception occurred: TaskError: b"b'pywikibot.Error: APIError: missingresult: No result in status data.\\n[help: See https://commons.wikimedia.org/w/api.php for API usage. Subscribe to the mediawiki-api-announce mailing list at <https://lists.wikimedia.org/postorius/lists/mediawiki-api-announce.lists.wikimedia.org/> for notice of API deprecations and breaking changes.]'"

Thanks, Yann (talk) 19:56, 10 June 2022 (UTC)

Check if the file appears in your uploads in the next hours. If not: Just try again. (Best in the night of a public holiday). It is only a hiccup of the MW software. --C.Suthorn (talk) 20:08, 10 June 2022 (UTC)

Video2commons down?

The interface doesn't load completely on my end, might it need rebooting? Only the top menu bar and than a white page loads. Vera ^(talk) 07:27, 13 July 2022 (UTC)

@Vera: I got it to start processing https://www.youtube.com/watch?v=TiGkU_eXJa8 with replacement of the onelink.me link by https://www.redbull.com/us-en/apps for Yann per #An exception occurred, 10-06-2022 above. — Jeff G. ツ_{please ping or talk to me} 12:57, 14 July 2022 (UTC)

That ended with:

An exception occurred: TaskError: b'b"pywikibot.Error: APIError: stashfailed: Error storing file in \'mwstore://local-multiwrite/local-temp/5/50/20220714134121!phpnqmHWG.webm\': backend-fail-internal; local-swift-eqiad\\n[help: See https://commons.wikimedia.org/w/api.php for API usage. Subscribe to the mediawiki-api-announce mailing list at <https://lists.wikimedia.org/postorius/lists/mediawiki-api-announce.lists.wikimedia.org/> for notice of API deprecations and breaking changes.]"'

. Trying again... — Jeff G. ツ_{please ping or talk to me} 13:45, 14 July 2022 (UTC)

@1Veertje and Yann: The second try worked, witness File:Experience the world's first ski descent of K2 with Andrzej Bargiel.webm. Four transcodes errored out, so I had to reset them. All transcodes have completed now, after 42 min 27 s. — Jeff G. ツ_{please ping or talk to me} 14:30, 14 July 2022 (UTC)

I'm having issues with uploading of video files to WIkimedia commons using video2commons

Hi, I've been trying to upload a couple of video files using the video2commons, but it's been returning error messages stopping me from moving ahead with the task. Is there another way of getting it done? Please, folks, I need your help. The files were converted from MP4 to Mweb before the upload. See the error here."Error: An exception occurred: FileNotFoundError: b"[Errno 2] No such file or directory: '/data/project/video2commons/video2commons/frontend/static/uploads/6ec04b3a-1268-11ed-aa9c-82d0630f66a4'".13:41, 2 August 2022 (UTC) — Preceding unsigned comment added by Olaniyan Olushola (talk • contribs)

@Olaniyan Olushola: Are these files already online somewhere? — Jeff G. ツ_{please ping or talk to me} 13:46, 2 August 2022 (UTC)

Hi Jeff, the files are not online; they are not yet published anywhere and are stored away on my hard drive.There are over 60 files in MP4 format; I need to convert them one after the other to the required format before the upload if I use the upload wizard. So far, I have converted less than ten files, and I still have a long way to go. Each file comes with an average size of 40MB. So, I am looking for a tool where I won't do the conversion Olaniyan Olushola (talk) 17:21, 2 August 2022 (UTC)

You don't need to use video2commons, if your files are already transcoded to webm and local files. You can use any upload tool (for example Upload Wizard). What is the file size of the videos? --C.Suthorn (talk) 16:33, 2 August 2022 (UTC)

There are 60+files and each has an average size of 40MB. Olaniyan Olushola (talk) 17:23, 2 August 2022 (UTC)

Commons talk:Video2commons/Archive 1

Very slow

Video formats

Translation of "Links"

Conversion of wav and mp3

Translation marks

Error message when I upload from Youtube to commons

Error when I try to convert a Youtube video

Signature extraction failed

Upload a new version of this file

Errno 116

Not importing subtitles from youtube

[BUG] file page was not created??

Is Twitter supported?

It doesn't work

Replace pipe | with | instead of |

Does the automatic upload summary contain private information?

Language

Thank you

DownloadError: ERROR: content too short

TaskError: pywikibot.Error

Quality options

Is it possible to give the tool more than one URL to do batch uploads

blacklist channel

WebM (VP9/Opus)

Support for Google Photos

Is there a way to pre-fill the description of the file?

Bugs & feature requests from WLMRO

Panoramic video is messed up

Multiple selection

Error uploading file

Uploading errors--I need help

Exitcode: 137

Error: An exception occurred: IOError: [Errno 30] Read-only file system

An exception occurred: DownloadError: ERROR: unable to download video data

Sound uploads

An exception occurred: <...> Target filename is invalid.

Note that you cannot upload a file if there is an existing description page on Commons by that name.

TaskError: pywikibot.Error: APIError: blocked

I added a 4.5min video from voanews.com yesterday but it is still in progress

Import from uploads

There is a problem on V2C website

Fails to convert video

There is a problem on V2C website (2)

v2c on Windows for Dummies

File exists with different extension error

Seems to think my file is a webpage? (youtube-dl problem)

V2C doesn't works

Not working

googlevideo.com - downloading URL has been blacklisted

DownloadError

video2commons is down

Your task is pending...

Error: Something went wrong while uploading

Safe YouTube alternative: VLC

YouTube produces error

Video page tagged as having no licence info

[Feature request] test file name before confirm

Error message

ERROR

An exception occurred, 10-06-2022

Video2commons down?

I'm having issues with uploading of video files to WIkimedia commons using video2commons

Navigation menu

Search