Commons:Bots/Requests/Noaabot

From Wikimedia Commons, the free media repository
Jump to navigation Jump to search

Noaabot (talk · contribs)

Operator: Fæ (talk · contributions · Statistics · Recent activity · block log · User rights log · uploads · Global account information)

Bot's tasks for which permission is being sought: Uploading of archives of images from the National Oceanic and Atmospheric Administration.

These are public domain and the background of the initial request and project can be found at Commons:Batch_uploading/Weather_maps#Coordination. In addition to the initial batch upload of around 20,000 images providing maps from September 2002 to the current day, there may be categorization and formatting changes as needed that can run from this account. Beta test images consisting of weather maps for the first year of the archive and the most recent month of maps, can be found at 2002 NCEP weather maps (610 maps) and 2013 NCEP weather maps (ongoing with the most recent weather maps being uploaded each day; these appear to be released after 2pm EST for the previous day's maps).

Partial supervision for archive. Beta testing then monitored runs would be expected, with unmonitored runs once uploads or changes are seen to be stable (i.e. 1,000 or more uploads or changes). Daily or weekly updates would be automatic with some regular oversight or in response to questions.

Automatic or manually assisted: Automatic.

Edit type (e.g. Continuous, daily, one time run): One time runs for the past 11 years of archive maps and then a daily or weekly automatic update. For the NCEP weather, 5 types of maps are made available each day and a weekly summary pdf is derived from these. The maps are published as gifs and Noaabot is converting these to pngs before uploading.

Maximum edit rate (e.g. edits per minute): Approximately 4 per minute.

Bot flag requested: (Y/N): Y

Programming language(s): Python

Fæ (talk) 11:12, 22 May 2013 (UTC)[reply]

Discussion

  • Not sure if the template is correct—the images do not come directly from NOAA, but from NCEP (National Centers for Environmental Prediction) — other than that, I'm quite OK with the way files are uploaded. odder (talk) 12:21, 22 May 2013 (UTC)[reply]
    Categories now changed from using "NOAA" to "NCEP". In theory the NCEP is a child of the "National Weather Service" (previously "Weather Bureau") which itself is a child of "NOAA". I would suggest avoiding making the category tree over-hierarchical until it starts to appear over-loaded or might be misleading. --Fæ (talk) 06:11, 24 May 2013 (UTC)[reply]
  • Looks OK for me. --EugeneZelenko (talk) 14:35, 22 May 2013 (UTC)[reply]
  • Files should have more categories. A category for the day and a category for the type. Also, are you going to upload the Daily Weather Map Weekly PDF Files? The pdfs contain vectorized maps.Smallman12q (talk) 00:58, 24 May 2013 (UTC)[reply]
  • Just as an outside thought, I believe that a category for each day is just too much, but I would support creating monthly categories; there should be around 150 files in each category, which would make them quite useful. odder (talk) 12:44, 24 May 2013 (UTC)[reply]
  • Why did bot upload images from USA military? --EugeneZelenko (talk) 14:17, 27 May 2013 (UTC)[reply]
  • I'd much prefer not to have them in both year and month, per COM:OVERCAT. Scrolling through is not much of an argument, because there are 5 different types in the year category, so you can hardly remember what it was 5 files back. Also I think a daily category would actually be useful for weather, because some people like comparing today's weather with last year's on the same day. --99of9 (talk) 18:48, 27 May 2013 (UTC)[reply]
    • The current directories provide a directory per year by type (example), so you do not have to browse the full year with all 5 map types. I am unsure what your expectation of a day category means. Is this something like "day 42" of all years (leap years being a problem), or "Monday", or something else? Any of this is do-able, but I feel a lot could be done with category intersections rather than hard categories, or by a user searching by dates. --Fæ (talk) 18:54, 28 May 2013 (UTC)[reply]
      • Yes, 2002_precipitation is a good category for browsing. But the files in there, are also in the parent cat Category:2002_NCEP_weather_maps, which if this convention is followed, will eventually have 5*365 files in it. I think they should be removed from that, since they can be immediately and directly placed in any relevant subcats at the time of upload. --99of9 (talk) 11:21, 30 May 2013 (UTC)[reply]
      • Sorry I didn't properly explain the daily category idea. What I mean is similar, but more user-friendly than "day 42". My category titles would be something like Category:NCEP weather maps for 11 September (although I can understand the argument for having the date the other way around for USA weather, current Commons date categorization is this way: Category:Days_in_September). I can think of a few things people might use these categories for (e.g. historical event research; or "wasn't it warmer last year?"). I don't think many users are sophisticated enough to do cat-intersects, and since it's easy to do, I'm not sure why we can't do it for them now? (On the other hand, I don't think "Monday" is correlated with weather patterns, so I don't think it's useful.) --99of9 (talk) 11:21, 30 May 2013 (UTC)[reply]
        • Okay, let me ponder it. The most recent uploads for days this week include the month category and it is easy enough in Python to name a category by the day of the month. I'll think about setting up an example day, and then uploading the maps for the full 11 years for that one day of the year, so we can see it in "action" before making a mare's nest of categories. Obviously "29 February" will end up a little sparse. --Fæ (talk) 11:27, 30 May 2013 (UTC)[reply]
  • I have gone along with your suggestions and implemented them in the upload, namely dropping the year category with all types and breaking down the day category with types. This will mean a bit of category emptying later on, and quite a bit of category creation (which I have not automated yet, but will ponder it). In the example of the file I just uploaded, File:2006-05-20 Max-min Temperature Map NOAA.png this means the following categories were added: --Fæ (talk) 15:40, 30 May 2013 (UTC)[reply]
Category:NCEP black and white daily max-min temperature maps - this can be switched off in {{NOAA-dailywxmap}}
Category:2006 NCEP black and white daily max-min temperature maps
Category:NCEP weather maps for May 2006
Category:NCEP black and white daily max-min temperature maps for 20 May

I think the first can be cut out because it's a parent of the second, and I doubt anyone will want to do a slideshow of over 12 years of files. Depending if you think people would prefer to scan a whole year or a month at a time, you could cut it down to just (with even more category creation):

Category:NCEP black and white daily max-min temperature maps for May 2006
Category:NCEP black and white daily max-min temperature maps for 20 May

If you need help with the category creation I have some scripts that might help. --99of9 (talk) 15:51, 30 May 2013 (UTC)[reply]

A Python code snippet (by email) might be helpful. Were I writing it, as I have the generated category name, I just need to call something to check "does this exist?" and if not, then I'll write the initial contents (which I have the basics already in the code to write in). For the existence check, rather than a failed page connection, a commons API call might be a quicker way of doing it. I'm not really stuck on this, it's just time to look it up and test it out. The '12 year' type cat is easily switched off in the template. I disagree with dropping the year of a type cat, this is rather useful for seeing the seasonal patterns over the year which would be much harder to do if broken into 12 month categories; though I'm not against having both.
Consider it pondered. It is easy enough to do a call like this and check for the 'missing' flag. I'm assuming this is slightly quicker than getting the category page, which I can do if this existence test fails. I'll add this in before running a bit more testing.
Category existence/creation routines now added, I am running through files for Category:NCEP weather maps for 21 May to check. Creating categories this way means they only get created when there is a file to populate them. --Fæ (talk) 10:06, 31 May 2013 (UTC)[reply]

Approved --99of9 (talk) 00:41, 4 September 2013 (UTC)[reply]

Retrieved from "https://commons.wikimedia.org/w/index.php?title=Commons:Bots/Requests/Noaabot&oldid=103371135"