Help talk:Extension:FileImporter

Jump to navigation Jump to search

About this board

Central feedback page for FileImporter/FileExporter.

Under 10 bytes file description page revisions break import to Commons for new Commons users

1
Summary by Thiemo Kreuz (WMDE)
Jeff G. (talkcontribs)

We have been having problems on Wikimedia Commons with new (to Commons) users of this tool not being allowed to import files with under 10 bytes file description page revisions, such as en:File:2018 AWS Sites ALL 03 29 2018.jpg and en:File:AntiBullyingCampaign.jpg, due to denial by c:Special:AbuseFilter/4. The current discussion is at c:COM:FILTERT#Report by Splattereel. Can this tool please be configured to not import under 10 bytes file description page revisions, or at least to pad the ones under 10 bytes with spaces or the 10 byte string "Padded4AF4"? See also phab:T215429 and phab:T213409.

Reply to "Under 10 bytes file description page revisions break import to Commons for new Commons users"
4nn1l2 (talkcontribs)
Reply to "language-tagged"

Attribution breaks in some cases, potentially violating licenses

5
Summary by Thiemo Kreuz (WMDE)
Peteforsyth (talkcontribs)

This is a great tool! However, I notice that when moving a file (example: File:Mary A Boren.jpg), the original uploader's name -- which in some cases is significant for attribution, i.e. with a CC BY or CC BY-SA license -- is originally linked to the user page on the origin wiki, but after file transfer it is linked to the user page on COMMONS (which may not exist). A simple change would avoid legal problems: change the link [[User:Name]] to [[languagecode:originwiki:User:Name]].

-Pete Forsyth (talk) 19:10, 1 February 2019 (UTC)

Thiemo Kreuz (WMDE) (talkcontribs)

Since the SUL finalisation (T37707) got resolved in 2015, the user pages on all Wikimedia wikis are guaranteed to the refer to the same legal person. So I have to ask. Can you please explain what exactly you mean when this "breaks", and what the "some cases" are that you are referring to?

Peteforsyth (talkcontribs)

Hi Thiemo, from the Creative Commons attribution license (v. 4.0):

If You Share the Licensed Material (including in modified form), You must: retain the following if it is supplied by the Licensor with the Licensed Material: identification of the creator(s) of the Licensed Material and any others designated to receive attribution, in any reasonable manner requested by the Licensor (including by pseudonym if designated);

I would point specifically to the words "retain" and "reasonable." Take a look at the following example:

  • File on English Wikipedia:
    • Username attributed on the Wikipedia page prior to move (blue link leading to genuine information):
  • File after move to Commons:
    • Username attributed on Commons after move, but before manual repair (redlink):

The photographer uploaded the file to English Wikipedia, and in so doing, defined the way he wanted to be attributed. In addition to his username, the method of attribution included a link to a page that has his username, and further information about himself, including links to blog and website.

After a move by this tool, that information is not retained. The blue link becomes a red link, which leads to a blank page inviting the reader to "create" it. It matters not to any reader who lacks intimate familiarity with the inner workings of wiki sites that there is some link between that blank page with the word "create" and some page on a different website which has the information they may have been seeking. To expect most users to get from the page they land on to the page originally linked as attribution would not be reasonable.

The solution to this problem, I think, is technically very simple: Simply have the tool insert the proper code in any relevant userpage links to retain the original link, as I did manually here.

-Pete Forsyth (talk) 19:18, 4 February 2019 (UTC)

Peteforsyth (talkcontribs)

To be clear, my primary concern is about the licensing requirement for attribution and the "author" field, not about provenance and the upload log. In this similar case, I don't see a major problem, even though this tool does result in the loss of some information (the user page of the original uploader). In an ideal world, moving the page would not result in the loss of that information; however, it is less important that the upload log be retained, because information about the author and about the uploader is kept separately, and authorship is more important (both legally and in practice). -Pete Forsyth (talk) 19:33, 4 February 2019 (UTC)

Thiemo Kreuz (WMDE) (talkcontribs)

Thanks a lot for the additional information, this is super helpful! I believe this can be solved together with what we already collected in T198584.

Reply to "Attribution breaks in some cases, potentially violating licenses"

Option to delete original (or mark for deletion)

2
Summary by Johanna Strodt (WMDE)
Peteforsyth (talkcontribs)

When moving a file, in most cases, the description page on the origin wiki becomes unnecessary. Could a checkbox be added (like there is on Commons Helper) which will either delete the description page (if the user has the requisite rights), or marks it for deletion (e.g., by adding the "NowCommons" template)? -Pete Forsyth (talk) 19:12, 1 February 2019 (UTC)

Johanna Strodt (WMDE) (talkcontribs)

@Peteforsyth Thanks a lot. There's a similar request on Phabricator (T196735), and I've added your feedback there.

Best,

Johanna

Project to import previously manually transferred files

2
Summary by Johanna Strodt (WMDE)
Rehman (talkcontribs)

Hello. Once the FileImporter passes all testing and becomes a fully enabled feature, are there any plans to [mass] import all of the historic content that was previously transferred without history (i.e. FAQ: What if I want to import a file to Commons that has been imported before (without the history)?

I understand that manually doing this would be almost impossible, mostly due to conflicting edit histories and having to painstakingly select correct history versions over thousands of files. Thus may I suggest that a simple tool be created (for admins) where we could:

  1. Compare both histories of localwiki and commons side by side, including deleted versions
  2. Select versions to undelete/transfer/delete, with the ability to undelete and delete files on the same screen/dashboard.
  3. Also with a AWB-style diff-comparison for descriptions and categories
  4. Maybe also features to support structured data.

Of course, we first need to figure out how to generate a list of files that were manually transferred before.

Just like me, I'm sure many of you feel the discomfort of knowing that many years worth of valuable edit history was lost when large numbers of files were manually transferred before this feature... I feel some level of effort should be put in at least creating a queue for the task, rather than depending on random people to select desired files to save history from... Just my thoughts. Best wishes from Colombo.

Johanna Strodt (WMDE) (talkcontribs)

Thanks @Rehman! There are no plans for such a feature yet. But I've created a Phabricator ticket so we can keep track of this request: T215008

Feel free to edit or comment there if I got something wrong.

Best,

Johanna

Hsarrazin (talkcontribs)

Hi, We have been trying to use it on frwikisource, for books that are now PD is US also :D

for configuration, I began by copying the enws file https://www.mediawiki.org/wiki/Extension:FileImporter/Data/fr.wikisource - but the configuration is quite tricky, and not easy to do...

we would like to automatically add the Book template, to add info... but, I do not understand how to do this :(

Johanna Strodt (WMDE) (talkcontribs)

@Hsarrazin Thanks for your feedback. I'm guessing you want to match the contents of the template:Book on frwikisource to the template:Book on Commons? And maybe do the same for template:Information?

Can you describe a bit more what it is that you'd like to do? An example file would also help.

Thanks a lot,

Johanna

Reply to "difficulties for configuration"

Kategorien hinzufügen

2
Summary by Thiemo Kreuz (WMDE)
Z thomas (talkcontribs)

Während des Imports habe ich die Möglichkeit, unter anderem den Dateinamen und die Beschreibung anzupassen.

Sinnvoll wäre es, auch gleich Kategorien hinzufügen zu können

Johanna Strodt (WMDE) (talkcontribs)

Hallo @Z thomas! Kleiner Nachtrag: Diese Frage habe ich jetzt auch in den FAQ ergänzt.

Viele Grüße

Johanna

Reply to "Kategorien hinzufügen"

Is "subst:" allowed in {{tl|Information}} template configuration?

4
Jarekt (talkcontribs)

I am trying to properly configure Extension:FileImporter/Data/en.wikipedia after there were requests to rewrite Information template on Commons to make it work with this extension. The issue is that w:en:Template:Information has some rarely used extra parameters as compared to c:Template:Information ("Location" and "Additional_information")and we would like not to loose their content. Traditionally when mapping from one template to another we use substitution, for example c:Template:Artwork/subst(information) can be used to map from c:Template:Information to c:Template:Artwork. I would like to use the same mechanism to handle w:en:Template:Information's "Location" and "Additional_information" so we can add them either to c:Template:Information's "description" field or to a separate field added using "Other fields" parameter. So can I use "Subst:" configuration file?

Johanna Strodt (WMDE) (talkcontribs)

Thanks for testing the feature and for your question, @Jarekt. Using "Subst:" currently isn't possible, but I've created a Phabricator ticket so T214365 we have this request on our radar.

Best,

Johanna

Jarekt (talkcontribs)

Johanna thanks for you reply. Is there another way one can use to map multiple fields in the source template into one?

Johanna Strodt (WMDE) (talkcontribs)

Hi @Jarekt. No, I'm afraid at the moment there's no way to merge multiple source parameters into one. I've updated the task description to make room for other solutions than "subst:". I hope that's alright with you.

Also, I've added your question to the FAQ, as I can imagine it's of interest for others, too.

Best,

Johanna

Reply to "Is "subst:" allowed in {{tl|Information}} template configuration?"

Option to delete file on transfer to Commons?

3
Summary by Johanna Strodt (WMDE)

Tracked in T196735

Sadads (talkcontribs)

So I am an admin on Enwiki, and after I do the import, I would like to be able to quickly fill out the deletion rational and delete the file that is unnecessarily hanging out on the local Wiki. Is there any way to toggle an option to go directly to the deletion interface? Otherwise, It takes me about 4 more gestures with my mouse to get to the Delete File interface. I am leaving the names on the files the same -- so it, in theory, should be something that could be made much more efficient.

Johanna Strodt (WMDE) (talkcontribs)

@Sadads Thanks for your comment! Currently, there's no way to do this, but this request has been raised before: T196735 I've added your comment there.

Have a happy weekend,

Johanna

Sadads (talkcontribs)
Deryck Chan (talkcontribs)

This is a great extension! I've started trying it out on yue.wp .

How do we localize the tool's messages? For example the name and description of the tool on Special:Preferences?

Thiemo Kreuz (WMDE) (talkcontribs)
Deryck Chan (talkcontribs)

(translatewiki:MediaWiki:Fileimporter-sourcetitleexists)

"The title you are currently trying to import to already exists on the source wiki."

Do you mean "destination wiki"?

Thiemo Kreuz (WMDE) (talkcontribs)
Deryck Chan (talkcontribs)

Thanks. I localized it with the meaning "The title you are trying to use is being used by another file on the source wiki."

Deryck Chan (talkcontribs)

yue and zh-hant all done! Thank you!
https://www.mediawiki.org/wiki/Help_talk:Extension:FileImporter
www.mediawiki.org
crawl=CC-MAIN-2019-09
391236
application/http; msgtype=response
sha1:AL6NMUGJIQMDJG242KOVZDZSXISX6PKQ
<urn:uuid:ff278efe-9cdb-46ec-984a-2ee34ec13a8a>
2019-02-18T12:22:20Z
208.80.154.224
text/html
sha1:PWVQGKGOULCIU5H6VQSLLZDIKOTNC6PN
<urn:uuid:2da001b3-2ac1-4278-8168-7916d29aa3c9>
https://www.mediawiki.org/wiki/Help_talk:Extension:FileImporter
response
<urn:uuid:f50ea406-97a7-4d99-b287-7b0d9f7abf31>
isPartOf: CC-MAIN-2019-09
publisher: Common Crawl
description: Wide crawl of the web for February 2019
operator: Common Crawl Admin (info@commoncrawl.org)
hostname: ip-10-141-122-116.ec2.internal
software: Apache Nutch 1.15 (modified, https://github.com/commoncrawl/nutch/)
robots: checked via crawler-commons 0.11-SNAPSHOT (https://github.com/crawler-commons/crawler-commons)
format: WARC File Format 1.1
conformsTo: http://iipc.github.io/warc-specifications/specifications/warc-format/warc-1.1/