User talk:Zhuyifei1999 Voyage Tips and guide

Contents

To help get you started contributing, we've created a tips for new contributors page, full of helpful links about policies and guidelines and style, as well as some important information on copyleft and basic stuff like how to edit a page. If you need help, check out Help, or post a message in the travellers' pub. If you are familiar with Wikipedia, take a look over some of the differences here. --Saqib (talk) 09:46, 11 October 2013 (UTC)Reply

I'm actually not the real maintainer of that, User:Mey2008 generates the information from data dumps, and I think he just went on vacation. However if you already do have your own list of items, I would be happy to add them, or even add you as a maintainer as well. We're still sorting out coordination and access problems, and maybe everything will shift to Meta. It would be great to see the Chinese Wikivoyage up and running, as I'm certain it's a pretty popular hobby. -- torty3 (talk) 23:56, 11 October 2013 (UTC)Reply

Hello Zhuyifei1999, why did you remove the redirect of Albina? This article was deliberately redirected to the region East Coast (Suriname) to avoid many small articles. If you revoke it, please remove the rest of the content. Thanks, --WiDi (talk) 18:58, 30 November 2013 (UTC)Reply

Sorry, I may be wrong. But the redirect was categorized, and CatScan2 counted it as an article while it isn't. Therefore I removed the redirect. --Zhuyifei1999 (talk) 03:26, 1 December 2013 (UTC)Reply

No problem. Maybe because it has content as it should be blank. I will remove the content and restore the redirect. Greetings. --WiDi (talk) 11:07, 1 December 2013 (UTC)Reply

Thanks. ;) --Zhuyifei1999 (talk) 12:32, 1 December 2013 (UTC)Reply

Hi! I have got access to tools.wmflabs. Now could you please share access to the scripts related to Wikivoyage? --Alexander (talk) 19:39, 24 May 2014 (UTC)Reply

Done You can now "$ become wikivoyage" ;) --Zhuyifei1999 (talk) 02:14, 25 May 2014 (UTC)Reply

• Maybe I'm too overzealous about reverting spam and misedits here in Wikivoyage, and was too hasty on blocking you. I have read your email and will listen to your reasoning. Ibaman (talk) 20:05, 4 August 2017 (UTC)Reply

  This will be somewhat of a long post, please give me some time. --Zhuyifei1999 (talk) 20:14, 4 August 2017 (UTC)Reply

In 2012, w:Wikipedia Zero program started in an attempt to provide free internet to some countries via zero rating, so people can gain access to knowledge for free. It was an quite successful, but it backfired.

In 2015, Wikimedia Commons had a significant increase in copyright violations. CheckUser / Outreach revealed they are from Angola, and the anti-abuse effort was tracked at c:User:Teles/Angola_Facebook_Case. the abuse slowly died out (Gunnex and Teles knows more), but eventually, other WP0 countries found their way to Wikimedia Commons as well.

In mid-2016, pirates from another WP0 country, Bangladesh, found their Wikimedia and started to upload videos to Commons. NahidSultan, a steward from Bangladesh noticed this and tracked the piracy at c:User:NahidSultan/Bangladesh_Facebook_Case. Around the same time, Steinsplitter, another Commons admin, and AlexZ, a WMF-er, had a chat on IRC regarding this abuse; it led to Steinsplitter's opening of Phabricator ticket phab:T129845 in Jul 2, in order to gain attention of WMF. You may have seen this ticket linked at a lot of places, but before July 2 of this year, this ticket was publicly visible. I'll elaborate on the events that make it private later.

As I have explained, the pirates are largely from WP0 countries with free access to the internet, but they did not choose Piratebay or other sites for piracy; they chose Wikimedia. Why? Because Wikipedia Zero only affects Wikimedia Production Sites, such as Wikipedia, Wikivoyage, Commons, Phabricator. After they found themselves blocked, they continued piracy using VPNs / other proxies despite costing money; why? Because they don't pirate for themselves; they pirate for the downloaders.

A similar project as Wikipedia Zero, w:Facebook_Zero, was a project started in 2010, and it is the project Wikipedia Zero tried to be like. These pirates, uses Facebook, with free access via Facebook Zero, and found that pirating videos on Wikimedia is a great and easy way to gain popularity on Facebook. Wikimedia admins are essentially playing whack-a-mole on their hundreds or even thousands of sockpuppets. Seeing us failing to prevent piracy, at the same time gaining more and more likes, was a great boost to their ego.

In from late 2015 to early 2016, out of request by another Commons admin Yann, I created a tool on Toolforge (a.k.a. Tool Labs), called videocommons, for the ease of transferring freely licensed or public domain videos from YouTube or other video-sharing sites (eg. Vimeo, Dailymotion) to Commons, in order to enrich Commons' video content. This was after Angola pirates died down, before Bangladesh case and before I became aware of Wikipedia Zero. It was a great success (even till now), and many users started to use this tool (there are competitors), but it again backfired when Bangladesh pirates started using it as well.

I do not remember how I started to understand the pirates, but I do started to notice a significant increase in redlink rate in the uploads log (i.e. Recent Changes, filtered by the tag). After phab:T131211 AbuseFilter started to identify uploads from WP0 IP ranges, I requested the same for tools uploads in phab:T131934. It turns out, however, that Wikimedia Labs was not zero rated, and the pirates do actually pay for the piracy. Unaware of a good way to stop the pirates, I didn't do much besides deleting their uploads and blocking accounts, but the piracy continued.

In April 19 of last year, Nahid, also a member of Wikimedia Bengladesh, posted that the Chapter is aware of the issue and will actively try to resolve it. They also had a post on their Facebook page, but in their own language that I could not understand (I did however do a Google Translate, but I no longer remember the details). The piracy almost immediately slowed down, but press coverage came.

In April 28, a person posted on motherboard.vice.com, explicitly supporting the piracy, and the pirates rekindled. The number of sockpuppet accounts recorded increased, from 300, to 400, to 500; it just seemed endless.

... Until we observed that the pirates, once used many tools for piracy (UploadWizard, Special:Upload, video2commons) became so obsessed with the convenience of video2commons, and the uploads of recorded socks are almost solely from video2commons. Being the creator/maintainer of the tool, I had to act, and on June 17 video2commons was no longer available to them without a significant effort on their side (think about 4 days of sleeper account, and having to do 50 edits for each sockpuppet). Very quickly, I no longer hear reports of these pirates, and to my knowledge, they died down. Last related comment was on June 16 2016, from Gunnex, comparing the number of socks (291 on April 16, 599 on June 8, and 676 on June 16 at the time of posting), implying some immediate action was needed. Next comment was MarkTraceur asking for an update on Dec 5 2016.

On 28 November 2016, Storkk, a Commons admin, posted on Commons Administrator's Noticeboard about his observation of files that are way too big for their dimensions. Analysis began, and we realized that you can simply hide arbitrary files in a JPEG by compressing it with RAR and byte-appending the RAR to the JPEG. Commons admin Jdx implemented a filter that sets a threshold for "way to big", then I came up with an idea December 5, and proposed some automatic non-statistics-based detection mechanism to find these embedded data in new uploads. It was not fruitful at the time.

While the filter for JPEG was rather successful in its threshold, problems began when the Myanmar started to switch formats. First to PNG, then to GIF, then to OGG/XCF/WEBM, etc. that the Abuse Filters simply do not provide enough information (variables) that we could set a reasonable threshold between "too big" or not. I asked again on 13:40, 28 December 2016 about the adminbot issue for automated deletion, as far as I remember, exchanged some emails with Commons B'crat Krd about the use of adminbots. I did not wait for an answer from the community due to the initial negative response in early December; I coded and did some initial testing for the bot in two days, and on December 29, a bot request for permissions was filed, immediately gaining likes. Due to posting a lot of technical information, Krd then warned me in email to keep technical stuffs in private cannot find a way to workaround the bot so easily. They did, unfortunately, so many of the work later for me later on the maintenance of the bot, and finding ways to not get the pirates workaround so easily.

On January 19, the bot was approved as adminbot and started to delete files within a minute of upload, this forced these "Embedded Data" pirates from Myanmar to quit.

On a side note: Bangladesh case came back on December. I'm not very aware of this second Bangladesh Case; Nahid knows more.

On March 26, Commons CheckUser posted that there was another abuse and they "now count Wikimedia Zero projects in 4 different countries as having coordinated file sharing campaigns on Commons." It is yet unclear to me what 4 countries they meant. On March 27, TJH2018 notified on AN that WP0 pirates are back, using Myanmar's "Embedded Data" method. I do not remember where, but it was investigated that these pirates belong to Moracco. On April 4, due to my speculations of the pirates monitoring us, I had an email exchange with Jdx and started phab:Z567, at the time semi-private, visible to whoever subscribed to the at-the-time-public phab:T129845. Reports on Z567 was often on new ways pirates are attempting to abuse the bot, and we gained contact with m:Wikimedia MA User Group in hope of them doing something similar to Wikimedia Bangladesh Chapter to relieve our whack-a-mole workload. That effort was not fruitful.

During some time later, Angola pirates came back, perhaps at the same time as Magog the Ogre's comment, perhaps not. We also see a lot of Bangladesh-case-style video piracy, except this time they are not using my tool. They are highly likely to have patterns (eg. "news" in file name). This provoked us to restrict video uploading, which was implemented with filter 180 and was switched to "disallow" instead of "log" on July 17. Another filter, 179, which matched against some patterns and checksums, are also in effect since June 8.

Due to the increasing complexity of of our defence systems, the pirates eventually started to find other wikis to abuse. Testwiki and Test2wiki was first, started during the Bangladesh-era. Then not long ago, Hebrew Wikipedia and Hungarian Wikipedia became abused. And now it has spread everywhere. Tbayer, an WMF-er, and Dispenser, who does most of our data analysis, may know more. Dispenser, in particular, coded an IRC bot that actively scans various wikis for audio/video uploads from newbies. Since June 22, Tbayer also do daily queries at phab:Z591 (so Z567 do not get flooded) of which files WP0 users are accessing, with his access to Private data at WMF cluster. I myself use these information to go to various wikis to tag almost-definitely-abusive uploads with {{delete|[[phab:T129845|WP0 abuse]]}}, and reporting various pirate accounts to stewards for CU/locking when necessary. (The one you blocked was locked after my report.)

One notable spread was the spread to Phabricator on June 17 as tacked in phab:T168142. This led to the hide of T129845 by a steward on July 2 and Z567 on June 17.

With increased attention, WMF is giving more and more attention to this issue. Recently, Wikimedia Community Liaisons team started to work on this issue.

T167400: Disable serving unpatrolled new files to Wikipedia Zero users shall invalidate the pirates' underlying motivation.

I hope this explains whatever I know. enwikivoyage is just yet another wiki being abused. We are fighting this abuse, but whack-a-mole is ineffective against thousands of sockpuppets across dozens of abuse-able wikis. If you are willing to join our effort, I would very much welcome you. And if you want access to any of the mentioned Phabricator tickets and/or discussions, please tell me your Phabricator username and I'll be happy to grant you access (in fact I'd be happy to grant any trusted people access so they can join our effort). --Zhuyifei1999 (talk) 22:28, 4 August 2017 (UTC)Reply

oh and courtesy ping to @Ibaman: to note that this is finished in case you want to read this long wall of text (I don't do this often but whatever) :/ --Zhuyifei1999 (talk) 22:30, 4 August 2017 (UTC)Reply

Btw I understand your actions. Jdx and I often block some people related to this abuse before they do a single edit / upload. c:Special:Log/block/Jdx c:Special:Log/block/Zhuyifei1999. Good part about wikis is that errors can be undone, and it's not the first time I was erroneously blocked (the other time was in 2013). --Zhuyifei1999 (talk) 22:38, 4 August 2017 (UTC)Reply

Wikivoyage has a very simple policy: no audio, no videos, images are only allowed when they are used in articles. Anyone uploading files without writing articles is an obvious suspect. Same would be true for the vast majority of Wikimedia projects. I sympathize with you, but I think that in this story WMF and Commons suffer from their own badly thought policies, and they deserve it in some sense. --Alexander (talk) 23:19, 4 August 2017 (UTC)Reply

Alexander: Many people only contributions are uploading pictures and other files. We are very thankful to them, as they are often good photographers. Personally, I rarely write articles nowadays. Most of my contributions are on multimedia (pictures, videos, etc.) by me or from public domain sources. Regards, Yann (talk) 00:03, 5 August 2017 (UTC)Reply

Yann, are you uploading multimedia files to Wikivoyage, or are you uploading photos to Commons and simply linking thumbnails here? Alexander is talking about people who upload multimedia files to Wikivoyage. That's not allowed, and we've had a good deal of trouble with multimedia-uploading vandals on this site in the last few months. I've banned a number of such accounts at sight, and I don't think I made any error. Ikan Kekek (talk) 00:13, 5 August 2017 (UTC)Reply

If that is per-policy, maybe you could implement an AbuseFilter to deny any non-image upload from newbies so we don't have to worry about enwikivoyage in a while. --Zhuyifei1999 (talk) 00:20, 5 August 2017 (UTC)Reply

Yes, it's per Wikivoyage:Image policy. I doubt that it's permanent, but it is currently and has forever, I think, been policy. However, I have no idea how to implement abuse filters. Ikan Kekek (talk) 00:37, 5 August 2017 (UTC)Reply

@Andrewssi2: --Zhuyifei1999 (talk) 00:40, 5 August 2017 (UTC)Reply

If "uploading files to Wikivoyage" is not allowed, why not turn off local uploads? Regards, Yann (talk) 11:06, 5 August 2017 (UTC)Reply

Images are only allowed when they are used in articles. It means that images are most welcome here, but they have to be relevant. Local uploads are the only way to avoid Commons monopoly on deciding which images are free and which are not. So, by all means, local uploads have to be allowed. --Alexander (talk) 16:39, 5 August 2017 (UTC)Reply

I guess the point I'm getting is that it would be good to block all uploads of video or audio files to Wikivoyage, since current policy is not to allow those. -- Ikan Kekek (talk) 23:31, 5 August 2017 (UTC)Reply

Hi Zhuyifei1999 , since you pinged me, how may I be of assistance here? --Andrewssi2 (talk) 03:42, 6 August 2017 (UTC)Reply

@Andrewssi2: Basically, WP0 pirates keeps uploading audio/videos for piracy. According to some admins here audio/video uploads are simply not allowed per-policy, so I was wondering if someone could implement an AbuseFilter to block all such uploads from newbies. I see many of the recent filters on Special:AbuseFilter are implemented by you, so maybe you can help? --Zhuyifei1999 (talk) 03:48, 6 August 2017 (UTC)Reply

I will need to see some examples of the abuse in order to create the strategy to prevent them. Looking at the previous cases above it seems that all their pages/files were deleted so it would be great to leave one up that can used as an example.

Just to clarify, videos are not allowed by policy. Images are sometimes allowed. --Andrewssi2 (talk) 03:58, 6 August 2017 (UTC)Reply

Unfortunately testing abuse filters in retrospect is difficult due to phab:T170249, so even if one or two uploads are not deleted they are next to useless for filter creation. One way to workaround is to setup a test filter and enforce the actions later when it seems ready. :( However, some of the public filters on Commons such as 154 156 162, and the details of Commons abuse log entries can be used as your reference when creating / debugging the filter.

Images are okay and out of scope of my request. The pirates rarely upload images for piracy. Mostly they are audio/videos, and sometimes (much rarer) PDFs. --Zhuyifei1999 (talk) 04:08, 6 August 2017 (UTC)Reply

@Andrewssi2: I've CC-ed you to phab:T129845 and phab:Z567 (feel free to unsubscribe if you don't like them), and copied our filter 179 and 180, which are most relevant to this WP0 abuse, to phab:Z567. --Zhuyifei1999 (talk) 04:18, 6 August 2017 (UTC)Reply

Sure, as the next step I will move this discussion to Wikivoyage_talk:Vandalism_in_progress . This is because I want the rest of community to have visibility of the potential filter, and review the behavior in order to ensure that it is consistent with policy and prior consensus. Andrewssi2 (talk) 05:10, 6 August 2017 (UTC)Reply