About | How to: upload files | How to: edit files | Advanced tips and tricks | Training | Projects |
OpenRefine is a free and open source (FOSS) tool with which you can (batch) edit and upload filesonWikimedia Commons. OpenRefine focuses on adding and editing structured data. OpenRefile is available as a cloud service on PAWS to Wikimedia users.
This page collects information about OpenRefine for the Wikimedia Commons community.
You can download the latest stable version of OpenRefine from its website. OpenRefine's manual includes detailed installation instructions; make sure to read these.
Some users are unable to install OpenRefine because of, for instance, firewall issues, or because their organization or company does not allow users to install external software. In that case, you can use Wikimedia's cloud version of OpenRefine on PAWS, which is described elsewhere on this page.
OpenRefine supports all kinds of media files that can be uploaded to Wikimedia Commons. It does not support the upload of data files.
Many upload tools, including OpenRefine, sometimes have trouble uploading TIFF files.
Please note that OpenRefine supports uploading local files up to 100MB, not larger. It is possible to upload larger files from URL though.
OpenRefine does not (yet) support Chunked Uploads, and hence only allows uploads of files from your local drive up to 100MB. See GitHub issue (help is very welcome to address and fix this issue). Uploading larger files to Wikimedia Commons is possible via URLs on the web. If that is not an option for you, please use Pattypan or the Upload Wizard.
OpenRefine can easily handle datasets of up to tens of thousands (potentially hundreds of thousands) of rows of data. The bottleneck is the speed of uploading files to Wikimedia Commons, which is regulated by the Wikimedia Commons API. For an upload of thousands of files at once (or more), you will need some patience and you will need to keep OpenRefine open.
This is not possible inside OpenRefine. We recommend using EXIFtool https://exiftool.org. This YouTube video explains the process quite clearly.
When running OpenRefine locally (on your own computer):
When running OpenRefine in the cloud (via Wikimedia PAWS):
250,627 files have been uploaded with OpenRefine.
Open Refine
in the category
Eggbeater
This online course is available at any time, for free, for anyone with a Wikimedia account. It can be followed at your own pace, with computer-graded exercises. Following the course takes an average of 6 to 8 hours.
Start the WikiLearn course: OpenRefine for Wikimedia Commons: the basics.
For uploading files to Wikimedia Commons, you need OpenRefine 3.7. Wikimedia Commons upload is not supported in OpenRefine 3.6 or earlier versions.
Follow the step by step guide (how-to) to edit Wikimedia Commons files with OpenRefine.
For editing Wikimedia Commons, you need OpenRefine 3.6 or newer. Wikimedia Commons is not supported in OpenRefine 3.5 or earlier versions. It is highly recommended to also install OpenRefine's Wikimedia Commons extension.
Follow the step by step guide (how-to) to edit Wikimedia Commons files with OpenRefine.
There is also a page with advanced tips and tricks, which include more instructions on working with manifests and reconciliation, retrieving EXIF, special GREL recipes, and more. Add your own!
OpenRefine can be downloaded as an application and works on desktop and laptop computers with Windows, Mac and Linux operating systems. It runs a small server on your computer and you then use a web browser to interact with it. It works best with browsers based on Webkit, such as Google Chrome, Chromium, Opera and Microsoft Edge, and is also supported on Firefox.
You can download OpenRefine here. Installation instructions are available in OpenRefine's user manual.
Additionally, you can also install OpenRefine's Wikimedia Commons extension. This is not necessary, but helpful for Wikimedia Commons batch editing. It offers:
Download and installation instructions are available at https://github.com/OpenRefine/CommonsExtension
If you are unable to install OpenRefine on your computer, or if it runs very slowly, then you can also use it in the cloud (on wmcloud.org through PAWS). Everyone with a Wikimedia account can access OpenRefine here. Visit https://hub-paws.wmcloud.org/, log in, and click on the OpenRefine (blue diamond) logo.
The Wikimedia Commons extension (mentioned above) is installed in OpenRefine on PAWS. Please note: with OpenRefine on PAWS it is NOT possible to upload files to Wikimedia Commons from your local computer.
Demo: start OpenRefine on Wikimedia PAWS
OpenRefine via PAWS is not managed by the OpenRefine team; if you have any questions or comments about this service, you can submit a ticket in the PAWS project on Phabricatororask a question on its talk page.
When | Activity | Links |
---|---|---|
July 2023-June 2024 | OpenRefine and Wikimedia Commons training and sustainability project (funded from Wikimedia Foundation funds) | |
August 13, 2022 | Tutorial: Batch uploading to Wikimedia Commons with OpenRefine at Wikimania 2022 | Etherpad / Video recording |
June 9, 2022 | OpenRefine and SDC editing tutorial, Wikidata Lab XXXIV | Video recording |
May 19, 2022 | One hour demo for beginners: Wikimedia Commons batch editing with OpenRefine (tutorial by Sandra Fauconnier), during Image Description Week | |
March–June 2022 | Monthly OpenRefine office hours | No notes/recordings (the meetings were informal) |
February 22, 2022 | OpenRefine community meetup with demo of Structured Data on Commons functionalities | Slides and meeting recording |
July 2021 – October 2022 | Development of Wikimedia Commons features for OpenRefine (funded by a Wikimedia Foundation Project Grant) |
|
| ||
---|---|---|
Features |
| |
Tools |
| |
About |
| |
Get involved |
| |
GLAM |
| |
Modeling |
| |
Discussion |
| |
In other projects |
| |
Related |
| |
|