Help:Converting
Converting a document from one format to another for Commons can be useful
- in order to make derivatives that are more readily accessible;
- because each format has its own advantages;
- if you need to have 2 or more books open at the same time, in different viewers.
So it can be a good idea have books in more than one format.
Converting PDF
[edit]- Although PDF documents are accepted by Commons, they can nevetheless be difficult to access.
Converting PDF to DJVU
[edit]See page Converting PDF to DjVu.
Converting PDF to images
[edit]See page Commons:Extracting_images_from_PDF#Extract_PDF_pages_as_images.
Extracting text from PDF
[edit]If the PDF contains the text in an easily extracted form, then use some of the following:
- Adobe Acrobat viewer's "Save as text"
- GSview's "Edit" / "Text extract..."
- STDUViewer's menu item File -> Export -> to text
- XPdf command line tools pdftotext, pdftohtml.
Otherwise, if the PDF has text as images:
- Follow the advice in "Converting PDF to images" above, then follow the advice in "Converting from image formats to text (OCR)" below.
Converting DjVu
[edit]Converting DjVu to other formats is useful because someone might not have a DjVu viewer installed, and other formats can be readily viewed in browser.
Converting DjVu to PDF
[edit]See page Help:Converting DjVu to PDF
Converting DjVu to images
[edit]Use DjVuLibre command line utilities ddjvu (DjVu decoder) or djvups (to convert to PostScript).
Extracting text from DjVu
[edit]WinDjView can do that or DjVuLibre command line tool djvutxt.
Converting images
[edit]Converting between image formats
[edit]- Use the (free for personal use) shareware IrfanView or XnView (and its command line tool NConvert), jpegcrop or the free software ImageMagick for advanced transformations
Processing images obtained from scanner
[edit]Images obtained from scanner usually require some processing before making a PDF or DJVU out of them: cropping, turning, splitting, reducing the size, converting to TIFF etc. The open-source application ScanTailor-Universal is designed for this purpose. It can be downloaded from the project's releases page.
Creating PDF from images
[edit]- img2pdf, an open-source command line program, is designed to convert images losslessly to PDF. It can also set metadata (such as the title and author) and how the resulting PDF file should be presented by a PDF viewing program.
The following command will take all files in the current folder and convert them into a single PDF named test.pdf with title and author metadata:
img2pdf --title "My First PDF" --author "Jack Example" --output test.pdf *
Note that this assumes the current directory does not contain non-image files or sub-folders. If all your source files are of a single type, such as JPEGs, you can specify *.jpg as the input instead. You can also specify multiple input files individually.
See img2pdf --help for everything img2pdf can do.
img2pdf is available from the Python Package Index and is also included in the repositories of many Linux distributions. A Windows executable is also available via the project's Appveyor.
- ImageMagick and GraphicsMagick can also be used to convert images to PDF files, if GhostScript is installed.
The following command will use ImageMagick's mogrify tool to convert all JPEG files to individual PDF files and place them in a subfolder named "pdf":
mogrify -format pdf -path pdf/ *.jpg
On some Linux distributions, the default ImageMagick security policy will block the program from handling PDF files. See this StackOverflow question for how to change the security policy.
Creating PDF from bitonal images
[edit]Bitonal images (ie. images that only contain a single shade of black and white) are a very efficient way of storing scanned documents that only contain text or other simple elements that only need two colors to be clearly represented. A high-quality bitonal text page is commonly only tens of kilobytes in size.
There are two bitonal compression methods used in PDF files, namely the CCITT Group 4 Fax compression and the JBIG2 compression. The latter is more efficient but has some perceived patent issues associated with it, resulting in JBIG2 encoding functionality often being missing or disabled in PDF creation software. However, it is sometimes possible to enable such functionality by installing the encoder yourself.
There are a number of tools for converting images to use bitonal compression. ScanTailor-Universal is an open-source tool for processing scanned pages that can output bitonal images. ImageMagick and GraphicsMagick are also able to do this with the -threshold option.
The following ImageMagick command converts all .jpg files in a folder into bitonal TIF files using Group 4 compression and places them into a folder named "bitonal":
mogrify -format tif -compress Group4 -path bitonal/ -threshold 50% *.jpg
Play around with the value of -threshold to find one that produces good results with your content.
JBIG2 conversion
[edit]OCRmyPDF is an open-source command line program mainly designed to add an OCR text layer to scanned PDF files. One of its additional features is its ability to optimize PDF files, which includes the conversion of other bitonal image formats to JBIG2. This requires the installation of the jbig2enc encoder. Compilation and installation instructions for Linux users are available here, and a third-party Windows executable is available here. The MacOS version packaged in Homebrew already includes jbig2enc.
See here for how to install OCRmyPDF on Windows. Many Linux distributions come with an OCRmyPDF package included in the repositories, though this may be outdated. OCRmyPDF is also available on pip.
The following command uses OCRmyPDF to add an OCR text layer to a PDF and arranges the PDF in a way that allows a web browser to start displaying it before it has been fully downloaded:
ocrmypdf --output-type pdf --fast-web-view 0 test.pdf test-OCR.pdf
OCRmyPDF by default losslessly optimizes the PDF, so bitonal images are automatically converted to JBIG2 if jbig2enc is installed.
If you want to skip the OCR process because you don't need to OCR the PDF, use --tesseract-timeout 0 to skip OCR altogether.
See OCRmyPDF's documentation for further usage examples.
Creating DjVu from images
[edit]Use DjVuLibre command line tools c44 (for color images) and cjb2 (for bitonal images) to convert images to single-page DjVu files, then join them with djvm.
Example c44 command line:
c44 [options] pnm-or-jpeg-file [djvufile]
The important options are -slice, -dpi and -mask. -slice sets the number of "slices" in each "chuck" (which are some tricky DjVu terms). Giving three numbers means there will be three "chunks", and four numbers will result in four chunks. These numbers seriously affect quality and size - experiment and check the documentation.
c44 -slice -slice 74+13+10 -dpi 150 pnm-or-jpeg-file [djvufile]
From the cjb2 bitonal encoder's self-explaining help:
Usage: cjb2 [options] <input-pbm-or-tiff> <output-djvu> Options are: -verbose Display additional messages. -dpi <n> Specify image resolution (default 300). -clean Cleanup image by removing small flyspecks. -lossy Lossy compression (implies -clean as well) -losslevel <n> Loss factor (implies -lossy, default 100) Encoding is lossless unless a lossy options is selected.
In many cases, -losslevel can be harmlessly set as low as 50.
For details, see DjVuLibre documentation.
Minidjvu allows to convert black & white single-page and multi-page TIFF, BMP and PBM to DjVu and reverse. It has a simple graphical shell. Its support for multi-page TIFF files is a notable advantage over DjVuLibre tools.
minidjvu 0.8 - encode/decode bitonal DjVu files
Usage:
single page encoding/decoding:
minidjvu [options] <input file> <output file>
multiple pages encoding:
minidjvu [options] <input file> ... <output file>
Formats supported:
DjVu (single-page bitonal), PBM, Windows BMP, TIFF.
Options:
-A, --Averaging: compute "average" representatives
-a <n>, --aggression <n>: set aggression level (default 100)
-c, --clean remove small black pieces
-d <n> --dpi <n>: set resolution in dots per inch
-e, --erosion sacrifice quality to gain in size
-i, --indirect: generate an indirect multipage document
-l, --lossy: use all lossy options (-s -c -m -e -A)
-m, --match: match and substitute patterns
-n, --no-prototypes: do not search for prototypes
-p <n>, --pages-per-dict <n>: pages per dictionary (default 10)
-r, --report: report multipage coding progress
-s, --smooth: remove some badly looking pixels
- DjVuSolo has the ability to create DjVu files.
- Some advanced scripts are available to do the job.
Converting from image formats to text (optical character recognition)
[edit]gImageReader
[edit]- gImageReader is an open-source application for extracting text from images and image-based PDF documents. It's able to automatically detect the layout of text and post-process the recognized text, including spell checking and removal of line breaks. It uses Tesseract for text recognition, and as such it supports any language that Tesseract does.
IrfanView
[edit]- Use IrfanView's Option menu "Start OCR" plugin (OCR is Optical character recognition) to extract the text. As of 2010-02-20 the KADMOS OCR plugin for Irfanview is limited to around six pages depending on your computer's free memory (about one gigabyte needed per 10 pages).
Pdf X-Change Viewer
[edit]- Use PDF-XChange Viewer's OCR function: Document -> OCR pages.
Tesseract engine
[edit]- Use the free Open Source Tesseract software (Linux, Mac OS X or Windows):
- download both the "tesseract" software and the "tessdata" language packs relevant to the languages or scripts appearing in the scanned document, and unpack them into the same folder; no installation is needed for the Windows executable
- obtain the highest resolution scan possible, and if necessary further enlarge the scan image (use fast resize, avoid resample filters) until characters are over 20 pixels high (experiment for best results) and save it (use ImageMagick or IrfanView)
- use the command tesseract IMAGEFILE OUTPUT
eScriptorium
[edit]- Use the free Open Source software eScriptorium with Kraken. The server component can be installed on Linux, macOS or Windows (with WSL). The client side is a web application which works in any modern web browser. It allows segmentation of text regions and lines and recognition of printed text and handwriting. Both segmentation and recognition can be trained.
Google has OCR. See:
- https://tools.wmflabs.org/ws-google-ocr/
- https://wikisource.org/wiki/Wikisource:Google_OCR
- https://cloud.google.com/vision/docs/ocr
In all cases, you may need to convert in sections. You will need to manually correct the generated text because the conversion is not perfect.
Converting video
[edit]Further help
[edit]You can ask for help at the Commons:Help desk.