index.md (2863B)
1 --- 2 title: "Extract Embedded Text from PDFs with Poppler in Ruby" 3 date: 2022-02-10T00:00:00+00:00 4 draft: false 5 canonical_url: https://www.viget.com/articles/extract-embedded-text-from-pdfs-with-poppler-in-ruby/ 6 --- 7 8 A recent client request had us adding an archive of magazine issues 9 dating back to the 1980s. Pretty straightforward stuff, with the hiccup 10 that they wanted the magazine content to be searchable. Fortunately, the 11 example PDFs they provided us had embedded text 12 content[^1], i.e. the 13 text was selectable. The trick was to figure out how to programmatically 14 extract that content. 15 16 Our first attempt involved the [`pdf-reader` 17 gem](https://rubygems.org/gems/pdf-reader/versions/2.2.1), which worked 18 admirably with the caveat that it had a little bit of trouble with 19 multi-column / art-directed layouts[^2], which was a lot of the content we were dealing 20 with. 21 22 A bit of research uncovered [Poppler](https://poppler.freedesktop.org/), 23 "a free software utility library for rendering Portable Document Format 24 (PDF) documents," which includes text extraction functionality and has a 25 corresponding [Ruby 26 library](https://rubygems.org/gems/poppler/versions/3.4.9). This worked 27 great and here's how to do it. 28 29 ## Install Poppler 30 31 Poppler installs as a standalone library. On Mac: 32 33 ``` 34 brew install poppler 35 ``` 36 37 On (Debian-based) Linux: 38 39 ``` 40 apt-get install libgirepository1.0-dev libpoppler-glib-dev 41 ``` 42 43 In a (Debian-based) Dockerfile: 44 45 ```dockerfile 46 RUN apt-get update && 47 apt-get install -y libgirepository1.0-dev libpoppler-glib-dev && 48 rm -rf /var/lib/apt/lists/* 49 ``` 50 51 Then, in your `Gemfile`: 52 53 ```ruby 54 gem "poppler" 55 ``` 56 57 ## Use it in your application 58 59 Extracting text from a PDF document is super straightforward: 60 61 ```ruby 62 document = Poppler::Document.new(path_to_pdf) 63 document.map { |page| page.get_text }.join 64 ``` 65 66 The results are really good, and Poppler understands complex page 67 layouts to an impressive degree. Additionally, the library seems to 68 support a lot more [advanced 69 functionality](https://www.rubydoc.info/gems/poppler/3.4.9). If you ever 70 need to extract text from a PDF, Poppler is a good choice. 71 72 [*John Popper photo by Gage Skidmore, CC BY-SA 73 3.0*](https://commons.wikimedia.org/w/index.php?curid=39946499) 74 75 76 [^1]: Note that we're not talking about extracting text from images/OCR; 77 if you need to take an image-based PDF and add a selectable text 78 layer to it, I recommend 79 [OCRmyPDF](https://pypi.org/project/ocrmypdf/). 80 81 [^2]: So for a page like this: 82 83 +-----------------+---------------------+ 84 | This is a story | my life got flipped | 85 | all about how | turned upside-down | 86 +-----------------+---------------------+ 87 88 `pdf-reader` would parse this into "This is a story my life got 89 flipped all about how turned upside-down," which led to issues when 90 searching for multi-word phrases.