pandoc 2.18

Click to expand changelog

• New input formats: endnotexml (EndNote XML bibliography), ris (RIS bibliography).

• A RIS bibliography file may now be used with --citeproc.

• Citeproc: Allow a formatted bibliography to be placed in metadata fields via a Div with class refs (#7969, #526). Thus, one can include a metadata field, say refs, whose content is an empty div with id refs, and the formatted bibliography will be put into this metadata field. It may then be interpolated into a template using the variable refs.

• Ensure that you don’t get PDF output to terminal. -t pdf now behaves like -t docx and gives an error unless the output is redirected.

• --version now prints hslua version (#7929) and Lua version (#7997, Albert Krewinkel).

• Change --metadata-file parsing so that, when the input format is not markdown or a markdown variant, pandoc’s markdown is used (#6832, #7926). When the input format is a markdown variant, the same format is used. Reason for the change: it doesn’t make sense to run the markdown parser with a set of extensions designed for a non-markdown format, and this dramatically limits what people can do in metadata files.

• Trim whitespace from math in --webtex (#7892). This fixes problems with –webtex and markdown output, when display math starts or ends with a newline.

• New exported module Text.Pandoc.Readers.EndNote, exporting readEndNoteXML, readEndNoteXMLCitation, and readEndNoteXMLReferences. [API change]

• --self-contained: issue warning rather than failing with an error if a resource can’t be found (#7904).

• New exported module, Text.Pandoc.Readers.RIS, exporting readRIS (#7894).

• LaTeX reader:

  • Handle subequations as inline math environment (#7883).
  • Rudimentary support for vbox (#7939).
  • Support \today (#7905).
  • Handle \label and \ref for footnotes (#7930).
  • Allow inline groups starting with \bgroup (#7953).
  • Use custom TokStream that keeps track of whether macros are expanded. This allows us to improve performance a bit by avoiding unnecessary runs of the macro expansion code (e.g. from 24 ms to 20 ms on our standard benchmark).
  • Further optimizations for inline parsing.
  • Better handling of \usepackage. If the package is local but causes parse errors, parse everything up to the error and skip the rest. Issue a CouldNotParseIncludeFile warning indicating that parsing failed at that point.
  • Text.Pandoc.Readers.LaTeX.Parsing: Monoid and Semigroup instances for TokStream.

• HTML reader:

  • Give warnings and emit empty note when parsing <a epub:type="noteref"> and the identifier doesn’t correspond to anything in the note table (#7884). Previously we just silently skipped these cases.
  • Fix parsing of epub footnotes (#7884).

• DocBook reader:

  • Handle complete set of entities as specified at https://www.w3.org/2003/entities/2007doc/byalpha.html (#7938).
  • Handle abstract in info section (#7747).
  • Improve info parsing.
  • Simplify metadata parsing code (#7747). Handle abstract as block-level content. Report skipped info elements with --verbose.
  • Handle address and coyright in metadata (#7747).

• DokuWiki reader:

  • Add DokuWiki table alignment (#5202, damon-sava-stanley).

• RST reader:

  • Fix treatment of headerless simple tables (#7902).
  • Wrap math in Span to preserve attributes (#7998, Albert Krewinkel). Math elements with a name, classes, or other fields are wrapped in a Span with these attributes.

• JATS reader:

  • Improve handling of fn-group elements (#6348, Albert Krewinkel). Footnotes in <fn-group> elements are collected and re-inserted into the document as proper footnotes in the place where they are referenced.
  • Handle pub-date (#8000).
  • Support PMID, DOI, issue in citations (#7995).
  • Improve refs parsing. Handle issn and isbn; use simpler form for issued date.
  • Strip ‘ref-’ from ref id in constructing CSL id. This allows better round-tripping, because the JATS writer adds the ref- prefix to the citation id to get the ref element’s id.

• Org reader:

  • Allow “:” in property drawer keys (Lucas V. R). Any non-space character is allowed as property drawer key, including “:” itself (so it is not really a delimiter). The real delimiter is a space character, so in a drawer like

    :PROPERTIES:
    ::k:ey:: value
    :END:
    

    “:k:ey:” is a key with value “value”.

  • Allow comments above property drawer.

  • More flexible LaTeX environments (Lucas V. R).

  • Handle #+bibliography: as metadata so that it can work with --citeproc.

  • Parse #+print_bibliography: as Div with id refs.

  • Allow multiple #+bibliography:.

• Markdown reader:

  • Allow one-column pipe tables with pipe on right (#7919).
  • Remove restriction on identifiers, so they no longer need to begin with a letter (#7920).

• Docx reader:

  • Enable citations extension for docx reader (#7840). When enabled, Zotero, Mendeley, and EndNote citations embedded in a docx are parsed as native pandoc citations. (When disabled, the generated citation text and bibliography are passed through as regular text.) The bibliography generated by the plugin is suppressed. Instead, bibliographic data embedded in citation items is added to the references metadata field so that it can be used with --citeproc.

• Docbook writer:

  • Interpret links without contents as cross-references (#7360, Jan Tojnar). Links without text contents are converted to <xref> elements. DocBook processors will generate appropriate cross-reference text when presented with an xref element.

• Docx writer:

  • Single numbering ID for examples (#7895, mjfs). This change ensures that example list items all belong to a single number sequence, so that if items are added or deleted in a word processor, the other items will renumber automatically.
  • Add bookmark with table id to table (#7989, Nikolai Korobeinikov, #7285). This allows tables with ids to be linked to.

• Ipynb writer:

  • Handle metadata better (#7928). Previously we used the markdown writer to render metadata. This had some undesirable consequences (e.g. en dash expanded to -- when smart enabled), so now we use the plain writer.

• LaTeX writer:

  • Avoid extra space before \CSLRightInline (#7932).
  • Add scrreport to chaptersClasses (#6168, ivardb).
  • Support page,trim,clip attributes on images (#7181).
  • Add () after booktabs rules (#8001). These commands take optional arguments with () and [], which can lead to problems if the content of the table cell begins with these characters.

• RST writer:

  • Support all standard metadata (“bibliographic”) fields.

• HTML writer: performance improvements.

• Org writer:

  • Stop indenting property drawers, quote blocks (#3245, Albert Krewinkel). This follows the current default org-mode behavior.

• Markdown writer:

  • Move table-related code into submodule (Albert Krewinkel).
  • Don’t produce redundant header identifier when the gfm_auto_identifiers extension is set (#7941).
  • Update escaping rules for \. We now escape \ only if raw_tex is enabled or it is followed by a non-alphanumeric.

• JATS writer:

  • Encode author “others” as <etal/> (Albert Krewinkel). Citeproc adopted the BibTeX convention to use the author name “others” when there are additional authors that are not named. JATS uses the <etal> element for this.
  • Avoid doubled ref-list element (#7990). Previously when generating JATS with the element_citations extension enabled, the references were put in a doubly-nested ref-list element (<ref-list><ref-list>...).
  • Keep edition info in element citations (#7993, Albert Krewinkel).
  • Fix handling of CSL variable ‘page’ (not ‘pages’ as we had before). It should go to ‘lpage’ and ‘rpage’, not ‘page-range’.

• EPUB writer: refactor for clarity (#7991, Jonathan Dönszelmann, Ola Wolska, Ivar de Bruin, Jaap de Jong).

• Custom writer (Albert Krewinkel):

  • Support new-style Writer function (Albert Krewinkel). See the documentation for custom writers for details.
  • Produce stacktrace if Writer function fails

• Text.Pandoc.Logging: add CouldNotParseIncludeFile constructor for LogMessage [API change].

• Text.Pandoc.Shared:

  • Put id attributes on TOC entries (#7907, damon-sava-stanley). Naming scheme of id is “toc-” + id of linked to header/section. Effects HTML, Markdown, Powerpoint, and RTF.
  • Define ordNub as alias for nubOrd from containers package (#7963, Albert Krewinkel).
  • Export ensureValidXmlIdentifiers. This function changes identifiers that don’t start with letters, and internal links to these identifiers, making them compatible with XML standards. The change is simple: we add id_ to the front. There is potential for duplication if there are already id_... identifiers defined, but this seems rare enough not to worry too much about.

• Ensure that valid XML identifiers are used in Docbook, EPUB, FB2, HTML4, S5, Slidy, Slideous, ICML, ODT, TEI writers. Thus, if you convert [anchor]{#1} and [link to](#1), id_1 will be used instead of 1 for the identifier.

• Lua (Albert Krewinkel).

  • Add module pandoc.layout to format and layout text.
  • Move custom writer code into Lua hierarchy.
  • Use pandoc-lua-marshal 0.1.5.
  • Allow any type of callable object as argument to List functions filter, map, and find_if. These previously required the argument to be of...

pandoc 2.17.1.1

Click to expand changelog

• Fix regression in 2.17.1 which caused problems finding default files in the default user data directory. (Reverts the item “logic bug in fullDefaultsPath”, which was misguided.)

• Sample custom writer: use single quotes for strings (#7487, Albert Krewinkel).

pandoc 2.17.1

Click to expand changelog

• Support pagedjs-cli as pdf engine (#7838, Albert Krewinkel). PagedJS is a polyfill and supports the Paged Media standards by the W3C. https://www.pagedjs.org/

• CommonMark reader: fix source position after YAML metadata (#7863).

• LaTeX reader:

  • Remove retokenizing in rawLaTeXParser.

  • Ensure that \raggedright doesn’t gobble an argument (#7757).

  • Improve descItem. For some reason we were skipping arbitrary blocks before \item. This is now changed to “skip whitespace and comments.”

  • Improve handling of \newif. Adding a pair of braces around the second argument of \def prevents LaTeX from an emergency stop on input like the following (#6096).

    \newif\ifepub
    \epubtrue
    \ifepub
    hi
    \fi
    

• Docx reader: Parse both Zotero citation and bibliography as FieldInfo (#7840).

• LaTeX writer:

  • Allow arbitrary frameoptions to be passed to a beamer frame, using the frameoptions attribute (#7869).
  • Add s and squeeze to recognized beamer frameoptions (#7869).

• Markdown writer: handle explicit column widths with pipe tables (#7847). If a table has explicit column width information and the content extends beyond the --columns width, we need to adjust the widths of the pipe separators to encode this width information.

• Docx writer: Separate tables even with RawBlocks between (#7224, Michael Hoffmann). Adjacent docx tables need to be separated by an empty paragraph. If there’s a RawBlock between tables which renders to nothing, be sure to still insert the empty paragraph so that they will not collapse together.

• Man writer: use custom font V for inline code (#7506). The V font is defined conditionally, so that it renders like CB in output formats that support that, and like B in those that don’t (e.g. the terminal). Aliases also defined for VI, VB, VBI.

• Asciidoc writer: Support checklists in asciidoctor writer (#7832, Nikolai Korobeinikov, ricnorr). The checklist syntax (similar to task_list in markdown) seems to be an asciidoctor-only addition.

• HTML writer:

  • Avoid duplicate “style” attributes on table cells (#7871).
  • Don’t break lines inside code elements. With the new (default) line wrapping of HTML, in conjunction with the default CSS which includes code { whitespace: pre-wrap; }, spurious line breaks could be introduced into inline code (#7858).

• Custom writer: preserve order of element attributes (#7489, Albert Krewinkel). Attribute key-value pairs are marshaled as AttributeList, i.e., as a userdata type that behaves both like a list and a map. This allows to preserve the order of key-value pairs.

• Switch to hslua-2.1 (Albert Krewinkel). This allows for some code simplification and improves stability.

• Don’t read files outside of user data directory (Even Brenden). If a file path does not exist relative to the working directory, and it does exist relative to the user data directory, but outside of of the user data directory, do not read it. This applies to readDataFile and readMetadataFile in PandocMonad and, by extension, any module that uses these by passing them relative paths.

• Text.Pandoc.Class.makeCanonical: Correctly handle consecutive “..”s at the beginning of a path (Even Brenden). Prior to this commit, ../../file would evaluate to file, when it should be unchanged.

• Search for metadata files in $DATADIR/metadata (#7851, Even Brenden). If files specified with --metadata-file are not found in the working directory, look in $DATADIR/metadata (#5876).

• Text.Pandoc.Class: export readMetadataFile [API change] (#5876).

• Text.Pandoc.Error: export new PandocCouldNotFindMetadataFileError constructor for PandocError [API change] (#5876).

• Avoid putting a frame around speaker notes in beamer (#7857). If speaker notes (a Div with class ‘notes’) occur right after a section heading, but above slide level, the resulting \note{..} caommand should not be wrapped in a frame, as that will cause a spurious blank slide.

• CSS in HTML template: adjust #TOC and h1 on mobile (#7835, Mauro Bieg).

• Text.Pandoc.Readers.LaTeX.Parsing: don’t export totoks. Make the first param of tokenize a SourcePos instead of SourceName, and use it instead of totoks.

• Text.Pandoc.Shared: Modify stringify so it ignores [Citation] inside Cite (#7855). Otherwise we’ll sometimes get two copies of things, one from the citationPrefix or citationSuffix and another from the embedded fallback text. When there is no fallback text, we’ll get no content. However, it really isn’t an alternative to just rely on the result of running query on the embedded Citations; this will result in a jumble of text rather than anything structured.

• Omit --enable-doc in the cabal haddock invocation in tools/build-and-upload-api-docs.sh.

• Text.Pandoc.App.Opt: fix logic bug in fullDefaultsPath. Previously we would (also) search the default user data directory for a defaults file, even if a different user data directory was specified using --data-dir. This was a mistake; if --data-dir is used, the default user data directory should not be searched.

• Text.Pandoc.Shared: defaultUserDataDir behavior change (#7842). If the XDG data directory is not defined (e.g. because it’s not supported in the OS or HOME isn’t defined), we return the empty string instead of raising an exception.

• Update command tests to distinguish stderr and test exit status.

• MANUAL: add that speaker notes can be used with beamer (#7856).

• Update build-and-upload-api-docs.sh.

• Document --trace option. Document no-check-certificate in defaults files. Document ‘sandbox’ option for defaults files. (#7873).

• Fix pattern syntax in sample readability custom reader.

• doc/custom-readers.lua: add example for “readable HTML.”

• Fix message in man page about where code can be found.

• manfilter.lua: remove extra indent in table cells with code blocks.

• Fix lua-filters documentation for table column widths (#7864).

• epub.doc: Update links to KindleGen (#7846, Benson Muite, Mauro Bieg). KindleGen has been deprecated and we need to link to archived versions.

• Use tables in defaults files documentation, so each default option is paired with the corresponding command-line option (Carsten Allefeld).

• Use skylighting 0.12.2.

• Add pandoc-lua-marshal to Nix shell (#7849, Even Brenden).

2.17.0.1

Click to expand changelog

• Require pandoc-lua-marshal 0.1.3.1 (#7831, Albert Krewinkel). Fixes a problem with List.includes and List.find that caused a Lua stackoverflow and subsequent program crash.

• HTML template: load header-includes before math (#7833, Kolen Cheung). MathJax expect the config comes before loading the MathJax script. This change of order allows one to config MathJax via header-includes, which loads before the MathJax script. Cf. #2750.

• When reading defaults file, stop at a line .... This line signals the end of a YAML document. This restores the behavior we got with HsYaml. yaml complains about content past this line. See #4627 (comment)

• Text.Pandoc.Citeproc: allow notes-after-punctuation to work with numerical styles that use superscripts (e.g. american-medical-association.csl), as well as with note styles. The default setting of notes-after-punctuation is true for note styles and false otherwise. This restores a behavior of pandoc-citeproc that wasn’t properly carried over to Citeproc (#7826, cf. jgm/pandoc-citeproc#384).

• Use commonmark-pandoc 0.2.1.2 (#7769).

• Add FAQ on images in ipynb containers (#7749, Kolen Cheung).

pandoc 2.17

Click to expand changelog

• Support markua as an output format (#1871, Tim Wisotzki and Saumel Lemmenmeier). Markua is a markdown variant used by Leanpub.

• Add text wrapping for HTML output (#7764). Previously the HTML writer was exceptional in not being sensitive to the --wrap option. With this change --wrap now works for HTML. The default (as with other formats) is automatic wrapping. Note that the contents of script, textarea, and pre tags are always laid out with the flush combinator, so that unwanted spaces won’t be introduced if these occur in an indented context in a template.

• Don’t read sources until in/out format are verified (#7797).

• Issue error with --list-extensions for invalid formats (#7797).

• Make --citeproc recognize .yml as well as .yaml extensions as YAML bibliography files (#7707, Jörn Krenzer).

• Use latest version of KaTeX with --katex.

• Fix parsing of footnotes in --metadata-file (#7813). Previously non-inline footnotes were not being parsed.

• ODT reader:

  • Parse list-header as a list item (Tuong Nguyen Manh).

• Commonmark reader:

  • Put sourcepos attribute on header, not enclosing div with -f commonmark+sourcepos (#7769).

• Markdown reader:

  • Don’t allow ^ at beginning of link or image label (#7723). This is reserved for footnotes. Fixes regression from 0a93acf.
  • Fix parsing of “bare locators” after author-in-text citations. Previously @item [p. 12; @item2] was incorrectly parsed as three citations rather than two. This is now fixed by ensuring that prefix doesn’t gobble any semicolons.
  • Revert changes to inlinesInBalancedBrackets (commit fa83246), which caused regressions.
  • Improve detection of pipe table line widths (#7713). Fixed calculation of maximum column widths in pipe tables. It is now based on the length of the markdown line, rather than a “stringified” version of the parsed line. This should be more predictable for users. In addition, we take into account double-wide characters such as emojis.

• Custom (Lua) readers:

  • First argument is now a list of sources instead of the concatenated text (Albert Krewinkel). The list structure can easily be converted to a string by applying tostring, but it is also possible to access the elements (each with a text and name). A small example is added to the custom reader documentation, showcasing its use in a reader that creates a syntax-highlighted code block for each source code file passed as input. Existing readers will still work through a fallback mechanism, issuing a deprecation notice.

• Org reader:

  • Parse official org-cite citations (#7329). We also support the older org-ref style as a fallback. We no longer support the “markdown style” or “Berkeley style” citations.
  • Support alphabetical (fancy) lists (Lucas Viana). When the fancy_lists extension is enabled, alphabetical list markers are allowed, mimicking the behaviour of Org Mode when org-list-allow-alphabetical is enabled.
  • Support counter cookies in lists (Lucas Viana). Such cookies are used to override the item counter in ordered lists. In org it is possible to set the counter at any list item, but since Pandoc AST does not support this, we restrict the usage to setting an offset for the entire ordered list, by using the cookie in the first list item.
  • Allow trailing spaces after key/value pairs in directives (Albert Krewinkel). Ensures that spaces at the end of attribute directives like #+ATTR_HTML: :width 100% (note the trailing spaces) are accepted.

• LaTeX reader:

  • Omit visible content for \label{...}. Previously we included the text of the label in square brackets, but this is undesirable in many cases. See discussion in #813 (comment).
  • Improve references (#813). Resolve references to theorem environments. Remove the Span caused by “label” in figure, table, and theorem environments; this had an id that duplicated the environments’ id.
  • Fix semantics of \ref. We were including the ams environment type in addition to the number. This is proper behavior for \cref but not for \ref. To support \cref we need to store the environment label separately.
  • Add babel mappings for Guajati (gu) and Oriya (or) (#7815).
  • Fix typo panjabi -> punjabi in babel mappings (#7814).

• HTML reader:

  • Parse attributes on links and images (#6970).

• Docx reader:

  • Handle multiple pic elements inside a drawing (#7786).
  • Change elemToParPart to return [ParPart] instead of ParPart. Also remove NullParPartconstructor, as it is no longer needed. This will allow us to handle elements that contain multiple ParParts, e.g. w:drawing elements with multiple pic:pic.

• DocBook reader:

  • Collapse internal spaces in literal and other similar tags (#7821), as the standard docbook toolchain does.
  • Be sensitive to spacing=“compact” in lists (#7799). When spacing="compact" is set, Para elements are turned into Plain, so we get a “tight” list.

• Markdown writer:

  • Add new exported function writeMarkua from Text.Pandoc.Writers.Markdown [API change] (#1871, Tim Wisotzki and Saumel Lemmenmeier).
  • Fix indentation issue in footnotes (#7801).
  • Avoid extra space before citation suffix if it already starts with a space.
  • Ensure semicolon between the locator and the next citation when an author-in-text citation has a locator and following citations.
  • Improve escaping for # (#7726).

• Custom (Lua) writers:

  • Allow variables to be set via second return value of Doc (#6731, Albert Krewinkel). New templates variables can be added by giving variable-value pairs as a second return value of the global function Doc. Example:

    function Doc (body, meta, vars)
      vars.date = vars.date or os.date '%B %e, %Y'
      return body, vars
    end
    

  • Provide global PANDOC_WRITER_OPTIONS (#6731, Albert Krewinkel).

  • Assign default Pandoc object to global PANDOC_DOCUMENT (Albert Krewinkel). The default Pandoc object is now non-strict, i.e., only the parts of the document that are accessed will be marshaled to Lua. A special type is no longer necessary. This change also makes it possible to use the global variable with library functions such as pandoc.utils.references, or to inspect the document contents with walk().

• LaTeX writer:

  • Fix typo panjabi -> punjabi in babel mappings (#7814).

• MediaWiki writer:

  • Remove redundant display text for wiki links (Jesse Hathaway).

• Docx writer:

  • Handle bullets correctly in lists by not reusing numIds (#7689, Michael Hoffmann). This fixes a bug in which a Div in a list item would receive bullets on its contained paragraphs.

• Org writer:

  • Fix list items starting with a code block or other non-paragraph content (#7810).
  • Avoid blank lines after tight sublists (#7810).
  • Fix extra blank line inserted after empty list item (#7810).
  • Don’t add blank line before lists (#7810).
  • Support starting number cookies (Lucas Viana). This is necessary for lists that start at a number other than 1.
  • Support the new org-cite syntax (#7329).

• Haddock writer:

  • Avoid blank lines after tight sublists (#7810).

• Ipynb writer:

  • Ensure deterministic order of keys.
  • Handle cell output with raw block of markdown (#7563, Kolen Cheung). Write RawBlock of markdown in code-cell output. This is designed to fit the behavior of #7561, which makes the ipynb reader parse code-cell output with mime “text/markdown” to a RawBlock of markdown. This commit makes the ipynb writer writes this RawBlock of markdown back inside a code-cell output with the same mime, preserving this information in round-trip.
  • In choosing between multiple output options, always favor those marked with the output format over images (Kolen Cheung). Previously, both fmt == f case and Image have a rank of 1.

• Ipynb reader & writer: properly handle cell “id” (#7728). This is passed through if it exists (in Nb4); otherwise the writer will add a random one so that all cells have an “id”.

• Ms writer:

  • Properly encode strings for PDF contents (#7731).

• JATS writer:

  • Keep quotes in element-citations (Albert Krewinkel). Fixed a bug that lead to quote characters being lost in element-citations.

• RTF writer:

  • Properly handle images in data URIs (#7771).

• Commonmark writer:

  • Allow ‘)’ delimiters on ordered lists.

• RST writer:

  • Avoid extra blank line after empty list item (#7810).

• HTML writer:

  • Make line breaks more consistent. With --wrap=none, we now output line breaks between block-level elements. Previously they were omitted entirely, so the whole document was on one line, unless there were literal line breaks in pre sections. This makes the HTML writer’s behavior more consistent with that of other writers. Also, regardless of wrap settings, put newline after <dd> and after block-level elements in the footnotes section. And add a line break between an img tag and the associated figcaption.
  • reveal.js: Make sure images with r-stretch are not in p tags. They must be direct children of the section. There was previously code to make this work with the older class name stretch, but the name has changed in reveal.js.
  • reveal.js: don’t add r-fit-text class to section. It must go on the header only.

• AsciiDoc writer:

  • Improve detection of intraword emphasis (#7803).

• OpenDocument writer:

  • Fix vertical alignment bug with...

pandoc 2.16.2

Click to expand changelog

• Add interface for custom readers written in Lua (#7669). Users can now do -f myreader.lua and pandoc will treat the script myreader.lua as a custom reader, which parses an input string to a pandoc AST, using the pandoc module defined for Lua filters. A sample custom reader can be found in data/creole.lua. Also see documentation in doc/custom-readers.md.

• New module Text.Pandoc.Readers.Custom, exporting readCustom [API change].

• Allow plain to be used in raw attribute syntax.

• Accept empty --metadata-file (#7675). This was a regression from 2.15 behavior.

• Markdown reader: Improve inlinesInBalancedBrackets. This is just a small improvement in terms of performance, but it’s simpler and more direct code. Also, we avoid parsing interparagraph spaces in balanced brackets, as the original did.

• BibTeX reader: Properly handle commented lines in BibTeX/BibLaTeX (#7668).

• RST reader: handle class attribute for for custom roles (#7699, willj-dev). Previously the class attribute was ignored, and the name of the role used as the class.

• DocBook reader:

  • Add <titleabbr> support (Rowan Rodrik van der Molen).
  • Support for <indexterm> (#7607, Rowan Rodrik van der Molen).

• LaTeX reader:

  • Add rudimentary support for \autoref (#7693).
  • Add ‘uri’ class when parsing \url, for consistency with treatment of autolinks in other formats (#7672).

• JATS reader: Capture alt-text in figures (#7703, Aner Lucero).

• MediaWiki writer: use HTML spans for anchors when header has id (#7697). We need to generate a span when the header’s ID doesn’t match the one MediaWiki would generate automatically. Note that MediaWiki’s generation scheme is different from pandoc’s (it uses uppercase letters, and _ instead of -, for example). This means that in going from markdown to mediawiki, we’ll now get spans before almost every heading, unless explicit identifiers are used that correspond to the ones MediaWiki auto-generates. This is uglier output but it’s necessary for internal links to work properly.

• Markdown writer: don’t create autolinks when this loses information (#7692). Previously we sometimes lost attributes when rendering links as autolinks.

• Text.Pandoc.Readers.Metadata: allow multiple YAML documents when parsing YAML for yamlBsToRefs. Some people use --- as the end delimiter in YAML bibliography files, which causes the yaml library to emit an error unless we explicitly allow multiple YAML documents (and just consider the first).

• JATS writer:

  • Ensure figures are wrapped with <p> in list items (Albert Krewinkel). This prevents the generation of invalid output.
  • Add URL to element citation entries (Albert Krewinkel). The URL of a reference, if present, is added in tag <uri> to element-citation entries.

• HTML writer: Don’t create invalid data- attribute for empty attribute key (#7546).

• LaTeX writer:

  • Babel mappings: use ancientgreek for grc.
  • With -t latex-smart, don’t generate \ldots from ellipsis (#7674). Instead just use unicode ellipsis.

• JATS template: fix equal-contrib attribute (Albert Krewinkel). The standard requires the value to be either yes or no, but is was set to true for authors who contributed equally.

• reveal.js template: Add disableLayout variable (Christophe Dervieux).

• Text.Pandoc.Error: sort errors in handleError by exit code (Albert Krewinkel).

• Text.Pandoc.Writers.Shared: Improve toLegacyTable (#7683, Christian Despres).

• Lua subsystem:

  • Include lpeg module (#7649, Albert Krewinkel). Compiles the lpeg library (Parsing Expression Grammars For Lua) into the program. Package maintainers may choose to rely on package dependencies to make lpeg available, in which case they can compile the with the constraint lpeg +rely-on-shared-lpeg-library. lpeg and re are always made available in global variables, without the need for a require.

  • Set lpeg and re as globals; allow shared lib access via require. The lpeg and re modules are loaded into globals of the respective name, but they are not necessarily registered as loaded packages. This ensures that

    • the built-in library versions are preferred when setting the globals,
    • a shared library is used if pandoc has been compiled without lpeg, and
    • the require mechanism can be used to load the shared library if available, falling back to the internal version if possible and necessary.

  • Fix argument order in constructor pandoc.Cite (Albert Krewinkel). This restores the old behavior; argument order had been switched accidentally in pandoc 2.15.

  • Add Pushable instance for ReaderOptions (Albert Krewinkel).

  • Allow to pass custom reader options to pandoc.read as an optional third argument (#7656, Albert Krewinkel). The object can either be a table or a ReaderOptions value like PANDOC_READER_OPTIONS. Creating new ReaderOptions objects is possible through the new constructor pandoc.ReaderOptions.

  • Display Pandoc values using their native Haskell representation (Albert Krewinkel).

  • Require latest hslua (2.0.1) (#7661, #7657, Albert Krewinkel). This fixes issues with

    • misleading error messages when a required function parameter is omitted;
    • absent properties still being listed in the output of pairs; and
    • alias accessing leading to errors instead of returning nil, e.g. with (pandoc.Str '').identifier.

  • Add missing space in “package not found” message (#7658, Albert Krewinkel).

• Update build files (#7696, Fabián Heredia Montiel). Drop old windows 32-bit constraints. Update cabal tested-with field to correspond to ci.yml matrix

• Remove unneeded package dependencies from benchmark target.

• Require ghc >= 8.6, base >= 4.12. This allows us to get rid of the old custom prelude and some crufty cpp. But the primary reason for this is that conduit has bumped its base lower bound to 4.12, making it impossible for us to support lower base versions.

• Require Cabal 2.4. Use wildcards to ensure that all pptx tests are included (#7677).

• Update bash_completion.tpl (S.P.H.).

• Add data/creole.lua as sample custom reader.

• Add doc/custom-readers.md and doc/custom-writers.md.

• doc/lua-filters.md: add section on global modules, including lpeg (Albert Krewinkel).

• MANUAL.txt: update table of exit codes and corresponding errors (Albert Krewinkel).

• Use latest texmath.

pandoc 2.16.1

Click to expand changelog

• Docx reader: don’t let first line indents trigger block quotes (#7655). This fixes a regression introduced in pandoc 2.15.

• Docx writer: use getTimestamp for modification times in reference.docx (#7654). This ensures that when SOURCE_DATE_EPOCH is set, the modification times of files taken from the reference.docx will be set deterministically, allowing for reproducible builds.

• Lua subsystem (Albert Krewinkel):

  • Load module pandoc.path on startup (#7524). Previously the module always had to be loaded via require 'pandoc.path'.
  • Fix typo in SoftBreak constructor.
  • Re-add content property to Strikeout elements. Fixes a regression introduced in 2.15.
  • Be more forgiving when retrieving the Image caption property. Fixes a regression introduced in 2.15.
  • Display Attr values using their native Haskell representation.
  • Allow omitting the 2nd parameter in pandoc.Code constructor. Fixes a regression introduced in 2.15 which required users to always specify an Attr value when constructing a Code element.
  • Allow to compare, show Citation values. Comparisons of Citation values are performed in Haskell; values are equal if they represent the same Haskell value. Converting a Citation value to a string now yields its native Haskell string representation.
  • Restore List behavior of MetaList (#7650). Fixes a regression introduced in 2.16 which had MetaList elements lose the pandoc.List properties.
  • Restore content property on Header elements.
  • Ensure Block elements have all expected properties.
  • Ensure Inline elements have all expected properties.

• Allow tasty-bench 0.3.x.

pandoc 2.16

Click to expand changelog

• Switch back from HsYAML to yaml for parsing YAML metadata (#6084). HsYAML is around 20 times slower in parsing large YAML bibliographies. In addition, HsYAML is not being actively maintained. This sets us back in our attempts to free ourselves from C dependencies (#4535). But I don’t see a good alternative until a faster pure Haskell parser is available. Notes:

  • We’ve removed the FromYAML instances for all types that had them, since this is a HsYAML-specific typeclass [API change]. (The yaml package just uses From/ToJSON instead of having a dedicated From/ToYAML class.)
  • Unlike HsYAML (in the configuration we were using), yaml parses ‘Y’, ‘N’, ‘Yes’, ‘No’, ‘On’, ‘Off’ as boolean values. Users may need to quote these when they are meant to be interpreted as strings. Similarly, ‘null’ is parsed as a YAML null value (and will be treated as an empty string by pandoc rather than the string ‘null’). Quoting it will force it to be interpreted as a string.
  • Some tests had to be adjusted accordingly.
  • Pandoc now behaves in a more useful way when the YAML metadata contains escaping errors: instead of just failng silently and falling back to some other interpretation of the section, it raises a YAML parsing error.

• Markdown writer: Ensure that special values are quoted in YAML metadata. These include “Y”, “yes”, “on”, and “off”, which are now (with yaml library) considered boolean values, as well as “null”.

• Change JSON encodings of some types.

  • For LineEnding use lowercase constructors, e.g. crlf, native.
  • For HTMLSlideVariant use lowercase constructors.
  • For ReaderOptions use e.g. default-image-extension instead of readerDefaultImageExtension for field names.
  • For Extension, use e.g. tex_math_dollars instead of Ext_tex_math_dollars as constructor.
  • For Extensions, use an array of Extensions, instead of an object wrapping the tag Extensions and an integer. (The integer representation is not supposed to be part of the public API.)
  • For Opt, use field names like tab-stop instead of optTabStop.

• Docx writer:

  • Add IDs to native_numbering test (Tristan Stenner).
  • Move “:” out of the caption bookmark (Tristan Stenner). This is needed so that native references to the figure are included as “As seen in Figure X, it is…” instead of “As seen in [Figure: X, it is…”

• Lua (Albert Krewinkel, except as noted):

  • Use hslua module abstraction where possible.

  • Fix placement of tests for Block elements in pandoc module tests

  • Increase strictness when getting attribute keys

  • Re-add t and tag property to Attr values. Removal of these properties from Attr values was a regression.

  • Fix pandoc.utils.stringify regression. The pandoc.utils.stringify function returned empty strings when called with a string argument.

  • Fix a copy/paste bug in Lua marshalling code (John MacFarlane, #7639). This caused links to be changed to figures when Lua filters changed link properties.

  • Re-add content property to Link elements (#7647). This was a regression introduced in version 2.15.

  • Generate constants in module pandoc programmatically.

  • Marshal SimpleTable, ListAttributes, Citation, and Block values as userdata objects. Properties of Block values are marshalled lazily, which generally improves performance considerably. Script users may also notice the following differences:

    • Block element properties can no longer be accessed by numerical indexing of the .c field. The .c property now serves as an alias for .content, so some filter that used this undocumented method for property access may continue to work, while others will need to be updated and use proper property names.
    • The marshalled Block elements now have a show method, and a __tostring metamethod. Both return the Haskell string representation of the element.
    • Block values now have the Lua type userdata instead of table.

• Add a short guide to pandoc’s sources (Albert Krewinkel).

• Fix epub files in epub reader tests, so that they are valid according to epubcheck (#7586).

• Allow time 1.13.

• Require latest skylighting (0.12.1).

• Fix build on GHC 9.2 (Joseph C. Sible).

• Fix trypandoc so it builds with aeson > 2.

pandoc 2.15

Click to expand changelog

• Add --sandbox option (#5045).

  • Add sandbox feature. When this option is used, readers and writers only have access to input files (and other files specified directly on command line). This restriction is enforced in the type system.
  • Filters, PDF production, custom writers are unaffected. This feature only insulates the actual readers and writers, not the pipeline around them in Text.Pandoc.App.
  • Note that when --sandboxed is specified, readers won’t have access to the resource path, nor will anything have access to the user data directory.

• --self-contained: Fix bug that caused everything to be made a data URI (#7635, #7367). We only need to use data URIs in certain cases, but due to a bug they were being used always.

• Pandoc will now fall back to latin1 encoding for inputs that can’t be read as UTF-8. This is what it did previously for content fetched from the web and not marked as to content type. It makes sense to do the same for local files. In this case a NotUTF8Encoded warning will be issued, indicating that pandoc is interpreting the input as latin1.

• Markdown reader:

  • Don’t parse links or bracketed spans as citations (#7632). Previously pandoc would parse [link to (@a)](url) as a citation; similarly [(@a)]{#ident}. This is undesirable. One should be able to use example references in citations, and even if @a is not defined as an example reference, [@a](url) should be a link containing an author-in-text citation rather than a normal citation followed by literal (url).
  • Fix interaction of --strip-comments and list parsing (#7521). Use of --strip-comments was causing tight lists to be rendered as loose (as if the comment were a blank line).
  • Fix parsing bug for math in bracketed spans and links (#7623). This affects math with unbalanced brackets (e.g. $(0,1]$) inside links, images, bracketed spans.
  • Fix code blocks using --preserve-tabs (#7573). Previously they did not behave as the equivalent input with spaces would.

• DocBook reader:

  • Honor linenumbering attribute (Samuel Tardieu). The attribute DocBook linenumbering="numbered" on code blocks maps to the numberLines class internally.

• LaTeX reader:

  • Implement siunitx v3 commands (#7614). We support \unit, \qty, \qtyrange, and \qtylist as synonynms of \si, \SI, \SIrange, and \SIlist.
  • Properly handle \^ followed by group closing (#7615).
  • Recognize that \vadjust sometimes takes “pre” (#7531).
  • Ignore (and gobble parameters of) CSLReferences environment (#7531). Otherwise we get the parameters as numbers in the output.
  • Restrict \endinput to current file (Simun Schuster).

• RST reader: handle escaped colons in reference definitions (#7568).

• HTML reader:

  • Handle empty tbody element in table (#7589).

• Ipynb reader (Kolen Cheung):

  • Get cell output mime from raw_mimetype in addition to format. (format is what the spec calls for, but raw_mimetype is often used in practice; see jupyter/nbformat#229).
  • Add more formats that can be handled as “raw” cells.
  • Fix mime type for rst.
  • Support text/markdown, which is now a supported mime type for raw output (#7561).

• RTF reader:

  • Support \binN for binary image data.
  • If doc begins with { … } only parse its contents. Some documents seem to have non-RTF (e.g. XML) material after the {\rtf1 ... } group.
  • Ignore \pgdsc group. Otherwise we get style names treated as test.
  • Better handling of \* and bookmarks. We now ensure that groups starting with \* never cause text to be added to the document. In addition, bookmarks now create a span between the start and end of the bookmark, rather than an empty span.

• Docx reader:

  • Avoid blockquote when parent style has more indent (Milan Bracke). When a paragraph has an indentation different from the parent (named) style, it used to be considered a blockquote. But this only makes sense when the paragraph has more indentation. So this commit adds a check for the indentation of the parent style.
  • Fix handling of empty fields (Milan Bracke). Some fields only have an instrText and no content, Pandoc didn’t understand these, causing other fields to be misunderstood because it seemed like a field was still open when it wasn’t.
  • Implement PAGEREF fields (Milan Bracke). These fields, often used in tables of contents, can be a hyperlink.
  • Fix handling of nested fields (Milan Bracke). Fields delimited by fldChar elements can contain other fields. Before, the nested fields would be ignored, except for the end, which would be considered the end of the parent field.
  • Add placeholder for word diagram instead of just omitting it (Ezwal).

• Org reader:

  • Don’t parse a list as first item in a list item (#7557).
  • Allow an initial :PROPERTIES: drawer to add to metadata (#7520).

• Docx writer:

  • Make id used in native_numbering predictable (#7551). If the image has the id IMAGEID, then we use the id ref_IMAGEID for the figure number. This allows one to create a filter that adds a figure number with figure name, e.g. <w:fldSimple w:instr=" REF ref_superfig "><w:r><w:t>Figure X</w:t> </w:r></w:fldSimple>. If an image lack an id, an id of the form ref_fig1 is used.

• Ensure we have unique ids for wp:docPr and pic:cNvPr elements (#7527, #7503).

• Handle SVG images (#4058). This change has several parts:

  • In Text.Pandoc.App, if the writer is docx, we fill the media bag and attempt to convert any SVG images to PNG, adding these to the media bag. The PNG backups have the same filenames as the SVG images, but with an added .png extension. If the conversion cannot be done (e.g. because rsvg-convert is not present), a warning is omitted.
  • In Text.Pandoc.Writers.Docx, we now use Word 2016’s syntax for including SVG images. If a PNG fallback is present in the media bag, we include a link to that too.

• Powerpoint writer (Emily Bourke):

  • Add support for more layouts (#5097). Up til now, four layouts were supported: “Title Slide” (used for the automatically generated metadata slide), “Section Header” (used for headings above slide level), “Two Column” (used when there’s a columns div), “Title and Content” (used for all other slides). We now support three additional layouts: “Comparison”, “Content with Caption”, and “Blank”. The manual describes the logic that determines which layout is used for a slide. Layouts may be customized in the reference doc.
  • Support specifying slide background images using a background-image attribute on the slide’s heading. Only the “stretch” mode is supported, and the background image is centred around the slide in the image’s larger axis, matching the observed default behaviour of PowerPoint.
  • Add support for incremental lists (through same methods as in other slide writers) (#5689).
  • Copy embedded fonts from reference doc.
  • Include all themes in output archive.
  • Fix list level numbering (#4828, #4663). In PowerPoint, the content of a top-level list is at the same level as the content of a top-level paragraph: the only difference is that a list style has been applied. Previously, the writer incremented the paragrap h level on each list, turning what should be top-level lists into second-level lists.
  • Line up list continuation paragraphs. This commit changes the marL and indent values used for plain paragraphs and numbered lists, and changes the spacing defined in the reference doc master for bulleted lists. For paragraphs, there is now a left-indent taken from the otherStyle in the master. For numbered lists, the number is positioned where the text would be if this were a plain paragraph, and the text is indented to the next level. This means that continuation paragraphs line up nicely with numbered lists. Existing reference docs may need to be modified so that otherStyle and bodyStyle indent levels match, for this feature to work with them.
  • Consolidate text runs when possible (jgm). This slims down the output files by avoiding unnecessary text run elements.
  • Support footers in the reference doc. There is one behaviour which may not be immediately obvious: if the reference doc specifies a fixed date (i.e. not automatically updating), and there’s a date specified in the metadata for the document, the footer date is replaced by the metadata date.
  • Fix presentation rel numbering. Before now, the numbering of rIds was inconsistent when making the presentation XML and when making the presentation relationships XML.
  • Don’t add relationships unnecessarily. Before now, for any layouts added to the output from the default reference doc, the relationships were unconditionally added to the output. However, if there was already a layout in slideMaster1 at the same index then that results in duplicate relationships.
  • If slide level is 0, don’t insert a slide break between a heading and a following table, “columns” div, or paragraph starting with an image.
  • Fix capitalisation of notesMasterId.
  • Restructure tests.

• Asciidoc writer:

  • Translate numberLines attribute to linesnum switch (Samuel Tardieu).
  • Improve escaping for -- in URLs (#7529).

• LaTeX writer:

  • Make babel use more idiomatic (#7604, hseg). Use babel’s bidi implementation. Import babel languages individually instead of as package options. Move header-includes to after babel setup so it can be modified.
  • Use babel, not polyglossia, with xelatex. Previously polyglossia worked better with xelatex, but that is no longer the case, so we simplify the code...

pandoc 2.14.2

Click to expand changelog

• Allow --slide-level=0 (#7476). When the slide level is set to 0, headings won’t be used at all in splitting the document into slides. Horizontal rules must be used to separate slides.

• Add RTF reader (#3982). rtf is now supported as an input format as well as an output format. New module Text.Pandoc.Readers.RTF (exporting readRTF). [API change]

• HTML reader: treat comments as blank when parsing (#7482).

• Markdown reader:

  • Fix raw LaTeX injection issue (#7497). Using a code block containing \end{verbatim}, one could inject raw TeX into a LaTeX document even when raw_tex is disabled. Thanks to Augustin Laville for noticing the bug.
  • Multimarkdown sub- and superscripts (#5512, OCzarnecki). Added an extension short_subsuperscripts which modifies the behavior of subscript and superscript, allowing subscripts or superscripts containing only alphanumerics to end with a space character (eg. x^2 = 4 or H~2 is combustible). This improves support for multimarkdown.

• RST reader: Fix :literal: includes (#7513). These should create code blocks, not insert raw RST.

• LaTeX reader:

  • Proper implicit grouping around environment macros.
  • Support \global before \def, \let, etc. (#7494).
  • Fix scope for LaTeX macros (#7494). They should by default scope over the group in which they are defined (except \gdef and \xdef, which are global). In addition, environments must be treated as groups.
  • Improve handling of plain TeX macro primitives (#7474). Fixed semantics for \let.
  • Implement \edef, \gdef, and \xdef.

• Docx reader: Improve docx reader’s robustness in extracting images (#7511). The docx reader made some assumptions about how docx containers were laid out that were not always true, with the result that some images in documents did not get extracted.

• LaTeX writer: Increase table column width precision (#7466, Peter Fabinski). In some cases, the rounding performed by the LaTeX table writer would introduce visible overrun outside the text area. This adds two more decimal places to the width values.

• Powerpoint writer:

  • Include image title in description (#7352, Emily Bourke). The image title (i.e. ![alt text](link "title")) was previously ignored when writing to pptx. This commit includes it in PowerPoint’s description of the image, along with the link.
  • Select layouts from reference doc by name (Emily Bourke). Until now, users had to make sure that their reference doc contains layouts in a specific order: the first four layouts in the file had to have a specific structure. Now the layout selection uses the layout names rather than order: users must make sure their reference doc contains four layouts with specific names, and if a layout with the right name isn’t found pandoc will emit a warning and use the corresponding layout from the default reference doc as a fallback.

• Docx writer: be sensitive to the native_numbering extension (#7499). Figure and table numbers are now only included if native_numbering is enabled. (By default it is disabled.) This is a behavior change with respect to 2.14.1, but the default behavior is now that of previous versions. The change was necessary to avoid incompatibilities between pandoc’s native numbering and third-party cross reference filters like pandoc-crossref.

• RTF writer:

  • Omit \bin in \pict. According to the spec, this is not needed or wanted when the data is in hexadecimal format, as here.
  • Emit ``` for section headings.

• RTF template: specify font family for fixed-width font f1. According to the spec, this is mandatory.

• LaTeX writer: Use ulem for underline (#7351). ulem is conditionally included already when the strikeout variable is set, so we set this when there is underlined text, and use \uline instead of \underline. This fixes wrapping for underlined text.

• Text.Pandoc.Citeproc:

  • Revise citeproc code to fit new citeproc 0.5 API (thanks to Benjamin Bray). Linkification of URLs in the bibliography is now done in the citeproc library, depending on the setting of an option. We set that option depending on the value of the metadata field link-bibliography (defaulting to true, for consistency with earlier behavior). If a DOI, PMID, PMCID, or URL field is present but not explicitly rendered, the title (or if no title, the whole entry) is hyperlinked. These changes implement the recommendations from the draft CSL v1.0.2 spec (Appendix VI): https://github.com/citation-style-language/documentation/blob/master/specification.rst#appendix-vi-links
  • Avoid odd handling of quotes. Recent citeproc changes allow us to ignore Quoted elements; citeproc now uses its own method for represented quoted things, and only localizes and flipflops quotes it adds itself. Convert Quoted in bib entries to special Spans before passing them off to citeproc. This ensures that we get proper localization and flipflopping if, e.g., quotes are used in titles (jgm/citeproc#87).
  • Removed quote localization from citeproc processing. This is now done in citeproc itself.

• Text.Pandoc.Logging: Add PowerpointTemplateWarning log message type [API change] (Emily Bourke).

• Text.Pandoc.Extension: Add Ext_short_subsuperscripts constructor to Extension [API change] (OCzarnecki).

• Various sample.lua editorial fixes (#7493, #7487, William Lupton).

• Bump base-compat version so we get compatibility with base 4.12.

• Use Prelude from base-compat for ghc 8.4 too.

• Add haskell-language-server to shell.nix (#7496, Emily Bourke).

• Tests.Helpers: export testGolden and use it in RTF reader. This gives a diff output on failure.

• Remove obsolete and incorrect sentence in --slide-level docs.

• Add internal module Text.Pandoc.Network.HTTP, exporting urlEncode.

• Text.Pandoc.Parsing: parseFromString: preserve at least the source directory (#7464). Previously we just set the source name to “chunk” when parsing from strings, to avoid misleading source positions. This had the side effect that rebase_relative_paths would break inside sections that were parsed as strings. So, now we use “ORIGINAL_SOURCE_PATH_chunk” instead of just “chunk”.

• Text.Pandoc.MIME: use image/x-xcf instead of application/x-xcf (#7454).

• Don’t compare cdLine in OOXML golden tests (Emily Bourke). The cdLine field gives the line of the file some CData was found on, which reflects irrelevant formatting differences.

• Provide more detailed XML diff in tests (Emily Bourke).

• OOXML tests: silence warnings. These can make the test output confusing, making people think tests are failing when they’re passing.

• INSTALL.md: Add GitLab CI/CD example (#7448, Veratyr).

• MANUAL.txt

  • Clarifications (William Lupton).
  • Add a note on security risks of include directives.

• Document use of the ‘underline’ class (#7492, #7484, William Lupton).

• Add a FAQ about the “Cannot allocate memory” error on M1 macs.

• Use texmath 0.12.3.1.

• Use released citeproc 0.5.

• Remove dependency on HTTP package (#7456, mt_caret).