Support Pandoc's .float table class to avoid longtable in twocolumn layouts

[cderv]

Pandoc's longtable (the default table environment for LaTeX output) is incompatible with twocolumn documents. This has come up before, #3314 for example, and again in a recent discussion comment:

https://github.com/orgs/quarto-dev/discussions/11669#discussioncomment-17905332

Pandoc has now added a way to opt a table out of longtable into a regular floating table environment instead. From the pandoc commit:

LaTeX writer: Provide a way to use table instead of longtable.

When the `float` class is attached to a table, a standard
floating `table` will be generated instead of a `longtable`.
Placement can be specified via the `latex-placement` attribute.
This will help, especially, those who use two-column layouts,
since `longtable` is incompatible with these.

And from the issue comment suggesting the workaround:

function Table(el)
  el.classes:insert("float")
  return el
end

This lands in Pandoc 3.10.1. We currently bundle 3.10,

quarto-cli/configuration

Line 16 in f01c303

export PANDOC=3.10

so it's not usable yet, but we could test it now with QUARTO_PANDOC=<path-to-3.10.1-build>.

If we want to auto-apply the .float class for twocolumn documents, so users don't have to write the Lua filter themselves, the scope isn't as simple as it looks.

For plain, uncaptioned tables (the case in #3314 and the discussion), the Table node reaches Pandoc's LaTeX writer unmodified, we only wrap a table into our own FloatRefTarget custom node when it carries a tbl- label:

quarto-cli/src/resources/filters/quarto-pre/parsefiguredivs.lua

Lines 524 to 560 in f01c303

	Table = function(el)
	if el.caption.long == nil then
	return nil
	end
	local last = el.caption.long[#el.caption.long]
	if not last or #last.content == 0 then
	return nil
	end
	-- check for tbl label
	local label = el.identifier
	local caption, attr = parseTableCaption(last.content)
	if startsWith(attr.identifier, "tbl-") then
	-- set the label and remove it from the caption
	label = attr.identifier
	attr.identifier = ""
	caption = createTableCaption(caption, pandoc.Attr())
	end
	-- we've parsed the caption, so we can remove it from the table
	el.caption.long = pandoc.Blocks({})
	if label == "" then
	return nil
	end
	local combined = merge_attrs(el.attr, attr)
	return construct({
	identifier = label,
	classes = combined.classes,
	attributes = as_plain_table(combined.attributes),
	type = "Table",
	content = pandoc.Blocks({ el }),
	caption_long = caption,
	}), false
	end,

For those, adding .float when classoption contains twocolumn should be straightforward, we already parse classoption the same way for booksidedness:

quarto-cli/src/resources/filters/layout/meta.lua

Lines 219 to 229 in f01c303

	function booksidedness(meta)
	local side = 'two'
	local classoption = readOption(meta, 'classoption')
	if classoption then
	for i, v in ipairs(classoption) do
	local option = pandoc.utils.stringify(v)
	if option == 'twoside=semi' then
	side = 'semi'
	elseif option == 'twoside' or option == 'twoside=on' or option == 'twoside=true' or option == 'twoside=yes' then
	side = 'two'
	elseif option == 'twoside=false' or option == 'twoside=no' or option == 'twoside=off' then

Cross-referenced tables (tbl- label) are more involved. We manually decrement LaTeX's table counter after every longtable, because longtable increments it in a way that double-counts against our own numbering:

quarto-cli/src/resources/filters/quarto-post/latex.lua

Lines 295 to 322 in f01c303

	return {
	traverse = "topdown",
	Div = handle_column_classes,
	Span = handle_column_classes,
	Table = handle_table_columns,
	PanelLayout = handle_panel_layout,
	-- Pandoc emits longtable environments by default;
	-- longtable environments increment the _table_ counter (!!)
	-- http://mirrors.ctan.org/macros/latex/required/tools/longtable.pdf
	-- (page 13, definition of \LT@array)
	--
	-- This causes double counting in our table environments. Our solution
	-- is to decrement the counter manually after each longtable environment.
	--
	-- This hack causes some warning during the compilation of the latex document,
	-- but the alternative is worse.
	FloatRefTarget = function(float)
	-- don't look inside floats, they get their own rendering.
	if float.type == "Table" then
	-- we have a separate fixup for longtables in our floatreftarget renderer
	-- in the case of subfloat tables...
	float.content = _quarto.ast.walk(quarto.utils.as_blocks(float.content), {
	traverse = "topdown",
	FloatRefTarget = function(float)
	return nil, false
	end,
	})

A regular floating table environment is expected to increment that counter, so switching the underlying environment there would make the decrement wrong. There's also raw-text parsing of \begin{longtable}...\end{longtable} for subtable/panel-layout fixups in floatreftarget.lua that assumes longtable specifically:

quarto-cli/src/resources/filters/customnodes/floatreftarget.lua

Lines 442 to 490 in f01c303

	local longtable_match, longtable_pattern = _quarto.modules.patterns.match_in_list_of_patterns(el.text, _quarto.patterns.latexLongtableEnvPatterns)
	if longtable_match and longtable_pattern then
	made_fix = true
	local raw = el
	-- special case for longtable floats in LaTeX
	local extended_pattern = {".-"}
	for _, pattern in ipairs(longtable_pattern) do
	table.insert(extended_pattern, pattern)
	end
	table.insert(extended_pattern, ".*")
	local longtable_preamble, longtable_begin, longtable_content, longtable_end, longtable_postamble = _quarto.modules.patterns.match_all_in_table(extended_pattern)(raw.text)
	if longtable_preamble == nil or longtable_begin == nil or longtable_content == nil or longtable_end == nil or longtable_postamble == nil then
	warn("Could not parse longtable parameters. This could happen because the longtable parameters\n" ..
	"are not well-formed or because of a bug in quarto. Please consider filing a bug report at\n" ..
	"https://github.com/quarto-dev/quarto-cli/issues/, and make sure to include the document that\n" ..
	"triggered this error.")
	return {}
	end
	-- Pandoc 3.8.1+ wraps a captionless table in a brace group that only
	-- scopes `\def\LTcaptype{none}`. We supply our own \caption and drop that
	-- definition, so the group is now pointless - and not inert: it breaks
	-- packages that move the environment out of the text flow (endfloat's
	-- \DeclareDelayedFloatFlavor*{longtable}{table}, #14741). Drop the braces
	-- with the definition, but only when provably Pandoc's own wrapper:
	-- preamble is just the brace + def, postamble is just the brace, and the
	-- block holds a single longtable. Otherwise strip the definition alone.
	local preamble_without_group, opened = longtable_preamble:gsub(
	"^(%s*){%s*\\def\\LTcaptype{none}[^\n]*\n(%s*)$", "%1%2")
	local postamble_without_group, closed = longtable_postamble:gsub(
	"^(%s*)}(%s*)$", "%1%2")
	local single_longtable =
	longtable_content:find("\\begin{longtable}", 1, true) == nil
	if opened > 0 and closed > 0 and single_longtable then
	longtable_preamble = preamble_without_group
	longtable_postamble = postamble_without_group
	else
	longtable_preamble =
	longtable_preamble:gsub("\\def\\LTcaptype{none}[^\n]*\n?", "")
	end
	-- split the content into params and actual content
	-- params are everything in the first line of longtable_content
	-- actual content is everything else
	local start, content = split_longtable_start(longtable_begin .. longtable_content)
	if start == nil or content == nil then
	warn("Could not parse longtable parameters. This could happen because the longtable parameters\n" ..
	"are not well-formed or because of a bug in quarto. Please consider filing a bug report at\n" ..
	"https://github.com/quarto-dev/quarto-cli/issues/, and make sure to include the document that\n" ..
	"triggered this error.")
	return {}

So a first version could be scoped to plain tables only, leaving tbl- labeled tables on the longtable path for now.

I haven't tested any of this against an actual 3.10.1 build yet, this is source-reading only.

Related: #3314, #14741 (different bug, same area of code)