Skip to content

gh-68475: Keep comments and processing instructions outside the root element - #157434

Open
serhiy-storchaka wants to merge 8 commits into
python:mainfrom
serhiy-storchaka:gh-68475-document-builder
Open

gh-68475: Keep comments and processing instructions outside the root element#157434
serhiy-storchaka wants to merge 8 commits into
python:mainfrom
serhiy-storchaka:gh-68475-document-builder

Conversation

@serhiy-storchaka

Copy link
Copy Markdown
Member

Comments and processing instructions outside of the root element were lost: the parser reported them to the target, but TreeBuilder had nowhere to put them, and ElementTree could not serialize them.

  • ElementTree gets the children attribute, a sequence of the children of the document: the root element and the comments and processing instructions around it. It can be modified like a list, with the restriction that there is at most one element (the root). ElementTree.write() serializes all children, ElementTree.iter() iterates over all of them, find*() still search from the root.
  • The new DocumentBuilder parser target is a TreeBuilder which keeps comments and PIs outside of the root element (when insert_comments or insert_pis is true) and returns the list of the children of the document from close(). ElementTree.parse() and parse() load such a list into children; XMLID() accepts it.
  • TreeBuilder is not changed: it still discards what is outside of the root element, and close() still returns the root element. The result of the target reaches the caller only through close(), as in XMLParser and lxml (see the discussion in Remove root attribute from XMLPullParser #63190).
  • DocumentBuilder is implemented in C too, sharing the TreeBuilder implementation; the parser calls the handlers directly for both exact types.

The document type declaration is left for a separate PR.

This is an alternative to #156719, which added get_document_children() to TreeBuilder.

… root element

ElementTree gets the children attribute, a view of the children of the
document, containing the root element and any number of comments and
processing instructions around it.  Adding a second element is an error.
iter() iterates over all of them, but find(), findall() and iterfind()
still search from the root element.

TreeBuilder collects the comments and processing instructions which occur
outside the root element and returns them, together with the root element,
from the new document() method.  This only happens when insert_comments or
insert_pis is set, so nothing changes for existing code.  parse() asks the
target for the document before close(), which releases it.

The C accelerator implements document() too, so that the feature works at
full parsing speed.
Registering the implementation with a cast is a call through a pointer to
an incorrect function type: it is warned about by the compiler, reported by
UBSan, and traps on WASI.
The name of the method which ElementTree.parse() looks up on the parser
target should not clash with an attribute of a custom target.
…ment-children

# Conflicts:
#	Modules/clinic/_elementtree.c.h
…ment-children

# Conflicts:
#	Modules/_elementtree.c
…nt_children()

TreeBuilder is unchanged: it discards comments and processing instructions
outside of the root element, and close() returns the root element.
The new DocumentBuilder subclass keeps them (when insert_comments or
insert_pis is true) and returns the list of the children of the document
from close(), the only result channel of the target protocol.
ElementTree.parse() loads such a list into children, XMLID() accepts it.

In C, the DocumentBuilder type shares the TreeBuilder struct and handlers;
the document list is NULL for a plain TreeBuilder.  The parser calls the
handlers directly for both exact types.
@read-the-docs-community

Copy link
Copy Markdown

Documentation build overview

📚 cpython-previews | 🛠️ Build #34535079 | 📁 Comparing 5bae975 against main (0ba7be9)

  🔍 Preview build  

3 files changed
± library/xml.etree.elementtree.html
± whatsnew/3.16.html
± whatsnew/changelog.html

@serhiy-storchaka

Copy link
Copy Markdown
Member Author

@scoder, this is the alternative to #156719 after your objection to adding a method to the target protocol.

TreeBuilder is now untouched. The new DocumentBuilder subclass keeps the comments and processing instructions outside of the root element and returns the list of the children of the document from close(), so the result of the target reaches the caller only through the channel which already exists, as you argued in bpo-18990 (gh-63190). ElementTree.parse() recognizes the list; iterparse(..., target=DocumentBuilder()) gets it as .root.

The document type declaration will be a separate PR on top of this one: DocumentType factory, DocumentBuilder.doctype(), and optionally the internal subset as the children of the DocumentType element.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant