Skip to content

Commit 3b57bf8

Browse files
[3.14] gh-156821: Fix inaccuracies in the xml.parsers.expat documentation (GH-156822) (GH-156853)
Corrected, in the documentation and in the docstrings: * GetInputContext() returns bytes, and its result extends to the end of the buffered input; * Parse() accepts a bytes-like object as well as a string, and ignores the encoding declaration for a string; * ParseFile() only supports binary files; * XML_ERROR_XML_DECL was described as XML_ERROR_NO_ELEMENTS; * UnparsedEntityDeclHandler is not restricted to Expat 1.2; * GetSpecifiedAttributeCount() needs ordered_attributes. Documented the intern parameter and attribute, namespace_prefixes, SkippedEntityHandler, XML_CTYPE_MIXED and XML_CTYPE_NAME, and EXPAT_VERSION, version_info and features. Removed obsolete notes about Expat 1.2 and 1.95.0, and :noindex: from the content model constants, which are not documented elsewhere; completed the list of attribute types in AttlistDeclHandler. (cherry picked from commit d16a691)
1 parent 07db3f4 commit 3b57bf8

3 files changed

Lines changed: 128 additions & 54 deletions

File tree

Doc/library/pyexpat.rst

Lines changed: 102 additions & 40 deletions
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,7 @@ the XML document.
3737
This module uses the :mod:`pyexpat` module to provide access to the Expat
3838
parser. Direct use of the :mod:`pyexpat` module is deprecated.
3939

40-
This module provides one exception and one type object:
40+
This module provides the following exception, type object and data items:
4141

4242

4343
.. exception:: ExpatError
@@ -55,6 +55,29 @@ This module provides one exception and one type object:
5555

5656
The type of the return values from the :func:`ParserCreate` function.
5757

58+
59+
.. data:: EXPAT_VERSION
60+
61+
The version string of the Expat library loaded by the interpreter,
62+
like ``'expat_2.8.4'``.
63+
64+
65+
.. data:: version_info
66+
67+
The version of the Expat library loaded by the interpreter,
68+
as a tuple of three integers: major, minor and micro version.
69+
70+
71+
.. data:: features
72+
73+
The list of the features with which the loaded Expat library
74+
was compiled, as ``(name, value)`` pairs.
75+
The value is only meaningful for features which have one,
76+
like ``'XML_CONTEXT_BYTES'`` or the default protection limits
77+
``'XML_BLAP_ACT_THRES'`` and ``'XML_AT_MAX_AMP'``;
78+
for other features, like ``'XML_DTD'`` and ``'XML_NS'``,
79+
the value is ``0`` and only the presence of the name is significant.
80+
5881
The :mod:`!xml.parsers.expat` module contains two functions:
5982

6083

@@ -63,7 +86,7 @@ The :mod:`!xml.parsers.expat` module contains two functions:
6386
Returns an explanatory string for a given error number *errno*.
6487

6588

66-
.. function:: ParserCreate(encoding=None, namespace_separator=None)
89+
.. function:: ParserCreate(encoding=None, namespace_separator=None, intern=None)
6790

6891
Creates and returns a new :class:`xmlparser` object. *encoding*, if specified,
6992
must be a string naming the encoding used by the XML data. Expat doesn't
@@ -110,6 +133,11 @@ The :mod:`!xml.parsers.expat` module contains two functions:
110133
http://www.python.org/ns/ elem1
111134
elem2
112135

136+
*intern*, if given, must be a dictionary.
137+
It is used to intern the names of elements and attributes,
138+
and is available as the :attr:`~xmlparser.intern` attribute.
139+
By default a new empty dictionary is created for every parser.
140+
113141
Due to limitations in the ``Expat`` library used by :mod:`pyexpat`,
114142
the :class:`xmlparser` instance returned can only be used to parse a single
115143
XML document. Call ``ParserCreate`` for each document to provide unique
@@ -132,18 +160,24 @@ XMLParser Objects
132160

133161
.. method:: xmlparser.Parse(data[, isfinal])
134162

135-
Parses the contents of the string *data*, calling the appropriate handler
136-
functions to process the parsed data. *isfinal* must be true on the final call
137-
to this method; it allows the parsing of a single file in fragments,
163+
Parses the contents of *data*,
164+
calling the appropriate handler functions to process the parsed data.
165+
*data* can be a :term:`bytes-like object` or a string.
166+
If it is a string, the encoding declaration in the XML data is ignored,
167+
and the data is parsed as already decoded text.
168+
*isfinal* must be true on the final call to this method;
169+
it allows the parsing of a single file in fragments,
138170
not the submission of multiple files.
139-
*data* can be the empty string at any time.
171+
*data* can be empty at any time.
140172

141173

142174
.. method:: xmlparser.ParseFile(file)
143175

144-
Parse XML data reading from the object *file*. *file* only needs to provide
145-
the ``read(nbytes)`` method, returning the empty string when there's no more
146-
data.
176+
Parse XML data reading from the object *file*.
177+
*file* only needs to provide the ``read(nbytes)`` method,
178+
which returns bytes, and an empty bytes object when there's no more data.
179+
Text files are not supported;
180+
use :meth:`Parse` for data which is already decoded.
147181

148182

149183
.. method:: xmlparser.SetBase(base)
@@ -163,9 +197,15 @@ XMLParser Objects
163197

164198
.. method:: xmlparser.GetInputContext()
165199

166-
Returns the input data that generated the current event as a string. The data is
167-
in the encoding of the entity which contains the text. When called while an
168-
event handler is not active, the return value is ``None``.
200+
Returns the input data which generated the current event
201+
as a :class:`bytes` object.
202+
The data is in the encoding of the entity which contains the text.
203+
It extends to the end of the currently buffered input,
204+
therefore it can contain also the data of the following events,
205+
and if the event was generated by a large amount of text,
206+
not all of it may be available.
207+
When called while an event handler is not active,
208+
the return value is ``None``.
169209

170210

171211
.. method:: xmlparser.ExternalEntityParserCreate(context[, encoding])
@@ -392,6 +432,22 @@ against some common XML vulnerabilities.
392432
default, this attribute is false; it may be changed at any time.
393433

394434

435+
.. attribute:: xmlparser.intern
436+
437+
The dictionary used to intern the names of elements and attributes.
438+
It is either the dictionary passed as the *intern* argument
439+
of :func:`ParserCreate`, or a new dictionary created for this parser.
440+
441+
442+
.. attribute:: xmlparser.namespace_prefixes
443+
444+
If set to a true value, and namespace processing is enabled,
445+
the namespace prefix is reported as the third part of the expanded name,
446+
separated by the namespace separator.
447+
Names which have no prefix are not affected.
448+
By default, this attribute is false; it may be changed at any time.
449+
450+
395451
The following attributes contain values relating to the most recent error
396452
encountered by an :class:`xmlparser` object, and will only have correct values
397453
once a call to :meth:`Parse` or :meth:`ParseFile` has raised an
@@ -455,8 +511,7 @@ otherwise stated.
455511
encoding of the document text, and an optional "standalone" declaration.
456512
*version* and *encoding* will be strings, and *standalone* will be ``1`` if the
457513
document is declared standalone, ``0`` if it is declared not to be standalone,
458-
or ``-1`` if the standalone clause was omitted. This is only available with
459-
Expat version 1.95.0 or newer.
514+
or ``-1`` if the standalone clause was omitted.
460515

461516

462517
.. method:: xmlparser.StartDoctypeDeclHandler(doctypeName, systemId, publicId, has_internal_subset)
@@ -465,14 +520,12 @@ otherwise stated.
465520
...``). The *doctypeName* is provided exactly as presented. The *systemId* and
466521
*publicId* parameters give the system and public identifiers if specified, or
467522
``None`` if omitted. *has_internal_subset* will be true if the document
468-
contains an internal document declaration subset. This requires Expat version
469-
1.2 or newer.
523+
contains an internal document declaration subset.
470524

471525

472526
.. method:: xmlparser.EndDoctypeDeclHandler()
473527

474-
Called when Expat is done parsing the document type declaration. This requires
475-
Expat version 1.2 or newer.
528+
Called when Expat is done parsing the document type declaration.
476529

477530

478531
.. method:: xmlparser.ElementDeclHandler(name, model)
@@ -487,12 +540,16 @@ otherwise stated.
487540
declaration declares three attributes, this handler is called three times, once
488541
for each attribute. *elname* is the name of the element to which the
489542
declaration applies and *attname* is the name of the attribute declared. The
490-
attribute type is a string passed as *type*; the possible values are
491-
``'CDATA'``, ``'ID'``, ``'IDREF'``, ... *default* gives the default value for
543+
The attribute type is a string passed as *type*:
544+
``'CDATA'``, ``'ID'``, ``'IDREF'``, ``'IDREFS'``, ``'ENTITY'``,
545+
``'ENTITIES'``, ``'NMTOKEN'`` or ``'NMTOKENS'``,
546+
an enumeration like ``'(x|y)'``,
547+
or a notation list like ``'NOTATION(n1|n2)'``.
548+
*default* gives the default value for
492549
the attribute used when the attribute is not specified by the document instance,
493550
or ``None`` if there is no default value (``#IMPLIED`` values). If the
494551
attribute is required to be given in the document instance, *required* will be
495-
true. This requires Expat version 1.95.0 or newer.
552+
true.
496553

497554

498555
.. method:: xmlparser.StartElementHandler(name, attributes)
@@ -528,10 +585,10 @@ otherwise stated.
528585

529586
.. method:: xmlparser.UnparsedEntityDeclHandler(entityName, base, systemId, publicId, notationName)
530587

531-
Called for unparsed (NDATA) entity declarations. This is only present for
532-
version 1.2 of the Expat library; for more recent versions, use
533-
:attr:`EntityDeclHandler` instead. (The underlying function in the Expat
534-
library has been declared obsolete.)
588+
Called for unparsed (NDATA) entity declarations.
589+
If this handler is not set, such declarations are reported by
590+
:attr:`EntityDeclHandler`, which is preferred for new code.
591+
(The underlying function in the Expat library has been declared obsolete.)
535592

536593

537594
.. method:: xmlparser.EntityDeclHandler(entityName, is_parameter_entity, value, base, systemId, publicId, notationName)
@@ -542,8 +599,7 @@ otherwise stated.
542599
``None`` for parsed entities, and the name of the notation for unparsed
543600
entities. *is_parameter_entity* will be true if the entity is a parameter entity
544601
or false for general entities (most applications only need to be concerned with
545-
general entities). This is only available starting with version 1.95.0 of the
546-
Expat library.
602+
general entities).
547603

548604

549605
.. method:: xmlparser.NotationDeclHandler(notationName, base, systemId, publicId)
@@ -597,7 +653,7 @@ otherwise stated.
597653

598654
.. method:: xmlparser.DefaultHandlerExpand(data)
599655

600-
This is the same as the :func:`DefaultHandler`, but doesn't inhibit expansion
656+
This is the same as the :attr:`DefaultHandler`, but doesn't inhibit expansion
601657
of internal entities. The entity reference will not be passed to the default
602658
handler.
603659

@@ -641,6 +697,16 @@ otherwise stated.
641697
:attr:`DefaultHandler` callback, if provided.
642698

643699

700+
.. method:: xmlparser.SkippedEntityHandler(entityName, is_parameter_entity)
701+
702+
Called for entity references which are not expanded,
703+
because the parser did not read the declaration of the entity.
704+
This happens when the external DTD subset or an external parameter entity
705+
is not parsed.
706+
*is_parameter_entity* is true for a parameter entity
707+
and false for a general entity.
708+
709+
644710
.. _expaterror-objects:
645711

646712
ExpatError Exceptions
@@ -746,35 +812,35 @@ The constants in the model type group are:
746812

747813

748814
.. data:: XML_CTYPE_ANY
749-
:noindex:
750815

751816
The element named by the model name was declared to have a content model of
752817
``ANY``.
753818

754819

755820
.. data:: XML_CTYPE_CHOICE
756-
:noindex:
757821

758822
The named element allows a choice from a number of options; this is used for
759823
content models such as ``(A | B | C)``.
760824

761825

762826
.. data:: XML_CTYPE_EMPTY
763-
:noindex:
764827

765828
Elements which are declared to be ``EMPTY`` have this model type.
766829

767830

768831
.. data:: XML_CTYPE_MIXED
769-
:noindex:
832+
833+
The named element allows character data, optionally interspersed with
834+
the named children; this is used for content models such as
835+
``(#PCDATA)`` and ``(#PCDATA | A | B)*``.
770836

771837

772838
.. data:: XML_CTYPE_NAME
773-
:noindex:
839+
840+
The model names a single element, as for ``A``.
774841

775842

776843
.. data:: XML_CTYPE_SEQ
777-
:noindex:
778844

779845
Models which represent a series of models which follow one after the other are
780846
indicated with this model type. This is used for models such as ``(A, B, C)``.
@@ -783,25 +849,21 @@ The constants in the quantifier group are:
783849

784850

785851
.. data:: XML_CQUANT_NONE
786-
:noindex:
787852

788853
No modifier is given, so it can appear exactly once, as for ``A``.
789854

790855

791856
.. data:: XML_CQUANT_OPT
792-
:noindex:
793857

794858
The model is optional: it can appear once or not at all, as for ``A?``.
795859

796860

797861
.. data:: XML_CQUANT_PLUS
798-
:noindex:
799862

800863
The model must occur one or more times (like ``A+``).
801864

802865

803866
.. data:: XML_CQUANT_REP
804-
:noindex:
805867

806868
The model must occur zero or more times, as for ``A*``.
807869

@@ -885,7 +947,7 @@ The ``errors`` module has the following attributes:
885947
.. data:: XML_ERROR_NO_ELEMENTS
886948

887949
The document contains no elements (XML requires all documents to contain exactly
888-
one top-level element)..
950+
one top-level element).
889951

890952

891953
.. data:: XML_ERROR_NO_MEMORY
@@ -988,7 +1050,7 @@ The ``errors`` module has the following attributes:
9881050

9891051
.. data:: XML_ERROR_XML_DECL
9901052

991-
The document contained no document element at all.
1053+
There was an error parsing the XML declaration.
9921054

9931055

9941056
.. data:: XML_ERROR_TEXT_DECL

Modules/clinic/pyexpat.c.h

Lines changed: 12 additions & 6 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)