Skip to content

Commit ddeb500

Browse files
Merge branch 'main' into gh-63102-pull-target
2 parents 85ded3e + e7a3937 commit ddeb500

4 files changed

Lines changed: 47 additions & 8 deletions

File tree

Doc/library/xml.etree.elementtree.rst

Lines changed: 35 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -160,8 +160,37 @@ some storage device. In such cases, blocking reads are unacceptable.
160160
Because it's so flexible, :class:`XMLPullParser` can be inconvenient to use for
161161
simpler use-cases. If you don't mind your application blocking on reading XML
162162
data but would still like to have incremental parsing capabilities, take a look
163-
at :func:`iterparse`. It can be useful when you're reading a large XML document
164-
and don't want to hold it wholly in memory.
163+
at :func:`iterparse`.
164+
165+
Note that both parsers build the tree incrementally: it is not freed
166+
incrementally, so every parsed element is kept until the whole document is
167+
read. To keep the memory usage low, get rid of the data which is not needed
168+
any more.
169+
170+
If the processed elements are large, it is enough to clear them.
171+
This works wherever they are in the tree,
172+
but the emptied elements are left in it::
173+
174+
for event, elem in ET.iterparse(source):
175+
if elem.tag == 'record':
176+
process(elem)
177+
elem.clear()
178+
179+
If an element has a large number of children,
180+
remove the processed children from it::
181+
182+
for event, elem in ET.iterparse(source, events=('start', 'end')):
183+
if event == 'start' and elem.tag == 'parent':
184+
parent = elem
185+
elif event == 'end' and elem.tag == 'child':
186+
process(elem)
187+
parent.remove(elem)
188+
189+
These examples are not universal,
190+
they only give an idea for two common cases.
191+
If you do not need a tree at all,
192+
parse with :class:`XMLParser` and a custom target instead;
193+
it is not built then, and nothing has to be removed.
165194

166195
Where *immediate* feedback through events is wanted, calling method
167196
:meth:`XMLPullParser.flush` can help reduce delay;
@@ -643,6 +672,10 @@ Functions
643672
for applications where blocking reads can't be made. For fully non-blocking
644673
parsing, see :class:`XMLPullParser`.
645674

675+
The tree is only built incrementally, it is not freed incrementally:
676+
every parsed element is kept until the whole document is read.
677+
See :ref:`elementtree-pull-parsing` for how to keep the memory usage low.
678+
646679
.. note::
647680

648681
:func:`iterparse` only guarantees that it has seen the ">" character of a

Lib/test/test_marshal.py

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -116,6 +116,14 @@ def test_bytes(self):
116116
for s in [b"", b"Andr\xe8 Previn", b"abc", b" "*10000]:
117117
self.helper(s)
118118

119+
@support.cpython_only
120+
def test_bytes_singleton(self):
121+
for version in range(marshal.version + 1):
122+
for sample in [b"", b"x"]:
123+
new = marshal.loads(marshal.dumps(sample, version))
124+
self.assertIs(new, sample)
125+
126+
119127
class ExceptionTestCase(unittest.TestCase):
120128
def test_exceptions(self):
121129
new = marshal.loads(marshal.dumps(StopIteration))
Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
:func:`marshal.load` and :func:`marshal.loads` can now get 1-byte string
2+
singletons. Patch by Victor Stinner.

Python/marshal.c

Lines changed: 2 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1388,16 +1388,12 @@ r_object(RFILE *p)
13881388
}
13891389
break;
13901390
}
1391-
v = PyBytes_FromStringAndSize((char *)NULL, n);
1392-
if (v == NULL)
1393-
break;
13941391
ptr = r_string(n, p);
13951392
if (ptr == NULL) {
1396-
Py_DECREF(v);
13971393
break;
13981394
}
1399-
memcpy(PyBytes_AS_STRING(v), ptr, n);
1400-
retval = v;
1395+
// Get a singleton for 1-byte string
1396+
retval = PyBytes_FromStringAndSize(ptr, n); // can be NULL
14011397
R_REF(retval);
14021398
break;
14031399
}

0 commit comments

Comments
 (0)