Skip to content

gh-160187: Reject ISO-2022 designations into the wrong G set - #160188

Merged
gpshead merged 2 commits into
python:mainfrom
gpshead:iso2022-wrong-g-set
Nov 1, 2026
Merged

gpshead merged 2 commits into
python:mainfrom
gpshead:iso2022-wrong-g-set

Conversation

@gpshead

@gpshead gpshead commented Oct 22, 2026 •

Copy link
Copy Markdown
Member

iso2022processesc() now requires the designation table entry's plane to match the G set the escape designates to, and accepts ASCII only for G0. Conforming encoders (including ours) never emit the escapes this rejects, but strict decoding of such malformed data now raises UnicodeDecodeError where it used to return wrong text.

The RuntimeError alone could be fixed in maintenance branches by having iso2022processg2() return an ordinary decode error instead of MBERR_INTERNAL.

Fixes #160187

ANT-2026-TH3IZKXQ

@gpshead gpshead self-assigned this Oct 22, 2026
@gpshead
gpshead marked this pull request as ready for review October 22, 2026 04:34
@gpshead
gpshead requested a review from corona10 as a code owner October 22, 2026 04:34
@gpshead gpshead added the sprint label Oct 22, 2026
@github-project-automation github-project-automation Bot moved this to Todo in Sprint Oct 22, 2026
@gpshead gpshead added the needs backport to 3.15 bugs and security fixes label Oct 22, 2026
The ISO-2022 decoders checked only that a designated charset belonged to
the encoding, not that it went to the code element the encoding assigns
it to.
@gpshead
gpshead force-pushed the iso2022-wrong-g-set branch from 65399ab to 7c83abe Compare October 22, 2026 04:45
@gpshead gpshead added the needs backport to 3.14 bugs and security fixes label Nov 1, 2026
@gpshead
gpshead enabled auto-merge (squash) November 1, 2026 06:05
@gpshead
gpshead disabled auto-merge November 1, 2026 06:05
@gpshead
gpshead enabled auto-merge (squash) November 1, 2026 06:05
@gpshead gpshead removed the needs backport to 3.14 bugs and security fixes label Nov 1, 2026
@gpshead
gpshead merged commit 5f28e31 into python:main Nov 1, 2026
61 checks passed
@github-project-automation github-project-automation Bot moved this from Todo to Done in Sprint Nov 1, 2026
@miss-islington-app

Copy link
Copy Markdown

Thanks @gpshead for the PR 🌮🎉.. I'm working now to backport this PR to: 3.15.
🐍🍒⛏🤖

@bedevere-app

bedevere-app Bot commented Nov 1, 2026

Copy link
Copy Markdown

GH-161064 is a backport of this pull request to the 3.15 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.15 bugs and security fixes label Nov 1, 2026
@bedevere-bot

Copy link
Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x Fedora Stable 3.x (tier-3) has failed when building commit 5f28e31.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1636/builds/3793) and take a look at the build logs.
  4. Check if the failure is related to this commit (5f28e31) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1636/builds/3793

Failed tests:

  • test_zipimport

Failed subtests:

  • testGetCompiledSource - test.test_zipimport.DeflateCompressedZipImportTestCase.testGetCompiledSource
  • testBoth - test.test_zipimport.UncompressedZipImportTestCase.testBoth
  • testGetCompiledSource - test.test_zipimport.UncompressedZipImportTestCase.testGetCompiledSource
  • testGetCompiledSource - test.test_zipimport.ZStdCompressedZipImportTestCase.testGetCompiledSource
  • test_syntax_error_in_source_with_valid_pyc - test.test_zipimport.UncompressedZipImportTestCase.test_syntax_error_in_source_with_valid_pyc
  • test_syntax_error_in_source_with_valid_pyc - test.test_zipimport.ZStdCompressedZipImportTestCase.test_syntax_error_in_source_with_valid_pyc
  • testBoth - test.test_zipimport.DeflateCompressedZipImportTestCase.testBoth
  • testBoth - test.test_zipimport.ZStdCompressedZipImportTestCase.testBoth
  • test_syntax_error_in_source_with_valid_pyc - test.test_zipimport.DeflateCompressedZipImportTestCase.test_syntax_error_in_source_with_valid_pyc

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (test.test_zipimport.UncompressedZipImportTestCase.testTraceback) ... ok


Traceback (most recent call last):
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.x.cstratak-fedora-stable-s390x/build/Lib/test/test_zipimport.py", line 286, in test_syntax_error_in_source_with_valid_pyc
    self.assertEqual(spec.origin, expected_path)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: '/var[84 chars]uild/build/test_python_927939æ/junk95142.zip/ziptestmodule.py' != '/var[84 chars]uild/build/test_python_927939æ/junk95142.zip/ziptestmodule.pyc'
- /var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.x.cstratak-fedora-stable-s390x/build/build/test_python_927939æ/junk95142.zip/ziptestmodule.py
+ /var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.x.cstratak-fedora-stable-s390x/build/build/test_python_927939æ/junk95142.zip/ziptestmodule.pyc
?                                                                                                                                                      +


Traceback (most recent call last):
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.x.cstratak-fedora-stable-s390x/build/Lib/test/test_zipimport.py", line 315, in testBoth
    self.doTest(pyc_ext, files, TESTMOD)
    ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.x.cstratak-fedora-stable-s390x/build/Lib/test/test_zipimport.py", line 160, in doTest
    self.doTestWithPreBuiltZip(expected_ext, *modules, **kw)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.x.cstratak-fedora-stable-s390x/build/Lib/test/test_zipimport.py", line 174, in doTestWithPreBuiltZip
    self.assertEqual(file, os.path.join(zip_path,
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                         *modules) + expected_ext)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: '/var[92 chars]ld/test_python_933280æ/junk95142.zip/pre/fix/ziptestmodule.py' != '/var[92 chars]ld/test_python_933280æ/junk95142.zip/pre/fix/ziptestmodule.pyc'
- /var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.x.cstratak-fedora-stable-s390x/build/build/test_python_933280æ/junk95142.zip/pre/fix/ziptestmodule.py
+ /var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.x.cstratak-fedora-stable-s390x/build/build/test_python_933280æ/junk95142.zip/pre/fix/ziptestmodule.pyc
?                                                                                                                                                              +


Traceback (most recent call last):
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.x.cstratak-fedora-stable-s390x/build/Lib/test/test_zipimport.py", line 315, in testBoth
    self.doTest(pyc_ext, files, TESTMOD)
    ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.x.cstratak-fedora-stable-s390x/build/Lib/test/test_zipimport.py", line 160, in doTest
    self.doTestWithPreBuiltZip(expected_ext, *modules, **kw)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.x.cstratak-fedora-stable-s390x/build/Lib/test/test_zipimport.py", line 174, in doTestWithPreBuiltZip
    self.assertEqual(file, os.path.join(zip_path,
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                         *modules) + expected_ext)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: '/var[92 chars]ld/test_python_927939æ/junk95142.zip/pre/fix/ziptestmodule.py' != '/var[92 chars]ld/test_python_927939æ/junk95142.zip/pre/fix/ziptestmodule.pyc'
- /var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.x.cstratak-fedora-stable-s390x/build/build/test_python_927939æ/junk95142.zip/pre/fix/ziptestmodule.py
+ /var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.x.cstratak-fedora-stable-s390x/build/build/test_python_927939æ/junk95142.zip/pre/fix/ziptestmodule.pyc
?                                                                                                                                                              +


Traceback (test.test_zipimport.ZStdCompressedZipImportTestCase.testTraceback) ... ok


Traceback (most recent call last):
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.x.cstratak-fedora-stable-s390x/build/Lib/test/test_zipimport.py", line 877, in testGetCompiledSource
    self.doTest(pyc_ext, files, TESTMOD, call=self.assertModuleSource)
    ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.x.cstratak-fedora-stable-s390x/build/Lib/test/test_zipimport.py", line 160, in doTest
    self.doTestWithPreBuiltZip(expected_ext, *modules, **kw)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.x.cstratak-fedora-stable-s390x/build/Lib/test/test_zipimport.py", line 174, in doTestWithPreBuiltZip
    self.assertEqual(file, os.path.join(zip_path,
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                         *modules) + expected_ext)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: '/var[92 chars]ld/test_python_933280æ/junk95142.zip/pre/fix/ziptestmodule.py' != '/var[92 chars]ld/test_python_933280æ/junk95142.zip/pre/fix/ziptestmodule.pyc'
- /var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.x.cstratak-fedora-stable-s390x/build/build/test_python_933280æ/junk95142.zip/pre/fix/ziptestmodule.py
+ /var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.x.cstratak-fedora-stable-s390x/build/build/test_python_933280æ/junk95142.zip/pre/fix/ziptestmodule.pyc
?                                                                                                                                                              +


Traceback (most recent call last):
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.x.cstratak-fedora-stable-s390x/build/Lib/test/test_zipimport.py", line 286, in test_syntax_error_in_source_with_valid_pyc
    self.assertEqual(spec.origin, expected_path)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: '/var[84 chars]uild/build/test_python_933280æ/junk95142.zip/ziptestmodule.py' != '/var[84 chars]uild/build/test_python_933280æ/junk95142.zip/ziptestmodule.pyc'
- /var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.x.cstratak-fedora-stable-s390x/build/build/test_python_933280æ/junk95142.zip/ziptestmodule.py
+ /var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.x.cstratak-fedora-stable-s390x/build/build/test_python_933280æ/junk95142.zip/ziptestmodule.pyc
?                                                                                                                                                      +


Traceback (most recent call last):
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.x.cstratak-fedora-stable-s390x/build/Lib/test/test_zipimport.py", line 877, in testGetCompiledSource
    self.doTest(pyc_ext, files, TESTMOD, call=self.assertModuleSource)
    ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.x.cstratak-fedora-stable-s390x/build/Lib/test/test_zipimport.py", line 160, in doTest
    self.doTestWithPreBuiltZip(expected_ext, *modules, **kw)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.x.cstratak-fedora-stable-s390x/build/Lib/test/test_zipimport.py", line 174, in doTestWithPreBuiltZip
    self.assertEqual(file, os.path.join(zip_path,
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                         *modules) + expected_ext)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: '/var[92 chars]ld/test_python_927939æ/junk95142.zip/pre/fix/ziptestmodule.py' != '/var[92 chars]ld/test_python_927939æ/junk95142.zip/pre/fix/ziptestmodule.pyc'
- /var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.x.cstratak-fedora-stable-s390x/build/build/test_python_927939æ/junk95142.zip/pre/fix/ziptestmodule.py
+ /var/lib/buildbot/worker/cstratak-fedora-stable-s390x/3.x.cstratak-fedora-stable-s390x/build/build/test_python_927939æ/junk95142.zip/pre/fix/ziptestmodule.pyc
?                                                                                                                                                              +


Traceback (test.test_zipimport.DeflateCompressedZipImportTestCase.testTraceback) ... ok

gpshead added a commit that referenced this pull request Nov 1, 2026
…H-160188) (#161064)

gh-160187: Reject ISO-2022 designations into the wrong G set (GH-160188)

The ISO-2022 decoders checked only that a designated charset belonged to
the encoding, not that it went to the code element the encoding assigns
it to.

ANT-2026-TH3IZKXQ
(cherry picked from commit 5f28e31)

Co-authored-by: Gregory P. Smith <68491+gpshead@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

ISO-2022 decoders accept charset designations into the wrong G set

2 participants