Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 15 additions & 4 deletions Doc/library/base64.rst
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ The :rfc:`4648` encodings are suitable for encoding binary data so that it can b
safely sent by email, used as parts of URLs, or included as part of an HTTP
POST request.

.. function:: b64encode(s, altchars=None)
.. function:: b64encode(s, altchars=None, *, wrapcol=0)

Encode the :term:`bytes-like object` *s* using Base64 and return the encoded
:class:`bytes`.
Expand All @@ -64,6 +64,15 @@ POST request.
May assert or raise a :exc:`ValueError` if the length of *altchars* is not 2. Raises a
:exc:`TypeError` if *altchars* is not a :term:`bytes-like object`.

If *wrapcol* is non-zero, the output will be represented in lines of
no more than *wrapcol* characters each, separated by a newline
(``b'\n'``) character.
If *wrapcol* is zero (default), the output will be represented as
a single line.

.. versionchanged:: next
Added the *wrapcol* parameter.


.. function:: b64decode(s, altchars=None, validate=False)

Expand Down Expand Up @@ -214,9 +223,11 @@ Refer to the documentation of the individual functions for more information.
instead of 4 consecutive spaces (ASCII 0x20) as supported by 'btoa'. This
feature is not supported by the "standard" Ascii85 encoding.

*wrapcol* controls whether the output should have newline (``b'\n'``)
characters added to it. If this is non-zero, each output line will be
at most this many characters long, excluding the trailing newline.
If *wrapcol* is non-zero, the output will be represented in lines of
no more than *wrapcol* characters each, separated by a newline
(``b'\n'``) character.
If *wrapcol* is zero (default), the output will be represented as
a single line.

*pad* controls whether the input is padded to a multiple of 4
before encoding. Note that the ``btoa`` implementation always pads.
Expand Down
21 changes: 16 additions & 5 deletions Doc/library/binascii.rst
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ The :mod:`binascii` module defines the following functions:

Valid base64:

* Conforms to :rfc:`3548`.
* Conforms to :rfc:`4648`.
* Contains only characters from the base64 alphabet.
* Contains no excess data after padding (including excess padding, newlines, etc.).
* Does not start with a padding.
Expand All @@ -67,15 +67,26 @@ The :mod:`binascii` module defines the following functions:
Added the *strict_mode* parameter.


.. function:: b2a_base64(data, *, newline=True)
.. function:: b2a_base64(data, *, wrapcol=0, newline=True)

Convert binary data to a line of ASCII characters in base64 coding. The return
value is the converted line, including a newline char if *newline* is
true. The output of this function conforms to :rfc:`3548`.
Convert binary data to a line(s) of ASCII characters in base64 coding,
as specified in :rfc:`4648`.

If *wrapcol* is non-zero, the output will be represented in lines of
no more than *wrapcol* characters each, separated by a newline
(``b'\n'``) character.
If *wrapcol* is zero (default), the output will be represented as
a single line.

If *newline* is true (default), a newline character will be added
at the end of the output.

.. versionchanged:: 3.6
Added the *newline* parameter.

.. versionchanged:: next
Added the *wrapcol* parameter.


.. function:: a2b_qp(data, header=False)

Expand Down
17 changes: 17 additions & 0 deletions Doc/whatsnew/3.15.rst
Original file line number Diff line number Diff line change
Expand Up @@ -428,6 +428,23 @@ argparse
inline code when color output is enabled.
(Contributed by Savannah Ostrowski in :gh:`142390`.)

base64
------

* Added the *pad* parameter in :func:`~base64.z85encode`.
(Contributed by Hauke Dämpfling in :gh:`143103`.)

* Added the *wrapcol* parameter in :func:`~base64.b64encode`.
(Contributed by Serhiy Storchaka in :gh:`143214`.)


binascii
--------

* Added the *wrapcol* parameter in :func:`~binascii.b2a_base64`.
(Contributed by Serhiy Storchaka in :gh:`143214`.)


calendar
--------

Expand Down
1 change: 1 addition & 0 deletions Include/internal/pycore_global_objects_fini_generated.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Include/internal/pycore_global_strings.h
Original file line number Diff line number Diff line change
Expand Up @@ -865,6 +865,7 @@ struct _Py_global_strings {
STRUCT_FOR_ID(which)
STRUCT_FOR_ID(who)
STRUCT_FOR_ID(withdata)
STRUCT_FOR_ID(wrapcol)
STRUCT_FOR_ID(writable)
STRUCT_FOR_ID(write)
STRUCT_FOR_ID(write_through)
Expand Down
1 change: 1 addition & 0 deletions Include/internal/pycore_runtime_init_generated.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 4 additions & 0 deletions Include/internal/pycore_unicodeobject_generated.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

23 changes: 13 additions & 10 deletions Lib/base64.py
Original file line number Diff line number Diff line change
Expand Up @@ -45,14 +45,18 @@ def _bytes_from_decode_data(s):

# Base64 encoding/decoding uses binascii

def b64encode(s, altchars=None):
def b64encode(s, altchars=None, *, wrapcol=0):
"""Encode the bytes-like object s using Base64 and return a bytes object.

Optional altchars should be a byte string of length 2 which specifies an
alternative alphabet for the '+' and '/' characters. This allows an
application to e.g. generate url or filesystem safe Base64 strings.

If wrapcol is non-zero, the output will be represented in lines of
no more than wrapcol characters each, separated by a newline (b'\\n')
character.
"""
encoded = binascii.b2a_base64(s, newline=False)
encoded = binascii.b2a_base64(s, wrapcol=wrapcol, newline=False)
if altchars is not None:
assert len(altchars) == 2, repr(altchars)
return encoded.translate(bytes.maketrans(b'+/', altchars))
Expand Down Expand Up @@ -327,9 +331,9 @@ def a85encode(b, *, foldspaces=False, wrapcol=0, pad=False, adobe=False):
instead of 4 consecutive spaces (ASCII 0x20) as supported by 'btoa'. This
feature is not supported by the "standard" Adobe encoding.

wrapcol controls whether the output should have newline (b'\\n') characters
added to it. If this is non-zero, each output line will be at most this
many characters long, excluding the trailing newline.
If wrapcol is non-zero, the output will be represented in lines of
no more than wrapcol characters each, separated by a newline (b'\\n')
character.

pad controls whether the input is padded to a multiple of 4 before
encoding. Note that the btoa implementation always pads.
Expand Down Expand Up @@ -566,11 +570,10 @@ def encodebytes(s):
"""Encode a bytestring into a bytes object containing multiple lines
of base-64 data."""
_input_type_check(s)
pieces = []
for i in range(0, len(s), MAXBINSIZE):
chunk = s[i : i + MAXBINSIZE]
pieces.append(binascii.b2a_base64(chunk))
return b"".join(pieces)
result = binascii.b2a_base64(s, wrapcol=MAXLINESIZE)
if result == b'\n':
return b''
return result


def decodebytes(s):
Expand Down
19 changes: 9 additions & 10 deletions Lib/email/base64mime.py
Original file line number Diff line number Diff line change
Expand Up @@ -83,16 +83,15 @@ def body_encode(s, maxlinelen=76, eol=NL):
if not s:
return ""

encvec = []
max_unencoded = maxlinelen * 3 // 4
for i in range(0, len(s), max_unencoded):
# BAW: should encode() inherit b2a_base64()'s dubious behavior in
# adding a newline to the encoded string?
enc = b2a_base64(s[i:i + max_unencoded]).decode("ascii")
if enc.endswith(NL) and eol != NL:
enc = enc[:-1] + eol
encvec.append(enc)
return EMPTYSTRING.join(encvec)
if not eol:
return b2a_base64(s, newline=False).decode("ascii")

# BAW: should encode() inherit b2a_base64()'s dubious behavior in
# adding a newline to the encoded string?
enc = b2a_base64(s, wrapcol=maxlinelen).decode("ascii")
if eol != NL:
enc = enc.replace(NL, eol)
return enc


def decode(string):
Expand Down
18 changes: 3 additions & 15 deletions Lib/email/contentmanager.py
Original file line number Diff line number Diff line change
Expand Up @@ -129,19 +129,6 @@ def _finalize_set(msg, disposition, filename, cid, params):
msg.set_param(key, value)


# XXX: This is a cleaned-up version of base64mime.body_encode (including a bug
# fix in the calculation of unencoded_bytes_per_line). It would be nice to
# drop both this and quoprimime.body_encode in favor of enhanced binascii
# routines that accepted a max_line_length parameter.
def _encode_base64(data, max_line_length):
encoded_lines = []
unencoded_bytes_per_line = max_line_length // 4 * 3
for i in range(0, len(data), unencoded_bytes_per_line):
thisline = data[i:i+unencoded_bytes_per_line]
encoded_lines.append(binascii.b2a_base64(thisline).decode('ascii'))
return ''.join(encoded_lines)


def _encode_text(string, charset, cte, policy):
# If max_line_length is 0 or None, there is no limit.
maxlen = policy.max_line_length or sys.maxsize
Expand Down Expand Up @@ -176,7 +163,7 @@ def normal_body(lines): return b'\n'.join(lines) + b'\n'
data = quoprimime.body_encode(normal_body(lines).decode('latin-1'),
maxlen)
elif cte == 'base64':
data = _encode_base64(embedded_body(lines), maxlen)
data = binascii.b2a_base64(embedded_body(lines), wrapcol=maxlen).decode('ascii')
else:
raise ValueError("Unknown content transfer encoding {}".format(cte))
return cte, data
Expand Down Expand Up @@ -234,7 +221,8 @@ def set_bytes_content(msg, data, maintype, subtype, cte='base64',
params=None, headers=None):
_prepare_set(msg, maintype, subtype, headers)
if cte == 'base64':
data = _encode_base64(data, max_line_length=msg.policy.max_line_length)
data = binascii.b2a_base64(data, wrapcol=msg.policy.max_line_length)
data = data.decode('ascii')
elif cte == 'quoted-printable':
# XXX: quoprimime.body_encode won't encode newline characters in data,
# so we can't use it. This means max_line_length is ignored. Another
Expand Down
9 changes: 2 additions & 7 deletions Lib/plistlib.py
Original file line number Diff line number Diff line change
Expand Up @@ -122,13 +122,7 @@ def __hash__(self):
r"\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f]")

def _encode_base64(s, maxlinelength=76):
# copied from base64.encodebytes(), with added maxlinelength argument
maxbinsize = (maxlinelength//4)*3
pieces = []
for i in range(0, len(s), maxbinsize):
chunk = s[i : i + maxbinsize]
pieces.append(binascii.b2a_base64(chunk))
return b''.join(pieces)
return binascii.b2a_base64(s, wrapcol=maxlinelength, newline=False)

def _decode_base64(s):
if isinstance(s, str):
Expand Down Expand Up @@ -385,6 +379,7 @@ def write_bytes(self, data):
maxlinelength = max(
16,
76 - len(self.indent.replace(b"\t", b" " * 8) * self._indent_level))
maxlinelength = maxlinelength // 4 * 4

for line in _encode_base64(data, maxlinelength).split(b"\n"):
if line:
Expand Down
7 changes: 2 additions & 5 deletions Lib/ssl.py
Original file line number Diff line number Diff line change
Expand Up @@ -1534,11 +1534,8 @@ def DER_cert_to_PEM_cert(der_cert_bytes):
"""Takes a certificate in binary DER format and returns the
PEM version of it as a string."""

f = str(base64.standard_b64encode(der_cert_bytes), 'ASCII', 'strict')
ss = [PEM_HEADER]
ss += [f[i:i+64] for i in range(0, len(f), 64)]
ss.append(PEM_FOOTER + '\n')
return '\n'.join(ss)
f = str(base64.b64encode(der_cert_bytes, wrapcol=64), 'ASCII')
return f'{PEM_HEADER}\n{f}\n{PEM_FOOTER}\n'

def PEM_cert_to_DER_cert(pem_cert_string):
"""Takes a certificate in ASCII PEM format and returns the
Expand Down
22 changes: 22 additions & 0 deletions Lib/test/test_base64.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
import base64
import binascii
import string
import sys
import os
from array import array
from test.support import cpython_only
Expand Down Expand Up @@ -172,6 +173,27 @@ def test_b64encode(self):
b"YWJjZGVmZ2hpamtsbW5vcHFyc3R1dnd4eXpBQkNE"
b"RUZHSElKS0xNTk9QUVJTVFVWV1hZWjAxMjM0NT"
b"Y3ODkhQCMwXiYqKCk7Ojw+LC4gW117fQ==")

eq(base64.b64encode(b"www.python.org", wrapcol=0), b'd3d3LnB5dGhvbi5vcmc=')
eq(base64.b64encode(b"www.python.org", wrapcol=8), b'd3d3LnB5\ndGhvbi5v\ncmc=')
eq(base64.b64encode(b"www.python.org", wrapcol=76), b'd3d3LnB5dGhvbi5vcmc=')
eq(base64.b64encode(b"www.python.org", wrapcol=1),
b'd\n3\nd\n3\nL\nn\nB\n5\nd\nG\nh\nv\nb\ni\n5\nv\nc\nm\nc\n=')
eq(base64.b64encode(b"www.python.org", wrapcol=sys.maxsize),
b'd3d3LnB5dGhvbi5vcmc=')
eq(base64.b64encode(b"www.python.org", wrapcol=sys.maxsize*2),
b'd3d3LnB5dGhvbi5vcmc=')
with self.assertRaises(OverflowError):
base64.b64encode(b"www.python.org", wrapcol=2**1000)
with self.assertRaises(ValueError):
base64.b64encode(b"www.python.org", wrapcol=-8)
with self.assertRaises(TypeError):
base64.b64encode(b"www.python.org", wrapcol=8.0)
with self.assertRaises(TypeError):
base64.b64encode(b"www.python.org", wrapcol='8')
eq(base64.b64encode(b"", wrapcol=0), b'')
eq(base64.b64encode(b"", wrapcol=8), b'')

# Test with arbitrary alternative characters
eq(base64.b64encode(b'\xd3V\xbeo\xf7\x1d', altchars=b'*$'), b'01a*b$cd')
eq(base64.b64encode(b'\xd3V\xbeo\xf7\x1d', altchars=bytearray(b'*$')),
Expand Down
37 changes: 37 additions & 0 deletions Lib/test/test_binascii.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
import binascii
import array
import re
import sys
from test.support import bigmemtest, _1G, _4G
from test.support.hypothesis_helper import hypothesis

Expand Down Expand Up @@ -479,6 +480,42 @@ def test_b2a_base64_newline(self):
b'aGVsbG8=\n')
self.assertEqual(binascii.b2a_base64(b, newline=False),
b'aGVsbG8=')
b = self.type2test(b'')
self.assertEqual(binascii.b2a_base64(b), b'\n')
self.assertEqual(binascii.b2a_base64(b, newline=True), b'\n')
self.assertEqual(binascii.b2a_base64(b, newline=False), b'')

def test_b2a_base64_wrapcol(self):
b = self.type2test(b'www.python.org')
self.assertEqual(binascii.b2a_base64(b),
b'd3d3LnB5dGhvbi5vcmc=\n')
self.assertEqual(binascii.b2a_base64(b, wrapcol=0),
b'd3d3LnB5dGhvbi5vcmc=\n')
self.assertEqual(binascii.b2a_base64(b, wrapcol=8),
b'd3d3LnB5\ndGhvbi5v\ncmc=\n')
self.assertEqual(binascii.b2a_base64(b, wrapcol=76),
b'd3d3LnB5dGhvbi5vcmc=\n')
self.assertEqual(binascii.b2a_base64(b, wrapcol=8, newline=False),
b'd3d3LnB5\ndGhvbi5v\ncmc=')
self.assertEqual(binascii.b2a_base64(b, wrapcol=1),
b'd\n3\nd\n3\nL\nn\nB\n5\nd\nG\nh\nv\nb\ni\n5\nv\nc\nm\nc\n=\n')
self.assertEqual(binascii.b2a_base64(b, wrapcol=sys.maxsize),
b'd3d3LnB5dGhvbi5vcmc=\n')
self.assertEqual(binascii.b2a_base64(b, wrapcol=sys.maxsize*2),
b'd3d3LnB5dGhvbi5vcmc=\n')
with self.assertRaises(OverflowError):
binascii.b2a_base64(b, wrapcol=2**1000)
with self.assertRaises(ValueError):
binascii.b2a_base64(b, wrapcol=-8)
with self.assertRaises(TypeError):
binascii.b2a_base64(b, wrapcol=8.0)
with self.assertRaises(TypeError):
binascii.b2a_base64(b, wrapcol='8')
b = self.type2test(b'')
self.assertEqual(binascii.b2a_base64(b), b'\n')
self.assertEqual(binascii.b2a_base64(b, wrapcol=0), b'\n')
self.assertEqual(binascii.b2a_base64(b, wrapcol=8), b'\n')
self.assertEqual(binascii.b2a_base64(b, wrapcol=8, newline=False), b'')

@hypothesis.given(
binary=hypothesis.strategies.binary(),
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Add the *wrapcol* parameter in :func:`binascii.b2a_base64` and
:func:`base64.b64encode`.
Loading
Loading