Per RFC 2047:
> [...] these encoding schemes allow the
> encoding of arbitrary octet values, mail readers that implement this
> decoding should also ensure that display of the decoded data on the
> recipient's terminal will not cause unwanted side-effects
It seems that the "quoted-word" scheme is a valid way to include
a newline character in a header value, just like we already allow
undecodable bytes or control characters.
They do need to be properly quoted when serialized to text, though.
This should fail for custom fold() implementations that aren't careful
about newlines.
(cherry picked from commit 097633981879b3c9de9a1dd120d3aa585ecc2384)
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Bas Bloemsaat <bas@bloemsaat.org>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* Remove backtracking when parsing tarfile headers
* Rewrite PAX header parsing to be stricter
* Optimize parsing of GNU extended sparse headers v0.0
(cherry picked from commit 34ddb64d088dd7ccc321f6103d23153256caa5d4)
Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
(cherry picked from commit f071f01b7b7e19d7d6b3a4b0ec62f820ecb14660)
Co-authored-by: Russell Keith-Magee <russell@keith-magee.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Authenticate socket connection for `socket.socketpair()` fallback when the platform does not have a native `socketpair` C API. We authenticate in-process using `getsocketname` and `getpeername` (thanks to Nathaniel J Smith for that suggestion).
(cherry picked from commit 78df1043dbdce5c989600616f9f87b4ee72944e5)
Co-authored-by: Seth Michael Larson <seth@python.org>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
[3.10] gh-112769: test_zlib: test_zlib: Fix comparison of ZLIB_RUNTIME_VERSION with non-int suffix
zlib-ng defines the version as "1.3.0.zlib-ng".
(cherry picked from commit d384813)
Co-authored-by: Miro Hrončok miro@hroncok.cz
The _private_networks variables, used by various is_private
implementations, were missing some ranges and at the same time had
overly strict ranges (where there are more specific ranges considered
globally reachable by the IANA registries).
This patch updates the ranges with what was missing or otherwise
incorrect.
100.64.0.0/10 is left alone, for now, as it's been made special in [1].
The _address_exclude_many() call returns 8 networks for IPv4, 121
networks for IPv6.
[1] https://github.com/python/cpython/issues/61602
In 3.10 and below, is_private checks whether the network and broadcast
address are both private.
In later versions (where the test wss backported from), it checks
whether they both are in the same private network.
For 0.0.0.0/0, both 0.0.0.0 and 255.225.255.255 are private,
but one is in 0.0.0.0/8 ("This network") and the other in
255.255.255.255/32 ("Limited broadcast").
---------
Co-authored-by: Jakub Stasiak <jakub@stasiak.at>
This fixes XML unittest fallout from the https://github.com/python/cpython/issues/115398 security fix. When configured using `--with-system-expat` on systems with older pre 2.6.0 versions of libexpat, our unittests were failing.
Co-authored-by: Sebastian Pipping <sebastian@pipping.org>
Use of a proxy is intended to defer DNS for the hosts to the proxy itself, rather than a potential for information leak of the host doing DNS resolution itself for any reason. Proxy bypass lists are strictly name based. Most implementations of proxy support agree.
(cherry picked from commit c43b26d02eaa103756c250e8d36829d388c5f3be)
Co-authored-by: Weii Wang <weii.wang@canonical.com>
Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods:
- `xml.etree.ElementTree.XMLParser.flush`
- `xml.etree.ElementTree.XMLPullParser.flush`
- `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled`
- `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled`
- `xml.sax.expatreader.ExpatParser.flush`
Based on the "flush" idea from https://github.com/python/cpython/pull/115138#issuecomment-1932444270 .
Includes code suggested-by: Snild Dolkow <snild@sony.com>
and by core dev Serhiy Storchaka.
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Doc/library/xml.rst: Document CVE-2023-52425 under "XML vulnerabilities"
(cherry picked from commit fbd40ce46e7335a5dbaf48a3aa841be22d7302ba)
Co-authored-by: Sebastian Pipping <sebastian@pipping.org>
gh-114572: Fix locking in cert_store_stats and get_ca_certs (GH-114573)
* gh-114572: Fix locking in cert_store_stats and get_ca_certs
cert_store_stats and get_ca_certs query the SSLContext's X509_STORE with
X509_STORE_get0_objects, but reading the result requires a lock. See
https://github.com/openssl/openssl/pull/23224 for details.
Instead, use X509_STORE_get1_objects, newly added in that PR.
X509_STORE_get1_objects does not exist in current OpenSSLs, but we can
polyfill it with X509_STORE_lock and X509_STORE_unlock.
* Work around const-correctness problem
* Add missing X509_STORE_get1_objects failure check
* Add blurb
(cherry picked from commit bce693111bff906ccf9281c22371331aaff766ab)
Co-authored-by: David Benjamin <davidben@google.com>
Manual backport due to code differences.
(cherry picked from commit e071b0d558b2f5cddd5a9fc6afadb4ba109ec77e)
Co-authored-by: Seth Michael Larson <seth@python.org>
gh-115133: Fix tests for XMLPullParser with Expat 2.6.0 (GH-115164)
Feeding the parser by too small chunks defers parsing to prevent
CVE-2023-52425. Future versions of Expat may be more reactive.
(cherry picked from commit 4a08e7b3431cd32a0daf22a33421cd3035343dc4)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-111239: Update Windows build to use zlib 1.3.1 (GH-114877)
(cherry picked from commit 618d7256e78da8200f6e2c6235094a1ef885dca4)
Co-authored-by: Zachary Ware <zach@python.org>
Raise BadZipFile when try to read an entry that overlaps with other entry or
central directory.
(cherry picked from commit 66363b9a7b9fe7c99eba3a185b74c5fdbf842eba)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* Fix a crash when pass UINT_MAX.
* Fix an integer overflow on 64-bit non-Windows platforms.
(cherry picked from commit 0daf555c6fb3feba77989382135a58215e1d70a5)
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
gh-109991: Update GitHub CI workflows to use OpenSSL 3.0.11 and multissltests to use 1.1.1w, 3.0.11, and 3.1.3.
(cherry picked from commit c88037d137a98d7c399c7bd74d5117b5bcae1543)
Output with one wheel:
```
❯ GITHUB_ACTIONS=true ./Tools/build/verify_ensurepip_wheels.py
Verifying checksum for /Volumes/RAMDisk/cpython/Lib/ensurepip/_bundled/pip-23.2.1-py3-none-any.whl.
Expected digest: 7ccf472345f20d35bdc9d1841ff5f313260c2c33fe417f48c30ac46cccabf5be
Actual digest: 7ccf472345f20d35bdc9d1841ff5f313260c2c33fe417f48c30ac46cccabf5be
::notice file=/Volumes/RAMDisk/cpython/Lib/ensurepip/_bundled/pip-23.2.1-py3-none-any.whl::Successfully verified the checksum of the pip wheel.
```
Output with two wheels:
```
❯ GITHUB_ACTIONS=true ./Tools/build/verify_ensurepip_wheels.py
::error file=/Volumes/RAMDisk/cpython/Lib/ensurepip/_bundled/pip-22.0.4-py3-none-any.whl::Found more than one wheel for package pip.
::error file=/Volumes/RAMDisk/cpython/Lib/ensurepip/_bundled/pip-23.2.1-py3-none-any.whl::Found more than one wheel for package pip.
```
Output without wheels:
```
❯ GITHUB_ACTIONS=true ./Tools/build/verify_ensurepip_wheels.py
::error file=::Could not find a pip wheel on disk.
```
(cherry picked from commit f8a047941f2e4a1848700c21d58a08c9ec6a9c68)
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
* In preauth tests of test_ssl, explicitly break reference cycles
invoving SingleConnectionTestServerThread to make sure that the
thread is deleted. Otherwise, the test marks the environment as
altered because the threading module sees a "dangling thread"
(SingleConnectionTestServerThread). This test leak was introduced
by the test added for the fix of issue gh-108310.
* Use support.SHORT_TIMEOUT instead of hardcoded 1.0 or 2.0 seconds
timeout.
* SingleConnectionTestServerThread.run() catchs TimeoutError
* Fix a race condition (missing synchronization) in
test_preauth_data_to_tls_client(): the server now waits until the
client connect() completed in call_after_accept().
* test_https_client_non_tls_response_ignored() calls server.join()
explicitly.
* Replace "localhost" with server.listener.getsockname()[0].
(cherry picked from commit 592bacb6fc0833336c0453e818e9b95016e9fd47)
Co-authored-by: Victor Stinner <vstinner@python.org>
Explicitly break a reference cycle when SSLSocket._create() raises an
exception. Clear the variable storing the exception, since the
exception traceback contains the variables and so creates a reference
cycle.
This test leak was introduced by the test added for the fix of GH-108310.
(cherry picked from commit 64f99350351bc46e016b2286f36ba7cd669b79e3)
Co-authored-by: Victor Stinner <vstinner@python.org>
Previously *consumed was not set in this case.
(cherry picked from commit b8b3e6afc0a48c3cbb7c36d2f73e332edcd6058c)
(cherry picked from commit f08e52ccb027f6f703302b8c1a82db9fd3934270)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-108310: Fix CVE-2023-40217: Check for & avoid the ssl pre-close flaw
Instances of `ssl.SSLSocket` were vulnerable to a bypass of the TLS handshake
and included protections (like certificate verification) and treating sent
unencrypted data as if it were post-handshake TLS encrypted data.
The vulnerability is caused when a socket is connected, data is sent by the
malicious peer and stored in a buffer, and then the malicious peer closes the
socket within a small timing window before the other peers’ TLS handshake can
begin. After this sequence of events the closed socket will not immediately
attempt a TLS handshake due to not being connected but will also allow the
buffered data to be read as if a successful TLS handshake had occurred.
Co-authored-by: Gregory P. Smith [Google LLC] <greg@krypto.org>
