Advertisement
Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- this patch series consists of 1 patches.
- [?1034hCc:
- displaying [PATCH] https: support tls sni (server name indication) for https urls (issue3090) ...
- Content-Type: text/plain; charset="us-ascii"
- MIME-Version: 1.0
- Content-Transfer-Encoding: 7bit
- Subject: [PATCH] https: support tls sni (server name indication) for https
- urls (issue3090)
- X-Mercurial-Node: bd1dc245a27e4a723f820c6261186022c28f74ac
- Message-Id: <bd1dc245a27e4a723f82.1377549437@localhost>
- User-Agent: Mercurial-patchbomb/2.4.2
- Date: Mon, 26 Aug 2013 14:37:17 -0600
- From: Alex Orange <crazycasta@gmail.com>
- To: mercurial-devel@selenic.com
- Cc: q
- # HG changeset patch
- # User Alex Orange <crazycasta@gmail.com>
- # Date 1377425168 21600
- # Node ID bd1dc245a27e4a723f820c6261186022c28f74ac
- # Parent d4a0055af149cdea20b3136b66cae8a24b2e2a98
- https: support tls sni (server name indication) for https urls (issue3090)
- SNI is a common way of sharing servers across multiple domains using separate
- SSL certificates. Python 2.x does not, and will not, support SNI according to:
- http://bugs.python.org/issue5639#msg192234. In order to support SNI pyOpenSSL
- and pyasn1 have been used to emulate Python's SSLSocket object (the one
- returned by wrap_socket). Additionally, changes are made to code in httpclient,
- sslutil, and url to pass the server hostname to the relavent wrap_socket
- wrapper function.
- The research I did led me to the conclusion that an OpenSSL based solution
- would best replicate that existing python ssl object. As opposed to a GNUTLS
- solution like PyGnuTLS (https://gitorious.org/pygnutls). The two packages I
- found that perform the basic functions of the python ssl object are pyOpenSSL
- (http://pythonhosted.org/pyOpenSSL/) and M2Crypto
- (http://www.heikkitoivonen.net/m2crypto/). M2Crypto does not support sending
- the host name as far as I can tell, which leaves pyOpenSSL. The shortcoming of
- both of these libraries is that they do not provide give the subjectAltName
- subobjects as separate objects. They give either the DER encoded raw data or
- an openssl command line style string "DNS:a.b.com, DNS:c.d.com, ...". I did not
- want to parse this string in case a certificate came up with a dNSName had the
- form 'a.b.com, DNS:c.d.com' which could conceivably lead to a security problem.
- In order to handle this problem I used pyasn1 to parse the subjectAltName data
- along with code taken from ndg-httpsclient. There appears to also be an
- way to do this directly through python's ssl module with an undocumented
- function called _test_decode_cert. However this would involve writing
- certificates to temporary files and then reading them back in. This seems like
- a bad idea both because the function is undocumented (and therefore vulnerable
- to going away without warning) and a possible security risk: writing to files
- and then reading them back in. Finally, to make the all of this as unobtrusive
- as possible I wrapped this functionality into a single class that emulates the
- python ssl class called SocketWrapper in opensslwrap.py.
- Finally, imports of pyOpenSSL and pyasn1 are tried before the import of ssl in
- httpclient/socketutil.py and sslutil.py. This prefers the use of this scheme
- over the built-in ssl module for those that have the supporting packages.
- diff -r d4a0055af149 -r bd1dc245a27e mercurial/httpclient/__init__.py
- --- a/mercurial/httpclient/__init__.py Fri Aug 23 16:16:22 2013 -0400
- +++ b/mercurial/httpclient/__init__.py Sun Aug 25 04:06:08 2013 -0600
- @@ -325,6 +325,8 @@
- raise Exception('ssl requested but unavailable on this Python')
- self.ssl = use_ssl
- self.ssl_opts = ssl_opts
- + if 'server_hostname' not in self.ssl_opts:
- + self.ssl_opts['server_hostname'] = self.host
- self._ssl_validator = ssl_validator
- self.host = host
- self.sock = None
- diff -r d4a0055af149 -r bd1dc245a27e mercurial/httpclient/opensslwrap.py
- --- /dev/null Thu Jan 01 00:00:00 1970 +0000
- +++ b/mercurial/httpclient/opensslwrap.py Sun Aug 25 04:06:08 2013 -0600
- @@ -0,0 +1,123 @@
- +import OpenSSL
- +import pyasn1.codec.der.decoder
- +import subjaltname
- +
- +import socket
- +
- +# Try to use python's builtin SSLError if we can, imitate it as best as we can
- +# if the import fails
- +try:
- + import ssl
- +except ImportError:
- + class ssl(object):
- + class SSLError(socket.error):
- + pass
- +
- +# Based heavily on code from:
- +# https://github.com/t-8ch/requests/blob/d7908a9fdef7bca16e384ca42478d69d1894c8b6/requests/packages/urllib3/contrib/pyopenssl.py check-code-ignore
- +CERT_NONE = OpenSSL.SSL.VERIFY_NONE
- +CERT_OPTIONAL = OpenSSL.SSL.VERIFY_PEER
- +CERT_REQUIRED = OpenSSL.SSL.VERIFY_PEER | \
- + OpenSSL.SSL.VERIFY_FAIL_IF_NO_PEER_CERT
- +
- +_PROTOCOL_SSLv23 = 2
- +_PROTOCOL_SSLv3 = 1
- +_PROTOCOL_TLSv1 = 3
- +_openssl_versions = {
- + _PROTOCOL_SSLv23: OpenSSL.SSL.SSLv23_METHOD,
- + _PROTOCOL_SSLv3: OpenSSL.SSL.SSLv3_METHOD,
- + _PROTOCOL_TLSv1: OpenSSL.SSL.TLSv1_METHOD,
- +}
- +
- +class socketwrapper(object):
- + def __init__(self, connection, sock):
- + self.connection = connection
- + self.sock = sock
- +
- + def makefile(self, mode, bufsize=-1):
- + return socket._fileobject(self.connection, mode, bufsize)
- +
- + def getpeercert(self, binary_form=False):
- + x509 = self.connection.get_peer_certificate()
- + if not x509:
- + raise ssl.SSLError('')
- +
- + if binary_form:
- + return OpenSSL.crypto.dump_certificate(
- + OpenSSL.crypto.FILETYPE_ASN1,
- + x509)
- +
- + dns_name = []
- + general_names = subjaltname.SubjectAltName()
- +
- + for i in range(x509.get_extension_count()):
- + ext = x509.get_extension(i)
- + ext_name = ext.get_short_name()
- + if ext_name != 'subjectAltName':
- + continue
- +
- + ext_dat = ext.get_data()
- + der_decoder = pyasn1.codec.der.decoder
- + decoded_dat = der_decoder.decode(ext_dat,
- + asn1Spec=general_names)
- +
- + for name in decoded_dat:
- + if not isinstance(name, subjaltname.SubjectAltName):
- + continue
- + for entry in range(len(name)):
- + component = name.getComponentByPosition(entry)
- + if component.getName() != 'dNSName':
- + continue
- + dns_name.append(('DNS', str(component.getComponent())))
- +
- + return {
- + 'subject': (
- + (('commonName', x509.get_subject().CN),),
- + ),
- + 'subjectAltName': dns_name
- + }
- +
- + # Pass any unhandle function calls on to connection
- + def __getattr__(self, name):
- + try:
- + return getattr(self.connection, name)
- + except AttributeError:
- + return getattr(self.sock, name)
- +
- +class OpenSSLReformattedError(Exception):
- + def __init__(self, e):
- + self.e = e
- +
- + def __str__(self):
- + return '*:%s:%s (glob)'%(self.e.args[0][0][1], self.e.args[0][0][2])
- +
- +
- +def wrap_socket(sock, keyfile=None, certfile=None, server_side=False,
- + cert_reqs=CERT_NONE, ssl_version=_PROTOCOL_TLSv1,
- + ca_certs=None, do_handshake_on_connect=True,
- + suppress_ragged_eofs=True, server_hostname=None):
- + ctx = OpenSSL.SSL.Context(_openssl_versions[ssl_version])
- + if certfile:
- + ctx.use_certificate_file(certfile)
- + if keyfile:
- + ctx.use_privatekey_file(keyfile)
- + if cert_reqs != CERT_NONE:
- + ctx.set_verify(cert_reqs, lambda a, b, err_no, c, d: err_no == 0)
- + if ca_certs:
- + try:
- + ctx.load_verify_locations(ca_certs, None)
- + except OpenSSL.SSL.Error, e:
- + raise ssl.SSLError('bad ca_certs: %r' % ca_certs,
- + OpenSSLReformattedError(e))
- +
- + cnx = OpenSSL.SSL.Connection(ctx, sock)
- + if server_hostname is not None:
- + cnx.set_tlsext_host_name(server_hostname)
- + cnx.set_connect_state()
- + try:
- + cnx.do_handshake()
- + except OpenSSL.SSL.Error, e:
- + raise ssl.SSLError('bad handshake',
- + OpenSSLReformattedError(e))
- +
- + return socketwrapper(cnx, sock)
- diff -r d4a0055af149 -r bd1dc245a27e mercurial/httpclient/socketutil.py
- --- a/mercurial/httpclient/socketutil.py Fri Aug 23 16:16:22 2013 -0400
- +++ b/mercurial/httpclient/socketutil.py Sun Aug 25 04:06:08 2013 -0600
- @@ -38,15 +38,27 @@
- logger = logging.getLogger(__name__)
- try:
- - import ssl
- - # make demandimporters load the module
- - ssl.wrap_socket # pylint: disable=W0104
- + # Required to successfully import opensslwrap
- + from OpenSSL import SSL
- + from pyasn1 import types
- +
- + import opensslwrap
- have_ssl = True
- + have_sni = True
- except ImportError:
- - import httplib
- - import urllib2
- - have_ssl = getattr(urllib2, 'HTTPSHandler', False)
- - ssl = False
- + opensslwrap = False
- + try:
- + import ssl
- + # make demandimporters load the module
- + ssl.wrap_socket # pylint: disable=W0104
- + have_ssl = True
- + have_sni = False
- + except ImportError:
- + import httplib
- + import urllib2
- + have_ssl = getattr(urllib2, 'HTTPSHandler', False)
- + have_sni = False
- + ssl = False
- try:
- @@ -75,8 +87,18 @@
- raise socket.error(msg)
- return sock
- -if ssl:
- - wrap_socket = ssl.wrap_socket
- +if opensslwrap:
- + CERT_NONE = opensslwrap.CERT_NONE
- + CERT_OPTIONAL = opensslwrap.CERT_OPTIONAL
- + CERT_REQUIRED = opensslwrap.CERT_REQUIRED
- +
- + wrap_socket = opensslwrap.wrap_socket
- +
- +elif ssl:
- + # Throw away the server hostname since Python 2.7 hasn't a clue about SNI
- + def wrap_socket(server_hostname=None, *args, **kwargs):
- + return ssl.wrap_socket(*args, **kwargs)
- +
- CERT_NONE = ssl.CERT_NONE
- CERT_OPTIONAL = ssl.CERT_OPTIONAL
- CERT_REQUIRED = ssl.CERT_REQUIRED
- @@ -120,7 +142,7 @@
- server_side=False, cert_reqs=CERT_NONE,
- ssl_version=_PROTOCOL_SSLv3, ca_certs=None,
- do_handshake_on_connect=True,
- - suppress_ragged_eofs=True):
- + suppress_ragged_eofs=True, server_hostname=None):
- """Backport of ssl.wrap_socket from Python 2.6."""
- if cert_reqs != CERT_NONE and ca_certs:
- raise CertificateValidationUnsupported(
- diff -r d4a0055af149 -r bd1dc245a27e mercurial/httpclient/subjaltname.py
- --- /dev/null Thu Jan 01 00:00:00 1970 +0000
- +++ b/mercurial/httpclient/subjaltname.py Sun Aug 25 04:06:08 2013 -0600
- @@ -0,0 +1,124 @@
- +"""NDG HTTPS Client package
- +
- +Use pyasn1 to provide support for parsing ASN.1 formatted subjectAltName
- +content for SSL peer verification. Code based on:
- +
- +http://stackoverflow.com/questions/5519958/how-do-i-parse-subjectaltname-extension-data-using-pyasn1
- +"""
- +__author__ = "P J Kershaw"
- +__date__ = "01/02/12"
- +__copyright__ = "(C) 2012 Science and Technology Facilities Council"
- +__license__ = "BSD - see LICENSE file in top-level directory"
- +__contact__ = "Philip.Kershaw@stfc.ac.uk"
- +__revision__ = '$Id$'
- +
- +from pyasn1.type import univ, constraint, char, namedtype, tag
- +
- +
- +class DirectoryString(univ.Choice):
- + """ASN.1 Directory string class"""
- + componentType = namedtype.NamedTypes(
- + namedtype.NamedType(
- + 'teletexString', char.TeletexString()),
- + namedtype.NamedType(
- + 'printableString', char.PrintableString()),
- + namedtype.NamedType(
- + 'universalString', char.UniversalString()),
- + namedtype.NamedType(
- + 'utf8String', char.UTF8String()),
- + namedtype.NamedType(
- + 'bmpString', char.BMPString()),
- + namedtype.NamedType(
- + 'ia5String', char.IA5String()),
- + )
- +
- +
- +class AttributeValue(DirectoryString):
- + """ASN.1 Attribute value"""
- +
- +
- +class AttributeType(univ.ObjectIdentifier):
- + """ASN.1 Attribute type"""
- +
- +
- +class AttributeTypeAndValue(univ.Sequence):
- + """ASN.1 Attribute type and value class"""
- + componentType = namedtype.NamedTypes(
- + namedtype.NamedType('type', AttributeType()),
- + namedtype.NamedType('value', AttributeValue()),
- + )
- +
- +
- +class RelativeDistinguishedName(univ.SetOf):
- + '''ASN.1 Realtive distinguished name'''
- + componentType = AttributeTypeAndValue()
- +
- +class RDNSequence(univ.SequenceOf):
- + '''ASN.1 RDN sequence class'''
- + componentType = RelativeDistinguishedName()
- +
- +
- +class Name(univ.Choice):
- + '''ASN.1 name class'''
- + componentType = namedtype.NamedTypes(
- + namedtype.NamedType('', RDNSequence()),
- + )
- +
- +
- +class Extension(univ.Sequence):
- + '''ASN.1 extension class'''
- + componentType = namedtype.NamedTypes(
- + namedtype.NamedType('extnID', univ.ObjectIdentifier()),
- + namedtype.DefaultedNamedType('critical', univ.Boolean('False')),
- + namedtype.NamedType('extnValue', univ.OctetString()),
- + )
- +
- +
- +class Extensions(univ.SequenceOf):
- + '''ASN.1 extensions class'''
- + componentType = Extension()
- + sizeSpec = univ.SequenceOf.sizeSpec
- +
- +
- +class GeneralName(univ.Choice):
- + '''ASN.1 configuration for X.509 certificate subjectAltNames fields'''
- + componentType = namedtype.NamedTypes(
- +# namedtype.NamedType('otherName', AnotherName().subtype(
- +# implicitTag=tag.Tag(tag.tagClassContext,
- +# tag.tagFormatSimple, 0))),
- + namedtype.NamedType('rfc822Name', char.IA5String().subtype(
- + implicitTag=tag.Tag(tag.tagClassContext,
- + tag.tagFormatSimple, 1))),
- + namedtype.NamedType('dNSName', char.IA5String().subtype(
- + implicitTag=tag.Tag(tag.tagClassContext,
- + tag.tagFormatSimple, 2))),
- +# namedtype.NamedType('x400Address', ORAddress().subtype(
- +# implicitTag=tag.Tag(tag.tagClassContext,
- +# tag.tagFormatSimple, 3))),
- + namedtype.NamedType('directoryName', Name().subtype(
- + implicitTag=tag.Tag(tag.tagClassContext,
- + tag.tagFormatSimple, 4))),
- +# namedtype.NamedType('ediPartyName', EDIPartyName().subtype(
- +# implicitTag=tag.Tag(tag.tagClassContext,
- +# tag.tagFormatSimple, 5))),
- + namedtype.NamedType('uniformResourceIdentifier', char.IA5String().\
- + subtype(implicitTag=tag.Tag(tag.tagClassContext,
- + tag.tagFormatSimple, 6))),
- + namedtype.NamedType('iPAddress', univ.OctetString().subtype(
- + implicitTag=tag.Tag(tag.tagClassContext,
- + tag.tagFormatSimple, 7))),
- + namedtype.NamedType('registeredID', univ.ObjectIdentifier().subtype(
- + implicitTag=tag.Tag(tag.tagClassContext,
- + tag.tagFormatSimple, 8))),
- + )
- +
- +
- +class GeneralNames(univ.SequenceOf):
- + '''Sequence of names for ASN.1 subjectAltNames settings'''
- + componentType = GeneralName()
- + sizeSpec = univ.SequenceOf.sizeSpec
- +
- +
- +class SubjectAltName(GeneralNames):
- + '''ASN.1 implementation for subjectAltNames support'''
- +# no-check-code
- diff -r d4a0055af149 -r bd1dc245a27e mercurial/sslutil.py
- --- a/mercurial/sslutil.py Fri Aug 23 16:16:22 2013 -0400
- +++ b/mercurial/sslutil.py Sun Aug 25 04:06:08 2013 -0600
- @@ -11,34 +11,53 @@
- from mercurial import util
- from mercurial.i18n import _
- try:
- - # avoid using deprecated/broken FakeSocket in python 2.6
- - import ssl
- - CERT_REQUIRED = ssl.CERT_REQUIRED
- + # Required to successfully import opensslwrap
- + from OpenSSL import SSL
- + from pyasn1 import types
- +
- + import httpclient.opensslwrap
- + CERT_REQUIRED = httpclient.opensslwrap.CERT_REQUIRED
- def ssl_wrap_socket(sock, keyfile, certfile,
- - cert_reqs=ssl.CERT_NONE, ca_certs=None):
- - sslsocket = ssl.wrap_socket(sock, keyfile, certfile,
- - cert_reqs=cert_reqs, ca_certs=ca_certs,
- - ssl_version=ssl.PROTOCOL_SSLv3)
- - # check if wrap_socket failed silently because socket had been closed
- - # - see http://bugs.python.org/issue13721
- - if not sslsocket.cipher():
- - raise util.Abort(_('ssl connection failed'))
- + cert_reqs=httpclient.opensslwrap.CERT_NONE,
- + ca_certs=None, server_hostname=None):
- + wrap_socket = httpclient.opensslwrap.wrap_socket
- + sslsocket = wrap_socket(sock, keyfile, certfile, cert_reqs=cert_reqs,
- + ca_certs=ca_certs,
- + server_hostname=server_hostname)
- return sslsocket
- except ImportError:
- - CERT_REQUIRED = 2
- + try:
- + # avoid using deprecated/broken FakeSocket in python 2.6
- + import ssl
- + CERT_REQUIRED = ssl.CERT_REQUIRED
- + def ssl_wrap_socket(sock, keyfile, certfile,
- + cert_reqs=ssl.CERT_NONE, ca_certs=None,
- + server_hostname=None):
- + sslsocket = ssl.wrap_socket(sock, keyfile, certfile,
- + cert_reqs=cert_reqs, ca_certs=ca_certs,
- + ssl_version=ssl.PROTOCOL_SSLv3)
- + # check if wrap_socket failed silently because socket had been
- + # closed
- + # - see http://bugs.python.org/issue13721
- + if not sslsocket.cipher():
- + raise util.Abort(_('ssl connection failed'))
- + return sslsocket
- + except ImportError:
- + CERT_REQUIRED = 2
- - import socket, httplib
- + import socket, httplib
- - def ssl_wrap_socket(sock, key_file, cert_file,
- - cert_reqs=CERT_REQUIRED, ca_certs=None):
- - if not util.safehasattr(socket, 'ssl'):
- - raise util.Abort(_('Python SSL support not found'))
- - if ca_certs:
- - raise util.Abort(_(
- - 'certificate checking requires Python 2.6'))
- + def ssl_wrap_socket(sock, key_file, cert_file,
- + cert_reqs=CERT_REQUIRED, ca_certs=None,
- + server_hostname=None):
- + if not util.safehasattr(socket, 'ssl'):
- + raise util.Abort(_('Python SSL support not found'))
- + if ca_certs:
- + raise util.Abort(_(
- + 'certificate checking requires Python 2.6'))
- - ssl = socket.ssl(sock, key_file, cert_file)
- - return httplib.FakeSocket(sock, ssl)
- + ssl = socket.ssl(sock, key_file, cert_file)
- + return httplib.FakeSocket(sock, ssl)
- def _verifycert(cert, hostname):
- '''Verify that cert (in socket.getpeercert() format) matches hostname.
- @@ -115,9 +134,14 @@
- self.ui.warn(_("warning: certificate for %s can't be verified "
- "(Python too old)\n") % host)
- return
- + try:
- + # work around http://bugs.python.org/issue13721
- + if not sock.cipher():
- + raise util.Abort(_('%s ssl connection error') % host)
- + except AttributeError:
- + # This is not the ssl object you are looking for
- + pass
- - if not sock.cipher(): # work around http://bugs.python.org/issue13721
- - raise util.Abort(_('%s ssl connection error') % host)
- try:
- peercert = sock.getpeercert(True)
- peercert2 = sock.getpeercert()
- diff -r d4a0055af149 -r bd1dc245a27e mercurial/url.py
- --- a/mercurial/url.py Fri Aug 23 16:16:22 2013 -0400
- +++ b/mercurial/url.py Sun Aug 25 04:06:08 2013 -0600
- @@ -181,7 +181,8 @@
- self.sock.connect((self.host, self.port))
- if _generic_proxytunnel(self):
- # we do not support client X.509 certificates
- - self.sock = sslutil.ssl_wrap_socket(self.sock, None, None)
- + self.sock = sslutil.ssl_wrap_socket(self.sock, None, None,
- + server_hostname=self.host)
- else:
- keepalive.HTTPConnection.connect(self)
- @@ -338,7 +339,7 @@
- _generic_proxytunnel(self)
- host = self.realhostport.rsplit(':', 1)[0]
- self.sock = sslutil.ssl_wrap_socket(
- - self.sock, self.key_file, self.cert_file,
- + self.sock, self.key_file, self.cert_file, server_hostname=host,
- **sslutil.sslkwargs(self.ui, host))
- sslutil.validator(self.ui, host)(self.sock)
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement