API tools faq
paste
codeaddict

Untitled

codeaddict
Mar 3rd, 2016
119
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
Python 19.35 KB | None | 0 0
raw download clone embed print report
  1. """
  2. author: _where
  3.  
  4. PVG HTTP Server.
  5.  
  6. Usage: python postviaget.py port   e.g python postviaget.py 80   to run on port 80
  7.  
  8. Implementation of popular HTTP protocols - GET, HEAD, POST, PUT, DELETE and CONNECT methods -
  9. via the GET protocol on BaseHTTPServer, thus acting as a HTTP server similar to Apache's httpd.
  10.  
  11. You are strongly advised to read the HTTP documentation (rfc2616) in order to grasp what's being done here.
  12.  
  13. Developed as a tunnel solution for organizations/isps where other protocols, except GET, are disabled.
  14.  
  15. On a lot of occassions, most isps allow free access to browsing a specific site.
  16. This is achieved through GET.
  17. POST, CONNECT and other protocols are mostly disabled.
  18. In such situations, this script makes it possible to implement these protocols through a standard HTTP
  19. proxy on the client machine, by tunneling requests based on forbidden protocols, to a server running this script,
  20. as a GET request.
  21. To achieve this, some custom headers are introduced.
  22.  
  23. The script assumes the organization/isp performs some form of response caching,
  24. thus chunked transfer-encoding for responses are not done. Responses are sent with
  25. a content-length.
  26.  
  27. Client proxy originally developed for: Header Handler
  28.  
  29. Header Definitions
  30. ======================
  31. H-Body:         This header, has as its value, data regarding the headers and body(if present) of
  32.                tunneled requests (forbidden protocols) encoded in base64. The decoded value of this
  33.                header should take the form:
  34.                    hostValue + delimiter + port+ delimiter + requestCaptured
  35.                An example of a request captured by a proxy on the client machine, to www.nairaland.com,
  36.                assuming the delimiter is '@HH@' will be:
  37.                    www.nairaland.com@HH@80@HH@request_headers_and_body
  38.  
  39. H-Chunked:      GET protocol, is not without its limitations. A standard GET request is limited in
  40.                size to ~12kb, since it's not allowed to have a body. What makes this fun, ;-)
  41.                is implementing this, with this limitation in mind. To achieve this, the proxy
  42.                appends this header to inform the script that requests will be sent in chunks.
  43.                The value is 1 to indicate this, and 0 to indicate otherwise. For a large upload
  44.                (POST request), of, for example 1mb, by the client, ~100 GET requests have to be made
  45.                from the client proxy to this script.
  46.                To implement ssl through the CONNECT protocol, requests are also sent in a similar fashion.
  47.            
  48. H-Position:     In order to hasten the upload process, tunneling is done in a fashion similar to the
  49.                UDP protocol.Requests from the client proxy could be sent concurrently, without waiting
  50.                for a response from this script. To achieve this, this header is appended to the request.
  51.                The position serves to inform the script of the order in which requests are sent from the
  52.                client proxy. The positional ordering starts from zero.
  53.                So, even if the 10th request arrives before the 1st one, the script knows how to
  54.                arrange and send accordingly. Since the script responds to each request received,
  55.                lost packets could be resent, if no response is delivered to the client proxy.
  56.  
  57. H-Secure:       To support requests sent in chunks, an id of some sort is needed in order to multithread
  58.                requests. The value of this header, is thus such id. Requests having same id, are treated
  59.                with the same socket connection to the target host. A value of zero indicates that requests
  60.                are neither sent in chunks, nor are https requests.
  61.                
  62. H-Share:        If authentication is to be performed, this header has to be appended to the original request format.
  63.                The current form of authentication is crude, as it just compares the value of this header to
  64.                preset strings. Implement appropriately, as desired.
  65.                
  66. H-Mode:         In supporting ssl/https, two modes are provided.
  67.                If the value of this header is 0, it is assumed that https requests captured by the
  68.                client proxy and tunneled to this script, are not being manipulated in any way
  69.                 - such as decryption. Thus, the script acts as a relay, forwarding requests
  70.                to the target host, and forwarding responses back. Plain sockets are thus used.
  71.                
  72.                If the value of this header is 1, it is assumed that https requests have being decrypted
  73.                by the client proxy. Thus, the script uses secure sockets to communicate with the
  74.                target host.
  75.                
  76. Thus a standard request to this server has to take the form:
  77.  
  78.            GET / HTTP/1.1
  79.            Host: ip_address_of_machine_running_this_script_on_port_80
  80.            H-Body: base64encoded_header_and_body_of_forbidden_protocols
  81.            H-share: auth_string_value
  82.            H-Chunked: 1
  83.            H-Position: 0
  84.            H-Secure: 43021
  85.            H-Mode: 0
  86.            
  87. """
  88.  
  89. __version__ = "0.0.1"
  90. # client dictionaries, if authentication is needed. Only client is madara.
  91.  
  92. g = {'madara': {}}       # dictionary used to service client requests. stores stuff
  93. h = {'madara': {}}       # dictionary used to service client requests. stores stuff
  94. num = {'madara': {}}     # dictionary used to keep track of next expected request. starts from zero.
  95. pSoc = {'madara': {}}    # dictionary used to service client requests. stores sockets.
  96. delimiter = '@HH@'       # delimiter used for body (H-body) header
  97. #actualOrder = []            
  98.  
  99. import BaseHTTPServer, socket, SocketServer, sys, base64, ssl, math, random,datetime,time
  100.  
  101. class ProxyHandler (BaseHTTPServer.BaseHTTPRequestHandler):
  102.     __base = BaseHTTPServer.BaseHTTPRequestHandler
  103.     __base_handle = __base.handle
  104.     server_version = "HH Server/" + __version__
  105.     rbufsize = 0                        # self.rfile should be unbuffered
  106.     username = ''
  107.  
  108.     def handle(self):
  109.         (ip, port) =  self.client_address
  110.         self.__base_handle()
  111.  
  112.     def outConnection(self, host, port, soc):
  113.         try: soc.connect((host, port))
  114.         except socket.error, arg:
  115.             self.send_error(404, "Connection error!")
  116.             return 0
  117.         return 1
  118.    
  119.     def do_GET(self):        
  120.         global g, h
  121.         if 'h-share' in self.headers:
  122.             self.username = str(self.headers['H-Share'])    
  123.             if self.username not in g:
  124.                 self.sendRes()
  125.                 self.connection.close()
  126.                 return
  127.        
  128.         if 'h-mode' in self.headers:
  129.             ex = int(self.headers['H-mode'])
  130.             if ex: self.mitm()
  131.             else: self.ssloverhttp()
  132.         else: self.ssloverhttp()
  133.    
  134.     def mitm(self):         # MAN-IN-THE-MIDDLE https
  135.         global g, h, pSoc, actualOrder
  136.         secure = int(self.headers['H-Secure'])
  137.         soc = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
  138.         requestBody = base64.b64decode(self.headers['H-Body'])
  139.         request = requestBody.split(delimiter)
  140.         host = request[0]
  141.         port = int(request[1])
  142.         dataSent = request[2]
  143.         if not int(self.headers['H-Chunked']):          # if requests are not chunked. normal requests less than 12kb
  144.            
  145.             if dataSent == 'REFRESH':                   #   to be sent initially by client proxy before any tunneling is done
  146.                 for i in (g, h, pSoc):
  147.                     for j in i:
  148.                         if type(j) is type(soc): j.close()
  149.                 g[self.username] = {}                   #   so as to clear dictionaries used to service previous session
  150.                 h[self.username] = {}
  151.                 pSoc[self.username] = {}
  152.                 self.wfile.write('HTTP/1.1 200 Continue\r\nContent-Length: 0\r\n\r\n')
  153.                 self.connection.close()
  154.                 return
  155.            
  156.             if secure:
  157.                 soc = ssl.wrap_socket(soc)   #    wrapper to convert plain socket to a secure socket
  158.            
  159.             try:                                        #  send request to target host and return response
  160.                 if self.outConnection(host, port, soc):
  161.                     soc.send(dataSent)
  162.                     self.read(soc)
  163.             finally:
  164.                 soc.close()
  165.                 self.connection.close()
  166.                
  167.         else:                   # for requests sent in chunks: UDP style
  168.            
  169.             """
  170.                The algo used here to keep track, is crude, but functional. Client proxy should
  171.                ensure as much as possible that order zero comes first. Other requests could come
  172.                in any particular order.
  173.                
  174.                Algorithm:
  175.                    Start from zero, expect request of position zero, if other position
  176.                    received, store value, and still expect position zero. Continue this way
  177.                    until position zero is received. When position zero is received,
  178.                    send zero, increment expected value to 1.
  179.                    If position 1 is already received, and stored, send 1, then increment expected
  180.                    to 2. If position 2 is not in store, wait till position 2 is received, then
  181.                    send, and look for three...
  182.                    
  183.                    For example, if five requests are sent by the client proxy and are
  184.                    received in the order: 5,3,0,4,1,2.
  185.                    
  186.                    It stores 5, stores 3, sends 0, stores 4, sends 1, sends 2, sends 3, sends 4
  187.                    and sends 5.
  188.            """
  189.            
  190.             position = int(self.headers['H-Position'])
  191.             if secure in g[self.username]:                     # if processing of request id is already started
  192.                 if position == num[self.username][secure]:     # if request is the next expected one, send immediately
  193.                     pSoc[self.username][secure].send(dataSent)
  194.                     t = num[self.username][secure] + 1
  195.                     while 1:                                   # loop to send stored requests in order
  196.                         if t in g[self.username][secure]: pSoc[self.username][secure].send(g[self.username][secure][t])
  197.                         else:
  198.                             num[self.username][secure] = t
  199.                             break
  200.                         t += 1
  201.                 else: g[self.username][secure][position] = dataSent;  #  store request body
  202.                 self.wfile.write('HTTP/1.1 200 Continue\r\nContent-Length: 0\r\n\r\n')  # inform proxy of receipt, so there's
  203.                 self.connection.close()                                                 # no need resending
  204.                 return
  205.            
  206.             else:                                               # if first request with that id
  207.                 g[self.username][secure] = {}                   # instantiate dictionary for that id
  208.                 split = dataSent.split(delimiter)
  209.                 self.outConnection(split[0], int(split[1]), soc)
  210.                 pSoc[self.username][secure] = soc               # store socket for that id in dictionary
  211.                 pSoc[self.username][secure].send(split[2])      # send immediately since position is zero
  212.                 num[self.username][secure] = 1
  213.                 self.wfile.write('HTTP/1.1 200 Continue\r\nContent-Length: 0\r\n\r\n')
  214.                 self.connection.close()
  215.                 return
  216.            
  217.     def ssloverhttp(self):                     # ssl relay
  218.         global g, h, pSoc
  219.         secure = int(self.headers['H-Secure'])
  220.         soc = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
  221.         requestBody = base64.b64decode(self.headers['H-Body'])
  222.         if int(self.headers['H-Chunked']):              # if requests are not chunked
  223.             split = requestBody.split(delimiter)
  224.             host = split[0]
  225.             port = int(split[1])
  226.             dataSent = split[2]
  227.            
  228.             if dataSent == 'REFRESH':      #   to be sent initially by client proxy before any tunneling is done
  229.                 g[self.username] = {}      #   so as to clear dictionaries used to service previous session
  230.                 h[self.username] = {}
  231.                 pSoc[self.username] = {}
  232.                 self.wfile.write('HTTP/1.1 200 Continue\r\nContent-Length: 0\r\n\r\n')
  233.                 self.connection.close()
  234.                 return
  235.            
  236.             if dataSent == 'PING':          #    used to inform script to send responses from id-specific socket
  237.                 if secure in pSoc[self.username]:
  238.                     #self.readResponse(pSoc[self.username][secure])                # if  caching is done by isp/organization
  239.                     self.readwriteChunkResponse(pSoc[self.username][secure])       # if chunked transfer-encoding is permitted
  240.                 else: self.constructResp(None)
  241.                 self.connection.close()
  242.                 return
  243.            
  244.             # same algo used in other mode. Transfer is still in a fashion similar to UDP
  245.             position = int(self.headers['H-Position'])
  246.             if secure in pSoc[self.username]:     #    send ssl request to target host as is, if socket already created
  247.                 if position == num[self.username][secure]:     # if request is the next expected one, send immediately
  248.                     pSoc[self.username][secure].send(dataSent)
  249.                     t = num[self.username][secure] + 1
  250.                    
  251.                     while 1:                                   # loop to send stored requests in order
  252.                         if t in g[self.username][secure]: pSoc[self.username][secure].send(g[self.username][secure][t])
  253.                         else:
  254.                             num[self.username][secure] = t
  255.                             break
  256.                         t += 1
  257.                        
  258.                 else: g[self.username][secure][position] = dataSent;  #  store request body
  259.                 self.wfile.write('HTTP/1.1 200 Continue\r\nContent-Length: 0\r\n\r\n')  # inform proxy of receipt, so there's
  260.                 self.connection.close()
  261.                 return
  262.            
  263.             else:
  264.                 """
  265.                    This is the first communication for ssl/https requests.
  266.                    Typically, this should not have a body, or if it does, it will be disregarded.
  267.                    When connection is successfully established to host, the script responds with a 200.
  268.                    The client could parse this, and modify, or send, as is to the browser.
  269.                    According to rfc2616, CONNECT requests should be handled this way.
  270.                """
  271.                 if self.outConnection(host, port, soc):
  272.                     g[self.username][secure] = {}
  273.                     pSoc[self.username][secure] = soc
  274.                     if dataSent == 'CONNECT': pass
  275.                     else: pSoc[self.username][secure].send(dataSent)
  276.                     num[self.username][secure] = 1
  277.                     self.wfile.write('HTTP/1.1 200 Continue\r\nContent-Length: 0\r\n\r\n')
  278.                     self.connection.close()
  279.                     return
  280.                    
  281.         else:                           # normal requests (POST, GET with body less that 12kb
  282.             split = requestBody.split(delimiter)
  283.             host = split[0]
  284.             port = int(split[1])
  285.             dataSent = split[2]
  286.             if self.outConnection(host, port, soc):
  287.                 soc.send(dataSent)
  288.                 self.read(soc, 20, secure)
  289.                 soc.close()
  290.                 self.connection.close()
  291.  
  292.     def read(self, soc, idlingTimeout=20, secure=0):    # normal response reading for GET/POST requests not sent in chunks
  293.         global g
  294.         count = 0
  295.         while 1:
  296.             count += 1
  297.             data = ''
  298.             try: data = soc.recv(8192)
  299.             except: pass
  300.             if data:
  301.                 self.wfile.write(data)
  302.                 count = 0
  303.             if count >= idlingTimeout: break
  304.         soc.close()
  305.         if secure in g[self.username]: g[self.username].pop(secure)
  306.    
  307.     def sendRes(self):         #   response for unauthorized clients. (Header Handler implementation)
  308.                                #   implement appropriately, to suit your needs.
  309.         body = "<html><head><title>Error</title></head><body>Client not authorized</body></html>"
  310.         self.wfile.write(self.protocol_version +
  311.                              " 200 OK\r\n")
  312.         self.wfile.write("Proxy-agent: %s\r\n" % self.version_string())
  313.         self.wfile.write('Content-Length: %s\r\n' % len(body))
  314.         self.wfile.write("Cache-control: no-cache, no-store, no-transform\r\n")
  315.         self.wfile.write("Pragma: no-cache\r\n")
  316.         self.wfile.write("Connection: close\r\n")
  317.         self.wfile.write("\r\n")
  318.         self.wfile.write(body)        
  319.  
  320.     def constructResp(self, response):
  321.         self.wfile.write('HTTP/1.1 200 OK\r\n')
  322.         self.wfile.write('Server: Header Handler\r\n')
  323.         self.wfile.write('Date: %s\r\n' % time.strftime("%c"))
  324.         self.wfile.write('Content-Type: text/html\r\n')
  325.        
  326.         if response is None:
  327.             self.wfile.write('Content-Length: 0\r\n')
  328.         else:
  329.             self.wfile.write('Content-Length: %s\r\n' % len(response))
  330.        
  331.         self.wfile.write('Cache-Control: private\r\n')
  332.         self.wfile.write('Connection: close\r\n')
  333.         self.wfile.write('\r\n')
  334.        
  335.         if response is not None: self.wfile.write(response)
  336.                
  337.     def readResponse(self, soc):                                   #         if caching is done by isp
  338.         data = ''
  339.         try: data = soc.recv(4092)
  340.         except: pass
  341.         if data: self.constructResp(data)
  342.         else: self.constructResp(None)
  343.    
  344.     def readwriteChunkResponse(self, soc, idlingTimeout=20):       #          else if chunked transfer-encoding is permitted by isp
  345.         self.wfile.write('HTTP/1.1 200 OK\r\n')
  346.         self.wfile.write('Server: Header Handler\r\n')
  347.         self.wfile.write('Date: %s\r\n' % time.strftime("%c"))
  348.         self.wfile.write('Content-Type: text/html\r\n')
  349.         self.wfile.write('Transfer-Encoding: chunked\r\n')
  350.         self.wfile.write('Cache-Control: private\r\n')
  351.         self.wfile.write('Connection: close\r\n')
  352.         self.wfile.write('\r\n')
  353.         count = 0
  354.         soc.setblocking(False)
  355.         while 1:
  356.             count += 1
  357.             data = ''
  358.             try: data = soc.recv(8192)
  359.             except: pass
  360.             if data:
  361.                 self.wfile.write('%x\r\n' % len(data))
  362.                 self.wfile.write(data)
  363.                 self.wfile.write('\r\n')
  364.                 count = 0
  365.             if count >= idlingTimeout: break
  366.         try: self.wfile.write('0\r\n\r\n')
  367.         except: pass
  368.  
  369. """
  370. Multi-threaded implementation of the PVG server.
  371. Spawns a new thread to handle each request.
  372. """
  373. class ThreadingHTTPServer (SocketServer.ThreadingMixIn,
  374.                            BaseHTTPServer.HTTPServer): pass
  375.  
  376. if __name__ == '__main__':
  377.     BaseHTTPServer.test(ProxyHandler, ThreadingHTTPServer)
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!