040721 开始定期的进行UC组中的,会课,进行Twisted 的讲解,大家的练习成果在此展示 -- dreamingk [2004-08-09 02:20:07]

1. TwistedTUT 01

  • Twisted TUT 练习1 要求
    • 完成一个服务器侦听2020端口
    • 用户telnet上去后输入一个url,服务器将这个url的信息抓回返回给用户
    • 完成一个客户端,传入一个参数,将得到的信息显示出来

1.1. *inux版本

利用 unix 平台的优点!

1.2. M$ 版本

Python!哪里也可以!

1.2.1. limodou版本

   1 #reactor version
   2 
   3 from twisted.internet import protocol, reactor, defer, utils
   4 from twisted.protocols import basic
   5 from twisted.web import client
   6 class FingerProtocol(basic.LineReceiver):
   7     def lineReceived(self, url):
   8         self.factory.getUrl(url
   9         ).addErrback(lambda _: "Internal error in server"
  10         ).addCallback(lambda m:
  11                       (self.transport.write(m+"\r\n"),
  12                        self.transport.loseConnection()))
  13 class FingerFactory(protocol.ServerFactory):
  14     protocol = FingerProtocol
  15     def getUrl(self, url):
  16         return client.getPage(url)
  17 reactor.listenTCP(2020, FingerFactory())
  18 reactor.run()

去掉了不必要的init函数,改了函数名及参数

   1 #application version
   2 
   3 from twisted.application import internet, service
   4 from twisted.internet import protocol, reactor, defer
   5 from twisted.protocols import basic
   6 from twisted.web import client
   7 
   8 class FingerProtocol(basic.LineReceiver):
   9     def lineReceived(self, url):
  10         self.factory.getUrl(url
  11         ).addErrback(lambda _: "Internal error in server"
  12         ).addCallback(lambda m:
  13                       (self.transport.write(m+"\r\n"),
  14                        self.transport.loseConnection()))
  15 class FingerFactory(protocol.ServerFactory):
  16     protocol = FingerProtocol
  17     def getUrl(self, url):
  18         return client.getPage(url)
  19 
  20 application = service.Application('finger', uid=1, gid=1)
  21 factory = FingerFactory()
  22 internet.TCPServer(2020, factory).setServiceParent(
  23 service.IServiceCollection(application))

   1 #client.py
   2 
   3 from socket import *
   4 
   5 ADDR = '127.0.0.1'
   6 PORT = 2020
   7 
   8 def sendraw(data):
   9     sendSock = socket(AF_INET, SOCK_STREAM)
  10     sendSock.connect((ADDR, PORT))
  11     sendSock.send(data+'\r\n')
  12     fp = file('data.txt', 'wb')
  13     while 1:
  14         text = sendSock.recv(1024)
  15         if text:
  16             fp.write(text)
  17         else:
  18             break
  19     sendSock.close()
  20     fp.close()
  21 
  22 if __name__ == '__main__':
  23     url = raw_input('enter url:')
  24     sendraw(url)

存在问题,如果使用telnet好象数据接收不全,但使用客户端程序还可以。

1.2.2. 0.706版本

   1 #server.py
   2 from twisted.internet import protocol, reactor, defer
   3 from twisted.protocols import basic
   4 from twisted.web import client
   5 class MyProtocol(basic.LineReceiver):
   6     def connectionMade(self):
   7         self.transport.write("Welcome,Input A URL:\r\n")
   8 
   9     def lineReceived(self, url):
  10         self.factory.getUser(url).addErrback(lambda _:
  11         "Get %s error in server \r\n"%url).addCallback(lambda m:
  12                       (self.transport.write(m+"\r\n"),
  13                        self.transport.loseConnection()))
  14 class MyFactory(protocol.ServerFactory):
  15     protocol = MyProtocol
  16     def __init__(self):
  17         pass
  18     def getUser(self, url):
  19         return client.getPage(url)
  20 reactor.listenTCP(2020, MyFactory())
  21 print "Server Started!"
  22 reactor.run()

我可不喜欢对着一个毫无提示的屏幕.

   1 #client.py
   2 import sys
   3 from socket import *
   4 
   5 def main():
   6     Sock = socket(AF_INET, SOCK_STREAM)
   7     err=Sock.connect_ex(("127.0.0.1", 2020))
   8     if(err):
   9         print "Connect Error"
  10     else:
  11         page = Sock.recv(8192)
  12         sys.stdout.write(page)
  13 
  14         url = raw_input('')
  15 
  16         Sock.send(url+'\r\n')
  17         page = Sock.recv(8192)
  18         sys.stdout.write(page)
  19 
  20     Sock.close()
  21 
  22 
  23 if __name__ == '__main__':
  24     main()

通过测试,发现在输入URL时得输全才能得到结果,如输入 "http://www.163.com" 会出错,而输入 "http://www.163.com/" 就可以

1.2.3. Zoom.Quiet版本

1.2.3.1. for reactor

   1 # -*- coding: gbk -*-
   2 # file zwget.py
   3 """ reactor version
   4 """
   5 from twisted.internet import protocol, reactor, defer, utils
   6 from twisted.protocols import basic
   7 from twisted.web import client
   8 import sys, os, string, time, re
   9 class FingerProtocol(basic.LineReceiver):
  10     def connectionMade(self):
  11         self.transport.write("Hollo! 输入URL吧::\r\n")
  12     def lineReceived(self, url):
  13         self.factory.getUrl(url
  14         ).addErrback(lambda _: "网络连接错误,或是URL错误!"
  15         ).addCallback(lambda m:
  16                       (self.echo(url,m))
  17                       )
  18     def echo(self,url,messg):
  19         label = time.strftime("%m%d%H%M%S",time.localtime(time.time()))
  20         if "bye"==url:
  21             self.transport.write("再见!\r\n")
  22             self.transport.loseConnection()
  23         else:
  24             if "URL错误" in messg:
  25                 self.transport.write(messg+"\r\n")
  26             else:
  27                 open("zgweb-%s.html"%label,"w").write(messg)
  28                 self.transport.write("抓取内容%s字节\r\n写入zgweb-%s.html\r\n 嗯嗯!再来!\r\n"%(
  29                                             len(messg),label)
  30                                      )
  31 class FingerFactory(protocol.ServerFactory):
  32     protocol = FingerProtocol
  33     def getUrl(self, url):
  34         return client.getPage(url)
  35 reactor.listenTCP(2020, FingerFactory())
  36 reactor.run()
  • 不断根据URL输入,将结果输出为.html文件,直到输入 bye ;
  • 感谢 0.706 的提示,知道了:

BaseProtocol --+    
               |    
        Protocol --+
                   |
                  LineReceiver
中,有从协议继承的  connectionFailed(self) 
 用以响应协议创建成功!

connectionFailedin Twisted API

1.2.3.2. for application

   1 # -*- coding: gbk -*-
   2 # file zawget.py
   3 """ application version
   4 """
   5 from twisted.internet import protocol, reactor, defer
   6 from twisted.protocols import basic
   7 from twisted.web import client
   8 from twisted.application import internet, service
   9 import sys, os, string, time, re
  10 class FingerProtocol(basic.LineReceiver):
  11     def connectionMade(self):
  12         self.transport.write("Hollo! 输入URL吧::\r\n")
  13     def lineReceived(self, url):
  14         self.factory.getUrl(url
  15         ).addErrback(lambda _: "网络连接错误,或是URL错误!"
  16         ).addCallback(lambda m:
  17                       (self.echo(url,m))
  18                       )
  19     def echo(self,url,messg):
  20         label = time.strftime("%m%d%H%M%S",time.localtime(time.time()))
  21         if "bye"==url:
  22             self.transport.write("再见!\r\n")
  23             self.transport.loseConnection()
  24         else:
  25             if "URL错误" in messg:
  26                 self.transport.write(messg+"\r\n")
  27             else:
  28                 open("zgweb-%s.html"%label,"w").write(messg)
  29                 self.transport.write("抓取内容%s字节\r\n写入zgweb-%s.html\r\n 嗯嗯!再来!\r\n"%(
  30                                             len(messg),label)
  31                                      )
  32 class FingerFactory(protocol.ServerFactory):
  33     protocol = FingerProtocol
  34     def getUrl(self, url):
  35         return client.getPage(url)
  36 application = service.Application('wget', uid=1, gid=1)
  37 factory = FingerFactory()
  38 internet.TCPServer(2020, factory).setServiceParent(
  39     service.IServiceCollection(application))

果然方便!其实就改最后的服务创建部分,三行而已!

1.2.3.3. for client

   1 # -*- coding: gbk -*-
   2 # file zcwget.py
   3 """ socket client version
   4 """
   5 from socket import *
   6 ADDR = '127.0.0.1'
   7 PORT = 2020
   8 def client():
   9     Sock = socket(AF_INET, SOCK_STREAM)
  10     err=Sock.connect_ex((ADDR, PORT))
  11     if(err): print "连接失败!"
  12     else:
  13         page = Sock.recv(8192)
  14         print page
  15         while 1:
  16             url = raw_input('')
  17             if "bye" in url: break
  18             else:
  19                 Sock.send(url+'\r\n')
  20                 page = Sock.recv(8192)
  21                 print page
  22     Sock.close()
  23 if __name__ == '__main__':
  24     client()
  • 不明白这客户端是什么意思呢? telnet 也算客户端吧? 嗯嗯,就是可以访问我们创建的服务的脚本是也乎!?
  • 不熟!参考前人的代码,唯一奇怪的是:
    • Sock.send(url+'\r\n') 为什么不能少 '\r\n' ?? 否则死也!

 我最早是只使用\n就出现你所说的问题。
后来看到transform返回时使用的是\r\n,
我想起来许多协议都要求行结束为\r\n,改过来就行了。
我想finger协议也是如此。使用\r\n而不是\\n应该是标准了。
======= 2004-07-24 22:57:48 您在来信中写道:=======

---Limodou