17/07/2011

讓Python 實現 P2P 功能[下]


A P2P File Sharing Application (Python version)

This page walks through an example of using the P2P framework described on this website to implement a simple P2P protocol and GUI in Python. The program developed here is a very simple file sharing application.

Protocol Design

Before beginning to write any code, we must first think about the design of the P2P protocol that we wish to implement. This is, of course, probably the hardest part. For a true, "pure" P2P network, every node (computer) will be running the exact same set of algorithms and code and there is no centralized server or otherwise special node to keep things organized. Thus, we need to decide on the set of messages that the various nodes will exchange between each other and then focus on the behavior of a single node when it receives each type of message.
In the protcol we develop here, there will be 9 types of messages exchanged between nodes. Each message type is identified by a 4-character string and may be accompanied with additional message data, summarized as follows:
  • NAME: Requests a peer to reply with its "official" peer id.
  • LIST: Requests a peer to reply with the list of peers that it knows about.
  • JOIN pid host port: Requests a peer to add the supplied host/port combination, associated with the node identified by pid, to its list of known peers.
  • QUER return-pid key ttl: Queries a peer to see if the peer has any record of a file name matching key. If so, send a RESP message back to the node identified by return-pid; if not, propagate the query to all known peers with a decreased ttl (time-to-live) value, unless ttl is already 0.
  • RESP file-name pid: Notifies a peer that the node specified by pid has a file with the given name.
  • FGET file-name: Request a peer to reply with the contents of the specified file.
  • QUIT pid: Indicate to a peer that the node identified by pid wishes to be unregistered from the P2P system.
  • REPL ...: Used to indicate an acknowledgement of the other message types above or to send back results of a successful request.
  • ERRO msg: Used to indicate an erroneous or unsuccessful request.
An essential component of any P2P application is to decide how peers actually join to form a network -- that is, how does each node become aware of other nodes in the network. A single node most often does not directly know about all the other nodes in the network but if there are enough nodes with links between them, they should all be able to interact somehow. In our example here, each peer keeps track of a fixed number of other nodes in its peer list. A new node needs to know about at least one peer that is already in the network in order to join. The new node then builds its own list of known peers by asking the initial contact for its list and repeatedly asking each peer in that list for their lists of known nodes, until the new node's list of peers is full (or there are no more peers to contact). The depth of this search is limited by a number-of-hops parameter and the algorithm ensures, of course, that the node itself is not added to its own peer list, nor are duplicate peer names.
In the file sharing application we develop here, each node also keeps track of a mapping of file names to peer identifiers where the files are actually stored. Peer identifiers are strings of the form "host:port", using IP addresses directly for routing purposes. In the next section, we discuss in more detail how a single node should respond to incoming messages of the various types summarized above, and then we are ready to look at the actual Python code, which is quite straightforward since the P2P framework takes care of most of the infrastructure associated with the application other than the initial formation of the network and the message handling. We will also look at writing a graphical user interface for the file sharing protocol.

Handling Message

In this section, we discuss in more detail how the first 7 messages listed above are handled by a peer. The last two message types, REPL and ERRO, are used for replies or responses to incoming messages of the other 7 types. Unless otherwise indicated, responses to the various types of messages are sent back using the same socket connection. In particular, it is only a QUER message that may not result in an immediate reply, so responses to QUER messages (i.e. RESP messages) will be sent back to a node at a later point over a new connection. Thus, a node will often not know immediately the result of a query because, of course, the P2P network will have to be searched to determine which node actually has the file that is being queried for.

NAME

When a peer receives a NAME message (and no additional data), it simply responds by sending back (in a REPL message) its peer id, of the form "host:port".

LIST

When a peer receives a LIST message (and no additional data), it responds by first sending a REPL message with the number of id's in its list of known peers. Then it sends that many additional REPL messages, including in the data portion of each message a string with the peer id, host address, and port number (all three separated by whitespace) associated with each known peer in its list.

JOIN

A JOIN message is accompanied with additional data specifying a sending peer id, IP address, and port number. The JOIN message is used to request a peer to insert the sending node into its list of known peers. If the peer's list is already full, or the specified id is already in the list (or incorrect arguments are specified with the JOIN message), than an ERRO message is sent back in response. Otherwise, the remote node's information is added to the list and a REPL acknowledgement is sent.

QUER

The QUER message is the most complex to handle in this application. Upon receipt of this type of message, if the correct number of additional data is provided (return-peer-id, key, and ttl value), then the peer responds simply with an acknowledgement REPL message; otherwise an ERRO message is returned. Before exiting the handler routine for this message type (and closing the socket connection), however, a separate thread is started that actually processes the query information.
The 'processQuery' thread operates as follows: If the key is found to be a substring of any of the file names in the peer's list of files, then the peer sends a RESP message to the peer that originally initiated the query (given by the return-peer-id argument of the QUER message; in our protocol here, the IP address and port number of the peer can be determined by parsing the return-peer-id string, which will be of the format "host:port"). If the key is not found to be a substring of any of the file names in the peer's list, and the ttl value is greater than 0 (zero), then a QUER message is sent to every node in the list of known peers, with the samereturn-peer-id and key arguments, and the ttl value decremented by 1.

RESP

A peer will receive a RESP message in response to a QUER message that it had previously sent out to others in the network. (The design of our protocol here does not preclude the possibility of peers receiving RESP messages for files they had not queried for -- this may be viewed as either a feature or a bug.) The RESP message includes a file name and the peer id of the node owning a copy of the file. Upon receiving a RESP message, a peer does not need to send any reply; if the file name is not already present in its file list, an entry will be added to the list indicating that the file may be found at the specified node.

FGET

The data of an FGET message consists of a file name. If the peer receiving the FGET message does not own a copy of the specified file or if it is not readable for some reason, an ERRO reply is sent. Otherwise, the entire file is sent as the payload of a REPL message.

QUIT

A QUIT message, including a peer id in the data of the message, tells a peer to remove the specified node's information from its list of known peers as that node is preparing to leave the network. Note that, as is the case in most P2P protocols, node may also leave the network unexpectedly, so some sort of "stabilization" routine should be run by every peer node to periodically update its list of peers.

File Sharing Application Code

Having discussed the file sharing protocol, you may now download and view the actual Python code. This file, btfiler.py, extends the Peer class and contains methods for handling each of the messages as described above, as well as a method, buildpeers for filling in the initial list of known peers as the node attempts to join the P2P network. The constructor registers the handler methods and performs other simple initialization tasks.
You may also at this point look over the GUI front end for the file sharing application. The code is in this file, filergui.py. The next section briefly discusses the contents of this module.
^ TOP

The GUI Application

The FilerGUI class extends the basic Frame class and adds components and event handlers to the window. The window looks like the following:
The constructor of the FilerGUI class constructs and adds the GUI components to the frame window. It then initializes a FilerPeer object listening on the server port specified as a parameter to the constructor. The constructor also takes as a parameter the peer id of the initial peer to contact in order to attempt to join the P2P network. It uses the buildpeers method of the peer class to populate the list of known peers as described in the earlier sections. It then starts the mainloop of the peer object in a separate thread.
The FilerGUI constructor also schedules two tasks to run periodically, every 3 seconds. One is a "stabilizer" routine that checks the list of known peers to ensure that they are all still alive. For this purpose, we just use the checklivepeers method that is provided by the framework. The other task that runs periodically is a "refresh" routine that copies the entries in the internal list of known peers and the list of known file names to the corresponding components of the GUI window: the "Available Files" and "Peer List" list boxes.
The FilerGUI class contains a createWidgets method that lays out all of the components in the window frame above and registers appropriate event handlers with each button. The handlers for each button perform the following tasks, each programmed in its own method entitled "onAdd", "onSearch", etc.:
  • Refresh: Synchronizes the contents of the GUI list components with the internal peer and file lists of the node. This operation happens automatically every few seconds.
  • Remove: Removes the selected node id from the peer list.
  • Rebuild: Takes a "host:port" string from the accompanying text box and uses it to invoke the buildpeers method of the Filer Peer class, attempting to populate the list of known peers.
  • Search: Sends a QUER message with the contents of the accompanying text box to all peers in the list. As discussed above, responses to the QUER message will not arrive immediately. RESP messages received in the background will be handled by the mainloop of the peer as described above in the "Handling Messages" section. If a new entry is in fact added to the file list then it will be reflected the next time the "refresh" routine is executed.
  • Add: Registers the specified file name as being locally stored on the peer. Shared files are stored in the same directory as the application's executable file.
  • Fetch: Downloads the selected file to the local computer by sending an FGET message to the peer that is listed as hosting the file.
Other than the component layout code and the event handlers above, the FilerGUI source file has a main method that launches the application, taking (1) a port number to listen on for incoming connections, (2) the maximum number of peers to store in the peer list, and (3) the peer id of an initial contact in the P2P network that the node wishes to join.

讓Python 實現 P2P 功能[上]


The P2P Framework Implementation (Python version)


This page walks through a Python implementation of the P2P framework library itself. I will assume that you are familiar with Python. I will also assume you are familiar with the general concepts of socket programming, though you may not necessarily have used the Python networking or threading libraries. The complete source code may be downloaded here: btpeer.py.

Initializing a peer

Let us first examine how a peer node is initialized. As discussed above, the overall operation of a node is managed by the Peer class. The constructor of the class stores the canonical identifier (name) of the peer, the port on which it listens for connections, and the maximum size of the list of known peers that the node will maintain (this can be set to 0 to allow an unlimited number of peers in the list). The constructor also initializes several fields whose use is described in the following subsections - shutdownhandlers, and router. The following definition illustrates the Python code for accomplishing these tasks:
    def __init__( self, maxpeers, serverport, myid=None, serverhost = None ):
	self.debug = 0

	self.maxpeers = int(maxpeers)
	self.serverport = int(serverport)

        # If not supplied, the host name/IP address will be determined
	# by attempting to connect to an Internet host like Google.
	if serverhost: self.serverhost = serverhost
	else: self.__initserverhost()

        # If not supplied, the peer id will be composed of the host address
        # and port number
	if myid: self.myid = myid
	else: self.myid = '%s:%d' % (self.serverhost, self.serverport)

        # list (dictionary/hash table) of known peers
	self.peers = {}  

        # used to stop the main loop
	self.shutdown = False  

	self.handlers = {}
	self.router = None
    # end constructor
Every peer node performs operations common to traditional network client and server applications. First, let us walk through the server-related operations, as described in the previous section.

The main loop

The main loop of a peer begins by setting up a socket that listens for incoming connections from other peers. To do this, we must (1) create the socket object, (2) set its options, (3) bind it to a port on which to listen for connections, and (4) actually begin listening for connections. Here is a Python method that accomplishes this, returning the initialized socket:

    def makeserversocket( self, port, backlog=5 ):
	s = socket.socket( socket.AF_INET, socket.SOCK_STREAM )
	s.setsockopt( socket.SOL_SOCKET, socket.SO_REUSEADDR, 1 )
	s.bind( ( '', port ) )
	s.listen( backlog )
	return s

The first statement creates a socket that will communicate using the IPv4 (AF_INET) protocol with TCP (SOCK_STREAM). By setting the SO_REUSEADDR option, the port number of the socket will be immediately reusable after the socket is closed (otherwise the operating system may prevent the port from being reused after the server exits, until a certain amount of time has passed). Then the socket is bound to the specified port and is set up to receive connections. The backlog parameter indicates how many incoming connections should be queued up. A value of 5 is conventionally used, although with a multithreaded server (as we are building) the parameter's value is not very significant.
Having created a server socket, the main loop of a peer loops continously, accepting connections. When an incoming connection is accepted, the server will have a new socket object used to send and receive data on the connection. The main loop then calls a separate method to handle communication with this connection in a new thread. A simple for of the main loop would thus look like this:

  s = self.makeserversocket( self.serverport )
  
  while 1:
     clientsock, clientaddr = s.accept()
  
     t = threading.Thread( target = self.__handlepeer, args = [ clientsock ] )
     t.start()

In reality, we also need to handle any errors that may occur in the process of accepting a connection, and we need to provide a mechanism so that the loop may somehow be nicely terminated (for example, when the user indicates that the program should exit). To do this, we set up the server socket to time out every 2 seconds (an arbitrary choice) and make the loop termination condition dependent on a boolean (instance) variable, shutdown. Also, we set up an exception handler to allow the main loop to be stopped by the user pressing the "Ctrl"+"Break" (or "Ctrl"+"c") keys. Here, then, is the complete mainloop method.
    def mainloop( self ):
	s = self.makeserversocket( self.serverport )
	s.settimeout(2)
	self.__debug( 'Server started: %s (%s:%d)'
		      % ( self.myid, self.serverhost, self.serverport ) )

	while not self.shutdown:
	    try:
		self.__debug( 'Listening for connections...' )
		clientsock, clientaddr = s.accept()
		clientsock.settimeout(None)

		t = threading.Thread( target = self.__handlepeer, args = [ clientsock ] )
		t.start()
	    except KeyboardInterrupt:
		self.shutdown = True
		continue
	    except:
		if self.debug:
		    traceback.print_exc()
		    continue
	# end while loop

	self.__debug( 'Main loop exiting' )
	s.close()
    # end mainloop method
The debug method will output various messages to an appropriate location - for example, the screen or a log file. The myid field is the identifier of the peer, and theserverhost field stores the peer's IP address (or host name). These values are initialized by the constructor for the peer object.

Handling a peer connection

The handlepeer method takes a newly formed peer connection, reads in a request from it, and dispatches the request to an appropriate handler (function or method) for processing. The particular handlers and types of requests will be specified by the programmer using this framework to implement a particular protocol. Thehandlepeer method simply looks for the appropriate handler for a message, if there is one registered with the peer object, and calls it.
handlepeer begins by encapsulating the socket connection in a PeerConnection object, to allow easy sending/receiving and encoding/decoding of P2P messages in the system.

	host, port = clientsock.getpeername()
	peerconn = BTPeerConnection( None, host, port, clientsock, debug=False )

Then, handlepeer attempts to receive some data from the connection and determine what to do with it:

	    msgtype, msgdata = peerconn.recvdata()
	    if msgtype: msgtype = msgtype.upper()
	    if msgtype not in self.handlers:
		self.__debug( 'Not handled: %s: %s' % (msgtype, msgdata) )
	    else:
		self.__debug( 'Handling peer msg: %s: %s' % (msgtype, msgdata) )
		self.handlers[ msgtype ]( peerconn, msgdata )

The handlers field is a dictionary (hash table), mapping message types (4-character strings) to function pointers. If the message type has a corresponding entry in the dictionary, the function pointer is extracted and invoked, passing it the PeerConnection object and the actual data of the message. Upon completion of Before returning, handlepeer closes the connection. Here, then, is the complete definition of the method:
    def __handlepeer( self, clientsock ):
	self.__debug( 'Connected ' + str(clientsock.getpeername()) )

	host, port = clientsock.getpeername()
	peerconn = BTPeerConnection( None, host, port, clientsock, debug=False )
	
	try:
	    msgtype, msgdata = peerconn.recvdata()
	    if msgtype: msgtype = msgtype.upper()
	    if msgtype not in self.handlers:
		self.__debug( 'Not handled: %s: %s' % (msgtype, msgdata) )
	    else:
		self.__debug( 'Handling peer msg: %s: %s' % (msgtype, msgdata) )
		self.handlers[ msgtype ]( peerconn, msgdata )
	except KeyboardInterrupt:
	    raise
	except:
	    if self.debug:
		traceback.print_exc()
	
	self.__debug( 'Disconnecting ' + str(clientsock.getpeername()) )
	peerconn.close()

    # end handlepeer method
^ TOP

Routing and sending messages

Using the addrouter method, the programmer may register a routing function (or method) with the Peer class to help decide how messages should be forwarded, given a destination peer id. The routing function should expect as a paremeter the name of a peer (which may not necessarily be present in the list of known peers of the node), and decide which of the known peer the message should be routed to next in order to (hopefully) reach the desired peer. The routing function should return a tuple of three values: (next-peer-id, host, port) where the host and port are the IP address of the peer identified by next-peer-id. If the message cannot be routed, the next-peer-id should be None.
The sendtopeer method takes a message type and data, along with a destination peer id, and uses the routing function to decide where to send the message next. If no routing function has been registered by the programmer, or if the routing function fails for some reason, the method fails. If the routing function successfully returns the next host/port combination to which the message should be sent, sendtopeer calls the connectandsend method to actually make the connection to the peer, package up, and send the data. If the programmer desires to receive a response from the next peer before the communication socket is closed, it will be returned by these methods.
    def sendtopeer( self, peerid, msgtype, msgdata, waitreply=True ):
	if self.router:
	    nextpid, host, port = self.router( peerid )
	if not self.router or not nextpid:
	    self.__debug( 'Unable to route %s to %s' % (msgtype, peerid) )
	    return None
	return self.connectandsend( host, port, msgtype, msgdata, pid=nextpid,
				    waitreply=waitreply )

    # end sendtopeer method
The connectandsend method connects and sends a message to a peer at the specified IP address and port. The host's reply, if desired by the caller, will be returned as a list of pairs, where each pair contains (1) the type and (2) the actual data of the message.
    def connectandsend( self, host, port, msgtype, msgdata, pid=None, waitreply=True ):
	msgreply = []   # list of replies
	try:
	    peerconn = BTPeerConnection( pid, host, port, debug=self.debug )
	    peerconn.senddata( msgtype, msgdata )
	    self.__debug( 'Sent %s: %s' % (pid, msgtype) )
	    
	    if waitreply:
		onereply = peerconn.recvdata()
		while (onereply != (None,None)):
		    msgreply.append( onereply )
		    self.__debug( 'Got reply %s: %s' % ( pid, str(msgreply) ) )
		    onereply = peerconn.recvdata()
	    peerconn.close()
	except KeyboardInterrupt:
	    raise
	except:
	    if self.debug:
		traceback.print_exc()
	
	return msgreply

    # end connectsend method
^ TOP

Additional methods

The Peer class provides methods supporting other fundamental functionalities of a peer node. Briefly, these include:
  • startstabilizer(stabilizer, delay): Runs the provided 'stabilizer' function in a separate thread, activating it repeatedly after every delay seconds, until theshutdown flag of the Peer object is set.
  • addhandler(msgtype, handler): Registers a handler function for the given message type with the Peer object. Only one handler function may be provided per message type. Message types do not have to be defined in advance of calling this method.
  • addrouter(router): Registers a routing function with this peer. Read the section on routing above for details.
  • addpeer(peerid, host, port): Adds a peer name and IP address/port mapping to the known list of peers.
  • getpeer(peerid): Returns the (host,port) pair for the given peer name.
  • removepeer(peerid): Removes the entry corresponding to the supplied peerid from the list of known pairs.
  • addpeerat(loc, peerid, host, port)getpeerat(loc)removepeerat(loc): Analogous to the prior three methods, except that they access direct (numeric) positions within the list of peers (as opposed to using the peerid as a hash key). These functions should not be used concurrently with the prior three.
  • getpeerids(): Return a list of all known peer ids.
  • numberofpeers():
  • maxpeersreached():
  • checklivepeers(): Attempt to connect and send a 'PING' message to all currently known peers to ensure that they are still alive. For any connections that fail, the corresponding entry is removed from the list. This method can be used as a simple 'stabilizer' function for a P2P protocol.

The PeerConnection class

The PeerConnection class encapsulates a socket connection to a peer node. An object of this class may be initialized with an already-connected socket, or with an IP address/port combination that will be used to open a new socket connection.

P2P Software_Opera[BT Webrowser]Torrent Finder



Torrent Finder是一個針對Chrome的種子搜尋外掛,他會搜尋超過150個的種子網站,如果不想在安裝此程式,也可以直接到下面網站尋找你要的資料.
 http://torrent-finder.info



Torrent Finder is a torrent search extension for Chrome which enables users to search over 150 top torrent sites and trackers from their Chrome browser the same way they used to search using http://torrent-finder.info/ form. It allows users to search search all sites on one page with an auto complete form and context menu option.

Main features :
► Search top torrent search engines and trackers from one page.
► Arrange sites on the main form on http://torrent-finder.info/.
► Holding Ctrl key with the return key searches Google instead of Torrent Finder.
► Auto-Complete Search suggestions from Google database.
► Contextmenu search option for selected text.
► Search Google from same form.

Some Sites we search:
IsoHunt ,Demonoid ,BitTorrentMonster ,The Pirate Bay ,MiniNova ,2torrents ,BiteNova ,Bush Torrent ,Snafit ,Meganova ,New Torrents ,Monova ,Torrent Reactor ,Torrent Box ,Torrent Portal ,Fulldls .com ,Torrent Valley ,bt junkie ,Riratic .org ,Pro Torrent ,BT-Chat ,Torrent Locomotive ,LiteBay ,Wtorrent ,SloTorrent .net ,TORRENTAT ,Zoozle ,Torrent Radar ,Dino Torrent ,Extreme Nova ,Nova Lite ,ExtraTorrent .com ,MyBitTorrent ,Piratenova .org ,The Pirate Box ,Saugstube ,Bitreactor .to ,Bitbull .to ,Torrent Galaxy ,Bit-Torrents ,Tokyo Toshokan ,Torrentz .ws ,Torrentz ,BitTorrent ,BtBot ,Torrentz ,flexbeta ,Orb ,Only Torrents ,Tracktrap ,Yotoshi v2 ,Torrents Hub ,Astatorrents ,Fkee BT ,Google ,Need Torrents ,Throughput .de ,Torrent Bay ,RockBox ,Torrent Matrix ,EmuParadise BT Tracker ,Zero Tracker ,h33t ,Conspiracy Central ,Indy Torrents ,Hunt Park Insider Futbol ,BNBT Tracker ,Games Torrents ,DescargasWEB ,DivXTotal ,TNT Village Scambio ,Smart Torrent ,Elite Team ,Tokyo Nights ,Podtropolis ,ebookshare .net ,Game Updates ,X TV Interactive ,Linux Tracker ,ZEOEZ ,LinuxTracker .de ,File List ,BitME .ORG ,TorrentDamage ,Bitsoup .org ,I love torrents ,Libitina .net ,UNDERGROUND ,Total Torrents ,DEVILOID .NET ,Badbits .org ,Cyber Storm ,Softorrent ,SWETorrents ,Peer Portal ,Super-Torrents ,Torrent Ice ,GeoTorrents ,Torrent Hackers ,Tri-Tavern ,EzTorrents ,Zomb Torrents ,Deep Space ,Torrent Leech ,IPTorrents ,Aradi Tracker ,Libble .com ,DSB Tracker ,Revolutiontt ,Shadow World ,DIDIDave ,Bitnation .com ,My Spleen ,Torrent Team .au ,High Speed Torrents ,Pimp Torrent ,katastrofa ,Extreme Share ,Torrent Bits ,TV Torrents ,ABetterPlace2B ,Xbox Sky ,FileMP3 .org ,MusicVids ,myMusixxTorrent ,Lankan Torrents ,Torrents .bol .bg ,Warez FR ,Snow Tigers ,Free Torrent ,Shared .cl ,ArebaBG ,Zentrale Multi Tracker ,Torrent Turkey ,Greek Tracker ,Witch Tracker ,Colombo ,BTParadise ,OUTLAW ,Doyana Kadar ,Arab Files ,Arab Films ,ToDo Torrents ,PirateBits ,XS Portal ,Torrents Mexico ,File .lv ,Danger .lv ,Torrent Land ,FatalBits ,AniRenA ,Dreamorabt .com ,Da Torrents ,Box Torrents ,DeadFrog Anime ,Frozen-Layer Network ,Download Anime ,SaiyaMan .Net ,Animersion ,Point Blank Tracker ,Arab Animes ,AnimeX ,Anime Yume ,Anime Suki ,and more...