Writing Network Programs with Python Nile University, CIT-614 Network Programming and Distributed Objects, By  Dr Sameh Al Ansary , modified by Ahmed Kishk
Agenda Talk  - Are We really Engineers? Python (Programming language).
Engineering? Software: is it really engineering? If it is not engineering, then what is it? Art?  Craft? Science?
Engineering? Art:   the production or expression of what is beautiful, appealing, or of more than ordinary significance. Craft:   A trade or occupation requiring special skill, especially manual skill. Science:   The systematic study of man and his environment based on the deductions and inferences that can be made, and the general laws that can be formulated, from reproducible observations and measurements of events and parameters within the universe.
Engineering? Engineering:   The art of making practical application of the knowledge of pure science. Simply it is a profession.
Engineering? The profession in which a knowledge of the mathematical and natural sciences gained by study, experience, and practice is applied with judgment to develop ways to utilize, economically, the materials and forces of nature for the benefit of mankind. Accreditation board for Engineering & Technology, 1996
Talk SEI (Software Engineering Institute) Software Product Line (Development, Testing and Production Environment) Skill y0ur self with Software Engineering professionalism (Testing -Architecture Design and Documentation) Doing Unit Testing, Regression Testing, Integration Testing, System Testing and Smoke Testing. Web Application Engineering You have to have critical thinking. Read Code Complete
To Find More…
Let’s Start our Lecture
Python Python is a programming language that lets you work more quickly and integrate your systems more effectively.  Download the compiler from  http://www.python.org/ PyDev is a plugin for python in Eclipse IDE. You can find installation steps at below link  http://pydev.org/manual_101_install.html
Python no explicit variable declarations Variables spring into existence by being assigned a value automatically destroyed when they go out of scope.
Python Run: Python >>>print “Hello MUFIX” Weak typed language: >>> x = 5 >>>x 5 >>>type(x) <type 'int'> >>>x = “MUFIX” >>>x MUFIX >>>type(x) <type 'str'>
Python .py file is called a module. To declare a function use “def:” Args separated by “,” No return types use the “return” keyword to return values
Python Indenting Code no explicit  begin or end,  and no curly  braces.
Python Lists Like java ArrayList but more powerful. >>> list = [“a”, “b”, “c”, “d”] >>>list [“a”, “b”, “c”, “d”] >>> List[0] “ a” >>> List[3] “ d”
Python Lists Negative List indices >>>list [“a”, “b”, “c”, “d”] >>>list[-1] “ d” >>>list[-3] “ b”
Python Lists Slicing a list >>>list [“a”, “b”, “c”, “d”] >>>list[1:3] [“b”, “c”] >>>list[1, -1] [“b”, “c”] >>>list[:3] [“a”, “b”, “c”] >>>list[:] [“a”, “b”, “c”, “d”]
Python Lists Operations: like java ArrayList methods List.append() List.insert() List.extend() List.index() List.pop()
Python Dictionaries Like Java HashMap but a simple one. A key / value pairs >>> Info = {“name”:”Ahmed”, “job”:”Software Engineer”} >>>Info[“name”] “ Ahmed” >>>Info[“College”] = “MUFIC” >>>Info {“name”:”Ahmed”, “job”:”Software Engineer”, “College”:”MUFIC”}
Python Dictionaries •  len(d) •  d[key] •  d[key] = value •  del d[key] •  key in d •  key not in d •  clear() •  copy() •  items() •  keys() •  values()
Tuples >>> T = (“a”, “b”, “c”, “d”) >>> T (“a”, “b”, “c”, “d”) Read-Only Lists Have no methods Like java Enums
Python Strings >>> names = “C++, Java, Python, Ruby, C#” >>>list = names.split(“,”) ['C++', ' Java', ' Python', ' Ruby', ' C#'] >>>names = ” ”.join(list) >>>names ” C++ Java Python Ruby C#”
Python and Files
To Find More…
Network Programming Browsers, chat clients, Downloads…..etc
Internet Socket and TCP/IP an  Internet socket  or  network socket  is an endpoint of a bidirectional inter-process communication flow across an Internet Protocol-based computer network, such as the Internet. Internet sockets  is also used as a name for an (API) for the TCP/IP protocol stack provided by the operating system.  Internet sockets delivers incoming data packets to the appropriate application process or thread, based on a combination of local and remote IP addresses and port numbers.
Socket types Datagram sockets , also known as connectionless sockets, which use User Datagram Protocol (UDP) Stream sockets , also known as connection-oriented sockets, which use Transmission Control Protocol (TCP) or Stream Control Transmission Protocol (SCTP). Raw sockets  (or  Raw IP sockets ), typically available in routers and other network equipment. Here the transport layer is bypassed, and the packet headers are not stripped off, but are accessible to the application. Application examples are Internet Control Message Protocol (ICMP, best known for the Ping sub operation), Internet Group Management Protocol (IGMP), and Open Shortest Path First (OSPF).
Connection-Oriented Services [blocked] [blocked] [blocked] Server Client When interaction is over, server loops to accept a new connection socket() bind() listen() accept() read() write() socket() connect() write() read()
First Network Program – Echo Server
Client
File Like Objects
HTTP RFC 2616
HTTP Request and Response
HTTP Request •  The format of the initial line for requests and responses is different •  The  Request line contains: –  Method: •  GET: The most common •  Others: POST, HEAD, PUT,… –  Request-URI •  Exampe:  bla/directory17/hello.html –  HTTP-Version •  Example The string: “HTTP/1.1”
HTTP Request
Request Methods
Response Initial line •  The Response initial (Called  Status line): –  HTTP-Version: –  Status-Code: three digits, e.g 200, 404 –  Reason-Phrase: OK, Not found •  Examples: –  200 OK –  404 Not Found •  (The requested resource doesn't exist.)
Response status codes
Minimal HTTP Client
HTTP Connection Management Serial connection:
Parallel connection Each transaction opens/closes a new connection, costing time and bandwidth.  Each new connection has reduced performance because of TCP slow start. There is a practical limit on the number of open parallel connections.
Persistent Connections TCP connections that are kept open after transactions complete are called  persistent  connections
Pipelined Connections Multiple requests can be enqueued before the responses arrive
To Find More… http://www.w3.org/Protocols/rfc2616/rfc2616.html
YouTube Downloader http://www.youtube.com/watch?v=QvsQ9hYKq7c
Proxy Demo http://www.youtube.com/watch?v=QvsQ9hYKq7c
Non Blocking Sockets [blocked] [blocked] [blocked] Server Client When interaction is over, server loops to accept a new connection socket() bind() listen() accept() read() write() socket() connect() write() read()
Non Blocking Sockets socket.setblocking(0)   Simply In Python, you use to make it non-blocking. send ,  recv ,  connect  and  accept  can return without having done anything. You can check return code and error codes and generally drive yourself crazy. Your app will grow large, buggy.
let’s skip the brain-dead solutions and do it right.   Use select.
Multiplexing Servers with select Let’s pick one of the previous solustion (Threads)  threads and processes don't really run in parallel. your operating system divides the computer's processing power among all active tasks. This process of switching between tasks is sometimes called   multiplexing . And this is the idea behind SELECT
Multiplexing Servers with select (CONTIUNED) Servers can apply this technique without Threading nor forking. By multiplexing client connections and the main dispatcher with the select system call, a single event loop can process clients and accept new ones in parallel . the magic behind this server structure is the operating system  select call select is asked to monitor a list of input sources, output sources, and exceptional condition sources and tells us which sources are  ready for processing
select-based echo server
select-based echo server (continued)
Questions

Mufix Network Programming Lecture

  • 1.
    Writing Network Programswith Python Nile University, CIT-614 Network Programming and Distributed Objects, By Dr Sameh Al Ansary , modified by Ahmed Kishk
  • 2.
    Agenda Talk - Are We really Engineers? Python (Programming language).
  • 3.
    Engineering? Software: isit really engineering? If it is not engineering, then what is it? Art? Craft? Science?
  • 4.
    Engineering? Art: the production or expression of what is beautiful, appealing, or of more than ordinary significance. Craft: A trade or occupation requiring special skill, especially manual skill. Science: The systematic study of man and his environment based on the deductions and inferences that can be made, and the general laws that can be formulated, from reproducible observations and measurements of events and parameters within the universe.
  • 5.
    Engineering? Engineering: The art of making practical application of the knowledge of pure science. Simply it is a profession.
  • 6.
    Engineering? The professionin which a knowledge of the mathematical and natural sciences gained by study, experience, and practice is applied with judgment to develop ways to utilize, economically, the materials and forces of nature for the benefit of mankind. Accreditation board for Engineering & Technology, 1996
  • 7.
    Talk SEI (SoftwareEngineering Institute) Software Product Line (Development, Testing and Production Environment) Skill y0ur self with Software Engineering professionalism (Testing -Architecture Design and Documentation) Doing Unit Testing, Regression Testing, Integration Testing, System Testing and Smoke Testing. Web Application Engineering You have to have critical thinking. Read Code Complete
  • 8.
  • 9.
  • 10.
    Python Python isa programming language that lets you work more quickly and integrate your systems more effectively. Download the compiler from http://www.python.org/ PyDev is a plugin for python in Eclipse IDE. You can find installation steps at below link http://pydev.org/manual_101_install.html
  • 11.
    Python no explicitvariable declarations Variables spring into existence by being assigned a value automatically destroyed when they go out of scope.
  • 12.
    Python Run: Python>>>print “Hello MUFIX” Weak typed language: >>> x = 5 >>>x 5 >>>type(x) <type 'int'> >>>x = “MUFIX” >>>x MUFIX >>>type(x) <type 'str'>
  • 13.
    Python .py fileis called a module. To declare a function use “def:” Args separated by “,” No return types use the “return” keyword to return values
  • 14.
    Python Indenting Codeno explicit begin or end, and no curly braces.
  • 15.
    Python Lists Likejava ArrayList but more powerful. >>> list = [“a”, “b”, “c”, “d”] >>>list [“a”, “b”, “c”, “d”] >>> List[0] “ a” >>> List[3] “ d”
  • 16.
    Python Lists NegativeList indices >>>list [“a”, “b”, “c”, “d”] >>>list[-1] “ d” >>>list[-3] “ b”
  • 17.
    Python Lists Slicinga list >>>list [“a”, “b”, “c”, “d”] >>>list[1:3] [“b”, “c”] >>>list[1, -1] [“b”, “c”] >>>list[:3] [“a”, “b”, “c”] >>>list[:] [“a”, “b”, “c”, “d”]
  • 18.
    Python Lists Operations:like java ArrayList methods List.append() List.insert() List.extend() List.index() List.pop()
  • 19.
    Python Dictionaries LikeJava HashMap but a simple one. A key / value pairs >>> Info = {“name”:”Ahmed”, “job”:”Software Engineer”} >>>Info[“name”] “ Ahmed” >>>Info[“College”] = “MUFIC” >>>Info {“name”:”Ahmed”, “job”:”Software Engineer”, “College”:”MUFIC”}
  • 20.
    Python Dictionaries • len(d) • d[key] • d[key] = value • del d[key] • key in d • key not in d • clear() • copy() • items() • keys() • values()
  • 21.
    Tuples >>> T= (“a”, “b”, “c”, “d”) >>> T (“a”, “b”, “c”, “d”) Read-Only Lists Have no methods Like java Enums
  • 22.
    Python Strings >>>names = “C++, Java, Python, Ruby, C#” >>>list = names.split(“,”) ['C++', ' Java', ' Python', ' Ruby', ' C#'] >>>names = ” ”.join(list) >>>names ” C++ Java Python Ruby C#”
  • 23.
  • 24.
  • 25.
    Network Programming Browsers,chat clients, Downloads…..etc
  • 26.
    Internet Socket andTCP/IP an  Internet socket  or  network socket  is an endpoint of a bidirectional inter-process communication flow across an Internet Protocol-based computer network, such as the Internet. Internet sockets  is also used as a name for an (API) for the TCP/IP protocol stack provided by the operating system. Internet sockets delivers incoming data packets to the appropriate application process or thread, based on a combination of local and remote IP addresses and port numbers.
  • 27.
    Socket types Datagramsockets , also known as connectionless sockets, which use User Datagram Protocol (UDP) Stream sockets , also known as connection-oriented sockets, which use Transmission Control Protocol (TCP) or Stream Control Transmission Protocol (SCTP). Raw sockets  (or  Raw IP sockets ), typically available in routers and other network equipment. Here the transport layer is bypassed, and the packet headers are not stripped off, but are accessible to the application. Application examples are Internet Control Message Protocol (ICMP, best known for the Ping sub operation), Internet Group Management Protocol (IGMP), and Open Shortest Path First (OSPF).
  • 28.
    Connection-Oriented Services [blocked][blocked] [blocked] Server Client When interaction is over, server loops to accept a new connection socket() bind() listen() accept() read() write() socket() connect() write() read()
  • 29.
    First Network Program– Echo Server
  • 30.
  • 31.
  • 32.
  • 33.
  • 34.
    HTTP Request • The format of the initial line for requests and responses is different • The Request line contains: – Method: • GET: The most common • Others: POST, HEAD, PUT,… – Request-URI • Exampe: bla/directory17/hello.html – HTTP-Version • Example The string: “HTTP/1.1”
  • 35.
  • 36.
  • 37.
    Response Initial line• The Response initial (Called Status line): – HTTP-Version: – Status-Code: three digits, e.g 200, 404 – Reason-Phrase: OK, Not found • Examples: – 200 OK – 404 Not Found • (The requested resource doesn't exist.)
  • 38.
  • 39.
  • 40.
    HTTP Connection ManagementSerial connection:
  • 41.
    Parallel connection Eachtransaction opens/closes a new connection, costing time and bandwidth. Each new connection has reduced performance because of TCP slow start. There is a practical limit on the number of open parallel connections.
  • 42.
    Persistent Connections TCPconnections that are kept open after transactions complete are called persistent connections
  • 43.
    Pipelined Connections Multiplerequests can be enqueued before the responses arrive
  • 44.
    To Find More…http://www.w3.org/Protocols/rfc2616/rfc2616.html
  • 45.
  • 46.
  • 47.
    Non Blocking Sockets[blocked] [blocked] [blocked] Server Client When interaction is over, server loops to accept a new connection socket() bind() listen() accept() read() write() socket() connect() write() read()
  • 48.
    Non Blocking Socketssocket.setblocking(0) Simply In Python, you use to make it non-blocking. send ,  recv ,  connect  and  accept  can return without having done anything. You can check return code and error codes and generally drive yourself crazy. Your app will grow large, buggy.
  • 49.
    let’s skip thebrain-dead solutions and do it right. Use select.
  • 50.
    Multiplexing Servers withselect Let’s pick one of the previous solustion (Threads) threads and processes don't really run in parallel. your operating system divides the computer's processing power among all active tasks. This process of switching between tasks is sometimes called multiplexing . And this is the idea behind SELECT
  • 51.
    Multiplexing Servers withselect (CONTIUNED) Servers can apply this technique without Threading nor forking. By multiplexing client connections and the main dispatcher with the select system call, a single event loop can process clients and accept new ones in parallel . the magic behind this server structure is the operating system select call select is asked to monitor a list of input sources, output sources, and exceptional condition sources and tells us which sources are ready for processing
  • 52.
  • 53.
  • 54.

Editor's Notes

  • #29 Python Network Programming LinuxWorld, New York, January 20, 2004 Steve Holden, Holden Web LLC A connection-oriented server creates a socket, binds it to one or more local ports on which it will listen for connections, and then puts the socket into the listening state to wait for incoming connections. At this point the server process blocks until a connection request arrives. A client creates its own socket, usually without specifying any particular port number, and then connects to the endpoint the server is listening on. The server’s accept() call returns a new socket that the server can use to send data across this particular connection.. The two parties then exchange data using read() and write() calls. The major limitation of this structure is the non-overlapped nature of the request handling in the server. Theoretically it&apos;s possible for the server to use its original socket to listen for further requests while the current request is being handled, but that isn&apos;t shown here. You will learn how to overcome this limitation using standard library classes.
  • #48 Python Network Programming LinuxWorld, New York, January 20, 2004 Steve Holden, Holden Web LLC A connection-oriented server creates a socket, binds it to one or more local ports on which it will listen for connections, and then puts the socket into the listening state to wait for incoming connections. At this point the server process blocks until a connection request arrives. A client creates its own socket, usually without specifying any particular port number, and then connects to the endpoint the server is listening on. The server’s accept() call returns a new socket that the server can use to send data across this particular connection.. The two parties then exchange data using read() and write() calls. The major limitation of this structure is the non-overlapped nature of the request handling in the server. Theoretically it&apos;s possible for the server to use its original socket to listen for further requests while the current request is being handled, but that isn&apos;t shown here. You will learn how to overcome this limitation using standard library classes.