brandon-rhodes / fopnp

Foundations of Python Network Programming (Apress) — scripts and examples

Foundations of Python Network Programming

Welcome!

This GitHub repository offers all of the example Python code from the Third Edition of Foundations of Python Network Programming as revised by Brandon Rhodes for Python 3:

From Amazon.com (affiliate link)
From Apress (the publisher)

Each chapter’s source code lives in its own directory:

• Chapter 1: Introduction to Client-Server Networking
• Chapter 2: UDP
• Chapter 3: TCP
• Chapter 4: Socket Names and DNS
• Chapter 5: Network Data and Network Errors
• Chapter 6: TLS/SSL
• Chapter 7: Server Architecture
• Chapter 8: Caches and Message Queues
• Chapter 9: HTTP Clients
• Chapter 10: HTTP Servers
• Chapter 11: The World Wide Web
• Chapter 12: Building and Parsing E-Mail
• Chapter 13: SMTP
• Chapter 14: POP
• Chapter 15: IMAP
• Chapter 16: Telnet and SSH
• Chapter 17: FTP
• Chapter 18: RPC

Most of this works under Python 2

If you are still using Python 2, you will still benefit from studying these examples as they are more carefully designed than the scripts in the previous edition and also use more modern third-party libraries.

The README.md inside each chapter directory — which GitHub will display automatically when you click a chapter title in the Table of Contents above — has a paragraph near the top stating whether the scripts work with Python 2 or not. Typically each script needs only a quick run through the 3to2 tool in order to operate flawlessly under the old version of the language.

The exceptions are:

• The TLS/SSL examples in Chapter 6 are specific to Python 3.4 and later because only with that version of the language did the ssl module gain safe enough defaults to be recommended for use with secure services. (An effort to backport these features to Python 2.7, however, does seem to be underway.)

• Two of the scripts in Chapter 8 run into a snag if translated to Python 2 because of modules that have switched whether they want to operate on bytes or on Unicode strings.

• The email scripts in Chapter 12 use a new and more convenient API that was added to the language in Python 3.4.

The other changes between Python 2 and 3 will be handled automatically by 3to2: print() is now a function, plain b'byte strings' now take a leading b, and many Standard Library modules were renamed during the transition.

The Network Playground

Many novice network programmers do not have interesting networks to explore — many homes have only a laptop plus a broadband modem to which the homeowner does not have access.

To remedy this situation, I have developed an entire Network Playground consisting of more than a half dozen hosts providing services from SSH and Telnet to a web server and FTP. The “hosts” are built using Docker, and I hope soon to package the whole thing up as a virtual machine that users can download.

The result will be that the user can get a prompt from which they can connect out to the example.com server farm and talk to all kinds of network services. And just as in a real network, they will find that these machines are several hops away — hops that can be inspected by normal networking tools like traceroute:

$ ssh h1

# traceroute www.example.com
traceroute to www.example.com (10.130.1.4), 30 hops max, 60 byte packets
 1  192.168.1.1 (192.168.1.1)  0.513 ms  0.208 ms  0.265 ms
 2  isp (10.25.1.1)  0.544 ms  0.220 ms  0.115 ms
 3  backbone (10.1.1.1)  0.364 ms  0.227 ms  0.252 ms
 4  example.com (10.130.1.1)  0.617 ms  0.355 ms  0.407 ms
 5  www.example.com (10.130.1.4)  1.301 ms  0.415 ms  0.522 ms

You can find the instructions here: The Network Playground.

fopnp's Issues

Unused argument in tcp_deadlock.py

On line 8 def server(host, port, bytecount): the server takes a bytecount argument that appears not be used inside the procedure.

Can't launch VM

Failed to open a session for the virtual machine playground-vm.

NamedPipe#0 failed to bind to local socket /home/brandon/.boot2docker/boot2docker-vm.sock (VERR_FILE_NOT_FOUND).

Result Code: NS_ERROR_FAILURE (0x80004005)
Component: Console
Interface: IConsole {8ab7c520-2442-4b66-8d74-4ff1e195d2b6}

Fix for broken ./build.sh (fails), need to add code to the /base/Dockerfile

For the life of me I couldn't get this to work (need to enable virtualization in the laptop bios). I later learned a lot more about box'es and vagrant (below is how I build the playground with 15.04 vivid64).

The instructions work 100%, but if you don't know what you are doing you might get messed up, so here is a detailed overview, hope it helps.

Install git, virtualbox and vagrant (sudo apt-get install each of them).

Then follow the instructions:
vagrant init brandon-rhodes/playground this command sets up the Vagrantfile with the commands to run

vagrant up --provider virtualbox this command runs the Vagrantfile, the playground isn't found locally so it goes to the Atlas web site and downloads the brandon-rhodes/playground box, which is a preconfigured/bult Ubuntu 14.04 box running Docker with 12 containers that it networks to form a network playground where learners can experiment with HTTP, DNS, SSH, SMTP, and other popular network protocols.

vagrant ssh this puts you into the VM you have access to all the playground code and files ( /home/vagrant/fopnp/ ) you can now run

./launch.sh this launches the playground.

ssh h1 now you are in the playground docker machine h1 and can run commands

traceroute www.example.com this runs a traceroute to the example web page.

Have Fun!

note you have machines h1 through h4 per the documentation.

Building your own: (you have access to all the files natively-- without being in the vagrant system)
Fortunately I have a super star son that worked with me and we figured it out. First on the Ubuntu laptop go into the bios system configuration and enable virtualization technology. (Calvin -- IndyPy Fame, found that issue).
To get the build.sh to run with vivid64 boxes, and then the launch.sh you need to go into the /fopnp/playground/base folder and edit the Dockerfile. The last RUN apt-get install -y python3-crypto python3-lxml python3-zmq
add the following after the zmq libffi-dev libssl-dev
so it looks like
RUN apt-get install -y python3-crypto python3-lxml python3-zmq libffi-dev libssl-dev
(you can use nano to edit the file).
I had to change the /fopnp/palyground/Vagrantfile (again us nano to edit it). The third line, config.vm.box = "ubunto/vivid64" I had to change to config.vm.box = "vivid64"

I am guessing that depends on the box you installed with Vagrant.

So I am running a Ubuntu 16.04 and a 14.04, and did the same installs on both. All is good.

1. in Terminal (open it with Ctrl+Alt+t) cd to the directory you want to install to (you might need to create the directory with mkdir myinstall)
2. sudo apt-get install the following git, virtualbox, and vagrant
3. vagrant box add ubuntu/vivid64
4. sudo git clone https://github.com/brandon-rhodes/fopnp.git
5. cd in to fopnp/playground/ and change the /fopnp/palyground/Vagrantfile (again us nano to edit it). The third line, config.vm.box = "ubunto/vivid64" I had to change to config.vm.box = "vivid64"
6. go into the /fopnp/playground/base folder and edit the Dockerfile. The last RUN apt-get install -y python3-crypto python3-lxml python3-zmq
   add the following after the zmq, libffi-dev libssl-dev
   so it looks like
   RUN apt-get install -y python3-crypto python3-lxml python3-zmq libffi-dev libssl-dev
7. now cd to fopnp/playground
8. vagrant up
9. vagrant ssh
10. ./build.sh this takes a long time
11. ./launch.sh

Now you can ssh h1 then run the traceroute www.example.com

WOW you got it !!!!

p.23 The "FAKE" Message Didn't Go Through

Hi Brandon,

I was trying to simulate forging a response from the server by jumping in and sending datagram before the server has a chance to send its own reply. I ran the server on my Macbook terminal, suspended it by Ctrl + Z, ran the client, and then typed in the three code lines on python3. All I got is the 4 returned from sock.sendto but the terminal running client is still waiting for server's response.

=====1st terminal=====
$ python3 p020_udp_local.py server
Listening at ('127.0.0.1', 1060)
^Z
[1]+  Stopped                 python3 p020_udp_local.py server
$

=====2nd terminal=====
$ python3 p020_udp_local.py client
The OS assigned me the address ('0.0.0.0', 56166)

=====3rd terminal=====
$ python3
Python 3.5.1 (v3.5.1:37a07cee5969, Dec  5 2015, 21:12:44) 
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

> > > import socket
> > > sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
> > > sock.sendto('FAKE'.encode('ascii'), ('127.0.0.1', 37821))
> > > 4

SocketServer request_queue_size

In the book, you recommended against using this because you can not limit the number of connected clients. From reading the documentation, we can set 'request_queue_size' which says:

The size of the request queue. If it takes a long time to process a single request, any requests that arrive while the server is busy are placed into a queue, up to request_queue_size requests. Once the queue is full, further requests from clients will get a “Connection denied” error. The default value is usually 5, but this can be overridden by subclasses.

Does this limit the number of clients? is it a new addition to the library?

Vagrant stuck on 'SSH auth method: private key'

OS: Ubuntu 16.04
After sudo apt-get install virtualbox and vagrant, I entered following commands:

$ vagrant init brandon-rhodes/playground
$ vagrant up --provider virtualbox

and then stuck on 'default: SSH auth method: private key'
Here's the error message:

Timed out while waiting for the machine to boot. 
...
If the box appears to be booting properly, you may want to increase the timeout ("config.vm.boot_timeout") value.

$ vagrant box list 
brandon-rhodes/playground (virtualbox, 1.0)
$ vagrant ssh
ssh_exchange_identification: read: Connection reset by peer
$ vagrant --version
Vagrant 1.8.1
$ vagrant up --provider virtualbox
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Checking if box 'brandon-rhodes/playground' is up to date...
==> default: VirtualBox VM is already running.
$ ./build.sh
...
Downloading/unpacking cryptography>=1.5 (from paramiko->-r /root/requirements2.txt (line 13))
  Running setup.py (path:/tmp/pip_build_root/cryptography/setup.py) egg_info for package cryptography
    error in cryptography setup command: Invalid environment marker: python_version < '3'
    Complete output from command python setup.py egg_info:
    error in cryptography setup command: Invalid environment marker: python_version < '3'
----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /tmp/pip_build_root/cryptography
Storing debug log for failure in /root/.pip/pip.log
The command '/bin/sh -c pip install -r /root/requirements2.txt' returned a non-zero code: 1

SSL read: error:00000000:lib(0):func(0):reason(0), errno 104

Hi Rhodes,

It's been a privilege to use your product and study stuffs from there. Unfortunately I've come across to an error but could not find any easy solution from google. I am using ubuntu 14.04 Lts and virtualbox 5.x

I've executed "vagrant up --provider virtualbox" command and was waiting for it to complete. Then after around 80% I got this error.

default: URL: https://atlas.hashicorp.com/brandon-rhodes/playground
==> default: Adding box 'brandon-rhodes/playground' (v1.0) for provider: virtualbox
default: Downloading: https://atlas.hashicorp.com/brandon-rhodes/boxes/playground/versions/1.0/providers/virtualbox.box

An error occurred while downloading the remote file. The error
message, if any, is reproduced below. Please fix this error and try
again.

Could you please have a look into this issue

docker: Error image fopnp/base:latest not found

Hi,
I got this error after running
./build.sh
./launch.sh

vagrant@vagrant-ubuntu-trusty-64:~/fopnp/playground$ ./launch.sh
++ dirname ./launch.sh
+ cd .
+ grep -q docker
++ groups
+ echo 'vagrant docker'
+ '[' '!' -x /sbin/brctl ']'
+ sudo mkdir -p /var/run/netns
+ sudo modprobe ip_nat_ftp nf_conntrack_ftp
+ start_container h1 fopnp/base 2201
+ hostname=h1
+ image=fopnp/base
+ port=2201
+ container=h1
++ docker inspect -f '{{.State.Pid}}' h1
++ true
+ pid=
+ '[' '' = '' ']'
+ '[' -n 2201 ']'
+ netopts=--publish=2201:22
++ readlink -f ..
+ docker run --name=h1 --hostname=h1 --dns=10.1.1.1 --dns-search=example.com --publish=2201:22 --volume=/home/vagrant/fopnp:/fopnp -d fopnp/base
Unable to find image 'fopnp/base:latest' locally
Pulling repository docker.io/fopnp/base
docker: Error: image fopnp/base:latest not found.
See 'docker run --help'.

appreciate any help
Thanks

IN means？

Is IN is a module?

"Building and Examining Packets"?

Hi Brandon,
In the second last paragraph of the introduction of Chapter 2 - UDP, you mentioned the "Building and Examining Packets" section near the end of Chapter 1, which builds raw ICMP packets and receives an ICMP reply. But I don't see there's such a section. Am I missing something here or do you mean the "Packet Fragmentation" section?
Thanks.

To recieve data contiuously from tcp barcode scanners

HI there, Actually im a beginner for python. I wanna recieve data contiuously from multiple tcp connected barcode scanners.

Can't get Vagrant box to communicate with host machine

Hey Brandon,

I am sorry for posting this here but I searched a million ways to get in touch with you and this was the best way I could come up with. I have your chapter 2 script download to by virtual box running Ubuntu 15 (guest machine) and also downloaded to my host computer (Macbook Pro). I have tried both your local and remote scripts, both as client and server on both machines, but cannot get them to communicate. I have even tried port forwarding 1060 (guest) to 8080 (host) in my Vagrant file and adjusted your script by changing the port on my host machine to 8080 as my default (and without making changes) but haven't had any luck. I am beating my head against the wall trying to figure this out and hope you can help me understand what I am doing wrong. I am going to read the first two chapters again this evening because I am sure I am missing something, likely trivial, and might be able to find the problem myself.

I did manage to get them to both work just fine running them in different terminals without Vagrant, however, I feel like it won't truly sink in unless I have two different machines that I can program to speak to one another.

Lastly, it threw me for a bit of a loop at the bottom of page 26 when you give the host name 'guinness' (great beer by the way) as your command line argument. (I have the 3rd edition). You quickly mentioned giving the host name of your other machine but for a simple minded beginner like myself I wish this would have been better explained than more or less a footnote. However, when running your remote script, I did list the host name of my Vagrant box as an argument 'vagrant-ubuntu-trusty-64' (as well as the empty string. Hopefully I have given you enough information to help me diagnose my issue. Anyway, any guidance would be greatly appreciated!

Thanks,

Tom

Capture 11 app_insecure doesn't start with flask version >2.0.0

Hey,

when trying to start the insecure application with python3 app_insecure.py I'm stuck with the follwoing error:

Traceback (most recent call last):
  File "/home/marius/projects/fopnp/py3/chapter11/app_insecure.py", line 12, in <module>
    get = Environment(loader=PackageLoader(__name__, 'templates')).get_template
  File "/home/marius/.local/lib/python3.9/site-packages/jinja2/loaders.py", line 286, in __init__
    spec = importlib.util.find_spec(package_name)
  File "/usr/lib64/python3.9/importlib/util.py", line 114, in find_spec
    raise ValueError('{}.__spec__ is None'.format(name))
ValueError: __main__.__spec__ is None

which seems to come from this line

get = Environment(loader=PackageLoader(__name__, 'templates')).get_template

Running the app_improved.py works without any issues.

Python Version: 3.9.1
Flask Version: 2.0.1

When I downgraded to flask 1.1.4 app_insecure.py works as expected.
Hope this issue can be a reference for anyone else coming accross this problem.
I'm also curious if there will be updated version for flask > 2.0.0.

Chapter 1 examples no longer work

The examples in Chapter 1 no longer work, because

(a) The Google Geocoder API now requires an API key, and (b) it now requires all requests to be made over https

I have made a fix in my fork of the repo (For Python 3 at least) that will work as soon as a valid API key is placed in a text file in the same directory. Please, let me know if you would like me to make a pull request.

How to connect to playground

I've got the playground-VM running but I don't see how to connect to it. The instructions just say to use SSH but no info on an address to connect to. I've tried to guess at the proper address based on the information in the README but have had no luck. Am i missing something or are the instructions just that vague?

Chapter 6 Certificate no longer accepted in Python 3.7

It seems that with Python 3.7 using create_default_context will no longer accept your example certificate.

ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: invalid CA certificate (_ssl.c:1051)

This works fine in 3.6

OSError with big_sender

Python3 attempting to run big_sender.py gives OSError 22 Invalid argument for line 14

Cant launch the playground

hi, im using ubuntu 14.04(64) on virtualbox with win 7 64 as host, the ovm doesnt seem to work on windows so i tried to build myself on ubuntu. I get as far as ./launch and recieve these errors:

2014-12-09 06:52:06-- ftp://ftp.nl.netbsd.org/vol/2/metalab/distributions/tinycorelinux/4.x/x86/tcz/bridge-utils.tcz
=> ‘bridge-utils.tcz’
Resolving ftp.nl.netbsd.org (ftp.nl.netbsd.org)... 192.87.102.42, 192.87.102.43, 2001:610:1:80aa:192:87:102:43, ...
Connecting to ftp.nl.netbsd.org (ftp.nl.netbsd.org)|192.87.102.42|:21... connected.
Logging in as anonymous ... Logged in!
==> SYST ... done. ==> PWD ... done.
==> TYPE I ... done. ==> CWD (1) /vol/2/metalab/distributions/tinycorelinux/4.x/x86/tcz ... done.
==> SIZE bridge-utils.tcz ... 49152
==> PASV ... done. ==> RETR bridge-utils.tcz ... done.
Length: 49152 (48K) (unauthoritative)

100%[================================================>] 49,152 --.-K/s in 0.09s

2014-12-09 06:52:07 (546 KB/s) - ‘bridge-utils.tcz’ saved [49152]

./launch.sh: 17: ./launch.sh: tce-load: not found
Container started: h1
Container started: h2
Container started: h3
Container started: h4
Container started: modemA
Container started: modemB
Container started: isp
Container started: backbone
Container started: example
Container started: ftp
Container started: mail
Container started: www
Created interface: h1-eth1
Created interface: h2-eth1
Created interface: h3-eth1
Created interface: h4-eth1
Created interface: modemA-eth1
Created interface: modemB-eth1
Cannot find device "homeA"
Created bridge: homeA
sudo: brctl: command not found
sudo: brctl: command not found
Bridged interface: modemA-eth1
sudo: brctl: command not found
Bridged interface: h1-eth1
sudo: brctl: command not found
Bridged interface: h2-eth1
sudo: brctl: command not found
Bridged interface: h3-eth1
sudo: brctl: command not found
Cannot find device "homeB"
Created bridge: homeB
sudo: brctl: command not found
Bridged interface: modemB-eth1
sudo: brctl: command not found
Bridged interface: h4-eth1
Created interface: example-eth1
Created interface: ftp-eth0
Error: argument "mail" is wrong: Invalid "netns" value

Cannot open network namespace "mail": No such file or directory
Cannot open network namespace "mail": No such file or directory
Created interface: mail-eth0
RTNETLINK answers: File exists
Created interface: www-eth0
sudo: brctl: command not found
Cannot find device "exampleCOM"
Created bridge: exampleCOM
sudo: brctl: command not found
Bridged interface: example-eth1
sudo: brctl: command not found
Bridged interface: ftp-eth0
sudo: brctl: command not found
Bridged interface: mail-eth0
sudo: brctl: command not found
Cannot find device "www-eth0"
Bridged interface: www-eth0
Cannot find device "eth2"
Cannot find device "eth0"
Cannot find device "eth1"
Cannot find device "eth1"
Cannot find device "eth0"
Cannot find device "eth0"
RTNETLINK answers: Network is unreachable
Cannot find device "eth0"
Cannot find device "eth1"
Cannot find device "eth2"
Cannot find device "eth0"
Cannot find device "eth1"
Cannot find device "eth2"
RTNETLINK answers: Network is unreachable
Cannot find device "eth1"
Cannot find device "eth1"
RTNETLINK answers: Network is unreachable
Cannot find device "eth0"
Cannot find device "eth0"
RTNETLINK answers: Network is unreachable
Cannot find device "eth0"
Cannot find device "eth0"
RTNETLINK answers: Network is unreachable
RTNETLINK answers: Network is unreachable
Cannot open network namespace "mail": No such file or directory
Cannot open network namespace "mail": No such file or directory

please help.

Chapter 06: lib.SSL_get_version segfaults on python3.6.6 darwin 16.7.0

I'm getting a segfault on my mac osx on the following line:

fopnp/py3/chapter06/test_tls.py

Line 81 in 2c1d0fc

version_bytestring = lib.SSL_get_version(struct.ssl)

.

SSL_get_version with debug prints

def SSL_get_version(ssl_sock):
    """Reach behind the scenes for a socket's TLS protocol version."""

    print('\n--START SSL_get_version--\n')

    lib = ctypes.CDLL(ssl._ssl.__file__)
    print('lib:', lib)

    lib.SSL_get_version.restype = ctypes.c_char_p
    print('lib.SSL_get_version.restype:', lib.SSL_get_version.restype)

    address = id(ssl_sock._sslobj)
    print('address:', address)

    struct = ctypes.cast(address, ctypes.POINTER(PySSLSocket)).contents
    print('struct:', struct)

    version_bytestring = lib.SSL_get_version(struct.ssl)
    print('doesnt get here')

    return version_bytestring.decode('ascii')

test_tls.py output:

Zachs-MBP:chapter06 zhammer$ ./test_tls.py localhost 1060 -a ca.crt

Address we want to talk to.......... ('localhost', 1060)
Peer certificate.................... provided
Name(s) on peer certificate......... localhost
Whether name(s) match the hostname.. Yes
Certificates loaded of type crl..... 0
Certificates loaded of type x509.... 1
Certificates loaded of type x509_ca. 0

--START SSL_get_version--

lib: <CDLL '/opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/lib-dynload/_ssl.cpython-36m-darwin.so', handle 7faf3c522df0 at 0x10cc14320>
lib.SSL_get_version.restype: <class 'ctypes.c_char_p'>
address: 4508905200
struct: <__main__.PySSLSocket object at 0x10cc11378>
Segmentation fault: 11

System info:

Zachs-MBP:chapter06 zhammer$ python3 --version
Python 3.6.6
Zachs-MBP:chapter06 zhammer$ uname -a
Darwin Zachs-MBP.fios-router.home 16.7.0 Darwin Kernel Version 16.7.0: Thu Jan 11 22:59:40 PST 2018; root:xnu-3789.73.8~1/RELEASE_X86_64 x86_64

Notes

I'm not familiar with ctypes or the ssl library, so I wasn't able to do much debugging on my end. Let me know what I can do to provide more info.

how to install it on windows?

Can't connect on OS X

Hello,

I got the playground to run on VirtualBox 4.3.20 on OS X by adding a local host adapter in VirtualBox settings and disabling the serial adapter. The host only adapter is set to 192.168.56.1 and I can't change the third octet to anything besides starting with a 5. I try ssh -p 220x brandon@localhost from terminal and am unable to connect. What should I change, need to add etc to get connected?

errors starting vm on windows 7

c:> .\VBoxHeadless.exe --startvm playground-vm

Error:

Error: failed to start machine. 
Error message: NamedPipe#0 failed to create named pipe 
\home\brandon\.boot2docker\boot2docker-vm.sock (VERR_INVALID_NAME)

Code Correction

This is regarding the book on Python3. Found a couple of errors with the code provided in the book. Could not find the code in Github, hence reporting the page numbers in the book.

Page 63:

mysock = server_sock.accept()
addr, port = mysock.getpeername()

accept() method is supposed to return a tuple with the active connection being the first item. accept()[0] should have been used here. Unless you do that, the next statement would report an error. I wonder how the second statement got executed in your code.

Page 90:
try:
host = sock.connect(address)
except socket.error as e:

sock.connect() returns a NoneType and yet is being assigned to a variable in here.

chapter 08 memcache not working

Running chapter08/squares.py just hangs up. It looks like the library memcache is not working like before.

`#!/usr/bin/env python3

Foundations of Python Network Programming, Third Edition

https://github.com/brandon-rhodes/fopnp/blob/m/py3/chapter08/squares.py

Using memcached to cache expensive results.

import memcache, random, time, timeit

def compute_square(mc, n):
value = mc.get('sq:%d' % n)
if value is None:
time.sleep(0.001) # pretend that computing a square is expensive
value = n * n
mc.set('sq:%d' % n, value)
return value

def main():
mc = memcache.Client(['127.0.0.1:11211'])

def make_request():
    compute_square(mc, random.randint(0, 5000))

print('Ten successive runs:')
for i in range(1, 11):
    print(' %.2fs' % timeit.timeit(make_request, number=2000), end='')
print()

if name == 'main':
main()`

opnp / py3 / chapter07 / client.py not work with python3.6

The box 'ubuntu/vivid64' could not be found

when I execute the "vagrant up --provider virtualbox" command, I get that error.
after searching for a while I found that the the ubuntu/vivid64 has been removed from the website (https://atlas.hashicorp.com/ubuntu).

Someone on stackoverflow has propose a solution for downloading the same version of ubuntu :
http://stackoverflow.com/questions/35812321/vagrant-up-the-box-ubuntu-vivid64-could-not-be-found

Python version used to compile the scripts

Hey,

This is regarding your book using Python3. I happened to observe that most of the scripts you've written are being executed using the command 'python' instead of 'python3'. Assuming that you're not using an alias, 'python' on Ubuntu should point to python2 rather than python3. This would result in a wrong interpreter being used to run the scripts. Also, you're using the hashbang '#! /usr/bin/env python3' in every script and still running them using the command 'python'. I think this needs to be corrected.

chapter 2: UDP client broadcasts go to any server

While reading the book, in chapter 2 in the section about UDP broadcasts (reading the e-book, so not sure the physical page number, sorry), it says that packets sent by the udp_broadcast.py client will be picked up by any instances of the udp_broadcast servers at the same time, which they are. But the book goes on to say that normal servers won't and that I could try running a few udp_remote instances to be convinced.

I couldn't find anything distinguishing the udp_broadcast server socket code from the udp_remote server socket code as far as socket configuration was concerned. So I tried running the udp_remote server at the same time as the udp_broadcast server and both servers received the 'Broadcast Datagram!' sent by the broadcast client. I'm not sure if the code or the book is wrong in this case, or if the situation is totally unique to my network settings somehow.

Here's a screenshot of the broadcasts being picked up by udp_remote.py:

So my question is, is there some socket option that's supposed to be set in the udp_broadcast server code that's supposed to make it behave differently from the udp_remote server code? Is the passage about regular servers not picking up broadcasts incorrect? Is there something about the way my network handles broadcasts that's non-standard and skewing my results?

sys info:
broadcast IP: 192.168.0.255
local: OSX 10.10.5, python 3.5, 192.168.0.175
remote: winserver 2012, cygwin, python 3.4.3, 192.168.0.125
router: tomato v1.25

Can't generate localhost.pem

Not sure if this is a bug or not, but anyway:

cd playground/certs
rm localhost.crt
make localhost.crt

results in :

openssl ca -batch -config ca.cnf -days 36500 \
	-keyfile ca.key -cert ca.crt \
	-in localhost.csr -outdir . -out localhost.crt
Using configuration from ca.cnf
Check that the request matches the signature
Signature ok
The Subject's Distinguished Name is as follows
countryName           :PRINTABLE:'us'
stateOrProvinceName   :PRINTABLE:'New York'
localityName          :PRINTABLE:'New York'
organizationName      :PRINTABLE:'Example from Apress Media LLC'
organizationalUnitName:PRINTABLE:'Foundations of Python Network Programming 3rd Ed'
commonName            :PRINTABLE:'localhost'
emailAddress          :IA5STRING:'[email protected]'
Certificate is to be certified until Apr 24 20:32:55 2117 GMT (36500 days)
failed to update database
TXT_DB error number 2
make: *** [Makefile:29: localhost.crt] Error 1

And localhost.crt is empty.

$ pacman -Qi openssl
Name            : openssl
Version         : 1.1.0.e-1
Description     : The Open Source toolkit for Secure Sockets Layer and Transport Layer Security
Architecture    : x86_64
URL             : https://www.openssl.org
Licenses        : custom:BSD
Groups          : None
Provides        : None
Depends On      : perl
Optional Deps   : ca-certificates [installed]
Required By     : apache  bind-tools  coreutils  cower  curl  erlang-nox  git  iputils  ldns  lib32-openssl  libarchive  libcurl-compat  libevent  libgit2  libmariadbclient  libsasl  libshout  libssh2
                  mariadb-clients  mongodb  net-snmp  nmap  ntp  openssh  perl-net-ssleay  poco  podofo  postfix  postgresql  postgresql-libs  pypy  python  python2  python25  python32  python33  python34  ruby
                  s-nail  toggldesktop  viber  virtualbox  w3m  wpa_supplicant
Optional For    : apr-util
Conflicts With  : None
Replaces        : None
Installed Size  : 6.23 MiB
Packager        : Pierre Schmitz <[email protected]>
Build Date      : Thu 16 Feb 2017 04:53:19 PM EET
Install Date    : Thu 27 Apr 2017 05:48:07 PM EEST
Install Reason  : Installed as a dependency for another package
Install Script  : No
Validated By    : Signature

Unable to properly start the image on a windows host

I am using Virtualbox in order to run the playground on a windows 8.1 machine.
I downloaded the image, imported and started it and I do see that I get my machine to listen on forwarded ports. When I connect using Putty (localhost:2201) I just get a black screen, it is not disconnecting but I am unable to do anything on these ports.

When I look at the console in virtualbox I just see a black screen, seems like the OS is not starting and nothing is really happening.

Can you suggest what to do?

chapter06/safe_tls.py does not work with Python 3.6

The server:

$ /usr/bin/python3.6 safe_tls.py -s localhost.pem localhost 8000
Listening at interface 'localhost' and port 8000
Connection from host '127.0.0.1' and port 40132
Traceback (most recent call last):
  File "safe_tls.py", line 50, in <module>
    server(args.host, args.port, args.s, args.a)
  File "safe_tls.py", line 35, in server
    ssl_sock = context.wrap_socket(raw_sock, server_side=True)
  File "/usr/lib/python3.6/ssl.py", line 401, in wrap_socket
    _context=self, _session=session)
  File "/usr/lib/python3.6/ssl.py", line 808, in __init__
    self.do_handshake()
  File "/usr/lib/python3.6/ssl.py", line 1061, in do_handshake
    self._sslobj.do_handshake()
  File "/usr/lib/python3.6/ssl.py", line 683, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: TLSV1_ALERT_UNKNOWN_CA] tlsv1 alert unknown ca (_ssl.c:749)

The client:

$ /usr/bin/python3.6 safe_tls.py -s localhost.pem localhost 8000
Listening at interface 'localhost' and port 8000
Connection from host '127.0.0.1' and port 40132
Traceback (most recent call last):
  File "safe_tls.py", line 50, in <module>
    server(args.host, args.port, args.s, args.a)
  File "safe_tls.py", line 35, in server
    ssl_sock = context.wrap_socket(raw_sock, server_side=True)
  File "/usr/lib/python3.6/ssl.py", line 401, in wrap_socket
    _context=self, _session=session)
  File "/usr/lib/python3.6/ssl.py", line 808, in __init__
    self.do_handshake()
  File "/usr/lib/python3.6/ssl.py", line 1061, in do_handshake
    self._sslobj.do_handshake()
  File "/usr/lib/python3.6/ssl.py", line 683, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: TLSV1_ALERT_UNKNOWN_CA] tlsv1 alert unknown ca (_ssl.c:749)

At the same time, using python3.4 works just fine

$ /usr/bin/python3.4 safe_tls.py -a ca.crt localhost 8000
Connected to host 'localhost' and port 8000
b'Simple is better than complex.'

$ /usr/bin/python3.4 safe_tls.py -s localhost.pem localhost 8000
Listening at interface 'localhost' and port 8000
Connection from host '127.0.0.1' and port 40140

I guess that this is normal behavior and that the fix is to regenerate the certificates.