[net] Ether/ne2k interrupt problem about elks

[mfld-fr]

Not really a bug. After your change, the ETH / NE2K driver requests the IRQ11, but in irqtab.S, the interrupt for the board is still fixed to IRQ9 (IRQ11 is surrounded by #if 0). You are the first one to need to set the interrupt number else than to 9 😄.

Could you please change that piece of code and test again, just to validate the fix, before we submit a clean patch ?

! IRQ9 is used by the Ethernet device

#ifdef CONFIG_ETH
_irq9:
	push    ax
	mov     ax, #9
	br      _irqit
#endif

#if 0
_irq10:
	push	ax
	mov	ax,#10
	br	_irqit
_irq11:
	push	ax
	mov	ax,#11
	jmp	_irqit

[Mellvik]

Thanks, that did it. A comment in ne2k.h about this would be useful. ELKS now initializes correctly on the compaq using the default config file (NE2K added). No packets sent or received yet, but I don’t have a working ne2k-system om QEMU either, so it’s under investigation. A QEMU command line that works w/ne2k would be helpful. I noticed that the FPU interrupt is commented out as well, does that mean the FPU is inactive even if it’s present and configured? —Helge

[mfld-fr]

The basic test for the ETH / NE2K driver is the eth command, that you could build in /elks/elkscmd/test/eth/. It tests the connection to the local network with ARP requests / replies. Don't forget to update the IP addresses in the code with the ones of your real local network, because the defaults are for QEMU as configured in qemu.sh.

[lithoxs]

Actually, two changes are needed in irqtab.S. The one pointed to by mfld-fr and another where the pointer to the ISR is placed in the interrupt vector table.
The code for irqtab.S has changed recently. The relevant parts for you case should now be:

#ifdef CONFIG_ETH_NE2K
#define METH 0x800
#else
...

#if 0
_irq9: ! Ethernet device
call _irqit
.byte 9
_irq10: ! USB
call _irqit
.byte 10
#endif
#ifdef CONFIG_ETH_NE2K
_irq11: ! Sound
call _irqit
.byte 11
#endif
#if 0
_irq12: ! Mouse
call _irqit

[lithoxs]

Regarding the FPU interrupt, ELKS does not know anything about the FPU.

[Mellvik]

Thanks, that helped - at least as far as QEMU is concerned. FOr the record - here’s the qemu cmd line that got it running (it took a while to figure out): qemu-system-i386 -k no -machine isapc -nodefaults -m 1 -fda full3.net.img -netdev user,id=mynet0,hostfwd=tcp:127.0.0.1:2323-10.0.2.15:23 -device ne2k_isa,irq=11,netdev=mynet0 Telnets OK both ways. Still no success with the physical machine — working on it. //Helge

[mfld-fr]

Hello, any news on that issue ?

[Mellvik]

Long story in which the first interface showed memory problems, the second seems to work hardware wise, but does not interact well with the driver. Needs debugging and ended up on the backburner before summer. I'll get back to it in the fall. Btw - re. FPU - someone said elks has no idea about fpu. Why is there a hw_fpu entry in config? Med vennlig hilsen/best regards, Helge Skrivervik myMAYDAY.com - 9201 7146

[mfld-fr]

Mmm... what is your CPU model ? Because I just remember I used some 80186 instructions in the Ne2k driver, and this could explain the failure.

[Mellvik]

Interesting. It is a 286. My .config says CONFIG_CPU_8086=y in case it matters. And - if I remember correctly - what happens is that the driver initializes OK. Then, when sending the first packet, it never stops sending. The Powersupply on the machine just broke, so I cannot recreate right now. —Helge

[mfld-fr]

Forget my last idea if you have a 80286.
About "never stops sending", could you please detail ?
Did you catch an Ethernet dump before the power supply break ?

[Mellvik]

No luck with the 286, but symptoms are repeatable on a similar 386 (Compaq portable): Boot -> NE2k configures ok -> login -> telnet 10.0.2.1 Then the TX LED on the interface lits permanently, the corresponding lamp on the switch blinks continuously, but no packets are received at the other end. The sending stops when hitting ^C and the system hangs. The RX LED on the interface blinks when packets are sent from the other end - ARP requests etc, but the N2K does not seem to be transmitting anything. The other end does not report any traffic at all, not even broadcasts. I really need to check whether the IF is actually working (with DOS or Solaris or something) before proceeding. —Helge

[mfld-fr]

Hello, any progress in your testing ? Were you able to test the HW with another OS ?

[Mellvik]

My 286 is again operational, network works with DOS, will download the latest ELKS and test ‘real soon now’ - meaning in the next few weeks. —Helge

[mfld-fr]

OK, thanks for the update. So the problem is specific to ELKS.

If you could connect point-to-point your target with the NE2K card to a PC, run the eth test program, reboot the target, then run the telnet client program, and capture the packets with WireShark, it would help us to diagnose.

[Mellvik]

I finally got around to it - configured up a new RaspberryPi for ELKS, loaded the elks sources from about a week ago. Here’s the status: - this is the 286 Compaq portable III - Boots DOS, network works fine with MTCP - Elks default kernel config, except for the obvious - enabling the nk2k - Elks: No change from the previous time I tested - Network stuff configures OK during boot - Any network activity on elks causes the traffic light on the ethernet switch to flash continuously until next boot - Elks responds to arp request - Inbound telnet: The console saus Telnetd: Accepted connection - No outbound packets from elks - except the arp response. The blinking on the switch does not cause visible traffic at the other end (tcpdump) - The blinking stops when elks is doing other stuff, such as reading the floppy Seems to me we have a hang/loop situation in the driver - any hint as to the next test? —Helge 10:52:53.428080 ARP, Request who-has 10.0.2.15 tell 10.0.2.1, length 28 10:52:53.429842 ARP, Reply 10.0.2.15 is-at 52:54:00:12:34:56 (oui Unknown), length 50

[mfld-fr]

Could you please take a photo of your ISA card, so that we could identify the main chip and check its datasheet against the current implementation ?

You could also disable all the network applets in /etc/rc.d/rc.sysinit (ktcp, telnetd, httpd, etc), only run the /bin/eth command and check the captured packets.

And last but not least, you could add some printk() in ne2k-main.c to try to find what is looping.

[Mellvik]

Thanks - Picture included. /bin/eth: I need help with this one. Several includes don’t work, which I could fix - except they exist in several places and are different. E.g. stdlib.h exists in 5 places, 3 version. Which one so I use? root@raspberrypi:/usr/src/elks-master/elkscmd/test/eth# make bcc -0 -O -ansi -I/usr/src/elks-master/include -I/usr/src/elks-master/elks/include -D__ELKS__ -DELKS_VERSION=\"0.2.0-prei86\" -c -o eth.o eth.c eth.c:6: error: Cannot open include file eth.c:7: error: Cannot open include file eth.c:8: error: Cannot open include file eth.c:9: error: Cannot open include file ../../Make.defs:104: recipe for target 'eth.o' failed make: *** [eth.o] Error 1 root@raspberrypi:/usr/src/elks-master/elkscmd/test/eth# vi eth.c [1]+ Stopped vi eth.c root@raspberrypi:/usr/src/elks-master/elkscmd/test/eth# find ../../../ -name stdlib.h ../../../cross/lib/bcc/include/stdlib.h ../../../cross/lib/bcc/include/bsd/stdlib.h ../../../cross/build/dev86/unproto/stdlib.h ../../../cross/build/dev86/libc/include/stdlib.h ../../../cross/build/dev86/libc/include/bsd/stdlib.h root@raspberrypi:/usr/src/elks-master/elkscmd/test/eth# find ../../../ -name stdlib.h -exec sum '{}' ';' 28901 2 30587 1 10493 1 28901 2 30587 1 Printk() - in process. —Helge

[mfld-fr]

Sorry, but I see no picture in your latest reply.

About /bin/eth, it looks like you forgot to set your build environment as explained in the main README : in the top directory (here .../elks-master), run . tools/env.sh before trying to invoke any make command.

Do you still have the legacy dev86 package installed on your system ? Please uninstall it if stills there : now ELKS relies on the auto-build & latest dev86 of @jbruchon, not the legacy one that is rather outdated and not able to run "in-tree".

[mfld-fr]

By the way, I just saw that there is a mistake in /elkscmd/Makefile(114) that prevents the test programs to be automatically built : please replace ifdef $(CONFIG_APP_TEST) by ifdef CONFIG_APP_TEST. Just fixed it for the next PR.

[Mellvik]

Hmmm, the pic was indeed included, I’ll reduce the size and try again (below). Just did the printk test: - after init, ne2k-main loops in select (normal initialization (std rc.sysinit), I can log in as usual while the select printk is looping). - telnet 10.0.2.1 and the select loop is replaced by an int-read-write loop which is causing the activity light on the switch. I don’t know which one came first of these. I’ll do the /bin/eth test later today. —Helge

[jbruchon]

You cannot post attachments of any kind (other than plain text) to Github issues; they are discarded and you're posting by email. Post a link to the image on an image host such as Imgur instead.

[mfld-fr]

Hey... I also just realized you are not capturing the packets on a point-to-point connection, but you are using a router. So if you are sniffing the packets behind that router, you won't be able to see the packets that are actually sent by ELKS, if they are malformed. Please make the capture directly on the NE2K side, not behind a router.

[Mellvik]

Don’t know how you arrived at that conclusion. This is indeed a p2p. The fact that there is a switch in between is just convenience, there is nothing else on the switch but the two parties. —Helge

[Mellvik]

OK, try this link for the picture(s). Sorry about that. https://www.icloud.com/sharedalbum/#B0N5ejO17dbZw6 <https://www.icloud.com/sharedalbum/#B0N5ejO17dbZw6> —Helge

[mfld-fr]

About the switch / router : I am wondering why the traffic LED on your switch / router is blinking while you capture no packet except the ARP ones ? I cannot explain this else than thinking that your switch / router discards some packets. This is why I am suggesting to connect the NE2K socket directly to the host socket where you run the capture program.

[mfld-fr]

Thanks for the photos. Unfortunately, I cannot find any datasheet for the W89C90, W89C901 or W89C902. I only found some references in the W89C94 one, but this latest is for PCI and has no detailed information on the NE2K implementation.

[Mellvik]

That’s a valid concern - I always figured the switch filtered invalid packets and noise. Which turns out to be the case. Making a direct connect didn’t change anything. The arp reply is still the only packet coming from ELKS. As to /bin/eth, I still fail to get it to compile. The build environ is in place, and the makefile change didn’t seem to change anything, I’ll have to take a closer look tomorrow. —Helge

[mfld-fr]

About /bin/eth, I checked on my side. After fixing the mistake as described above, and selecting the test applets in the configuration menu, that program is automatically built. Did you check you have no legacy dev86 in your path, just the one in /cross ?

[Mellvik]

The devil is always in the details. Thanks - Eth now compiles - see tcpdump below. I also added a picture to the album showing the screen. Eth sends a bunch of arp requests, receives replies, then bombs on read. I’m sure the output means a lot more to you than to me. If you need a more verbose TCPdump, let me know. BTW - here’s some additional info on the ethernet card (from the ebay listing where I got it): Danpex ISA NE2000 Network Card - W89C902P EN-2200T <http://rover.ebay.com/rover/0/e11401.m1842.l5919/7?euid=e8dc16c9e3954dab92f60f8ba6eb1549&bu=43585854729&loc=http%3A%2F%2Fwww.ebay.com%2Fulk%2Fvod%3Fitemid%3D152564787566%26transid%3D1573028734005&exe=16966&ext=43852&es=0&nqc=EEAABAACAABLAkAAiAAAAIQAAQAAADABAEAAABAkAEgAAAABFCAADAAAAAAAAKAAAIgAAAAABAAgBAAAAAABEAAAAAAAAAABggAAEAAQABQgAQABAgAEEAAAzgAAEBCCAABBQAAAAAIEEAACAAAEQgAAAAAAAQQBCSCIQAAAACAAAAAAAAAAACAC&nqt=EEAAFAACAABLAkAAiAAAAIQAAQAAADABAEAABBAkAEgAAAABFCAADAAAAAAAAKAAAIgAAAAABAAgBAAAABABEAAAAAAAAAABggAAEAAQABQgAQAFAgAEEAAAzgAAEBCCAABBQAAAAAIEEAACAAAEQgAAAAABAQwBCSCIQAAAACAAAAAAAAAAACAC&ec=1&sojTags=es=es,nqc=nqc,nqt=nqt,ec=ec,exe=exe,ext=ext,bu=bu> Further, Wikipedia has a comment that some winbond based NE2K cards ignore the reset signal. —Helge 08:02:22.382859 ARP, Request who-has 10.0.2.2 tell 10.0.2.15, length 46 08:02:22.382945 ARP, Reply 10.0.2.2 is-at b8:27:eb:9a:77:bc (oui Unknown), length 28 08:02:22.467194 ARP, Request who-has 10.0.2.2 tell 10.0.2.15, length 46 08:02:22.467247 ARP, Reply 10.0.2.2 is-at b8:27:eb:9a:77:bc (oui Unknown), length 28 08:02:22.551343 ARP, Request who-has 10.0.2.2 tell 10.0.2.15, length 46 08:02:22.551393 ARP, Reply 10.0.2.2 is-at b8:27:eb:9a:77:bc (oui Unknown), length 28 08:02:22.635642 ARP, Request who-has 10.0.2.2 tell 10.0.2.15, length 46 08:02:22.635689 ARP, Reply 10.0.2.2 is-at b8:27:eb:9a:77:bc (oui Unknown), length 28 08:02:22.719767 ARP, Request who-has 10.0.2.2 tell 10.0.2.15, length 46 08:02:22.719828 ARP, Reply 10.0.2.2 is-at b8:27:eb:9a:77:bc (oui Unknown), length 28 08:02:22.803731 ARP, Request who-has 10.0.2.2 tell 10.0.2.15, length 46 08:02:22.803777 ARP, Reply 10.0.2.2 is-at b8:27:eb:9a:77:bc (oui Unknown), length 28 08:02:22.888276 ARP, Request who-has 10.0.2.2 tell 10.0.2.15, length 46 08:02:22.888387 ARP, Reply 10.0.2.2 is-at b8:27:eb:9a:77:bc (oui Unknown), length 28 08:02:22.972219 ARP, Request who-has 10.0.2.2 tell 10.0.2.15, length 46 08:02:22.972304 ARP, Reply 10.0.2.2 is-at b8:27:eb:9a:77:bc (oui Unknown), length 28 08:02:23.056468 ARP, Request who-has 10.0.2.2 tell 10.0.2.15, length 46 08:02:23.056524 ARP, Reply 10.0.2.2 is-at b8:27:eb:9a:77:bc (oui Unknown), length 28 08:02:23.140544 ARP, Request who-has 10.0.2.2 tell 10.0.2.15, length 46 08:02:23.140610 ARP, Reply 10.0.2.2 is-at b8:27:eb:9a:77:bc (oui Unknown), length 28 08:02:23.224778 ARP, Request who-has 10.0.2.2 tell 10.0.2.15, length 46 08:02:23.224827 ARP, Reply 10.0.2.2 is-at b8:27:eb:9a:77:bc (oui Unknown), length 28 08:02:23.308991 ARP, Request who-has 10.0.2.2 tell 10.0.2.15, length 46 08:02:23.309037 ARP, Reply 10.0.2.2 is-at b8:27:eb:9a:77:bc (oui Unknown), length 28 08:02:23.393087 ARP, Request who-has 10.0.2.2 tell 10.0.2.15, length 46 08:02:23.393149 ARP, Reply 10.0.2.2 is-at b8:27:eb:9a:77:bc (oui Unknown), length 28 08:02:23.477395 ARP, Request who-has 10.0.2.2 tell 10.0.2.15, length 46 08:02:23.477449 ARP, Reply 10.0.2.2 is-at b8:27:eb:9a:77:bc (oui Unknown), length 28 08:02:23.561484 ARP, Request who-has 10.0.2.2 tell 10.0.2.15, length 46 08:02:23.561535 ARP, Reply 10.0.2.2 is-at b8:27:eb:9a:77:bc (oui Unknown), length 28 08:02:23.645694 ARP, Request who-has 10.0.2.2 tell 10.0.2.15, length 46 08:02:23.645740 ARP, Reply 10.0.2.2 is-at b8:27:eb:9a:77:bc (oui Unknown), length 28 08:02:23.729880 ARP, Request who-has 10.0.2.2 tell 10.0.2.15, length 46 08:02:23.729939 ARP, Reply 10.0.2.2 is-at b8:27:eb:9a:77:bc (oui Unknown), length 28 08:02:23.813980 ARP, Request who-has 10.0.2.2 tell 10.0.2.15, length 46 08:02:23.814023 ARP, Reply 10.0.2.2 is-at b8:27:eb:9a:77:bc (oui Unknown), length 28 08:02:23.898316 ARP, Request who-has 10.0.2.2 tell 10.0.2.15, length 46 08:02:23.898367 ARP, Reply 10.0.2.2 is-at b8:27:eb:9a:77:bc (oui Unknown), length 28 08:02:23.982451 ARP, Request who-has 10.0.2.2 tell 10.0.2.15, length 46 08:02:23.982499 ARP, Reply 10.0.2.2 is-at b8:27:eb:9a:77:bc (oui Unknown), length 28 08:02:24.066861 ARP, Request who-has 10.0.2.2 tell 10.0.2.15, length 46 08:02:24.066925 ARP, Reply 10.0.2.2 is-at b8:27:eb:9a:77:bc (oui Unknown), length 28 08:02:24.150922 ARP, Request who-has 10.0.2.2 tell 10.0.2.15, length 46 08:02:24.150970 ARP, Reply 10.0.2.2 is-at b8:27:eb:9a:77:bc (oui Unknown), length 28 08:02:24.218516 ARP, Request who-has 10.0.2.2 tell 10.0.2.15, length 46 08:02:24.218564 ARP, Reply 10.0.2.2 is-at b8:27:eb:9a:77:bc (oui Unknown), length 28 08:02:24.284728 ARP, Request who-has 10.0.2.2 tell 10.0.2.15, length 46 08:02:24.284778 ARP, Reply 10.0.2.2 is-at b8:27:eb:9a:77:bc (oui Unknown), length 28 08:02:24.350888 ARP, Request who-has 10.0.2.2 tell 10.0.2.15, length 46 08:02:24.350931 ARP, Reply 10.0.2.2 is-at b8:27:eb:9a:77:bc (oui Unknown), length 28 08:02:24.417173 ARP, Request who-has 10.0.2.2 tell 10.0.2.15, length 46 08:02:24.417221 ARP, Reply 10.0.2.2 is-at b8:27:eb:9a:77:bc (oui Unknown), length 28

[Mellvik]

A further dive into this makes me thing there is a timing problem in the NE2k code. Observations: From the 386 compaq: (25MHz 386) If CPU-config is set to 286, the 386 hangs on Ethernet init If CPU is set to 8086, the Ethernet IF initializes, ktcpd & telnet starts and the telnet command does not hang the system, but there is no activity on the net whatsoever. Same config, and the eth test program fails immediately - even the address set/get test fails (see picture). 286 (12MHz): The eth test (ARP-loop) fails after 26 read/write cycles. Read /dev/eth: Unknown error 512 (seemingly ERESTARTSYS)

[mfld-fr]

Just a question : /bin/eth should be run alone, with all the other internet applets down. Are you sure you ran it after removing ktcp, telnetd, etc, from the configuration ? Otherwise it conflicts with ktcp on the /dev/eth resource.

[mfld-fr]

For the other tests, please proceed step by step : first remove all applets, and try to pring and connect from outside. Then start only ktcp, and try again (ping and connect). Then start only ktcp & telnetd, then retry.

[Mellvik]

Yes, ktcp and telnetd are off during the eth run. No, there is no icmp in here as far as I can tell, so ping has never worked. I didn’t even think icmp was implemented. And finally, ‘connect from outside’ - I don’t know what that means. In order to connect there has to be something to connect to. I seem to be missing something here.

[mfld-fr]

Ok so then your test on the ARP loop is meaningful. It shows the data path is good, but I agree, is not stable, as the test loop does not complete. Thanks to your traces, it also shows that there is only one interrupt for two operations (write + read), where two are expected (one for the end of the write, one for data available for read). Let me dig into the code to try to understand this wrong behavior...

About 'connect from outside' : I mean initiate the connection from the host side (= inbound), with the ping and telnet clients, without any reader, to see how the driver would behave. Expected is a 'read interrupt' for each packet ELKS would receive until buffer exhaustion. And yes, ICMP looks like to be implented in ktcp, according to its sources. Please do not forget to capture the packets in P2P in all that test cases.

[Mellvik]

Summary from today’s test: In ne2k-main.c: The status flag ne2k_inuse is never set. Should probably be set in ne2k_open. Don't think it affects this problem though. In eth - the test program: It always reports mac address mismatch even though the addresses match. (memcmp?) Kernel: The last few bytes of the kernel are overwritten - probably by the ne2k driver. ne2k_main.o is the last object in the generated kernel, containing printk static strings at the end. As I add printk messages for debugging, the final few bytes of the last string get screwed. The reason this has not been discovered before is that the error message (ne2k not found) never got printed - the card was always there since the probe is a dummy … When ktcp is active (presumably listening), it loops continuously in select(in). Add a printk() to ne2k_select, and there is one single output just after ktcp starts, then - given an interrupt from the keyboard (or anywhere I suppose), the loop starts. It can be terminated with a ^C from the keyboard which terminates both ktcp and the shell. All of the above applies to qemu and physical. Connecting to the physical machine from the outside gives no response when ktcp is off, presumably because the interface is shut down (not opened) and interrupts are off. Ktcp started, arp works as before, nothing else. Starting telnetd does not change anything. And tracing interrupts is useless because of the memory leak above (I will add more printk's to the code in order to get meaningful messages).

[Mellvik]

… And some more: Legend (printk from ne2k-main.c): Si - select in, Ir - Interrupt Receive, Rd - read packet, Wr - Write packet, It - Interrupt transmit 16:39:22.773340 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from b8:27:eb:9a:77:bc (oui Unknown), length 343 Command: ktcp 10.0.2.15 /dev/eth& telnet 10.0.2.2 Response: Si Ir Si Rd Si [no touching the keyboard since then we would have a select loop again] Nothing on the wire from these two commands. The following starts after at telnet 10.0.2.15 from the outside. Ir Si Rd Wr It Ir Si Rd Si Ir Si Rd Si Ir Si Rd Si Ir Sd Ri Sd Ir etc. 16:40:20.819173 ARP, Request who-has 10.0.2.15 tell 10.0.2.2, length 28 16:40:20.823314 ARP, Reply 10.0.2.15 is-at 52:54:00:12:34:56 (oui Unknown), length 50 16:40:20.823360 IP 10.0.2.2.37142 > 10.0.2.15.telnet: Flags [S], seq 1455799474, win 29200, options [mss 1460,sackOK,TS val 1774337165 ecr 0,nop,wscale 7], length 0 16:40:21.828105 IP 10.0.2.2.37142 > 10.0.2.15.telnet: Flags [S], seq 1455799474, win 29200, options [mss 1460,sackOK,TS val 1774338174 ecr 0,nop,wscale 7], length 0 16:40:23.908116 IP 10.0.2.2.37142 > 10.0.2.15.telnet: Flags [S], seq 1455799474, win 29200, options [mss 1460,sackOK,TS val 1774340253 ecr 0,nop,wscale 7], length 0 16:40:27.754255 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from b8:27:eb:9a:77:bc (oui Unknown), length 343 16:40:27.988123 IP 10.0.2.2.37142 > 10.0.2.15.telnet: Flags [S], seq 1455799474, win 29200, options [mss 1460,sackOK,TS val 1774344333 ecr 0,nop,wscale 7], length 0 16:40:36.228138 IP 10.0.2.2.37142 > 10.0.2.15.telnet: Flags [S], seq 1455799474, win 29200, options [mss 1460,sackOK,TS val 1774352573 ecr 0,nop,wscale 7], length 0 16:40:52.868127 IP 10.0.2.2.37142 > 10.0.2.15.telnet: Flags [S], seq 1455799474, win 29200, options [mss 1460,sackOK,TS val 1774369212 ecr 0,nop,wscale 7], length 0 16:41:31.318063 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from b8:27:eb:9a:77:bc (oui Unknown), length 343 There is a new picture available as well. New test: Restarting the system, starting ktcp, then a ping from the outside, ktcp goes into an endless loop: Wr It Rd Wr It Rd etc. forever. No Receive interrupt, but the ICMP packet seems to be read correctly (debug from ktcp: IP Got packet of size : 88 Version/IHL:4/5 TypeOfService/Length: 0/84 ID/FLAGS/FRAGMENT OFFSET: -20966/64 TTL: 64 PROTOCOL: 1 SADDR: 10.0.2.2 DADDR: 10.0.2.15 CHECK SUM = 0 8 0 3B DE 51 E2 0 etc. etc. IP : ICMP packet )

[mfld-fr]

Thank you very much for all that test data ! Still working on it...

[Mellvik]

OK, and now that we’ve started, some more observations (this is all quemu, not related to my ne2k interface or the physical machine): - When ktcp starts, there is one call to select (in). - Any keypush on the console causes an endless loop calling ne2k select (input). - ^C on the console aborts not only the loop, but ktcp, some times even the shell. - Changing ktcp to ignore SIGINT does just that, the loop becomes uninterruptible. ==>This means that even though ktcp is in the background, it receives interrupts from the keyboard, which it should not. Closing 0,1 and 2 in ktcp does not change that. - The select loop terminates on any incoming packet, and ktcp behaves normally until the next keypush. This is all repeatable. I don’t have sufficient knowledge of the inside of the system to analyze this, but it seems to me that there is a signal handling problem in ash and a problem in the select handler (or maybe the keyboard handler/ntty code). Even if the ktcp process owns the keyboard, a keypush interrupt should not trigger the ethernet driver. It is likely that the loop is caused by this confusion: The keypush triggers the wrong flag or interrupt handler indicating that there is a ethernet packet ready, then select (in) loops until there is one, as indicated by an external packet terminating the loop. My 2cents.

[mfld-fr]

@Mellvik : I fixed the problem of the screwed characters in #210.

[mfld-fr]

@Mellvik : I fixed the problem of the endless looping in #212.

[mfld-fr]

@Mellvik : could you please rebase your IRQ 11 commits on the latest upstream master and test again on your physical machines ? This is to know if we still have a problem specific to your Ethernet adapter, in addition to the bad terminal management tracked by #211.

[Mellvik]

Will do - btw my current ne2k adapter has IRQ9, making life easier. At some point down the road we need to make the IRQ a config option.

[Mellvik]

Ne2k test - not much change from before: The keyboard is now partly disabled (extra keys) - no lights, no capslock numlock etc. Possibly a config issue, will look into that. Network: same pattern as before - arp ok, no ICMP Echo response, no outgoing packets from elks Outgoing telnet hangs the system. Needs ctl-alt-del. Incoming telnet: No response, as before, no outgoing traffic on the wire. I will put back the print's in the driver to get more details. Cannot kill the telnetd process. Shell Interrupt problem (this is in Qemu): Repeatable, but does not happen every time: hitting ^C at the shell prompt cause a call to select(in) in the ne2k driver (!), and the message 'Signal received: 2' (I put back a printk("Si") in the ne2k driver) Not reproducible until reboot. Hints as to further debugging welcome.

[Mellvik]

Additional info re. ‘Signal received’ from my previous message (possibly #125): — This happens exactly once per boot, the first INT (^C) after bootup, and regardless of whether we’re logged in or not. - The message seems to be originating from telnetd, indicating that it is not completely detached from stdio. - Later ^Cs at the console seem to be ignored, even by the shell. - This may be related to what has previously been reported as the shell interrupt problem (#125). If telnetd isn’t running, the first ^C is normal, a 2nd ^C terminates the shell like before, but no ‘cannot cd to XXX’ or similar.

[mfld-fr]

Let us forget the terminal stuff for now (that is reproducible in QEMU and so could be tested later without your real HW), and let us focus on the "no TX" problem with your ETH board, and only with inbound telnet to avoid any side effect of a key press in the ELKS console (select only the ktcp and telnetd applets in the configuration).

From what you describe since the beginning, I am starting to think that your board does not reset the TXP bit 2 in the command register (CR offset 00h) after transmitting the first and only packet (i.e. the ARP reply).

In current implementation, the driver puts the packet to send in the board TX buffer, then sets this bit to trigger the TX, and will block any more write / send until this bit is reset (see ne2k_tx_stat() call in ne2k_write()).

So I would add some traces in the ne2k_write() to confirm that this function completes on first call (= the observed ARP reply), and is blocked on second call (= the expected reply to TCP SYN packet from telnet), because ne2k_tx_stat() never returns NE2K_STAT_TX and ne2k_write() enters interruptible_sleep_on(), causing ktcp to hang.

[Mellvik]

OK, take a look at the last picture of the shared album. Printk’s from incoming telnet starts at the login prompt. There is no ARP exchange, the compaq was just rebooted and the host has the address already so it doesn’t ask. The first incoming packet starts the loop which is endless unless terminated by a (fortunately working) ^C. Indicating that it is actually telnet that is looping… The packet trace isn’t all that interesting. After three unanswered telnet SYNs, the host sends a new ARP and never gets an answer. The legend from the printk’s is like before except that the ‘res’ variable is included from both reads and writes. —Helge 18:21:04.460011 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from b8:27:eb:9a:77:bc (oui Unknown), length 343 18:21:27.810930 IP 10.0.2.2.37218 > 10.0.2.15.telnet: Flags [S], seq 3876232944, win 29200, options [mss 1460,sackOK,TS val 2212404241 ecr 0,nop,wscale 7], length 0 18:21:28.868101 IP 10.0.2.2.37218 > 10.0.2.15.telnet: Flags [S], seq 3876232944, win 29200, options [mss 1460,sackOK,TS val 2212405298 ecr 0,nop,wscale 7], length 0 18:21:30.948114 IP 10.0.2.2.37218 > 10.0.2.15.telnet: Flags [S], seq 3876232944, win 29200, options [mss 1460,sackOK,TS val 2212407378 ecr 0,nop,wscale 7], length 0 18:21:32.868082 ARP, Request who-has 10.0.2.15 tell 10.0.2.2, length 28 18:21:33.908077 ARP, Request who-has 10.0.2.15 tell 10.0.2.2, length 28 18:21:34.948073 ARP, Request who-has 10.0.2.15 tell 10.0.2.2, length 28

[mfld-fr]

OK, what is the meaning of R0 / R1, or W2 in that screenshot ?

[mfld-fr]

Could you also activate the commented traces in the file ktcp/deveth.c, function deveth_send(), to see if at least ktcp tries to respond to the telnet SYNs ?

[Mellvik]

See the last sentence in my message... More in the morning.

[mfld-fr]

Sorry, but I still not understand your R0 / R1 / W2. The res variable could be a return value from a lower level function, or the size of the received / sent packet. What actual value are you displaying in your trace ?

[mfld-fr]

And last but not least, what is the speed of this R0W2It endless loop ? Very fast or an iteration each second ? If the period is around one second, I think I understood the current behavior (there is a retry loop in ktcp_run() every second).

[Mellvik]

More details. The looping seems to be as fast as the machine can go (or rather, as fast as the screen can print) — a couple of screen-lines per second. The W0/R1 etc provides the return value (res) from the stat call for both read and write, like this: res = ne2k_rx_stat (); printk("R%d", res); deveth.c output: see new screenshot. The leading ‘IrR1Si’ sequence is not related to the telnet-session.

[mfld-fr]

Thanks for these latest info. Forget my previous though about the TXP bit, it is working as expected. So ELKS actually responds to the TCP SYNs and the packets are really sent on the wire (I guess the router traffic LED is blinking at the same speed as that loop, right ?), as we get the TX interrupt after the write.

So why do you not capture any packet from ELKS ? I guess this is because these packets are too short, and are discarded by your host. Ethernet requires a minimal packet size of 64. QEMU is more tolerant and works with little packets.

Let me prepare a patch to confirm that analysis...

[Mellvik]

Yes, this is - at least partly - correct. The traffic indicator on the switch blinks continuously - on the compaq-port, nothing on the port going to the host. Connecting directly to the host, the host light blinks just like the switch port did, but captures no packets. So the packets are indeed invalid, and the switch does what it should, filter them. The traffic is not responding to telnet syns though. It is triggered by the first incoming SYN, but after that, ELKS is looping as fast as it can. I get maybe 4 SYN packets from the host in 10 seconds while the compaq is running wild.

[mfld-fr]

Yes, in addition to the invalid packet size, there is another problem in the select() that does not honour the timeout of 1 second before ktcp tries again to resend the SYN ACK. But let us fix that quick looping in a second step, it should not be a blocking point in the traffic.

[mfld-fr]

Here is the patch to have an ETH packet minimum size of 64 bytes.

In ne2k_main.c, line 128:

		// Client should write packet once
		// otherwise end of packet will be lost

		if (len > MAX_PACKET_ETH) len = MAX_PACKET_ETH;
		memcpy_fromfs (send_buf, data, len);

		if (len < 64) len = 64;  /* issue #133 */
		res = ne2k_pack_put (send_buf, len);

Could you please apply that patch and test again the inbound telnet ?

[Mellvik]

No more looping. ELKS responds (on the screen) once to every incoming packet. Still, only one (the first) packet hits the wire, this time with zero content, so the host continues to hit it with ARP requests. See trace and new screenshot. —Helge 11:38:33.196823 ARP, Request who-has 10.0.2.15 tell 10.0.2.2, length 28 11:38:33.386579 00:00:00:00:00:00 (oui Ethernet) > 00:00:00:00:00:00 (oui Ethernet) Null Information, send seq 0, rcv seq 0, Flags [Command], length 50 0x0000: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 0x0010: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 0x0020: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 0x0030: 0000 .. 11:38:34.274787 ARP, Request who-has 10.0.2.15 tell 10.0.2.2, length 28 11:38:35.314760 ARP, Request who-has 10.0.2.15 tell 10.0.2.2, length 28 11:38:36.354937 ARP, Request who-has 10.0.2.15 tell 10.0.2.2, length 28 11:38:37.394768 ARP, Request who-has 10.0.2.15 tell 10.0.2.2, length 28 11:38:38.434761 ARP, Request who-has 10.0.2.15 tell 10.0.2.2, length 28 11:38:39.649428 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from b8:27:eb:9a:77:bc (oui Unknown), length 343

[mfld-fr]

Is this capture really made in P2P, not through the switch / router ?

[mfld-fr]

Could you please clean and rebuild all ELKS ? I cannot understand how just adding a line to force the minimal size could cause such a big effect...

[Mellvik]

You are absolutely right. I did screw up while editing the ne2k-main.c file. Sorry about that! Now fixed - the results are entirely different. Dramatic improvement I’d say. I get a login prompt from telnet, get a shell and can actually execute a couple of commands before telnetd goes into a loop writing W1 continuously on the console. I will have to look more closely at the pretext before the loop tomorrow, but the packet trace is included below. The command history after the login via telnetd is a ‘ls /‘ and a couple of <CR>s. I’m always connected via a switch - which is equivalent to bare wire. It’s just convenient for my setup. A router is a different animal altogether, and wouldn’t even pass the ARP. Let me know if there is more to look at tomorrow. —Helge 17:13:54.823880 ARP, Request who-has 10.0.2.15 tell 10.0.2.2, length 28 17:13:55.013548 ARP, Reply 10.0.2.15 is-at 52:54:00:12:34:56 (oui Unknown), length 50 17:13:55.013610 IP 10.0.2.2.35052 > 10.0.2.15.telnet: Flags [S], seq 1234129115, win 29200, options [mss 1460,sackOK,TS val 2913226362 ecr 0,nop,wscale 7], length 0 17:13:55.014957 ARP, Reply 10.0.2.15 is-at 52:54:00:12:34:56 (oui Unknown), length 50 17:13:55.154163 ARP, Request who-has 10.0.2.2 tell 10.0.2.15, length 50 17:13:55.154238 ARP, Reply 10.0.2.2 is-at b8:27:eb:9a:77:bc (oui Unknown), length 28 17:13:55.155555 ARP, Request who-has 10.0.2.2 tell 10.0.2.15, length 50 17:13:55.155602 ARP, Reply 10.0.2.2 is-at b8:27:eb:9a:77:bc (oui Unknown), length 28 17:13:55.337031 IP 10.0.2.15.telnet > 10.0.2.2.35052: Flags [S.], seq 676, ack 1234129116, win 256, options [mss 1024], length 0 17:13:55.337166 IP 10.0.2.2.35052 > 10.0.2.15.telnet: Flags [.], ack 1, win 29200, length 0 17:13:55.337556 IP 10.0.2.2.35052 > 10.0.2.15.telnet: Flags [P.], seq 1:25, ack 1, win 29200, length 24 [telnet DO SUPPRESS GO AHEAD, WILL TERMINAL TYPE, WILL NAWS, WILL TSPEED, WILL LFLOW, WILL LINEMODE, WILL NEW-ENVIRON, DO STATUS [|telnet] 17:13:55.338901 IP 10.0.2.15.telnet > 10.0.2.2.35052: Flags [S.], seq 676, ack 1234129116, win 256, options [mss 1024], length 0 17:13:55.338969 IP 10.0.2.2.35052 > 10.0.2.15.telnet: Flags [.], ack 1, win 29200, length 0 17:13:55.550770 IP 10.0.2.15.telnet > 10.0.2.2.35052: Flags [.], ack 25, win 256, length 0 17:13:55.551484 IP 10.0.2.15.telnet > 10.0.2.2.35052: Flags [.], ack 25, win 256, length 0 17:13:55.825174 IP 10.0.2.15.telnet > 10.0.2.2.35052: Flags [P.], seq 1:43, ack 25, win 256, length 42 17:13:55.825261 IP 10.0.2.2.35052 > 10.0.2.15.telnet: Flags [.], ack 43, win 29200, length 0 17:13:55.826530 IP 10.0.2.15.telnet > 10.0.2.2.35052: Flags [P.], seq 1:43, ack 25, win 256, length 42 17:13:55.826603 IP 10.0.2.2.35052 > 10.0.2.15.telnet: Flags [.], ack 43, win 29200, length 0 17:13:56.111959 IP 10.0.2.15.telnet > 10.0.2.2.35052: Flags [P.], seq 43:85, ack 25, win 256, length 42 17:13:56.112023 IP 10.0.2.2.35052 > 10.0.2.15.telnet: Flags [.], ack 85, win 29200, length 0 17:13:56.113295 IP 10.0.2.15.telnet > 10.0.2.2.35052: Flags [P.], seq 43:85, ack 25, win 256, length 42 17:13:56.113358 IP 10.0.2.2.35052 > 10.0.2.15.telnet: Flags [.], ack 85, win 29200, length 0 17:14:13.922959 IP 10.0.2.2.35052 > 10.0.2.15.telnet: Flags [P.], seq 25:31, ack 85, win 29200, length 6 17:14:14.102495 IP 10.0.2.15.telnet > 10.0.2.2.35052: Flags [.], ack 31, win 256, length 0 17:14:14.103215 IP 10.0.2.15.telnet > 10.0.2.2.35052: Flags [.], ack 31, win 256, length 0 17:14:14.282099 IP 10.0.2.15.telnet > 10.0.2.2.35052: Flags [P.], seq 85:87, ack 31, win 256, length 2 17:14:14.282181 IP 10.0.2.2.35052 > 10.0.2.15.telnet: Flags [.], ack 87, win 29200, length 0 17:14:14.283368 IP 10.0.2.15.telnet > 10.0.2.2.35052: Flags [P.], seq 85:87, ack 31, win 256, length 2 17:14:14.283442 IP 10.0.2.2.35052 > 10.0.2.15.telnet: Flags [.], ack 87, win 29200, length 0 17:14:21.063730 IP 10.0.2.2.35052 > 10.0.2.15.telnet: Flags [P.], seq 31:37, ack 87, win 29200, length 6 17:14:22.594779 IP 10.0.2.2.35052 > 10.0.2.15.telnet: Flags [P.], seq 31:37, ack 87, win 29200, length 6 17:14:25.634795 IP 10.0.2.2.35052 > 10.0.2.15.telnet: Flags [P.], seq 31:37, ack 87, win 29200, length 6 17:14:27.565892 IP 10.0.2.15.telnet > 10.0.2.2.35052: Flags [.], ack 37, win 256, length 0 17:14:27.566040 IP 10.0.2.2.35052 > 10.0.2.15.telnet: Flags [P.], seq 37:39, ack 87, win 29200, length 2 17:14:27.567808 IP 10.0.2.15.telnet > 10.0.2.2.35052: Flags [.], ack 37, win 256, length 0 17:14:27.804226 IP 10.0.2.15.telnet > 10.0.2.2.35052: Flags [.], ack 37, win 256, length 0 17:14:27.805432 IP 10.0.2.15.telnet > 10.0.2.2.35052: Flags [.], ack 37, win 256, length 0 17:14:28.047780 IP 10.0.2.15.telnet > 10.0.2.2.35052: Flags [P.], seq 87:93, ack 37, win 256, length 6 17:14:28.047878 IP 10.0.2.2.35052 > 10.0.2.15.telnet: Flags [.], ack 93, win 29200, length 0 17:14:28.049698 IP 10.0.2.15.telnet > 10.0.2.2.35052: Flags [P.], seq 87:93, ack 37, win 256, length 6 17:14:28.049771 IP 10.0.2.2.35052 > 10.0.2.15.telnet: Flags [.], ack 93, win 29200, length 0 17:14:28.314727 IP 10.0.2.15.telnet > 10.0.2.2.35052: Flags [.], ack 37, win 256, length 0 17:14:28.315971 IP 10.0.2.15.telnet > 10.0.2.2.35052: Flags [.], ack 37, win 256, length 0 17:14:28.553756 IP 10.0.2.15.telnet > 10.0.2.2.35052: Flags [.], ack 39, win 256, length 0 17:14:28.554852 IP 10.0.2.15.telnet > 10.0.2.2.35052: Flags [.], ack 39, win 256, length 0 17:14:28.970887 IP 10.0.2.15.telnet > 10.0.2.2.35052: Flags [P.], seq 93:157, ack 39, win 256, length 64 17:14:28.970951 IP 10.0.2.2.35052 > 10.0.2.15.telnet: Flags [.], ack 157, win 29200, length 0 17:14:28.972954 IP 10.0.2.15.telnet > 10.0.2.2.35052: Flags [P.], seq 93:157, ack 39, win 256, length 64 17:14:28.973020 IP 10.0.2.2.35052 > 10.0.2.15.telnet: Flags [.], ack 157, win 29200, length 0 17:14:29.222214 IP 10.0.2.15.telnet > 10.0.2.2.35052: Flags [P.], seq 157:173, ack 39, win 256, length 16 17:14:29.222288 IP 10.0.2.2.35052 > 10.0.2.15.telnet: Flags [.], ack 173, win 29200, length 0 17:14:29.223526 IP 10.0.2.15.telnet > 10.0.2.2.35052: Flags [P.], seq 157:173, ack 39, win 256, length 16 17:14:29.223586 IP 10.0.2.2.35052 > 10.0.2.15.telnet: Flags [.], ack 173, win 29200, length 0 17:14:29.452012 IP 10.0.2.15.telnet > 10.0.2.2.35052: Flags [P.], seq 173:179, ack 39, win 256, length 6 17:14:29.452085 IP 10.0.2.2.35052 > 10.0.2.15.telnet: Flags [.], ack 179, win 29200, length 0 17:14:29.453311 IP 10.0.2.15.telnet > 10.0.2.2.35052: Flags [P.], seq 173:179, ack 39, win 256, length 6 17:14:29.453369 IP 10.0.2.2.35052 > 10.0.2.15.telnet: Flags [.], ack 179, win 29200, length 0

[mfld-fr]

So nice 😃 !

Yes, you are right for the switch nature, I don't know why I started to consider it as a router earlier in the discussion ?..

There is still some extra and useless traffic because your HW is rather fast, coupled to the select() ineffective timeout, but it looks like we are done to make the NE2K driver & board to basically work.

Let us then close this long issue by submitting the above patch, and focus on other defective parts in dedicated issues (#213 #211 #125 #124 and so on).

[pawosm-arm]

Here is the patch to have an ETH packet minimum size of 64 bytes.

memcpy_fromfs(send_buf, data, len);
if (len < 64U) len = 64U;  /* issue #133 */
...
res = len;

^^ I don't think this is right. The value returned through res isn't telling how many bytes were put into send buffer.

IMO it should be like this:

                if (len > MAX_PACKET_ETH) len = MAX_PACKET_ETH;
                memcpy_fromfs(send_buf, data, len);

                if (ne2k_pack_put(send_buf,
                     (len < 64) ? 64 : len /* issue #133 */
                   )) {
                        res = -EIO;
                        break;
                }

                res = len;

[ghaerr]

Hello @pawosm-arm,

I can see you've been intricately inspecting driver source :)

You are correct on the point that the write return value isn't what the user specified when len < 64, as it is returned larger, which is usually not a good thing, since the user's buffer wasn't actually advanced. Note that the len can be returned smaller (also never handled nor needed to be in ktcp) when len > MAX_PACKET_ETH. The driver has made it clear that multiple writes for a packet are not supported.

However, the current code does return the amount of bytes sent on the wire, if you include the garbage bytes required for proper operation. It could be said it would be better to return the larger len to inform the application of what happened.

That said, the code could be changed to what you're talking about with less code using:

if (len > MAX_PACKET_ETH) len = MAX_PACKET_ETH;
res = len [add this statement]
...
res = len; [remove this statement]

I'll have to think which return approach would be better. In general, ktcp can't do anything different either way, and the only real purpose in the return value is to report anomalies, which is currently only checked for read returns. Write return values are ignored because of the higher probability of packet loss anyways.

Thank you!