HostFS+TCPIP

for bbc micro/electron hardware, peripherals & programming issues (NOT emulators!)
dominicbeesley
Posts: 868
Joined: Tue Apr 30, 2013 11:16 am
Contact:

Re: HostFS+TCPIP

Post by dominicbeesley » Fri Oct 11, 2019 11:34 am

I'd still recommend *DCD or *DCHDIR as *HCF seems a bit unintuitive and doesn't fit with *DIN - your choice though! One thing I keep meaning to add is a *HELP to both the client rom and the server as I keep having to open up the sources to remember what the commands are!

Why must folders start with "_"?

@tom_seddon, I'll put you down for a card too.

I spent some time last night improving my schematic design. I fixed the silly errors and turned all the diodes the right way round. I've now also added a serial EEPROM for storing the MAC address, local IP address, etc this will mean an extra 74x74, 74x245 and a 24AA64 eeprom which might need a slightly larger board. I was toying with the idea of an optional Real Time Clock but not sure that that is really necessary whereas the eeprom probably is?

D

User avatar
sweh
Posts: 2089
Joined: Sat Mar 10, 2012 12:05 pm
Location: New York, New York
Contact:

Re: HostFS+TCPIP

Post by sweh » Fri Oct 11, 2019 1:36 pm

dominicbeesley wrote:
Fri Oct 11, 2019 11:34 am
Why must folders start with "_"?
I needed a way of distinguishing between a directory used for *DIN purposes and a directory used for folder purposes. I didn't want people to *DIN a folder, or *HCF to a disk :-)

I originally was going to use something like "NAME.FOLDER" but then thought it might be nice for the folders to be all listed together (and likely first) in an "ls". So I thought "!FOLDER" but that messed around with ksh prompts. Then "@FOLDER" but that was messy. "-FOLDER" has Unix CLI complications. So "_FOLDER" :-)

If you have another idea, I'm willing to entertain it. That's why it's on a branch and not on master yet, 'cos it's not fixed and still a work in progress :-)

I think I'd like to stick with "folder" terminology, though, to avoid confusion between "*DIR" directories and the host directories :-)
Rgds
Stephen

dominicbeesley
Posts: 868
Joined: Tue Apr 30, 2013 11:16 am
Contact:

Re: HostFS+TCPIP

Post by dominicbeesley » Fri Oct 11, 2019 3:29 pm

Ah, I see, that does make sense now, though I'd maybe hide the "_" at the client end as it's a server detail?

D

User avatar
sweh
Posts: 2089
Joined: Sat Mar 10, 2012 12:05 pm
Location: New York, New York
Contact:

Re: HostFS+TCPIP

Post by sweh » Fri Oct 11, 2019 4:06 pm

dominicbeesley wrote:
Fri Oct 11, 2019 3:29 pm
Ah, I see, that does make sense now, though I'd maybe hide the "_" at the client end as it's a server detail?
I thought about that, but I was being lazy. And then I thought "consistency" 'cos if I do "mkdir _foobar" on the Linux machine I can "*HCF _foobar" on the Beeb.

But mostly laziness :-)

I might stop being lazy, though!
Rgds
Stephen

dominicbeesley
Posts: 868
Joined: Tue Apr 30, 2013 11:16 am
Contact:

Re: HostFS+TCPIP

Post by dominicbeesley » Sat Oct 12, 2019 10:04 am

I sent off the mk2 boards late last night to jlpcb. Threes will take a couple of weeks to make and ship and hopefully won't get but with ridiculous duty charges.

In three meantime I'll start looking at the code. I'm not sure whether to drive straight in and just support hostfs first or to start by implementing Sprows socket API first...

User avatar
Elminster
Posts: 3925
Joined: Wed Jun 20, 2012 8:09 am
Location: Essex, UK
Contact:

Re: HostFS+TCPIP

Post by Elminster » Sat Oct 12, 2019 1:56 pm

He uses some standard code

I had the base some where, oh yes lwIP.

I used it for notes when I write telnet on the beeb

dominicbeesley
Posts: 868
Joined: Tue Apr 30, 2013 11:16 am
Contact:

Re: HostFS+TCPIP

Post by dominicbeesley » Sat Oct 12, 2019 2:24 pm

Have you got sources?...I'm slowly disassembling the LANManager ROM, mainly to make sure I return errors and statuses the same way...

User avatar
Elminster
Posts: 3925
Joined: Wed Jun 20, 2012 8:09 am
Location: Essex, UK
Contact:

Re: HostFS+TCPIP

Post by Elminster » Sat Oct 12, 2019 8:03 pm

It is open source, will find link later

https://en.wikipedia.org/wiki/LwIP

I asked Sprow years ago as I had work out how to process some bits that weren’t in his doc

I have the version number he used somewhere. I was hoping someone would put it in one of the emulators to make it easier to develop my Telnet program, but it never happened.

Edit: lwip version 1.3.0

dominicbeesley
Posts: 868
Joined: Tue Apr 30, 2013 11:16 am
Contact:

Re: HostFS+TCPIP

Post by dominicbeesley » Sun Oct 13, 2019 1:40 pm

That's strange lwIP looks to be written in C and the code in the ROM (so far) looks like hand coded assembler. Also, the emails I got from Sprow don't mention this, but he did say it was Berkley Sockets API where as lwIP isn't or at least the pdf I found says it's not? Are we talking about the same thing or have I just not dug far enough into the ROM (I was disassembling LANManager 0.37).

I've made a small start (from scratch) on this. I don't really need Sprow's code much as the stuff already built into the WizNet chip does most of the network stack stuff I'd just like to have more information on the error codes returns in blk%?3 as these don't look to be standard Berkley i.e. &88 for method not implemented?

Did you get anything working with telnet? It would be good to have some software to test against my API

D

User avatar
Elminster
Posts: 3925
Joined: Wed Jun 20, 2012 8:09 am
Location: Essex, UK
Contact:

Re: HostFS+TCPIP

Post by Elminster » Sun Oct 13, 2019 5:29 pm

LwIP is the tcp/ip stack on sprow master Ethernet card. The rom would be the acorn interface to the card.

So you won’t be using LwIP I expect but your own tcp/ip stack. But the rom might make more sense cross referenced to the stack that was used.

Telnet was never finished, debugging was a pain due to no emulator support for sprow card. On my todo list to restart. I had nula support for ansi working from memory for playing muds etc. Think I posted some videos.

Edit: just remembered it is huge so it uses shadow ram and co pro.

dominicbeesley
Posts: 868
Joined: Tue Apr 30, 2013 11:16 am
Contact:

Re: HostFS+TCPIP

Post by dominicbeesley » Sun Oct 13, 2019 7:40 pm

Ah i get you. Sorry i was being thick.

I think I'll have to carry on debugging Sprows rom for error numbers but ill get the lwip sources too.

BITD when i did Berkley Sockets for my day job I remember it being a case of trawling Linux sources to get clues as to likely error numbers...

User avatar
Elminster
Posts: 3925
Joined: Wed Jun 20, 2012 8:09 am
Location: Essex, UK
Contact:

Re: HostFS+TCPIP

Post by Elminster » Wed Oct 16, 2019 8:48 pm

Forgot to ask, why don't you have room for Sprow ethernet card in Master? I have 'most' master mods and it still fits. Something to do with Blitter board?

dominicbeesley
Posts: 868
Joined: Tue Apr 30, 2013 11:16 am
Contact:

Re: HostFS+TCPIP

Post by dominicbeesley » Thu Oct 17, 2019 12:39 am

I've plenty of room in the master...I wanted this for the model B which is what I use mostly for the Blitter stuff - I've still not got the blitter working in the Master. I tried a couple of weekends ago but didn't get very far...yet

D

User avatar
Elminster
Posts: 3925
Joined: Wed Jun 20, 2012 8:09 am
Location: Essex, UK
Contact:

Re: HostFS+TCPIP

Post by Elminster » Thu Oct 17, 2019 8:27 am

Ah that makes sense. Indeed interweb via cable on the Beeb would be a welcome addition.

But you need to work harder on the Blitter I want to put it in a Master :twisted: :)

dominicbeesley
Posts: 868
Joined: Tue Apr 30, 2013 11:16 am
Contact:

Re: HostFS+TCPIP

Post by dominicbeesley » Thu Oct 17, 2019 8:43 am

So do i but we are waiting to move house and I've not got room for both. I'll post an update on the master to the other thread this weekend.

User avatar
Elminster
Posts: 3925
Joined: Wed Jun 20, 2012 8:09 am
Location: Essex, UK
Contact:

Re: HostFS+TCPIP

Post by Elminster » Thu Oct 17, 2019 8:48 am

dominicbeesley wrote:
Thu Oct 17, 2019 8:43 am
So do i but we are waiting to move house and I've not got room for both. I'll post an update on the master to the other thread this weekend.
The same issue as me, well I am not moving, but the other way around. I have room to have Electron and Master out, but not the Beeb. Waiting to win the lottery so I can build Man Cave. Good luck on the move, I vowed to never move again when we last moved 7 or so years ago.

dominicbeesley
Posts: 868
Joined: Tue Apr 30, 2013 11:16 am
Contact:

Re: HostFS+TCPIP

Post by dominicbeesley » Thu Oct 17, 2019 2:29 pm

This move seems to be taking the longest but we've already moved 2 times in 3 years.

Made a bit of progress late last night / early this morning. I've got TCP working slowly but reliably, well enough to do a simple web client. There's still plenty of software to write. This time I've decided to do it properly. I'll implement Sprow's API with suitable adjustments and improvements and then use that in the main HostFS port. This may slow it down a little but probably not on larger transfers as once you get into big transfers the bulk of the optimisation is in the memory shuffling loops.
20191017_150712-s.jpg
This has proved quite a challenge on the 6502 after the ease of the 6809. The Wiznet want's to do everything in 16bit chunks which was easy on the 6809 but on the 6502 odd bytes and pointers need to be saved somewhere so I've ended up having to abuse page 0 a lot (stacking and borrowing pointers) to allow this to work without the FS being selected.

Ive borrowed the following locations:

Code: Select all

\ these pointers are "borrowed from the OS, we need to save them and restore"
zp_mos_txtptr				= &F2		\ OS Text pointer (eg star commands and filenames)
zp_wiz_tmpptr				= zp_mos_txtptr	\ used as a general pointer in e.g. recv/send 
zp_mos_genPTR				= &F6		\ PHROM/RFSROM/OSRDSC/OSWRSC pointer
zp_wiz_priptr				= zp_mos_genPTR \ used to store memory pointer
zp_cfs_w				= &E2		\ something Cassettey -
zp_wiz_curchan				= zp_cfs_w
zp_mos_input_buf			= &E8		\ pointer to text input buffer
zp_wiz_tmp				= zp_mos_input_buf
I think these should be ok as they're all foreground locations and I put them back when I've finished

So far only TCP works and only in client mode, the rest should be fairly straightforward. Then I want to have a go at DHCP and DNS resolution though this might have to wait as it looks like it might be a bit of a minefield, unlike the Sprow card the WizNet doesn't have this built in.

After that I'll start with the HostFS side in earnest which should be more straightforward.... :^o

The new Mk.2 board is on the way from China, not much difference other than corrected addresses and the eeprom but I get excited nonetheless. It's now with China GB Mail - no idea how long that is likely to take but hopefully I won't get stung by Customs this time!
mk2-jlpcb.jpg
D

User avatar
Elminster
Posts: 3925
Joined: Wed Jun 20, 2012 8:09 am
Location: Essex, UK
Contact:

Re: HostFS+TCPIP

Post by Elminster » Thu Oct 17, 2019 2:52 pm

Okay I guess you can put me down for a test/early adopter board. Maybe with the possibility of a larger audience, it might gee me up to looking at my Telnet program again. Although I wrote it in BASIC, hence huge and slow, maybe I need to write it in ASM. Or C and cross compile.

User avatar
jgharston
Posts: 3696
Joined: Thu Sep 24, 2009 11:22 am
Location: Whitby/Sheffield
Contact:

Re: HostFS+TCPIP

Post by jgharston » Thu Oct 17, 2019 10:17 pm

dominicbeesley wrote:
Thu Oct 17, 2019 2:29 pm
I've borrowed the following locations:

Code: Select all

\ these pointers are "borrowed from the OS, we need to save them and restore"
zp_mos_txtptr				= &F2		\ OS Text pointer (eg star commands and filenames)
zp_wiz_tmpptr				= zp_mos_txtptr	\ used as a general pointer in e.g. recv/send 
zp_mos_genPTR				= &F6		\ PHROM/RFSROM/OSRDSC/OSWRSC pointer
zp_wiz_priptr				= zp_mos_genPTR \ used to store memory pointer
zp_cfs_w				= &E2		\ something Cassettey -
zp_wiz_curchan				= zp_cfs_w
zp_mos_input_buf			= &E8		\ pointer to text input buffer
zp_wiz_tmp				= zp_mos_input_buf
I think these should be ok as they're all foreground locations and I put them back when I've finished
If you're working in the foreground you can stack and restore all of &F2/&F3 and &F5 to &FB, &FD/&FE. Typically, code would stack'n'restore the command workspace at &A8-&AF or the temporary filing system space at &B0-&BF* (as long as you don't issue a filing system or MOS call). (Not the perm. FS space at &C0-&CF 'cos some filing system may be using it in the background.)

* Or even &A8-&BF if you are desperate for 24 zero-page locations. The UTILS in DFS ROMs uses &A8-&B2, using &B0-&B2 very transitorily** knowing that nothing can use them while it's using them, and knowing that it loses them as soon as it's finished with them.

** *BUILD puts a control block for OSWORD &00 at &AE-&B2, the very first thing OSWORD &00 does is copy the control block elsewhere, so &B0-&B2 are immediately released.

Code: Select all

$ bbcbasic
PDP11 BBC BASIC IV Version 0.25
(C) Copyright J.G.Harston 1989,2005-2015
>_

User avatar
jgharston
Posts: 3696
Joined: Thu Sep 24, 2009 11:22 am
Location: Whitby/Sheffield
Contact:

Re: HostFS+TCPIP

Post by jgharston » Thu Oct 17, 2019 10:26 pm

dominicbeesley wrote:
Sun Oct 13, 2019 1:40 pm
I've made a small start (from scratch) on this. I don't really need Sprow's code much as the stuff already built into the WizNet chip does most of the network stack stuff I'd just like to have more information on the error codes returns in blk%?3 as these don't look to be standard Berkley i.e. &88 for method not implemented?
When I typed up the API for the Wiki, as the command codes where those used in the RISC OS API, I assumed the return codes were also those used in the RISC OS API.

Code: Select all

$ bbcbasic
PDP11 BBC BASIC IV Version 0.25
(C) Copyright J.G.Harston 1989,2005-2015
>_

User avatar
jgharston
Posts: 3696
Joined: Thu Sep 24, 2009 11:22 am
Location: Whitby/Sheffield
Contact:

Re: HostFS+TCPIP

Post by jgharston » Thu Oct 17, 2019 10:29 pm

dominicbeesley wrote:
Thu Oct 10, 2019 3:14 pm
I've yet to think it through thoroughly but this should all be possible with the flexibility of the HostFS idea. The permissions would be down to the "server" - i.e. the perl script at the moment but of course other servers could do what they liked.
Yes, it's always down to the server in any server/client system to determine what it allows to the client. The client must never impose its own beliefs of what it thinks the server's functionality should be. Psychic abilities are not yet something that can be coded.

Code: Select all

$ bbcbasic
PDP11 BBC BASIC IV Version 0.25
(C) Copyright J.G.Harston 1989,2005-2015
>_

dominicbeesley
Posts: 868
Joined: Tue Apr 30, 2013 11:16 am
Contact:

Re: HostFS+TCPIP

Post by dominicbeesley » Fri Oct 18, 2019 11:28 am

jgharston wrote:
Thu Oct 17, 2019 10:26 pm
dominicbeesley wrote:
Sun Oct 13, 2019 1:40 pm
I've made a small start (from scratch) on this. I don't really need Sprow's code much as the stuff already built into the WizNet chip does most of the network stack stuff I'd just like to have more information on the error codes returns in blk%?3 as these don't look to be standard Berkley i.e. &88 for method not implemented?
When I typed up the API for the Wiki, as the command codes where those used in the RISC OS API, I assumed the return codes were also those used in the RISC OS API.
I'm not sure that they are - and more importantly I can't find out what the Risc OS error codes are. Often one needs to know for example is there something returned like EWOULDBLOCK for recv.

I asked Sprow by email:
> Cool, that's enough for me thanks though when you say best translation
> you mean to a BASIC/MOS-ey type number?

Yes. On the beeb the error number space is only 8 bits, so usually I pick the best existing error number match even if that might bend the meaning a bit!
I don't want to nag him to find out what numbers he actually picked. I've given up for now on the disassembly of the LANManager ROM as it was taking a lot of time and not very enlightening re the error numbers. I've started poking around in the ROOL sources too to try and find the numbers they used.

D

User avatar
jgharston
Posts: 3696
Joined: Thu Sep 24, 2009 11:22 am
Location: Whitby/Sheffield
Contact:

Re: HostFS+TCPIP

Post by jgharston » Fri Oct 18, 2019 6:00 pm

dominicbeesley wrote:
Fri Oct 18, 2019 11:28 am
I asked Sprow by email:
> Cool, that's enough for me thanks though when you say best translation
> you mean to a BASIC/MOS-ey type number?
Yes. On the beeb the error number space is only 8 bits, so usually I pick the best existing error number match even if that might bend the meaning a bit!
I don't want to nag him to find out what numbers he actually picked. I've given up for now on the disassembly of the LANManager ROM as it was taking a lot of time and not very enlightening re the error numbers. I've started poking around in the ROOL sources too to try and find the numbers they used.
Ah, so mapping the return codes to error numbers. So something like 'host not found' would return 214, sorta thing.

It rings a bell, I think I might have some notes from the original discussion on the mailing list ages ago.

Code: Select all

$ bbcbasic
PDP11 BBC BASIC IV Version 0.25
(C) Copyright J.G.Harston 1989,2005-2015
>_

dominicbeesley
Posts: 868
Joined: Tue Apr 30, 2013 11:16 am
Contact:

Re: HostFS+TCPIP

Post by dominicbeesley » Sat Oct 19, 2019 8:36 pm

Anything you could find would be a help if only to verify what I'm doing.

Thanks

D

User avatar
jgharston
Posts: 3696
Joined: Thu Sep 24, 2009 11:22 am
Location: Whitby/Sheffield
Contact:

Re: HostFS+TCPIP

Post by jgharston » Sun Oct 20, 2019 12:19 am

dominicbeesley wrote:
Sat Oct 19, 2019 8:36 pm
Anything you could find would be a help if only to verify what I'm doing.
Thanks
Is it for the OSWORD &C0 interface? That should just return the result of the command, these sort of things:

#define socket_EBADF 0x9u /*Bad descriptor*/
#define socket_EAGAIN 0xBu /*No more ports*/
#define socket_EFAULT 0xEu /*Bad address*/
#define socket_EINVAL 0x16u /*Invalid argument*/
#define socket_EWOULDBLOCK 0x23u /*Operation would block*/
etc.

Error numbers are the numbers used when an error is generated when calling a filing system call or command, not when calling an OSWORD call, OSWORD calls return a result with numbering specific to the interface it is accessing. For instance, LOAD "-lanman-jim" could generate an error 214,"Not found", but OSWORD &C0,socket_Accept,0,0 would return result=&0E. (Guessing a bit at specific numbers, I've forgotten where I've put my sample code.)

Code: Select all

$ bbcbasic
PDP11 BBC BASIC IV Version 0.25
(C) Copyright J.G.Harston 1989,2005-2015
>_

dominicbeesley
Posts: 868
Joined: Tue Apr 30, 2013 11:16 am
Contact:

Re: HostFS+TCPIP

Post by dominicbeesley » Sun Oct 20, 2019 9:14 am

That's what i thought. But it doesnt - see Sprows comments earlier. I want to make whatever i do compatible with his but I'm fast thinking that might be impossible if i can't find what error numbers he used. So far I've been returning an Exxx type number but then any software that relied on those wouldn't work on the LANManager ROM.

The only error number I've managed to greet from the disassembly is &88 for function not implemented which doesn't seem to correspond to any known sockets or BBC error number

Post Reply