Viewdata Service

Got a programming project in mind? Tell everyone about it!
User avatar
John_Newcombe
Posts: 55
Joined: Tue Nov 28, 2017 9:19 am
Contact:

Re: Viewdata Service

Post by John_Newcombe » Fri Jul 20, 2018 8:24 am

Ok, I am not sure I fully understand but if you are asking whether T* could handle a telnet neg then yes it probably could. I have a telnet parser that could be used. Also, I wonder if things would be simpler if we used some form of in band method. E.g. by having enhanced commands etc. *time# *date# etc that perhaps updated the system message line or logo or something.

At the begining of each fully rendered page, the first char code is the clearscreen, currently 2 are sent as it helps with emulators. What about squeezing some meta data in between them, that would be so simple to implement.

J

User avatar
Elminster
Posts: 2589
Joined: Wed Jun 20, 2012 8:09 am
Location: Essex, UK
Contact:

Re: Viewdata Service

Post by Elminster » Fri Jul 20, 2018 8:29 am

Telnet Server. Hmmmm. I could then use the Master Telent Client on it.

viewtopic.php?f=53&t=13228&hilit=btelnet

User avatar
John_Newcombe
Posts: 55
Joined: Tue Nov 28, 2017 9:19 am
Contact:

Re: Viewdata Service

Post by John_Newcombe » Fri Jul 20, 2018 8:51 am

I often use telnet to test the service. I could re-enable the command to set the client to char mode rather than line mode if you are up for testing it.

User avatar
Elminster
Posts: 2589
Joined: Wed Jun 20, 2012 8:09 am
Location: Essex, UK
Contact:

Re: Viewdata Service

Post by Elminster » Fri Jul 20, 2018 9:20 am

Well it could be a carrot that gets me to finish the btelnet program.

Although what use to drive me nuts is that none of the emulators support the Sprow Ethernet card, so has to be tested on real hardware, and the program is to large to easily edit on real hardware, so continually copying them between machines.

But realistically It will be a couple of weeks before I get time to do anything, other non Acorn commitments..... :cry:

User avatar
John_Newcombe
Posts: 55
Joined: Tue Nov 28, 2017 9:19 am
Contact:

Re: Viewdata Service

Post by John_Newcombe » Fri Jul 20, 2018 10:32 am

I have now added some date/time meta-data to the header line of each frame (apart from telesoftware download frames).

The meta-data (UTC Date and time only at the moment) sits between the first character 0x0C and a subsequent 0x0C. The char 0x0C is a clear screen character which means that normal users of the system will not see this data, however, clients will.

A header row will look something like the following:

"\x0c20180720T1005Z\x0c\0x11\x1b\x42T\x1b\x41E\x1b\x46L\x1b\x44S\x1b\x47T\x1b\x45A\x1b\x43R ..."

The date and time in UTC is shown as follows 20180720T1007Z, meaning 10:05, 20 July 2018 UTC.

PLEASE NOTE, this is only appears on the initial page.
Last edited by John_Newcombe on Sat Aug 11, 2018 8:19 am, edited 3 times in total.

User avatar
John_Newcombe
Posts: 55
Joined: Tue Nov 28, 2017 9:19 am
Contact:

Re: Viewdata Service

Post by John_Newcombe » Fri Jul 20, 2018 10:40 am

Elminster wrote:
Fri Jul 20, 2018 9:20 am
Well it could be a carrot that gets me to finish the btelnet program.

Although what use to drive me nuts is that none of the emulators support the Sprow Ethernet card, so has to be tested on real hardware, and the program is to large to easily edit on real hardware, so continually copying them between machines.

But realistically It will be a couple of weeks before I get time to do anything, other non Acorn commitments..... :cry:
I have used BeebEm in Wine on a Mac by setting the emulators Comm port to the IP/Port in the 'RS232 Destination Settings'. But Richard Russells BBC Basic example gets preference now. The only issue I have with the BBC Basic example client is that it is too fast and the 'Page not found' messages etc. are overwritten before they can be read. It is a great example though and I can easily make a few changes to it, however, I tend to do the real testing on a BBC B.

Adding the Telnet negotiation to set the client works fine, however the Telnet parser I have to handle the responses has a bug. I will fix this over the next week or so, hopefully it will be ready for you when you have time.

J

User avatar
John_Newcombe
Posts: 55
Joined: Tue Nov 28, 2017 9:19 am
Contact:

Re: Viewdata Service

Post by John_Newcombe » Fri Jul 20, 2018 6:40 pm

John_Newcombe wrote:
Fri Jul 20, 2018 10:32 am
I have now added some date/time meta-data to the header line of each frame (apart from telesoftware download frames).
I have had second thoughts about this being on all pages. Maybe just the welcome page. I can be seen at 2400 baud using a BBC with Comstar and it is a little distracting. Will consider it some more.

colonel32
Posts: 48
Joined: Wed Jan 18, 2017 7:59 pm
Location: USA
Contact:

Re: Viewdata Service

Post by colonel32 » Fri Jul 20, 2018 7:04 pm

I'm still considering what you wrote John, and haven't had time to reply in detail.

I agree whatever you or I do should not be visible in COMMSTAR or at slow speeds. I haven't got telnet option negotiation fully working yet, but this was the reason I preferred the out-of-band idea. Specifically, IAC WILL/DO NEW-ENVIRON IS USER lets you send arbritrary key/value pairs in either direction. There's no requirement to send the full set of vars each time, so individual short vars can flow in a call/response way.

I wasn't really asking you anything in my last post, more describing my plans in the hope that it woulds start an interesting discussion. Which it has!

User avatar
Elminster
Posts: 2589
Joined: Wed Jun 20, 2012 8:09 am
Location: Essex, UK
Contact:

Re: Viewdata Service

Post by Elminster » Fri Jul 20, 2018 7:18 pm

John_Newcombe wrote:
Fri Jul 20, 2018 10:40 am

I have used BeebEm in Wine on a Mac by setting the emulators Comm port to the IP/Port in the 'RS232 Destination Settings'. But Richard Russells BBC Basic example gets preference now. The only issue I have with the BBC Basic example client is that it is too fast and the 'Page not found' messages etc. are overwritten before they can be read. It is a great example though and I can easily make a few changes to it, however, I tend to do the real testing on a BBC B.

Adding the Telnet negotiation to set the client works fine, however the Telnet parser I have to handle the responses has a bug. I will fix this over the next week or so, hopefully it will be ready for you when you have time.

J
Btelnet talks tcp/ip it doesn’t use rs232.

Beebem runs native on Mac but several features missing, I have that and B2 but generally use b-em.

User avatar
John_Newcombe
Posts: 55
Joined: Tue Nov 28, 2017 9:19 am
Contact:

Re: Viewdata Service

Post by John_Newcombe » Wed Jul 25, 2018 11:08 pm

I like the idea of the telnet approach, but equally something that could be used by all clients including none tcp and none telnet could be useful. Particularly as I am working on a serial server that will allow v23 modem connection using a sip/voip service. :)

In the meantime the date and time is now sent on the welcome page only. It was too intrusive on all pages.

Only on ACTON at the moment, (port 6504).

User avatar
Elminster
Posts: 2589
Joined: Wed Jun 20, 2012 8:09 am
Location: Essex, UK
Contact:

Re: Viewdata Service

Post by Elminster » Thu Jul 26, 2018 7:22 am

Are you talking to a voip service directly or running some thing like a local asterisk server? A local PBX would be more flexible but more faff. Not used it for a decade but I did setup and admin a Asterisk for an office of 10 using combination of PSTN and voip. Never tried using it for viewdata dial in though :)

User avatar
John_Newcombe
Posts: 55
Joined: Tue Nov 28, 2017 9:19 am
Contact:

Re: Viewdata Service

Post by John_Newcombe » Fri Jul 27, 2018 3:06 am

At the moment I have a sip with sipgate and a Grsnstream adapter hooked up to a v23 modem.

Darren Storer got me on to this. Asterisk, could be an option as there is a git hub project for a virtual v23 modem. But would need some time (or help) to set this all up. I guess it would support multiple connections though.

First things first. Just adding live weather for all 27000 cities and towns in the UK at the moment... :)
Last edited by John_Newcombe on Fri Jul 27, 2018 3:11 am, edited 1 time in total.

User avatar
Elminster
Posts: 2589
Joined: Wed Jun 20, 2012 8:09 am
Location: Essex, UK
Contact:

Re: Viewdata Service

Post by Elminster » Fri Jul 27, 2018 2:44 pm

I did a lot of asterisk but not touched it for a 10 years, so must have changed, had both normal phone lines and voip going to 10 handsets okay. Was early days then, used to get a lot of echo, that would probably drive a modem nuts.

I still use the same voip provider a decade later but just going through a Cisco/linksys/sipura (keep getting bought box) and a soft phone. sip with gradwell.

I haven’t looked at using modems over voip, although no reason I couldn’t.

Soruk
Posts: 91
Joined: Mon Jul 09, 2018 10:31 am
Location: Basingstoke, Hampshire
Contact:

Re: Viewdata Service

Post by Soruk » Fri Jul 27, 2018 9:00 pm

Just as a demo (and for the sheer hell of it) - I've implemented a quick and dirty (and very basic) client for Matrix Brandy BASIC. It's included as part of the package from version 1.21.6.
Last edited by Soruk on Fri Jul 27, 2018 9:03 pm, edited 1 time in total.

User avatar
Richard Russell
Posts: 353
Joined: Sun Feb 27, 2011 10:35 am
Location: Downham Market, Norfolk
Contact:

Re: Viewdata Service

Post by Richard Russell » Sun Jul 29, 2018 1:13 pm

Soruk wrote:
Fri Jul 27, 2018 9:00 pm
Just as a demo (and for the sheer hell of it) - I've implemented a quick and dirty (and very basic) client for Matrix Brandy BASIC.
Any particular reason you didn't use the BBCSDL client as a starting point? It no doubt uses some language extensions not available in Brandy, but since it runs without modification on Windows, Linux, Mac OS, Raspberry Pi, Android and iOS the BBC BASIC code is fairly generic.

Richard.

P.S. As I've noted elsewhere, the telesoftware download capability in my client is currently broken following a change at Telstar. I'll fix this when I next update it.

User avatar
Richard Russell
Posts: 353
Joined: Sun Feb 27, 2011 10:35 am
Location: Downham Market, Norfolk
Contact:

Re: Viewdata Service

Post by Richard Russell » Sun Jul 29, 2018 3:51 pm

I see that the gateway from Telstar to Colossal Cave Adventure is up, but a rather important feature seems to have been forgotten: when it prompts for user input it fails to enable the caret (text cursor). As a result the mobile (Android and iOS) versions of my client don't display the on-screen touch keyboard, making user interaction impossible. :(

I would appreciate it if you could address this problem and enable the caret (character code 0x11) when user input is required and disable it (character code 0x14) when it's not. This will cause the mobile clients to enable and disable the on-screen keyboard as required. Incidentally this feature works as expected in CCl4.

Richard.

Soruk
Posts: 91
Joined: Mon Jul 09, 2018 10:31 am
Location: Basingstoke, Hampshire
Contact:

Re: Viewdata Service

Post by Soruk » Sun Jul 29, 2018 6:19 pm

Richard Russell wrote:
Sun Jul 29, 2018 1:13 pm
Soruk wrote:
Fri Jul 27, 2018 9:00 pm
Just as a demo (and for the sheer hell of it) - I've implemented a quick and dirty (and very basic) client for Matrix Brandy BASIC.
Any particular reason you didn't use the BBCSDL client as a starting point? It no doubt uses some language extensions not available in Brandy, but since it runs without modification on Windows, Linux, Mac OS, Raspberry Pi, Android and iOS the BBC BASIC code is fairly generic.

Richard.

P.S. As I've noted elsewhere, the telesoftware download capability in my client is currently broken following a change at Telstar. I'll fix this when I next update it.
I did have a quick look - but as the SWI interface is completely unimplemented in Brandy on non-RISCOS builds, I started from scratch for a network interface, deciding with extening OPENUP so BGET# and BPUT# read and write the network channel instead of a file.

As for the Videotex emulation, I started from scratch, pretty much making it up as I went along having never seen any documentation for the standard. Not to mention, if I took yours and just adjusted it for Brandy's network interface that would be rather going against your copyright!
Last edited by Soruk on Sun Jul 29, 2018 8:53 pm, edited 1 time in total.

User avatar
Richard Russell
Posts: 353
Joined: Sun Feb 27, 2011 10:35 am
Location: Downham Market, Norfolk
Contact:

Re: Viewdata Service

Post by Richard Russell » Sun Jul 29, 2018 9:42 pm

Soruk wrote:
Sun Jul 29, 2018 6:19 pm
the SWI interface is completely unimplemented in Brandy on non-RISCOS builds
If Brandy did not support SYS on any platform it would at least be consistent (if very limiting) but to support it on RISC OS but not (say) on Windows seems highly unhelpful! In any case my client doesn't directly use SYS to access the network, rather it calls routines in a BASIC library. In principle a Brandy client could do the same and thereby offload OS differences to the library.
Not to mention, if I took yours and just adjusted it for Brandy's network interface that would be rather going against your copyright!
My EULA states "The supplied example programs and libraries remain the Copyright © 2001-2018 of R.T.Russell and others, but you may incorporate them in whole or in part within your own programs so long as any copyright notice is retained", so you are fully entitled to adapt my code. After all, the example programs supplied with 'my' BBC BASICs are as much intended to demonstrate coding techniques, and what is possible, as they are to perform a specific function. It would be pretty daft to restrict how people can use them.

Richard.

Soruk
Posts: 91
Joined: Mon Jul 09, 2018 10:31 am
Location: Basingstoke, Hampshire
Contact:

Re: Viewdata Service

Post by Soruk » Mon Jul 30, 2018 12:19 pm

Richard Russell wrote:
Sun Jul 29, 2018 9:42 pm
Soruk wrote:
Sun Jul 29, 2018 6:19 pm
the SWI interface is completely unimplemented in Brandy on non-RISCOS builds
If Brandy did not support SYS on any platform it would at least be consistent (if very limiting) but to support it on RISC OS but not (say) on Windows seems highly unhelpful! In any case my client doesn't directly use SYS to access the network, rather it calls routines in a BASIC library. In principle a Brandy client could do the same and thereby offload OS differences to the library.
That would, of course, make logical sense. My little Telstar client was just something quickly thrown together as a functioning demo of Brandy makig network connections.
Richard Russell wrote:
Sun Jul 29, 2018 9:42 pm
Not to mention, if I took yours and just adjusted it for Brandy's network interface that would be rather going against your copyright!
My EULA states "The supplied example programs and libraries remain the Copyright © 2001-2018 of R.T.Russell and others, but you may incorporate them in whole or in part within your own programs so long as any copyright notice is retained", so you are fully entitled to adapt my code. After all, the example programs supplied with 'my' BBC BASICs are as much intended to demonstrate coding techniques, and what is possible, as they are to perform a specific function. It would be pretty daft to restrict how people can use them.
Thank you for the clarification. I have since been looking through your version, and while some things are confusing me a bit (I've never seen variables of the type varname& before) I have used it as an guide to fix a few oddities, and in due course might even implement Telesoftware downloading! But, it certainly appears to behave itself now on both Telstar and Tetrachloromethane - and helped me track down some bugs in the new networking code (which was its other reason for existing).

User avatar
Richard Russell
Posts: 353
Joined: Sun Feb 27, 2011 10:35 am
Location: Downham Market, Norfolk
Contact:

Re: Viewdata Service

Post by Richard Russell » Mon Jul 30, 2018 3:57 pm

Soruk wrote:
Mon Jul 30, 2018 12:19 pm
I've never seen variables of the type varname& before
If I'm trying to maximise compatibility with Acorn's BBC BASICs (and Brandy) I avoid language extensions that they don't support, but it wasn't a primary goal in this case. I find the additional variable types in my versions very useful:

Code: Select all

varname& = 234 : REM Unsigned 8-bit variable (0-255)
varname# = 123.456 : REM 64-bit float ('double')
varname%% = 1234567890123456789 : REM Signed 64-bit integer variable
These are rarely essential; in my versions a regular suffixless variable like 'varname' is capable of containing any numeric value they can with no loss of precision. But they can give a useful reduction of memory usage, e.g. DIM array&(10000) requires only 10001 bytes of memory, and occasionally it is necessary to force a specific type (for example when passing a value to an API function using SYS). And of course they are vital as structure members, if one needs to control the layout in memory precisely.

Because all versions of BBC BASIC support 8-bit indirection (?address) they already have the ability to handle 8-bit unsigned values internally, so I've always been surprised that Sophie didn't provide it as a variable type. Apart from (I would expect) a tiny amount of extra code to handle arrays of that type, it should have been achievable with very little effort. Of course you may argue that I should have suggested it to her when BASIC 5 was proposed, but it never occurred to me at the time.

Sorry for the off-topic post.

Richard.

Soruk
Posts: 91
Joined: Mon Jul 09, 2018 10:31 am
Location: Basingstoke, Hampshire
Contact:

Re: Viewdata Service

Post by Soruk » Mon Jul 30, 2018 4:27 pm

My last post on the topic here too - just to say thank you, that gives a lot of clarity to understanding what's happening.

User avatar
John_Newcombe
Posts: 55
Joined: Tue Nov 28, 2017 9:19 am
Contact:

Re: Viewdata Service

Post by John_Newcombe » Mon Jul 30, 2018 10:19 pm

Richard Russell wrote:
Sun Jul 29, 2018 3:51 pm

I would appreciate it if you could address this problem and enable the caret (character code 0x11) when user input is required and disable it (character code 0x14) when it's not. ...

Richard.
I will sort that out, appologies for missing it. Am away this week but will sort it when I get back.

User avatar
John_Newcombe
Posts: 55
Joined: Tue Nov 28, 2017 9:19 am
Contact:

Re: Viewdata Service

Post by John_Newcombe » Mon Jul 30, 2018 10:34 pm

Richard Russell wrote:
Sun Jul 29, 2018 1:13 pm
P.S. As I've noted elsewhere, the telesoftware download capability in my client is currently broken following a change at Telstar. I'll fix this when I next update it.
What was the change that broke the client?
J

User avatar
Richard Russell
Posts: 353
Joined: Sun Feb 27, 2011 10:35 am
Location: Downham Market, Norfolk
Contact:

Re: Viewdata Service

Post by Richard Russell » Mon Jul 30, 2018 10:49 pm

John_Newcombe wrote:
Mon Jul 30, 2018 10:34 pm
What was the change that broke the client?
The subpage letter specified in the |G record used to be off-by-one (for example it said 'b' when the actual page was 'c' and so on). You have corrected that, which is good, but unfortunately I had inadvertently coded my client to depend on the original error. :oops:

Richard.

User avatar
John_Newcombe
Posts: 55
Joined: Tue Nov 28, 2017 9:19 am
Contact:

Re: Viewdata Service

Post by John_Newcombe » Tue Jul 31, 2018 7:23 am

Richard Russell wrote:
Sun Jul 29, 2018 3:51 pm

I would appreciate it if you could address this problem and enable the caret (character code 0x11) when user input is required and disable it (character code 0x14) when it's not. ...

Richard.
I have added the caret with a quick hack in live. This has been fixed in the main branch also.

Now that I finally have the Telesoftware spec, i can see that there is one other change required in the Telesoftware header. I will make the change when I return at the end of the week and let you know the details.

J
Last edited by John_Newcombe on Tue Jul 31, 2018 7:23 am, edited 1 time in total.

User avatar
Richard Russell
Posts: 353
Joined: Sun Feb 27, 2011 10:35 am
Location: Downham Market, Norfolk
Contact:

Re: Viewdata Service

Post by Richard Russell » Tue Jul 31, 2018 9:18 am

John_Newcombe wrote:
Tue Jul 31, 2018 7:23 am
I have added the caret with a quick hack in live. This has been fixed in the main branch also.
Thanks. I can confirm that Colossal Cave Adventure is now playable on Android and iOS.
Now that I finally have the Telesoftware spec, i can see that there is one other change required in the Telesoftware header. I will make the change when I return at the end of the week and let you know the details.
One reason why the change you made broke my client is that the 'header' frame, unlike all the others, does not contain |G and |I sequences. This means that my client doesn't know what subframe the main telesoftware frames begin on, and it makes a false assumption based on the earlier 'off by one' error. It would be more consistent to include |G and |I sequences on the header frame as well; it's possible that this would make my client work without alteration, but I'll fix it anyway when I next update BBCSDL.

Richard.

User avatar
John_Newcombe
Posts: 55
Joined: Tue Nov 28, 2017 9:19 am
Contact:

Re: Viewdata Service

Post by John_Newcombe » Tue Jul 31, 2018 10:42 am

That is exactly the change I want to make. :D

The docs I had were pre these sequences and the code I disassembled did not add them to the header. Now that I have the original docs, I will sort this at the weekend when I am back.

J

https://glasstty.com/wiki/index.php/Fil ... ow_Res.pdf
Last edited by John_Newcombe on Tue Jul 31, 2018 10:50 am, edited 4 times in total.

User avatar
John_Newcombe
Posts: 55
Joined: Tue Nov 28, 2017 9:19 am
Contact:

Re: Viewdata Service

Post by John_Newcombe » Thu Aug 02, 2018 5:14 pm

Richard Russell wrote:
Tue Jul 31, 2018 9:18 am
One reason why the change you made broke my client is that the 'header' frame, unlike all the others, does not contain |G and |I sequences.
Another 'hot fix' to live to add the |G|I sequences to the header frame. These changes have been made to the main branch also.

colonel32
Posts: 48
Joined: Wed Jan 18, 2017 7:59 pm
Location: USA
Contact:

Re: Viewdata Service

Post by colonel32 » Thu Aug 02, 2018 5:58 pm

Telesoftware works for me in Richard's client now. Nice one John.

Good find on the telesoftware protocol document, too.
Last edited by colonel32 on Thu Aug 02, 2018 5:58 pm, edited 1 time in total.

User avatar
John_Newcombe
Posts: 55
Joined: Tue Nov 28, 2017 9:19 am
Contact:

Re: Viewdata Service

Post by John_Newcombe » Fri Aug 03, 2018 3:12 pm

colonel32 wrote:
Thu Aug 02, 2018 5:58 pm
Good find on the telesoftware protocol document, too.
Darren Storer is to be thanked for the document, not sure where he found it but I am pleased he did. :)
Last edited by John_Newcombe on Fri Aug 03, 2018 3:12 pm, edited 1 time in total.

Post Reply