Viewdata Service

Got a programming project in mind? Tell everyone about it!
User avatar
John_Newcombe
Posts: 69
Joined: Tue Nov 28, 2017 9:19 am
Contact:

Viewdata Service

Post by John_Newcombe » Mon Apr 16, 2018 9:33 pm

Hi All, thanks for welcoming me to the group. Whilst I have only recently acquired a BBC (Issue 7), I have a history with them that spans back to the day they were launched.

I am interested in investigating what can be done to present Viewdata pages over the RS432 port from some sort of server. The aim would be to perhaps create a simple networked server (Pi/Arduino etc.) that could automate the generation of pages based on a news feed of some sort.

I would not be looking to use a Prestel adapter or other modem, however, I would like to use an existing 'Prestel' client program for example Comstar or the Acornsoft ROM. The Acornsoft PRESTEL ROM would require that I simulate the presence of an adapter, but how hard can it be, right?

I cannot find anything out there that supplies a Viewdata services, I would love to hear about anything that exists. I am aware of the Vbit project though.

I just love the idea of switching on my BBC Micro and checking the news headlines. I would be really interested in your thoughts.

User avatar
BigEd
Posts: 2092
Joined: Sun Jan 24, 2010 10:24 am
Location: West
Contact:

Re: Viewdata Service

Post by BigEd » Tue Apr 17, 2018 6:51 am

Welcome! An interesting idea. It certainly should be possible. This blog post
https://www.retrocomputers.online/2013/ ... -viewdata/
makes a successful connection to a particular viewdata service, via a Raspberry Pi and a serial port. That service is still live but I don't know if it has any news feed.

This post
https://medium.com/@cq94/get-your-minit ... 8c42f1ea17
directs us to the French Wikipedia article on Minitel microservers
https://fr.wikipedia.org/wiki/Micro-serveurs_Minitel
but of course you want something local, not dial-up, and with source code, and with the hope of making a gateway to the web (or to an RSS feed...)

This project
https://github.com/cquest/pynitel
is a Minitel server in python - a server of local content, not a web gateway. And Minitel isn't quite exactly the same thing.

User avatar
vanpeebles
Posts: 470
Joined: Wed Nov 28, 2012 10:01 am
Location: UK
Contact:

Re: Viewdata Service

Post by vanpeebles » Tue Apr 17, 2018 7:20 am

Some years ago, I used a viewdata service called silicon village. I had email via my speccy, and would email my pc owning friends, much to their amusement. :lol:

User avatar
BigEd
Posts: 2092
Joined: Sun Jan 24, 2010 10:24 am
Location: West
Contact:

Re: Viewdata Service

Post by BigEd » Tue Apr 17, 2018 7:22 am

That could be worth a bit of an investigation!
https://www.viewdata.org.uk/index.php?c ... troduction

It leads us to this thread:
CommunITel Viewdata
viewtopic.php?t=9625

guesser
Posts: 210
Joined: Mon Jun 26, 2006 9:21 pm

Re: Viewdata Service

Post by guesser » Tue Apr 17, 2018 11:43 am

Some members of York Hackspace are working on a project to emulate a v23 modem and a viewdata server to breath life back into a ZX Spectrum modem.
They have a few posts on the hackspace blog and a couple of project on github for the modem and server. Most of the info is about the modem stuff and physical layer at the moment.
A web based teletext editor which can export as Mode 7 screen memory: https://zxnet.co.uk/teletext/editor

User avatar
vanpeebles
Posts: 470
Joined: Wed Nov 28, 2012 10:01 am
Location: UK
Contact:

Re: Viewdata Service

Post by vanpeebles » Wed Apr 18, 2018 11:57 am

I think I would of been on it, mid 90s, I still have the letters somewhere. It had kind of been left alone even then. You wrote and got a demo time limited account, and were meant to pay a sub once it ran out, but the demo account just kept going.

User avatar
John_Newcombe
Posts: 69
Joined: Tue Nov 28, 2017 9:19 am
Contact:

Re: Viewdata Service

Post by John_Newcombe » Thu Apr 19, 2018 5:33 pm

Many thanks for the replies, appreciated. I had great fun touring the the fish.cc4.org site using a BBC Micro with Comstar and a WiModem232. As an aside, does anyone no of any Viewdata server software thats in the public domain?

I had a play around and knocked up a Python based Viewdata service on a Digital Ocean server I have. This simply presents a static page at the moment (see below). I also have a piece of code that manipulates a BBC RSS feed to create news pages, it is not working too well yet but with some work it could be OK.

I am not sure where this is going, but I thought I would give it the name TELSTAR for now as this was the name of a small bulletin board I ran back in 83. It was an anagram of my friend and co-author T Slater.

The image below is a representation of my static page which was created in a Teletext editor and then re-coded into Prestel Esc sequences. The software I used was Commstar as I can set it to Prestel mode with 1200 full duplex and 8 data/1 stop. This allows me to use the WiModem232 to telnet to the service.

If I take it any further, I will keep you posted.

Image
Last edited by John_Newcombe on Thu Jun 07, 2018 1:57 pm, edited 1 time in total.

User avatar
BigEd
Posts: 2092
Joined: Sun Jan 24, 2010 10:24 am
Location: West
Contact:

Re: Viewdata Service

Post by BigEd » Thu Apr 19, 2018 7:05 pm

Splendid!

I found some Ruby code here but probably you're already ahead with your Python code.

User avatar
John_Newcombe
Posts: 69
Joined: Tue Nov 28, 2017 9:19 am
Contact:

Re: Viewdata Service

Post by John_Newcombe » Fri Apr 20, 2018 6:01 pm

Here is the test page should anyone be interested.

Server: glasstty.com
Port: 6502

Using Commstar set the emulation to Prestel set Tx/Rx to match your telnet modem e.g. 1200/1200 and (for now at least) set 8 data and 1 stop bit. Prestel, was 7 bit even but as these things often cannot be set on telnet modems, I will implement it in software at some point down the line.

Enjoy!

Image
Last edited by John_Newcombe on Thu Jun 07, 2018 1:57 pm, edited 1 time in total.

User avatar
Lardo Boffin
Posts: 1278
Joined: Thu Aug 06, 2015 6:47 am
Contact:

Re: Viewdata Service

Post by Lardo Boffin » Fri Apr 20, 2018 7:46 pm

Hello John

Random question but are you a big fan of Apple stuff?

Lardo
BBC model B 32k issue 4, 16k sideways RAM, Watford 12 ROM board, Viglen twin 40/80 5.25" discs, Acorn 6502 coproc
BBC model B 32k issue 7, turboMMC, Opus Challenger 3 512k, Pi 3 coproc
BBC Master, Datacentre + HDD, pi co-proc

User avatar
John_Newcombe
Posts: 69
Joined: Tue Nov 28, 2017 9:19 am
Contact:

Re: Viewdata Service

Post by John_Newcombe » Fri Apr 20, 2018 11:23 pm

In terms of retro, I am a big fan of the Apple II, I have several of them. I am not much of a gamer, I tend to use both the Beeb and the A2s for assembler stuff and silly projects like this Viewdata one. I am clearly drawn towards serial comms as the A2 project I am working on is a linux terminal for my ARM cluster.

Why do you ask?

User avatar
John_Newcombe
Posts: 69
Joined: Tue Nov 28, 2017 9:19 am
Contact:

Re: Viewdata Service

Post by John_Newcombe » Tue Apr 24, 2018 11:26 am

I now have a working service, all written is Python, which is remarkable in itself as I only learn't the language this week. :shock:

All news sections are automatically updated from BBC RSS feeds and the frames are automatically built and pickled every 10mins (or will be once the cron job has been set up).

Static pages have been created from some simple templates so its a little sparse at the moment, but it's a start. Images shown are using BeebEm on a Mac with 'socat' providing the link between the emulated serial port and the viewdata server. Works better on the BBC Itself but screen shots are harder.

Anyone know anything about how to implement downloading of software?

Let me know what you think?

glasstty.com:6502

Enjoy!

Image
Image
Last edited by John_Newcombe on Thu Jun 07, 2018 1:57 pm, edited 1 time in total.

User avatar
vanpeebles
Posts: 470
Joined: Wed Nov 28, 2012 10:01 am
Location: UK
Contact:

Re: Viewdata Service

Post by vanpeebles » Tue Apr 24, 2018 11:33 am

I downloaded a terminal program for the speccy via viewdata. You had to watch it in real time, it would slowly fill the screen up with "junk" per page, and keep going so like page 1/15 then 2/15 etc Once it got to the end, you were prompted to save it to tape. I can't remember much about that bit. I remember all the pages of data though.

User avatar
BigEd
Posts: 2092
Joined: Sun Jan 24, 2010 10:24 am
Location: West
Contact:

Re: Viewdata Service

Post by BigEd » Tue Apr 24, 2018 11:41 am

Great progress!

OneSwitch
Posts: 91
Joined: Tue Nov 22, 2011 5:50 pm
Contact:

Re: Viewdata Service

Post by OneSwitch » Tue Apr 24, 2018 6:02 pm

Hugely impressive. Be amazing to get a mix of new and original Prestel content up for public access.

User avatar
John_Newcombe
Posts: 69
Joined: Tue Nov 28, 2017 9:19 am
Contact:

Re: Viewdata Service

Post by John_Newcombe » Tue Apr 24, 2018 11:28 pm

I may create a teletext to Prestel converter so that some of the art and pages created at edit.tf etc, can be hosted. In the mean time a FTSE section has been added.

As an aside, I added some Telnet negotiation stuff to put Telnet clients into character mode, good for testing etc. There are one or two issues to resolve and there is still loads to do to make it more useable.

glasstty.com:6502

Image
Last edited by John_Newcombe on Thu Jun 07, 2018 5:09 pm, edited 1 time in total.

User avatar
John_Newcombe
Posts: 69
Joined: Tue Nov 28, 2017 9:19 am
Contact:

Re: Viewdata Service

Post by John_Newcombe » Wed Apr 25, 2018 6:12 pm

Image
Last edited by John_Newcombe on Thu Jun 07, 2018 5:09 pm, edited 1 time in total.

User avatar
Lardo Boffin
Posts: 1278
Joined: Thu Aug 06, 2015 6:47 am
Contact:

Re: Viewdata Service

Post by Lardo Boffin » Wed Apr 25, 2018 6:22 pm

!!!!! I was just reading that post on my iPhone! I so want to be able to do that on a beeb.
BBC model B 32k issue 4, 16k sideways RAM, Watford 12 ROM board, Viglen twin 40/80 5.25" discs, Acorn 6502 coproc
BBC model B 32k issue 7, turboMMC, Opus Challenger 3 512k, Pi 3 coproc
BBC Master, Datacentre + HDD, pi co-proc

User avatar
lcww1
Posts: 280
Joined: Wed Mar 15, 2017 11:16 pm
Location: Moon Base Alpha
Contact:

Re: Viewdata Service

Post by lcww1 » Wed Apr 25, 2018 6:54 pm

@John_Newcombe - that is magnificent! =D> =D> :D

Bat
Posts: 165
Joined: Tue Mar 13, 2018 1:17 pm
Contact:

Re: Viewdata Service

Post by Bat » Wed Apr 25, 2018 7:19 pm

Hi,
That's a huge achievement in a short period of time.
How do the rest of us get to use it though?
Excuse my ignorance but I'm still quite new here :oops:
Cheers,
Gavin :)

User avatar
John_Newcombe
Posts: 69
Joined: Tue Nov 28, 2017 9:19 am
Contact:

Re: Viewdata Service

Post by John_Newcombe » Wed Apr 25, 2018 10:05 pm

Using the service requires either a BBC with a telnet modem (see below) or an emulator some software and much patience. The BBC route is easier and most reliable but requires a modem such as the item here https://www.cbmstuff.com/proddetail.php?prod=WiModem232. The guy will post to UK and in my experience Customs don't seem to get involved which keeps the price reasonable. I do have a 'home-brew' telnet modem running on an Arduino with an ethernet shield built for another project, but it really needs to support character mode and at the moment it is stuck on line mode. Once I have this working I will post the details on https://glasstty.com.

I think the longer term solution is to build a software based 'Prestel Adapter' that can deliver results on an emulator such as BeebEm. However if you really want to get frustrated, the following is a description of how I have used BeebEm on my Mac without such a product.

The process works using the latest Windows version (run under Wine) and the version ported for Mac. I have only tried this on my Mac it should work on Linux but for Windows, I am afraid you are on your own, however, the following may help. The idea is as follows:

BeebEm has the ability to connect the BBC Emulated RS423 as a client to a TCP Port (Comm menu). This means that if I run software such as 'socat' on my Mac to listen to that port and direct it to a terminal session on the Mac, I can control my Mac from the Emulated BBC. Once I have control I can telnet to any Viewdata service I fancy and away we go.

So, assuming you have researched 'socat' and installed it with your favourite installer ('Ports' or 'Brew' for example), you can run the following command...

socat tcp-l:25232,reuseaddr,fork exec:/bin/bash,pty,setsid,setpgid,stderr,ctt

This will listen on TCP port 25232 for incoming data and present it to a bash shell (command line)

On BeebEm set the RS423 port to <ip>:port 25232 where <ip> is you own computers IP Address (do not use 127.0.0.1). This connects BeebEm to the instance of socat using TCP port 25232. With me so far?

Run ComStar in ROM using *C* select Prestel Mode (shift 3), press I to set the speed to 1200/1200 and press 5 to set 8 dat 1 stop. Press return to get back at ComStars menu, then press C for Chat. Then press Ctrl M (the return key in Prestel mode sends a # not a CR, Ctrl M sends a CR). At this point you should see the command prompt. Simply run the following command followed by Ctrl M, ignoring the fact that the typed characters are not displayed correctly.

telnet glasstty.com 6502

You may be lucky, you may get corrupted screens you may not see all the screens returned. I have no idea why it is so bad but occasionally its enough to experience it and get some screen shots.

I really hope this helps, Enjoy!
Last edited by John_Newcombe on Thu Apr 26, 2018 6:10 pm, edited 1 time in total.

User avatar
John_Newcombe
Posts: 69
Joined: Tue Nov 28, 2017 9:19 am
Contact:

Re: Viewdata Service

Post by John_Newcombe » Wed Apr 25, 2018 10:07 pm

Thanks to Daniel Jameson I now have the Telesoftware spec and have started building an encoder for the service. I am hoping I can get software downloads working soon.

User avatar
vanpeebles
Posts: 470
Joined: Wed Nov 28, 2012 10:01 am
Location: UK
Contact:

Re: Viewdata Service

Post by vanpeebles » Thu Apr 26, 2018 9:12 am

Will it get to the point where people can ring your house with modems and connect like the old days? :D

User avatar
hoglet
Posts: 7501
Joined: Sat Oct 13, 2012 6:21 pm
Location: Bristol
Contact:

Re: Viewdata Service

Post by hoglet » Thu Apr 26, 2018 10:57 am

John,

This is very nice indeed.

I was able to connect this morning from BeebEm on Windows 7 using a gateway program called tcpser.exe:
https://www.irrelevant.com/bbc/TestBeebEmComms.rar

You need two files from this archive:
- tcpser.exe
- cygwin1.dll

Copy these into a new directory (I created one called tcpser under BeebEm)

From a Windows Command Prompt, cd to this directory and run:

Code: Select all

tcpser.exe -v 25232 -p 6400 -l 4 -s 9600
The output should be:

Code: Select all

2018-04-26 11:38:25:7536648:INFO:Server socket bound to port
2018-04-26 11:38:25:7536648:INFO:Server socket listening for connections
2018-04-26 11:38:25:7536648:INFO:Opening ip232 device
2018-04-26 11:38:25:7536648:INFO:Server socket bound to port
2018-04-26 11:38:25:7536648:INFO:Server socket listening for connections
2018-04-26 11:38:25:7536648:INFO:ip232 device configured
2018-04-26 11:38:25:7543896:INFO:Control Lines: DSR:1 DCD:0 CTS:1
2018-04-26 11:38:25:7543896:INFO:Disconnecting modem
2018-04-26 11:38:25:7543896:INFO:Disconnecting
2018-04-26 11:38:25:7543896:INFO:Control Lines: DSR:1 DCD:0 CTS:1
Here are some notes for then setting up BeebEm:

Unzip Commstar 3.52 for the Master into BeebEm/BeebFile/M128

Edit BeebEm/roms.cfg and add M128/COMSTAR352.rom as ROM 8

Code: Select all

M128\MOS.ROM
M128\TERMINAL.ROM
M128\VIEW.ROM
M128\ADFS.ROM
M128\BASIC4.ROM
M128\EDIT.ROM
M128\VIEWSHT.ROM
M128\DFS.ROM
M128\COMSTAR352.rom
RAM
RAM
RAM
RAM
EMPTY
EMPTY
EMPTY
EMPTY
Start BeebEm

Select the following menus:
- Hardware/BBC Model/Master 128
- Comms/RS423 Destination/IP:localhost:25323 is ticked
- Comms/RS423 Destination/IP:IP232 mode is ticked
- Comms/RS423 On/Off is ticked

At this point, there should be two more lines in the tcpser console:

Code: Select all

2018-04-26 11:39:52:7543896:INFO:Accepting ip232 connection...
2018-04-26 11:39:52:7543896:INFO:Connection accepted from 127.0.0.1
*INSERT 8

Ctrl Break

*COMMSTAR

Press # to select prestel mode

Press I and then select:
- Even, No Parity, 1 stop bit (option 5)
- Send 9600 baud
- Receive 9600 baud
- return to exit

Press C to enter Chat Mode

Type ATD<ctrl M> and you should get the response OK

Type ATDGLASSTTY.COM:6502<ctrl M> to connect to John's service (very carefully, as delete does not work)

At this point the tcpser console log should look like:

Code: Select all

2018-04-26 11:48:19:6029320:INFO:Server socket bound to port
2018-04-26 11:48:19:6029320:INFO:Server socket listening for connections
2018-04-26 11:48:19:6029320:INFO:Opening ip232 device
2018-04-26 11:48:19:6029320:INFO:Server socket bound to port
2018-04-26 11:48:19:6029320:INFO:Server socket listening for connections
2018-04-26 11:48:19:6029320:INFO:ip232 device configured
2018-04-26 11:48:19:6036568:INFO:Control Lines: DSR:1 DCD:0 CTS:1
2018-04-26 11:48:19:6036568:INFO:Disconnecting modem
2018-04-26 11:48:19:6036568:INFO:Disconnecting
2018-04-26 11:48:19:6036568:INFO:Control Lines: DSR:1 DCD:0 CTS:1
2018-04-26 11:48:55:6036568:INFO:Accepting ip232 connection...
2018-04-26 11:48:55:6036568:INFO:Connection accepted from 127.0.0.1
2018-04-26 11:48:58:6061440:INFO:DTR has gone high
2018-04-26 11:49:01:6061440:INFO:DTR has gone low
2018-04-26 11:49:01:6036568:INFO:Disconnecting modem
2018-04-26 11:49:01:6036568:INFO:Disconnecting
2018-04-26 11:49:01:6036568:INFO:Control Lines: DSR:1 DCD:0 CTS:1
2018-04-26 11:49:53:6036568:INFO:taking modem off hook
2018-04-26 11:49:53:6036568:INFO:Connecting
2018-04-26 11:49:53:6036568:INFO:Connection to GLASSTTY.COM established
2018-04-26 11:49:53:6036568:INFO:Control Lines: DSR:1 DCD:1 CTS:1
Now hit return one more time and you should be in.
prestel0.png
prestel1.png
Simply wonderful! I must try to get this working on my real Master.

(There is some more help here)

Dave
Last edited by hoglet on Thu Apr 26, 2018 11:10 am, edited 1 time in total.

User avatar
BigEd
Posts: 2092
Joined: Sun Jan 24, 2010 10:24 am
Location: West
Contact:

Re: Viewdata Service

Post by BigEd » Thu Apr 26, 2018 11:07 am

This is a great service!

You remind me Dave, I had success in the past using 'ser2net' which is a Debian/Ubuntu standard. That was for inbound connections: ssh in through the firewall and routed to a Beeb.

User avatar
hoglet
Posts: 7501
Joined: Sat Oct 13, 2012 6:21 pm
Location: Bristol
Contact:

Re: Viewdata Service

Post by hoglet » Thu Apr 26, 2018 11:11 am

BigEd wrote: You remind me Dave, I had success in the past using 'ser2net' which is a Debian/Ubuntu standard. That was for inbound connections: ssh in through the firewall and routed to a Beeb.
Yes, I also used that on my Internet-Connected Atom Experiment

User avatar
John_Newcombe
Posts: 69
Joined: Tue Nov 28, 2017 9:19 am
Contact:

Re: Viewdata Service

Post by John_Newcombe » Thu Apr 26, 2018 2:39 pm

Ok, I have re-coded the front end a little. You should now be able to connect directly from BeebEm with no intermediate software needed.

On the MacOS Version of BeebEm, simply put the address 188.166.148.126 and port 6502 into the Comm/Remote Ethernet Port settings (set as client), make sure the RS232 is set to 'On' and away you go. The MacOS version does tend to corrupt the screens after a few pages, but hey ho.

The Windows version requires that you set the following two parameters in ~\My Documents\BeebEm\Preferences.cfg

IP232customip=188.166.148.126
IP232customport=00006502

Then restart BeebEm and use the Comm/RS232 Destination to IP:Custom Setting. Make sure the RS232 is set to 'On'.

I have been unable to test this as I do not have a Windows machine. I can run the Windows version of BeebEm with 'Wine' on Mac and Linux but IPv4 doesn't seem to be supported in that environment. Bah!
Last edited by John_Newcombe on Thu Apr 26, 2018 3:07 pm, edited 1 time in total.

User avatar
John_Newcombe
Posts: 69
Joined: Tue Nov 28, 2017 9:19 am
Contact:

Re: Viewdata Service

Post by John_Newcombe » Thu Apr 26, 2018 3:05 pm

vanpeebles wrote:Will it get to the point where people can ring your house with modems and connect like the old days? :D
Remarkably I do have a couple of Dowty Quattro Modems I could use and a Phone line. The thing is I can rent a Digital Ocean server for less than the cost of powering a small laptop at home. BT used servers in London (and elsewhere) I am using a server in London it all fits.

User avatar
danielj
Posts: 6588
Joined: Thu Oct 02, 2008 4:51 pm
Location: Manchester
Contact:

Re: Viewdata Service

Post by danielj » Thu May 03, 2018 12:20 pm

Just to note, I've forked the ESP8266 modem firmware across into the stardot repository to have a play with. As the RS423 is TTL rather than RS232 levels, we should be able to get away with using something like this:

https://www.ebay.co.uk/itm/1-x-NodeMcu- ... 3154085546

Together with a domino DIN plug to connect to these from beebs/masters. Anyone want to run a BBS from a Master? :D

d.

User avatar
Elminster
Posts: 3081
Joined: Wed Jun 20, 2012 8:09 am
Location: Essex, UK
Contact:

Re: Viewdata Service

Post by Elminster » Thu May 03, 2018 12:41 pm

Might do one day. I have a phone line that is almost never used. (Use VOIP or mobile, dont give out the number and it is ex-directory). I just have one thing that auto dials out, need o upgrade it to internet version but invovles spending money for no real benefit (other than being able to do a BBC bulleting board!)

Post Reply