Ceefax Engineering Test Page - BeebEm 4.13 & B-em

[guesser]

Excuse the bump, I noticed this page comes up when googling about the ceefax engineering test page so I thought I would chime in and say I've also had a go at recreating that early version of the page from Alan Pemberton's photo. I located a copy of the datasheet for the 74S262 character generator chip which includes the bitmaps for the font so from that I drew myself a cheat sheet of tifax rom symbols to teletext control codes and from there transcribed the whole test card minus the header row.

Here it is in the format of a URL hash for the javascript teletext frame editor: http://edit.tf/#0:QIECBAgQIECBAgQIECBAg ... ECBAgQIECA

It's not something that the modern teletext decoder in my TV likes at all however!

TV displaying recreated ceefax engineering test page

I've also attached a dump in the form of comma delimited hex bytes. This is formatted as would be sent, i.e. all 42 relevant bytes per line and with the parity set. I can dump it to another format if anyone needs, though the editor above is able to export to various presumably standard(?) formats too.

[jgharston]

Samples.zip contains the Teletext Engineering test page as downloaded directly off the air as page 99 (9D.9).

[guesser]

That's a different version of the page though
There seem to be a lot of versions, including some experimental tests etc. I don't suppose you have an idea when the page was captured? The green "Engineering" on row 24 is interesting.

One thing I've noticed which I assume is down to the software for the teletext adapter is that captured pages always have the double height rows duplicated. Obviously because that is what the beeb needs in its memory to generate the display correctly. Unfortunately it means that the dumps are a record of what the beeb displayed rather than what the decoder received.

[1024MAK]

I'm pretty sure the text for double height was transmitted twice, one per line. As sometimes if the data was corrupted due to poor signal or interference, only the top half of the double height text was displayed.

If you want, I can dig out some captured data from a teletext adaptor for an Atari ST. Mind, not sure how many, if any has double height text.

Mark

[jgharston]

That's a different version of the page though
There seem to be a lot of versions, including some experimental tests etc. I don't suppose you have an idea when the page was captured? The green "Engineering" on row 24 is interesting.

Sometime in 1986-1987. Only rows 0 to 23 are the raw data, the rest is padding to 1024 bytes with ABZTtxt data in the padding.

[guesser]

They may have done early on, but they got very efficient at wringing the maximum bandwidth available out of the available VBI lines. Only the first line of the double height text needs to be transmitted so usually the one below isn't sent. In the case of the test card though (at least that early one in the photographs) there's a full line of clock cracker with the appropriate number at the end.

[BeebMaster]

My understanding was that only the first row of double height text was sent, but then again, that would mean that you couldn't have double-height text in two different colours or two different backgrounds, wouldn't it? And what if only part of the line was double height?

All my captured Teletext pages are screen dumps of the output of mode 7, I never captured any pages with the non-displaying rows or even the identification bytes at the beginning of each row, so I suppose I don't have much of a record of the technical side of Teletext, just a record of what actually displayed.

I don't recall having an issue with control codes appearing as characters 0-31 but I did get problems with the Beeb Teletext chip not being able to display black text. I wrote a little converter which would go through mode 7 RAM and replace the black code with the blue code - something like *BLACKTOBLUE. I have wondered if it's possible to change the Teletext chip inside a Beeb for one that does support black text.

The engineering page was still being transmitted right up to the analogue switch off, I have several copies downloaded when capturing whole channels' worth of Teletext pages, principally in the autumn of 2009 before the Granada region went off. I can't recall offhand the page number, it was in the first magazine, may have been page 169 or 197, something like that. Next time I dip into my Teletext page storage I'll retrieve it and put it here.

[BeebMaster]

Thought my website might be a good source for this kind of thing, especially anything that happened more than 5 years ago.

Here are screengrabs of the Engineering pages from the British Garden Birds laserdisc:

[Dreamland Fantasy]

My understanding was that only the first row of double height text was sent, but then again, that would mean that you couldn't have double-height text in two different colours or two different backgrounds, wouldn't it? And what if only part of the line was double height?

My understanding is that in teletext transmissions only the first row of double height text is required as the teletext decoder should automatically "fill in the blanks" for the second row.

Of course with the BBC Micro you do need to specify both rows which allows for different colours per row. I don't know if teletext transmissions allow this (I can't recall any teletext pages like this) or if it is a case that any colour changes, etc. would override the automatic doubling of the row.

Kind regards,

Francis

[guesser]

My understanding was that only the first row of double height text was sent, but then again, that would mean that you couldn't have double-height text in two different colours or two different backgrounds, wouldn't it? And what if only part of the line was double height?

That's something that I think you can only do on a beeb etc. because you are in control of what data you put in the screen memory for the SAA5050 to read. A teletext decoder is supposed to display either blank space or the bottom half of double height text in the same colours as the top row.

I don't recall having an issue with control codes appearing as characters 0-31 but I did get problems with the Beeb Teletext chip not being able to display black text. I wrote a little converter which would go through mode 7 RAM and replace the black code with the blue code - something like *BLACKTOBLUE. I have wondered if it's possible to change the Teletext chip inside a Beeb for one that does support black text..

As far as I know, only that Texas Instruments 74S262 chip ever had a way to display any symbols for the control codes. Black foreground is something that doesn't exist in what is now called "Level 1" teletext, though some decoders (like my TV) do implement it. That is why on the photo I posted earlier the red line numbers aren't visible because they are preceded by the alphanumeric black code which should be ignored for Level 1 teletext
The current etsi standard does warn that alpha and mosiac black "should be used with caution as it is interpreted by
some existing Level 1 and Level 1.5 decoders."

The engineering page was still being transmitted right up to the analogue switch off, I have several copies downloaded when capturing whole channels' worth of Teletext pages, principally in the autumn of 2009 before the Granada region went off. I can't recall offhand the page number, it was in the first magazine, may have been page 169 or 197, something like that. Next time I dip into my Teletext page storage I'll retrieve it and put it here.

I have been given one dump of the page from a tape from 1983 but it has a few errors. Hopefully if I can gather together enough copies I can take a kind of average as different bytes will have been lost in transmission each time
It's frustrating as there are endless photos and video of it since it seems to have been transmitted in vision quite a lot. That's enough to make something that looks the same, but doesn't give away what is present in the parts you can't see

I have some small dumps from 2006 and 2010, one which I made, and a couple I found online as examples to use with beebem, but alas none contain the engineering test page. It's possible it just wasn't transmitted in the time the dump was running as they're only a couple of seconds worth each and I suppose the test page was transmitted pretty infrequently.

[guesser]

Of course with the BBC Micro you do need to specify both rows which allows for different colours per row. I don't know if teletext transmissions allow this (I can't recall any teletext pages like this) or if it is a case that any colour changes, etc. would override the automatic doubling of the row.

Once the decoder has seen a double height control code on a line it should ignore anything on the line below and display only the bottom half of the double height text and fill the rest with the same background colour as the top row. As for what an individual television set etc. will do... well that's another matter entirely

[BeebMaster]

I have some small dumps from 2006 and 2010, one which I made, and a couple I found online as examples to use with beebem, but alas none contain the engineering test page. It's possible it just wasn't transmitted in the time the dump was running as they're only a couple of seconds worth each and I suppose the test page was transmitted pretty infrequently.

Possibly I may be able to help. I have a few Super VHS tapes I made at the time, but my experience of capturing pages of Teletext from these recordings is pretty poor, we always had bad analogue TV reception anyway. More hopeful might be the laserdisc, I can make another capture of the engineering page stored on there to get to the actual downloaded data (rather than a screendump of the displayed page) if that would be any use.

[1024MAK]

In this post there is an attachment with some teletext data in.

Mark

[guesser]

I'd never heard of teletext on laserdisc before, that's very interesting
Getting a raw dump of the contents would be interesting to me from a preservation point of view. I'm fairly new to the teletext "scene" but have an isatiable thirst for info at the moment
I dumped my captures live off the air using a PCI capture card, that would work for recordings and the laser discs too. There's a chap who is recovering lots from s-vhs tapes called Jason, that's where the 1983 capture of the testcard I have came from. There's a fairly active Facebook group with some knowledgeable teletext bods. Until I saw that I didn't realise there were more than a couple of people still interested in this stuff!

[guesser]

In this post there is an attachment with some teletext data in.

Hi Mark. Those ttx format pages are certainly easy enough to do something useful with
The file "CH4_349A.TTX" demonstrates the "only the top row of double height text" thing nicely with the ad for loans.co.uk at the bottom of the page, the line below is all spaces

[Dreamland Fantasy]

Thought my website might be a good source for this kind of thing, especially anything that happened more than 5 years ago.

Here are screengrabs of the Engineering pages from the British Garden Birds laserdisc:

Do you have this file in something like a .bmp format? I would be interested in a version of the image that doesn't have the JPEG compression artefacts.

Kind regards,

Francis

[jgharston]

Do you have this file in something like a .bmp format? I would be interested in a version of the image that doesn't have the JPEG compression artefacts.

Will this do? (Screenshot from BeebEm)

[1024MAK]

In this post there is an attachment with some teletext data in.

Hi Mark. Those ttx format pages are certainly easy enough to do something useful with
The file "CH4_349A.TTX" demonstrates the "only the top row of double height text" thing nicely with the ad for loans.co.uk at the bottom of the page, the line below is all spaces

Ahh, maybe that's partly where the confusion creeps in. Part of the reason that I thought that the double height text included the text on the second line, is because the files produced by the Atari teletext decoder software (the fixed format file type) are all the same size. As if there are a fixed number of characters/codes per line, and a fixed number of lines per page.

I will try to find some time to extract more of my teletext archive. Maybe on Saturday or Sunday.

I also do (did) have some generated by a PC (as in IBM compatible computer) that has a TV decoder card which also does (did) teletext decoding. This machine still boots off it's original HDD. But the added second much larger drive (used for most programs and most data) is a Western Digital HDD, which got the head knocking/banging fault some years ago. There should be a partial back-up on a Zip disk, but it's a case of finding it! Also I forget what format the program stored pages in. I remember it did picture formats, and plain text formats, but can't for the life of me remember if it did a "raw" format. So this may will take rather a lot longer.

Mark

[Dreamland Fantasy]

Do you have this file in something like a .bmp format? I would be interested in a version of the image that doesn't have the JPEG compression artefacts.

Will this do? (Screenshot from BeebEm)

Thanks for that Jonathan, but I was actually more interested in the screengrab artefact present in that particular image. I'm experimenting with simulating screen artefact and I was looking for a "clean" image without the JPEG artefact also being present.

Kind regards,

Francis

[jgharston]

Ahh, maybe that's partly where the confusion creeps in. Part of the reason that I thought that the double height text included the text on the second line, is because the files produced by the Atari teletext decoder software (the fixed format file type) are all the same size. As if there are a fixed number of characters/codes per line, and a fixed number of lines per page.

People seem to be confusing the teletext decoder and the teletext display electronics. The teletext display system, usually a SAA505x, just displays whatever is in the memory that it uses. The teletext decoder, usually a SAA502x and SAA503x, does not do any displaying, it decodes the datastream from the broadcast television signal and fills the memory used by the teletext display system. The teletext display system blindly displays whatever is in that memory. The teletext decoder will fill two lines of memory when a line with a double-height character is in it. The display hardware does not add the extra row, the decoder does. So, any dump of the memory being displayed will have two rows of characters for a double-height line. Any dump of the data the decoder uses to fill the display memory will have only one row IF the transmitter chose to only send one row.

Using Acorn Teletext parlance, *TRANSFER fetches whatever data is sent for the requested page, so will fetch no data where no data is sent. *DISPLAY will copy that received daya into the display memory, filling in blank rows where no row was received and filling in extra rows for double-height lines where needed. The display system does not do line doubling, the decoder does the line doubling when giving the data to the display system.

[BeebMaster]

Do you have this file in something like a .bmp format? I would be interested in a version of the image that doesn't have the JPEG compression artefacts.

I have all my DVD recordings which I use to make screens like that. I can't record at any higher quality, my DVD recorder already is recording at the highest setting (you get 1 hr of video on a DVD) but I've gone back to that video file and processed it again as BMP instead of JPG.

Unfortunately I can't upload a bitmap as an attachment to this forum, and zipping it completely defeats the object of being able to DISPLAY the new images as comparison with the jpeg already shown above, so I've had to upload them elsewhere and link here:





The jpeg compression artefacts are gone, but there is still a lot of blurring at the edges of the darker colours. I don't know if I can do anything about that, it's present in the original recording so it would probably have to involve using new equipment.

Here's the output of the avconv command I used if that's any use for anything:

Code: Select all

bm-laptop@bm-laptop:/media/BMNFS/Pictures/Finished/May 2011/Capture/Finished$ avconv -i "May2011A.VRO" -r 0.5 -q 1 -ss 63 -t 8 May2011a%03d.bmp
avconv version 11.2-6:11.2-1, Copyright (c) 2000-2014 the Libav developers
  built on Jan 18 2015 05:12:33 with gcc 4.9.2 (Ubuntu 4.9.2-10ubuntu2)
Input #0, mpeg, from 'May2011A.VRO':
  Duration: 00:00:28.81, start: 0.284778, bitrate: 174139 kb/s
    Stream #0.0[0x80]: Audio: ac3, 48000 Hz, stereo, fltp, 384 kb/s
    Stream #0.1[0x1e0]: Video: mpeg2video (Main), yuv420p, 720x576 [PAR 64:45 DAR 16:9], 8820 kb/s, 25 fps, 90k tbn, 50 tbc
Output #0, image2, to 'May2011a%03d.bmp':
  Metadata:
    encoder         : Lavf56.1.0
    Stream #0.0: Video: bmp, bgr24, 720x576 [PAR 64:45 DAR 16:9], q=2-31, 200 kb/s, 0.50 fps, 0.50 tbn, 0.50 tbc
    Metadata:
      encoder         : Lavc56.1.0 bmp
Stream mapping:
  Stream #0:1 -> #0:0 (mpeg2video (native) -> bmp (native))
Press ctrl-c to stop encoding
[mpeg2video @ 0xd27060] warning: first frame is no keyframe
frame=    4 fps=  0 q=0.0 Lsize=       0kB time=6.00 bitrate=   0.0kbits/s    its/s    
video:4860kB audio:0kB other streams:0kB global headers:0kB muxing overhead: unknown

I can get a new dump of Teletext data for the engineering page, although that's a bit more involved as I will have to go back to the laserdisc and grab it again using the Teletext adapter.

[Dreamland Fantasy]

Do you have this file in something like a .bmp format? I would be interested in a version of the image that doesn't have the JPEG compression artefacts.

I have all my DVD recordings which I use to make screens like that. I can't record at any higher quality, my DVD recorder already is recording at the highest setting (you get 1 hr of video on a DVD) but I've gone back to that video file and processed it again as BMP instead of JPG.

I am experimenting with simulating artefacts that occur with TV/video so what you have posted is perfect! Many thanks for that!

Here is an example of the output of my Mode 7 / teletext converter:

And the differences between the above and your screengrab that I can use for analysing:

Kind regards,

Francis

[BeebMaster]

Glad it's been of use. Incidentally, the Conceal/Flash/Gone doesn't seem to be in the laserdisc engineering page.

Comparing the BMP and the JPG, I can see how much better quality the bitmap is, especially considering both images came from the same source video, but each bitmap screen is 1.2MB compared with 40-50KB for the average JPG screen, so I don't think I could use bitmap images for my screengrabs. Next time I do a recording I might try processing the screens to BMP first then change them to JPG for the final set and see if that makes any difference.

[BeebMaster]

Well now, this is interesting. Here's the original JPG image before it was processed into the image on my website:



That looks to be nearly as good as the BMP, and it's only 107K. I use Phatch to process the images into picture sets, and I use the JPG quality setting at 90% which decimates the file size. That's mainly for camera images, so I can get the file sizes down from several MB to under 200K. In future I won't reduce the quality for screenshots.

[BigEd]

BTW, you lose no information by putting things in a zip file, so BMP-in-zip would be just as good, quality-wise, as BMP. If you can produce PNG format that should also be lossless.

[guesser]

Glad it's been of use. Incidentally, the Conceal/Flash/Gone doesn't seem to be in the laserdisc engineering page.

How do you mean? That text is hidden by conceal codes so is only visible when "revealed"

[simonm]

I'm currently playing with some Mode7 teletext content. Having read this thread its clear theres a couple of differences between the teletext chip in the Beeb and how it renders raw teletext byte stream. For instance if I load the TESTPAGE from jgharstons Samples.zip directly to &7C00 I note that the double height is incorrect (missing bottom line) and the hold graphics have slightly different results.
Are these the only two 'differences' to be mindful of?

[Dreamland Fantasy]

I'm currently playing with some Mode7 teletext content. Having read this thread its clear theres a couple of differences between the teletext chip in the Beeb and how it renders raw teletext byte stream. For instance if I load the TESTPAGE from jgharstons Samples.zip directly to &7C00 I note that the double height is incorrect (missing bottom line) and the hold graphics have slightly different results.
Are these the only two 'differences' to be mindful of?

A teletext decoder in e.g. a TV will automatically fill in the 2nd line of double height text. I'm not aware of any differences in the hold graphics function though (some of the test pages are coded differently and can give different results), although I may be wrong in that.

Oh, and don't use BeebEm for Mode 7 teletext stuff as its implementation is flawed.

Kind regards,

Francis

[simonm]

Oh, and don't use BeebEm for Mode 7 teletext stuff as its implementation is flawed.

Oops! That might explain a few things...
The hold graphics difference I'm seeing is that BeebEm seems to use a held control code one byte sooner than the equivalent teletext render.

[guesser]

Hold mosaics is notoriously awkward to implement correctly.
If you are playing with editing teletext stuff you may want to look at a dedicated teletext editor as they have options to show visible control codes, grid lines, and other editing aids. There's a windows program wxTED, the edit.tf online javascript teletext editor, and finally another javascript editor that is still very new and as yet has no name...
(full disclosure - I wrote that third one )

Edit-tf is more "beeb-ey" using the SAA5050 character sets and allowing the whole frame to be edited, mine is more "teletext-ey" using the bitmaps from the ETSI teletext standard and applying the rules strictly to ensure pages are suitable for transmission.

Edit.tf has a range of export formats which includes raw binary output (with either 00-1F, or 80-9F control codes) which should be easy to get into a Beeb.

Finally to keep this reply vaguely on topic here's the recreated engineering test page from the 'Alan Pemberton tifax photo' in my editor.