Clock Signal: now also for Linux and BSD

want to talk about MESS/model b/beebem/b-em/electrem/elkulator? do it here!
ThomasHarte
Posts: 458
Joined: Sat Dec 23, 2000 5:56 pm
Contact:

Clock Signal: now also for Linux and BSD

Postby ThomasHarte » Thu Nov 23, 2017 1:30 am

Clock Signal is my emulator of the Acorn Electron and a bunch of other machines that Acorn didn't make. Its particular claims to fame are:

  • a desire to be entirely invisible — emulator as medium, not message;
  • emulation of the entire stack of things that comprised your Electron — the machine itself, and a CRT, fed either by a composite or by an RGB signal. Composite display is an actual GPU-powered composite encoding and decoding of video, not a post-processing effect. Whether composite or RGB, the painted surface is simulated phosphor and will update as quickly as your machine's refresh rate, being entirely independent of the 50Hz rate of the Electron and thereby seeking to avoid classic 50Hz-to-60Hz emulation judder; and
  • audio uses a broadly similar signals processing approach, being internally sampled at a very high resolution and then resampled downward. Speech effects like Exile are not handled as a special case, they just work.

Supported Electron file formats are the usual suspects: UEF (including HQ), CSW, SSD, DSD, ADF. Tapes can usually be speed loaded regardless of file format; the only known exception is Way of the Exploding Fist as it uses a custom loader.

Binaries are provided for macOS. A Scons-powered build script is provided for Linux and BSD; prerequisites are SDL 2, ZLib and OpenGL (or, more likely, Mesa).

This is the first attempt to build Linux so feedback is sought.

Releases accumulate on GitHub. The one at the top of that page is the latest.

User avatar
Matt Godbolt
Posts: 180
Joined: Mon Jul 31, 2006 10:02 am
Location: Chicago
Contact:

Re: Clock Signal: now also for Linux and BSD

Postby Matt Godbolt » Thu Nov 23, 2017 1:48 am

Wow great stuff!! Looking forward to checking this out :-)

ThomasHarte
Posts: 458
Joined: Sat Dec 23, 2000 5:56 pm
Contact:

Re: Clock Signal: now also for Linux and BSD

Postby ThomasHarte » Thu Nov 23, 2017 3:20 am

Matt Godbolt wrote:Wow great stuff!! Looking forward to checking this out :-)

Alas it is presently OpenGL 3.2 powered, so I will not shortly be asking you for Emscripten tips. Also, I've uncovered a bug already, with the Oric display being very incorrect on X11.

A couple of boasts I forgot to make though:

It's the closest emulator yet as to timing on memory access; a regression in Mode 3 since last I checked is probably its only deviation from correctness, so for this test:
Mode 0: 14.70
Mode 1: 14.75
Mode 2: 14.90
Mode 3: 7.74
Mode 4: 7.07
Mode 5: 7.07
Mode 6: 7.07

I was completely unaware of the Mode 3 error before composing this post; will file it as a bug.

It similarly properly models the Electron output as interlaced, and obeys all of Paul Boddie's other observations as to behaviour.

paulb
Posts: 811
Joined: Mon Jan 20, 2014 9:02 pm
Contact:

Re: Clock Signal: now also for Linux and BSD

Postby paulb » Thu Nov 23, 2017 12:01 pm

ThomasHarte wrote:It similarly properly models the Electron output as interlaced, and obeys all of Paul Boddie's other observations as to behaviour.


Nice to see you doing more emulator work, Thomas!

I hope my observations are still valid given hoglet's heroic investigations with FPGA implementations, given that my document mostly started as speculation about the various mechanisms and probably wasn't too far from what emulator developers have been doing all along. I do remember some timing exercises that refined some of my assumptions somewhat (probably the ones you've mentioned) and the awkward realisation that the ULA really does have to throw away potentially usable cycles in modes 0 to 3 because the CPU cannot interact with the RAM fast enough.

guesser
Posts: 172
Joined: Mon Jun 26, 2006 9:21 pm

Re: Clock Signal: now also for Linux and BSD

Postby guesser » Thu Nov 23, 2017 5:18 pm

ThomasHarte wrote:[*] emulation of the entire stack of things that comprised your Electron — the machine itself, and a CRT, fed either by a composite or by an RGB signal. Composite display is an actual GPU-powered composite encoding and decoding of video, not a post-processing effect.


Very interesting! Is speccy emulation on the roadmap for the future? I'd love to see what your code makes of my silly "chromatrons" game :D
A web based teletext editor which can export as Mode 7 screen memory: https://zxnet.co.uk/teletext/editor

ThomasHarte
Posts: 458
Joined: Sat Dec 23, 2000 5:56 pm
Contact:

Re: Clock Signal: now also for Linux and BSD

Postby ThomasHarte » Thu Nov 23, 2017 9:52 pm

Now available on master, and included in a fresh source and Mac release: the Mode 3 timing mistake is fixed — usually around 11.91s, depending on how promptly I release the enter key. So I really think this is the most-accurate emulator to date.

paulb wrote:
ThomasHarte wrote:It similarly properly models the Electron output as interlaced, and obeys all of Paul Boddie's other observations as to behaviour.


Nice to see you doing more emulator work, Thomas!

I hope my observations are still valid given hoglet's heroic investigations with FPGA implementations, given that my document mostly started as speculation about the various mechanisms and probably wasn't too far from what emulator developers have been doing all along. I do remember some timing exercises that refined some of my assumptions somewhat (probably the ones you've mentioned) and the awkward realisation that the ULA really does have to throw away potentially usable cycles in modes 0 to 3 because the CPU cannot interact with the RAM fast enough.

I had definitely previously believed it to be a progressive not-exactly-50Hz machine with a real-time clock interrupt that always fell at the end of a scan line. All of those errors are corrected.

guesser wrote:
ThomasHarte wrote:[*] emulation of the entire stack of things that comprised your Electron — the machine itself, and a CRT, fed either by a composite or by an RGB signal. Composite display is an actual GPU-powered composite encoding and decoding of video, not a post-processing effect.


Very interesting! Is speccy emulation on the roadmap for the future? I'd love to see what your code makes of my silly "chromatrons" game :D

I think it might be interesting to try the Spectrum, especially as my 8272 has still not necessarily earned my trust and adding +3DOS to the things I've tested that with might lead to some revelations. But I worry that the main part of getting the user-experience I'd really want would be an endless indeterminate slog through figuring out how to accelerate different loading schemes. On the CPC I've just left tapes to load at their natural speed. I suspect that the immediate future is going to be spent dealing with all the latent issues suddenly revealed by attempting to operate with an entirely different outer program (i.e. the SDL code) in a different environment under a different compiler.

Though I had a pleasing experience with the Oric: its composite signal is generated from an asynchronously-clocked ROM-based lookup table, and throwing those numbers straight in seems to result in the proper colours out.

User avatar
Matt Godbolt
Posts: 180
Joined: Mon Jul 31, 2006 10:02 am
Location: Chicago
Contact:

Re: Clock Signal: now also for Linux and BSD

Postby Matt Godbolt » Fri Nov 24, 2017 7:14 pm

ThomasHarte wrote:
Matt Godbolt wrote:Wow great stuff!! Looking forward to checking this out :-)

Alas it is presently OpenGL 3.2 powered, so I will not shortly be asking you for Emscripten tips.


Haha no worries. The only emscripten I know is enough to get BeebAsm to work in BeedIDE :)

Built pretty easily - nice work. I sent some PRs to fix errors GCC 4.8 picked up. Will try building it against 7.2. There's some strict aliasing warnings which will probably bite on newer GCCs. If you're interested in help fixing those, let me know!

User avatar
Matt Godbolt
Posts: 180
Joined: Mon Jul 31, 2006 10:02 am
Location: Chicago
Contact:

Re: Clock Signal: now also for Linux and BSD

Postby Matt Godbolt » Fri Nov 24, 2017 7:16 pm

I'd love to try and port the composite output to jsbeeb though...will be digging around for info. Do you have any links to sources you used for this?

ThomasHarte
Posts: 458
Joined: Sat Dec 23, 2000 5:56 pm
Contact:

Re: Clock Signal: now also for Linux and BSD

Postby ThomasHarte » Fri Nov 24, 2017 9:19 pm

Matt Godbolt wrote:
ThomasHarte wrote:
Matt Godbolt wrote:Wow great stuff!! Looking forward to checking this out :-)

Alas it is presently OpenGL 3.2 powered, so I will not shortly be asking you for Emscripten tips.


Haha no worries. The only emscripten I know is enough to get BeebAsm to work in BeedIDE :)

I previously though that was part of the JSBeeb toolchain, but now I see that it is actually a JavaScript-first implementation. Obviously I wasn't paying enough attention. Very poor form.

Matt Godbolt wrote:Built pretty easily - nice work. I sent some PRs to fix errors GCC 4.8 picked up. Will try building it against 7.2. There's some strict aliasing warnings which will probably bite on newer GCCs. If you're interested in help fixing those, let me know!

I've definitely made a few implementational missteps along the way: I started this emulator two or three years ago because I had never used modern C++ and had completely forgotten what of the '90s-style C++ I once knew. So a lot of the older sections look a lot more like a C programmer trying to find his way than do the new sections, especially the union that underlies static analyser results. I think I also initially leaned far too heavily on shared_ptr, being an Objective-C/Swift transferee. I've generally been going back and fixing such things as and when my knowledge proceeds and time allows but it's a learning process. The survival of some phoney aliasing assumptions would be no surprise.

That all being admitted, apparently I'm using GCC 7.2 over in Ubuntu world, so probably all is already well. On the Mac I'm on "Apple LLVM" (surely they haven't forked their own project?) version 9.0.0 with clang-900.0.38.

PRs merged by the way, thanks!
Matt Godbolt wrote:I'd love to try and port the composite output to jsbeeb though...will be digging around for info. Do you have any links to sources you used for this?

I'll see whether I can dig anything up, but the genesis of all that is very long-winded. I wrote a ZX80/81 emulator back in 2011, which is a machine where the programmer has a great deal of programmatic control over the video output, and probably started reading about it then. But I'm not sure there's a lot to it given that you're emulating a CRTC so must have syncs under control, so I think all you really need to compose the PAL output stream is:
  • keep track of a colour subcarrier at 4.43361875Mhz;
  • translate to YUV;
  • output Y*0.8 + (U*cos(t) + V*sin(t))*0.2, with t being the appropriate measure of the colour subcarrier.

Then to convert back:
Y = input with lowpass filter applied; subcarrier/4 is a good choice;
U = 2 * (input - Y)*cos(t), with even more aggressive lowpass filter applied, probably subcarrier/8; and
V = 2 * (input - Y)*sin(t), with same lowpass filter as for U.

Then colour space convert and output that.

I'm unaware of any particular quirks in how Acorn's hardware produces a composite wave, so for the Electron I'm running exactly by the book from RGB. That's not true of the other machines; both the 2600 and the Vic-20 produce their outputs directly as amplitude + phase offset of a single subcarrier-frequency wave, and the Oric uses a lookup table extracted from the original hardware.

The (most) hand-waving justification for the mathematics being:

As U and V are added on as amplitude modulations of a 4.43361875Mhz, of course low-pass filtering the whole signal will recover Y.

Given the single amplitude modulation of k*sin(t), you can't get back to k by dividing by sin(t) because of sin(t)'s annoying habit of crossing zero. But if you multiply by sin again and throw in a trig identity:

k * sin(t) * sin(t) = k * sin^2(t) = k * 1/2(1 - cos(2t)) = k/2 - k/2 * cos(2t)

... so if you lowpass filter that to kill the cos(2t) term, you get k/2 back out. And then clearly the same logic applies to the cos term because that's just sin with a different phase. And, to be really precise about it:

k * sin(t) * sin(t) + v * cos(t) * sin(t) = ... k as above ... + 1/2(sin(t + t) + sin(t - t)) = ... + sin(2t)/2 + sin(0)/2 = ... + sin(2t)/2

So the fact of composition of two out-of-phase signals just adds some more noise in the 2t frequency range, which you also throw away via the lowpass filtering.

I was originally applying another instance of my Kaiser-Bessel filter to the problem, but getting a decent answer out of that created a bit of a bandwidth issue on the GPU: there's only one texture sample for each two source data samples but it still adds up. So I took a hacky shortcut: I sample my original composite wave at four times the colour subcarrier then just perform an exact average over each group of four to recover Y. Which is a bit brute-force, but is effectively a comb filter, I guess.
Last edited by ThomasHarte on Fri Nov 24, 2017 9:42 pm, edited 1 time in total.

User avatar
Matt Godbolt
Posts: 180
Joined: Mon Jul 31, 2006 10:02 am
Location: Chicago
Contact:

Re: Clock Signal: now also for Linux and BSD

Postby Matt Godbolt » Fri Nov 24, 2017 9:33 pm

Wow: thanks for the deep info! A lot to digest...I'll see what I can knock up :) Thank you!

ThomasHarte
Posts: 458
Joined: Sat Dec 23, 2000 5:56 pm
Contact:

Re: Clock Signal: now also for Linux and BSD

Postby ThomasHarte » Fri Nov 24, 2017 9:43 pm

Matt Godbolt wrote:Wow: thanks for the deep info! A lot to digest...I'll see what I can knock up :) Thank you!

I'll probably have missed something out, but I'm sure you can figure out the point: add the extra two channels as amplitude modulations of ~4.43Mhz subcarriers that are exactly out of phase, multiply again by sin and cos to extract them, use lowpass filtering at various stages to get rid of bits of signal you don't want, use the trig identities to justify why what you're doing works. Easy!

User avatar
Pernod
Posts: 1167
Joined: Fri Jun 08, 2012 10:01 pm
Location: Croydon, UK
Contact:

Re: Clock Signal: now also for Linux and BSD

Postby Pernod » Tue Jan 16, 2018 1:43 am

I haven't tried this (no Mac) but am curious how your Electron reports BASIC with *ROMS in Advanced ROM Manager. A photo at viewtopic.php?f=7&t=12677#p166755 reports it in page 10, but both Elkulator and MAME report page 11. Also tried ElectrEm but didn't like me trying to get it to load ARM.

I expect it's related to the ULA ROM paging that's been discussed over the years so wondered if Clock Signal gets it right.
- Nigel

BBC Model B, ATPL Sidewise, Acorn Speech, 2xWatford Floppy Drives, AMX Mouse, Viglen case, BeebZIF, etc.

ThomasHarte
Posts: 458
Joined: Sat Dec 23, 2000 5:56 pm
Contact:

Re: Clock Signal: now also for Linux and BSD

Postby ThomasHarte » Tue Jan 16, 2018 2:16 am

As currently released, it refuses to believe that Acorn ROMs can be anything other than 16kb in size. But even after doubling up the file contents to get it to load, it gives the wrong answer: slot 11.

So, no, it gets it wrong.

Work to do!

EDIT: current implementation is:

Code: Select all

// latch the paged ROM in case external hardware is being emulated
active_rom_ = (Electron::ROMSlot)(*value & 0xf);

// apply the ULA's test
if(*value & 0x08) {
   if(*value & 0x04) {
      keyboard_is_active_ = false;
      basic_is_active_ = false;
   } else {
      keyboard_is_active_ = !(*value & 0x02);
      basic_is_active_ = !keyboard_is_active_;
   }
}


So there's a genuine attempt to apply what I understand to be the received wisdom. I'll start with a forum search, possibly I'm behind.

(EDIT2: 8kb ROM loading is now enabled in the version on master, for those building at home; I released a new version earlier in the day with some MSX fixes so it'll probably be a while before I release a new Mac binary)

User avatar
Pernod
Posts: 1167
Joined: Fri Jun 08, 2012 10:01 pm
Location: Croydon, UK
Contact:

Re: Clock Signal: now also for Linux and BSD

Postby Pernod » Tue Jan 16, 2018 1:56 pm

ThomasHarte wrote:So there's a genuine attempt to apply what I understand to be the received wisdom. I'll start with a forum search, possibly I'm behind.

The only detailed discussion I could find is at viewtopic.php?f=3&t=1178&start=30, which your current implementation is based upon.

I'm also curious as to what Elkman reports on a real Electron, specifically the empty sockets being reported as 4k. This is MAME with a ROMBOX+ attached:
0016.png
- Nigel

BBC Model B, ATPL Sidewise, Acorn Speech, 2xWatford Floppy Drives, AMX Mouse, Viglen case, BeebZIF, etc.

ThomasHarte
Posts: 458
Joined: Sat Dec 23, 2000 5:56 pm
Contact:

Re: Clock Signal: now also for Linux and BSD

Postby ThomasHarte » Fri Mar 23, 2018 2:19 am

Very minor announcement: a new release enables HFE support for the Electron. So if you have any disk images in that format, or create any because you have an HxC, please don't hesitate to test.

As I've yet to implement FDI or anything equivalent, HFE is now the most accurate supported file format.

ThomasHarte
Posts: 458
Joined: Sat Dec 23, 2000 5:56 pm
Contact:

Re: Clock Signal: now also for Linux and BSD

Postby ThomasHarte » Wed May 16, 2018 2:04 am

Another small release: ADF support was broken, specifically sectors would not all be transcribed to the disk surface. It was a failure properly to factor out the relevant logic when implementing support for the analogous MSX file format, so had probably been broken January. But now is no longer. I strongly suspect that I'm the first to notice.

Check it out in the usual place.

EDIT: actually, it strikes me that people around here are sufficiently open to other platforms that when I'm announcing an update or fix to the Acorn side of things I can slip this in: since the last announcement in March, the emulator has seen a substantial improvement in its Vic-20 emulation (colours especially; I intuited that the colour phase offsets formally documented for the TED were probably the same as for the VIC, and they are — also I added an S-Video route for video, optionally rescuing the Vic from composite world) and gained Apple II emulation.

Uninterestingly for a user, the code base briefly topped 50,000 lines during this development cycle, though it's nudged back below since. Though that's for nine machines, not counting trivial variants, so only a bit more than 5,500 lines per machine. Sort of.