The Hidden Fifth Column: Integrating Wizzo with RISC OS 3.11

discuss general risc os software applications and utilities
Related forum: adventures


Post Reply
philpem
Posts: 569
Joined: Fri Apr 04, 2014 6:42 pm
Contact:

The Hidden Fifth Column: Integrating Wizzo with RISC OS 3.11

Post by philpem » Mon Jul 13, 2020 3:49 pm

I've been having a play around with some of the RISC OS ROM modding tools dotted around Stardot, with varying degrees of success. I think I've found a "better" way which doesn't involve huge amounts of ROM hackery.

Many of you will be familiar with the 5th Column ROM socket. As it happens, the Podule module (the part of RISC OS which handles Podules and extension ROMs) scans the entire MEMC ROM are for 5th Column ROMs. You'll also know that the ARM250 machines don't have one, which rules out adding Wizzo to the RISC OS ROM. Because of the ">2MB checksum issue", the RISC OS ROM has to stay below 2MB (until the bug is fixed).

Or does it?

Let's take a look at the 5th Column ROM documentation... (https://www.chiark.greenend.org.uk/~the ... Column.txt)
Extension ROMs appear in the ROM area of the memory map, ie between &03400000 and &03FFFFFF. An extension ROM set must end on a 64K boundary or at the start of another extension ROM. This is normally not a problem as it is unlikely you would want to use a ROM smaller than a 27128 (16K), and the normal way of addressing this would mean that the ROM would be visible in 1 byte out of each word, ie within a 64K addressable area.
The eagle-eyed among you will have noticed what I noticed: the RISC OS ROM is normally mapped in between &03800000 and 0x39FFFFF, if we have a standard 2MB ROM set fitted. But the ARM250 machines (and probably others) can accept a pair of 16Mbit 16-bit EPROMs, giving a total of 4MBytes of ROM space. That means you can have a ROM which looks like this:

Code: Select all

+------------+ &38000000
|   RISC OS  |
|    2Meg    |
|            |
+------------+ &3A000000
| filler     |
| &FFFFFFFF  |
+------------+ &3C000000 less 64k
| 5th Column |
| ROM        |
| (64k)      |
+------------+ &3C000000
This gets even better -- because the RISC OS ROMs are 32-bits wide from the ARM's perspective (two ROMs, each 16-bit wide) -- the code can be executed in-place, provided it doesn't try to alter itself. Altering its workspace (the correct way per the PRMs!) is fine.

We need to do a few slightly special things. First, each module needs to be prefixed with its length, a bit like the "module chain" in the RISC OS ROM. Second, each module must begin on a multiple of 4 bytes (the two least significant address bits must be zero). If you breach the second condition, RISC OS will start booting, but throw an Address Exception when it starts to load your module. Unaligned accesses aren't allowed for program code.

Some of you might remember my project to rebuild an Acorn A4 version of Wizzo, with the A4's I2C and Portable ROMs included. I updated the 5th Column ROM tools from that project to account for the 32-bit requirements. The result is on Github: https://github.com/philpem/riscos-romtools/ (5thColumn directory).

So how do we use this?
  • Get the modules you need together. I grabbed Wizzo 3V15's IDEFS and IDEFSFiler.
  • Edit config.yml to list the ROMs in the order you'd like them included.
  • Run mkrom.py to create a 5th Column ROM. You'll get a 64kbyte "rom.bin" as output.
  • Run 'dd if=/dev/null of=padding.bin bs=65536 count=31' -- this produces a 2MB-less-64k padding image. (Ideally you'd pad with 0xFF)
  • Use stripe.py to split the ROM into Low and High words.
  • Program some EPROMs!
If that's too much like hard work, here's an srecord one-liner to combine the RO3.11 and 5th-column ROMs and add 0xFF padding (which most EPROM programmers will skip):

Code: Select all

srec_cat \
  RO311 -Binary \
  -fill 0x00 0 0x3F0000 \
  5thcolumn/rom.bin -Binary -Offset=0x3F0000 \
  -Output EXT -Binary
If you want to make a larger 5th Column ROM, change the parameters in config.yml and recalculate the offset (should be 0x400000 less the length of the 5th Column ROM).

You'll still need to use other tools (like qUE and Flibble's "Roll your own RISC OS" from viewtopic.php?f=29&t=14164) if you want to remove or replace RISC OS core modules like ADFS, but this is a great option for adding new things to the ROM. Especially as it nicely sidesteps the ROM checksum bug.

If you want to add things to ResourceFS, you don't need to patch the Messages module. All you need is a simple module which contains a resource file block (see https://www.riscosopen.org/wiki/documen ... isterFiles) and some ResourceFS calls.
The RISC OS 3.60 "Messages" module is a nice simple example: https://gitlab.riscosopen.org/RiscOS/So ... e/RO_3_60/

Obviously this only applies if you want to add things to ResourceFS. If you want to remove them, you'll need to patch the Messages module.

I'm also not entirely sure whether ResourceFS will access the files in-place in ROM, or if it will copy them to RAM. I'm sure someone here knows which option ResourceFS takes... but for now, it might be an idea not to bake a copy of ArcElite or SparkFS into the ROM...

Enjoy!
Phil

steve3000
Posts: 2284
Joined: Sun Nov 25, 2012 12:43 am
Contact:

Re: The Hidden Fifth Column: Integrating Wizzo with RISC OS 3.11

Post by steve3000 » Mon Jul 13, 2020 7:21 pm

Great work/investigations! This sounds like a very simple way of adding new modules to RISC OS ROMs without all the pain of rolling the whole RISC OS back together.

I’m assuming this works...it certainly sounds like it should? (I don’t think you actually say whether it does though?)

As for replacing RISC OS core modules such as ADFS, what happens if you just put the replacement module in there as an extension module? I would expect RISC OS initialises extension modules after core modules, so may well just replace the core module with the newer version provided as an extension module?

And to your question about ResourceFS, I looked at this many years ago, and am sure it runs from ROM without copying anything into RAM...

philpem
Posts: 569
Joined: Fri Apr 04, 2014 6:42 pm
Contact:

Re: The Hidden Fifth Column: Integrating Wizzo with RISC OS 3.11

Post by philpem » Mon Jul 13, 2020 7:52 pm

steve3000 wrote:
Mon Jul 13, 2020 7:21 pm
Great work/investigations! This sounds like a very simple way of adding new modules to RISC OS ROMs without all the pain of rolling the whole RISC OS back together.

I’m assuming this works...it certainly sounds like it should? (I don’t think you actually say whether it does though?)
Oh it definitely works! In fact, here's my A4000 motherboard running Wizzo 3v15 from the "extension ROM":
2020-07-13 19.27.47.jpg
2020-07-13 19.27.41.jpg
steve3000 wrote:
Mon Jul 13, 2020 7:21 pm
As for replacing RISC OS core modules such as ADFS, what happens if you just put the replacement module in there as an extension module? I would expect RISC OS initialises extension modules after core modules, so may well just replace the core module with the newer version provided as an extension module?
I expect it'd initialise the old version of ADFS (in ROM), then the Podule module would detect the extension ROM and load the newer version. You'd just have to make sure the one in the Extension ROM had a higher version number.
steve3000 wrote:
Mon Jul 13, 2020 7:21 pm
And to your question about ResourceFS, I looked at this many years ago, and am sure it runs from ROM without copying anything into RAM...
Nice! Though I can't imagine it's terribly fast with ~400ns ROM access time. Though perhaps still better than CompactFlash?

Cheers
Phil.

User avatar
danielj
Posts: 8439
Joined: Thu Oct 02, 2008 5:51 pm
Location: Manchester
Contact:

Re: The Hidden Fifth Column: Integrating Wizzo with RISC OS 3.11

Post by danielj » Mon Jul 13, 2020 8:06 pm

Game. Changer. Great work!

User avatar
myelin
Posts: 954
Joined: Tue Apr 26, 2016 10:17 pm
Location: Mountain View, CA, USA
Contact:

Re: The Hidden Fifth Column: Integrating Wizzo with RISC OS 3.11

Post by myelin » Tue Jul 14, 2020 1:18 am

Well, that's interesting! Excellent discovery!

I had assumed that extension ROMs would have to live in the "low rom" section, between 0x3400000 and 0x37FFFFF, i.e. where the IOEB puts them on the A5000, but you've just confirmed that RISC OS does indeed check the *entire* ROM space.
SW/EE from New Zealand, now in Mountain View, CA, making BBC/Electron hardware projects for fun.
Most interesting: Arcflash, FX2+PiTubeDirect Tube/Cartridge adapter, USB cart interface.

User avatar
danielj
Posts: 8439
Joined: Thu Oct 02, 2008 5:51 pm
Location: Manchester
Contact:

Re: The Hidden Fifth Column: Integrating Wizzo with RISC OS 3.11

Post by danielj » Tue Jul 14, 2020 7:14 am

Phil, are you using 27C160s? And what speed?

d.

philpem
Posts: 569
Joined: Fri Apr 04, 2014 6:42 pm
Contact:

Re: The Hidden Fifth Column: Integrating Wizzo with RISC OS 3.11

Post by philpem » Tue Jul 14, 2020 11:11 am

myelin wrote:
Tue Jul 14, 2020 1:18 am
Well, that's interesting! Excellent discovery!

I had assumed that extension ROMs would have to live in the "low rom" section, between 0x3400000 and 0x37FFFFF, i.e. where the IOEB puts them on the A5000, but you've just confirmed that RISC OS does indeed check the *entire* ROM space.
To be fair, the Podule module source code and the 5thColumn document both say that RISC OS scans the whole ROM space :)

The only time I had this fail was when I had the 5thColumn ROM immediately following RISC OS. That was before I fixed some bugs in the 5th-column ROM linker -- so the reason for failure could have been those issues. I have more EPROMs on the way, so I'll likely be doing more tests.

Cheers
Phil

Kazzie
Posts: 1793
Joined: Sun Oct 15, 2017 8:10 pm
Location: North Wales
Contact:

Re: The Hidden Fifth Column: Integrating Wizzo with RISC OS 3.11

Post by Kazzie » Tue Jul 14, 2020 12:03 pm

So this week it seems I'm learning about Extension ROMs. :)

The document linked in the OP states:
The availability, size and number of extension ROM sockets depends on which type of RISC OS computer you are using.
which is a politician's answer if ever I saw one. :roll:

Having gone digging further, am I right in saying:
  • Some machines (A5000?) have an extra socket on the motherboard for fitting an extension ROM
  • The clever thing here is using extra-large ROMs (on boards that have large enough sockets) to put an extension ROM alongside the regular RISC OS ROM
  • This method will work for any machine with 32-pin sockets (e.g. my A420/1), or a suitable ROM carrier board fitted?
BBC Model B 32K issue 7, Sidewise ROM board with 16K RAM
Archimedes 420/1 upgraded to 4MB RAM, ZIDEFS with 512MB CF card
RiscPC 600 under repair
Acorn System 1 home-made replica

User avatar
IanS
Posts: 1413
Joined: Mon Aug 31, 2009 7:02 pm
Contact:

Re: The Hidden Fifth Column: Integrating Wizzo with RISC OS 3.11

Post by IanS » Tue Jul 14, 2020 1:00 pm

Kazzie wrote:
Tue Jul 14, 2020 12:03 pm
Having gone digging further, am I right in saying:
  • Some machines (A5000?) have an extra socket on the motherboard for fitting an extension ROM
  • The clever thing here is using extra-large ROMs (on boards that have large enough sockets) to put an extension ROM alongside the regular RISC OS ROM
  • This method will work for any machine with 32-pin sockets (e.g. my A420/1), or a suitable ROM carrier board fitted?
Only the A5000 and A4 had a built-in 5th column rom socket, which were both only 8 bit, so code couldn't execute in-place.
On machines with 4 rom sockets (A3xx/A4xx) you would need to connect and extra address line to access the upper area of the larger roms (LA21). Depending on the carrier board this can be quite easy or may require track cuts.. The A540 already has a link for feeding LA21 into pin 1 of the roms sockets.

User avatar
IanJeffray
Posts: 300
Joined: Sat Jun 06, 2020 3:50 pm
Contact:

Re: The Hidden Fifth Column: Integrating Wizzo with RISC OS 3.11

Post by IanJeffray » Tue Jul 14, 2020 4:05 pm

So a little "loader module" in ExtnROM space, that scooped up a module chain that was crafted in to the wasted "2MB-64K" space and OS_Module 10'd them, would be nice. Any reason that sort of plan wouldn't work?

User avatar
IanJeffray
Posts: 300
Joined: Sat Jun 06, 2020 3:50 pm
Contact:

Re: The Hidden Fifth Column: Integrating Wizzo with RISC OS 3.11

Post by IanJeffray » Tue Jul 14, 2020 4:26 pm

I also wrote a simple RISC OS utility a very long time ago, to create modules that contain ResourceFS files, just as Phil suggested. http://ian.jeffray.co.uk/riscos/resrom114.zip

User avatar
IanS
Posts: 1413
Joined: Mon Aug 31, 2009 7:02 pm
Contact:

Re: The Hidden Fifth Column: Integrating Wizzo with RISC OS 3.11

Post by IanS » Tue Jul 14, 2020 4:49 pm

IanJeffray wrote:
Tue Jul 14, 2020 4:26 pm
I also wrote a simple RISC OS utility a very long time ago, to create modules that contain ResourceFS files, just as Phil suggested. http://ian.jeffray.co.uk/riscos/resrom114.zip
What OS is this expected to work on?
Didn't seem happy on 3.71 in RPCemu.
Attachments
ResRom.png
ResRom.png (4.91 KiB) Viewed 1242 times

User avatar
IanJeffray
Posts: 300
Joined: Sat Jun 06, 2020 3:50 pm
Contact:

Re: The Hidden Fifth Column: Integrating Wizzo with RISC OS 3.11

Post by IanJeffray » Tue Jul 14, 2020 5:10 pm

What OS is this expected to work on?
Yoiks. It was most recently tweaked for Select (back in '02 !)

I've just tested this on Select 3i4 (4.39), 3.71, and 3.10 all on real hardware -- no issues on any. Did you do anything in particular before this exploded so violently? It's a fairly trivial bit of BASIC. :?

[Edit: Also tested with RPCemu with 3.71 and 4.39, no issues, and with VirtualRPC-SA, with 3.71, 4.02, 4.39, no issues... no idea what's upset things for you @IanS :? ]
Last edited by IanJeffray on Tue Jul 14, 2020 5:35 pm, edited 1 time in total.

steve3000
Posts: 2284
Joined: Sun Nov 25, 2012 12:43 am
Contact:

Re: The Hidden Fifth Column: Integrating Wizzo with RISC OS 3.11

Post by steve3000 » Tue Jul 14, 2020 5:29 pm

IanJeffray wrote:
Tue Jul 14, 2020 4:05 pm
So a little "loader module" in ExtnROM space, that scooped up a module chain that was crafted in to the wasted "2MB-64K" space and OS_Module 10'd them, would be nice. Any reason that sort of plan wouldn't work?
Yes I was thinking whether something along those lines might be possible (if OS_Module 10 will accept ROM area addresses...can’t see why not?). Because if you did this, the extra modules could execute from ROM rather than being copied into RAM as ExtensionROMs are...

User avatar
IanJeffray
Posts: 300
Joined: Sat Jun 06, 2020 3:50 pm
Contact:

Re: The Hidden Fifth Column: Integrating Wizzo with RISC OS 3.11

Post by IanJeffray » Tue Jul 14, 2020 5:40 pm

steve3000 wrote:
Tue Jul 14, 2020 5:29 pm
IanJeffray wrote:
Tue Jul 14, 2020 4:05 pm
So a little "loader module" in ExtnROM space, that scooped up a module chain that was crafted in to the wasted "2MB-64K" space and OS_Module 10'd them, would be nice. Any reason that sort of plan wouldn't work?
Yes I was thinking whether something along those lines might be possible (if OS_Module 10 will accept ROM area addresses...can’t see why not?). Because if you did this, the extra modules could execute from ROM rather than being copied into RAM as ExtensionROMs are...
I wasn't sure how/what would map that bit of ROM in to MEMC, that's all. Whether the whole 12MB area (0x3400000 to 0x3ffffff) was considered mapped-in by then or what. My RO knowledge doesn't go quite that deep.

User avatar
IanS
Posts: 1413
Joined: Mon Aug 31, 2009 7:02 pm
Contact:

Re: The Hidden Fifth Column: Integrating Wizzo with RISC OS 3.11

Post by IanS » Tue Jul 14, 2020 10:50 pm

IanJeffray wrote:
Tue Jul 14, 2020 5:10 pm
What OS is this expected to work on?
Yoiks. It was most recently tweaked for Select (back in '02 !)

I've just tested this on Select 3i4 (4.39), 3.71, and 3.10 all on real hardware -- no issues on any. Did you do anything in particular before this exploded so violently? It's a fairly trivial bit of BASIC. :?

[Edit: Also tested with RPCemu with 3.71 and 4.39, no issues, and with VirtualRPC-SA, with 3.71, 4.02, 4.39, no issues... no idea what's upset things for you @IanS :? ]
I tried it on real hardware, using RISC OS 3.11, and it works fine, so it must be something odd about my RPCemu. I did try it a few times on RCPemu, but it's not the latest version, so probably safe to ignore my bug report. Sorry.

I'm thinking I can use it to include !hform and partitioning tools on ROM for my ide podules. It will use ram (only an 8-bit rom), but the module can always be unplugged when not needed. Looks very useful. Thanks.

sirbod
Posts: 1133
Joined: Mon Apr 09, 2012 9:44 am
Location: Essex
Contact:

Re: The Hidden Fifth Column: Integrating Wizzo with RISC OS 3.11

Post by sirbod » Wed Jul 15, 2020 6:27 am

steve3000 wrote:
Tue Jul 14, 2020 5:29 pm
IanJeffray wrote:
Tue Jul 14, 2020 4:05 pm
So a little "loader module" in ExtnROM space, that scooped up a module chain that was crafted in to the wasted "2MB-64K" space and OS_Module 10'd them, would be nice. Any reason that sort of plan wouldn't work?
Yes I was thinking whether something along those lines might be possible (if OS_Module 10 will accept ROM area addresses...can’t see why not?). Because if you did this, the extra modules could execute from ROM rather than being copied into RAM as ExtensionROMs are...
That is how extension ROM's are handled if they're 32bit, there's no need for extra code. The Kernel just runs through the Modules chain and issues OS_Module 10 - identical to how it inserts its own ROM Modules.

If you add the extension ROM to the end of the OS ROM, Modules will be execute in place and if you create a dedicated 5th column ROM they'll be copied to RAM, as its 8 bit.

This obviously means you need to ensure any Modules added via an embedded extension ROM are ROM safe, or insert them into RAM via a !App in a host ResourceFS Module. This is exactly how RISC OS 3.20 handles non-ROM safe Modules, such as VProtect, CallASWI etc

steve3000
Posts: 2284
Joined: Sun Nov 25, 2012 12:43 am
Contact:

Re: The Hidden Fifth Column: Integrating Wizzo with RISC OS 3.11

Post by steve3000 » Wed Jul 15, 2020 4:30 pm

sirbod wrote:
Wed Jul 15, 2020 6:27 am
That is how extension ROM's are handled if they're 32bit, there's no need for extra code. The Kernel just runs through the Modules chain and issues OS_Module 10 - identical to how it inserts its own ROM Modules.
Ah ok, that makes a lot of sense. So no need to create as a 5th col. ROM unless there’s a need to run the module from RAM. :)

sirbod
Posts: 1133
Joined: Mon Apr 09, 2012 9:44 am
Location: Essex
Contact:

Re: The Hidden Fifth Column: Integrating Wizzo with RISC OS 3.11

Post by sirbod » Thu Jul 16, 2020 5:22 pm

The RISC OS code that handles Module setup following POST can be viewed here

The order of Module initialisation is as follows:
  1. Initialise Podule Module (which must immediately follow UtilityModule in the ROM)
  2. Add ROM Modules to the Module node list
  3. Add Podule hosted Modules to the Module node list
  4. Add Extension ROM Modules to the Module node list
  5. Initialise the latest version of Modules in the node list that aren't unplugged
Podule and non-32bit extension ROM Modules are copied to the RMA before being initialised.

User avatar
IanS
Posts: 1413
Joined: Mon Aug 31, 2009 7:02 pm
Contact:

Re: The Hidden Fifth Column: Integrating Wizzo with RISC OS 3.11

Post by IanS » Wed Aug 26, 2020 1:44 pm

Using one of these viewtopic.php?p=267437#p267437 the resrom tool viewtopic.php?p=278763#p278763 I built a wizzo podule for my A4000, and included the IDE format tool. So I don't have to load the wizzo modules from disk. Unfortunately it uses up the only podule slot.

Duiring the Abug call last night we got discussing being able to add a rom to the A4000 (or A30x0) without using the podule slot.

Based on the information in this thread it should be possible to add an 8-bit 5th column rom into the usual rom space in the memory map above the usual roms. If I make an adapter that replaces the lower rom (data bits 0-15), uses LA21 to decide if the existing rom or an 8-bit eprom/flash chip appears in the upper area, would that work?

Can anyone see an issue with that?

User avatar
IanJeffray
Posts: 300
Joined: Sat Jun 06, 2020 3:50 pm
Contact:

Re: The Hidden Fifth Column: Integrating Wizzo with RISC OS 3.11

Post by IanJeffray » Wed Aug 26, 2020 4:24 pm

Is that faff better than just making some 4MB ROMs? According to this viewtopic.php?f=16&t=20295#p284691 Phil confirmed this works, and I was about to figure out what ICs I need to try the exact same for myself - I'm surprised if it's as easy as just plonking bigger ROMs in place.

OTOH, I do rather like keeping things as original as possible - so an adaptor to add 5th column ROM capability to existing machines and allow use of the original ROMs would be rather cute. The A4 schematics are available and clearly show how the 5th col ROM is connected up with two size options controlled via a link.
5thColA4.PNG
A4 ROM section

sirbod
Posts: 1133
Joined: Mon Apr 09, 2012 9:44 am
Location: Essex
Contact:

Re: The Hidden Fifth Column: Integrating Wizzo with RISC OS 3.11

Post by sirbod » Fri Aug 28, 2020 2:44 am

I'd use a larger ROM and embed an Extension ROM at the end, provided the Modules are ROM safe. If they're not ROM safe, I'd embed them into a Resource Module that RMLoads them - which is what I did in RISC OS 3.20.

5th column is 8 bit, so will always load the Modules into the RMA. Real world 5th column use is clearly limited to the few machines that support it. Recreating it on other hardware although interesting, sounds like a lot of effort when Extension ROM's have been proven to work.

philpem
Posts: 569
Joined: Fri Apr 04, 2014 6:42 pm
Contact:

Re: The Hidden Fifth Column: Integrating Wizzo with RISC OS 3.11

Post by philpem » Tue Sep 01, 2020 3:16 am

IanJeffray wrote:
Wed Aug 26, 2020 4:24 pm
Is that faff better than just making some 4MB ROMs? According to this viewtopic.php?f=16&t=20295#p284691 Phil confirmed this works, and I was about to figure out what ICs I need to try the exact same for myself - I'm surprised if it's as easy as just plonking bigger ROMs in place.

OTOH, I do rather like keeping things as original as possible - so an adaptor to add 5th column ROM capability to existing machines and allow use of the original ROMs would be rather cute. The A4 schematics are available and clearly show how the 5th col ROM is connected up with two size options controlled via a link.

5thColA4.PNG
ST Micro M27C160-100 are what you want. That's 100ns, 16 MBit per chip, for a total of 32 MBits (4 MiB).

If you want to program a "stock" 2MB image, you want the M27C800-100F1.

Cheers
Phil.

Post Reply

Return to “32-bit acorn software: other”