Roll your own RISC OS 3.1x

discuss general risc os software applications and utilities
Related forum: adventures


Post Reply
User avatar
flibble
Posts: 770
Joined: Tue Sep 22, 2009 11:29 am
Contact:

Roll your own RISC OS 3.1x

Post by flibble » Tue Dec 05, 2017 12:11 am

Following on the discussions in the 'RISC OS 3.20' thread I thought I'd relearn how to use some very old programs and thought people might appreciate some 'tutorial' style instructions.


Roll your own RISC OS image


These are some basic instructions that will hopefully allow you to roll your own custom RISC OS ROM image for 3.1x era machines. It doesn't discuss burning ROMs for use in real hardware as I have never taken it that far.

Why?

Mainly, just because. You can add updates from Acorn and other companies that would normally need 'softloading' (aka loading into RAM), thus saving memory and fixing some of the bugs. You can also add your own apps to the ROM, or even just customise it with your own sprites. You could have an idea for a custom embedded use for a machine, this will allow you to ship a ROM only version.

Requirements
  • A RISC OS 3.1x ROM image, 3.11 will be used in these examples. https://4corn.co.uk/aview.php?sPath=/roms
  • Two instances of an emulator capable of running RISC OS 3.1x, one will be used for building (the build machine), one used for testing (the test machine). HostFS will be incredibly useful here, so either RedSquirrel or Arculator (if compiling from source).
    I used arculator in this example.
  • Software on the build machine
  • Ideas for what you want to add to the ROM
    • Official Acorn updates such as Newlook, the universal boot, !System updates etc
    • Your own or someone else's apps so they are always available.
    • Third party modules
    • Changing the version number so you can be the first to run RISC OS 7
Caveats

Binary munging ROM images like this is prone to making mistakes, you'll find some software and modules are not capable of being placed in ROM (e.g anything that uses temporary disc files relative to its install, or anything that writes to its own module space as workspace). Certain modules have cross dependencies that need to be figured out. You have a hard constraint on the size of the ROM, so sometimes you have to pick what to leave out when you add your own stuff.

Though probably the biggest caveat is that this has not actually been overly tested and certainly not on real hardware. It should work, but do not use one time programmable ROMs until you test it.

However if you find bugs in the builder or shortcomings, please report them as we may be able to fix them.

Phase 1 - RISC OS 3.11 with an extra ROM module

Setup an emulator image with RISC OS 3.11 for the build machine. Install !Sparkplug, optionally install !Zap.

Unzip ROMBUILD
1.png
Run 'LocalDir' to set the working directory to the ROMBuilder directory.
Run 'CreateList' to create a manifest for your new ROM based on the contents of the RISC OS 3.1 ROM you're running, in the file 'ModuleList'
2.png
As you can see 'ModuleList' contains a list of entries to put in the ROM and the place the 'Build' program should load them from.

The choices for places are
ROM: The module from the running version of RISC OSs ROM image
RMA: The module from RAM having been loaded from disc earlier (I don't use this option really)
Disc: A named module in the 'Modules' subdirectory of the 'ROMBuilder' directory
Bootstrap, taildata, SET: don't worry about these yet, we'll come back to them

Add a module,

For this example I'm going to use the OSVersion module, it changes the output of *FX0 to a custom string, which should make it nice and obvious that we have a new ROM.

https://www.marutan.net/download/riscos/osver.zip

Download to the build machine, optionally edit the version number string in 'MakeMod' and rerun it, place the OSVersion module built into the 'ROMBuilder.Modules' directory.
3.png
Next edit the 'ModuleList' manifest file, and add this module into the image at the end, but before the SET: and taildata: section.
4.png
Now run 'Build' to make your ROMImage, this creates the file 'ROMImage' in the 'ROMBuilder' directory.
5.png
Copy this 'ROMImage' file out of your build machine, via hostfs and place it in the ROMs directory of your emulated test machine, boot up and cross your fingers.
6.png
Pat yourself on the back, you've just built your first ROM.

BUG: ROM Image generated is 2049Kb in size not 2048Kb, investigating.

Phase 2 Changing resources in ResourceFS
Manu newer RISC OS modules also need newer resources (such as templates, messages files, icons) to work properly, these are stored in ResourceFS. ResourceFS is the files you can see when you middle click over 'Apps' and click 'Open '$''.

You can add or remove to these files by editing the 'Messages' module, to edit it use the Messages program.
messages.zip
(134.9 KiB) Downloaded 65 times
Unzip this file in your host operating system direct into the hostfs directory of your emulator.

To run the Messages program, we also need a copy of the CallASWI, SharedUnixLib and FPEmulator modules to be available.
message_support.zip
(16.84 KiB) Downloaded 78 times
Also unzip this to the hostfs directory of your emulator and double click to load them ... this is because I'm being lazy and don't want to setup a full boot sequence for this build machine. You'll need these module loaded every time you reboot and want to run the messages programs
2.1.png
Next we need a copy of the RISC OS 3.11 Messages module to work with, using !Zap, middle click on the iconbar icon and select 'Create' -> 'Get Module' and pick 'Messages' from the list. Save a copy of the module from Zap to the messages directory.

Whilst you're looking at the module code we need to determine what the r0 argument is to the SWI XResourceFS_RegisterFiles and XResourceFS_DeregisterFiles. It's not very obvious in the 3.11 module, but it should be E0 in hex.

This offset is where in the module the chain of files stored starts and differs between versions of the Messages module.
2.2.png
Enter the messages directory, double click 'ThisDir+', create a directory in the messages directory called 'Working'
2.3.png
Start a task window and enter the following

*read Messages e0
2.4.png
This takes a while to run but afterwards, in the directory 'Working' you will have the directory tree of all the files in the Messages module.
2.5.png
Making a change

For this example, we will try something very very simple, we'll create a basic file and save it to the top of the resource tree
2.6.png
Rebuilding the Messages module

Again, from a task window, make sure you're in the messages directory and enter the following

*write
2.7.png
This creates a new module called 'Output', copy this file to the ROMBuilder.Modules directory and rename it to Messages.

Edit the ROMBuilder.ModuleList file and change the entry for Messages from ROM: to Disc:, to use our newly created version.
2.8.png
Then build the romimage in the same way we did in Phase 1. Double click 'LocalDir' and double click 'Build'

Again copy the romimage out to your test emulator and run.
2.9.png
Why this is almost competent

Phase 3 Adding lots more and trying to be clever

Coming Soon!

steve3000
Posts: 2246
Joined: Sun Nov 25, 2012 12:43 am
Contact:

Re: Roll your own RISC OS 3.1x

Post by steve3000 » Tue Dec 05, 2017 12:44 am

Ooh very nice :)

Do the build tools include calculating the ROM checksums and a tool to generate 'striped' 8 bit or 16 bit ROM image files for burning? (The later is simple to write if not...)

User avatar
flibble
Posts: 770
Joined: Tue Sep 22, 2009 11:29 am
Contact:

Re: Roll your own RISC OS 3.1x

Post by flibble » Tue Dec 05, 2017 2:09 am

steve3000 wrote:Do the build tools include calculating the ROM checksums and a tool to generate 'striped' 8 bit or 16 bit ROM image files for burning? (The later is simple to write if not...)
I don't believe there's checksum code in there. I think the Striped versions are made from the programs SplitPage and SplitWord are for, but have never tested.

User avatar
flibble
Posts: 770
Joined: Tue Sep 22, 2009 11:29 am
Contact:

Re: Roll your own RISC OS 3.1x

Post by flibble » Wed Dec 06, 2017 10:26 pm

now with Phase 2 instructions in original post.

philpem
Posts: 439
Joined: Fri Apr 04, 2014 6:42 pm
Contact:

Re: Roll your own RISC OS 3.1x

Post by philpem » Sun Jul 12, 2020 4:27 pm

Holy necrothreads, Batman!

I've been poking at the "RISC OS 3.20" project which seems to have similar goals to this. To that end, I've put together some tools to pad the ROM up to 4MB and fix the checksums and ROM size in the header:
https://github.com/philpem/riscos-romtools/

I'm getting a couple of errors at the moment:
  • After adding the Percussion module, there's a "Cannot fill" error when "Filling for set address &3978718" before adding SharedCLibrary.
  • After adding the NetUtils module, there's a second "Cannot fill" error for address &39FECA0 before the Taildata.
This is with a fresh RO3.11 boot, and no modifications to the ModuleList. The ROM built is not identical to the original RO3.11 ROM. The difference is in the Taildata -- the output ROM has an extra 0x0A byte before it.

The bug turns out to be in the way the Build program terminates the module chain:

Code: Select all

PRINT"* Terminating Module chain":BPUT#fileout,STRING$(4,CHR$(0))
There are two occurrences of this, which need to be patched as follows:

Code: Select all

PRINT"* Terminating Module chain":FOR tby%=1 TO 4:BPUT#fileout,0:NEXT tby%
There's probably a better way to fix this, but I'm not a BBC BASIC expert...

A further bug is that the Taildata is one byte short. Again, easy patch. Look out for this chunk of code:

Code: Select all

ENDWHILE:tailsize-=((workspace-&3800000)+1)
SYS"OS_GBPB",2,fileout,workspace,tailsize
And change it to:

Code: Select all

ENDWHILE:tailsize-=(workspace-&3800000)
SYS"OS_GBPB",2,fileout,workspace,tailsize
If both these bugs are fixed, Build produces a perfect copy of the RISC OS 3.11 ROM.

The Taildata, as it happens, is the credits data for the "TEAM" easter-egg in RO3.11. It's in the following format:
  • Every name (or rather the contents of the text box) is terminated with a linefeed (0x0A byte).
  • The entire list is terminated (after the final linefeed) with a null byte.
  • There are three copies of the data in the ROM, though this may not be terribly important.
Looking at the RO3.6 "Switcher" source code on ROOL Gitlab (https://gitlab.riscosopen.org/RiscOS/So ... s/Switcher) and an Armalyser disassembly of the TaskManager module, OS_Module 19/18 are used to find the end of the module chain. TaskManager assumes the credits taildata (heroes list) follows immediately afterwards.
I've no idea how this was appended to the RO3.6 ROMs, Bigsplit2 doesn't seem to do it, and it's hard to check out the RO3_60 branch now ROOL have moved to Git.

If you want to get rid of the hero list, pull the Taildata from a file, and have that file contain a single null byte. I'd use a 4-byte word of zero for safety's sake. This will free up about 4K of ROM for you.

Cheers
Phil.

philpem
Posts: 439
Joined: Fri Apr 04, 2014 6:42 pm
Contact:

Re: Roll your own RISC OS 3.1x

Post by philpem » Sun Jul 12, 2020 4:55 pm

One more for the road. Here's my patched version of Build, with the following changes:
  • Fix the Stray-0x0A bug when adding the Module Trailer (which broke binary reproducibility)
  • If no filler is required, indicate that as a comment instead of throwing an error.
  • If the ROM address has gone past the specified filler address, Build will STOP and tell you how many bytes you need to free up.
I've also thrown in two Taildata files:
  • NoHero contains a short "sorry" message in place of the Heroes list.
  • ShortHero contains only one copy of the Heroes list (in case you can't live without the easter egg)
It's not necessary to force the address of the Taildata, you can remove the SET command immediately before it. Allowing SharedCLibrary to move is probably a bad idea!

Once you have your "ROMImage" file, grab my tools from Github: https://github.com/philpem/riscos-romtools/
Run 'make' to build Patchsum and Checksum.
Use 'pad4mb.sh' to pad the ROM to the 4MB (edit the script beforehand if you want to pad to 2MB), then patch the checksum with Patchsum.
Try your image out under Arculator.
Split the ROM image in two, and program a pair of 27C800 (2MB) or 27C160 (4MB) EPROMs with your custom image.

Round 3, patching in the new ADFS and Wizzo...

Cheers
Phil

EDIT 13 Jul - replaced ZIP file, it was corrupt...
Attachments
build.zip
(3.02 KiB) Downloaded 6 times
Last edited by philpem on Mon Jul 13, 2020 11:33 am, edited 1 time in total.

philpem
Posts: 439
Joined: Fri Apr 04, 2014 6:42 pm
Contact:

Re: Roll your own RISC OS 3.1x

Post by philpem » Mon Jul 13, 2020 2:28 am

Another bug. The padding logic is completely wrong (specifically, it's backwards). When fixed, it creates an empty Module which causes an address exception crash on boot. The module is full of NOPs, which are out-of-range addresses in the context of a module header.

The workaround is to pad any modules you're replacing (e.g. ADFS) to the length of the existing module.

I can think of a few ways to fix this:
  • Modify Build (again) to add the padding onto the end of the preceding module, then backtrack and patch its module length. This seems like the cleanest solution.
  • Check there are at least 44 bytes of padding needed (per http://www.riscos.com/support/developer ... dules.html - module header) and fill with zeroes instead. Cross fingers that RO doesn't kick up a fuss about a title string.
  • Do the second if there's space, otherwise do the first (limiting the added length of the module, though this won't matter as it's running from ROM)
For now, I've done the second, without the length check. It's as dumb as bricks, but it means I can shove the patched ADFS 2.68 in (replacing RO3.11's 2.67) without having to pad the module up to the length of the old one.


There's also a second buglet -- Modules need to start on a 4-byte boundary, otherwise RISC OS will crash when executing them. This is the same rule as applies to 32-bit wide 5th Column ROMs which execute-in-place.
If you try to integrate Wizzo 3v15 (for example) - you'll find the IDEFS Filer needs 2 null bytes tacking on the end to make its length a multiple of 4 bytes. Ideally Build should do this automatically, perhaps with a warning.

Disappointingly I still haven't been able to figure out why 4MB ROMs don't generate correct checksums during selftest. It won't stop the machine booting, but it will add an annoying delay while the machine blinks out the error codes. Arculator will sit for what seems like forever on the green "POST success" screen but will carry on booting.
Something which happens later in boot is clearly fixing this, because *Save and the Podule module (5th Column ROM support) can access the second 2MB half perfectly fine.


I'm probably going to park this for a while -- it turns out you can install a 4MB EPROM set in an A4000, and put a 5th Column ROM in the second half. This turns out to be a far easier way to add things to RO3.11 than picking apart the module chain. That is to say, unless you want to replace e.g. ADFS with a later version (2.68 seems popular -- remember to remove the ADFSUtils module).

I'll see about putting together a thread on tacking 5th Column ROMs onto the end of RISC OS. It's a fun little gig.

Cheers
Phil
Last edited by philpem on Mon Jul 13, 2020 10:35 am, edited 1 time in total.

User avatar
jgharston
Posts: 4039
Joined: Thu Sep 24, 2009 12:22 pm
Location: Whitby/Sheffield
Contact:

Re: Roll your own RISC OS 3.1x

Post by jgharston » Mon Jul 13, 2020 10:19 am

The "disc:" prefix really should be "file:". Functionality not implementation and all that. I pull my files in from net: not from adfs: :)

Code: Select all

$ bbcbasic
PDP11 BBC BASIC IV Version 0.32
(C) Copyright J.G.Harston 1989,2005-2020
>_

User avatar
flibble
Posts: 770
Joined: Tue Sep 22, 2009 11:29 am
Contact:

Re: Roll your own RISC OS 3.1x

Post by flibble » Sat Aug 01, 2020 7:57 pm

Philpem: sorry for no replying earlier, but thank you for all the bug reports and testing, I've read them, will hack them in, but I've been distracted by other things.

I think the project as a whole required me to do a little bit more work on the C linked modules (paint, draw, edit, filer action), for the first 3 you can use disc based versions from 3.5, but filer action was always an arse. I think the hack of setting the position of the CLIB is only half the requirement, you also need to fix the location of clib linked module also, as I think it contains backwards relative pointers.

Also I note on another thread you've managed to put the 5th column rom on the end of the regular rom image, this is a very clever hack!

Post Reply

Return to “32-bit acorn software: other”