A new MENU system for MMC/Gotek

[tricky]

######################################################################################################
EDIT: latest versions of GOTEK.ZIP and BEEB.ZIP here via DropBox, no login should be required.
GOTEK.zip/GOTEK should be extracted to the root of your USB stick and contains an FF.CFG and a bunch of DSKA0000.ssd files.
BEEB.zip should be extracted to the root of your storage card and contains BEEB.MMB which should work with any MMC type device.
Backup you files first and please note that FF.CFG will set indexed mode, you can set this back on a different stick for file/folder browsing.
I have added a sound toggle on TAB, this is represented in the bottom right by <) sound on and <X sound off.
######################################################################################################

GOTEK.zip/GOTEKmenu.ssd contains some GOTEK utilities:
*DIN (drive) image_number, is for inserting a disc image into a drive.
*DBOOT image_number, should insert the disc image and then try to boot it like SHIFT-BREAK.
*DBOOT0, is a shortcut for *DBOOT 0 and is what is used as !BOOT on all the discs in the menu except the first.

I've started work on a new menu system for the beeb, the idea was to hopefully make it easier to add and remove games than the Stairway to Hell archive, whilst still keeping the great functionality and also supporting GOTEKs using FlashFloppy (once the relevant FF update is complete).

I have a couple of basic C++ programs (will unify later), one that reads the Something Special discs and extracts which games that are on there but not much else. The other parses a dump of pages from bbcmicro.co.uk, extracting the game info and downloading the .SSDs and creating a .CSV.

I have also written (again basic C++) a program to read the .CSV and .SSDs and merge the files onto a set of game discs that will be copied to MMC or GOTEK (and maybe later DataCentre if required). It also produces encoded names and meta data which is then compressed and currently combined with a tiny program to extract them a page at a time, although this will be replaced by a proper menu along the lines of the STH one later.

There are currently around 4,000 games on bbcmicro.co.uk, which combine to about 800 .SSDs and about 50KB of titles and indexing etc (which compresses to about half that still allowing random access). The number of discs could be reduced, but I start a new one if I find a duplicate file name (mostly because of duplicate games).

I am attaching the sample program that just dumps the titles (+ publisher for duplicated names) to the screen, it currently uses OSWRCH, but could be sped up considerably if required.

The input format (on the PC) is currently: Title,Extra,Sort,SSD per line where Extra is displayed if to games have matching Titles. Sort is used to sort the entries and also to decide when the first letter changes to allow jumping in the final menu, this allows "The..." to not be in the Ts if required.

To build your own menu, see this post and the links to downloads viewtopic.php?f=2&t=16070&p=262481#p262481.

[pau1ie]

Cool! Are you aware of what we call the spreadsheet? That might make it easier to refresh the database. There is also a method to download all the game images and screenshots, but I can't remember the URL at present. I made a start on developing a desktop application for the website and was wondering if some kind of API would be beneficial, though as always I am concerned that I don't want it to be too much hassle to maintain. I have put that to one side for the time being. I look forward to seeing how this develops!

Edit: typo in url

[pau1ie]

The download link is archdl.php. it does take a long time to load if changes have been made to the site because it zips everything up. The archive format could be improved to match up with the spreadsheet better.

[tricky]

Thanks, that would have saved an hour
I do use a spreadsheet to edit the .CSV, but this way it is product agnostic
I can easily change the order of the columns to make it compatible with the downloaded one and already have a bunch of rules to trim most titles to fit in 35 characters.
I think that the main problem is having the user pick which titles to include. The categories and clones might come in handy here.

[tricky]

Ah sorry pau1ie, thanks, now that I have read your post properly, the spreadsheet makes much more sense.

[tricky]

I have included parsing the "spreadsheet" html but have hit strange OSRDCH behaviour.
If I */!BOOT (or *!BOOT) OSRDCH reads keyboard input as expected.
If I SHIFT-BREAK (with OPT 4 2 - *RUN) OSRDCH doesn't return.
I have tried many *FX/OSBYTE call combinations to "fix" this but I'm obviously missing something basic (or BASIC).
I have tried various combinations of:

Code: Select all

	lda #2  : ldx #0  : jsr OSBYTE ;;   2 select input stream *FX 2 0 kbd input, no RS423
	lda #3  : ldx #0  : jsr OSBYTE ;; *FX 3 0 !423 VDU printer(by VDU 2) spooled (by SPOOL) *FX 3 84 only VDU output (and VDU 1 next to printer)
	lda #4  : ldx #1  : jsr OSBYTE ;; *FX 4 1 crsr keys don't edit, COPY<>^v:&87..&8B
	lda #&7E : jsr OSBYTE ;; may not be needed

with various parameters, but before I start single stepping the OS, does anyone have any suggestions please?

[tricky]

Scratch that, it seems to be beebem (at least the version that I am using) related, the power on beep never ends either, so I hadn't fixed that either! I have so many versions of beebem with different hacks and fixes that I'm not sure which is which any more!
So, summary, I'm a dummy and wasted most of my morning!
Here is menu that doesn't launch anything, but does let you see what is on bbcmicro.co.uk - RETURN quits!
I haven't hooked up jumping to letters yet, but the cursor keys should work.

[tricky]

Now with 0 and A-Z to jump to page where that section begins.

[tricky]

After a few hick-ups and reflashing my GOTEKs via USB-serial, I have the menu working on them.
I can select a game and then have the GOTEK select the appropriate image and launch the game.
It should already work on MMC, now that I am calling OSWORD instead of OSBYTE having set up for OSWORD!


URL:https://youtu.be/o_PBs-tvWuw

[danielj]

Fantastic work - Keir's implemented all the selection stuff now?

d.

[tricky]

He has implemented selection by index and was talking about maybe adding dir and CD functionality

[danielj]

Ah, I see the issue thread . If he does that what would be nice is to be able to return index#, dir?, filename (similar to the beebscsi direct file access method) and then you could generate a menu system that just worked from the directory structure on the usb.

[ctr]

Scratch that, it seems to be beebem (at least the version that I am using) related, the power on beep never ends either, so I hadn't fixed that either!

If you use *OPT4,2 then your code is entered with interrupts disabled. You have to enable interrupts to stop the beep, get keyboard input, etc. I don't know if it's a feature of beebem or a particular DFS or what. But I've had the same issue.

[tricky]

Thanks, I never knew that

[tricky]

I have written a *DIN for the GOTEK using Flash Floppy v0.12

This will work best (or probably only) with

Code: Select all

nav-mode = indexed

in your FF.CFG
This requires images to be named DSKA0999.xxx where 999 is the disc number and xxx is either .SSD or .DSD (or whatever else is supported).

Code: Select all

*DIN (drive) image

The drive is optional and will use the current drive if omitted, image is a number from 0 to 999.

The current version of Flash Floppy (v0.12) doesn't update the 7 segment display after changing images, but it will update on the first access of the selected image.

To select disk, *DIN will seek to track 254, this may harm a real drive.
If you have both a GOTEK and a real drive connected and issue a *DIN that starts the real drive, I would CTRL-BREAK immediately.

The idea for *DIN is that it is compatible with the one used by "MMC" ROMS and should allow the MMC menu to work on a GOTEK - I haven't tested this yet. A GOTEK/FlashFloppy won't swap back to disk 0 on a CTRL-BREAK, so you will need to click both buttons together to change back or put a copy of *DIN on each disk.

DIN uses &70..&79 and &80..&8E inclusive for various work space although this may change later.
DIN is relocatable and doesn't care what it is called. To change it, *load and *save with a new load/exec address and name.

I am planning on creating a *DBOOT, which is basically a *DIN followed by whatever the *OPT 4 x is set to.
I am also going to create a !BOOT that when * RUN will *DBOOT 0. This is what I am going to use on my MENU system to get back to the main menu.

[jms2]

This is really excellent stuff, thanks for working on it!

I find the Gotek much easier to manage than the MMC, because you don't need any special software to add files to it. However the inability to use the games menu is a bit of a problem, so your development is something that I'm really keen on.

There are a couple of technical points that I'd like to understand a bit better:

To select disk, *DIN will seek to track 254, this may harm a real drive.

I assume this is the protocol for communicating with Flashfloppy - attempt to select the non-existent track 254?

The idea for *DIN is that it is compatible with the one used by "MMC" ROMS and should allow the MMC menu to work on a GOTEK - I haven't tested this yet.

Obviously for this to work in practice, the images on the Gotek will need to have the same numbering as those on the MMC card. To me this raises the perplexing question of how to manage hundreds of disc images and be able to add to them in future, rather than just have a static unchanging set. Adding new games to the menu on an MMC card is a real pain.

On my Gotek I have got folders for BBC, Electron and Master, so I can swap the same drive between different machines. This works fine, but I wonder if it is the best way to do it - especially if the STH collection is going to be included on there.

[tricky]

Yes, 254 is the single density access, 255 is the original double density.
There is talk of adding directory access.
I don't know if the access is limited to indexed mode, I didn't check. If not, you could pit the indexed part in root and the rest in folders.

[danielj]

It is, according to my reading of it, limited to indexed mode at the moment.

d.

[KenLowe]

It is, according to my reading of it, limited to indexed mode at the moment.

I've been doing a bit of testing, and nav-mode = default also seems to work fine.

The order in which the images are added to the USB drive sets its image number. I think it's down to the way the files are added into the FAT32 directory tables.

If you have an OLED display you'll see that there's a field xxx/yyy at the bottom left, where xxx is the currently selected image number (actual file name is shown above), and yyy is the total number of images on the USB. The xxx image number is what will be loaded if you select that number via the BBC menu.

You'll need to delete or hide any .hfe files though, as they prevent it from working correctly.

I prefer it this way, since it is possible to retain meaningful disk image file names that are displayed on the OLED (instead of using the DSKAxxxx.ssd format required by the indexed mode). However, the downside is that loading image xxx from the BBC will not necessarily load the same image on every system. It depends on the sequence in which the files were added to the USB disk.

[tricky]

...The order in which the images are added to the USB drive sets its image number. I think it's down to the way the files are added into the FAT32 directory tables...

Do you know if the files in sub-directories are counted from where the directory entry is, or if the directories are after the files in a directory?

As in, if you put the files that you wanted to access in order in the root folder with everything else in sub-directories and then added more root files later, would it still work?

Here is *DBOOT (drive) image

It hasn't had much testing and is as *DIN, except that it reads half of the catalogue into &A00..&AFF to read the two bits to determine how to handle !BOOT.

I was expecting an OSBYTE command to read the current value, but I can't find it. As long as you don't need the cassette/serial buffer until after the new disc has started to !BOOT, it should work.

If anyone has a simple legal way to do this I will take a look.

The current plan for the Menu is to create a folder (currently on windows) and copy all the required files in to it.

I've been trying to come up with a good way to select which images to put on the menu, but short of something fancy, it currently checks what is in the bbcmicro.co.uk "spreadsheet" or .CSV against what is in the gameing/discs/... folders and builds the menu from the intersection.

Games can be excluded by removing them from the "spreadsheet"/csv or from the gaming/discs folder.
Adding a new game, not in the "spreadsheet" would require adding a line to the CSV (best done with a spreadsheet).
Currently only discs that *EXEC their !BOOT are supported, (all bbcmicro.co.uk discs do AFAIK) but I will fix that at some point.

[tricky]

Menu is all working, but packing the games on to .SSD means that only 511 of the ~800 discs can be accessed (matches MMC though) so I will have to pack to .DSD and swap sides as needed.
I was hoping to avoid this to mean that whatever drive the menu was started on, it could stay on to avoid issues with floppy+gotek, but I might just force drive 0/2.

[KenLowe]

...The order in which the images are added to the USB drive sets its image number. I think it's down to the way the files are added into the FAT32 directory tables...

Do you know if the files in sub-directories are counted from where the directory entry is, or if the directories are after the files in a directory?

As in, if you put the files that you wanted to access in order in the root folder with everything else in sub-directories and then added more root files later, would it still work?

It doesn't seem to like folders particularly well. As a test I did the following with a freshly FAT32 formatted USB skick:

Code: Select all

Copy root1.ssd to the root
Copy root2.ssd to the root
Create folder A
Copy A1.ssd to folder A
Copy A2.ssd to folder A
Copy root3.ssd to the root

Then on the BBC, I tried to step through the images by increasing the file index (cmd 4, parameter 0) by 1 and then doing a *CAT, and the following would happen:

Code: Select all

Index = 0: Would load root1.ssd
Index = 1: Would load root2.ssd
Index = 2: Would load A1.ssd
Index = 3: Would load A1.ssd
Index = 4: Would load A1.ssd
Index = 0: Would load A1.ssd

Once I've navigated into folder A, I seem to get stuck there with the first image. I have to use the rotary knob on the Gotek to get out again. I wonder if disabling autoselect-folder-secs might change that behaviour. I'll give that a try and report back.

Edit: Setting autoselect-folder-secs=0 didn't seem to be make any difference.

[tricky]

I think Keirf mentioned something about only counting files in the root directory.
I wonder if it is stopping after the number of files in the root folder, but in the order they were added.

[KenLowe]

Ah. Ok. I could add some more files to the root, then, to see if I can get any further through the sub directory. I'll try that tomorrow.

[sweh]

Cool! Are you aware of what we call the spreadsheet?

Is there a CSV version of this?

I was thinking of writing (in my copious free time) a menu front end to MB's "30th anniv" disk collection. I did a cruddy version, based purely off the menus on each disk, but something similar to the old STH menu using the extended descriptions from your spreadsheet mapping back to the DSDs (I can use the disc name to identify the DSD, and the "ssd" name to map the relevant menu entry, so I can then CHAIN the right file).

If no CSV is available then don't worry; I can kludge the html into something usable (sed -n -e 's!,!!g' -e s!</td><td>!,!gp).

[tricky]

I may have tweaked a couple of bits, bbcmicro.zip attached.
I also have a program that parses the basic menu on each of the 30th discs to grab the data from there, but I have only checked it on a couple of discs and the spreadsheet has extra info.

[KenLowe]

I think Keirf mentioned something about only counting files in the root directory.
I wonder if it is stopping after the number of files in the root folder, but in the order they were added.

Added 4 more images to the root of the USB, but that didn't make any difference. Tried with autoselect-folder-secs set at 0 (disabled) and 2 (default). No difference.

[KenLowe]

It hasn't had much testing and is as *DIN, except that it reads half of the catalogue into &A00..&AFF to read the two bits to determine how to handle !BOOT.

I was expecting an OSBYTE command to read the current value, but I can't find it. As long as you don't need the cassette/serial buffer until after the new disc has started to !BOOT, it should work.

If anyone has a simple legal way to do this I will take a look.

Why don't you read it from the DFS workspace. According to the Advanced Disk User Guide, it's available at &0F06 bits 4 & 5. This of course assumes no other ROM has already claimed the workspace from &0E00. I'm sure there's probably a way to determine the start of the DFS workspace, and then it would be a case of adding &106 to it to get the correct data.

[tricky]

I had forgotten about the catalogue that is usually stored there.
I've just realised that in this case, the dfs won't have read the new catalogue, so reading half of it manually seems a reasonable solution, other than using the wrong buffer.

[tricky]

I have what should be a working GOTEK menu system of everything on bbcmicro.co.uk as of last week.
Files that have "STD" (Save To Disk - I assume) have their files writable, a spare catalogue entry and spare track on the image.
Images that don't STD have their .DSD files truncated to save USB space (.SSDs fitted in 64MB, but due to extra padding, the .DSDs don't).

A .zip of the archive is available for direct download from my DropBox (no login required AFAIK).
Download the zip at the top of the page and extract to a USB stick.

SHIFT-BREAK to boot the menu and then SHIFT-BREAK to swap back to disk 0 and re-boot the menu.
I have only tried this on my Master Compact, but other than DFS differences, I think it should work with any model/controller.

This assumes that your GOTEK is drives 0/2 and I can't guarantee that it won't make your beeb spontaneously combust or delete all the data from every computer in the house - use at your own risk!

I will release the menu maker and menu code at some point, but it needs tidying up and more automation.
I better way to select which files to include in the menu would also be good too.

The last page of the menu has nine spare rows which correspond to :2.$.--B4A-- .. :2.$.--B52-- (although you can only use the first seven without deleting some files) on side 2 of DSKA0395.DSD. The names won't show up in the menu but you should be able to add your own "secret" entries after padding out the .DSD to full size!

The first half might work on an "MMC", but I haven't tested it yet and haven't uploaded the BEEB.MMB.
Without more creative packing, ~800 .SSDs are required for the full set and BEEB.MMB is limited to 511 (although SmartSPI does support up to 8 archives).