Disc Image Manager

discuss pc<>acorn file transfer issues and the use of other utils
User avatar
lurkio
Posts: 3665
Joined: Wed Apr 10, 2013 12:30 am
Location: Doomawangara
Contact:

Re: Disc Image Manager

Post by lurkio »

The two XOR For Schools .SSDs in the zipfile uploaded by BillCarr both fail to open in DIM.

:!:
User avatar
geraldholdsworth
Posts: 956
Joined: Tue Nov 04, 2014 9:42 pm
Location: Inverness, Scotland
Contact:

Re: Disc Image Manager

Post by geraldholdsworth »

lurkio wrote:
Fri May 28, 2021 12:38 pm
The two XOR For Schools .SSDs in the zipfile uploaded by BillCarr both fail to open in DIM.
As does Zirconium uploaded by Treborwall. I found another bug in the ID-ing process. I'll check the XOR SSDs to see if they open in the fixed version.

EDIT: They do open...although one takes a while to open, which is worrying.

EDIT 2:Ahh...couldn't handle zero length files. Can now. :D
User avatar
geraldholdsworth
Posts: 956
Joined: Tue Nov 04, 2014 9:42 pm
Location: Inverness, Scotland
Contact:

Re: Disc Image Manager

Post by geraldholdsworth »

Version 1.27 is now available. Not much changed:
  • Fixed a bug where some DFS images would not be read.
  • Zero length files in DFS images were causing the application to take a long time reading in the image.
  • Circumnavigated a compiler bug (see earlier posts from lurkio) where multi-blocked GZip files where not getting inflated (this affected compressed UEF files). I have since found out that this bug within the Lazarus library was reported last year, after I posted my 'fix' for it on the Lazarus forum.
markusher
Posts: 86
Joined: Tue Jul 17, 2018 10:55 am
Contact:

Re: Disc Image Manager

Post by markusher »

Hi Gerald

I finally had some time to have a look at DIM, great work! I've been getting more and more frustrated with the bugs and limitations in ADFS explorer and this is a very useful tool. The addition of files direct from a !SparkFS archive is especially useful and I tried this out first in order to restore an old backup. I created an ADFS hard drive, worked out first I have to click on the root directory to add files (would be nice if it was selected already by default on creation of a new image) and set it to work with the .zip file.

After several files and directories had been added it aborted. Looking at the archive file, it seems there is a file with 0kb and this is the one that caused the hiccup. I can send you the archive file if you like, although it is a little large (for a BBC) at just under 8MB.
-Mark
4 x BBC, 1 Viglen BBC, 3 Masters, BBC AIV Domesday System, EO1S & E20 Filestore, A4000, RISC PC 600,700, StrongArm. Probably more I've missed...
User avatar
geraldholdsworth
Posts: 956
Joined: Tue Nov 04, 2014 9:42 pm
Location: Inverness, Scotland
Contact:

Re: Disc Image Manager

Post by geraldholdsworth »

Hi Mark,

Thank you for your kind words.

I have had some issues with zero length files across the various supported formats. I think, at the start, I assumed that there wouldn't be any such files...and now recently I've been coming across them, causing issues in various parts of the code. I think I can see where the issue is, but I'll take your archive file to test it out anyway (I'll PM you my email address).
User avatar
geraldholdsworth
Posts: 956
Joined: Tue Nov 04, 2014 9:42 pm
Location: Inverness, Scotland
Contact:

Re: Disc Image Manager

Post by geraldholdsworth »

Thank you to Mark for his various Spark and disc image samples, and some very good suggestions - I have made a load of improvements for version 1.28. Thank you also to David Pilling for his assistance in properly decoding the Spark format.

In this issue:
  • With the Spark archive sample that Mark sent, having over 2000 files in it, showed up the speed (or, rather, lack of) that files are de-archived. I've attempted to speed this along by bypassing the GUI during decompression (for over a certain number of files), but the time is mostly taken up by the decompression code (which I didn't write, as it is part of a library).
  • During an import, and de-archive, of a Spark archive, any errors are skipped over, but logged with the error log being displayed at the end.
  • A warning is shown prior to a de-archive if the current open image is not suitable for the number of files (i.e. if some directories have more than 47 or 77 files).
  • The root node is automatically selected when an image is opened (or created).
  • And a bug fix: ADFS Big Directory Hard Drive images where not being recognised by RISC OS.
Coming up...
  • The ability to pick and choose which files you want to import from a Spark archive or another disc image.
  • Dealing with Acorn File Server (AFS) format.
User avatar
geraldholdsworth
Posts: 956
Joined: Tue Nov 04, 2014 9:42 pm
Location: Inverness, Scotland
Contact:

Re: Disc Image Manager

Post by geraldholdsworth »

Just knocked up, and published, a user guide for Disc Image Manager. This is based on the next version, 1.29.
User avatar
geraldholdsworth
Posts: 956
Joined: Tue Nov 04, 2014 9:42 pm
Location: Inverness, Scotland
Contact:

Re: Disc Image Manager

Post by geraldholdsworth »

Version 1.29 is now available.

I've changed it so that Spark archives are treated as a filing system - in other words, just like any other image. So you can load one in without having an image already open. In addition, the Spark module now supports !PackDir archives. This was done because a lot of the operations where just duplicating what happens when you import any other image to an open image, so it made sense.

In addition, when importing an image into another image a dialogue now opens for you to pick which files/directories you wish to import and those that you wish to leave.

You can now also duplicate a file within the same directory (either by drag and drop or via the buttons or menus), and files can be moved around in a UEF...although, not currently through the command line (which will come).

And there is now a preferences window - not much in there, aside from being able to select which tile (or none) you want as a background, whether to create inf files on download, and whether to force interleaving with ADFS 'L' images (Mark Usher sent me an ADFS L image which wasn't interleaved, so I was able to find a way of working out if they were interleaved or not, just by counting the number of broken directories).

There's quite a few other changes, but mostly to do with tidying up or making things make more sense. A few bug fixes too.
User avatar
geraldholdsworth
Posts: 956
Joined: Tue Nov 04, 2014 9:42 pm
Location: Inverness, Scotland
Contact:

Re: Disc Image Manager

Post by geraldholdsworth »

We are now at version 1.30. Main change (well, pretty much the only change) is the addition of the support for Acorn FileStore. Currently, only Level 3 is supported, as I have not got any examples for Level 2 or Level 4, nor have I any information about Level 4. So if anyone has examples, and information, it would be gratefully received.

Also, the Level 3 support is at beta status - meaning that I've only tested it on the small amount of samples I have. But they do seem to cover most eventualities. However, if you find one that doesn't work, please send it along. My email address is in both the documentation and on the About box in the application.

Only other change is the addition of extra tiling options.

For those that are only interested in the binaries, and can't be bothered navigating GitHub to get them, I have them pointed to on my own website (still on GitHub, mind).
markusher
Posts: 86
Joined: Tue Jul 17, 2018 10:55 am
Contact:

Re: Disc Image Manager

Post by markusher »

The link from your webpage to the windows 64bit image is broken. Results in 404.
-Mark
4 x BBC, 1 Viglen BBC, 3 Masters, BBC AIV Domesday System, EO1S & E20 Filestore, A4000, RISC PC 600,700, StrongArm. Probably more I've missed...
User avatar
geraldholdsworth
Posts: 956
Joined: Tue Nov 04, 2014 9:42 pm
Location: Inverness, Scotland
Contact:

Re: Disc Image Manager

Post by geraldholdsworth »

Cheers Mark. Fixed now.
User avatar
hoglet
Posts: 10351
Joined: Sat Oct 13, 2012 7:21 pm
Location: Bristol
Contact:

Re: Disc Image Manager

Post by hoglet »

Gerald,

There is a version of the Atom Software Archive that is packaged as an Econet L3 File Server DIsk Image:
https://github.com/hoglet67/AtomSoftwar ... BSCSI0.zip

This comprises approx 4,600 small files spread across quite a large number of directories.

I've just tried opening the scsi0.dat file with the Linux version of Disc Image Manager (using the binary in github). It starts to read the files, but after about a minute the process seems to have consumed all my available RAM (I'm on an 8GB machine) and the machine grinds to a halt.
Screenshot from 2021-06-25 10-59-45.png
It's possible there is something wrong with the format of the image (since I wrote the software the generates it). It does work when it's loaded into an L3 FileServer. The directory hierarchy includes the ^ directory back-links, so I did wonder if that was causing things to get confused. It's possible I have not implemented these correctly.

Dave
User avatar
geraldholdsworth
Posts: 956
Joined: Tue Nov 04, 2014 9:42 pm
Location: Inverness, Scotland
Contact:

Re: Disc Image Manager

Post by geraldholdsworth »

hoglet wrote:
Fri Jun 25, 2021 11:10 am
I've just tried opening the scsi0.dat file with the Linux version of Disc Image Manager (using the binary in github). It starts to read the files, but after about a minute the process seems to have consumed all my available RAM (I'm on an 8GB machine) and the machine grinds to a halt.
I'll take a look. The version of Disc Image Manager online may also be at fault reading this image - it was only tested with the images I had. I've since had some more and have been able to improve the AFS reading capabilities. So this image should help me also.

Cheers,

Gerald.
cmorley
Posts: 1541
Joined: Sat Jul 30, 2016 8:11 pm
Location: Oxford
Contact:

Re: Disc Image Manager

Post by cmorley »

hoglet wrote:
Fri Jun 25, 2021 11:10 am
It's possible there is something wrong with the format of the image (since I wrote the software the generates it). It does work when it's loaded into an L3 FileServer. The directory hierarchy includes the ^ directory back-links, so I did wonder if that was causing things to get confused. It's possible I have not implemented these correctly.
I tried loading it with ADFS explorer and it doesn't show thousands of files...
adfsexplorer.png
User avatar
hoglet
Posts: 10351
Joined: Sat Oct 13, 2012 7:21 pm
Location: Bristol
Contact:

Re: Disc Image Manager

Post by hoglet »

cmorley wrote:
Fri Jun 25, 2021 12:40 pm
I tried loading it with ADFS explorer and it doesn't show thousands of files...
That's because they are in the Acorn File Server AFS0 partition, not the ADFS Partition.

On ADFS, the AFS0 partiton just appears as a single file (L3DIRS I think). Edit: On reflection, this is incorrect, the space for the AFS0 files is not visible at all in the ADFS volume.

It's a rather confusing combination of formats!

Dave
User avatar
geraldholdsworth
Posts: 956
Joined: Tue Nov 04, 2014 9:42 pm
Location: Inverness, Scotland
Contact:

Re: Disc Image Manager

Post by geraldholdsworth »

I do believe I know what is happening...I haven't taken account of the '^' parent directory pointer (mainly because I hadn't seen an image with this in), so it is reading the same directories over and over again.

While doing this, I found a bug, which may come in handy for you - in the latest version online I introduced a facility where a debug log file is produced in your temporary files. This can be turned on and off in the preferences. Unfortunately, it just produces the log file whatever this is set to (fixed now in 1.32). OK, this is all fine, but where is this log file? Just go to the About box and click on the disc icon - it'll report the path there (I really need to change that so it copies it to the clipboard...in fact, I'll do that in a mo').
User avatar
jgharston
Posts: 4457
Joined: Thu Sep 24, 2009 12:22 pm
Location: Whitby/Sheffield
Contact:

Re: Disc Image Manager

Post by jgharston »

geraldholdsworth wrote:
Fri Jun 04, 2021 4:48 pm
Coming up...
  • The ability to pick and choose which files you want to import from a Spark archive or another disc image.
  • Dealing with Acorn File Server (AFS) format.
You import *to*, you export *from*.

(Unless you're talking international trade....)

Code: Select all

$ bbcbasic
PDP11 BBC BASIC IV Version 0.36
(C) Copyright J.G.Harston 1989,2005-2020
>_
User avatar
jgharston
Posts: 4457
Joined: Thu Sep 24, 2009 12:22 pm
Location: Whitby/Sheffield
Contact:

Re: Disc Image Manager

Post by jgharston »

geraldholdsworth wrote:
Fri Jun 25, 2021 1:42 pm
I do believe I know what is happening...I haven't taken account of the '^' parent directory pointer (mainly because I hadn't seen an image with this in), so it is reading the same directories over and over again.
FSFloppy1 has '^' links in the filesystem. I was made for me by somebody on here.

Parent directory
----------------
The parent directory if it exists it is always the first entry in the
directory, (so is at offset &0011) the 'Next' entry is &FFFF, the SIN
points to the parent and all other bytes are &00. Note: this includes
the filename entry, so it appears to be named "", and includes the
Attribute entry, so appears to not actually be a directory, it has access "/".

Code: Select all

$ bbcbasic
PDP11 BBC BASIC IV Version 0.36
(C) Copyright J.G.Harston 1989,2005-2020
>_
User avatar
jgharston
Posts: 4457
Joined: Thu Sep 24, 2009 12:22 pm
Location: Whitby/Sheffield
Contact:

Re: Disc Image Manager

Post by jgharston »

Ah! I see the problem with Hoglet's image. The parent directory actually exists in the directory as an entry called "^" and is in the linked list of entries, so is visible to *CAT and any other directory-scanning code.

It's supposed to not be in the linked list, and it's supposed to be called "" (ie &00, &00, &00, &00, &00, &00, &00, &00, &00, &00), and it's required to be in the first directory slot at offset &0011, and it required to have a linked list 'Next' entry of &FFFF. Apparents the server code specifically looks at offset &0011 for the &FFFF Next entry to determine that the parent directory link exists.

Code: Select all

$ bbcbasic
PDP11 BBC BASIC IV Version 0.36
(C) Copyright J.G.Harston 1989,2005-2020
>_
User avatar
jgharston
Posts: 4457
Joined: Thu Sep 24, 2009 12:22 pm
Location: Whitby/Sheffield
Contact:

Re: Disc Image Manager

Post by jgharston »

Arrghh!!!

Can't... Stop.... Infinite... Loooppppp.
Task
Manager
Not
Responding
Need
Emergency
Paperclip.....
Image2.gif
^^^ Needs a CANCEL button. :)

Code: Select all

$ bbcbasic
PDP11 BBC BASIC IV Version 0.36
(C) Copyright J.G.Harston 1989,2005-2020
>_
User avatar
geraldholdsworth
Posts: 956
Joined: Tue Nov 04, 2014 9:42 pm
Location: Inverness, Scotland
Contact:

Re: Disc Image Manager

Post by geraldholdsworth »

jgharston wrote:
Fri Jun 25, 2021 5:47 pm
Can't... Stop.... Infinite... Loooppppp.
Not any more...I got around it and got it opening. :D
jgharston wrote:
Fri Jun 25, 2021 5:47 pm
^^^ Needs a CANCEL button. :)
Now there's an idea...thank you Jonathan.
User avatar
geraldholdsworth
Posts: 956
Joined: Tue Nov 04, 2014 9:42 pm
Location: Inverness, Scotland
Contact:

Re: Disc Image Manager

Post by geraldholdsworth »

jgharston wrote:
Fri Jun 25, 2021 5:25 pm
FSFloppy1 has '^' links in the filesystem. I was made for me by somebody on here.
Ah...so I have seen one. And DIM opens it OK (well, v1.31 does...not sure about 1.30).
Thank you Jonathan.

Free Space Map next...just can't get my head around where to find it - particularly on Hoglet's image.
User avatar
hoglet
Posts: 10351
Joined: Sat Oct 13, 2012 7:21 pm
Location: Bristol
Contact:

Re: Disc Image Manager

Post by hoglet »

jgharston wrote:
Fri Jun 25, 2021 5:34 pm
It's supposed to not be in the linked list, and it's supposed to be called "" (ie &00, &00, &00, &00, &00, &00, &00, &00, &00, &00), and it's required to be in the first directory slot at offset &0011, and it required to have a linked list 'Next' entry of &FFFF. Apparents the server code specifically looks at offset &0011 for the &FFFF Next entry to determine that the parent directory link exists.
Thanks, I'll make a note to fix the generator code:
https://github.com/hoglet67/AtomSoftwar ... e/issues/1
User avatar
hoglet
Posts: 10351
Joined: Sat Oct 13, 2012 7:21 pm
Location: Bristol
Contact:

Re: Disc Image Manager

Post by hoglet »

geraldholdsworth wrote:
Fri Jun 25, 2021 6:07 pm
Free Space Map next...just can't get my head around where to find it - particularly on Hoglet's image.
Do remember, my image may have issues.

Damn... my hidden agenda is now becoming obvious!
User avatar
geraldholdsworth
Posts: 956
Joined: Tue Nov 04, 2014 9:42 pm
Location: Inverness, Scotland
Contact:

Re: Disc Image Manager

Post by geraldholdsworth »

hoglet wrote:
Fri Jun 25, 2021 6:10 pm
Do remember, my image may have issues.
Ahh...I see. I'll stop looking.
hoglet wrote:
Fri Jun 25, 2021 6:10 pm
Damn... my hidden agenda is now becoming obvious!
Is this to try and break Disc Image Manager?
Which is fine - just means I have more ammo to improve it.

I've also come up with an ingenious method to prevent infinite loops - I take a note of every directory that has been referenced, so if it is referenced again, it is removed. This way, I can now open that image and remove all directories with the name '^', as they all appear to reference the root (or a parent).
User avatar
geraldholdsworth
Posts: 956
Joined: Tue Nov 04, 2014 9:42 pm
Location: Inverness, Scotland
Contact:

Re: Disc Image Manager

Post by geraldholdsworth »

I've just updated this to 1.31.

I've incorporated the changes noted above, and some suggestions/bugs from Mark Usher. I've still got some more to go, but I feel I need to do an overhaul of how the free space is recorded and reported, which means a major code change. So I'm releasing this version so I've got something to fall back on if it all goes pear shaped :D

So, what did I do? (I need to write this stuff down, otherwise I forget)...I fixed a few bugs:
  • Windows did not update the window when the texture was changed.
  • The log file was produced whether the option was set or not.
  • Renaming a file, then moving away, left the edit box still open.
There's also some added or improved features:
  • Added the file extensions for Acorn File Server and Spark files (*.afs and *.zip);
  • The File Viewer window can be resized, with certain tabs, and the size is remembered while the image is open. Text files with a &0D line ending are now displayed;
  • Downloading/extracting a single file uses the name specified in the save box and not the original name;
  • There was also some discussion about the naming (and extension) of AFS0. I now use Acorn FS (for File Server, but can equally be FileStore). The file extension is still afs (for both Level 2 and 3) - this may need some discussion in the forum, so opinions please.
  • Found that there was two (well, three) different methods of ADFS/AFS interleaving. After discussion with Jonathan Harston, these are referred to as Sequential, Interleaved and Multiplex (SEQ, INT and MUX). The method is now automatically detected, hopefully. I have since found a fourth (which is included, but not currently selectable).
  • It will now load in AFS Level 2 images.
  • It will now read and display the free space map for Acorn FS.
  • When saving an image, the original directory from whence it came from will default to being the save location.
  • and, thank you to Dave (Hoglet), it now deals with directories referencing parent directories and ending up in an infinite loop.
Kevin Edwards
Posts: 163
Joined: Tue Mar 14, 2006 9:16 pm
Contact:

Re: Disc Image Manager

Post by Kevin Edwards »

Hi,

I just updated to the latest version of DIM and it looks and works great.

One thing I did notice is that some disc images generate an error about not being recognised when you load them - this error was occuring with previous releases too.

The DFS disc images load and work fine in B-Em and BeebEm and I wondered what criteria is used for verifying if a DFS image is valid or not.
DiskImageManager_Error1.PNG
DiskImageManager_Error1.PNG (21.51 KiB) Viewed 405 times
I get this for 400K 80T/DS images that originally were exported from a DataCentre setup. Browsing the image using a hex editor doesn't show anything odd.

Any suggests or info greatly received.

Thanks,
Kevin.
User avatar
lurkio
Posts: 3665
Joined: Wed Apr 10, 2013 12:30 am
Location: Doomawangara
Contact:

Re: Disc Image Manager

Post by lurkio »

Kevin Edwards wrote:
Mon Jun 28, 2021 1:44 pm
Any suggests or info greatly received.
Could you possibly upload one of the troublesome disc-images that might help Gerald diagnose the problem?

:?:
Kevin Edwards
Posts: 163
Joined: Tue Mar 14, 2006 9:16 pm
Contact:

Re: Disc Image Manager

Post by Kevin Edwards »

Thanks.

I shall have a look for ones that are less 'sensitive'.
User avatar
geraldholdsworth
Posts: 956
Joined: Tue Nov 04, 2014 9:42 pm
Location: Inverness, Scotland
Contact:

Re: Disc Image Manager

Post by geraldholdsworth »

Kevin Edwards wrote:
Mon Jun 28, 2021 1:44 pm
The DFS disc images load and work fine in B-Em and BeebEm and I wondered what criteria is used for verifying if a DFS image is valid or not.
DFS images doesn't give much to go on to identify one. So I've got to fall back on assumptions to ID one, and hopefully it doesn't get IDed as something else.
Kevin Edwards wrote:
Mon Jun 28, 2021 1:44 pm
Any suggests or info greatly received.
Is it just double sided images that fail, or single sided too? One assumption is that DS images are always interleaved. An assumption I made for ADFS 'L' images, until I came across one that wasn't. However, as the first two sectors should be in the same place whether interleaved or not makes me think it is something else.
I'd suggest sending me an example, that fails, and I'll work out what is going on.

Cheers,

Gerald.
Post Reply

Return to “software & utilities for the pc, mac or unix”