Reducing Basic Program Size

[rmbrowngr]

Does anyone have some tips about reducing the size of a Basic program to share? I have a program on my Electron which I need to almost half in size to fit in mode 1 on an unmodified Electron. Currently running it with a MRB.

I generally know I should reduce the number of lines of code by combining them and remove any REMs. Not sure how aggressive I can be in removing spaces. Worst case I could look to move parts of code to assembly. Would be interested to hear about text compression as the program does have some pages of text to display.

[cmorley]

I use JGH's CRUNCH program. http://mdfs.net/System/Library/

I don't know if this works on an electron but if not you could run it on an emulated B to do the cruching.

[lurkio]

For compressing a BASIC program, see the Pack option in the PRES Advanced BASIC Editor ROMs (*B):

• viewtopic.php?t=7719&start=30#p81362

For text compression, see Robico's MIDGE system:

• viewtopic.php?f=32&t=17104&p=237194#p236858

[jgharston]

I use JGH's CRUNCH program. http://mdfs.net/System/Library/

I don't know if this works on an electron but if not you could run it on an emulated B to do the cruching.

Works with any 6502 BBC BASIC. The Electron uses the bog standard BASIC 2.

[Richard Russell]

I don't know if this works on an electron but if not you could run it on an emulated B to do the cruching.

In principle you could even compress the code using the BBC BASIC for Windows or BBC BASIC for SDL 2.0 cruncher, although it would involve some conversion because of the slightly different tokenised formats. Those crunchers are very aggressive, they even remove spaces necessary to allow the program to be edited so it's vital to keep a clean version.

[julie_m]

One way programs can use space unnecessarily is having two copies of things in memory: one copy in a program statement defining it, and another copy in its intended location in memory.

If you have any chunks of assembly language in your program, you can *SAVE the assembled code and strip them out. Function key and character definitions likewise (*SAVE FKEYS B00 C00 / *SAVE VDU23 C00 D00 / *SAVE BOTH B00 D00). Similarly if you have a bunch of DATA statements and are inserting values into memory with the ?, ! and $ operators, *SAVE the result and you can free up the space they occupied.

Static variable definitions also can be farmed out into a *EXEC file as immediate-mode statements; just be sure to start the program using GOTO as opposed to RUN which will wipe them out again.

Text compression is a subject about which books could be (and have been) written. There are many ways to do it, depending how much text you have to compress and how many repeating bits it contains. Emulation freaks can cheat with host-side tools

What's the program? It's hard to suggest much, because every case is unique.

[lurkio]

Static variable definitions also can be farmed out into a *EXEC file as immediate-mode statements; just be sure to start the program using GOTO as opposed to RUN which will wipe them out again.

Neat idea.

Code: Select all

10 CLS
20 PRINT'"Program begins"'
30 *EXEC INIT
40 END
50 PRINT'"Program continues"'
60 PRINT "A = ";A
70 PRINT "B = ";B
80 PRINT'"Etc."
90 END

$.INIT:

Code: Select all

A = 1
B = 2
GOTO 50

https://bbc.godbolt.org/?autoboot&nosee ... k/exec.ssd

[julie_m]

Yeah, it's a neat trick!

I use some csplit, sed and awk tomfoolery on the host side to produce the variable dumps used by the various build files for BCP (not quite the same thing, as they actually have line numbers). I've also got Perl scripts for things like extracting variable definitions and spotting DEFPROC and DEFFN statements that aren't actually called up.

[rmbrowngr]

Static variable definitions also can be farmed out into a *EXEC file as immediate-mode statements; just be sure to start the program using GOTO as opposed to RUN which will wipe them out again.

Neat idea.

Code: Select all

10 CLS
20 PRINT'"Program begins"'
30 *EXEC INIT
40 END
50 PRINT'"Program continues"'
60 PRINT "A = ";A
70 PRINT "B = ";B
80 PRINT'"Etc."
90 END

$.INIT:

Code: Select all

A = 1
B = 2
GOTO 50

https://bbc.godbolt.org/?autoboot&nosee ... k/exec.ssd

That’s neat.

[lurkio]

Yeah, it's a neat trick!

I did vaguely the same sort of thing in a much more convoluted way here:

• viewtopic.php?f=2&t=9061&p=151067#p151067

[flaxcottage]

Other ways of reducing the memory overhead of BASIC programs that I have used in the past include;

• Using the resident integer variables rather than user defined variables. The memory in page &400 is already reserved.

• When using strings, define the string to the maximum size needed. A$=STRING$(25," "):A$="" will reserve 32 bytes for the string A$ so that a line such as FOR X%=1to25:A$=A$+CHR$(X%):NEXT will only use 32 bytes for A$ whereas if the definition were left out the storage needed for A$ would be 8+16+24+32 bytes as BASIC assigns string memory in 8 byte chunks and there is no garbage collection.

• Any code which is repeated more than once could be profitably placed in a procedure, function or subroutine.

• Use functions rather than procedures where a value is to be returned.

• Use indirection operators (?, ! and $) to poke values directly into memory in pages &900 and &A00 then *SAVE and *LOAD the data.

• Any character definitions could be set up in a loader program or an EXEC file.

• Use files to store DATA statements' data and read it from the file.

• Use very short variable/procedure/function names.

• Rewrite the code to be more efficient.

• Avoid recursion as this may need a lot of temporary memory.

• Use overlay techniques to load procedures only when needed. These procedures will overwrite procedures no longer needed.

• Use machine code. Small machine code routines stored in zero page &70 to &8F or other memory pages, eg. &700, &800, &900, &A00, &B00, &C00 can dramatically reduce the amount of BASIC code needed.

[jgharston]

I've also got Perl scripts for things like extracting variable definitions and spotting DEFPROC and DEFFN statements that aren't actually called up.

Which the *link command also at http://mdfs.net/Software/CommandSrc/BasUtils/ does. (docs)

The only thing my BASIC cunrching tools don't do yet is variable crunching. It's something I keep meaning to get around to doing - maybe at Christmas after I've done my tax return.

One way programs can use space unnecessarily is having two copies of things in memory: one copy in a program statement defining it, and another copy in its intended location in memory.

As you mentioned, the classic example is DATA statements - the data exists twice, in the DATA statements themselves, and in the variable heap once they've been read. A lot of DATA use can be rewitten to avoid this duplication, for example the classic "day of week" function can be written:
DEF FNmon(mon%)=MID$("JanFebMarAprMayJunJulAugSepOctNovDec",mon%*3-2,3)

See http://beebwiki.mdfs.net/Data_without_DATA

[julie_m]

If you've got Perl code capable of parsing BBC BASIC programs, Perl almost does the rest for you automatically. You just need something like

Code: Select all

%float = %int = %str = ();
$float = $int = $str = "B";
# .....
if ($varname =~ /\$$/) {
    $varname = $str{$varname} = $str++;
}
elsif ($varname =~ /%$/) {
    $varname = $int{$varname++} = $int++;
}
else {
    $varname = $float{$float++} = $float++;
};

[lurkio]

Nothing to see here.

[lurkio]

I really do think that the Pack routine in the PRES Advanced BASIC Editor ROMs doesn't get enough love when it comes to its ability to compress programs. It handles REMs, spaces, comments, "Variables", "unused singles", and concatenation.

I tried running it on John's listing for Sphinx Adventure, which wouldn't even fit into RAM on a Model B with DFS -- I had to switch to Master mode in BeebEm, run Pack on the listing, and then save the compressed program to disc. The compressed version of the program will now load into RAM on a Model B with DFS -- though admittedly when you run it it promptly crashes with an out-of-memory error!

But the point is that the Pack routine managed to reduce the size of the program by 1271 bytes. Not bad!

• Sphinx.ssd

  The two versions of the Sphinx program on a disc-image. S is uncompressed. S2 is compressed.

  (49.75 KiB) Downloaded 25 times

[daveejhitchins]

It's already been mentioned, however, here it is again . . . The Advanced BASIC Editor - may I say - is probably the best around for compressing BASIC programs. There's very little it misses and here'e the link again. The BASIC Editor is pretty good too! I use it on the Electron, Master and BeebEm (Mac version) all the time - it really helps that it's an integrated system . . .

I've just added the User Guide to the end of the above linked thread.

Dave H

[lurkio]

I've just added the User Guide to the end of the above linked thread.

Thanks, Dave. Here's the bit from the manual that describes what the Pack routine does:

3.4.1 P – Pack Program

The object of Pack is to try to reduce the amount of memory occupied by the program. It does this by five means, each of which can be selected or not by answering the questions prompted at the start:

​REMs?
​Spaces?
​Comments?
​Variables?
​Use unused singles?
​Concatenate?

​(answer "Y" to remove REMs etc)

REMs: Removes all the REM statements from the program. Ensure that the program contains no GOTO/GOSUBs to lines consisting of REM statements only.

Spaces: Removes all unnecessary spaces, including those normally required for correct syntax during input, i.e. IF A=B THENC=0 becomes IFA=BTHENC=0, which could not be entered directly as BTHENC would be taken as a single variable. This means that great care must be taken editing a program if it has been packed.

Comments: Removes comments from any assembler listings. Note that this only affects a comment from the "\" delimiter up to the next separator (colon or CR), where deleting REMs removes the whole of the rest of the line.

Variables: This attempts to shorten all variable, PROC and FN names to one or two letters. The rules are:

​(a)​If the name is already single character, does nothing.
​(b)​Tries the first letter of the name as a single letter name.
​(c)​If the name is only two letters long, does nothing.
​(d)​Tries a two letter name consisting of the first two letters of the current name. If this fails it takes the first and third letters, then the first and fourth etc. If all these fail it tries the first letter plus a letter from A to z in turn.
​(e)​If all combinations should fail, leaves the variable name unchanged.
​(f)​PROC and FN names are always condensed to a single letter name, following the general rules above.


As each variable is changed the old and new names will be displayed in pairs, together with the number of occurrences. i.e.:

​A$ [6]
​Klingon% [3] K%
​Key% [15] Ke%
​Record$ [69] R$
​PROCwrite [4] PROCw

The order of change is first letter alphabetic, then in program order. The length of a name excludes any % $ ( suffixes.

Use unused singles: When packing variable names the resultant short names normally start with the same letter as the original. By using 'unused singles' further space can be saved, at the expense of program legibility. The program will try any unused initial letters as single character names before reverting to two characters, as in (c) above.

Concatenate: This attempts to add short lines together to form lines up to the maximum length allowed, thus saving the line overhead bytes (4 bytes per line, less 1 for the extra colon required). Certain lines cannot be concatenated, such as those including IF,DATA,REM,DEF etc and those that are targets of GOTOs etc.; these are all left alone.

Before packing, the value of TOP is shown, as packing proceeds a second line shows the current value of TOP as the program is shortened. The total number of bytes saved is shown on completion.

In order to get the best result from packing, variable names should be packed BEFORE concatenating lines. Also note that concatenation may produce errors if Spaces have not been stripped out first.


ALWAYS SAVE A COPY OF THE PROGRAM BEFORE PACKING IT.

[Richard Russell]

Here's the bit from the manual that describes what the Pack routine does

Pretty much identical to what the BBC BASIC for Windows cruncher does, except that when shortening variable names BB4W tries to maximise performance (speed) as well as reducing size. It does this by ensuring that the replacement variable names (usually one or two characters) are distributed evenly through the alphabet, including the little-used initial characters _ (underscore) and ` (grave accent). This is because of the way BBC BASIC indexes dynamic variables by their initial letter.

[rmbrowngr]

I've just added the User Guide to the end of the above linked thread.

Thanks, Dave. Here's the bit from the manual that describes what the Pack routine does:

3.4.1 P – Pack Program

The object of Pack is to try to reduce the amount of memory occupied by the program...

I'll have a go at using the pack function of the ABE to see what the affect is. I had some issues using the ABE with BeebSCI which I'll post in a different thread.

[1024MAK]

Can we please not quote entire posts where they are rather long. Either crop the quoted text, or use the normal “post reply” button.

Mark

[jgharston]

Being charitable, the quote block wasn't quoting a long post, but quoting external content.

[jgharston]

Being charitable, the quote block wasn't quoting a long post, but quoting external content.

Edit: Ah, I missed that the long external quote had been itself quoted, and edited out. Long day, not conntrating properly....

[lurkio]

The Advanced BASIC Editor - may I say - is probably the best around for compressing BASIC programs.

More evidence of its effectiveness:

• viewtopic.php?f=32&t=18667&p=259537#p259354

[Richard Russell]

As each variable is changed the old and new names will be displayed in pairs, together with the number of occurrences. i.e.:
Klingon% [3] K%

I'd not spotted that example before, but it's not a safe substitution. 'Klingon%', as a normal dynamic variable, will be deleted by a CLEAR (or CHAIN) but 'K%', as a static integer variable, won't. This could cause a program to misbehave, for example:

Code: Select all

   10 Klingon% = 123
   20 CLEAR
   30 Klingon% = Klingon% + 5
   40 PRINT Klingon%

   60 K% = 123
   70 CLEAR
   80 K% = K% + 5
   90 PRINT K%

For that reason my crunchers never replace a variable name with one of the static integer variables (@% to Z%).

[lurkio]

As each variable is changed the old and new names will be displayed in pairs, together with the number of occurrences. i.e.:
Klingon% [3] K%

I'd not spotted that example before, but it's not a safe substitution. 'Klingon%', as a normal dynamic variable, will be deleted by a CLEAR (or CHAIN) but 'K%', as a static integer variable, won't. This could cause a program to misbehave

And if you say yes to all the options including "Use unused singles?" then the Pack routine in the PRES Advanced BASIC Editor does indeed compress your example program down to the following...

Code: Select all

10A%=123:CLEAR:A%=A%+5:PRINTA%:K%=123:CLEAR:K%=K%+5:PRINTK%

...which behaves differently from the original program! Here's the output of the Packed version:

Code: Select all

>RUN
       128
       128
>_

EDIT: You can avoid this particular issue by saying yes to everything except "Use unused singles?":

Code: Select all

   10Kl%=123:CLEAR:Kl%=Kl%+5:PRINTKl%:K%=123:CLEAR:K%=K%+5:PRINTK%

[Richard Russell]

You can avoid this particular issue by saying yes to everything except "Use unused singles?":

But what happens in that case for a variable like 'klingon%' (note with a lowercase 'k')? Does it, or does it not, abbreviate it to 'k%' (which is of course entirely safe since it's not a static variable)?

[lurkio]

You can avoid this particular issue by saying yes to everything except "Use unused singles?":

But what happens in that case for a variable like 'klingon%' (note with a lowercase 'k')? Does it, or does it not, abbreviate it to 'k%' (which is of course entirely safe since it's not a static variable)?

Perhaps surprisingly, it does:

Code: Select all

   10 klingon% = 123
   20 CLEAR
   30 klingon% = klingon% + 5
   40 PRINT klingon%

The above compresses down to the following when you say yes to everything except "Use unused singles?":

Code: Select all

   10k%=123:CLEAR:k%=k%+5:PRINTk%

But I've now noticed another wrinkle:

Code: Select all

   10 Klingon% = 123
   20 CLEAR
   30 Klingon% = Klingon% + 5
   40 PRINT Klingon%

The above compresses down to the following when you say yes to everything except "Use unused singles?"!:

Code: Select all

   10K%=123:CLEAR:K%=K%+5:PRINTK%

That doesn't seem intuitive to me. What's the point of saying no to "Use unused singles?" in this case? It doesn't seem to make any difference from saying yes!

So it seems that if you say yes to "Variables?", then -- even if you say no to "Use unused singles?" -- you can't stop Pack from replacing a "regular" integer variable with a resident integer var if one is available and its name is the same as the first letter of the variable to be replaced.

[Richard Russell]

That doesn't seem intuitive to me.

Indeed not, but I suppose it could simply be a bug.

[daveejhitchins]

I have a list of 'upgrades' I'd like to see in the ABE - maybe I should start a thread to see what others would like - including any 'feature' changes as mentioned above!

One item that does need attention is the User Guide. It has all the information in there but finding it is a nightmare. It needs completely reformatting. On my list, of course . . .

Dave H

[Rich Talbot-Watkins]

I'd not spotted that example before, but it's not a safe substitution. 'Klingon%', as a normal dynamic variable, will be deleted by a CLEAR (or CHAIN) but 'K%', as a static integer variable, won't. This could cause a program to misbehave, for example:

Code: Select all

   10 Klingon% = 123
   20 CLEAR
   30 Klingon% = Klingon% + 5
   40 PRINT Klingon%

   60 K% = 123
   70 CLEAR
   80 K% = K% + 5
   90 PRINT K%

For that reason my crunchers never replace a variable name with one of the static integer variables (@% to Z%).

Do you remember if the BBC BASIC specification required this to work, or whether it's just an undefined consequence of how BBC BASIC implements variable assignment (allocating a new variable and defaulting it to zero prior to evaluating the RHS)? It has always seemed to me an extremely hacky paradigm!