PLASMA virtual machine

[SteveF]

I was bored and poking around on the 6502.org forum and I found this thread about the PLASMA VM for the Apple: http://forum.6502.org/viewtopic.php?f=2&t=2981 Linked from that thread is a presentation on YouTube by the author: https://youtu.be/RrR79WVHwJo?t=11m1s

It looked pretty cool and I thought I'd have a go at porting it to the BBC. It's still fairly hacky, but if anyone's interested they can take a look at what I've done here: https://github.com/ZornsLemma/PLASMA/tree/bbc

I've attached a copy of the plasma.ssd disc image it builds and you can play around with that on a real BBC or an emulator:

Code: Select all

MODE 7
*PLASMA
+HELLO
+TEST
+HANOI
+SIEVE
+DUMP
+MODE

Building the plasma.ssd more or less follows the Apple build instructions (which are present in the README which is cloned into my repository); the makefile has been extended to build the SSD as well. You'll need a copy of beebasm; I use it as a convenient way to bundle the files up from the build environment into an SSD, not as an assembler.

The VM seems to have some pretty clever support for banked memory on the Apple 2 and 3; I haven't looked into that aspect of things yet, so currently it just runs in main RAM, but I think there's potential for all sorts of enhancements, e.g.:

• the VM could be put into a ROM (fairly easy)
• we could probably have a BAS128-style version with 64K of workspace
• a 'hi' version for a 6502 second processor would probably give about 55K of workspace (fairly easy)
• it would be nice to be able to "link" a PLASMA program into a standalone executable or ROM image

I haven't yet ported the "sandbox" shown in the youtube video; I suspect this will be doable using mode 7 to provide an Apple-ish text screen, but memory might be a problem with the current version of the VM with no banked RAM support.

[ThomasHarte]

It works on an emulated Electron right up until it attempts to scroll the screen, at which point everything goes crazy. You've said to use Mode 7 quite clearly, so no great surprise, but is that the end of that?

[SteveF]

I'm surprised it doesn't work, actually; it won't currently work in modes 0-2 so I just specified mode 7 to be safe. It works on an (emulated) BBC in mode 6 and it uses the OS for everything so I'd expect it to work just fine on an Electron. (HANOI won't work as it's written to use mode 7 graphics, but everything else should be fine.)

My best guess - but it is only a guess - would be that some stray Apple-ish memory location left in the code is being touched in a way that doesn't upset a BBC but does upset an Electron. I'll see if I can try it myself, but what exactly goes wrong? Does just pressing RETURN repeatedly at the "PLASMA:" prompt until it scrolls cause a problem, or do you have to be running a particular test program?

[ThomasHarte]

I'm surprised it doesn't work, actually;

You're right to be; having tried again, I obviously made some sort of mistake at my end. I'll try to figure out what it was through academic interest but it's unlikely to be particularly relevant. My error, I apologise.

[SteveF]

Good to know, thanks. It's quite possible there's some sort of intermittent bug (not necessarily Electron specific) - I did get the odd glitch during development, I *think* these were caused by the emulator seeing a partial update to the disc image, but they might be genuine. I'd like it to be as robust as possible so let me know if you manage to reproduce it.

[BigEd]

Great idea to bring this to the Beeb!

[SteveF]

I've spent some more time fiddling with this so here's a new version of the ssd. The source code is on github in the repo mentioned above.

Some notes:

• There are now two versions of the virtual machine - 'PLASMA' and 'PLAS128'. Both relocate themselves down to PAGE on startup (instead of always running at &2000 as before); this gives more free memory, especially on machines where PAGE=&E00.
• PLASMA uses main memory as workspace, so it's the version of choice for a machine without sideways RAM. PLASMA will automatically use all the available memory on a 6502 second processor, so there's no need for a separate 'HIPLASMA'; on a 6502 second processor, there's currently about 54K of workspace free at the prompt.
• PLAS128 uses up to 64K of sideways RAM to store PLASMA bytecodes, leaving main RAM free for assembly language routines and program data. It doesn't insist on a full 64K of sideways RAM and it doesn't need shadow RAM, although obviously if you have them you will get more space free. (Shadow mode isn't forced on automatically, you need to do something like 'MODE 135' before '*PLAS128' to activate it.) PLAS128 isn't compatible with the 6502 second processor; currently it will fail, I think I can make it execute on the host and ignore the second processor, but I haven't done it yet.
• PLASMA should work on an Electron, though I haven't tested it. I suspect PLAS128 won't, as I think the ROM paging on the Electron isn't controlled by writing to &FE30 as it is on the BBC. Depending on how ROM paging works on the Electron, it would probably be possible to produce an Electron-specific build of PLAS128.
• Compiled PLASMA programs should execute happily on either PLASMA or PLAS128, obviously subject to them having enough memory available. Let me know if you find any incompatibilities. Having said that, PLASMA is definitely slightly simpler so please do try both if you have problems.
• There are a couple of new test programs, '+PCW' will run some Personal Computer World benchmarks and '+MODE2' is a torture test for the mode changing implementation.
• There's a first cut at a 32-bit arithmetic library; this is used by the PCW benchmarks.

I couldn't resist benchmarking the performance against BASIC. This is obviously a bit unfair, as BASIC is using 32-bit integers whereas PLASMA integers are 16-bit. I coded up a special 32-bit version of intmath for PLASMA, but the other benchmarks are using normal 16-bit integers. I think intmath32 is slow on PLASMA for two reasons - firstly a function call is needed for each arithmetic operation, which adds overhead, and secondly I just grabbed some multiply and divide code off the web (credits in the source) and these probably aren't as well-tuned as the BBC BASIC code.

Code: Select all

                Master    Master    BBC B     Master        Master   Master
                BASIC 4   BASIC 4   BASIC 2   BAS128 1.10   PLASMA   PLAS128
                          (shadow)
intmath16                                                    0.97     0.99
intmath32        3.12      3.32      4.32      5.48          4.73     4.81
textscrn        21.59     22.93     23.15     23.52         16.12    16.42
grafscrn        21.02     22.90     22.09     30.08         17.15    17.75

It's worth noting that because of the design of the PLASMA VM, PLAS128 doesn't suffer a significant performance penalty compared with PLASMA, unlike BAS128 compared to BASIC.

(Incidentally, and nothing to do with PLASMA, for some reason my BASIC benchmark times are significantly slower than those reported by PCW:

Code: Select all

                Master    BBC B     PCW BBC B   PCW Master
                BASIC 4   BASIC 2   BASIC 2     BASIC 4   
                          
intmath32        3.12      4.32      2.6         2.5      
textscrn        21.59     23.15     13.7        14.2      
grafscrn        21.02     22.09     21.5        22.0      

)

I have to say that the more I play with it the more impressed I am with the design of the PLASMA VM. It's a very powerful and elegant thing. The PLASMA language is also pretty good, although there are a couple of sharp corners I keep cutting myself on. I can't help wondering if the PLASMA VM would make a good target for a C compiler...

[SteveF]

I've just spent a day grovelling around in the innards of the virtual machine trying to track down a mysterious failure in a PLASMA program I've been trying to write. In the end it turned out to be as simple as the fact that my relatively complex program was overflowing the 512 byte symbol table allocated by the module loading code in bbcmd.pla; I had no idea there was such a limit until I managed to figure this out.

I think I can actually dynamically grow the symbol table, and if I can't I'll bump the size up from 512 bytes and put an explicit error in if it overflows, but I thought I'd post here and mention it in case anyone else is experimenting with this and runs across the same problem. If you want to change this yourself in the meantime, all you need to do is alter the constant just under the comment '// Init symbol table.' in vmsrc/bbcmd.pla from $200 to say $400.

Trying to see the bright side of this, I do now have a much better understanding of all the magic symbol lookups and relocations going on in loadmod()...

[SteveF]

Here's another update; the source code is on my 'bbc' branch at github (see the first post in the thread). Changes:

• Benchmarks re-worked and split into BENCH1 and BENCH2. The idea of this is to allow BENCH1 (which needs to be able to select mode 3) to be run on a model B with PAGE=&1900 and no shadow RAM. Ultimately I think the solution to very tight memory in the larger screen modes would be a language ROM version of the VM; this would free up the 5K or so of main RAM taken by the VM, and that makes all the difference. I hope to be clever with that and allow the rest of the ROM bank to be used for bytecode if running from sideways RAM; this is something I'm trying to resist the temptation to do until the VM is closer to being otherwise "finished".
• Added detection of some out-of-memory conditions, in particular heap allocations and frame stack allocations which fail now generate an error. There are still places where it's possible to crash things by running out of memory (the example which springs to mind would be trying to run a program with +PROGRAM where the module is too big for the free memory) but I think this should catch the main "subtle" cases where a program starts running fine and then starts trampling over memory it shouldn't do. There's a small performance impact to this (mainly the frame allocation), but it's fairly negligible - the new fncall benchmark execution time goes from 4.27s without to 4.34s with this checking.
• Experimental and incomplete C-style "stdio" library added; there's a test/demonstration of this (+STDIOTS). There's also a version of the PCW 'store' benchmark using it, but that's disabled at the moment. It's not very fast right now but it's still a work in progress so I haven't tried optimising it.
• A few new functions in the 32-bit arithmetic library (DWORD)
• Experimental and incomplete bytecode optimiser (optimise.py); this isn't used by default, but there's some commented out code in the makefile to apply it to the PLASMA executable and the same could be done for any other module.
• The 'CS' (constant string) VM opcode now uses a per-function-call hash table to avoid the problem noted in the README under "In-line String Literals" where use of a string literal inside a loop allocates a fresh copy of the string on every pass. (This only applies to PLAS128; PLASMA never suffered from this as it has a flat address space and doesn't need to copy constant strings in the first place.) However, if you want to write "portable" PLASMA code which would run well on an Apple II or III, you should continue to use the workaround advised in the README. (I might see if David is interested in taking this change upstream later on, but I want to wait until it's had some testing first.) There's a test program for this (+STRPOOL)
• There's a pdump.py Python program which can be used to dump out the contents of a compiled PLASMA module; it doesn't currently disassemble bytecodes, but I created it while trying to debug the problem I noted in my previous post. What might be moderately useful is that it shows (at the bottom of the output) how much memory is needed once the module has been loaded - modules are much larger on disc than they are once loaded, as they contain relocation data at the end which is discarded after they're loaded.
• Some bugs in setjmp()/longjmp()/OS error handling have been fixed. There's a new test program +SETJMP as well.
• The fixed-size symbol table has been increased from 512 bytes to 1024 bytes. An error is generated if the symbol table becomes full instead of trampling over random memory. (It's not currently possible to increase this dynamically at runtime.)

If anyone would like even more detail or has any other questions please ask!

[BigEd]

Excellent!

...I think this should catch the main "subtle" cases where a program starts running fine and then starts trampling over memory it shouldn't do. There's a small performance impact to this (mainly the frame allocation), but it's fairly negligible - the new fncall benchmark execution time goes from 4.27s without to 4.34s with this checking.

I'm reminded of this note by Tony Hoare relating to the early 1960s:

In designing the machine code to be output by the [single pass 8k] Elliott Algol compiler, I took it as an over-riding principle that no program compiled from the high level language could ever run wild. Our customers had to accept a significant performance penalty, because every subscripted array access had to be checked at run time against both upper and lower array bounds. They knew how often such a check fails in a production run, and they told me later that they did not want even the option to remove the check.

[SteveF]

Nice quote. It's not that strict - PLASMA arrays aren't bounds checked and there isn't much scope for changing that [1] - but I'm used to C/C++ and can handle that. But the mysterious crashes part way through program runs on a program that worked fine before I made some tiny change were driving me crazy, especially as the VM is effectively alpha/beta quality and could be harbouring latent bugs, so I think the heap/frame checking is well worth it, especially as it is so cheap.

[1] This is about the PLASMA language; it would, I think, be perfectly possible to write a compiler (or port an existing compiler) for a language with bounds-checked arrays to the PLASMA VM.

[SteveF]

I've continued to fiddle with this, so here's another update. I've attached a zipped SSD and the source is on github.

The sample ROGUE game has been ported.

It's quite memory hungry - it might be possible to optimise the memory use, but I didn't really want to make changes to it other than those required to port it. It runs under PLAS128 on a Master or PLASMA on a 6502 second processor on any machine; other combinations might work but I haven't tested extensively. (You should get an error message if it doesn't have enough memory rather than a crash.) Type '*PLASMA' or '*PLAS128' to start the VM, then type '+ROGUE' to start the game. You need to be in a 40 column mode to use it, mode 134 (shadow mode 6) is best as it uses the '\' character which appears as '1/2' in mode 7.

It's a pretty unforgivingly hardcore game (but then I've always been crap at nethack, which is the only roguelike I've ever played before this). The tile symbols can be seen in samplesrc/rogue.pla:

const FLOOR_TILE = '.'
const WALL_TILE = '#'
const TORCH_TILE = '*'
const PIT_TILE = ' '
const DOOR_TILE = '+'
const LOCKED_TILE = '%'
const WATER1_TILE = '>'
const WATER2_TILE = '<'
const KEY_TILE = ','
const RAFT_TILE = '@'
const GOLD_TILE = '$'
const FOOD_TILE = '&'
const ENTER_TILE = '-'
const EXIT_TILE = '='

Keys can also be seen in samplesrc/rogue.pla, but here's my reading of the code:

• I/J/K/M - move north/west/east/south
• left/right arrows (also A/D) - rotate on the spot 45 degrees left or right
• up/down arrow (also W/S) - move one space forwards or backwards in the direction you're facing
• Q - run forward in the direction you're facing
• SPACE - 'operate on' the tile you're facing (to open or unlock a door, or go through the exit)
• RETURN - pick up an object on the tile you're occupying
• O - turn lamp off
• </> - turn lamp down/up
• X - quit

If anyone is having a go at writing their own programs with this port of PLASMA, the following changes are relevant:

• New 'MODE' command supported at the PLASMA prompt to change mode. (The parsing is actually very lax, and you can get away with things like 'M OD FOO 1 28', if that sort of thing floats your boat. )
• loadmod() no longer loads the entire module twice; it reads the first 128 bytes with OSGBPB then loads the entire module just once aftwards. I haven't actually timed it, but this should make program startup noticeably faster.
• Acorn OS interface tidied up. There's now an inc/acornc.plh header which contains nothing but constants for use with the standard call() function to call an operating system routine. This is good enough for most OS calls. The inc/acornos.plh header declares a dependency on the ACORNOS module, which takes up a little bit of space at runtime but provides a few wrappers for OS calls where call() isn't ideal (e.g. OSBGET wants as little overhead as possible).
• Acorn port-specific changes in inc/cmdsys.plh have been moved out into inc/cmdsysac.plh, which should be mildly helpful for anyone trying to port code to or from the Apple version.
• The optimiser (optimise.py) is still extremely hacky and experimental, but it's had some more work done and it is currently enabled by default in the makefile for the build of the PLASMA executable. It saves 150-ish bytes; every little helps! In principle this can be used on any PLASMA program, I'm just being cautious about turning it on. Look at the 'python optimise.py' command in the makefile to see how to use it.
• New xheapalloc() function added; this is like heapalloc() but will raise an error if the allocation fails. It's intended for use in cases where a program can't do anything sensible if the allocation fails.
• New callalloca() function added; this allows you to allocate arrays of >255 bytes on the frame stack which are automatically freed on function exit. Normally you can only have 255 bytes of local variables per function. This might be of interest to upstream once it's had a bit of testing.
• The symbol table is now dynamically allocated rather than being a fixed size. It also starts off using otherwise wasted space in the &400-&800 region of language workspace, although currently this is deliberately restricted to help shake out any bugs in the dynamic allocation.
• PLAS128 only: Bug in handling of the per-function constant string pool when using setjmp()/longjmp() has been fixed; previously strings in the pool allocated between a setjmp() and a longjmp() could be corrupted after the longjmp() call.
• PLAS128 only: Bug in rdstr()/gets() which could corrupt the string pool fixed
• PLAS128 only: Now tries to force itself to run in the I/O processor if a second processor is active. This seems to work but it makes me feel uneasy; I may turn this off later and just make it refuse to run (as BAS128 does).
• Additional checks for running out of memory added, most notably to loadmod() - it should now not be possible to crash the VM by trying to load a program which won't fit in memory.
• 'Q' (QUIT) command no longer supported at the PLASMA prompt; you can exit by issuing a * command to enter another language, as before.
• PLASMA only: When used with a second processor, pressing BREAK previously left the VM in a mangled state; it should now reinitialise correctly.

Internal changes to the VM, of largely academic interest unless you're playing around with the VM code yourself:

• Bug in code to relocate VM down to OSHWM on startup fixed; previously it worked most of the time (any given build either worked or it didn't, presumably all the ones previously released worked), but if two relocations happened to be a multiple of 256 bytes apart it would fail. (So adding debug code to try to figure out what was going on would often make the problem disappear. )
• I finally found the '--report' option to ACME (my bad; I was really badmouthing ACME to myself for not having this, and it was there all along...) and the build process now generates a .lst file for builds of the VM showing the assembled code. For this to be most useful, you need to temporarily disable the relocation down to OSHWM by commenting out the 'python add-relocations.py' line in the makefile, otherwise the addresses at runtime don't match those in the .lst file.
• Latest upstream changes merged; these just silence some compiler warnings and don't do anything very exciting.
• Some progress has been made on rationalising the constants used for various memory locations; 'vmsrc/acornc.inc' has been added.
• PLAS128 only: CS opcode string pool has been tweaked so that new strings go at the head of the linked lists in the string pool's hash table instead of the tail. This was done primarily to help fix the bug noted above, but as a side effect it may help performance of constant string lookups in inner loops and the code is slightly simpler.
• Some dead code has been deleted.

Known bugs:

• PLAS128 only: Using mode() can interact badly with constant strings previously allocate in any of the functions higher up the call stack. Just use mode() ASAP and it's not a problem...

[BigEd]

Great work! Always good to have an application to test out a language.

Could you sketch out the memory map for the two cases? (For example, is there a distinction between user program and user variables? Does it use all sideways RAM or just some? How much of the 64k in the copro gets used?)

Edit: oops, I see there are three cases: your PLASMA port (but not your rogue port) will run on an unexpanded Beeb or Electron in the lower 32k - is that right?

[SteveF]

Could you sketch out the memory map for the two cases? (For example, is there a distinction between user program and user variables? Does it use all sideways RAM or just some? How much of the 64k in the copro gets used?)

Sure, thanks for giving me the opportunity to waffle some more.

The following keeps talking about 'modules'. You probably know what they are, but just for the benefit of any other readers, a module is what you get by compiling a PLASMA source file. It's a bit like an object file in a traditional compiled language like C, although it's really a bit more like a shared object as they are linked together dynamically at run time. A module includes a list of dependencies on other modules, so taking the ROGUE game as an example:

• The top-level module ROGUE, which is what you load to run the game, has dependencies on CMDSYS (the built-in module containing some basic functions), ROGUEMAP, ROGUECOMBAT and ROGUEIO.
• ROGUEMAP has dependencies on CMDSYS and ROGUEIO
• ROGUEIO has dependencies on CMDSYS and ACORNOS
• ROGUECOMBAT has dependencies on CMDSYS and ROGUEMAP

When you tell the VM to load ROGUE by typing '+ROGUE', it loads it and then recursively loads all the dependencies so all the necessary code and data is present in memory.

A module contains assembly language code (for 'asm myfunction' definitions in the source code), PLASMA bytecode (for 'def myfunction' definitions in the source code), global variables and information on the symbols exported and imported by that module, which the VM uses to perform the dynamic linking. I'll note in passing that assembly language code in a module doesn't get relocated, so it must be written to be position independent (e.g. using Bxx branches instead of JMP).

With that out of the way, let's talk about the memory maps.

The 'PLASMA' executable runs on an unexpanded machine or on a second processor. It has a flat memory map, which looks roughly like this.

• OSHWM to HEAPSTART=OSHWM+6K - the PLASMA VM itself (6K is a rough figure, and I just made the name HEAPSTART up)
• HEAPSTART to HIMEM - user programs and data. HIMEM varies with mode on an unexpanded machine as you'd expect, on a second processor it's always &F800. This area is further broken down into:
  • HEAPSTART to HEAPEND - the heap, which grows up from HEAPSTART towards higher addresses. Modules are loaded and executed here, program global variables live here along with the program code, and user programs can allocate space using heapalloc() and release it with heaprelease(). User program heap allocations are automatically freed on program exit. (There is support for user modules remaining resident after execution, and my brief experiments suggest this works fine, but I haven't played with it much yet.)
  • IFP to HIMEM - the frame stack, which grows down from HIMEM towards lower addresses. Used to hold local variables for functions which are executing.

On this model, whatever memory is free is available for program code and user data as required; you can have a small program operating on a large amount of data, or vice versa. On a Master with no second processor in mode 7, there's 21K available for programs and data at the PLASMA prompt; on a second processor there's 53K available.

As noted above, the PLASMA language allows you to write functions in either assembly language ('asm myfunction') or PLASMA ('def myfunction'), the latter being compiled into PLASMA bytecodes. Under the 'PLASMA' executable, both of these types of functions are loaded onto the heap and executed from there. 'PLAS128' uses up to 64K of sideways RAM to hold PLASMA bytecodes, but the VM, assembly language functions and all user data (globals, heap allocations, local variables on the frame stack) live in main RAM exactly as in the 'PLASMA' executable.

The PLAS128 VM is a bit larger due to the extra complexity of handling the sideways RAM paging, so you get a bit less main RAM free - which means you have a bit less space available for globals, data on the heap, assembly language code and local variables - in return for not using main RAM for the bytecodes which hopefully make up the bulk of your program. PLAS128 benefits from shadow RAM just as PLASMA does, because it raises HIMEM, but it's not mandatory.

There are a few extra subtleties to mention regarding PLAS128:

• The first 9 bytes of each 16K bank are deliberately left untouched to avoid the risk of them accidentally looking like valid ROMs to the OS
• A single module can't span two 16K sideways RAM bank, so if you load a module with 12K of bytecode and then another module with 5K of bytecode, the last 4K in the first sideways RAM bank will be wasted. This also means you can't load a module which has more than 15.xK of bytecode.
• Every bytecode function, including those loaded into sideways RAM, needs an entry in the definition table in main RAM, which takes about 5 bytes per function, and exported functions additionally need a symbol table entry, which takes about len(function_name)+2 bytes.
• Constant strings in PLASMA bytecode need to be copied into main RAM so they can be accessed; the frame stack is used to contain a per-function string pool in addition to the local variables for that function. This obviously uses up RAM while the function is executing, but it doesn't count against the 255 byte limit on a function's local variables.

Eventually I intend to produce a third version, a "normal" Acorn OS language ROM which runs from a single sideways ROM or RAM bank. This would get the VM code out of main RAM (making use of modes 0-2 more practical without shadow RAM) and I expect it to be possible to allow the remaining 10K or so of the sideways RAM bank to be used for bytecodes if running from sideways RAM instead of sideways ROM. (That 10K could *almost* be used for heap, but problems would occur if a user program made a heap allocation, happened to get an address in that 10K of sideways RAM and passed it to an OS call.)

It has occurred to me that as long as no module exceeds 64K, there is very little practical limitation on the total size of the PLASMA bytecodes forming a program. It would therefore probably be possible to produce a version for the Matchbox co-pro with 1MB of banked RAM which uses (say) 6-7K for the VM, a 16K window onto 1MB of bytecode space and the remaining 37K or so for the heap (assembly language code and user data) and frame stack. Similarly, I think it would be possible to produce a 'PLAS256' which uses more than 64K of sideways RAM for bytecodes, but given that even PLAS128 hasn't had a serious hammering yet I have resisted the temptation to produce something so unnecessary. (It would be sort of cool in a very very geeky way, but part of the fun of something like this IMO is building something that actually works well, even if it works at something of no practical use, and without a huge program needing the space provided by these hypothetical versions they'd never get enough testing to be satisfyingly proved to work.)

[BigEd]

That's great information, and very clear, thanks!

Are you saying that a bytecode function with string constants will copy those strings into the heap every time it's entered? That could be a performance gotcha.

Do note that the latest release of PiTubeDirect also offers banked memory in the 6502 model, so, with that or the Matchbox, that's two ways of having a large-memory fast 6502.

(As a wild idea, which you might like or not, it should be possible to put a C implementation of the PLASMA VM onto the Pi, as yet another flavour of CPU, even with say a megabyte of flat RAM space, or with banked memory. That would then be the fastest PLASMA on the planet, and still hooked up to a Beeb and with the usual Acorn MOS facilities for I/O.)

[SteveF]

Are you saying that a bytecode function with string constants will copy those strings into the heap every time it's entered? That could be a performance gotcha.

More or less. What actually happens is that when the CS ("constant string") PLASMA opcode is executed, the string gets copied onto the frame stack (not the heap). If the string is used inside an if, for example, the copying won't occur unless the body of the if is executed. You have the option to allocate the string as a global to avoid any overhead here, and this is the recommended technique (have a look at the README - which I haven't modified from upstream - under "In-Line String Literals") as without the hash table, every time the CS opcode is executed it copies the string again. You can see an example of this in samplesrc/sieve.pla, where a strPrimes global is used to hold a constant string (though in this case it's not used in a loop, and this really isn't necessary, except perhaps on grounds of using consistent coding standards).

The Acorn port will actually look up the string in a per-function-call hash table and avoid copying it to the frame stack multiple times if the same CS opcode is executed more than once (due to a loop, typically). This avoids the "I am eating all your memory" problem illustrated in the README, but if you want to write portable code or you need a constant string in a performance critical inner loop you probably want to allocate the string as a global to avoid any of these overheads.

Do note that the latest release of PiTubeDirect also offers banked memory in the 6502 model, so, with that or the Matchbox, that's two ways of having a large-memory fast 6502.

This is good news. However, I don't have any real hardware available most of the time, so if I was ever going to do such a port I'd probably need an implementation inside an emulator. If anyone else would like to have a go in the meantime I'm happy to offer advice, otherwise it's something I might look at in future, probably after the language ROM version.

(As a wild idea, which you might like or not, it should be possible to put a C implementation of the PLASMA VM onto the Pi, as yet another flavour of CPU, even with say a megabyte of flat RAM space, or with banked memory. That would then be the fastest PLASMA on the planet, and still hooked up to a Beeb and with the usual Acorn MOS facilities for I/O.)

That sounds pretty cool, and I think it would be capable of running the same compiled modules as the current port as well. It would need to include a 6502 emulator for modules containing assembly code, though it might be possible to just say that isn't supported. I am not sure if it would be possible to support more data space than 64K, given the assumption that a word is 16 bits and that you can stuff the return value of (say) heapalloc() into one. However, 64K of data space is still more than any other version of PLASMA offers AFAIK, on the Apple or the Acorn. You could almost certainly have a large code space though, and as you say it would be very fast with the bytecodes being interpreted by the Pi's CPU.

I can't see myself doing any direct work on this (e.g. given my lack of real hardware), but if you or anyone else wants to pursue this I'd be very happy to work with you to ensure as much compatibility as we can between this "super PLASMA" and the port I've been working on for non-Pi hardware.

[BigEd]

Oh, that string handling approach is about as good as it could possibly be! No gotcha there, which is great.

Good point about a possible C model needing (or nearly needing) to support 6502 code too. It's only a wild idea at present!

In the best of all possible worlds, the memory banking system now offered by the Matchbox and the Pi would be so well-received that it is also supported in at least one emulator! But we're not there yet. It would, unfortunately, probably come with a speed penalty.

[SteveF]

Good point about a possible C model needing (or nearly needing) to support 6502 code too. It's only a wild idea at present!

Sure, I understand! But just to state the obvious - the 6502 support would probably be easyish and could re-use some existing C or ARM 6502 emulation code, as it's not having to worry about cycle exact emulation, interactions with other emulated hardware (like CRTC controllers) and it's not all that performance critical. It would also be the natural interface point to the OS routines on the host, because (probably; obviously there's lots of scope for different approaches) none of the PLASMA bytecodes executed by the C VM would ever try to call the OS routines directly; they'd instead call a 6502 wrapper like the existing 'asm call' function to do the job.

[BigEd]

Just to note: David Schmenk [resman over on 6502.org] just commented on this port:

I had no idea they were doing this. Just made my day. The PLASMA VM isn't tied too directly to the Apple II. I've built versions for the Apple I and the Apple /// without much effort. The IDE is a bit of a hack job, though. It doesn't use relocatable modules and is just a big, ugly, monolithic blob that contains the editor, compiler, and runtime all together. It was written early on, before all the relocatable module stuff was solidified to prove it could be done (it only uses the main 64K, none of the banked memory). It still gives you about 16K of code space and about 8K of compiled code space to play with. Someday I should rewrite it as modules it which will get the banked memory support involved.

Kudos to the Beeb team.

Dave...

It would be great if the overall result is a more portable, more modular PLASMA!

[resman]

As Ed mentioned, I had no idea so much work had gone in to porting PLASMA to the Beeb. What an incredible effort! As you can see, the PLASMA VM itself is fairly portable, but the rest of the infrastructure to support the underlying platform is held together with duck tape and bailing wire. So to get this far with such poorly documented code is nothing short of miraculous.

I thought I would fill in a few of the gaps that I've left. First, the ROGUE game is actually a test of a visibility algorithm I was playing with. It goes well above and beyond most other rogue-like games in that line-of-site and distance visibility is strategic. I wrote up directions on how to play, but never checked it in. So I just rectified that:
https://github.com/dschmenk/PLASMA/blob ... uctions.md

Much of the reason behind the crusty state of the innards of PLASMA is that it is a tool for me to build other programs that just couldn't be built without it. Once I got something working, I moved on to using it on another project. One project PLASMA is key to is the Lawless Legends/Ancient Legends game engine: http://www.lawlesslegends.com
That is where the constant string (CS opcode) came from. The one person who as written as almost as much PLASMA as me, Martin Haye, wanted in-line strings to simplify the code. He was right, but I never came up with a real satisfactory implementation for when strings sat in extended/banked/external memory. I like what you've done here and perhaps its time to fix CS for real.

Also, there is a simple C version of the PLASMA VM. It's already in the source tree. Mainly used for verification of the VM and compiler, not really a complete PLASMA system. But a good place to start. You can run it by typing 'make test' in the PLASMA/src directory.

Others have done some cool stuff combining PLASMA with another project of mine, Apple2Pi: https://www.youtube.com/watch?v=wZSLrq5 ... ZZzydgqMWV, https://github.com/dschmenk/apple2pi
A CS professor in New Zealand is extending the PLASMA IDE to cross compile the source on the Pi, then copy the binary back to the Apple II for execution using the A2Pi communication channel; kind of similar to the Tube interface (in concept, not implementation).

Again, congratulations on your impressive work. It's always humbling to see someone else find value in a project of mine.

Dave...

[SteveF]

I haven't looked at this forum for about a week, so this was a nice surprise! Thanks for your kind words Dave, much appreciated - and thanks again for PLASMA, it's great, it has a very "big computer" feel about it which is impressive on an 8-bit system. (Dynamic linking!)

I certainly noticed the visibility algorithm in my brief games of ROGUE, but I hadn't realised it was so sophisticated. The Lawless Legends thing looks very cool and it's great to see PLASMA getting some use on big projects.

Having got the Beeb port into a semi-decent state, I've temporarily stopped work on it. Apart from general life getting in the way , I also started tinkering with creating an LLVM backend to emit PLASMA bytecode. I got as far as knocking up a crude proof of concept in Python to convince myself I could generate decent code for a stack architecture like PLASMA from LLVM IR. I then got stuck trying to climb the LLVM learning curve; I've created a stub backend in C++ but can't convince clang to use it, and I've been procrastinating on diving into the code to find out what I've missed. This is a project which is well outside my comfort zone, so I figured a) my chances of success are small but b) I might learn something...

I still intend to return to the Beeb port and perhaps polish off some of the rough edges, and once it's stable I definitely want to do that third version (single 16K language ROM) as well. It would be great if you wanted to take something like the CS opcode change from the Beeb port upstream; I'd hope to be able to keep this port in sync with upstream if it doesn't change too fast.

I would encourage any interested Beeb developers to try writing a PLASMA program and give this port a go; I'll try to keep an eye on this thread to help answer any questions!

Cheers,

Steve

[SteveF]

Here's another update, mostly revolving around upstream changes but with a few small tweaks to the Acorn port. I still have plans to do more with the Acorn port but I thought it would be better to pick up the latest upstream changes first.

Merged in latest upstream changes (version 00.99), including:

• The arguments to the call() function have been re-ordered - the address to call is now the first argument instead of the last. If you've written any of your own PLASMA programs, you'll need to fix any uses of call() or you'll get strange crashes. (All the sample programs have been updated.)
• Some PLASMA opcodes have changed their behaviour, so compiled modules are not compatible between this version and the previous versions.
• The PLASMA compiler now has a -O option to optimise the generated code. (It doesn't do everything my optimise.py does yet, but it's way less hacky.)
• The PLASMA compiler now allows the number of values returned by a function to be specified, so you can specify 0 return values to get slightly tighter code if a function has no useful return value, and you can return multiple values as well. See the (unmodified from upstream) README.md file for details.
• PLAS128's per-function-call string pool for the CS opcode is now based on the new upstream implementation instead of my excessively complex one. Mine used a 16-element hash table with the strings stored in linked lists, the upstream one uses a single contiguously stored list of strings. This shrinks the VM quite considerably and addresses the niggly known bug about changing mode after executing a CS opcode; because the string pool no longer contains any absolute addresses, it can be relocated with impunity as HIMEM moves around. Embarrassingly, the new implementation is also slightly faster, at least in the simple cases I've tried. (Code for which performance is absolutely critical might want to continue to use global strings to avoid any overhead, but I find it hard to imagine it would really be necessary.)

Acorn port specific changes:

• The VM source code has been tidied a bit with fewer literal constants and some small optimisations.
• A small bug in error handling in the DUMP example has been fixed; it previously didn't close open file handles correctly under all circumstances.
• There's a new SYMTBL test to allow the non-contiguous symbol table to be examined.

The source is at https://github.com/ZornsLemma/PLASMA/tree/stardot5 and I've attached the SSD built from that source.

[BigEd]

Sounds great - I haven't had a go yet but I like the look of this project.

[SteveF]

Time for another update. As always, the code can be seen on the 'bbc' branch of my repo as mentioned in the first post, and I've attached a build of the plasma.ssd it generates.

The latest upstream changes have been merged, including:

• Lambda functions are now supported by the PLASMA compiler
• New CFFB VM opcode to allow more compact representation of small negative constants
• Magic constant in binary modules has changed, so modules are not compatible between this version and older versions
• New optimisations added to the compiler's peephole optimiser; the hacky optimise.py optimiser is now almost redundant. Just specify '-O' when invoking the compiler to allow optimisation.

Changes specific to the Acorn port include:

A new 'driver' utility has been added which should make invoking the compiler a lot more convenient. Simple example:

Code: Select all

steven@nine:/tmp/hello$ ls -l
total 4
-rw-r--r-- 1 steven steven 64 Aug 27 21:47 hello.pla
steven@nine:/tmp/hello$ cat hello.pla
import cmdsys
    predef puts
end

puts("Hello, world.\n")
done
steven@nine:/tmp/hello$ plasmac.py --ssd hello.pla
steven@nine:/tmp/hello$ ls -l
total 204
-rw-r--r-- 1 steven steven     64 Aug 27 21:47 hello.pla
-rw-r--r-- 1 steven steven 204800 Aug 27 21:47 hello.ssd

The hello.ssd file is a bootable Acorn DFS disc image which can be used directly with an emulator (or transferred to a real machine, of course). SSD creation is handled by David Boddie's Python module - thanks David!

There's also experimental support for building standalone executables - just add '--standalone' to the plasmac.py command line. The resulting program effectively takes over the machine as the current language, so it should be written to never return.

plasmac.py also tries to do the right thing if the code it's compiling imports other modules, e.g. it will automatically attempt to locate and compile these and include them on an SSD image without being explicitly told to.

I've added a new demonstration program (sort.pla, with subsidiary modules sort1.pla and sort2.pla) which shows how a large program can be split into parts which don't all have to be in memory simultaneously. Use "+SORT" to run this at the PLASMA prompt; it does a very slow and silly bubble sort which avoids having the code to display the array and perform one pass of the bubble sort in memory simultaneously, just as an example.

There are some experimental additions to the compiler's peephole optimiser.

[davidb]

SSD creation is handled by David Boddie's Python module - thanks David!

You're welcome! Let me know if you encounter any problems with it.

[BrianH]

Hi SteveF.

So I only recently got introduced to PLASMA. I think what Dave has created is really cool, and I share your interest in having it eventually be available for other 8-bit retro computers. I've been looking at the changes you've made for the Beeb in your GitHub repo. I have also done some brief experimentation with it on my own BBC Master 128.

My main struggle is with delineating between things you added/changed for convenience or access to "advanced Beeb features" vs. things you added/changed just to get a basic PLASMA system working. I'm new to PLASMA and probably being naive, but I'm hoping that a relatively simple port of the PLVM and the most important modules (cmdsys?) is enough to get a functioning PLASMA up-and-running on the Beeb. Even if I was limited to 32KB for code and data, I'd be Ok with that for a long time. I want to be able to create small apps in PLASMA that probably only need 16KB for code, very little dynamic data, and access to the video/console. So I am sort of agnostic right now on support for sideways RAM & ROM or lots of other hooks into the Beeb's MOS or DOS facilities.

Anyway, my agenda here is to see if I can pick your brain over what a minimal port looks like for the Beeb, and whether it is practical to start with a minimal port, and maybe add the more advanced Beeb capabilities using optional PLASMA modules?

I also created this "placeholder" for a Porting Guide on the Wiki. Perhaps the best way that we could all collaborate is if you were interested in contributing something to that document here: https://github.com/dschmenk/PLASMA/wiki/Porting-Guide

Best.
BrianH

[SteveF]

Hi Brian,

I hope this isn't too waffly a reply; I can see a few ways you could choose to go with this so I've tried to sketch out various possibilities. Apologies if this comes across as either patronising or lacking in detail or both, it's not intended...

Just a general observation first: I am going to suggest that you look at some of the older versions in my repository in what follows. Please be aware, if you aren't already, that the VM is still evolving upstream - new opcodes get added and some opcodes get repurposed. So if you do decide to do something starting with an older version of my repository, you will either need to a) stick with the upstream PLASMA compiler from the same date or b) port the newer upstream changes into the older version from my repository.

I have made a few changes to the PLASMA compiler itself (toolsrc/*), but I don't think these are really related to the Acorn port - they're experimental optimisations which would be equally useful (or useless) for the Apple version. I've sent some of them upstream already, the ones which haven't been upstreamed I am still evaluating. So one immediate and fairly cheap win if you're after minimal divergence from upstream would be to switch to using the upstream version of the compiler (but from a suitable date, as noted above). On the other hand, barring bugs in my experimental optimisations, any code you write should work equally well with my compiler or the upstream version - they just aren't that different.

Moving on to the VM itself, I think it's useful to consider the assembly language part first (vmsrc/plvmbb*.s for my port, vmsrc/plvm01.s for the Apple I version, vmsrc/plvm02.s for the Apple II version). If I just look through a diff between plvm02.s and plvmbb.s, I think the main changes in my port are:

• Changes in the addresses of various ZP locations. I think this is mostly inevitable.
• Some new macros to check for things like stack underflow/overflow. These could be easily removed to reduce diverge from upstream.
• PLAS128 (the banked RAM support). Manually removing all the conditionally compiled code for the PLAS128 case would remove most/all of this divergence.
• A bit of code reordering to keep the VM size down. It would be a little bit tedious to undo this, but not particularly difficult.
• Some extra (and relatively self-contained) code to relocate the VM code automatically to PAGE to give as much free RAM as possible. This could easily be removed.

So that part really hasn't diverged all that much. It might therefore be worth you re-using this (perhaps with the changes suggested above, according to taste) for producing a minimal port.

The part of the VM written in PLASMA itself (vmsrc/bbcmd.pla for my port, vmsrc/a1cmd.pla for the Apple I version, vmsrc/a2cmd.pla for the Apple II version) - let's call this the "shell" for want of a better name - has certainly undergone a lot more modification in my port. I have tried to avoid making any changes which would cause avoidable source-code incompatibilities from upstream - i.e. I hope that a PLASMA program which doesn't use Apple-specific memory addresses or OS calls would compile and run unaltered on my port. I also tried to factor out new functionality like the 32-bit arithmetic library and some of the Acorn OS support into its own loadable modules, as you suggest (DWORD and ACORNOS, respectively).

Where I included Acorn-specific code in the core VM it was either because the core VM itself needed it anyway or because it was tied so closely to the core VM implementation that separating it out would be infeasible:

• The mode() command, for example, needs different code in the PLASMA and PLAS128 versions of the VM, which would get messy if it lived in a loadable module. (mode() is also exposed at the PLASMA prompt so in a very weak sense the core VM itself needs it, but I only added support at the prompt because it was cheap and felt a lot nicer - previously you had to change mode before starting the VM.)
• setjmp() and longjmp() were my solution to mapping Acorn OS errors (generated via BRK) into the PLASMA VM. Again, these are tied quite closely to the VM internals and differ between PLASMA and PLAS128.[There have been some similar functions added upstream recently which I need to consider as possible replacements; I am still porting the latest upstream code so I haven't got round to looking at this yet.]

I also made changes to the core VM which scratched my own itches in playing around with PLASMA, but which aren't really Acorn-specific:

• Checking for heap full and generating an error if this occurs
• Allowing the symbol table to grow without bounds
• The callalloca() function as a way to have large local arrays

In principle I could submit these to upstream for inclusion, but for various reasons I haven't - although I have come close to doing this for callalloca(), as it's relatively unintrusive and I find it useful. (On the other hand, I mainly find it useful for being able to allocate a 256-byte local array to deal with things which could be that large on the Acorn OS, like pathnames, so it may be that on the Apple this is less valuable.) An Acorn port without these would be completely possible and would be simpler (if perhaps more frustrating to develop on) and closer to upstream. However, I think if they're going to be there, they have to be in the core VM rather than in modules.

It would be possible to unpick some or all of these changes in the shell, but the changes for the growable symbol table are pretty intrusive and you might find it easier to start from scratch with the upstream 'cmd.pla' or 'a1cmd.pla' and perhaps just use my port's code as a source of ideas and fragments of code.

The first version of my port posted in this thread (and in the github repo with the 'stardot1' tag) was IIRC based on the upstream Apple I code. The Apple I code is much simpler than the Apple II code - there's no banked RAM support, for example - and of course my port had had less time to diverge from upstream as well. So you might like to take a look at that, and if you want something with the latest upstream features, you could perhaps try porting the latest Apple I VM - maybe using my 'stardot1' tag as a base, or perhaps porting 'a1cmd.pla' but re-using the assembly language part of the latest version of my port.

If you really want to go minimal, you could also consider throwing away the shell and *replacing* it with your own application code. (This is what the --standalone option in my plasmac.py driver does, but it's quite fiddly as it tries to work with arbitrary PLASMA source code.) That way you don't get any dynamic module loading - everything's effectively statically linked up front - but you also avoid wasting RAM on code you don't want. By throwing away the shell, it also automatically removes most of the divergence between my port and upstream. I've created a simple example of this (based off the old 'stardot1' repo for the simplest, least-modified version of the VM) here: https://github.com/ZornsLemma/PLASMA/tr ... n-stardot1 If you build that, the Acorn "PLASMA" executable file on the SSD is actually just a standalone "Hello, world!" program. Have a look at vmsrc/bbcmd.pla.

To sum up:

• If you want a minimal port which doesn't diverge too much from upstream, I think you might well find it a useful time saver to start with the assembly language part of my port (vmsrc/plvmbb*.s), either as-is or manually removing the features you don't want. Doing this would also mean you'd get to re-use any future work I do to port new upstream versions.
• I am open to be convinced, but I don't think I can factor out any of my changes to the core VM into loadable modules - I've already done this where I thought I could. They could certainly be removed completely in a fork/alternate port, but I think that if they are present, they have to be in the core VM.
• If you do want a minimal port, the upstream Apple I code is probably a good place to start as its hardware is probably most like that of a standard 32K BBC B. The 'stardot1' tag in my repo is an outdated version of such a port which might be helpful to you.
• If you can live without the "environment" provided by the PLASMA shell, you could go very minimal by replacing the shell with your own application.

I'd certainly be happy to contribute to the wiki porting guide but at this rather waffly stage I thought it would be better just to post this here. If you'd like to copy any of this onto the wiki, do feel free!

Cheers.

Steve

[BrianH]

Hi Steve.

Thanks for the very detailed response. This is absolutely clarifying many of the questions I was thinking about. I'm going to do some more experimenting with your port to try to validate whether a minimal VM port can really work for me, combined with the assumptions I am making about my workflow.

This may be a point of divergence in how we view workflow, but I am thinking mainly of:
1. Edit application code in Linux
2. Compile/assemble using PLASMA/ACME in Linux
3. Test simple console-style apps using the Linux PLVM and console I/O
4. Test apps that use Beeb-specific modules using Beeb PLVM and Linux Beeb emulator
5. Finally, move app to real Beeb, for standalone execution/testing.

If that turns out to be a realistic (and perhaps preferred by me) workflow, then you can see how the minimal port with minimal divergence from upstream sort of becomes the easiest path. Well, it seems easiest until or unless I want to build sophisticated apps and access more of the memory in my Master 128.

Cheers.

[resman]

I've made a first pass at multiple platform support in the mainline PLASMA project. Mind you it is very preliminary and as I told Brian, I'm a makefile moron. But is does successfully build apple and commodore 64 versions (usually). The C64 requires different zero page locations for the VM, so it was a good test to make sure all the libraries built against the proper plvmzp.inc file. I tried to keep as much common code as possible. I'm sure there will be cases I forgot where apple-isms still exist. Just need to put those in the platform specific subdirectory.

Give it a glance and let me know what I got wrong. And yes, the makefile is an abomination.

Dave...

[SteveF]

Thanks Dave (and Brian)! I noticed the issue for this was closed the other day. I'll try to take a look this weekend.