Help required for the new BBC games archive

feedback, questions and discussion relating to the Complete BBC Games Archive (beta site now open!)
User avatar
pau1ie
Posts: 668
Joined: Thu May 10, 2012 9:48 pm
Location: Bedford
Contact:

Re: Help required for the new BBC games archive

Post by pau1ie » Tue May 24, 2016 7:32 pm

I have made a start. I don't want to say where because it is my owncloud server, so I don't want search engines indexing vulnerabilities if I can help it, but I can PM anyone the link if they like. Note I have not implemented downloads yet, so if you just want the games you are better downloading the images already uploaded to the forums!

I think Richard is correct that we need to keep to do this one step at a time,

So far I have the list of games, which can be filtered by publisher or year.
I need to add category too.
And a user interface for all these.

The big thing I am missing is screenshots. I have done the first 40 or so, but there are a couple of thousand games.

This proposed site and Daves are similar in some ways, but I think having a BBC site is worthwhile as it allows things like the play button. I have to hold myself back from this, I think it will be the first thing I want to do after meeting the first specification.

Richard - you mentioned you would have time to work on it. Are you interested in collaborating? If so how do we share code? There is always the chance you will throw up your hands in horror at my code and refuse to touch it, as it isn't all that pretty.

User avatar
sydney
Posts: 2423
Joined: Wed May 18, 2005 9:09 am
Location: Newcastle upon Tyne
Contact:

Re: Help required for the new BBC games archive

Post by sydney » Tue May 24, 2016 7:47 pm

pau1ie wrote:I have made a start. I don't want to say where because it is my owncloud server, so I don't want search engines indexing vulnerabilities if I can help it, but I can PM anyone the link if they like.
I'll have the link please so I can see the current game list.
The big thing I am missing is screenshots. I have done the first 40 or so, but there are a couple of thousand games.
How many screenshots per game and how do you want me to get them to you? What size and format? Beebem goes up to 1280 x 1024 and offers BMP, JPG, GIF or PNG.I could put them in my dropbox and send you a link if you want.Once I've had a look at the list I'll make a start.

User avatar
pau1ie
Posts: 668
Joined: Thu May 10, 2012 9:48 pm
Location: Bedford
Contact:

Re: Help required for the new BBC games archive

Post by pau1ie » Tue May 24, 2016 8:12 pm

Lee - Can you post the spreadsheet and disc images that you sent me here?

I have not yet imported the latest spreadsheet, so that would be better, I am using the spreadsheet at viewtopic.php?f=32&t=8270&p=114370&hili ... es#p114370 at present.

I'll PM the link anyway.

For screenshots I was just using what b-em does by default on my computer which is 676 × 548. I don't imagine there is any point going bigger because pixels are just duplicated? So long as they are consistent we can easily use a batch process to resize them which I may do for thumbnails at some stage.

I settled on jpg as I assumed the images would end up smaller.

At first just one image per game. I can see the value in having a gallery for each game, but it's not in the original spec.

Thanks!

PaulH

User avatar
sydney
Posts: 2423
Joined: Wed May 18, 2005 9:09 am
Location: Newcastle upon Tyne
Contact:

Re: Help required for the new BBC games archive

Post by sydney » Tue May 24, 2016 8:19 pm

I think Lee has sent me the same files. Is there any preference on the order in which you need them? Alphabetical / by disk / by genre? I can start on games beginning with A or on DISC001.

EDIT: Just seen you're part way through 'A' so I'll start in 'B' if that's ok?

User avatar
Arcadian
Site Admin
Posts: 2973
Joined: Fri Nov 24, 2000 12:16 pm
Contact:

Re: Help required for the new BBC games archive

Post by Arcadian » Tue May 24, 2016 8:28 pm

pau1ie wrote:Richard - you mentioned you would have time to work on it. Are you interested in collaborating? If so how do we share code?
We could always set you both up with access to a private forum (together with any other contributors) if that's any use?
For a "Complete BBC Games Archive" visit www.bbcmicro.co.uk

User avatar
Arcadian
Site Admin
Posts: 2973
Joined: Fri Nov 24, 2000 12:16 pm
Contact:

Re: Help required for the new BBC games archive

Post by Arcadian » Tue May 24, 2016 8:33 pm

Oh and re: the screenshots, as long as you've got 'smoothing' turned off* the resultant images should only be a maximum of 8 colours, meaning if you save as PNG the file size should be very small (and not only that, they'll also be lossless, compared with e.g. JPG).

* excluding Mode 7 screens

But thanks Paul1e, Richard for stepping forward - can't wait to see how this progresses!
For a "Complete BBC Games Archive" visit www.bbcmicro.co.uk

User avatar
pau1ie
Posts: 668
Joined: Thu May 10, 2012 9:48 pm
Location: Bedford
Contact:

Re: Help required for the new BBC games archive

Post by pau1ie » Tue May 24, 2016 8:48 pm

sydney wrote:I think Lee has sent me the same files. Is there any preference on the order in which you need them? Alphabetical / by disk / by genre? I can start on games beginning with A or on DISC001.

EDIT: Just seen you're part way through 'A' so I'll start in 'B' if that's ok?
That's a good idea! Can you name the files so its obvious what they are?
Arcadian wrote:
pau1ie wrote:Richard - you mentioned you would have time to work on it. Are you interested in collaborating? If so how do we share code?
We could always set you both up with access to a private forum (together with any other contributors) if that's any use?
I was thinking maybe using a source control system. I don't think theres a need to keep anything private - I'd quite like to release the bits of the website so anyone can recreate it easily. We have had a few websites disappear, so I would like to give this one the best chance to come back from the dead should that be necessary. Of course that's just my opinion, and it is a community effort.

User avatar
leenew
Posts: 3889
Joined: Wed Jul 04, 2012 3:27 pm
Location: Doncaster, Yorkshire
Contact:

Re: Help required for the new BBC games archive

Post by leenew » Tue May 24, 2016 10:08 pm

Seeing as Dave E. has (I think :D ) kindly offered that we may use his screen shots..... are we not going down that route? Seeing as he has taken about a million :shock:

Lee.
Oh! Ps. I will post the latest spreadsheet tomorrow evening as I am doing a couple of alterations on it tomorrow afternoon.
I will have to pop the game images somewhere as they are about 18mb.
Lee.

User avatar
sydney
Posts: 2423
Joined: Wed May 18, 2005 9:09 am
Location: Newcastle upon Tyne
Contact:

Re: Help required for the new BBC games archive

Post by sydney » Tue May 24, 2016 10:38 pm

I've done about 30 so far but will stop until I know we need them.

User avatar
simonm
Posts: 285
Joined: Mon May 09, 2016 2:40 pm
Contact:

Re: Help required for the new BBC games archive

Post by simonm » Tue May 24, 2016 10:50 pm

I was thinking maybe using a source control system. I don't think theres a need to keep anything private - I'd quite like to release the bits of the website so anyone can recreate it easily. We have had a few websites disappear, so I would like to give this one the best chance to come back from the dead should that be necessary. Of course that's just my opinion, and it is a community effort.
Good idea. GitHub is free and easy to use for this type of thing. I couldn't agree more that the more freely available the code is, the more likely it is that it will be developed and maintained by multiple folks within the community.

That said, I really would think carefully about committing to building a new website from scratch, as although it's a seductive idea, in practice it's extremely hard work to build something that works well, scales well, and is secure. Plus the ongoing cost and administration of server hosting and backup tends to be higher for a bespoke solution.

If the primary goal is making archived bbc gaming related content freely available, it would surely be better to expend energy on creating that content, instead of building website tech from the ground up, so I'd suggest a CMS would be a better/faster option. As someone pointed out, yes you might have to keep them updated (depending on your hosting service), but at least you have a well known format for data that can be easily backed up and/or migrated across servers in emergencies.

One CMS option would be MediaWiki- if a more community oriented approach is desirable. This allows content to be open sourced, so the burden of work doesn't fall on just a few mods. I personally think Wikis look a bit dull, and they can easily fizzle out if the community lose interest and stop adding content.

Another option is WordPress- it's a really good package now, the WP themes are amazingly versatile and configurable.
Here's an example of an arcade games oriented theme -would look ace if all those thumbnails were Beeb games! :) With that theme, (and I'm guessing) it looks like you can author a page for each game and add links, comments ratings etc.
The good thing about WP themes is that you can hack them about to customize them even further (eg. hack in buttons to the above theme for Downloading SSD files, Play in JS beeb etc.). Doing this would likely be much easier than writing a website from scratch.
I'd suggest getting a good theme to start out with, then check that into Github, for open source code customization of it to suit whatever folks think is needed. The good thing is that there's literally thousands of themes out there, and a huge knowledge base of how to build/customize them. WP is not just for blogs and bog standard websites anymore.

In terms of hosting, a cloud service would be the easiest option. wordpress.com has all-inclusive custom domain hosting from £133/year. Siteground.com offers custom WP hosting from £36/year or so. Auto upgrades, backups and stuff like that. There's a wealth of other options on the cloud hosting front.


Food for thought anyways.

User avatar
pau1ie
Posts: 668
Joined: Thu May 10, 2012 9:48 pm
Location: Bedford
Contact:

Re: Help required for the new BBC games archive

Post by pau1ie » Tue May 24, 2016 11:53 pm

Teaser. image. It is a start.
Attachments
site.jpg

User avatar
Arcadian
Site Admin
Posts: 2973
Joined: Fri Nov 24, 2000 12:16 pm
Contact:

Re: Help required for the new BBC games archive

Post by Arcadian » Wed May 25, 2016 10:14 am

Like the look of that! :)

Very promising start, thanks for sharing!
For a "Complete BBC Games Archive" visit www.bbcmicro.co.uk

User avatar
sydney
Posts: 2423
Joined: Wed May 18, 2005 9:09 am
Location: Newcastle upon Tyne
Contact:

Re: Help required for the new BBC games archive

Post by sydney » Wed May 25, 2016 10:21 am

sydney wrote:I've done about 30 so far but will stop until I know we need them.
Anyone want to say whether we are going to use Dave's images or not. I could be screenshotting (is that a word?) right now.

User avatar
leenew
Posts: 3889
Joined: Wed Jul 04, 2012 3:27 pm
Location: Doncaster, Yorkshire
Contact:

Re: Help required for the new BBC games archive

Post by leenew » Wed May 25, 2016 10:52 am

Dave E. Needs to reply and give us a line of communication really... [-o<
Lee

User avatar
daveejhitchins
Posts: 4955
Joined: Wed Jun 13, 2012 5:23 pm
Location: Newton Aycliffe, County Durham
Contact:

Re: Help required for the new BBC games archive

Post by daveejhitchins » Wed May 25, 2016 11:22 am

Does this mean we have all the titles with Primary and Secondary Genres attached :mrgreen:

Cant imagine why I would want those . . . :roll:

By the way . . . All looking good - a round of applause to all involved =D> =D> =D> =D> It's all really appreciated!

Dave H :D
Parts: UM6502CE, GAL22V10D, GAL16V8D, AS6C62256A, TC514400AZ, WD1772, R6522, TMS27C512, AT28C256
Products: ARA II, ARA III, ABR, ATI, AP6, MGC, AP5 . . .
For a price list, contact me at: Retro Hardware AT dave ej hitchins DOT plus DOT com

User avatar
pau1ie
Posts: 668
Joined: Thu May 10, 2012 9:48 pm
Location: Bedford
Contact:

Re: Help required for the new BBC games archive

Post by pau1ie » Wed May 25, 2016 3:43 pm

daveejhitchins wrote:Does this mean we have all the titles with Primary and Secondary Genres attached :mrgreen:
I haven't done genres yet. I want to get search on date and publisher working (hopefully this evening) then I will reimport the new spreadsheet and work on genres. Should primary and secondary genres be reflected in the UI? or can I just make it so a game can be in any number of genres?
daveejhitchins wrote: Cant imagine why I would want those . . . :roll:
I'll bite. Why would you want the genres?

Changing the subject - Does anyone have any design expertise? I thought the colours of the site should in some way reflect the BBC computer. (Beige, red, dark brown). If someone could come up with a nice palette that would be good. If not I'll have a go once the coding is done.

Cheers

PaulH

User avatar
daveejhitchins
Posts: 4955
Joined: Wed Jun 13, 2012 5:23 pm
Location: Newton Aycliffe, County Durham
Contact:

Re: Help required for the new BBC games archive

Post by daveejhitchins » Wed May 25, 2016 3:52 pm

pau1ie wrote:I'll bite. Why would you want the genres?
This for this . . .

Dave H :D
Parts: UM6502CE, GAL22V10D, GAL16V8D, AS6C62256A, TC514400AZ, WD1772, R6522, TMS27C512, AT28C256
Products: ARA II, ARA III, ABR, ATI, AP6, MGC, AP5 . . .
For a price list, contact me at: Retro Hardware AT dave ej hitchins DOT plus DOT com

User avatar
Arcadian
Site Admin
Posts: 2973
Joined: Fri Nov 24, 2000 12:16 pm
Contact:

Re: Help required for the new BBC games archive

Post by Arcadian » Wed May 25, 2016 5:59 pm

pau1ie wrote:Should primary and secondary genres be reflected in the UI? or can I just make it so a game can be in any number of genres?
Single top-level genre, multiple sub-genres?

I think if you use the top-level genres I suggested on the mockup, you'll be able to identify a single "main" genre for each game relatively easily.
For a "Complete BBC Games Archive" visit www.bbcmicro.co.uk

User avatar
pau1ie
Posts: 668
Joined: Thu May 10, 2012 9:48 pm
Location: Bedford
Contact:

Re: Help required for the new BBC games archive

Post by pau1ie » Wed May 25, 2016 7:27 pm

daveejhitchins wrote:Does this mean we have all the titles with Primary and Secondary Genres attached :mrgreen:
Does Lees spreadsheet contain what you want? There is a copy here

viewtopic.php?f=32&t=8270&p=114370&hili ... es#p114370

I believe he is about to upload an updated version soon.

User avatar
leenew
Posts: 3889
Joined: Wed Jul 04, 2012 3:27 pm
Location: Doncaster, Yorkshire
Contact:

Re: Help required for the new BBC games archive

Post by leenew » Wed May 25, 2016 7:32 pm

Latest spreadsheet attached, I have never been happy with the genre section though!
BBC Disc And Tapes NEW.xls
(911.5 KiB) Downloaded 38 times
The spreadsheet now contains a "filename" section, that refers to the filename of the SSD for that game.
Some small errors have been squashed, and some renaming done, to help the automation of the DSD to SSD script match up to the spreadsheet (alphabetically speaking)... I think that does not make sense :D

Paul - check your mail!

Lee.
Last edited by leenew on Thu May 26, 2016 9:38 am, edited 1 time in total.

User avatar
pau1ie
Posts: 668
Joined: Thu May 10, 2012 9:48 pm
Location: Bedford
Contact:

Re: Help required for the new BBC games archive

Post by pau1ie » Wed May 25, 2016 9:47 pm

Thanks Lee

I haven't made as much progress as I'd hoped this evening. I can search on year, and am working on publisher. I don't think I'll get another chance to work on it till about Monday.

By the way, I notice that Escape from Ebony Tower has year 984 at one point (missing the initial 1)

Cheers

PaulH

User avatar
daveejhitchins
Posts: 4955
Joined: Wed Jun 13, 2012 5:23 pm
Location: Newton Aycliffe, County Durham
Contact:

Re: Help required for the new BBC games archive

Post by daveejhitchins » Thu May 26, 2016 11:54 am

Yes, it was an enormous help. I'm adding sub Genres as we speak! I'm now going to follow the Beeb archive method.

Dave H :D
Parts: UM6502CE, GAL22V10D, GAL16V8D, AS6C62256A, TC514400AZ, WD1772, R6522, TMS27C512, AT28C256
Products: ARA II, ARA III, ABR, ATI, AP6, MGC, AP5 . . .
For a price list, contact me at: Retro Hardware AT dave ej hitchins DOT plus DOT com

User avatar
Arcadian
Site Admin
Posts: 2973
Joined: Fri Nov 24, 2000 12:16 pm
Contact:

Re: Help required for the new BBC games archive

Post by Arcadian » Thu May 26, 2016 12:00 pm

leenew wrote:I have never been happy with the genre section though!
Yeah - I think the spreadsheet, in particular the genre sections, could probably do with a few tweaks.

For example, if we work on the basis that each game will have one main Genre* and can be labelled with, say, between zero and three 'Sub Genres' - e.g. an Action game such as Planetoid could be 1) an Arcade Clone 2) a Horizontal Shooter 3) a Scrolling Shooter - so in that case we'd need a new column inserting for Genre, plus a third sub-genre column adding.

* Proposed main genres = Action, Compilation, Educational, Leisure, Puzzle, Racing / Driving, Role Playing, Simulation, Sports, Strategy, Text/Graphic Adventure (Comments?) These top-level genres have been based of mobygames.com, btw - with one addition: Leisure.

Also currently missing is a column to denote the Status e.g. Commercial Release (or Type In listing, Public Domain, Demo) because I think end-users would find it useful to toggle between the display of commercial games only/all games.

Finally, I wonder if the "DiscXXX-" prefixes should be removed from the filenames? (I know that if I had downloaded a big collection of emulator images, I'd prefer to have them named in alphabetical order of title, as locating files would be a lot easier/faster). Obviously would need to ensure that there were no duplicate filenames. So perhaps append the publisher name to the end of every filename e.g. JetSetWilly-Tynesoft.ssd

But before anybody changes the spreadsheet - could Paul perhaps provide some tips on the how we could best be modifying it to suit future imports into his web database?
For a "Complete BBC Games Archive" visit www.bbcmicro.co.uk

User avatar
pau1ie
Posts: 668
Joined: Thu May 10, 2012 9:48 pm
Location: Bedford
Contact:

Re: Help required for the new BBC games archive

Post by pau1ie » Thu May 26, 2016 4:37 pm

Arcadian wrote:could Paul perhaps provide some tips on the how we could best be modifying it to suit future imports into his web database?
When I start working on genre I will just import the whole thing as is - I need to do this as I haven't got the filenames in my present dataset. Then I will put the data where I want it. One line per game is good, so if you want extra secondary genres, just use extra columns.

I think you need to decide what you are not happy about. I can see some minor inconsistencies that will be easy to fix (e.g. board games have the type (e.g. reversi) in both genre1 and genre2, I can easily strip it out of genre1 and make sure it is in genre2, so I wouldn't bother spending time on that, but if there are any additional subgenres you want, then add those in.

Regarding the genre list, I don't really care, but I would suggest looking at the list we already have, stripping anything in brackets and using that as a sub category, combining similar categories, then looking at removing genres that are not used by many games (e.g. Spray em up, run n gun, reaction tester, pong, multi event, maze shooter, breakout, and my favourites: eh and WTF). Maybe there should be a category of Other, but I think it should have more than 10 games in it to be useful. Much of this can be done quite easily with SQL, so it doesn't have to be perfect when I import it. It is always going to be subjective.

I'd suggest status wasn't in the original spec so should be left for phase 2...

I hope this helps.

Cheers

PaulH

User avatar
Dave_E
Posts: 757
Joined: Fri Jun 16, 2006 12:55 pm
Location: Middlesbrough
Contact:

Re: Help required for the new BBC games archive

Post by Dave_E » Fri May 27, 2016 10:42 am

I don't know what you guys have but the complete list of games that I know of for the BBC is here on my own site:

http://www.everygamegoing.com/landing/m ... games.html

You can click a game for its information. Lots of games have genre and brief details.

Might be good to hear from PaulH and see a copy of his spreadsheet so I can see if we can avoid you guys duplicating a lot of the work that's already been done.

User avatar
leenew
Posts: 3889
Joined: Wed Jul 04, 2012 3:27 pm
Location: Doncaster, Yorkshire
Contact:

Re: Help required for the new BBC games archive

Post by leenew » Fri May 27, 2016 10:51 am

Hi,
Here is the latest spreadsheet with a bit more tidying up.
MB Discs NEW.xls
(466 KiB) Downloaded 34 times
Apart from discs 105 to 107 (which I am looking at) the filename section is complete.
Arcadian's suggestion of stripping the disc number has been noted.

DaveE. I think it is screenshots that would save a massive amount of work for this site :D

Lee.

User avatar
PitfallJones
Posts: 447
Joined: Fri Feb 22, 2008 3:44 pm
Contact:

Re: Help required for the new BBC games archive

Post by PitfallJones » Fri May 27, 2016 11:33 am

Hi there,

My old program BBCLIST has 1000s of screenshots:

http://www.stairwaytohell.com/essential ... t-v1.0.zip

And also has a database of ~2100 games
bbclist.jpg
- PJ

User avatar
pau1ie
Posts: 668
Joined: Thu May 10, 2012 9:48 pm
Location: Bedford
Contact:

Re: Help required for the new BBC games archive

Post by pau1ie » Thu Jun 02, 2016 11:12 pm

I have uploaded my latest effort to my personal server. It still looks pretty much the same as the screenshot apart from the search form is filled out. I am now working on genre.

Progress against spec is:

- stores standard, unprotected ssd/dsd disc images only - Not done
- stores limited Metadata e.g. Year, Done
+ Genre, Sub Genre, In Progress
+ Possibly Author. Not in spreadsheet, so out of scope for now.
- searchable and browsable by Title, Done
+ Publisher, Done
+ Year Done

I have narrowed down genre more or less as I thought, mostly by incorporating those only used by 1-3 games into others, so the list is something like this, where count is the number of games in the genre. I haven't spent ages on it, mostly put them into other, so I hope I haven't butchered things too much.

+----------+-------------------+
| count(*) | genre |
+----------+-------------------+
| 7 | Beat 'Em Up |
| 8 | RPG |
| 8 | Avoid 'em |
| 12 | Collect 'Em Up |
| 14 | Ball Game |
| 14 | Sports |
| 15 | Card Game |
| 17 | Lunar Lander |
| 17 | Other |
| 21 | Quiz |
| 24 | Catch 'em |
| 27 | Bat And Ball |
| 30 | Traditional Games |
| 34 | Strategy |
| 36 | Platform Puzzler |
| 44 | Driving |
| 46 | Board Game |
| 53 | Arcade Adventure |
| 54 | Puzzle |
| 64 | Sport |
| 86 | Simulation |
| 190 | Shooter |
| 210 | Adventure |
| 224 | Platformer |
| 262 | Space Shooter |
| 355 | Maze |

User avatar
Dave_E
Posts: 757
Joined: Fri Jun 16, 2006 12:55 pm
Location: Middlesbrough
Contact:

Re: Help required for the new BBC games archive

Post by Dave_E » Fri Jun 03, 2016 7:38 am

I'm fine for you to use screenshots. However, if you want to give me a spec for an API to get them then that might be useful (unless you're going to right click download each and every one).

User avatar
leenew
Posts: 3889
Joined: Wed Jul 04, 2012 3:27 pm
Location: Doncaster, Yorkshire
Contact:

Re: Help required for the new BBC games archive

Post by leenew » Fri Jun 03, 2016 8:39 am

Keep up the good work Paul! =D>
Check your e-mail too, for the latest info, and some author info.
@DaveE, thanks again for use of the screenshots! I hope Paul knows how to hoover them up in bulk :mrgreen:

Lee.

Post Reply