485 replies to this topic

[headkaze]

GameEx Database Project

What is it?

This is a new project to create databases for GameEx that will include information about games for many different systems and emulators supported by GameEx.

It allows us to:

- Sort games by name, category, publisher, date and number of players
- Read descriptions about games
- Use 3 different naming conventions for our game sets (GoodTool/NoIntro/TOSEC)
- Create online databases for popular games

What about the current databases?

The current databases are incomplete and the format for them has changed. All the databases will be re-created.

How do we collect the data?

Using an advanced software program called "GameEx DBM" (GameEx Database Manager) which is used to manage, maintain and allow data entry via a friendly user interface as well as provide tools for conversion, searching and scraping the web, syncronising data across databases and management of naming conventions (ie. GoodTool/NoIntro/TOSEC)

Team Members

Project / Software Manager: HeadKaze
Data Manager: Flash
Data Entry: Flash, Lawrence

Database Stats (Generated by GameEx DBM v2.0 By Ben Baker on 12/08/2009)

[td=0]46[/td][td=0]0.0%[/td][td=0]58.7%[/td][td=0]41.3%[/td][td=0]0.0%[/td][td=0]Unfinished[/td][td=0]Acorn Archimedes[/td][td=0]PC[/td][td=0]Flash[/td][td=0]Flash[/td][td=0]1.034[/td][td=0]116[/td][td=0]100.0%[/td][td=0]0.0%[/td][td=0]0.0%[/td][td=0]0.0%[/td][td=0]Unfinished[/td][td=0]Acorn BBC Micro[/td][td=0]PC[/td][td=0]Flash[/td][td=0]Flash[/td][td=0]1.34[/td][td=0]1708[/td][td=0]99.9%[/td][td=0]0.0%[/td][td=0]0.1%[/td][td=0]0.0%[/td][td=0]Unfinished[/td][td=0]Acorn Electron[/td][td=0]PC[/td][td=0]Flash[/td][td=0]Flash[/td][td=0]1.271[/td][td=0]632[/td][td=0]99.8%[/td][td=0]0.0%[/td][td=0]0.2%[/td][td=0]0.0%[/td][td=0]Unfinished[/td][td=0]Amstrad CPC[/td][td=0]PC[/td][td=0]Flash[/td][td=0]Flash[/td][td=0]1.601[/td][td=0]7400[/td][td=0]15.9%[/td][td=0]0.0%[/td][td=0]0.0%[/td][td=0]84.1%[/td][td=0]Unfinished[/td][td=0]Apple II[/td][td=0]PC[/td][td=0]Flash[/td][td=0]Flash[/td][td=0]1.262[/td][td=0]1517[/td][td=0]44.2%[/td][td=0]0.3%[/td][td=0]0.0%[/td][td=0]55.4%[/td][td=0]Unfinished[/td][td=0]Atari 8-bit[/td][td=0]PC[/td][td=0]Flash[/td][td=0]Flash[/td][td=0]1.317[/td][td=0]2611[/td][td=0]22.0%[/td][td=0]0.8%[/td][td=0]1.3%[/td][td=0]75.9%[/td][td=0]Unfinished[/td][td=0]Atari ST[/td][td=0]PC[/td][td=0]Flash[/td][td=0]Flash[/td][td=0]1.577[/td][td=0]2302[/td][td=0]65.3%[/td][td=0]0.1%[/td][td=0]0.0%[/td][td=0]34.6%[/td][td=0]Unfinished[/td][td=0]Commodore 128[/td][td=0]PC[/td][td=0]Flash[/td][td=0]Flash[/td][td=0]1.017[/td][td=0]31[/td][td=0]29.0%[/td][td=0]0.0%[/td][td=0]0.0%[/td][td=0]71.0%[/td][td=0]Progressing[/td][td=0]Commodore 64[/td][td=0]PC[/td][td=0]Flash[/td][td=0]Flash[/td][td=0]1.552[/td][td=0]19211[/td][td=0]98.8%[/td][td=0]0.6%[/td][td=0]0.6%[/td][td=0]0.1%[/td][td=0]Unfinished[/td][td=0]Commodore Amiga[/td][td=0]PC[/td][td=0]Flash[/td][td=0]Flash[/td][td=0]1.833[/td][td=0]4731[/td][td=0]47.3%[/td][td=0]0.1%[/td][td=0]0.0%[/td][td=0]52.6%[/td][td=0]Unfinished[/td][td=0]Commodore PET_CBM[/td][td=0]PC[/td][td=0]Flash[/td][td=0]Flash[/td][td=0]1.008[/td][td=0]33[/td][td=0]66.7%[/td][td=0]0.0%[/td][td=0]0.0%[/td][td=0]33.3%[/td][td=0]Unfinished[/td][td=0]Commodore Plus 4[/td][td=0]PC[/td][td=0]Flash[/td][td=0]Lawrence[/td][td=0]1.084[/td][td=0]654[/td][td=0]100.0%[/td][td=0]0.0%[/td][td=0]0.0%[/td][td=0]0.0%[/td][td=0]Unfinished[/td][td=0]Commodore VIC-20[/td][td=0]PC[/td][td=0]Flash[/td][td=0]Flash[/td][td=0]1.094[/td][td=0]1057[/td][td=0]10.6%[/td][td=0]0.0%[/td][td=0]0.0%[/td][td=0]89.4%[/td][td=0]Unfinished[/td][td=0]Dragon 32-64[/td][td=0]PC[/td][td=0]Flash[/td][td=0]Flash[/td][td=0]1.05[/td][td=0]573[/td][td=0]16.9%[/td][td=0]1.0%[/td][td=0]0.0%[/td][td=0]82.0%[/td][td=0]Unfinished[/td][td=0]MGT Sam Coupe[/td][td=0]PC[/td][td=0]Flash[/td][td=0]Flash[/td][td=0]1.045[/td][td=0]828[/td][td=0]87.1%[/td][td=0]0.5%[/td][td=0]0.0%[/td][td=0]12.4%[/td][td=0]Unfinished[/td][td=0]Microsoft MSX[/td][td=0]PC[/td][td=0]Flash[/td][td=0]Flash[/td][td=0]1.318[/td][td=0]2079[/td][td=0]32.5%[/td][td=0]4.4%[/td][td=0]2.2%[/td][td=0]60.9%[/td][td=0]Unfinished[/td][td=0]Microsoft MSX 2[/td][td=0]PC[/td][td=0]Flash[/td][td=0]Flash[/td][td=0]1.29[/td][td=0]1543[/td][td=0]47.7%[/td][td=0]0.0%[/td][td=0]0.1%[/td][td=0]52.2%[/td][td=0]Unfinished[/td][td=0]Tangerine Oric[/td][td=0]PC[/td][td=0]Flash[/td][td=0]Flash[/td][td=0]1.042[/td][td=0]932[/td][td=0]100.0%[/td][td=0]0.0%[/td][td=0]0.0%[/td][td=0]0.0%[/td][td=0]Unfinished[/td][td=0]Memotech MTX[/td][td=0]PC[/td][td=0]Flash[/td][td=0]Flash[/td][td=0]1.012[/td][td=0]51[/td][td=0]100.0%[/td][td=0]0.0%[/td][td=0]0.0%[/td][td=0]0.0%[/td][td=0]Finished[/td][td=0]ScummVM[/td][td=0]PC[/td][td=0]Flash[/td][td=0]Flash[/td][td=0]1.07[/td][td=0]88[/td][td=0]0.0%[/td][td=0]100.0%[/td][td=0]0.0%[/td][td=0]0.0%[/td][td=0]Progressing[/td][td=0]Sinclair ZX-81[/td][td=0]PC[/td][td=0]Flash[/td][td=0]Flash[/td][td=0]1.057[/td][td=0]1103[/td][td=0]98.7%[/td][td=0]1.3%[/td][td=0]0.0%[/td][td=0]0.0%[/td][td=0]Unfinished[/td][td=0]Sinclair ZX Spectrum[/td][td=0]PC[/td][td=0]Flash[/td][td=0]Flash[/td][td=0]1.37[/td][td=0]5526[/td][td=0]99.2%[/td][td=0]0.7%[/td][td=0]0.1%[/td][td=0]0.0%[/td][td=0]Unfinished[/td][td=0]Tandy TRS-80 CoCo[/td][td=0]PC[/td][td=0]Flash[/td][td=0]Flash[/td][td=0]1.046[/td][td=0]158[/td][td=0]58.2%[/td][td=0]0.0%[/td][td=0]0.0%[/td][td=0]41.8%[/td][td=0]Unfinished[/td][td=0]Tandy TRS-80[/td][td=0]PC[/td][td=0]Flash[/td][td=0]Flash[/td][td=0]1.022[/td][td=0]72[/td][td=0]98.6%[/td][td=0]1.4%[/td][td=0]0.0%[/td][td=0]0.0%[/td][td=0]Unfinished[/td][td=0]TI-99_4A[/td][td=0]PC[/td][td=0]Flash[/td][td=0]Flash[/td][td=0]1.023[/td][td=0]56[/td][td=0]89.3%[/td][td=0]10.7%[/td][td=0]0.0%[/td][td=0]0.0%[/td][td=0]Progressing[/td][td=0]Future Pinball[/td][td=0]Pinball[/td][td=0]Flash[/td][td=0]Flash[/td][td=0]1.069[/td][td=0]344[/td][td=0]100.0%[/td][td=0]0.0%[/td][td=0]0.0%[/td][td=0]0.0%[/td][td=0]Unfinished[/td][td=0]Visual Pinball[/td][td=0]Pinball[/td][td=0]Flash[/td][td=0]Lawrence[/td][td=0]1.128[/td][td=0]2125[/td][td=0]100.0%[/td][td=0]0.0%[/td][td=0]0.0%[/td][td=0]0.0%[/td][td=0]Unfinished[/td][td=0]Visual PinMAME[/td][td=0]Pinball[/td][td=0]Flash[/td][td=0]Flash[/td][td=0]1.002[/td][td=0]0[/td][td=0]NaN%[/td][td=0]NaN%[/td][td=0]NaN%[/td][td=0]NaN%[/td][td=0]Finished[/td][td=0]Sega Naomi[/td][td=0]Arcade[/td][td=0]Flash[/td][td=0]Flash[/td][td=0]1.071[/td][td=0]92[/td][td=0]0.0%[/td][td=0]100.0%[/td][td=0]0.0%[/td][td=0]0.0%[/td]

Status	Name	System	Creator	Author	Version	NumGames	Unfinished	Finished	Progressing	New
Progressing	Atari 2600	Console	Flash	Flash	1.087	5387	99.7%	0.3%	0.0%	0.0%
Finished	Atari 5200	Console	Flash	Flash	1.09	141	0.0%	100.0%	0.0%	0.0%
Finished	Atari 7800	Console	Flash	Flash	1.065	82	0.0%	100.0%	0.0%	0.0%
Finished	Atari Jaguar	Console	Flash	Flash	1.107	91	0.0%	100.0%	0.0%	0.0%
Finished	Coleco Vision	Console	Flash	Flash	1.149	276	0.0%	100.0%	0.0%	0.0%
Progressing	Commodore Amiga CD32	Console	Flash	Flash	1.113	148	70.9%	5.4%	23.6%	0.0%
Finished	Fairchild Channel F	Console	Flash	Flash	1.022	44	0.0%	100.0%	0.0%	0.0%
Progressing	GCE Vectrex	Console	Flash	Flash	1.023	231	85.7%	11.7%	2.6%	0.0%
Finished	Mattel Intellivision	Console	Flash	Flash	1.084	220	0.0%	100.0%	0.0%	0.0%
Unfinished	Microsoft Xbox	Console	Flash	Flash	1.381	888	100.0%	0.0%	0.0%	0.0%
Progressing	NEC PC-FX	Console	Flash	Flash	1.028	22	0.0%	100.0%	0.0%	0.0%
Finished	NEC SuperGrafx	Console	Flash	Flash	1.021	5	0.0%	100.0%	0.0%	0.0%
Unfinished	NEC TurboGrafx CD	Console	Flash	Flash	1.075	810	11.7%	0.1%	0.0%	88.1%
Progressing	NEC TurboGrafx-16	Console	Flash	Flash	1.069	365	98.1%	1.9%	0.0%	0.0%
Progressing	Nintendo GameCube	Console	Flash	Flash	1.266	523	94.8%	5.2%	0.0%	0.0%
Finished	Nintendo N64	Console	Flash	Flash	1.39	496	0.0%	100.0%	0.0%	0.0%
Unfinished	Nintendo NES	Console	Flash	Flash	1.421	5997	15.8%	0.1%	0.4%	83.8%
Progressing	Nintendo SNES	Console	Flash	Flash	1.316	2383	93.2%	6.6%	0.1%	0.1%
Unfinished	Panasonic 3DO	Console	Flash	Flash	1.096	205	53.2%	1.5%	0.5%	44.9%
Unfinished	Philips CD-i	Console	Flash	Flash	1.049	137	35.8%	0.7%	0.7%	62.8%
Unfinished	Philips Odyssey	Console	Flash	Flash	1.003	6	100.0%	0.0%	0.0%	0.0%
Unfinished	Philips Odyssey 2	Console	Flash	Flash	1.019	160	31.3%	0.6%	2.5%	65.6%
Progressing	Sega 32X	Console	Flash	Flash	1.077	71	0.0%	49.3%	50.7%	0.0%
Unfinished	Sega CD	Console	Flash	Flash	1.117	145	93.8%	2.1%	4.1%	0.0%
Finished	Sega Dreamcast	Console	Flash	Flash	1.31	400	0.0%	98.3%	0.0%	1.8%
Unfinished	Sega Genesis	Console	Flash	Flash	1.434	1837	38.6%	0.3%	0.7%	60.4%
Unfinished	Sega Master System	Console	Flash	Flash	1.194	690	42.3%	0.1%	0.0%	57.5%
Finished	Sega Saturn	Console	Flash	Flash	1.573	663	0.0%	99.5%	0.5%	0.0%
Unfinished	Sega SG-1000	Console	Flash	Flash	1.058	151	100.0%	0.0%	0.0%	0.0%
Finished	SNK Neo-Geo CD	Console	Flash	Flash	1.024	86	0.0%	100.0%	0.0%	0.0%
Finished	SNK Neo-Geo	Console	Flash	Flash	1.039	152	0.0%	75.7%	24.3%	0.0%
Progressing	Sony Playstation	Console	Flash	Flash	1.461	1402	99.4%	0.6%	0.0%	0.0%
Unfinished	Sony PlayStation 2	Console	Lawrence	Flash	1.354	5338	0.0%	0.1%	0.0%	99.9%
Finished	Atari Lynx	Handheld	Flash	Flash	1.06	91	0.0%	100.0%	0.0%	0.0%
Progressing	Bandai WonderSwan Color	Handheld	Flash	Flash	1.021	92	0.0%	95.7%	4.3%	0.0%
Unfinished	Bandai WonderSwan	Handheld	Flash	Flash	1.019	27	96.3%	3.7%	0.0%	0.0%
Unfinished	Nintendo DS	Handheld	Lawrence	Flash	1.109	3024	0.0%	0.0%	0.1%	99.9%
Unfinished	Nintendo Game Boy Advance	Handheld	Flash	Flash	1.317	11150	6.2%	0.0%	0.0%	93.7%
Unfinished	Nintendo Game Boy Color	Handheld	Flash	Flash	1.209	3635	8.0%	0.1%	0.0%	92.0%
Unfinished	Nintendo Game Boy	Handheld	Flash	Flash	1.327	2508	13.7%	1.2%	0.1%	85.1%
Progressing	Nintendo Virtual Boy	Handheld	Flash	Flash	1.02	43	0.0%	55.8%	44.2%	0.0%
Progressing	Sega Game Gear	Handheld	Flash	Flash	1.215	428	0.0%	7.7%	92.3%	0.0%
Unfinished	SNK Neo-Geo Pocket Color	Handheld	Lawrence	Flash	1.041	335	0.0%	0.0%	0.0%	100.0%
Progressing	SNK Neo-Geo Pocket	Handheld	Lawrence	Flash	1.019	9	0.0%	0.0%	100.0%	0.0%
Unfinished	Sony PSP	Handheld	Flash	Flash	1.223	939	35.8%	0.1%	0.0%	64.1%
Finished	Watara Supervision	Handheld	Flash	Flash	1.013

Totals: 109452 Games, 51.3% Unfinished, 3.4% Finished, 0.7% Progressing, 44.6% New

Attached Files

•  GameExDBM.png   44.36K   136 downloads

[chriss]

great idea, this tool might be quite handy for me... for example the c64 roms need a while to show up, with an mdb file it would take no time (if i get this right ... )

i don't know if this will be possible but since you had a similar option in whdloader i thought i might ask:

i have c64 collection as goodgb64. the naming is [gamename] (number).zip but the screenshots from gamebase64 are just like [gamename].zip
so will it be possible to rename the screenshots so they display with the roms?

[PimpDaddyStu]

looking good Headkaze, I don't mind helping out again and don't worry I don't claim ownership on anything I post here, it's a public forum and anything I do on it is for the benefit and use of anyone

Stu

[headkaze]

PimpDaddyStu, on Jun 13 2008, 04:31 AM, said:

looking good Headkaze, I don't mind helping out again and don't worry I don't claim ownership on anything I post here, it's a public forum and anything I do on it is for the benefit and use of anyone

Stu

I know that mate, but recent events have made me have to re-think the way group projects are maintained here. There is probably nothing to worry about anymore, but it's just to be sure. Once bitten twice shy if you know what I mean.

Anyway the scrape has finished. The Microsoft Windows one took a bit longer than I expected. Another thing I noticed is the ESRB ratings didn't get scraped because it shows the rating system for Australia for me. So maybe if I could get someone from the US to scrape the Windows database again? It didn't take that long to scrape really. We might not even be using the ESRB ratings anyway so I'll need to find out from Tom what data he's going to be using from the databases before I worry too much about it.

I'll PM you with details on how to download the project when I need your help. Thanks Stu

[starbuck]

A while back I asked about this being incorporated into GameEx. Thought would be nice if on the fly you could update the datebase from the GameEx screen and then have it sent to the same palce as GameEx live goes. That way each time there was a update thru GameEx we all would get updated databases. With all the GameEx communty comtributing to updating and maintanining the database, we would have the most up to date dbs for any frontend.

Looks great

[bkenobi]

If it's giving you Australia ratings, you could go through a US based proxy server or access Moby's US page directly (if possible).

[headkaze]

bkenobi, on Jun 14 2008, 12:51 AM, said:

If it's giving you Australia ratings, you could go through a US based proxy server or access Moby's US page directly (if possible).

I'm guessing you got my PM about this. So I take your lack of reply as you not being interested in this project?

Anyone else from the US that can help with the scraping?

[bkenobi]

PM replied to.

[shaunopp]

headkaze, on Jun 14 2008, 09:16 AM, said:

I'm guessing you got my PM about this. So I take your lack of reply as you not being interested in this project?

Anyone else from the US that can help with the scraping?

I'm happy to help with a little coaching. Not a big DB guru but I'm sure I could get it with some guidance.

A description of the big picture for this may help people jump onboard. So from what I understand, your app scrapes the Moby site for data and creates a huge game database. That DB can be referred to by GameEx for descriptive info such as genre, yr published etc.. So whenever someone adds an emulator it's associated games will have way more descriptive information than they currently do?

I think maybe many people didn't respond because they (like me) aren't as familiar with the whole concept and the benefits. Cmon headkaze, sell us on it!

[Tempest]

I'll help too...I would also require coaching!

[headkaze]

Well to be honest I was sort of hoping some of the people who helped create the old databases would be willing to help out with the new ones. That way I wouldn't need to explain what it's all about.

Well now I think about it it would be of benefit if we get some new people involved in this project. The difference between how the data was collected then and now is that the last time MobyGames.com was scraped it was using an AutoIt script that would require you to open Microsoft Access and Internet Explorer so it could read the data off the Window and fill in a database. While it did this it would lock up the entire computer and once the information was collected it had to run through PimpDaddyStu's Fuzzy String Matcher program to get the GoodName and then I had to write a program to scrape AllGame.com to get the single categories required by GameEx. As you can see it was quite a messy process to create a database for GameEx.

So this is where my application comes in, it scrapes Moby, it does the fuzzy string matching, it outputs to the Access Database (mdb) format GameEx uses, it doesn't take up much CPU or bandwidth, it can be run at any time to update databases and it's all done from a GUI. In other words my program does the job that it took 3 others to do. But even now, some names won't get matched so there is some need for user intervention. My software helps with that also.

So now I've explained what it is, I need volenteer's to fill in missing names. I will explain how to do that in the PM to the people who volenteer. Just remember it's a very easy thing to do, all it is is pressing buttons and filling in data using a bit of common sense. Most of the names are matched using the fuzzy string matcher, so it's only the ones it misses that need filling in manually.

The actual amount of work involved depends on the database, so what I'll do is go through and sort out which ones will take the longest to fill in and which ones will take the least amount of time, and people can decide which databases they'd like to work on. Some databases have got a 100% name match while some have 10%. It all depends on how close the Moby name and the dat file names match up.

I would prefer people who volenteer to understand what GoodTools, No-Intro and TOSEC is and what they mean. It would help if people understood how databases work in GameEx, what a Microsoft Access database is and how to view and edit them. So I do need people who actually understand what most of this stuff is already. If I'm desperate for volenteers I'll probably settle for anyone who can press a button lol

Right now though there are some things I'm working on with Tom to make the program function better and also I want to make sure he's happy with the output it will produce before I upload it for people here. I'll make another post when I'm ready to do that. In the mean time people are still welcome to post here suggestions and ideas for the program before it gets released. I'm sure people will come up with additional ideas once I've sent it out to people, so it will probably be released in new versions as that happens.

[Brian Hoffman]

Send me the info and what you need done still.
Im sure I can dig up some time..... somewhere

[headkaze]

Thanks PimpDaddyStu, bkenobi, shaunopp, Tempest and Brian. I'll be PMing you all with instructions when I've got everything ready.

I'm thinking once you figure out how easy it is to scrape and update a database, I'm sure you won't mind keeping them up-to-date too. Anyway I'll give you intructions on how in a PM soon.

Stay tuned and thanks again for all the responses!

[ReFleX]

I would like to chime in on this...

It would be nice to have a few databases combined in addition to their separated brethren. For example, since GoodTools has 32x and Genesis combined, a combined database would be helpful. Gameboy / Gameboy Color and NeoGeo Pocket / Pocket Color are also the same way.

This wouldn't take any extra work, just include an extra database with these combined so they match the goodtools.

[headkaze]

This project is still happening. I've re-scraped all the databases to fix some of the issues with the previous scrape. I've managed to scrape the ESRB data too.

Right now were just waiting on Tom to answer some of my questions related to the formatting of the databases. I will be PMing the volunteer's as soon as the software is ready.

I have uploaded a table that shows the total number of games in each database and the percentage of GoodTool/NoIntro/TOSEC fuzzy name matches. It should give us an indication of how much work will be needed to fill in the remaining names for each system.

Moby Name to GoodTool/NoInto/TOSEC Fuzzy Match

[greatflash]

headkaze, on Jun 22 2008, 11:21 AM, said:

This project is still happening. I've re-scraped all the databases to fix some of the issues with the previous scrape. I've managed to scrape the ESRB data too.

Right now were just waiting on Tom to answer some of my questions related to the formatting of the databases. I will be PMing the volunteer's as soon as the software is ready.

I have uploaded a table that shows the total number of games in each database and the percentage of GoodTool/NoIntro/TOSEC fuzzy name matches. It should give us an indication of how much work will be needed to fill in the remaining names for each system.

Moby Name to GoodTool/NoInto/TOSEC Fuzzy Match

Nice work mate,

Looks like the Saturn DB will need a bit of work (have all U, and all Pal that is not in the U set, and 60% jap)
and what about my poor little zx81 (poor thing)

Did you get anywhere regard alternative names? (did you see my reply re: that in my saturn db problem post?)

All the best you work-aholic.. Hic!

[headkaze]

greatflash, on Jun 23 2008, 05:07 AM, said:

Nice work mate,

Looks like the Saturn DB will need a bit of work (have all U, and all Pal that is not in the U set, and 60% jap)
and what about my poor little zx81 (poor thing)

Did you get anywhere regard alternative names? (did you see my reply re: that in my saturn db problem post?)

All the best you work-aholic.. Hic!

It's not really going to matter what set's people have because I'm using the dat files which have all games listed already. The databases I am concerned about though are the ones that don't have a GoodTools/NoIntro or TOSEC dat file at all. So we may need people with big sets to make dat's for them.

I'm not sure I understood your idea regarding alternate names. I'm still waiting for Tom to get back to me on how he wants to implement it (which is more important than how I do things in my program). I've proposed that we have 3 columns for each naming convention GoodName/NoIntro/TOSEC where you could set the naming system for each emulator. His suggestion was to separate the databases so that each naming convention had it's own database. My problem with the latter is managing all the different sets of databases. I personally liked the idea of using a column for each one but it's up to Tom how he implements it.

Perhaps you could try explaining it to me again? Bare in mind it's more important to sell the idea to Tom than me

[greatflash]

headkaze, on Jun 23 2008, 04:21 PM, said:

It's not really going to matter what set's people have because I'm using the dat files which have all games listed already. The databases I am concerned about though are the ones that don't have a GoodTools/NoIntro or TOSEC dat file at all. So we may need people with big sets to make dat's for them.

I'm not sure I understood your idea regarding alternate names. I'm still waiting for Tom to get back to me on how he wants to implement it (which is more important than how I do things in my program). I've proposed that we have 3 columns for each naming convention GoodName/NoIntro/TOSEC where you could set the naming system for each emulator. His suggestion was to separate the databases so that each naming convention had it's own database. My problem with the latter is managing all the different sets of databases. I personally liked the idea of using a column for each one but it's up to Tom how he implements it.

Perhaps you could try explaining it to me again? Bare in mind it's more important to sell the idea to Tom than me

I do prefer the idea of keeping the 3 naming conventions within the one database. Not every dat contains every file. So, some games are missing from one dat and present in another. by having 3 contained in one db then people are still able to mix and match. also, not everyone has a nice clean set.
The naming problems also happen with ISO images. We have tosec ISO, but by looking at the file for the saturn, it expects you to have files as cue, iso, and mp3/wave.. who has them like that??? Mine are all .cdi (selfboots) but named to follow the conventions set by other dats.

With regards my idea. It was to include a relational reference to the naming conventions (so you could have more than 3). It is hard to explain.. The idea was to have a (*) header file that contains a common name with the other data that the rom requires within one record (name, publisher, year, etc), adding to the end of the record a flag that cross references other records that contain just the name (tosec, good, no-intro, generic, etc). The master record (*) would then relate within your editor to the subsequent naming records.
When you output the db, you could then convert it from a relational to a list database by creating records of the various names and appending the data from the master record (*) to each.
All i was trying to suggest is that the 3 naming conventions are not enough. Take a system that has no dat's, and a game that has "the xxxx - yyyy of zzzz" this could be written in so many ways. I was just looking for a way to add alternatives to the db because if people have collected lots of no-dat roms then there is a good chance they will have named them to follow their own style and this would have been a way to add a few guesses to the db list.
This suggestion was mainly made in case Tom decided he would like to keep the original format for db's. Using this format would allow you to do as you suggested with the multi-names and to continue to use one dat. Using the flag to link the matching records would also allow you to input a list db and return it to a relational db and vise-versa. All you need is to only allow data editing of the master record (*) and name editing of the naming records, updating the maser edits to the naming records on output.
God, I know i go on.

I hope you can follow this?

Included a little piccy (hope that helps)

All the best

Attached Files

•  My_little_Piccy.jpg   138.19K   64 downloads

[PimpDaddyStu]

3 MDB's for each system would be a complete mess to look after but I know I suggested extra columns the last time we were working on the MDB's and he wasn't keen on doing anything with the current system then either, heh what can you do, I guess we just have to make the most of what we're given.

Stu

[bkenobi]

Just a thought, but if you *must* use 3 databases for each, I hope 2 (or maybe all 3) would be linked to a master database where the actual data resides. In this case, the extra databases wouldn't need updating, they would just be there to make things easier for Tom. I personally think it's pretty silly to do it that way unless it's a major undertaking to get the correct solution working, but hey, not my call.

I also had another thought... I don't know how much data there is contained in all the databases (I haven't looked recently), but perhaps this info could just be pulled from a server so it could be maintained at a central location and a local cached copy used if no update was desired/needed. Obviously it would be slower that way, but if data is small, then it wouldn't be too bad. Also, it would only need updating if a flag showed your version wasn't up to date. And, it would only need to update out of date info. I haven't messed with this stuff, so I don't know how hard it is putting hash tables together vs just eating the bandwidth of the download. Anyway, just a thought.