You must be logged in to post messages.
Please login or register

The Town Square
Moderated by Jayhawk

Hop to:    
Welcome! You are not logged in. Please Login or Register.50 replies, Sticky
Caesar IV Heaven » Forums » The Town Square » A call to make a modding Toolkit for the CB series.
Bottom
Topic Subject:A call to make a modding Toolkit for the CB series.
« Previous Page  1 2  Next Page »
user3
Pleb
posted 10-16-17 03:45 ET (US)         
City Builder executables points to a "phrase" in the ENG file(for english installs). I'm assuming other language installs use different file types(?SPA,FRE,GER...).

The ENG file structure is fairly simple. Using a hex editor; my wife prefered UltraEdit and contributed funds, but I am a cheapskate and use HxD(stable since 2009 and simple enough for me), you can view the simple structure.

At the top of the file(it's header), is the offset value for each "phrase" delimited by the hex value of '00', on either side, as opposed to a 'space' value of 20 that 'Notepad' displays.

Since 2003(Empress mod), people have been editing this file, for Emperor, using a hex editor. I am sure others have done simular ENG editing for the earlier games.
What is needed is a programmed text editor that has some 'Notepad' fuctionality, find and replace but with restrictions so as not replace every instance without user input or nag popup dialogs. The displayed phrases should be stacked(verticle list), rather then word-wrapped. The 'save' function obviously needs to conform to the systems expectation.

Why bother, you ask. With an editor in a modding toolkit, modders can change the "phrases" length, something that cant be done with a hex editor. Modders could even shorten the exteranious phrases never displayed in normal game play. I dont much care if it messes up the 'programmers dash' as they never released it to the public and I doubt Activision maintains a copy that could be released.

Notepad++ looks like a good candidate as it is open sourced. BEOS & Linux enthusists inform me 'Emacs' could do the job with its extensive library, but none have made a custom editor, so it might be over-complicated for our task.

With this one starter tool, we could continue onto Help file corrections, correcting the message concoctions, etc. Even get enough interest that people might start editing the Monuments and Temples, again. Dream a little think how your skillset can help the HG Modding Toolkit.

So, am I beating a horse skeleton here or what?
Pardon any spelling mistakes, new windows install still in progress.
AuthorReplies:
Pecunia
CBH Seraph
posted 10-17-17 02:16 ET (US)     1 / 50       
I'm quite intimate with both .eng file formats. The easiest thing to do for me would be to create a program to convert those files to easily edited XML and back to .eng again. How does that sound?

Angel Pecunia | HG Downloads programmer and techie
Caesar IV Heaven | Children of the Nile Heaven | Citadel of Poseidon (Zeus/Poseidon) | The Great Wall (Emperor)
user3
Pleb
posted 10-17-17 03:40 ET (US)     2 / 50       
Wonderful Pecunia, Armel will love the editing help. Perhaps work with him to polish the final product, yes.
Pecunia
CBH Seraph
posted 10-21-17 18:18 ET (US)     3 / 50       
Small update on this: I have implemented a first version which can read/write the *_Text.eng files, but not yet the *_MM.eng files. You can download the pre-release version (plus source code) here. Let me know any comments/suggestions

Angel Pecunia | HG Downloads programmer and techie
Caesar IV Heaven | Children of the Nile Heaven | Citadel of Poseidon (Zeus/Poseidon) | The Great Wall (Emperor)
user3
Pleb
posted 10-21-17 20:59 ET (US)     4 / 50       
Awesome software!
I just converted Emperor to C3(inspired to install it)non-MM.ENGs; Perfection in readability with the grouping classes.
I have no doubt that it will reverse the process just as easily.
I will do a little test editing tomorrow, just to be thorough.

Thanks Pecunia!
-user3

update:
XML to ENG Converter worked perfectly for C3, Cleo, Posd & Emperor. Shortened and lengthened phrases. Have not messed with the mm files as your still working on that.


Many Thanks, Pecunia!
-user3

[This message has been edited by user3 (edited 10-22-2017 @ 02:55 AM).]

Pecunia
CBH Seraph
posted 10-22-17 05:14 ET (US)     5 / 50       
Thanks for testing!

MM files are more complex, with lots more info beside the text strings, such as: X/Y coordinates for titles, subtitles and videos, additional pictures, dialog position and size, message types...
I still need to do some more investigation where each field is used in the games. I know how C3 uses them, but haven't looked into the other games in depth yet.

To be continued!

Angel Pecunia | HG Downloads programmer and techie
Caesar IV Heaven | Children of the Nile Heaven | Citadel of Poseidon (Zeus/Poseidon) | The Great Wall (Emperor)
PCDania
Pleb
(id: PCD)
posted 10-22-17 11:50 ET (US)     6 / 50       
Wow, I could really have used that program back in the day when I tried translating the game into Danish. Not sure xml was released back the though.

"Cats are the only animals that are both uber, pwns and 1337 at the same time." -King Euric
By reading this fine print your soul is now the exclusive property of HeavenGames.
Pecunia
CBH Seraph
posted 10-22-17 16:06 ET (US)     7 / 50       
user3: I have just released version 0.2 with support for MM files, but only from ENG to XML, not the other way yet. Could you check it out and see if this XML format is something we can work with?

PCDania: XML was around back then, but not as popular as today with good support in editors. How far did you get with the translation?

Angel Pecunia | HG Downloads programmer and techie
Caesar IV Heaven | Children of the Nile Heaven | Citadel of Poseidon (Zeus/Poseidon) | The Great Wall (Emperor)
user3
Pleb
posted 10-22-17 18:04 ET (US)     8 / 50       
Looking good, but perhaps convert the @? formatting to html5 equivalent. Most of the formatting is understandable in its present form for me. I loaded an Emperor save, viewing the xml in notepad++, viewing the hex in HxD to get a quick understanding of the '@' format meanings.

Idea: perhaps make it browser displayable so the person can see their work before converting back to eng, but it is not essential to do this.(shrug) It is usable as is, and your time is precious.

You are astonishing, Pecunia!
-user3

PS: Let see if I got these things correct
Window = dialog popup relative to mouse position or absolute position of city view, system corrects for out-of-bounds numbers. Not sure, as I'm testing with a widescreen version which may mess-up offsets.
Image1: bmp pointer @G### ???--not sure how this works as a tagline--???
title: self explainitory -gets Bold+centered formatting-
subtitle: seems broken/not displayed ingame(shrug)
Contents: what gets displayed in the message id
@### = message id to link/jump to (space+puncuation delimits end, so (& amp) have to be used within link)
@L = new line(cr+lf)
@H = unsure, reset formatting or font weight
@B = bold
@P = lf+cr+tab/indent
Hmm, too difficult to get the html to work without a strict style sheet, I think. Perhaps a 'formattinghelp.txt(cheat sheet)' at a later date.

[This message has been edited by user3 (edited 10-22-2017 @ 08:07 PM).]

Pecunia
CBH Seraph
posted 10-23-17 01:51 ET (US)     9 / 50       
Thanks again for testing!

Window: yes. The x/y is relative to some bounding box on the screen that the system decides. For C3/Pharaoh, this is usually a 640x480 rectangle in the center of the screen. The width and height params are in multiples of 16, so if you see "20" for height, that means 20*16 = 320 pixels. Would it be more logical to rename this to "dialog" instead of "window"?

Image1: the graphic is a game dependent index into one of the graphics files. In C3, it's used for extra images on the manual entries about history. I'm not sure if they're used in the other games. There's also a second image (image2), but I need to check if it's used anywhere. If not, I can just rename 'image1' to 'image'. If you see 'image1' tags with an id of 0, ignore those: I'm going to leave them out in the next version.

Subtitle: it depends on the type of message. In C3, the mission introduction texts are also stored in the MM file, and the subtitle is displayed on the mission briefing screen.

The @-formatting: this is exactly how the strings are stored in the text files, and also the format used for custom adventure text in Zeus/Poseidon. And indeed, converting from/to a html-like format will require checks that are not really worth the effort.

You're right about the meaning of different types of @-tags. I'm also not sure about the @H: it's the first time I've seen that one. I think it's only used in Emperor.

At a later date there will be an extensive explanation under the 'Help' button that's right now not doing anything

Angel Pecunia | HG Downloads programmer and techie
Caesar IV Heaven | Children of the Nile Heaven | Citadel of Poseidon (Zeus/Poseidon) | The Great Wall (Emperor)

[This message has been edited by Pecunia (edited 10-23-2017 @ 01:52 AM).]

PCDania
Pleb
(id: PCD)
posted 10-23-17 02:08 ET (US)     10 / 50       
@Pecunia

Can't remember exactly but at least 90%, most difficult was the MM file. Remainder could not be translated because it would break the game because of those off set values for each "phrase". I first tried using Notepad but found out not all the spaces were spaces. Then I found a free hex editor which made it much easier. The problem with not being able to translate everything was that some words are longer in Danish so the marker showing the end (as I saw it) got moved thereby breaking the game. Mostly text went out of the side of the screen or got very big, a few times buildings and resources changed names.

What the original program Impressions Games (IG) used goes, then as far as I understand from a comment from a former IG programmer, that program was lost some time in the early 2000s (maybe when IG closed in 2004?) so Activision has never had that program.

I hope I can get a copy of the final program so I can translate the game again as I have lost the original files. They were on a 3.5" floppy and haven't had such a drive for ages so they got thrown out the last time I moved.

"Cats are the only animals that are both uber, pwns and 1337 at the same time." -King Euric
By reading this fine print your soul is now the exclusive property of HeavenGames.

[This message has been edited by PCDania (edited 10-23-2017 @ 02:55 AM).]

user3
Pleb
posted 10-23-17 04:21 ET (US)     11 / 50       
I'm also not sure about the @H: it's the first time I've seen that one. I think it's only used in Emperor.
Correct about @H only used in Emperor, checked C3 and Posd.

After sleeping I figured it out, for Emperor @H is the medium yellow font color(also used for title tag, bold, maybe subtitle), @### Links are medium green font color . The font colors differ for each game;(shrug) Probably hardcoded in an exe resource.
Would it be more logical to rename this to "dialog" instead of "window"?
Dialog, popup box, panel, placard all seem better descripters. Use whatever makes most sense.

Thanks Pecunia for your valued time on this project.
-user3

[This message has been edited by user3 (edited 10-23-2017 @ 04:45 AM).]

Pecunia
CBH Seraph
posted 10-23-17 15:29 ET (US)     12 / 50       
@PCDania: wow, that must have been a lot of trial and error with your translation back in the days, with only a hex editor!
The program is open source, will stay on Github and you'll be able to download the final program from my Github project as well.

@user3: @B is also Emperor-only, indicating Big title-like text, and @H might be short for heading. The colors for bold/links/etc are game-dependent indeed. There's a graphics file called '*Fonts.sg2/.sg3' for every game, which contains images of all letters that make up text in the game.
Next project: put my SG-reader program on Github, and maybe create an SG-writer as well?

Angel Pecunia | HG Downloads programmer and techie
Caesar IV Heaven | Children of the Nile Heaven | Citadel of Poseidon (Zeus/Poseidon) | The Great Wall (Emperor)
user3
Pleb
posted 10-23-17 18:21 ET (US)     13 / 50       
Next project: put my SG-reader program on Github, and maybe create an SG-writer as well?
!!!Stupendous!!!
I'm no good with graphics and I'm sure you will have no lack of testers; but if you need a noobish point of view, just ask.

My heart is thumping in excitement
-user3
Pecunia
CBH Seraph
posted 10-24-17 14:33 ET (US)     14 / 50       
Aaaaand, new version again: v0.3.
New stuff: final format for the message XML file, convert message XML files back to ENG, some polishing: added window title, app icon. Regarding the app icon: I know my drawing skills are not as good as my programming skills... if someone wants to provide an alternative app icon, please do

Angel Pecunia | HG Downloads programmer and techie
Caesar IV Heaven | Children of the Nile Heaven | Citadel of Poseidon (Zeus/Poseidon) | The Great Wall (Emperor)
user3
Pleb
posted 10-24-17 16:25 ET (US)     15 / 50       
I have the unfortunate task of reporting this error

ERROR: Invalid XML: end element </content> not found

I converted from EmperorMM.eng to xml then back to eng without even loading the xml for editing. No eng file created.

I skimmed the xml and could not find an error in tagging

[This message has been edited by user3 (edited 10-24-2017 @ 04:49 PM).]

Pecunia
CBH Seraph
posted 10-25-17 01:07 ET (US)     16 / 50       
Aha, found it. I tested with C3 and Zeus and those work fine, but Emperor does something different with links to other messages: for every other game, it's "@123link&text&here", but for Emperor, it's "&123link text here(0e)", where the (0e) is a special character that's not allowed in XML and looks like a space...
I'll create a fix when I get home from work. Thanks for reporting

Angel Pecunia | HG Downloads programmer and techie
Caesar IV Heaven | Children of the Nile Heaven | Citadel of Poseidon (Zeus/Poseidon) | The Great Wall (Emperor)
PCDania
Pleb
(id: PCD)
posted 10-25-17 02:04 ET (US)     17 / 50       
How does Pharaoh behave?

"Cats are the only animals that are both uber, pwns and 1337 at the same time." -King Euric
By reading this fine print your soul is now the exclusive property of HeavenGames.
Pecunia
CBH Seraph
posted 10-25-17 02:10 ET (US)     18 / 50       
Pharaoh should be good in 0.3
PCDania
Pleb
(id: PCD)
posted 10-25-17 09:02 ET (US)     19 / 50       
It seems Pharaoh_Text.eng is ok when converted to .xml and back to .eng.
Unfortunately Pharaoh_MM.eng is different from the original when converted to .xml and back to .eng. I seems to remember Impressions Games made a few changes to the game engine when they made Pharaoh.

Maybe the file Pharaoh_MM.txt in the Pharaoh main folder is of help.

If you don't have the files I can provide you with copies. I have the original fully patched game (Pharaoh + Cleopatra incl patches).

"Cats are the only animals that are both uber, pwns and 1337 at the same time." -King Euric
By reading this fine print your soul is now the exclusive property of HeavenGames.

[This message has been edited by PCDania (edited 10-25-2017 @ 09:03 AM).]

Pecunia
CBH Seraph
posted 10-25-17 10:27 ET (US)     20 / 50       
Fixed the Emperor bug: v0.3.1.

@PCDania: yes, there are differences, but they are in unused fields only. For example, the ENG file can have an X and Y coordinate for the title, subtitle and video strings, even when the string itself is not defined. I ignore those X/Y coordinates in the XML output and the result is that they are not in the resulting ENG file either. Furthermore, there are "empty" message entries which have no text at all but are still counted as non-empty in the original ENG file. And as last one: there's a spurious "@18" string in the Pharaoh ENG which does not correspond to any known text field...

I've analyzed all game .exe files and they do not use the fields that change in the ENG -> XML -> ENG conversion. Therefore, I chose to ignore any "rubbish" that the files accumulated over the years, to make editing the texts as simple as possible

The change between C3 and Pharaoh: the number of message entries in the MM file went from 400 to 1000. Other than that there are no changes to the language files.

And I have all games, both as old CD-ROM versions, as well as the versions from GOG, I bought Emperor last weekend to complete the set

Angel Pecunia | HG Downloads programmer and techie
Caesar IV Heaven | Children of the Nile Heaven | Citadel of Poseidon (Zeus/Poseidon) | The Great Wall (Emperor)
PCDania
Pleb
(id: PCD)
posted 10-25-17 13:00 ET (US)     21 / 50       
Unfortunately it looks like I lost the CDs for Caesar 3 and Pharaoh when I moved. I still have the games installed on my computer in a faint hope they will show up eventually. Hurray for gog.com where I have bought them long time ago so I had a backup I only need Caesar, Caesar II, Caesar IV and Emperor at gog.com.

"Cats are the only animals that are both uber, pwns and 1337 at the same time." -King Euric
By reading this fine print your soul is now the exclusive property of HeavenGames.
user3
Pleb
posted 10-25-17 13:54 ET (US)     22 / 50       
I have converted all eng files for C3 to xml then back to eng; V0.3.1 . I will do the original files backup and play C3 to ensure no C3 program hick-ups. Hopefully, today or tomorrow I will have time to complete my C3 check.

I apologize for this useless whine/complaint.
Multi-file processing requires user copy and paste operations, as the output file is not cleared. Also, the 'conversion output:' doesn't print a successful operation. Since I had no errors, failure printout is unknown.

A batch file operation would be nice 'eye-candy'.
*.eng and *.xml for the input field.

I guess these issues can be put on a back-burner, as they are not essential. The converter is working so don't break it, yes.


Thanks for your time and patience, Pecunia.
-user3

ps: I use the 'Great Empires Collection II' and Emperor disk is circa 2003(requires patch); both are US distributions.

updates:
C3 check was perfection. Moving onto Cleopatra.
Cleo check was perfection. Moving onto Posd.
Posd check was perfection. Moving onto Emperor after a break.
Emperor check was perfection. Im happy with the product at this stage, completely usable.

[This message has been edited by user3 (edited 10-26-2017 @ 03:48 PM).]

user3
Pleb
posted 10-26-17 15:40 ET (US)     23 / 50       
Sorry to report an issue with Emperor Messages.

When the new years festival occurs the message is not within a dialog, but rather the title is displayed mid-screen left justified; then Emperor freezes without a crash.
Since the test campaign doesn't have any scripting yet; I request a commodity and got the same result when it arrived.

Others need to check the long term impact on other games. I m willing to do Emperor as Im currently testing out a campaign project.

-user3

[This message has been edited by user3 (edited 10-26-2017 @ 04:08 PM).]

Pecunia
CBH Seraph
posted 10-27-17 00:22 ET (US)     24 / 50       
Was that just with converting the ENG to XML and back again? I'll take a look at it this weekend.

Angel Pecunia | HG Downloads programmer and techie
Caesar IV Heaven | Children of the Nile Heaven | Citadel of Poseidon (Zeus/Poseidon) | The Great Wall (Emperor)
user3
Pleb
posted 10-27-17 00:45 ET (US)     25 / 50       
I did not do any editing of the XML, just converted ENG->XML->ENG.
« Previous Page  1 2  Next Page »
You must be logged in to post messages.
Please login or register

Hop to:    

Caesar IV Heaven | HeavenGames