Districts

[instafluff0]

@maxpetul , I'm a huge fan of your work with C3X, and as many others have said, thank you!

Basically, I'm interested in creating a mod based on C3X (fork here) which adds "districts (light)", or essentially the visual elements of districts from Civ 6 (literally, as tiles on the map), only without adjacency bonuses or anything like that. So a building like barracks would require having an encampment district within a city's radius, etc.

Let's please leave aside for now whether this is a good idea and a million-and-one other details about gameplay I need to figure out. Long-story-short, I tried really hard to enjoy Civ 6, and loved the urban sprawl look, but in terms of gameplay nothing has been as fun as Civ 3. So trying to make a happy marriage of the two...

Visually, something like this:

I've reviewed the repository extensively as best I can, and also familiarized myself with Ghidra and analyzed the exe code and confirmed I can see the correspondence with the gog mem addresses in civ_prog_objects.

I have a number of questions though that I hope won't be too onerous, and that you can point me in the right directions and teach me how to fish. Thank you in advance for any guidance you can provide 🙏

How a district would be built

I'm thinking workers (not cities) would build a district as an additional action. So for example:

1. Worker moves to flat terrain tile within radius of city
2. Additional button shows up in actions, like 'Build Encampment' or 'Build Neighborhood', which each take 10 turns for one worker (districts available will have dependencies on a tech)
3. 10 turns pass
4. District pcx image shows up on improved tile
5. Cities for which district are in radius are now able to build barracks, for example
6. I haven't yet decided if a district could still be 'worked' by the city, and what yields they would offer, as they'd need to somehow offset the loss of potential mines, irrigation, etc.

Obviously there are many details and checks involved that I'll need to figure out, but I'm trying to conceptually keep it simple for now. Also, as you can see in the example image, I'm thinking of re-using the city pcx art for a given culture & era as the 'Neighborhood' art. Others I'll make new pcx files for.

Questions

1. I need to add a worker job for this to the Worker_Jobs enum, but also any idea
   • Where I could set the cost per turn? (ie, to make it take 10 turns)
   • Where to set the dependency between an improvement and a given tech?
   • Where to set the yield value of a given improvement (i.e., how yield would be calculated for a given terrain with a district improvement)
2. I assume I would need to implement something like init_stackable_command_buttons and related functions, but I'm unclear on the relationship between the dimensions & number of images in pcx files and the params that go into Sprite_slice_pcx, e.g., Sprite_slice_pcx (sprite, __, pcx, 0, 0, 64, 32, 1, 1);.
3. What function(s) draw the terrain improvement images on tiles?
4. What function(s) determine buildings a city can build? I'm assuming I'll need to intercept or rework that to take districts in city radius into consideration.

Sorry for the long post. Any pointers you have about how you'd approach this (like, a step-by-step laundry list of how you would approach this would be amazing) and in particular how to work with pcx images (tutorials in Civ fanatics tend not approach this from a code perspective, so seem less helpful), would be greatly appreciated. Thank you again in advance.

Edit: Thinking about this a bit more, I'm wondering if the approach you take with set_up_stack_worker_buttons, specifically this loop could be adapted to intercept and add the Build Encampment, Build Campus, etc. buttons at runtime, rather than formally adding as worker job types and so on. Is this what you'd recommend?

[maxpetul]

Hello! You're very welcome, and I'm glad you enjoy my work. I'd be very happy to have collaborators on this project.

You're jumping in the deep end by starting off with adding a new worker action. The game's base set of worker jobs was unfortunately hard coded in many ways. Not in all ways, for example the list of worker jobs read from the scenario data is dynamically allocated (see 0x9C7324, the WorkerJobs array inside the BIC data object). So there shouldn't be a problem adding a new entry to that array. Inside that array is where the turns-to-complete and required tech ID are stored for each worker job. Turns-to-complete is used at 0x46159E in Unit::work_simple_job, and the tech requirement is enforced, I believe, in Unit::can_perform_command.

The yield increase from completing a worker job comes from the tile and terrain. Completing the job sets an overlay bit on the tile and then the terrain type defines a bonus yield for that overlay. For example, see the call at 0x5D7779 in Map::calc_shield_yield_at. That call is to Tile::m18_Check_Mines (0x5EA810) to check if there's a mine on the tile, and if there is, it adds the mining bonus for the tile's terrain type to the yield. However, to customize tile yield you don't have to bother with overlay bits or terrain, the simplest approach would be to patch Map::calc_shield_yield_at to add however many shields you want to tiles with your new improvement. (Though, the despotism penalty makes that a bit awkward.)

To answer your other questions:

2. Sprite_slice_pcx takes a PCX Image, upper left x & y coordinates, a width & height, and two mystery parameters, in that order. I don't know what the last two params are for, and for their values I simply copied what the original game passes when creating the command button sprites.

3. The drawing of the map is annoyingly split up into dozens of virtual methods. The ones you're looking for are at:

• 0x4C4060 -- void Map_Renderer::m38_Draw_Irrigation (Map_Renderer * this, int terrain_index, int sprite_index, Map_Renderer * map_renderer, int pixel_x,int pixel_y)
  • Called from 0x5F61A0 -- void Map_Renderer::m11_Draw_Tile_Irrigation (Map_Renderer * this, int visible_to_civ, int tile_x, int tile_y, int param_4, int param_5, int param_6)
• 0x4C4480 -- void Map_Renderer::m41_Draw_Mines
  • Called from 0x5F6B20 -- void Map_Renderer::m12_Draw_Tile_Buildings (Map_Renderer * this, int param_1, int tile_x, int tile_y, Map_Renderer *map_renderer, int pixel_x,int pixel_y)

To draw a new kind of tile improvement, I'd probably patch m21_Draw_Tile_Buildings. Though there are other approaches, for example you could patch m41_Draw_Mines if you wanted to make your improvement secretly a mine under the hood.

4. Building availability is determined by City::can_build_improvement at 0x4BFF80. It would be straight forward to patch that and have it require a special improvement within the city radius. I've already done something similar in patching City::can_build_unit to enforce building prereqs and unit limits.

To approach this, I would first of all focus on getting the worker job working since that's the most technically uncertain part of what you're planning to do. I'm certain it's possible to make districts appear on the map and to add a new button for them to the interface. If there's a technical roadblock here, it's probably going to be in creating the worker job or getting the AI to use it. One problem I can see already is that worker jobs correspond to a chunk of unit states, so the worker job enum's build mines (0) through build barricade (0xC) corresponds to unit states 2 through 0xE. The next unit state after those is intercept (0xF) so trying to repurpose that might cause issues. Another problem I can see is that there's a second list of worker jobs in the Map object of type Map_Worker_Job_Info. That array is fixed in size so it won't be possible to extend it like the one in the BIC data.

Keep in mind that if it turns out to be too difficult to add something, another approach is to repurpose one of the game's existing things with a tag along the side to treat it specially. For a specific example, I never added a real stack bombard command to the game. Under the hood, the stack bombard command is just the regular bombard command but the mod makes a note when it's activated that it's the stack version. Then it intercepts the point at which bombard commands are executed by a click on the map and performs stack bombard if the flag is set for it. You could do something similar by making districts be mines under the hood. Tiles with districts on them would be set as mined and the mod would track somewhere which ones actually had districts. Then you'd intercept all the relevant points in the logic, including where the mine graphic is drawn, the yield is calculated, the job is finished to produce the mine, and so on, in order to make the district-flagged tiles behave specially.

Yes, I would recommend inserting any additional buttons at runtime. The base game sets up the command buttons by looping over the possible unit actions and there's no reason to bother with trying to modify that. For an example of adding a new button, it's probably better to look at what I did for the stack bombard button. The stack worker command buttons only replace their non-stack versions.

Hope all that helps. By the way, if you want my Ghidra project, I could send it to you. I have well over a thousand functions named in the GOG version, many more than are listed in civ_prog_objects.csv. I also have all of Civ3Conquests.h imported, I wrote a Python script to do that long ago. It would be a lot more convenient for you than recreating everything.

[instafluff0]

Hello! You're very welcome, and I'm glad you enjoy my work. I'd be very happy to have collaborators on this project.

Awesome! Thanks very much and likewise. The information above is really helpful, and through playing around a bit (the tip you gave on debugging by inspecting at runtime was very helpful) I tentatively came to many of the same conclusions you mention. (I saw 42, the base number of worker jobs, I think, hard-coded all over the place, for example).

Keep in mind that if it turns out to be too difficult to add something, another approach is to repurpose one of the game's existing things with a tag along the side to treat it specially. For a specific example, I never added a real stack bombard command to the game. Under the hood, the stack bombard command is just the regular bombard command but the mod makes a note when it's activated that it's the stack version.

Right, as I dug in a bit I realized this! I really like the approach. I'm also strongly considering the approach of having the districts be represented by a mine/irrigation under the hood, and short-circuiting the functions you mention.

By the way, if you want my Ghidra project, I could send it to you.

Yes please! Let me know what works for you, my email is in my profile.

I also have all of Civ3Conquests.h imported, I wrote a Python script to do that long ago. It would be a lot more convenient for you than recreating everything.

This would be great as well 🙏

[maxpetul]

Yes please! Let me know what works for you, my email is in my profile.

Great, I'll email you about it.

This would be great as well 🙏

Alright, I uploaded it to the repo here: https://github.com/maxpetul/C3X/blob/master/Ghidra%20scripts/Civ3TypesFromHeader.py
Not sure how useful it will be, really, it's an old script that I hacked together just to get the basic contents of Civ3Conquests.h loaded into Ghidra. It probably won't even work with the state that header file is in now, and maybe won't be compatible with newer versions of Ghidra. But anyway, there it is if you want to have a look at it.

[instafluff0]

Ok, I was successfully able to get the custom worker buttons to create districts showing up, basically taking over free buttons, as in the bombard button case. I also confirmed a slight tweak to patch_Main_GUI_handle_button_press seems to then work as expected in the district-related build command being captured after click.

I've next tried to patch m12_Draw_Tile_Buildings. I basically followed the pattern I observed in m19_Draw_Tile_by_XY_and_Flags, which is also a patch of a Map_Renderer method.

• Entry in civ_prog_objects
• patch_Map_Renderer_m12_Draw_Tile_Buildings function in injected_code (which currently does nothing but call the base method)

This seems to compile fine, but the game appears to crash immediately upon starting in the map view:

I'm mystified why this would be, and have checked all the method params and so on and can't find any issues. As far as I can tell, I don't think I need to make any changes to CX3.h or Civ3Conquests.h, but could be wrong.

[maxpetul]

I'm glad to see you're making progress. You've made a mistake in your civ_prog_objects entry. The addresses on "repl vptr" lines must be of the v-pointers themselves in the object's vtable, not the addresses of the functions to be replaced. In this case, the address should be 0x66A538, it's easy to find since it's 28 bytes below the address of method m19 in the same vtable.

[instafluff0]

Great, thanks, I'll take a look

[instafluff0]

Next question: I have a Tile object and want to enumerate all surrounding tiles with the same owner within the current config city_work_radius. What is the best way to do so?

I've reviewed the ni* related functions, checked the source and used Copilot quite a bit to try to grasp but need some help please. If you could share a snippet or something similar that would be great.

Use case: I need to check if surrounding Tiles have districts in a few functions.

---

Edit: I just stumbled on the FOR_TILES_AROUND macro and think I have the answer on this one, nevermind.

[maxpetul]

Alright, but just to be clear, there's a second neighbor_index_to_diff function for city work areas called patch_ni_to_diff_for_work_area. The FOR_TILES_AROUND macro is how you'd do this for every case of looping around neighboring tiles except for looping over a city's work area. The exception is due to the unfortunate complication that the game's standard way of indexing neighboring tiles doesn't coincide with city cultural borders. I decided when I was working on the expanded work areas that I wanted them to match cultural borders, so that e.g. a work radius of 3 matched exactly level 3 culture, so had to write a second set of neighbor index -related functions to handle that. For an example of how to loop over tiles in a possibly expanded work area, see has_resources_required_by_building_r, specifically the loop for (int n = 0; n < is->workable_tile_count; n++) {...}.

[instafluff0]

Hey @maxpetul, update time: I've made huge progress with the Districts feature.

The README in my districts_v1 branch mostly sums it up.

In a nutshell:

• Standard Districts are fully configurable and customizable
• After a city builds a building that is dependent on a District, the art updates to show the building on the map
• Neighborhoods (special district type) allow urban sprawl
• Wonder districts (special district type, also fully configurable) allow Wonders to be shown on the map, or even destroyed and rebuilt
• Distribution Hubs (special district type) allow "breadbaskets" (for both food and shields), effectively distributing food & shields from one area of a civ to ALL civ cities connected by trade route
• District art can vary by culture/era/buildings
• Districts can have food/shield/gold/science/culture/defense bonuses, and be shared between cities
• AI builds Districts (this took a while to figure out but seems to be fully working)
• Fully compatible with day/night cycle.

So I kinda went overboard but wow, it basically became an expansion pack.

I'm not quite ready to make a PR yet and need more time to test and clean things up. I'm also a bit concerned about how huge an addition this would be to C3X in terms of code and complexity, and fully understand you are inundated with various requests and also are doing the Lua implementation. So I'd be amenable to either merging or keeping Districts as a separate mod, though my preference would be to merge it in if you were ok with it (and it didn't dramatically increase the maintenance burden).

[maxpetul]

Wow, that's very impressive work! Like before, I don't think it makes sense to have multiple versions of C3X around with different feature sets. It's better to keep it all together. Though one reason you might want to post your own versions is so you're not tied to my release schedule which can be pretty slow. But in this case, districts could be included immediately in R25 since I can post that any time now. There's nothing more that really needs to be done for the final version of R25, I'm just giving it some time for bug reports to come in. If you wanted to do an R25 Preview 2 for the districts, I'd be fine with that too.

[instafluff0]

Thanks very much! I think that with so many people waiting for the official release of R25 w/ city count limit lifted, etc., I don't want to hold that up. I also suspect Districts will need a lengthy feedback period of who knows how long. So I'd suggest let's move ahead with R25 as-is and tentatively plan for Districts to be in R26. I agree about posting my own version - I'll do that similar to the Day/Night Cycle and that way interested folks can do beta testing without complicating the current release cycle.

On an unrelated note, I was able to figure out most of the technical issues I'd been stuck on, but one minor but annoying one kind of remains:

Can you figure out why rivers are shown on top of sprites (I think) rendered after them? In impl_m19_Draw_Tile_by_XY_and_Flag buildings appears to come after roads/railroads, and in any case I thought the lower tiles are rendered afterward, and thus buildings should appear on top of rivers, yet that's not the case. Curiously, cities on rivers don't have the same problem.

[maxpetul]

Alright, my plan was to wait a week or two for R25 anyway, that's way I think it's plausible the districts feature will be done in time. It appears to be coming along quickly. Even if it's not, we can always have an R26 Preview 1 for that.

Looks like map drawing is done over multiple passes. Just as a quick test, I inserted a breakpoint on m19_Draw_Tile_by_XY_and_Flags conditioned on tile_x/y == 50, and found that triggers five times each time I move the camera. Presumably you could edit the flags so it draws tile buildings on the same later passes as the rivers.

There are also checks for cities, colonies, radars, fortresses, barb camps, etc. on tiles in m15_Draw_Tile_Rivers_and_Flood_Plains. It's probably appropriate to edit those to include a check for districts as well. Although I don't know what the point of those checks are or if that edit would fix the problem with rivers getting drawn overtop.

[instafluff0]

Awesome. I think at my current pace (I definitely can't keep it up, but am racing to get these ideas out of my head and into code) I can have things ready for testing in a week or so. My main concern is I've been so busy writing code and doing art, I haven't yet progressed in a game into the industrial or modern era. I don't know how the AI will perform long-term and how the art will look. But maybe that's ok and it'd be better to have feedback and ideas anyway.

Dumb question: I'd like to patch m15_Draw_Tile_Rivers_and_Flood_Plains but I don't see it in Ghidra.

Do we just not know where in memory that virtual method lies? I confess I've been heavily reliant on your identification of the variables and functions and haven't investigated enough myself to know why it's not listed.

[maxpetul]

The names of all the map renderer vtable functions were found by Antal before me. That's why they're named in Civ3Conquests.h but not defined in Ghidra. I haven't bothered to go through and define them all, just the ones I'm interested in looking at. It's easy to find the address of a vtable function once you have any other function in the same vtable because they're all listed in order in the table. You can just go to, for example, m14_Draw_Tile_Railroads then look at the references to it, 0x66A540 is the reference in the main map renderer vtable, double click on that to go to the vtable, and there m15 is listed right after m14. I've also created bookmarks for all the vtables that I've identified so you can jump to them that way.