Pragma to prevent symbol from being optimized away

[firelizzard18]

I would like to create symbols (variables, constants, functions) that may not be directly referenced, but need to exist in the executable for one reason or another. I'm imagining adding a keep bool field to compiler.globalInfo and case "//go:used" or case "//go:keep" to globalInfo.parsePragmas. Then something would need to be added to keep the symbol, maybe in compiler.getGlobal.

I would be happy to create a PR implementing this, but I can't for the life of me figure out how to mark a global as "this will be used, do not optimize it away". There must be some way to do it, because clang respects __attribute((used)), but googling "clang used attribute" is not effective.

[aykevl]

What is your use case here? I suspect there is a simpler solution, but I'd like to know first what you need this for.

I can imagine two cases:

1. It is used by the program in the same compile unit. In that case, simply using it keeps the symbol alive.
2. It is used outside of the compile unit. In that case, you need to set the linkage to external (TinyGo defaults to internal linkage, comparable to C static functions). For functions there is //export with exactly that purpose.

There is also the SimpleDCE pass that I've been trying to get rid of, but it does not apply in the runtime package.

[firelizzard18]

As per #863, the Teensy 3.6 requires 16 bytes at 0x400 for flash configuration. Currently, I am achieving this by including targets/teensy36.s (plus a linker rule). I really don't like having this defined in an assembly file. I added support to tinygo for //go:section name pragmas. With this pragma and the linker rule, I moved the flash configuration to src/machine/board_teensy36.go:

//go:section .flashconfig
var FlashConfig = [16]byte{
	0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
	0xFF, 0xFF, 0xFF, 0xFF, 0xDE, 0xF9, 0xFF, 0xFF,
}

However, I can't find any way to have FlashConfig show up in the final binary that doesn't involve a bogus reference to FlashConfig. But it doesn't make sense for any code to reference FlashConfig. My bogus reference is println(string(machine.FlashConfig[:])), which is definitely not something I'd actually want to do.

[firelizzard18]

In 42c2940 you can see the change that this allows me to make

[aykevl]

I really don't like having this defined in an assembly file.

Why not? It's a low-level detail of this particular platform. In fact, you could even put it directly in the linker script. The GBA target uses assembly for this purpose:

tinygo/targets/gameboy-advance.s

Lines 8 to 17 in 1d913a6

	.fill 156,1,0 // Nintendo Logo Character Data (8000004h)
	.fill 16,1,0 // Game Title
	.byte 0x30,0x31 // Maker Code (80000B0h)
	.byte 0x96 // Fixed Value (80000B2h)
	.byte 0x00 // Main Unit Code (80000B3h)
	.byte 0x00 // Device Type (80000B4h)
	.fill 7,1,0 // unused
	.byte 0x00 // Software Version No (80000BCh)
	.byte 0xf0 // Complement Check (80000BDh)
	.byte 0x00,0x00 // Checksum (80000BEh)

[firelizzard18]

If it's in an assembly file, it is harder to find, and it is much less accessible to Go developers who are not familiar with assembly. IMO putting it in the linker script is worse because those are black magic to me and I'm guessing to most developers.

I'd argue that the flash configuration for the K66 isn't the same as your example for the GBA. I'm not familiar with programming the GBA, but it looks like that is a series of essentially magic values that must be those exact values (for the most part) for everything to work. Whereas the flash configuration of the K66 is a well documented feature of the microcontroller that some users may want to modify. Granted, you can brick your chip if you're not careful, but "WARNING HERE BE DRAGONS" (plus details) in the field's comment would suffice.

[aykevl]

IMO putting it in the linker script is worse because those are black magic to me and I'm guessing to most developers.

Fair enough, they used to be black magic to me too (and I still don't understand them completely). Linker script should probably only be used when there are no other options.

I'd argue that the flash configuration for the K66 isn't the same as your example for the GBA. I'm not familiar with programming the GBA, but it looks like that is a series of essentially magic values that must be those exact values (for the most part) for everything to work.

True. I suspect they are modified a bit by the flashing programs (because they need to include the Nintendo logo and stuff) but they have to be in that particular location.

Whereas the flash configuration of the K66 is a well documented feature of the microcontroller that some users may want to modify.

Can you give an example of something users might want to modify?

In general I'm rather hesitant with pragmas. I have indeed added some but I've also removed some again (such as //go:volatile and //go:interrupt). And I don't see many uses for putting globals in sections apart from specific "these bytes should go exactly here". Another reason I'm hesitant is because globals are normally mutable in Go. With some compiler tricks and heuristics they often become immutable but they're not really guaranteed to be immutable. In general, I would suggest to put the configuration in an assembly file for now and move them over to Go when there is a clear use case for them. Nothing is set in stone after #863 is finished.

[firelizzard18]

I could read through the documentation and try to invent a reason (probably related to the Flash Option Register (FOPT) boot options), but I don't myself have a reason to change those fields. It's more a matter of principle. Ideally I'd like them to be user-modifiable without touching src or targets. This feels like one of those moments where I'm working with some tool/library/framework and want to change something only to discover it's been hardcoded (cue foul language). So my thought process is, "I can imagine being in a situation where I would want to change this and being very frustrated that I can't and still use the tool as provided, unmodified, thus I should do what I can to make it easily overridable."

That being said, I switched back to an assembly file on #863 because I was getting unpredictable, non-deterministic linker errors. Something isn't working right with my pragma.

[aykevl]

"I can imagine being in a situation where I would want to change this and being very frustrated that I can't and still use the tool as provided, unmodified, thus I should do what I can to make it easily overridable."

One thing I learned was to not provide flexibility when there isn't a clear use case. It's the KISS principle. I've often seen cases where something was supposed to be flexible but when you actually try to use the feature it doesn't quite fit whatever you're doing. It's also how I interpret the Go philosophy: provide few ways to do something that are well tested and work in real-world software. Providing more options may help someone but increases complexity for everyone else.

Anyway, I think this is enough bikeshedding already.

[firelizzard18]

So should I close this? I was using //go:keep for the flash config, but regardless of the concerns expressed here, that didn't even work reliably (lld would fail, depending on conditions I do not understand). And I was using //go:section for the flash config and for replicating the memory layout that Paul uses for the teensy core. And while it is disappointing that addresses 0x1D0 through 0x400 are wasted with the current linking strategy, I could not reliably get the reset handler to fit in that space (adding GC and scheduler support increases code size, sometimes beyond that limit).

[firelizzard18]

I have isolated the changes I made and pushed them to firelizzard18:linker-pragmas for posterity, so I can remove them from #863 without losing any work.

[firelizzard18]

And I don't see many uses for putting globals in sections apart from specific "these bytes should go exactly here"

On the K66, the USB descriptor table must be 512-byte aligned. Given that the table needs to be accessible as a variable in the machine package, I'm not sure how else to do this.

The BDT resides on a 512-byte boundary in
system memory and is pointed to by the BDT Page Registers.

The BDT page registers are three 8 bit registers, and the register used for the low part of the address, BDTPAGE1, does not allow writes to its lowest bit. This means that it is impossible to use a BDT that is not aligned properly.