Skip to content

Conversation

@andyross
Copy link
Contributor

@andyross andyross commented Nov 8, 2024

This is getting hot again so re-pushing a stale PR with a few bits added. Support for Zephyr builds for the Audio DSPs on MT8186, MT8188 and MT8196 SOCs.

@andyross
Copy link
Contributor Author

andyross commented Nov 8, 2024

Add shotgun gaggle of Xtensa reviewers.

Copy link
Contributor

@nordicjm nordicjm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See query about cpu cluster first, cpu clusters should be used if you are targetting a specific CPU/cluster on a SoC, the SoC name should not include the CPU

teburd
teburd previously approved these changes Nov 14, 2024
@andyross
Copy link
Contributor Author

OK, I puzzled out the bits needed to add the (IMHO, still needless) cpucluster to the soc tree. Please see if this matches what you expect.

I will say though, the process... wasn't smooth. Things like "CONFIG_SOC value does not match string defined in soc.yml" or "Cannot find soc directory" should be obvious error messages and instead are inscrutable secondary build failures. I spent a lot more time than I'd have expected stuffing message(WARNING ...) lines into cmake files. Also... the DTS filename changes. Why? The DTS file is a board asset, not a SOC thing. It lives in the board directory. It used to be named the same as the board, but now it isn't, because of something that changed in the SOC layer?

(Also rebased around a collision with llext support having been added to mt8195)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes are OK but you could alternatively rename the board to mt8188, then the filenames would be mt8188_adps.dts as it was before, and means your minimal build string would be west build -b mt8188//adps. Please squash the changes to the socs/boards added in earlier commits into the earlier commits

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd really rather have it be a separate patch if it's OK with you, for symmetry with the older already-merged device that is being changed and just to have a record I know I can look back to.

And the name still creeps me out. Again, this is not an integration for the whole laptop chip. It's just audio firmware.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about if you flip the order of converting existing boards to the new format as the first commit then have the new commits adding the final board names in the commits without having a fixup commit at the end?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does that really rise to a -1? It's a non-trivial rebase, and I personally like the order here as it makes the "complicated" change obvious.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are adding a commit 04ed839 and then fixing it in the next commit eab814b so yes. It's fine to have the commits in that order, but the first one should be changed to add the new boards with the correct names outright, then the second commit for updating existing boards, there is no point to adding a commit which has no use, is wrong and is fixed up in a later commit.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have a whole porting guide here: https://docs.zephyrproject.org/latest/hardware/porting/board_porting.html#hw-model-v2
We have an automatic porting script here: https://github.com/zephyrproject-rtos/zephyr/blob/main/scripts/utils/board_v1_to_v2.py
We have a whole branch here showing conversions we did for in-tree boards: https://github.com/zephyrproject-rtos/zephyr/tree/collab-hwm
Why do we need a new board to be added that, as part of the same PR, adds the board wrongly in the first commit by not using the things specifically defined for hwmv2 and also should anyone want to bisect a failure for this board would result in them having to use a different board name if they get to the first commit?

If you think the zephyr documentation is lacking, you can open an issue about it or submit a PR improving it, if you want to reference to see how you ported something then you can host that yourself locally on or github or anywhere

Copy link
Contributor Author

@andyross andyross Nov 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

adds the board wrongly in the first commit by not using the things specifically defined for hwmv2

Yeah, I don't want this to become a big monkey fight, but that's distressingly overstated. In fact I checked: there's nothing in the existing docs that demands the use of a cpucluster, it just talks about what the syntax means. The demand for particular structure was your personal design aesthetic. And... that's fine! Your code, your model, you get to make demands like that.

But it's not correct to characterize the early patches as "incorrect" either, given that they matched the idioms used by many (probably most) existing boards in the tree. So if you do make those demands, part of the job of being a good maintainer is staying with the PR as developers push back with questions and attempts at compromise and not walking away in pique.

Copy link
Contributor

@nordicjm nordicjm Nov 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not my personal design aesthetic, it's the design and whole purpose of hwmv2 (which was pioneered and designed by Torsten not myself) and all boards (in tree that is, I don't really care about out of tree boards, people can do as they please) need to follow that. hwmv2 was agreed by people to be brought in and used going forward with all the features it has, a lot of time was spent on things like terminology and having the right parts of devices in SoCs in discussions to get the system that everyone wanted. Is hwmv2 the best thing since sliced bread? No, but is it a huge marked improvement over the wild west that was hwmv1 which had so many irregularities and people just doing whatever which was specific to them and incompatible with others, it brings organisation to what previously was chaos. Is hwmv2 complete? No, is hwmv2 unfathomable? No, can hwmv2 be improved to make things easier and better to support for more complex use cases? Absolutely, hence why we have open issues for improving features and adding new functionality. And really porting a board or SoC should not be hard, I ported 100s in the conversion all manually by hand, made a lot of mistakes that needed to get patched up but the conversion was not hard but I work on the build system and looked at hwmv2 before that anyway, so if you find something to be hard in doing that or documentation is wrong then by all means open issues about them. On the board/SoC PRs I've seen with adsp (which have all been NXP except for this) they have all be added as CPU clusters.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Jaime... I did what you asked, two weeks ago, even, and you said at the time it was correct. I'm only talking about your characterization of the patch changing the board names to add cpucluster bits as "wrong", when again we have multiple boards in tree right now that don't conform to that scheme, and likely won't.

So I view that change as "development" (especially since it took me a few hours to figure out) and think it should be a separate patch in the tree. And I continue not to understand the level of argument about it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If there are other boards not complying then they should be fixed, these are not boards/socs I use so I don't know of any

Comment on lines 206 to 207
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this not needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Zephyr will zero its own bss after boot, so no. It's a hygiene thing only (to guarantee that all memory looks consistent, even things like .noinit sections), but disabled because on one of the boards I have it appears to glitch the bus and reboot the whole device.

@andyross andyross force-pushed the mtk-updates-2 branch 2 times, most recently from 6cf4082 to 19d548e Compare November 15, 2024 19:54
@kartben
Copy link
Contributor

kartben commented Nov 27, 2024

It's NAKing a script because the imports statements... aren't sorted?!

Not even "not sorted alphabetically". It has its own ideas about package order that it doesn't seem to document.

Even better, apparently we can now no longer submit a change to a python script if it happens to include a hex constant with (gasp) lower case digits. I'll fix the scripts in this PR out of necessity and malicious compliance, but something seriously needs to change with this CI rule; it's not going to scale at all.

@andyross if there are scripts that you're including verbatim from other projects, like it looks to be the case based on copyright notices, it would be fine to just add them to ruff-excludes.toml IMO. cc @pdgendt

@andyross
Copy link
Contributor Author

No, they're my scripts. Just not new in this PR yet still failing on ruff checks because like most linters it doesn't understand the idea of "only check the deltas" (which is a really hard problem). It's the standard complaint about formatters: if the existing source isn't valid already, they have the effect of disallowing all changes.

@andyross
Copy link
Contributor Author

LOL. I've iteratively fixed the ruffination errors I can duplicate. Now it's failing with:

 Python format error:Run 'ruff format soc/mediatek/mt8xxx/gen_img.py'
File:soc/mediatek/mt8xxx/gen_img.py
 Python format error:Run 'ruff format soc/mediatek/mt8xxx/mtk_adsp_load.py'
File:soc/mediatek/mt8xxx/mtk_adsp_load.py
Compliance error, check for error messages in the "Run Compliance Tests" step
You can run this step locally with the ./scripts/ci/check_compliance.py script.

... there's no error there! What do I fix?!

@kartben
Copy link
Contributor

kartben commented Nov 27, 2024

LOL. I've iteratively fixed the ruffination errors I can duplicate. Now it's failing with:

 Python format error:Run 'ruff format soc/mediatek/mt8xxx/gen_img.py'
File:soc/mediatek/mt8xxx/gen_img.py
 Python format error:Run 'ruff format soc/mediatek/mt8xxx/mtk_adsp_load.py'
File:soc/mediatek/mt8xxx/mtk_adsp_load.py
Compliance error, check for error messages in the "Run Compliance Tests" step
You can run this step locally with the ./scripts/ci/check_compliance.py script.

... there's no error there! What do I fix?!

Didn't realize you moved the files. So yeah, the ruff-excludes.toml entries needed to be updated after all :) which would have had the original intended behavior of not complaining about existing files.
Not sure about the remaining error though :/ I guess I would try to run the ruff format commands locally to see if they output any error that the CI script would not be showing

@andyross
Copy link
Contributor Author

Ooph. Got it to work. What that text was trying to tell me is that "ruff format" reported that it had made changes to the file, but none that it logged. You can't just run ruff and fix what it complains about, you need to run ruff and commit what it doesn't complain about. :)

@andyross
Copy link
Contributor Author

Sigh, unbelievable. Yes, indeed I did move the files as part of the big record demanded above for hwmv2 compliance. But obviously my brain didn't file that as a "move" since it was just a minor change request trying to get this thing merged. And in any case I guess I didn't realize there was a pre-existing exclude list added for scripts already in-tree (FWIW: git understands the idea of file motion, it wouldn't be impossible to write a checker that could update the .toml based on the PR being examined).

This... really isn't supposed to be this difficult. There was a time when you could add a new board to Zephyr with half an hour of work and someone would merge it instantly. Because it's a new board; it's by definition not going to break anything and you can always fix it later iteratively. Now we're a giant tangle of fiefdoms and processes and only-mostly-documented idioms, and the only tool anyone has to exert control is the -1 button in GitHub.

These are very similar devices to mt8195, minimal changes needed
beyond boilerplate configuration.

In the process, this reworks the board/soc layout to be HWMv2
compliant, with "adsp" becoming a CPU cluster beneath the SOC.  So the
name of the boards to west become e.g. "mt8195/mt8195/adsp" (which can
be shortened to "mt8195//adsp" if desired).

Note that the cpuclk driver is not yet ported, it works only with 8195
(the clocking/power architecture seems similar between the parts, but
the graph of wells and clocks is different and historically these have
been three separate drivers in SOF).  The biggest changes are in the
image/loader scripts, which needed some rework for cross-device
portability.

Signed-off-by: Andy Ross <[email protected]>
Simple docs for this board family.  Not a lot of complexity currently.

Signed-off-by: Andy Ross <[email protected]>
This got missed by the first version, but recent SOF needs it for
STRUCT_SECTION_ITERABLE usage.

Signed-off-by: Andy Ross <[email protected]>
Add a twister YAML file for these devices.  Right now they're flagged
as xt-clang only, though xcc and SOF-derived gcc toolchains are known
to work too.  We'll enable the zephyr SDK once sdk-ng support lands.

Signed-off-by: Andy Ross <[email protected]>
The early boot function got renamed to a pseudo-standard "z_prep_c",
but this isn't an actual API and doesn't have a prototype in the
headers anywhere, so the compiler started whining about an undeclared
function.

Signed-off-by: Andy Ross <[email protected]>
New platform has different mappings.  Auto-detect rather than parse
dts or similar, as this is is really just a simple format for testing.

Signed-off-by: Andy Ross <[email protected]>
Add Zephyr support for the Audio DSP on the MT8196 SOC.  This is a
very similar device to previous designs.  Most of this patch is just
DTS.

The biggest delta is the more complicated second level interrupt
controller, though it is still able to be represented using some
vaguely clever DTS config over the older intc_mtk_adsp driver.

Also the memory layout is slightly different, requiring a little
indirection to set the initial boot stack address and log output
buffer.  And the timer "irq_ack" register bits moved.

Signed-off-by: Andy Ross <[email protected]>
These in-tree scripts fail the new ruff checks.  Clean things up so
modifications can merge.

Signed-off-by: Andy Ross <[email protected]>
These devices have an architecturally fixed 13 MHz clock device.  But
thankfully you can put a default into a DTS binding so we don't have
to repeat it for all of them.

Signed-off-by: Andy Ross <[email protected]>
@andyross
Copy link
Contributor Author

Fixed indentation in DTS. Any other complaints? Would be really, really nice to see a few +1's and get this merged before people start to disappear around the holidays.

Just as an aside @nordicjm, as far as the C++ namespace point, you're definitely in the minority there. Consensus is generally that "needless enforced top-level indentation" is a bad thing and most of the community has disallowed it. See these two popular style guides, for example:

https://llvm.org/docs/CodingStandards.html#namespace-indentation
https://google.github.io/styleguide/cppguide.html#Namespace_Formatting

Also clang-format won't do it and AFAICT can't be made to; you can see the examples in their docs: https://clang.llvm.org/docs/ClangFormatStyleOptions.html

I don't think it's really that much a stretch to imagine something similar in DTS context, the reason is identical (there's a top-level scope that is enforced by syntax and can't be avoided, so effectively the whole file gets indented for no particularly good reason). Something to consider for the future when we've all calmed down a bit.

Copy link
Contributor

@nordicjm nordicjm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for updating

@nordicjm nordicjm removed the Architecture Review Discussion in the Architecture WG required label Nov 27, 2024
@nordicjm
Copy link
Contributor

Just as an aside @nordicjm, as far as the C++ namespace point, you're definitely in the minority there. Consensus is generally that "needless enforced top-level indentation" is a bad thing and most of the community has disallowed it. See these two popular style guides, for example:

Interesting, was not aware of those. The codebase is so old now that I don't remember if it was the IDE that added the indentation or if I did, presumably it was me...

namespace Ui
{
    class AutMainWindow;
}

Just created a new project in the IDE and must have been me that indented it

namespace Ui {
class MainWindow;
}

@nordicjm nordicjm requested review from gmarull and nashif November 27, 2024 15:36
Copy link
Contributor

@kartben kartben left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I won't make this a blocker but the following will need to be addressed as follow-ups please:

  • documentation as per the comments ;
  • mt8195_adsp board being renamed to mt8195 will need to be mentioned in the 4.1 migration guide

Comment on lines +1 to +5
.. _boards-mtk_adsp:

Mediatek Audio DSPs
###################

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will need some follow-up work as it's not really aligned with the rest of boards' docs.
mediatek/index.rst should be the index of all Mediatek boards, and each board should have its own doc file.

I appreciate you want some common doc bits across all similar boards, and it's fair to want to group everything right in the same file, but this top-level index.rst should really be for all Mediatek boards, whatever they might be (see index.rst files for all other vendors).

Note that for this reason, the boards listed here lead nowhere for they appear to the documentation infrastructure as "doc-less".

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Heh. Well I do need to point out that when that patch was written, that directory was exclusively to Audio DSPs. :) But sure, will figure it out. Presumably I can have stub pages and link to the shared one or whatever.

@@ -0,0 +1,5 @@
boards:
- name: mt8186
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here and for other boards -- this will help boards show nicely under their commercial name in the board catalog I just linked before.

Suggested change
- name: mt8186
- name: mt8186
- full_name: MT8186

Copy link
Contributor

@ajf58 ajf58 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

I've raised a couple of nits, but nothing that should gate this being merged onto main.


.. code-block:: console

user@dev_host:~$ west build -b mt8186//adsp samples/hello_world
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
user@dev_host:~$ west build -b mt8186//adsp samples/hello_world
user@dev_host:~$ west build -b mt8186/adsp samples/hello_world

nit/not a blocker, but is there an extra forward slash that crept in?

**********

The MT8195 toolchain is already part of the Zephyr SDK, so builds for
the ``mt8195//adsp`` board should work out of the box simply following
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
the ``mt8195//adsp`` board should work out of the box simply following
the ``mt8195/adsp`` board should work out of the box simply following

nit: As above, not a blocker, but has a double slash crept in?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope, see above. :) That's the real name, it's shorthand for "mt8195/mt8195/adsp"

Comment on lines +52 to +62
ostimer64: ostimer64@10b83080 {
compatible = "mediatek,ostimer64";
reg = <0x10b83080 28>;
};

ostimer0: ostimer@10b83000 {
compatible = "mediatek,ostimer";
reg = <0x10b83000 16>;
interrupt-parent = <&core_intc>;
interrupts = <18 0 0>;
};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: could/should these entries be in reversed order, given their addresses?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They're not the same kind of device. The first is the 13MHz 64 bit upcounter used for system timing, the second is a 32 bit 26MHz down-counting interrupt source. The timer driver uses them both, but in fact the second has multiple instantiations on the chips (though not used by Zephyr), so they make sense to declare separately. As far as order: my sense of aesthetics likes having the master counter first, I guess.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK. For sure, they're different, I'm more looking at this with an eye on https://docs.kernel.org/devicetree/bindings/dts-coding-style.html#order-of-nodes . It's not a thing mandated/enforced on the codebase, and there's plenty of idioms/prior art that don't make this. Not a blocker.

Comment on lines +65 to +75
ostimer64: ostimer64@1080d080 {
compatible = "mediatek,ostimer64";
reg = <0x1080d080 28>;
};

ostimer0: ostimer@1080d000 {
compatible = "mediatek,ostimer";
reg = <0x1080d000 16>;
interrupt-parent = <&intc23>;
interrupts = <11 0 0>;
};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: As above, should these nodes be the other way round?

Comment on lines +79 to +89
ostimer64: ostimer64@1a00b080 {
compatible = "mediatek,ostimer64";
reg = <0x1a00b080 28>;
};

ostimer0: ostimer@1a00b000 {
compatible = "mediatek,ostimer";
reg = <0x1a00b000 16>;
interrupt-parent = <&intc_g1>;
interrupts = <8 0 0>;
};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: As above.

@kartben kartben merged commit b4fb833 into zephyrproject-rtos:main Nov 28, 2024
29 of 30 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.