merging language specific tasks with a tag's tasks

[nhomble]

Background

I have tasks like :build for my specific language, like typescript.

I am also exploring some code generation from specifications like openapi spec. I didn't consider the moon generate because this seems more for project scaffolding and not prebuild generation (but please point me to separate docs if that's incorrect). To structure this, I have certain moon projects that are basically stubs with just the openapi specification with a tag: codegen.

The output of the task is going to be typescript project.

NOTE the output is not always typescript. Sometimes we are also generating other configuration files out of the openapi spec. So from the same openapi spec, but different moon project stub, we won't always create a typescript project post generation.

Problem

I am unable to make :codegen a dependency of typescript :build

Attempts so far

implicitDeps

I have tried implicitDeps: ['^:codegen'] in the tasks/typescript.yml but I don't see anything in the action-graph --dependents.

adding :build to the codegen tasks

I considered adding a :build task in the tasks/tag-codegen.yml, but this also fails because not all codegen tasks create a typescript project. You could wonder, why not create a separate tag? But the tag would be something like codegen-typescript which makes me think I am missing some composition feature of tasks since the idea of typescript already exists.

[milesj]

@nhomble For ^:codegen, does a project that inherits the build task depend on a project that has a codegen task? That's the only way that would work.

From your post, it kind of sounds like there is no codegen task, just a tag file called codegen. What's in that tag file?

[milesj]

@nhomble To confirm, the setup looks like this. Project A has a tag codegen (and inherits the codegen task), and project B inherits the typescript tasks (and inherits the build task) and depends on project A?

What's the output of moon project a --json and moon project b --json? The same could be said for tasks, moon task <target> --json.

Additionally, you can maybe try this for the build task:

deps:
  - target: '^:codegen'
    optional: true

[nhomble]

Actually project A and B are the same thing. I apologize for not posting the exact configuration but you can imagine its got a lot of noise with other things going on and its internal.

Project structure:

project-a/
  moon.yml
  generated/
    package.json
    foo.ts
    tsconfig.json
project-b/
  moon.yml
  other-configuration.txt

In project-a/moon.yml

type: library
language: typescript
tags:
  - codegen

and then project-b/moon.yml

type: configuration
tags:
  - codegen

---

my intention with

type: library
language: typescript
tags:
  - codegen

was to add the codegen step and then reuse the :build step in the typescript-library task definition

---

Finally with the task structure, codegen is a task that lives in a tags-codegen yml which is separate from build which is in typescript-library. Somehow, I want the build to depend on :codegen. With the wrinkle that not all codegen tagged projects are typescript projects and not all typescript projects need codegen

[milesj]

Ah ok, that makes sense. I think this should work if you add it to the typescript build task.

deps:
  - target: 'codegen'
    optional: true

^:codegen is tasks from dependency projects, while codegen is a task within itself. If that still doesn't work, I'll try to recreate it locally, it may be a bug.

[nhomble]

this got me somewhere :) - I missed this section in the docs: https://moonrepo.dev/docs/config/project#optional-1. Sorry about that.

all dependencies are required to exist when tasks are being built and expanded, but this isn't always true when dealing with composition and inheritance

I think this exactly describes what I am trying to achieve.

[milesj]

Let me know if that works!

[nhomble]

Hey @milesj, since implementing this solution, we've had some additional dependency issues with other pipeline steps. Tasks on our applications that depend on the generated library for this optional codegen step, don't have the codegen step. So we have to explicitly invoke :codegen in our pipeline script rather than relying on the moon DAG.

I tried to come up with a representative example here: https://github.com/nhomble/example-moon-typescript-codegen. If you'd prefer a separate discussion, I'd be happy to repost somewhere else too.

[milesj]

@nhomble Oh, I think it's because the codegen folder does not have a package.json. The optionalDependencies is basically pointing to nothing that can be inferred.

[nhomble]

That makes sense... I suppose we could include the package.json in the stub moon project. We were hoping to keep the stub as light as possible. I'll confirm the results tonight

[milesj]

You could also explicitly set the project dependency: https://moonrepo.dev/docs/config/project#dependson

[nhomble]

First, I wanna say thanks Miles for all of the discussion so far!

The good news is that I can create the action graph I need on main. 2 important changes:

• package.json (and tsconfig) are committed
• implicitDeps now includes ^:codegen

What's still a gap for me is that I wasn't expecting to need ^:codegen, I was hoping to just add a dependency to ^:codegen to the :build task defined within application-library (this was the initial reason for opening the discussion).

Here's an example diff that illustrates my confusion: nhomble/example-moon-typescript-codegen#2

[milesj]

That makes sense. I know this works for qualified targets (has project scope), but the ^ parent scope may not support this based on the order of operations. I'll have to check.