🚀 Announcing Beta Release: --dry_run

[kushalshit27]

🎉 We’re excited to announce the beta release of --dry_run support in 8.14.0-beta.0 🚀

📘 Learn more about how to use it here: Using Dry Run

Previous Discussions thread

💬 We welcome your feedback on this feature, as your insights are essential to helping us improve and deliver the best possible experience. 🙌

[alvaroloes]

Love this! I have been integrating this in my CI this week.

My use case is the following:

• I have a Github "deploy" action that uses the dry run to generate the differences.
• Those differences are then pasted into an "approval" Github issue so that the team can check them and approve.
• If approved, the "deploy" action continues and deploys the changes to the Auth0 tenant.

Let me give you some feedback after testing it for some days:

1. [BUG] I could only make it work by using the env var AUTH0_DRY_RUN=true. The flag didn't work
2. [MINOR] For me, the DELETE resources are always shown, even if I don't use the AUTH0_ALLOW_DELETE=true env var. Nothing bothering, though.
3. [FEATURE] The interactive mode is great when working locally, but for a CI like the one I described above, it complicates things. My goal is to generate the JSON with the differences so that they can be shown in the Github action, but there is no way to run the dry-run in non-interactive mode. I simulated a user hitting the "down arrow" and "enter" with the expect library, but it would be ideal to have a command flag like --dry-run-option=export.
4. [FEATURE] When there are differences in an email template or the code of an action, the full content of both versions are shown in the corresponding property of the JSON file, even if only one line changed. It would be great to include just the diff

Thanks a lot!

[kushalshit27]

Hi, @alvaroloes ,
Thank you for taking the time to provide such detailed feedback — we truly appreciate it and we are carefully reviewing each point.

[BUG] I could only make it work by using the env var AUTH0_DRY_RUN=true. The flag didn't work

Question: Is it working on local, not on CI?

a0deploy --version                                         
## 8.14.0-beta.0

## Example
a0deploy import -c=config.json --input_file=local --dry_run 

Thanks again!

[alvaroloes]

It doesn't work locally, haven't tried it yet in CI. This is how I would run it:

~/projects/aifindr-auth0 git:[main]
a0deploy --version                                         
8.14.0-beta.0
~/projects/aifindr-auth0 git:[main]
a0deploy import --dry-run -c=tenants/saas-non-prod.json --input_file=tenant-template/tenant.yaml

It doesn't work, so I run it like this:

~/projects/aifindr-auth0 git:[main]
AUTH0_DRY_RUN=true a0deploy import -c=tenants/saas-non-prod.json --input_file=tenant-template/tenant.yaml

[kushalshit27]

This is strange, 🤔 just ran it:

[alvaroloes]

Ohh! I was getting nuts until I noticed. I was using the flag --dry-run, but it is --dry_run (with underscore).

Ok, my bad. Thanks for your time. I'll keep testing this.

So far, I have noticed some inconsistencies in the JSON diff when I add a new App directly in the tenant.yml file. It seems it mess with other apps.
I'll get more details and let you know

[simonpain]

Can I add a 2nd vote for the non-interactive dry-run option for CI. Happy to raise it as a separate comment if that help. Many Thanks.

[alvaroloes]

Hey @kushalshit27 here is some more feedback about the JSON diff that is calculated. I have found what I consider incorrect behavioiur in two cases

First case: array order

1. I added a new permission to a specific rol in the tenant.yml file.
2. Run the dry-run, it told me the right changes, and then imported the tenant.
3. I run the dry-run again and, suddenly, it told me there were a lot of changes. Those changes are related because the order of the permissions in the remote is different than those in the tenant.yml file.
4. The only way I can get these changes to go away is if I put the permission in tenant.yml exactly in the same position as Auth0 decides to place it after the import.

Check the screenshot to see this clearly:

Ideal behavior for me: The diff in these arrays should not be order-dependant.

Second case: Adding new app

While the above situation seems a minor thing, it gets worse in other situations.

1. I created a new App (LLMFindr app") in a non-prod tenant and exported it.
2. Then I wanted to import it into the prod tenant. I run the dry-run, and then I saw changes that made me cancel the import.
   • The new app was going to be created. This is correct.
   • But then, I saw that another app was going to be updated. When I review the JSON changes, it seems that apps were mixed together. Here is the output of the dry-run:

Note: "AIFindr app" is an app that already existed. "LLMFindr app" is the new one:

Simulating deployment... The following changes are proposed:

| Resource        | Status   | Name / Identifier                |

|-----------------|----------|----------------------------------|
| Clients         | CREATE   | LLMFindr app                     |
|                 | UPDATE   | LLMFindr app                     |

As you see above. Everything seems right, but I don't understand why it creates the new app and also updates it.

  "clients": [
    "Value difference for [AIFindr app.name]: local:LLMFindr app vs remote:AIFindr app",
    "Array length difference for [AIFindr app.allowed_logout_urls]: local:1 vs remote:13",
    "Array length difference for [AIFindr app.callbacks]: local:1 vs remote:25",
    "Array length difference for [AIFindr app.web_origins]: local:1 vs remote:9"
  ],

However, when I saw the JSON details, you can see that it is trying to update the old app with the values of the new app.

I'm not sure if this is related to the order again, but It might be. However, this time I didn't find a workaround to avoid it.

Maybe this is just a dry-run glitch and the actual import will work without problems, but I don't want to risk it, especially because I would be deploying to prod (and I haven't had the time to create another tenant to test this)

I hope all this helps to keep improving the tool. Thanks!