SAM build with project references above the Lambda project (NativeAOT .NET 8)

[kyl3t]

Hi,
I have a question regarding running sam build on a .NET 8 Serverless NativeAOT project with project references referencing an adjacent directory.

My project structure is as below, building worked fine when building only the Cwoc.Lambda project but ever since including shared projects from the shared folder, the docker run command that is eventually called is not able to locate these files as they aren't mounted.

Please let me know of any workarounds or options to address this.
Thanks

.
└── src/
    ├── apps/
    │   └── Cwoc/
    │       ├── Cwoc.sln
    │       └── Cwoc.Lambda/
    │           ├── serverless.yaml
    │           └── ...
    └── shared/
        ├── CA.C7.API
        ├── CA.Email
        └── ...

sam build --template ./apps/Cwoc/Cwoc.Lambda/serverless.yaml
Building codeuri: /Users/kt/Source/custom/src/apps/Cwoc/Cwoc.Lambda runtime: dotnet8 metadata: {'Tool': 'Amazon.Lambda.Annotations'} architecture: arm64 functions:                                              
CwocLambdaFunctionsOrderTxHandlerGenerated, CwocLambdaFunctionsOrderRxHandlerGenerated, CwocLambdaFunctionsStockSyncHandlerGenerated                                                                  
Workflow DotnetCliPackageBuilder does not support value "False" for building in source. Using default value "True".                                                                                                                                                     
 Running DotnetCliPackageBuilder:GlobalToolInstall                                                                                                                                                                                                                      
Tool 'amazon.lambda.tools' was reinstalled with the stable version (version '5.10.7').                                                                                                                                                                                  
 Running DotnetCliPackageBuilder:RunPackageAction                                                                                                                                                                                                                       
Amazon Lambda Tools for .NET Core applications (5.10.7)                                                                                                                                                                                                                 
Project Home: https://github.com/aws/aws-extensions-for-dotnet-cli, https://github.com/aws/aws-lambda-dotnet                                                                                                                                                            
                                                                                                                                                                                                                                                                        
Executing publish command                                                                                                                                                                                                                                               
Starting container for native AOT build using build image: public.ecr.aws/sam/build-dotnet8:latest-arm64.                                                                                                                                                               
... invoking 'docker run --name tempLambdaBuildContainer-6e26ef82-2544-427d-83cd-b3d780edf5e1 --rm --volume "/Users/kt/Source/custom/src/apps/Cwoc":/tmp/source/ -i -u 501:20 -e DOTNET_CLI_HOME=/tmp/dotnet -e XDG_DATA_HOME=/tmp/xdg 
public.ecr.aws/sam/build-dotnet8:latest-arm64 dotnet publish "/tmp/source/Cwoc.Lambda" --output "/tmp/source/Cwoc.Lambda/bin/Release/net8.0/publish" --configuration "Release" --framework "net8.0" --runtime linux-arm64   
/p:GenerateRuntimeConfigurationFiles=true --self-contained True  /p:StripSymbols=true' from directory /Users/kt/Source/custom/src/apps/Cwoc                                                                                            
... docker run: An issue was encountered verifying workloads. For more information, run "dotnet workload update".                                                                                                                                                       
... docker run:   Determining projects to restore...                                                                                                                                                                                                                    
... docker run:   Skipping project "/shared/CA.C7.API/CA.C7.API.csproj" because it was not found.                                                                                                                                                   
... docker run:   Skipping project "/shared/CA.Protocols.SFTP/CA.Protocols.SFTP.csproj" because it was not found.                                                                                                                                       
... docker run:   Skipping project "/shared/CA.C7.API/CA.C7.API.csproj" because it was not found.                                                                                                                                                   
... docker run:   Skipping project "/shared/CA.Protocols.SFTP/CA.Protocols.SFTP.csproj" because it was not found.                                                                                                                                       
... docker run:   Restored /tmp/source/Cwoc/Cwoc.csproj (in 4.07 sec).                                                                                                                                                      
... docker run:   Restored /tmp/source/Cwoc.Lambda/Cwoc.Lambda.csproj (in 8.59 sec).                                                                                                                                        
... docker run: /var/lang/bin/sdk/8.0.303/Microsoft.Common.CurrentVersion.targets(2165,5): warning : The referenced project '../../../shared/CA.C7.API/CA.C7.API.csproj' does not exist.                                                            
[/tmp/source/Cwoc/Cwoc.csproj]                                                                                                                                                                                              
... docker run: /var/lang/bin/sdk/8.0.303/Microsoft.Common.CurrentVersion.targets(2165,5): warning : The referenced project '../../../shared/CA.Protocols.SFTP/CA.Protocols.SFTP.csproj' does not exist.                                                
[/tmp/source/Cwoc/Cwoc.csproj]                                                                                                                                                                                              
... docker run: /tmp/source/Cwoc/Di.cs(4,7): error CS0246: The type or namespace name 'CA' could not be found (are you missing a using directive or an assembly reference?)                                                               
[/tmp/source/Cwoc/Cwoc.csproj]                                                                                                                                                                                              
... docker run: /tmp/source/Cwoc/Di.cs(5,7): error CS0246: The type or namespace name 'CA' could not be found (are you missing a using directive or an assembly reference?)                                                               
[/tmp/source/Cwoc/Cwoc.csproj]                                                                                                                                                                                              
... docker run: /tmp/source/Cwoc/Handlers/OrderRxHandler.cs(4,7): error CS0246: The type or namespace name 'CA' could not be found (are you missing a using directive or an assembly reference?)                                          
[/tmp/source/Cwoc/Cwoc.csproj]                                                                                                                                                                                              
... docker run: /tmp/source/Cwoc/Handlers/OrderRxHandler.cs(5,7): error CS0246: The type or namespace name 'CA' could not be found (are you missing a using directive or an assembly reference?)                                          
[/tmp/source/Cwoc/Cwoc.csproj]                                                                                                                                                                                              
... docker run: /tmp/source/Cwoc/Handlers/OrderRxHandler.cs(6,7): error CS0246: The type or namespace name 'CA' could not be found (are you missing a using directive or an assembly reference?)                                          
[/tmp/source/Cwoc/Cwoc.csproj]                                                                                                                                                                                              
... docker run: /tmp/source/Cwoc/Handlers/OrderTxHandler.cs(3,7): error CS0246: The type or namespace name 'CA' could not be found (are you missing a using directive or an assembly reference?)                                          
[/tmp/source/Cwoc/Cwoc.csproj]                                                                                                                                                                                              
...                          
[/tmp/source/Cwoc/Cwoc.csproj]                                                                                                                                                                                              
ERROR: Container build returned 1                                                                                                                                                                                                                                       

Build Failed
Error: DotnetCliPackageBuild

[Beau-Gosse-dev]

Under the hood, sam build uses dotnet lambda publish as shown here: https://github.com/aws/aws-lambda-builders/blob/beac4185ed6318584e14de569ded9c842c9b91b4/aws_lambda_builders/workflows/dotnet_clipackage/actions.py#L105

dotnet lambda publish will use the location of your .sln file to mount to docker as shown here:

aws-extensions-for-dotnet-cli/src/Amazon.Lambda.Tools/LambdaPackager.cs

Line 150 in 7918ffc

var directoryToMountToContainer = Utilities.GetSolutionDirectoryFullPath(workingDirectory, projectLocation, codeMountDirectory);

Unfortunately, that isn't a perfect solution, since as shown in your case, the .sln file isn't in a folder that contains all other projects.

You can run dotnet lambda package --help to see all the extra parameters that dotnet lambda package can take.

One of those is --code-mount-directory | Path to the directory to mount to the build container. Otherwise, look upward for a solution folder. You can use this to tell it to mount src/ instead, but only if you're using dotnet lambda directly.

However, you're using sam build, so I don't think you'll be able to pass --code-mount-directory into sam since it won't know what that is.

You could try moving your .sln file to a folder with all projects under it, but that might not be feasible. Or you could try using dotnet lambda directly for this Lambda, instead of sam, but that is probably even less feasible.

I see you already opened an issue with SAM, which is good. Hopefully we can prioritize either passing in extra args for .NET or set the code mount directory there. Maybe this other proposal would solve this: aws/aws-sam-cli#5546

[Beau-Gosse-dev]

You could also try some combination of --use-container --build-in-source --base-dir directly into the SAM CLI, but I don't remember that working for this use-case in the past.

[kyl3t]

Thanks Beau, that's really useful and good to know that there isn't any obvious solutions at this stage.
I was able to rework the folder structure of the project to make this work.

.
└── src/
    ├── shared/
    │   ├── SharedLibrary1
    │   └── SharedLibrary2
    ├── App1.Lambda/
    │   ├── serverless.yaml
    │   └── ...
    ├── App2.Lambda/
    │   ├── serverless.yaml
    │   └── ...
    ├── App1
    ├── App2
    ├── App1.sln
    └── App2.sln

This way I can run sam build --template ./App1.Lambda/serverless.yaml with src as the working directory which builds fine.
I will definitely have more use cases in the future where the shared libraries need to live further away from the lambda projects so will be keeping an eye on the solution you suggested.

Appreciate the snappy and informative response here!

[normj]

Some extra info for you. The back door for getting parameters set to the dotnet lambda package is to have a aws-lambda-tools-defaults.json file in the directory where the Lambda function is at. In there you can specify command line parameters like this.

{
  "code-mount-directory": "../../../"
}

[kyl3t]

Thanks @normj - that worked perfectly locally and on the GitHub action with the initial folder structure.

[github-actions[bot]]

Hello! Reopening this discussion to make it searchable.