aws sso login Error in creating session object : profile "default" is configured to use SSO but is missing required configuration: sso_region, sso_start_url

[ptnutrien]

2023-06-01T20:14:16Z [Info] Initializing AWS X-Ray daemon 3.3.7
2023-06-01T20:14:16Z [Debug] Listening on UDP xray-daemon:2000
2023-06-01T20:14:16Z [Info] Using buffer memory limit of 78 MB
2023-06-01T20:14:16Z [Info] 1248 segment buffers allocated
2023-06-01T20:14:16Z [Debug] Using proxy address:
2023-06-01T20:14:17Z [Error] Error in creating session object : profile "default" is configured to use SSO but is missing required configuration: sso_region, sso_start_url

services:
xray-daemon:
command:
- '-o'
- '--log-level=debug'
- '--bind=xray-daemon:2000'
build:
context: ./
dockerfile: Dockerfile.xray
volumes:
- ~/.aws/:/root/.aws/

FROM amazonlinux
RUN yum install -y unzip
RUN curl -o daemon.zip https://s3.us-east-2.amazonaws.com/aws-xray-assets.us-east-2/xray-daemon/aws-xray-daemon-linux-3.x.zip
RUN unzip daemon.zip && cp xray /usr/bin/xray
ENTRYPOINT ["/usr/bin/xray", "-t", "0.0.0.0:2000", "-b", "0.0.0.0:2000"]
EXPOSE 2000/udp
EXPOSE 2000/tcp

[ptnutrien]

I was able to fix the first error

services:
xray-daemon:
command: ["-o", '--log-level=debug']
image: amazon/aws-xray-daemon
volumes:
- ~/.aws/:/root/.aws/
environment:
- AWS_REGION=us-east-2

but now it is giving other errors, its like it doesn't use the sso config

Debug] processor: segment batch size: 2. capacity: 50
xray-daemon-1 | 2023-06-02T16:17:16Z [Error] Unable to sign request: NoCredentialProviders: no valid providers in chain. Deprecated.
xray-daemon-1 | For verbose messaging see aws.Config.CredentialsChainVerboseErrors

[ptnutrien]

I am creating an xray daemon like this

xray-daemon:
command: ["-o", '--log-level=dev']
image: amazon/aws-xray-daemon
volumes:
- ~/.aws/:/root/.aws/
environment:
- AWS_REGION=us-east-2
- AWS_PROFILE=default
- AWS_SDK_LOAD_CONFIG=1

I added region because I was getting an error even though region is specified in my config, I am using
aws configure sso
to generate config file
[default]
sso_session = newbie
sso_account_id = ACCOUNT_ID
sso_role_name = the_role
region = us-east-2
output = json
[sso-session newbie]
sso_start_url = obfuscated
sso_region = us-east-1
sso_registration_scopes = sso:account:access

However, I keep getting this error
[Error] Unable to sign request: NoCredentialProviders: no valid providers in chain. Deprecated.
xray-daemon-1 | For verbose messaging see aws.Config.CredentialsChainVerboseErrors

after compiliing locally and running I get this
./xray -v
AWS X-Ray daemon version:
xray-mac-arm64 % ./xray -o -n us-east-2
2023-06-02T13:40:20-07:00 [Info] Initializing AWS X-Ray daemon
2023-06-02T13:40:20-07:00 [Info] Using buffer memory limit of 163 MB
2023-06-02T13:40:20-07:00 [Info] 2608 segment buffers allocated
2023-06-02T13:40:20-07:00 [Error] Error in creating session object : profile "default" is configured to use SSO but is missing required configuration: sso_region, sso_start_url

But as you can see above these values are set

this pr should have fixed this?
https://github.com/aws/aws-xray-daemon/issues/147

I have verified the aws sdk go from this link aws/aws-sdk-go#4054 (comment)

Version 1.44.275
Trying V1....
COULD NOT DO V1 profile "default" is configured to use SSO but is missing required configuration: sso_region, sso_start_url

Trying V2....
Account: ACCOUNT_ID
UserID:USER
ARN: arn:aws:sts::

[wangzlei]

Checking doc AWS SSO Support in the AWS SDK for Go v1

    session.Options{
        SharedConfigState: session.SharedConfigEnable, // Must be set to enable
        Profile:           "dev-profile",
    }

X-Ray Daemon code does contain the similar code SharedConfigEnable https://github.com/aws/aws-xray-daemon/blame/56bcdadc0e5808f4428ed6e3e54a88a2ceca2f82/pkg/conn/conn.go#L294

[ptnutrien]

How can I enable the session.SharedConfigEnable ?
I have - AWS_SDK_LOAD_CONFIG=1
I don't understand how to set the session.Options for this library to be able to use the SharedConfigState

I have reviewed the doc you shared as well as
https://docs.aws.amazon.com/xray/latest/devguide/xray-daemon-configuration.html
https://docs.aws.amazon.com/sdk-for-go/api/aws/session/

[quetzalcoatl]

I'm struggling with setting up a local instance of xra-daemon on my Windows developer machine.
A bit of a different issue, but similar symptoms, so I thought I'll pass you what I found.

The package I'm using is: https://s3.us-east-2.amazonaws.com/aws-xray-assets.us-east-2/xray-daemon/aws-xray-daemon-windows-process-3.x.zip

My set up is, basically, from scratch (remove all profiles from ~/.aws/config, remove ~/.aws/sso/cache completely):
(yes, I said I'm on Windows and now you see I'm using / and ~ -- that's because I'm using git-bash, not cmd.exe).

aws configure sso --no-browser --profile my-profile
#   it asks for data, I provide (data below is anonymized)
#session:    sess01
#sso url:     https://xxxxxxx.awsapps.com/start#/
#account:  123456789
#role:         MyDesiredRole
#region:     eu-west-1
#output:     yaml

#then, some time later
aws sso login --no-browser --profile my-profile
# it gives me a browser link, I open it, and accept

After that, I'm able to use aws cli etc.
However, when I run xray-process-3.x, it provides the same error as reported here:

> export AWS_PROFILE=my-profile
> ./xray_windows.exe -l dev -o -n eu-west-1
2023-07-12T12:12:41+02:00 [Info] Initializing AWS X-Ray daemon 3.3.7
2023-07-12T12:12:41+02:00 [Debug] Listening on UDP 127.0.0.1:2000
2023-07-12T12:12:41+02:00 [Info] Using buffer memory limit of 324 MB
2023-07-12T12:12:41+02:00 [Info] 5184 segment buffers allocated
2023-07-12T12:12:41+02:00 [Debug] Using proxy address:
2023-07-12T12:12:41+02:00 [Debug] Fetch region eu-west-1 from commandline/config file
2023-07-12T12:12:41+02:00 [Error] Error in creating session object : profile "my-profile" is configured to use SSO but is missing required configuration: sso_region, sso_start_url

I've got an idea, and I edited the ~/.aws/config and literally copied sso_region, sso_start_url sections from sso-session sess01 into profile my-profile:

[default]
region = eu-north-1
[profile my-profile]
sso_session = sess01
sso_account_id = 1234567890
sso_role_name = MyDesiredRole
region = eu-west-1
output = yaml
sso_start_url = https://xxxxxxx.awsapps.com/start#/
sso_region = eu-west-1
[sso-session sess01]
sso_start_url = https://xxxxxxx.awsapps.com/start#/
sso_region = eu-west-1
sso_registration_scopes = sso:account:access

and this change allowed the xray-windows.exe to start!

> export AWS_PROFILE=my-profile
> ./xray_windows.exe -l dev -o -n eu-west-1
xray_windows.exe -o -n eu-west-1
2023-07-12T12:19:04+02:00 [Info] Initializing AWS X-Ray daemon 3.3.7
2023-07-12T12:19:04+02:00 [Info] Using buffer memory limit of 324 MB
2023-07-12T12:19:04+02:00 [Info] 5184 segment buffers allocated
2023-07-12T12:19:04+02:00 [Info] Using region: eu-west-1
2023-07-12T12:19:04+02:00 [Info] HTTP Proxy server using X-Ray Endpoint : https://xray.eu-west-1.amazonaws.com
2023-07-12T12:19:04+02:00 [Info] Starting proxy http server on 127.0.0.1:2000

After I got this, I accidentaly found this similar report elsewhere, and the person there did almost exactly the same: gruntwork-io/terragrunt#2604 (comment)
This post may be important. It mentions that the tools are picky about the SSO URL, and that they had to keep an eye on #/ at the end of it.

However, that's not the end of it, at least for me.
When my application actually did something and tried sending traces/segments to this now-finally-running xray-daemon, the daemon said:

2023-07-12T12:19:06+02:00 [Error] Sending segment batch failed with: SSOProviderInvalidToken: the SSO session has expired or is invalid
caused by: open C:\Users\xxxxxxx\.aws\sso\cache\6b818xxxxxxxxx.json: The system cannot find the file specified.

But on my computer, I had different files there:

$ ls sso/cache/
4915fxxxxxxxxxxxxxx.json  87bbbxxxxxxxxxxxx.json

I played a bit, and discovered that the hash-name of the file wanted by xray-daemon is related to the sso url from the profile (not the sso-session) from the ~/.aws/config file.

    https://xxxxxx.awsapps.com/start#/    -> looks for: 6b818xxxxxx.json
    https://xxxxxx.awsapps.com/start      -> looks for: b382dxxxxxx.json
    https://xxxxxx.awsapps.com/start#     -> looks for: 03350xxxxxx.json

So, for some unknown reason, the xray-daemon reads the config properly (well, almost, it ignores sso-session, and gets all properties from parent profile), and then it generates a hashname for the credentials file, but generates it differently than the aws sso login did.

I tried playing around with copying files. I had those 4915fxxxxxxxxxxxxxx.json 87bbbxxxxxxxxxxxx.json files, 4915 didn't work, but 87bb worked:

> cp ~/.aws/sso/cache/87bbbxxxxxxxxx.json ~/.aws/sso/cache/6b818xxxxxxxxx.json
> export AWS_PROFILE=my-profile
> xray_windows.exe -l dev -o -n eu-west-1
2023-07-12T12:52:33+02:00 [Info] Initializing AWS X-Ray daemon 3.3.7
2023-07-12T12:52:33+02:00 [Debug] Listening on UDP 127.0.0.1:2000
2023-07-12T12:52:33+02:00 [Info] Using buffer memory limit of 324 MB
2023-07-12T12:52:33+02:00 [Info] 5184 segment buffers allocated
2023-07-12T12:52:33+02:00 [Debug] Using proxy address:
2023-07-12T12:52:33+02:00 [Debug] Fetch region eu-west-1 from commandline/config file
2023-07-12T12:52:33+02:00 [Info] Using region: eu-west-1
2023-07-12T12:52:33+02:00 [Debug] ARN of the AWS resource running the daemon:
2023-07-12T12:52:33+02:00 [Debug] No hostname set for telemetry records
2023-07-12T12:52:33+02:00 [Debug] No Instance Id set for telemetry records
2023-07-12T12:52:33+02:00 [Debug] Using Endpoint: https://xray.eu-west-1.amazonaws.com
2023-07-12T12:52:33+02:00 [Debug] Telemetry initiated
2023-07-12T12:52:33+02:00 [Info] HTTP Proxy server using X-Ray Endpoint : https://xray.eu-west-1.amazonaws.com
2023-07-12T12:52:33+02:00 [Debug] Using Endpoint: https://xray.eu-west-1.amazonaws.com
2023-07-12T12:52:33+02:00 [Debug] Batch size: 50
2023-07-12T12:52:33+02:00 [Info] Starting proxy http server on 127.0.0.1:2000
2023-07-12T12:52:37+02:00 [Debug] processor: sending partial batch
2023-07-12T12:52:37+02:00 [Debug] processor: segment batch size: 1. capacity: 50
2023-07-12T12:52:38+02:00 [Info] Successfully sent batch of 1 segments (0.772 seconds)
2023-07-12T12:53:33+02:00 [Debug] Send 1 telemetry record(s)

So, it works now. Yay!
It may look simple here, because it's trimmed down to the point, but it took me a lot of hours over several sessions fo trying to get it working (including running a linux version in a docker container, or trying the window-service version, etc -- all had the same initial behavior of ignoring the sso-session and only looking at the profile).

It seems the solution is 100% repeatable. If I observe the logs of xray-daemon and read from the error line what was the hash-name.json of the expected credential file, and I can copy the correct credential file so that expected file exists, and it works beautifuly.

But - I had 2 credential files. Here's how they looked like (anonymized ofc, and FORMATTED):

4915fxxxxxxxxxxxxxx.json:
{
  "clientId": "xxxxx some hash or token",
  "clientSecret": "xxxxx typical JWT token",
  "expiresAt": "2023-10-10T10:13:56Z",
  "scopes": ["sso:account:access"]
}

87bbbxxxxxxxxxxxx.json:
{
  "startUrl": "https://xxxxxx.awsapps.com/start#/",
  "region": "eu-west-1",
  "accessToken": "xxxxxx some hash or token",
  "expiresAt": "2023-07-12T11:14:06Z",
  "clientId": "xxxx some hash or token, THE SAME as in the other file",
  "clientSecret": "xxxxx typical JWT token, THE SAME as in the other file",
  "registrationExpiresAt": "2023-10-10T10:13:56Z",   ## note that it is the same as "expiresAt" in the other file
  "refreshToken": "xxxx some hash or token"
}

The xray daemon wanted the second file. That one having attributes like registrationExpiresAt or refreshToken. Judging by names and expiry dates, I guess that the first file are the 'profile registration' and that the second one (wanted by xray daemon) is 'current session', or something like that.

So, the full workaround was, for me:

• edit ~/.aws/config and copy sso_start_url and sso_region from [sso-session] into its related [profile]
• remember to set AWS_PROFILE to have the profile name
• run the xray daemon, wait a bit for some activity
• read out the logs from xray daemon
• if it complains about SSOProviderInvalidToken: .... caused by: open SOMEPATH\.aws\sso\cache\FILENAME.json
• go to that folder SOMEPATH\.aws\sso\cache
• look for a JSON file that contains registrationExpiresAt and refreshToken
• copy that file into new name of FILENAME.json
• no need to even restart xray daemon, it should notice the new file immediately

HTH