Skip to content

Commit 4db924c

Browse files
dylan-smithdylan-smith
authored
Validate bbs2gh env vars in the generated script (#923)
<!-- For the checkboxes below you must check each one to indicate that you either did the relevant task, or considered it and decided there was nothing that needed doing --> - [x] Did you write/update appropriate tests - [x] Release notes updated (if appropriate) - [x] Appropriate logging output - [x] Issue linked - [x] Docs updated (or issue created) - [x] New package licenses are added to `ThirdPartyNotices.txt` (if applicable) Implements part of #629 Added some powershell to the start of the script generated by `bbs2gh generate-script` that validates the proper environment variables are set before executing the rest of the script. This one was a bit trickier than `ado2gh` because depending on what args you pass to `generate-script` the script will expect different sets of env vars. This was the pseudocode I came up with before coding this up: ``` always validate GH_PAT always validate BBS_PASSWORD unless args.Kerberos if args.BbsUsername.IsNullOrWhitespace() and !args.Kerberos validate BBS_USERNAME if args.AwsBucketName.HasValue() or args.AwsRegion.HasValue() Validate AWS_ACCESS_KEY_ID Validate AWS_SECRET_ACCESS_KEY if AwsBucketName.IsNull and AwsRegion.IsNull Validate AZURE_STORAGE_CONNECTION_STRING if SmbUser.HasValue Validate SMB_PASSWORD ``` These are all the various validation messages: ``` if (-not $env:GH_PAT) { Write-Error "GH_PAT environment variable must be set to a valid GitHub Personal Access Token with the appropriate scopes. For more information see https://docs.github.com/en/migrations/using-github-enterprise-importer/preparing-to-migrate-with-github-enterprise-importer/managing-access-for-github-enterprise-importer#creating-a-personal-access-token-for-github-enterprise-importer" exit 1 } else { Write-Host "GH_PAT environment variable is set and will be used to authenticate to GitHub." } if (-not $env:BBS_USERNAME) { Write-Error "BBS_USERNAME environment variable must be set to a valid user that will be used to call BBS API's to generate a migration archive." exit 1 } else { Write-Host "BBS_USERNAME environment variable is set and will be used to authenticate to BBS APIs." } if (-not $env:BBS_PASSWORD) { Write-Error "BBS_PASSWORD environment variable must be set to a valid password that will be used to call BBS API's to generate a migration archive." exit 1 } else { Write-Host "BBS_PASSWORD environment variable is set and will be used to authenticate to BBS APIs." } if (-not $env:AZURE_STORAGE_CONNECTION_STRING) { Write-Error "AZURE_STORAGE_CONNECTION_STRING environment variable must be set to a valid Azure Storage Connection String that will be used to upload the migration archive to Azure Blob Storage." exit 1 } else { Write-Host "AZURE_STORAGE_CONNECTION_STRING environment variable is set and will be used to upload the migration archive to Azure Blob Storage." } if (-not $env:AWS_ACCESS_KEY_ID) { Write-Error "AWS_ACCESS_KEY_ID environment variable must be set to a valid AWS Access Key ID that will be used to upload the migration archive to AWS S3." exit 1 } else { Write-Host "AWS_ACCESS_KEY_ID environment variable is set and will be used to upload the migration archive to AWS S3." } if (-not $env:AWS_SECRET_ACCESS_KEY) { Write-Error "AWS_SECRET_ACCESS_KEY environment variable must be set to a valid AWS Secret Access Key that will be used to upload the migration archive to AWS S3." exit 1 } else { Write-Host "AWS_SECRET_ACCESS_KEY environment variable is set and will be used to upload the migration archive to AWS S3." } if (-not $env:SMB_PASSWORD) { Write-Error "SMB_PASSWORD environment variable must be set to a valid password that will be used to download the migration archive from your BBS server using SMB." exit 1 } else { Write-Host "SMB_PASSWORD environment variable is set and will be used to download the migration archive from your BBS server using SMB." } ``` Note: I updated this PR to target another PR branch (instead of main) so the diff is correct for reviewers. Will retarget it to main once that other PR merges. <!-- For docs we should review the docs at: https://docs.github.com/en/early-access/github/migrating-with-github-enterprise-importer and the README.md in this repo If a doc update is required based on the changes in this PR, it is sufficient to create an issue and link to it here. The doc update can be made later/separately. The process to update the docs can be found here: https://github.com/github/docs-early-access#opening-prs The markdown files are here: https://github.com/github/docs-early-access/tree/main/content/github/migrating-with-github-enterprise-importer -->
1 parent 37cef2a commit 4db924c

File tree

3 files changed

+281
-4
lines changed

3 files changed

+281
-4
lines changed

‎RELEASENOTES.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,5 @@
22
- Update `gh bbs2gh generate-script` so it supports more than 25 projects/repos
33
- Rename `--bbs-project-key` to `--bbs-project` in `gh bbs2gh generate-script` for consistency
44
- Fix a bug where `create-team` might not work due to a race condition
5-
- When using `gh ado2gh generate-script` the script will now validate that the necessary environment variables are set
5+
- When using `gh ado2gh generate-script` or `gh bbs2gh generate-script` the script will now validate that the necessary environment variables are set
66
- Make API calls to GitHub.com that require pagination more resilient by retrying on HTTP failures

‎src/OctoshiftCLI.Tests/bbs2gh/Handlers/GenerateScriptCommandHandlerTests.cs‎

Lines changed: 208 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -82,7 +82,211 @@ public async Task No_Projects()
8282
await _handler.Handle(args);
8383

8484
// Assert
85-
_mockFileSystemProvider.Verify(m => m.WriteAllTextAsync(It.IsAny<string>(), It.Is<string>(script => TrimNonExecutableLines(script, 9, 0) == "")));
85+
_mockFileSystemProvider.Verify(m => m.WriteAllTextAsync(It.IsAny<string>(), It.Is<string>(script => TrimNonExecutableLines(script, 33, 0) == "")));
86+
}
87+
88+
[Fact]
89+
public async Task Validates_Env_Vars()
90+
{
91+
// Arrange
92+
_mockBbsApi.Setup(m => m.GetProjects()).ReturnsAsync(Enumerable.Empty<(int Id, string Key, string Name)>());
93+
94+
// Act
95+
var args = new GenerateScriptCommandArgs()
96+
{
97+
BbsServerUrl = BBS_SERVER_URL,
98+
GithubOrg = GITHUB_ORG,
99+
SshUser = SSH_USER,
100+
SshPrivateKey = SSH_PRIVATE_KEY,
101+
Output = new FileInfo(OUTPUT),
102+
};
103+
await _handler.Handle(args);
104+
105+
var expected = @"
106+
if (-not $env:GH_PAT) {
107+
Write-Error ""GH_PAT environment variable must be set to a valid GitHub Personal Access Token with the appropriate scopes. For more information see https://docs.github.com/en/migrations/using-github-enterprise-importer/preparing-to-migrate-with-github-enterprise-importer/managing-access-for-github-enterprise-importer#creating-a-personal-access-token-for-github-enterprise-importer""
108+
exit 1
109+
} else {
110+
Write-Host ""GH_PAT environment variable is set and will be used to authenticate to GitHub.""
111+
}
112+
113+
if (-not $env:BBS_PASSWORD) {
114+
Write-Error ""BBS_PASSWORD environment variable must be set to a valid password that will be used to call Bitbucket Server/Data Center API's to generate a migration archive.""
115+
exit 1
116+
} else {
117+
Write-Host ""BBS_PASSWORD environment variable is set and will be used to authenticate to Bitbucket Server/Data Center APIs.""
118+
}
119+
120+
if (-not $env:BBS_USERNAME) {
121+
Write-Error ""BBS_USERNAME environment variable must be set to a valid user that will be used to call Bitbucket Server/Data Center API's to generate a migration archive.""
122+
exit 1
123+
} else {
124+
Write-Host ""BBS_USERNAME environment variable is set and will be used to authenticate to Bitbucket Server/Data Center APIs.""
125+
}
126+
127+
if (-not $env:AZURE_STORAGE_CONNECTION_STRING) {
128+
Write-Error ""AZURE_STORAGE_CONNECTION_STRING environment variable must be set to a valid Azure Storage Connection String that will be used to upload the migration archive to Azure Blob Storage.""
129+
exit 1
130+
} else {
131+
Write-Host ""AZURE_STORAGE_CONNECTION_STRING environment variable is set and will be used to upload the migration archive to Azure Blob Storage.""
132+
}";
133+
134+
// Assert
135+
_mockFileSystemProvider.Verify(m => m.WriteAllTextAsync(It.IsAny<string>(), It.Is<string>(script => TrimNonExecutableLines(script, 9, 0) == TrimNonExecutableLines(expected, 0, 0))));
136+
}
137+
138+
[Fact]
139+
public async Task Validates_Env_Vars_BBS_USERNAME_Not_Validated_When_Passed_As_Arg()
140+
{
141+
// Arrange
142+
_mockBbsApi.Setup(m => m.GetProjects()).ReturnsAsync(Enumerable.Empty<(int Id, string Key, string Name)>());
143+
144+
// Act
145+
var args = new GenerateScriptCommandArgs()
146+
{
147+
BbsServerUrl = BBS_SERVER_URL,
148+
GithubOrg = GITHUB_ORG,
149+
SshUser = SSH_USER,
150+
SshPrivateKey = SSH_PRIVATE_KEY,
151+
Output = new FileInfo(OUTPUT),
152+
BbsUsername = BBS_USERNAME,
153+
};
154+
await _handler.Handle(args);
155+
156+
var expected = @"
157+
if (-not $env:BBS_USERNAME) {
158+
Write-Error ""BBS_USERNAME environment variable must be set to a valid user that will be used to call BBS API's to generate a migration archive.""
159+
exit 1
160+
} else {
161+
Write-Host ""BBS_USERNAME environment variable is set and will be used to authenticate to BBS APIs.""
162+
}";
163+
164+
// Assert
165+
_mockFileSystemProvider.Verify(m => m.WriteAllTextAsync(It.IsAny<string>(), It.Is<string>(script => !TrimNonExecutableLines(script, 0, 0).Contains(TrimNonExecutableLines(expected, 0, 0)))));
166+
}
167+
168+
[Fact]
169+
public async Task Validates_Env_Vars_BBS_PASSWORD_Not_Validated_When_Kerberos()
170+
{
171+
// Arrange
172+
_mockBbsApi.Setup(m => m.GetProjects()).ReturnsAsync(Enumerable.Empty<(int Id, string Key, string Name)>());
173+
174+
// Act
175+
var args = new GenerateScriptCommandArgs()
176+
{
177+
BbsServerUrl = BBS_SERVER_URL,
178+
GithubOrg = GITHUB_ORG,
179+
SshUser = SSH_USER,
180+
SshPrivateKey = SSH_PRIVATE_KEY,
181+
Output = new FileInfo(OUTPUT),
182+
Kerberos = true,
183+
};
184+
await _handler.Handle(args);
185+
186+
var expected = @"
187+
if (-not $env:BBS_PASSWORD) {
188+
Write-Error ""BBS_PASSWORD environment variable must be set to a valid password that will be used to call BBS API's to generate a migration archive.""
189+
exit 1
190+
} else {
191+
Write-Host ""BBS_PASSWORD environment variable is set and will be used to authenticate to BBS APIs.""
192+
}";
193+
194+
// Assert
195+
_mockFileSystemProvider.Verify(m => m.WriteAllTextAsync(It.IsAny<string>(), It.Is<string>(script => !TrimNonExecutableLines(script, 0, 0).Contains(TrimNonExecutableLines(expected, 0, 0)))));
196+
}
197+
198+
[Fact]
199+
public async Task Validates_Env_Vars_AWS()
200+
{
201+
// Arrange
202+
_mockBbsApi.Setup(m => m.GetProjects()).ReturnsAsync(Enumerable.Empty<(int Id, string Key, string Name)>());
203+
204+
// Act
205+
var args = new GenerateScriptCommandArgs()
206+
{
207+
BbsServerUrl = BBS_SERVER_URL,
208+
GithubOrg = GITHUB_ORG,
209+
SshUser = SSH_USER,
210+
SshPrivateKey = SSH_PRIVATE_KEY,
211+
AwsBucketName = AWS_BUCKET_NAME,
212+
Output = new FileInfo(OUTPUT),
213+
};
214+
await _handler.Handle(args);
215+
216+
var expected = @"
217+
if (-not $env:AWS_ACCESS_KEY_ID) {
218+
Write-Error ""AWS_ACCESS_KEY_ID environment variable must be set to a valid AWS Access Key ID that will be used to upload the migration archive to AWS S3.""
219+
exit 1
220+
} else {
221+
Write-Host ""AWS_ACCESS_KEY_ID environment variable is set and will be used to upload the migration archive to AWS S3.""
222+
}
223+
if (-not $env:AWS_SECRET_ACCESS_KEY) {
224+
Write-Error ""AWS_SECRET_ACCESS_KEY environment variable must be set to a valid AWS Secret Access Key that will be used to upload the migration archive to AWS S3.""
225+
exit 1
226+
} else {
227+
Write-Host ""AWS_SECRET_ACCESS_KEY environment variable is set and will be used to upload the migration archive to AWS S3.""
228+
}";
229+
230+
// Assert
231+
_mockFileSystemProvider.Verify(m => m.WriteAllTextAsync(It.IsAny<string>(), It.Is<string>(script => TrimNonExecutableLines(script, 0, 0).Contains(TrimNonExecutableLines(expected, 0, 0)))));
232+
}
233+
234+
[Fact]
235+
public async Task Validates_Env_Vars_AZURE_STORAGE_CONNECTION_STRING_Not_Validated_When_Aws()
236+
{
237+
// Arrange
238+
_mockBbsApi.Setup(m => m.GetProjects()).ReturnsAsync(Enumerable.Empty<(int Id, string Key, string Name)>());
239+
240+
// Act
241+
var args = new GenerateScriptCommandArgs()
242+
{
243+
BbsServerUrl = BBS_SERVER_URL,
244+
GithubOrg = GITHUB_ORG,
245+
SshUser = SSH_USER,
246+
SshPrivateKey = SSH_PRIVATE_KEY,
247+
Output = new FileInfo(OUTPUT),
248+
AwsBucketName = AWS_BUCKET_NAME,
249+
};
250+
await _handler.Handle(args);
251+
252+
var expected = @"
253+
if (-not $env:AZURE_STORAGE_CONNECTION_STRING) {
254+
Write-Error ""AZURE_STORAGE_CONNECTION_STRING environment variable must be set to a valid Azure Storage Connection String that will be used to upload the migration archive to Azure Blob Storage.""
255+
exit 1
256+
} else {
257+
Write-Host ""AZURE_STORAGE_CONNECTION_STRING environment variable is set and will be used to upload the migration archive to Azure Blob Storage.""
258+
}";
259+
260+
// Assert
261+
_mockFileSystemProvider.Verify(m => m.WriteAllTextAsync(It.IsAny<string>(), It.Is<string>(script => !TrimNonExecutableLines(script, 0, 0).Contains(TrimNonExecutableLines(expected, 0, 0)))));
262+
}
263+
264+
[Fact]
265+
public async Task Validates_Env_Vars_SMB_PASSWORD()
266+
{
267+
// Arrange
268+
_mockBbsApi.Setup(m => m.GetProjects()).ReturnsAsync(Enumerable.Empty<(int Id, string Key, string Name)>());
269+
270+
// Act
271+
var args = new GenerateScriptCommandArgs()
272+
{
273+
BbsServerUrl = BBS_SERVER_URL,
274+
GithubOrg = GITHUB_ORG,
275+
Output = new FileInfo(OUTPUT),
276+
SmbUser = SMB_USER,
277+
};
278+
await _handler.Handle(args);
279+
280+
var expected = @"
281+
if (-not $env:SMB_PASSWORD) {
282+
Write-Error ""SMB_PASSWORD environment variable must be set to a valid password that will be used to download the migration archive from your BBS server using SMB.""
283+
exit 1
284+
} else {
285+
Write-Host ""SMB_PASSWORD environment variable is set and will be used to download the migration archive from your BBS server using SMB.""
286+
}";
287+
288+
// Assert
289+
_mockFileSystemProvider.Verify(m => m.WriteAllTextAsync(It.IsAny<string>(), It.Is<string>(script => TrimNonExecutableLines(script, 0, 0).Contains(TrimNonExecutableLines(expected, 0, 0)))));
86290
}
87291

88292
[Fact]
@@ -102,7 +306,7 @@ public async Task No_Repos()
102306
await _handler.Handle(args);
103307

104308
// Assert
105-
_mockFileSystemProvider.Verify(m => m.WriteAllTextAsync(It.IsAny<string>(), It.Is<string>(script => TrimNonExecutableLines(script, 9, 0) == "")));
309+
_mockFileSystemProvider.Verify(m => m.WriteAllTextAsync(It.IsAny<string>(), It.Is<string>(script => TrimNonExecutableLines(script, 33, 0) == "")));
106310
}
107311

108312
[Fact]
@@ -488,6 +692,7 @@ private string TrimNonExecutableLines(string script, int skipFirst = 9, int skip
488692
.Skip(skipFirst)
489693
.SkipLast(skipLast);
490694

491-
return string.Join(Environment.NewLine, lines);
695+
var result = string.Join(Environment.NewLine, lines);
696+
return result;
492697
}
493698
}

‎src/bbs2gh/Handlers/GenerateScriptCommandHandler.cs‎

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,29 @@ private async Task<string> GenerateScript(GenerateScriptCommandArgs args)
6464
content.AppendLine(VersionComment);
6565
content.AppendLine(EXEC_FUNCTION_BLOCK);
6666

67+
content.AppendLine(VALIDATE_GH_PAT);
68+
if (!args.Kerberos)
69+
{
70+
content.AppendLine(VALIDATE_BBS_PASSWORD);
71+
}
72+
if (args.BbsUsername.IsNullOrWhiteSpace() && !args.Kerberos)
73+
{
74+
content.AppendLine(VALIDATE_BBS_USERNAME);
75+
}
76+
if (args.AwsBucketName.HasValue() || args.AwsRegion.HasValue())
77+
{
78+
content.AppendLine(VALIDATE_AWS_ACCESS_KEY_ID);
79+
content.AppendLine(VALIDATE_AWS_SECRET_ACCESS_KEY);
80+
}
81+
else
82+
{
83+
content.AppendLine(VALIDATE_AZURE_STORAGE_CONNECTION_STRING);
84+
}
85+
if (args.SmbUser.HasValue())
86+
{
87+
content.AppendLine(VALIDATE_SMB_PASSWORD);
88+
}
89+
6790
var projects = args.BbsProject.HasValue()
6891
? new List<string>() { args.BbsProject }
6992
: (await _bbsApi.GetProjects()).Select(x => x.Key);
@@ -232,5 +255,54 @@ function Exec {
232255
if ($lastexitcode -ne 0) {
233256
exit $lastexitcode
234257
}
258+
}";
259+
private const string VALIDATE_GH_PAT = @"
260+
if (-not $env:GH_PAT) {
261+
Write-Error ""GH_PAT environment variable must be set to a valid GitHub Personal Access Token with the appropriate scopes. For more information see https://docs.github.com/en/migrations/using-github-enterprise-importer/preparing-to-migrate-with-github-enterprise-importer/managing-access-for-github-enterprise-importer#creating-a-personal-access-token-for-github-enterprise-importer""
262+
exit 1
263+
} else {
264+
Write-Host ""GH_PAT environment variable is set and will be used to authenticate to GitHub.""
265+
}";
266+
private const string VALIDATE_BBS_USERNAME = @"
267+
if (-not $env:BBS_USERNAME) {
268+
Write-Error ""BBS_USERNAME environment variable must be set to a valid user that will be used to call Bitbucket Server/Data Center API's to generate a migration archive.""
269+
exit 1
270+
} else {
271+
Write-Host ""BBS_USERNAME environment variable is set and will be used to authenticate to Bitbucket Server/Data Center APIs.""
272+
}";
273+
private const string VALIDATE_BBS_PASSWORD = @"
274+
if (-not $env:BBS_PASSWORD) {
275+
Write-Error ""BBS_PASSWORD environment variable must be set to a valid password that will be used to call Bitbucket Server/Data Center API's to generate a migration archive.""
276+
exit 1
277+
} else {
278+
Write-Host ""BBS_PASSWORD environment variable is set and will be used to authenticate to Bitbucket Server/Data Center APIs.""
279+
}";
280+
private const string VALIDATE_AZURE_STORAGE_CONNECTION_STRING = @"
281+
if (-not $env:AZURE_STORAGE_CONNECTION_STRING) {
282+
Write-Error ""AZURE_STORAGE_CONNECTION_STRING environment variable must be set to a valid Azure Storage Connection String that will be used to upload the migration archive to Azure Blob Storage.""
283+
exit 1
284+
} else {
285+
Write-Host ""AZURE_STORAGE_CONNECTION_STRING environment variable is set and will be used to upload the migration archive to Azure Blob Storage.""
286+
}";
287+
private const string VALIDATE_AWS_ACCESS_KEY_ID = @"
288+
if (-not $env:AWS_ACCESS_KEY_ID) {
289+
Write-Error ""AWS_ACCESS_KEY_ID environment variable must be set to a valid AWS Access Key ID that will be used to upload the migration archive to AWS S3.""
290+
exit 1
291+
} else {
292+
Write-Host ""AWS_ACCESS_KEY_ID environment variable is set and will be used to upload the migration archive to AWS S3.""
293+
}";
294+
private const string VALIDATE_AWS_SECRET_ACCESS_KEY = @"
295+
if (-not $env:AWS_SECRET_ACCESS_KEY) {
296+
Write-Error ""AWS_SECRET_ACCESS_KEY environment variable must be set to a valid AWS Secret Access Key that will be used to upload the migration archive to AWS S3.""
297+
exit 1
298+
} else {
299+
Write-Host ""AWS_SECRET_ACCESS_KEY environment variable is set and will be used to upload the migration archive to AWS S3.""
300+
}";
301+
private const string VALIDATE_SMB_PASSWORD = @"
302+
if (-not $env:SMB_PASSWORD) {
303+
Write-Error ""SMB_PASSWORD environment variable must be set to a valid password that will be used to download the migration archive from your BBS server using SMB.""
304+
exit 1
305+
} else {
306+
Write-Host ""SMB_PASSWORD environment variable is set and will be used to download the migration archive from your BBS server using SMB.""
235307
}";
236308
}

0 commit comments

Comments
 (0)