Skip to content
This repository was archived by the owner on Apr 13, 2020. It is now read-only.

Commit 978116f

Browse files
authored
Creates new Variablegroup with a link to Azure Key Vault (#73)
* Adding Variable group implementation * added new environment variables in example file. * refactoted code and added comments * Changed node mode to development in webpack config and added debug statements in variable group * Fixed naming issue in the production build. Added Service Point interfaces * removed tslint disabling * Fixed linting issues * updated mock config file and extracted azure-devops interface * Made some config variables nullable * Updated mock config file with variablegroup info * Added interfaces for service connection and implemented feedback comments * Fixed lint issue * renamed service connection to service endpoint to be consistent with rest api * Fixed file reanming issue * refatored azdo.ts methods to be singleton * Fixed lint issues * adding tests * uncommented code * Fixed unit test cases * added more test cases and created new section for vsts variables * added more test cases and created new section for vsts variables * Added test cases * updated test description * updated comments * refactored code and added create command for variable group * added docs * updated docs and added valiadtion to check variable group type * Moved ts-node to dev dependencies * updated docs * renamed file * Fixed type refactoring issue * fixed doc error * changed docs and fixed unit test * Force merging master * added variable group types back after force overwrite
1 parent 3ae45c0 commit 978116f

File tree

18 files changed

+1958
-3
lines changed

18 files changed

+1958
-3
lines changed

.env.example

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,3 +3,10 @@ AZURE_TENANT_ID="AAD tenant id"
33
AZURE_CLIENT_ID="Azure service principal client Id"
44
AZURE_CLIENT_SECRET="Azure service principal client secret/password"
55
AZURE_SUBSCRIPTION_ID="Azure subscription id"
6+
7+
AZDO_PERSONAL_ACCESS_TOKEN=" Azure DevOps Personal Access Token"
8+
9+
AZDO_SERVICE_CONNECTION_SUBSCRIPTION_ID="Azure Key Vault Subscription Id"
10+
AZDO_SERVICE_CONNECTION_CLIENT_ID="Azure Service Principal Id with [get, list] access to Key Vault secrets"
11+
AZDO_SERVICE_CONNECTION_CLIENT_SECRET="Azure Service Principal secret/password"
12+
AZDO_SERVICE_CONNECTION_TENANT_ID="AAD Tenant Id for Azure Service Principal Id"

README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@ Commands:
3737
service Create and manage services for a Bedrock project.
3838
infra Manage and modify your Bedrock infrastructure.
3939
hld Commands for initalizing and managing a bedrock HLD repository.
40+
variable-group Creates Variable Group in Azure DevOps project.
4041
```
4142

4243
## `spk` commands docs
@@ -47,6 +48,7 @@ Commands:
4748
- [spk init](./docs/init.md)
4849
- [spk project](./docs/project-management.md)
4950
- [spk service](./docs/service-management.md)
51+
- [spk variable group](./docs/variable-group.md)
5052

5153
## Getting Started
5254

docs/variable-group.md

Lines changed: 121 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,121 @@
1+
# Variable Group
2+
3+
Create
4+
[variable group](https://docs.microsoft.com/en-us/azure/devops/pipelines/library/variable-groups)
5+
in Azure DevOps project.
6+
7+
Usage:
8+
9+
```
10+
spk variable-group [command] [options]
11+
```
12+
13+
Commands:
14+
15+
- [Variable Group](#variable-group)
16+
- [Prerequisites](#prerequisites)
17+
- [Commands](#commands)
18+
- [create](#create)
19+
20+
Global options:
21+
22+
```
23+
-v, --verbose Enable verbose logging
24+
-h, --help Usage information
25+
```
26+
27+
## Prerequisites
28+
29+
An existing
30+
[Azure DevOps project](https://azure.microsoft.com/en-us/services/devops/) is
31+
required to create a Variable Group. In addition, to link secrets from an Azure
32+
key vault as variables in Variable Group, you will need an existing key vault
33+
containing your secrets and the Service Principal for authorization with Azure
34+
Key Vault.
35+
36+
1. Use existng or
37+
[create a service prrincipal either in Azure Portal](https://docs.microsoft.com/en-us/azure/active-directory/develop/howto-create-service-principal-portal)
38+
or
39+
[with Azure CLI](https://docs.microsoft.com/en-us/cli/azure/create-an-azure-service-principal-azure-cli?view=azure-cli-latest).
40+
2. Use existing or
41+
[create a Key Vault in Azure Portal](https://docs.microsoft.com/en-us/azure/key-vault/quick-create-portal)
42+
or
43+
[with Azure CLI](https://docs.microsoft.com/en-us/azure/key-vault/quick-create-cli).
44+
3. Give the service principal `get` and `list` access in Azure Key Vault. Follow
45+
step 2 from
46+
[these instructions](https://docs.microsoft.com/en-us/azure/devops/pipelines/library/variable-groups?view=azure-devops&tabs=yaml#link-secrets-from-an-azure-key-vault).
47+
48+
## Commands
49+
50+
### create
51+
52+
Add a new variable group in Azure DevOps project
53+
54+
```
55+
Usage:
56+
spk variable-group create|c [options]
57+
58+
Options:
59+
-f, --file <file> Path to the yaml file that contains variable group manifest
60+
-o, --org-name <org> Azure DevOps organization name; falls back to azure_devops.org in spk config
61+
-p, --project <project> Azure DevOps project name; falls back to azure_devops.project in spk config
62+
-t, --personal-access-token <token> Personal access token from Azure DevOps org; falls back to azure_devops.access_token in spk config
63+
```
64+
65+
Variable Group Yaml Manifest File Format
66+
67+
1. Variable Group manifest with variables stored in Azure DevOps
68+
69+
```
70+
name: "myvg"
71+
description: "myvg description"
72+
type: "Vsts"
73+
variables:
74+
storage-account-name:
75+
value: fabrikamstorage
76+
storage-account-access-key:
77+
value: "fabrikamstorage access key"
78+
isSecret: true
79+
```
80+
81+
_*NOTE:*_
82+
83+
- `variables` value can also support json format as shown below.
84+
```
85+
variables: { storage-account-name: { value: "fabrikamstorage" }, storage-account-access-key: {value: "fabrikamstorage access key", isSecret: true } }
86+
```
87+
88+
2. Variable Group manifest with varibles linking to secrets in Azure Key Vault
89+
90+
```
91+
name: "myvg"
92+
description: "myvg description"
93+
type: "AzureKeyVault"
94+
variables:
95+
storage-account-name:
96+
enabled: true
97+
storage-account-access-key:
98+
enabled: true
99+
personal-access-token:
100+
enabled: true
101+
key_vault_provider:
102+
name: "mykeyvault" # name of the Azure Key Vault with Secrets
103+
service_endpoint: # service endpoint is required to authorize with Azure Key Vault
104+
name: "service-connection-name" # service endpoint name to find an existing endpoint or to create a new endpoint with this name
105+
subscription_id: "subscription-id" # Azure Subscription id where Key Vault exist
106+
subscription_name: "sub-name" # Azure Subscription name where Key Vault exist
107+
service_principal_id: "service-principal-id" # Service Principal Id that has 'Get' and 'List' in Key Vault Access Policy
108+
service_principal_secret: "service-principal-secret" # Service Principal secret
109+
tenant_id: "tenant-id" # AAD Tenant Id for the above Service Principal
110+
111+
```
112+
113+
**_NOTE:_**
114+
115+
- SPK will create a new service endpoint when it does not exist by
116+
`service_endpoint: name`.
117+
118+
- `variables` value can also support json format as shown below.
119+
```
120+
variables: { fabrikam-storage-account: { enabled: true }, fabrikam-storage-account-access-key: { enabled: true }, azdo-personal-access-token: { enabled: true } }
121+
```

package.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@
3737
"shx": "^0.3.2",
3838
"ts-jest": "^24.0.2",
3939
"ts-loader": "^6.0.4",
40+
"ts-node": "^8.4.1",
4041
"tslint": "^5.19.0",
4142
"tslint-config-prettier": "^1.18.0",
4243
"typescript": "^3.6.2",
Lines changed: 127 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,127 @@
1+
import { VariableGroup } from "azure-devops-node-api/interfaces/ReleaseInterfaces";
2+
import commander from "commander";
3+
import fs from "fs";
4+
import { readYaml } from "../../config";
5+
import { IAzureDevOpsOpts } from "../../lib/git";
6+
import {
7+
addVariableGroup,
8+
addVariableGroupWithKeyVaultMap
9+
} from "../../lib/pipelines/variableGroup";
10+
import { logger } from "../../logger";
11+
import { IVariableGroupData } from "../../types";
12+
13+
/**
14+
* Adds the create command to the variable-group command object
15+
*
16+
* @param command Commander command object to decorate
17+
*/
18+
export const createCommandDecorator = (command: commander.Command): void => {
19+
command
20+
.command("create")
21+
.alias("c")
22+
.description("Add a new variable group in Azure DevOps project.")
23+
.option(
24+
"-f, --file <variable-group-manifest-file-path>",
25+
"Path to the yaml file that contains variable group manifest."
26+
)
27+
.option(
28+
"-o, --org-name <organization-name>",
29+
"Azure DevOps organization name; falls back to azure_devops.org in spk config"
30+
)
31+
.option(
32+
"-p, --project <project>",
33+
"Azure DevOps project name; falls back to azure_devops.project in spk config"
34+
)
35+
.option(
36+
"-t, --personal-access-token <personal-access-token>",
37+
"Personal access token associated with the Azure DevOps org; falls back to azure_devops.access_token in spk config"
38+
)
39+
.action(async opts => {
40+
try {
41+
if (!opts.file) {
42+
logger.error(
43+
"You need to specify a file with variable group manifest"
44+
);
45+
return;
46+
}
47+
48+
const { file, orgName, project, personalAccessToken } = opts;
49+
50+
logger.debug(
51+
`opts: ${file}, ${orgName}, ${project}, ${personalAccessToken}`
52+
);
53+
54+
// type check
55+
if (typeof orgName !== "undefined" && typeof orgName !== "string") {
56+
throw Error(
57+
`--org-name must be of type 'string', ${typeof orgName} specified.`
58+
);
59+
}
60+
61+
if (typeof project !== "undefined" && typeof project !== "string") {
62+
throw Error(
63+
`--project must be of type 'string', ${typeof project} specified.`
64+
);
65+
}
66+
67+
if (
68+
typeof personalAccessToken !== "undefined" &&
69+
typeof personalAccessToken !== "string"
70+
) {
71+
throw Error(
72+
`--personal-access-token must be of type 'string', ${typeof personalAccessToken} specified.`
73+
);
74+
}
75+
76+
const accessOpts: IAzureDevOpsOpts = {
77+
orgName,
78+
personalAccessToken,
79+
project
80+
};
81+
logger.debug(`access options: ${JSON.stringify(accessOpts)}`);
82+
83+
await create(opts.file, accessOpts);
84+
85+
logger.info(
86+
"Successfully added a variable group in Azure DevOps project!"
87+
);
88+
} catch (err) {
89+
logger.error(`Error occurred while creating variable group`);
90+
logger.error(err);
91+
}
92+
});
93+
};
94+
95+
/**
96+
* Loads varible group manifest from a given filename
97+
*
98+
* @param filepath file to read manifest
99+
* @param accessOpts Azure DevOps access options from command options to override spk config
100+
*/
101+
export const create = async (
102+
filepath: string,
103+
accessOpts: IAzureDevOpsOpts
104+
) => {
105+
logger.info("Creating variale group");
106+
try {
107+
fs.statSync(filepath);
108+
const data = readYaml<IVariableGroupData>(filepath);
109+
logger.debug(`Varible Group Yaml data: ${JSON.stringify(data)}`);
110+
111+
// validate variable group type
112+
113+
let variableGroup: VariableGroup;
114+
115+
if (data.type === "AzureKeyVault") {
116+
variableGroup = await addVariableGroupWithKeyVaultMap(data, accessOpts);
117+
} else if (data.type === "Vsts") {
118+
variableGroup = await addVariableGroup(data, accessOpts);
119+
} else {
120+
throw new Error(
121+
`Varible Group type "${data.type}" is not supported. Only "Vsts" and "AzureKeyVault" are valid types and case sensitive.`
122+
);
123+
}
124+
} catch (err) {
125+
throw err;
126+
}
127+
};
Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
import { Command } from "../command";
2+
import { createCommandDecorator } from "./create";
3+
export const variableGroupCommand = Command(
4+
"variable-group",
5+
"Creates Variable Group in Azure DevOps project.",
6+
[createCommandDecorator]
7+
);

src/config.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ let spkConfig: IConfigYaml = {}; // DANGEROUS! this var is globally retrievable
2121
*
2222
* @param filepath filepath of the yaml file
2323
*/
24-
const readYaml = <T>(filepath: string): T => {
24+
export const readYaml = <T>(filepath: string): T => {
2525
if (fs.existsSync(filepath)) {
2626
const contents = fs.readFileSync(filepath, "utf8");
2727
return yaml.safeLoad(contents) as T;

src/index.ts

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@ import { infraCommand } from "./commands/infra";
2020
import { initCommandDecorator } from "./commands/init";
2121
import { projectCommand } from "./commands/project";
2222
import { serviceCommand } from "./commands/service";
23+
import { variableGroupCommand } from "./commands/variable-group";
2324

2425
////////////////////////////////////////////////////////////////////////////////
2526
// Instantiate core command object
@@ -33,7 +34,14 @@ const rootCommand = Command(
3334
},
3435
initCommandDecorator
3536
],
36-
[deploymentCommand, projectCommand, serviceCommand, infraCommand, hldCommand]
37+
[
38+
deploymentCommand,
39+
projectCommand,
40+
serviceCommand,
41+
infraCommand,
42+
hldCommand,
43+
variableGroupCommand
44+
]
3745
);
3846

3947
////////////////////////////////////////////////////////////////////////////////

0 commit comments

Comments
 (0)