Skip to content

Commit 9c1497c

Browse files
tyler-dunkeltyler-dunkel
authored
Merge pull request #84 from cloudgraphdev/alpha
Beta release
2 parents 43c04b7 + 62922f0 commit 9c1497c

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

51 files changed

+1801
-126
lines changed

‎CHANGELOG.md‎

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,63 @@
1+
# [0.67.0-alpha.6](https://github.com/cloudgraphdev/cloudgraph-provider-azure/compare/0.67.0-alpha.5...0.67.0-alpha.6) (2023-07-19)
2+
3+
4+
### Features
5+
6+
* collect crawl error logs ([bd58d1d](https://github.com/cloudgraphdev/cloudgraph-provider-azure/commit/bd58d1dce0cd55d86828b0a463d41061352f2517))
7+
8+
# [0.67.0-alpha.5](https://github.com/cloudgraphdev/cloudgraph-provider-azure/compare/0.67.0-alpha.4...0.67.0-alpha.5) (2023-06-14)
9+
10+
11+
### Features
12+
13+
* add publicIp missing services ([33ae7da](https://github.com/cloudgraphdev/cloudgraph-provider-azure/commit/33ae7da24dd69264344016bf4af3b25a6264c43d))
14+
* Added missing subnets ids and endpoint policies for subnets ([baa0a6e](https://github.com/cloudgraphdev/cloudgraph-provider-azure/commit/baa0a6e72510f70e217a5c516479db8d5d05294f))
15+
* **azure:** Add missing dataFactory services ([2b9a48b](https://github.com/cloudgraphdev/cloudgraph-provider-azure/commit/2b9a48b9e848aad2775069167dcb406ad0dc8f77))
16+
* **azure:** add route table services ([5573e8e](https://github.com/cloudgraphdev/cloudgraph-provider-azure/commit/5573e8eeee9387d50e440e7d1aecd4c529f3c531))
17+
* Fetched records for private dns ([83cbdd8](https://github.com/cloudgraphdev/cloudgraph-provider-azure/commit/83cbdd809f81e750d5abc83bf503dba47e12fe50))
18+
* Fetched virtual network links for private zone ([414ceec](https://github.com/cloudgraphdev/cloudgraph-provider-azure/commit/414ceece3af505c18a76d6141d63a8d85e20067b))
19+
20+
# [0.67.0-alpha.4](https://github.com/cloudgraphdev/cloudgraph-provider-azure/compare/0.67.0-alpha.3...0.67.0-alpha.4) (2023-06-09)
21+
22+
23+
### Features
24+
25+
* **azure:** add missing mySQL Server services ([90131c8](https://github.com/cloudgraphdev/cloudgraph-provider-azure/commit/90131c83d231a8ef3d2f29b23412e978688e32dc))
26+
27+
# [0.67.0-alpha.3](https://github.com/cloudgraphdev/cloudgraph-provider-azure/compare/0.67.0-alpha.2...0.67.0-alpha.3) (2023-06-07)
28+
29+
30+
### Features
31+
32+
* **azure:** Add missing postgreSql server services ([8aeff96](https://github.com/cloudgraphdev/cloudgraph-provider-azure/commit/8aeff969f44cfa8751fefacbbca6597cf7213cd1))
33+
34+
# [0.67.0-alpha.2](https://github.com/cloudgraphdev/cloudgraph-provider-azure/compare/0.67.0-alpha.1...0.67.0-alpha.2) (2023-06-06)
35+
36+
37+
### Bug Fixes
38+
39+
* Added missing attribute for cosmosdb schema ([cde0c79](https://github.com/cloudgraphdev/cloudgraph-provider-azure/commit/cde0c791374d648f77541b1189ddbd3062ea67aa))
40+
41+
42+
### Features
43+
44+
* Exposed containersIds for cosmosdb account ([8b45320](https://github.com/cloudgraphdev/cloudgraph-provider-azure/commit/8b4532048fc8137539489adb38eee101ff867039))
45+
46+
# [0.67.0-alpha.1](https://github.com/cloudgraphdev/cloudgraph-provider-azure/compare/0.66.0...0.67.0-alpha.1) (2023-06-06)
47+
48+
49+
### Features
50+
51+
* **azure:** Add missing SQL Server services ([41bffd4](https://github.com/cloudgraphdev/cloudgraph-provider-azure/commit/41bffd48c728303d0d697509c9efa9a5c7442166))
52+
* **azure:** Support missing storageBlob and storageContainer services ([b179ebf](https://github.com/cloudgraphdev/cloudgraph-provider-azure/commit/b179ebf76107a6285c978067b79f544e995df66f))
53+
54+
# [0.66.0](https://github.com/cloudgraphdev/cloudgraph-provider-azure/compare/0.65.0...0.66.0) (2023-05-16)
55+
56+
57+
### Features
58+
59+
* add new regions ([6e79b4d](https://github.com/cloudgraphdev/cloudgraph-provider-azure/commit/6e79b4dbc6d1adb8f6c2893e5c4e8fb03b0060c3))
60+
161
# [0.66.0-beta.1](https://github.com/cloudgraphdev/cloudgraph-provider-azure/compare/0.65.0...0.66.0-beta.1) (2023-05-16)
262

363

‎README.md‎

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,13 @@ Scan cloud infrastructure via the [Azure SDK](https://github.com/Azure/azure-sdk
44

55
<!-- toc -->
66

7-
- [Install](#install)
8-
- [Authentication](#authentication)
9-
- [Supported Services](#supported-services)
10-
<!-- tocstop -->
7+
- [CloudGraph Azure Provider](#cloudgraph-azure-provider)
8+
- [Docs](#docs)
9+
- [Install](#install)
10+
- [Authentication](#authentication)
11+
- [Supported Services](#supported-services)
12+
- [Development](#development)
13+
- [Testing](#testing)
1114

1215
## Docs
1316

@@ -95,6 +98,7 @@ CloudGraph needs read permissions in order to ingest your data. To keep things e
9598
| postgreSqlServers | resourceGroup, databasePostgreSql |
9699
| privateDns | resourceGroup |
97100
| publicIp | networkInterface, resourceGroup |
101+
| publicIpPrefix | |
98102
| recoveryInstances | recoveryVaults, resourceGroup |
99103
| recoveryPolicies | recoveryVaults, resourceGroup |
100104
| recoveryVaults | recoveryInstances, recoveryPolicies, resourceGroup |
@@ -104,6 +108,8 @@ CloudGraph needs read permissions in order to ingest your data. To keep things e
104108
| replicationNetworks | resourceGroup |
105109
| replicationPolicies | resourceGroup |
106110
| resourceGroup | **all services** |
111+
| routeFilter | |
112+
| routeTable | |
107113
| securityAssessments | |
108114
| securityContacts | |
109115
| securityGroup | networkInterface, resourceGroup |

‎package.json‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
22
"name": "@cloudgraph/cg-provider-azure",
3-
"version": "0.66.0-beta.1",
3+
"version": "0.67.0-alpha.6",
44
"description": "CloudGraph provider plugin for Azure used to fetch Azure cloud data.",
55
"publishConfig": {
66
"registry": "https://registry.npmjs.org/",
@@ -73,7 +73,7 @@
7373
"@azure/identity": "^2.0.4",
7474
"@azure/storage-blob": "^12.8.0",
7575
"@azure/storage-queue": "^12.9.0",
76-
"@cloudgraph/sdk": "0.22.1",
76+
"@cloudgraph/sdk": "0.23.0",
7777
"@graphql-tools/load-files": "^6.5.3",
7878
"@graphql-tools/merge": "^8.2.3",
7979
"@microsoft/microsoft-graph-client": "^3.0.2",

‎src/enums/schemasMap.ts‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,8 @@ export default {
7777
[services.replicationNetworks]: 'azureReplicationNetwork',
7878
[services.replicationPolicies]: 'azureReplicationPolicy',
7979
[services.resourceGroup]: 'azureResourceGroup',
80+
[services.routeFilter]: 'azureRouteFilter',
81+
[services.routeTable]: 'azureRouteTable',
8082
[services.securityAssesments]: 'azureSecurityAssesment',
8183
[services.securityContacts]: 'azureSecurityContact',
8284
[services.securityGroup]: 'azureNetworkSecurityGroup',

‎src/enums/serviceAliases.ts‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,7 @@ export default {
5656
[services.postgreSqlServers]: 'postgreSqlServers',
5757
[services.privateDns]: 'privateDnsZones',
5858
[services.publicIp]: 'publicIps',
59+
[services.publicIpPrefix]: 'publicIpPrefixes',
5960
[services.recoveryVaults]: 'recoveryVaults',
6061
[services.recoveryInstances]: 'recoveryInstances',
6162
[services.recoveryPolicies]: 'recoveryPolicies',
@@ -65,6 +66,8 @@ export default {
6566
[services.replicationNetworks]: 'replicationNetworks',
6667
[services.replicationPolicies]: 'replicationPolicies',
6768
[services.resourceGroup]: 'resourceGroups',
69+
[services.routeFilter]: 'routeFilters',
70+
[services.routeTable]: 'routeTables',
6871
[services.securityAssesments]: 'securityAssesments',
6972
[services.securityContacts]: 'securityContacts',
7073
[services.securityGroup]: 'securityGroups',

‎src/enums/serviceMap.ts‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,7 @@ import AzurePolicyAssignment from '../services/policyAssignment'
5151
import AzurePostgreSqlServer from '../services/postgreSqlServers'
5252
import AzurePrivateDns from '../services/privateDns'
5353
import AzurePublicIp from '../services/publicIp'
54+
import AzurePublicIpPrefix from '../services/publicIpPrefix'
5455
import AzureRecoveryVault from '../services/recoveryVaults'
5556
import AzureRecoveryInstance from '../services/recoveryInstances'
5657
import AzureRecoveryPolicy from '../services/recoveryPolicies'
@@ -89,6 +90,8 @@ import AzureBilling from '../services/billing'
8990
import AzureLogProfiles from '../services/logProfiles'
9091
import Subscription from '../services/subscription'
9192
import AzureNetworkWatcher from '../services/networkWatcher'
93+
import AzureRouteTable from '../services/routeTable'
94+
import AzureRouteFilter from '../services/routeFilter'
9295

9396
/**
9497
* serviceMap is an object that contains all currently supported services for AZURE
@@ -160,6 +163,7 @@ export default {
160163
[services.postgreSqlServers]: AzurePostgreSqlServer,
161164
[services.privateDns]: AzurePrivateDns,
162165
[services.publicIp]: AzurePublicIp,
166+
[services.publicIpPrefix]: AzurePublicIpPrefix,
163167
[services.recoveryVaults]: AzureRecoveryVault,
164168
[services.recoveryInstances]: AzureRecoveryInstance,
165169
[services.recoveryPolicies]: AzureRecoveryPolicy,
@@ -169,6 +173,8 @@ export default {
169173
[services.replicationNetworks]: AzureReplicationNetwork,
170174
[services.replicationPolicies]: AzureReplicationPolicy,
171175
[services.resourceGroup]: AzureResourceGroup,
176+
[services.routeFilter]: AzureRouteFilter,
177+
[services.routeTable]: AzureRouteTable,
172178
[services.securityAssesments]: AzureSecurityAssesments,
173179
[services.securityContacts]: AzureSecurityContacts,
174180
[services.securityGroup]: AzureNetworkSecurityGroup,

‎src/enums/services.ts‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,7 @@ export default {
6262
postgreSqlServers: 'postgreSqlServers',
6363
privateDns: 'privateDns',
6464
publicIp: 'publicIp',
65+
publicIpPrefix: 'publicIpPrefix',
6566
recoveryVaults: 'recoveryVaults',
6667
recoveryInstances: 'recoveryInstances',
6768
recoveryPolicies: 'recoveryPolicies',
@@ -71,6 +72,8 @@ export default {
7172
replicationNetworks: 'replicationNetworks',
7273
replicationPolicies: 'replicationPolicies',
7374
resourceGroup: 'resourceGroup',
75+
routeFilter: 'routeFilter',
76+
routeTable: 'routeTable',
7477
securityAssesments: 'securityAssesments',
7578
securityContacts: 'securityContacts',
7679
securityGroup: 'securityGroup',

‎src/properties/logger.ts‎

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,8 @@ export default {
2020
foundAKSManagedClusters: (num: number): string =>
2121
`Found ${num} AKS managed clusters`,
2222
/* App Gateway */
23-
foundApplicationGateway: (num: number): string => `Found ${num} Application Gateways`,
23+
foundApplicationGateway: (num: number): string =>
24+
`Found ${num} Application Gateways`,
2425
/* App Service */
2526
foundAppServiceEnvironments: (num: number): string =>
2627
`Found ${num} app service environments`,
@@ -152,11 +153,14 @@ export default {
152153
`Found ${num} private dns zones`,
153154
// Public Ips
154155
foundPublicIps: (num: number): string => `Found ${num} public ips`,
156+
foundPublicIpPrefixes: (num: number): string =>
157+
`Found ${num} public ip prefixes`,
155158
// Recovery Vaults
156159
foundRecoveryVaults: (num: number): string => `Found ${num} recovery vaults`,
157160
foundRecoveryInstances: (num: number): string =>
158-
`Found ${num} recovery instances`,
159-
foundRecoveryPolicies: (num: number): string => `Found ${num} recovery policies`,
161+
`Found ${num} recovery instances`,
162+
foundRecoveryPolicies: (num: number): string =>
163+
`Found ${num} recovery policies`,
160164
// RedisCache
161165
foundRedisCaches: (num: number): string => `Found ${num} Redis caches`,
162166
// Replication Appliances
@@ -173,6 +177,10 @@ export default {
173177
`Found ${num} replication policies`,
174178
// Resource Groups
175179
foundResourceGroups: (num: number): string => `Found ${num} resource groups`,
180+
// Route Filters
181+
foundRouteFilters: (num: number): string => `Found ${num} route filters`,
182+
// Route Tables
183+
foundRouteTables: (num: number): string => `Found ${num} route tables`,
176184
// Security Assesments
177185
foundSecurityAssesments: (num: number): string =>
178186
`Found ${num} security assesments`,

‎src/services/cosmosDb/format.ts‎

Lines changed: 24 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -31,9 +31,11 @@ const formatRestoreParameters = (
3131
const { restoreTimestampInUtc, databasesToRestore, ...rest } = restoreParameters
3232
return {
3333
restoreTimestampInUtc: restoreTimestampInUtc?.toISOString(),
34-
databasesToRestore: databasesToRestore?.map(db => ({ id: generateUniqueId({
35-
...db
36-
}), ...db })) || [],
34+
databasesToRestore: databasesToRestore?.map(db => ({
35+
id: generateUniqueId({
36+
...db
37+
}), ...db
38+
})) || [],
3739
...rest,
3840
}
3941
}
@@ -94,6 +96,20 @@ export default ({
9496
tables = [],
9597
} = service
9698

99+
let containersIds: string[] = []
100+
const sqlDatabases = databases?.map(({ id: databaseId, options, data, ...rest }) => {
101+
const containers = new Set(data.map(d => d.id))
102+
containersIds = Array.from(containers)
103+
return ({
104+
id: databaseId,
105+
...rest,
106+
options: {
107+
throughput: options?.throughput,
108+
maxThroughput: options?.autoscaleSettings?.maxThroughput,
109+
},
110+
})
111+
}) || []
112+
97113
return {
98114
id,
99115
name,
@@ -133,10 +149,10 @@ export default ({
133149
failoverPolicies?.map(fp => ({ id: fp.id, ...fp })) || [],
134150
virtualNetworkRules:
135151
virtualNetworkRules?.map(vn => ({ id: vn.id, ...vn })) || [],
136-
privateEndpointConnections: privateEndpointConnections?.map(({ id: endpointId, privateEndpoint, ...pe}) => ({
137-
id: endpointId,
152+
privateEndpointConnections: privateEndpointConnections?.map(({ id: endpointId, privateEndpoint, ...pe }) => ({
153+
id: endpointId,
138154
privateEndpointId: privateEndpoint?.id,
139-
...pe
155+
...pe
140156
})) || [],
141157
enableMultipleWriteLocations,
142158
enableCassandraConnector,
@@ -160,14 +176,8 @@ export default ({
160176
disableLocalAuth,
161177
capacityTotalThroughputLimit: capacity?.totalThroughputLimit,
162178
tags: formatTagsFromMap(Tags),
163-
databases: databases?.map(({ id: databaseId, options, ...rest }) => ({
164-
id: databaseId,
165-
...rest,
166-
options: {
167-
throughput: options?.throughput,
168-
maxThroughput: options?.autoscaleSettings?.maxThroughput,
169-
},
170-
})) || [],
179+
databases: sqlDatabases,
171180
azureTables: tables?.map(at => ({ id: at.id, ...at })) || [],
181+
containersIds
172182
}
173183
}

‎src/services/cosmosDb/schema.graphql‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -303,4 +303,5 @@ type azureCosmosDb implements azureResource
303303
databases: [azureCosmosDbDatabase]
304304
azureTables: [azureCosmosDbTable]
305305
resourceGroup: [azureResourceGroup] @hasInverse(field: cosmosDb)
306+
containersIds: [String] @search(by: [hash])
306307
}

0 commit comments

Comments
 (0)