Skip to content

Commit 74ea6c5

Browse files
ayush3797yogivkAniruddh25seantleonardJerryNixon
authored
[Cherry-pick] Bug fixes/Upgrades/Enhancements to release 1.1 (#2217)
## Why make this change? Ports bug-fixes to release/1.1: 1.) #2214 2.) #2215 3.) #2208 4.) #2216 --------- Co-authored-by: Yogi K <125512104+yogivk@users.noreply.github.com> Co-authored-by: Aniruddh Munde <anmunde@microsoft.com> Co-authored-by: Sean Leonard <sean.leonard@microsoft.com> Co-authored-by: Jerry Nixon <1749983+JerryNixon@users.noreply.github.com>
1 parent df426a4 commit 74ea6c5

13 files changed

+288
-37
lines changed

CODEOWNERS

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,3 +2,10 @@
22
# the repo. Unless a later match takes precedence,
33
# review when someone opens a pull request.
44
* @mbhaskar @Aniruddh25 @seantleonard @mathos1432 @gledis69 @aaronburtle @tarazou9 @ayush3797 @abhishekkumams @aaronpowell @severussundar @ravishetye @rohkhann @neeraj-sharma2592 @sourabh1007
5+
6+
code_of_conduct.md @jerrynixon
7+
contributing.md @jerrynixon
8+
license.txt @jerrynixon
9+
readme.md @jerrynixon
10+
security.md @jerrynixon
11+
support.md @jerrynixon

SUPPORT.md

Lines changed: 23 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,35 @@
11
# Support
22

3+
Data API builder is an open-source project with contributions from Microsoft and the community to build a configuration-based engine in a Docker-friendly container that easily adds a secure and feature-rich Data API over databases.
4+
35
## How to file issues and get help
46

5-
### Azure support tickets
7+
### GitHub Issues
8+
9+
Engineers who contribute to and work on Data API builder periodically monitor and review issues added to [the repository's issues list](https://github.com/Azure/data-api-builder/issues). The period of review is not set by a service level agreement (SLA), and resolution is not guaranteed. When an issue is raised, a member of the team will respond as soon as possible; please be patient.
10+
11+
Repository issues track bugs, questions, and feature requests. Creating issues requires a GitHub account; issue templates prompt for details that will help engineers reproduce your issue and more effectively complete an investigation. Repository issues may be added to the backlog by engineers and become triaged in our future roadmap.
12+
13+
We recommend searching for existing issues similar to yours before creating a new one. Commenting on existing issues helps coalesce recurring issues and helps engineers evaluate severity and find better resolutions.
14+
15+
### Before Submitting an Issue
616

7-
While Data API builder is in public preview, it is not eligible for Microsoft support. Please use [GitHub Issues](https://github.com/Azure/data-api-builder/issues) to file bugs and feature requests.
17+
First, please do a search in [open issues](https://github.com/Azure/data-api-builder/issues) to see if the issue or feature request has already been filed. Use this [query](https://github.com/Azure/data-api-builder/issues?q=is%3Aopen+is%3Aissue+sort%3Areactions-%2B1-desc+label%3Aenhancement+) to search for the most popular feature requests.
18+
If you find your issue already exists, make relevant comments and add your [reaction](https://github.com/blog/2119-add-reactions-to-pull-requests-issues-and-comments). Use a reaction in place of a "+1" comment.
819

9-
### GitHub issues
20+
:+1: - upvote
1021

11-
We use [GitHub Issues](https://github.com/azure/data-api-builder/issues) to track bugs, questions, and feature requests. Opening GitHub issues requires you to have a free to create GitHub account. There is a GitHub Issues template to help you provide the information we need to investigate your issue.
22+
:-1: - downvote
1223

13-
We recommend that you search through the existing issues to see if there is a similar issue that you can comment on. If you find an existing issue, please add a comment to the existing issue instead of creating a new one.
24+
### Azure services support tickets
1425

15-
When an issue is raised, a member of the team will respond as soon as possible, so please be patient.
26+
If your solution hosts Data API builder containers using Azure Static Web Apps, Azure Container Apps, Azure Container Instances, Azure Kubernetes Services, or Azure Web Apps for Containers, then the cause of your issue could be that service. If you wish to open a support ticket with that service, do so in your [Azure portal](https://learn.microsoft.com/en-us/azure/azure-portal/supportability/how-to-create-azure-support-request).
27+
28+
### Azure Database support tickets
29+
30+
If your solution uses an Azure database, including Azure SQL, Azure Cosmos DB, Azure Database for PostgreSQL, Azure Database for MySQL, and Azure SQL Data Warehouse, then the cause of your issue could be with that database. If you wish to open a support ticket with that database, do so in your [Azure portal](https://learn.microsoft.com/en-us/azure/azure-portal/supportability/how-to-create-azure-support-request).
31+
32+
Data API builder containers using Azure Static Web Apps, Azure Container Apps, Azure Container Instances, Azure Kubernetes Services, or Azure Web Apps for Containers, then the cause of your issue could be that service. If you wish to open a support ticket with that service, do so in your [Azure portal](https://learn.microsoft.com/en-us/azure/azure-portal/supportability/how-to-create-azure-support-request).
1633

1734
## Resources
1835

@@ -23,5 +40,4 @@ When an issue is raised, a member of the team will respond as soon as possible,
2340
## Security issues
2441

2542
Security issues and bugs should be reported privately, via email, to the Microsoft Security Response Center (<secure@microsoft.com>). You should receive a response within 24 hours.
26-
2743
For more information, review [SECURITY.md](SECURITY.md).

src/Cli.Tests/EndToEndTests.cs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -178,7 +178,7 @@ public void TestInitializingRestAndGraphQLGlobalSettings()
178178
[DataRow(CliBool.None, "cosmosdb_nosql", DatabaseType.CosmosDB_NoSQL, DisplayName = "Init command without '--graphql.multiple-create.enabled' option for cosmosdb_nosql database type")]
179179
public void TestEnablingMultipleCreateOperation(CliBool isMultipleCreateEnabled, string dbType, DatabaseType expectedDbType)
180180
{
181-
List<string> args = new() { "init", "-c", TEST_RUNTIME_CONFIG_FILE, "--connection-string", SAMPLE_TEST_CONN_STRING, "--database-type", dbType };
181+
List<string> args = new() { "init", "-c", TEST_RUNTIME_CONFIG_FILE, "--connection-string", dbType == "postgresql" ? SAMPLE_TEST_PGSQL_CONN_STRING : SAMPLE_TEST_CONN_STRING, "--database-type", dbType };
182182

183183
if (string.Equals("cosmosdb_nosql", dbType, StringComparison.OrdinalIgnoreCase))
184184
{

src/Cli.Tests/TestHelper.cs

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,8 @@ public static class TestHelper
1313

1414
public const string SAMPLE_TEST_CONN_STRING = "Data Source=<>;Initial Catalog=<>;User ID=<>;Password=<>;";
1515

16+
public const string SAMPLE_TEST_PGSQL_CONN_STRING = "Host=<>;Database=<>;username=<>;password=<>";
17+
1618
// test schema for cosmosDB
1719
public const string TEST_SCHEMA_FILE = "test-schema.gql";
1820
public const string DAB_DRAFT_SCHEMA_TEST_PATH = "https://github.com/Azure/data-api-builder/releases/download/vmajor.minor.patch/dab.draft.schema.json";
@@ -157,6 +159,68 @@ public static Process ExecuteDabCommand(string command, string flags)
157159
},
158160
""entities"": {}";
159161

162+
/// <summary>
163+
/// Configuration with unresolved environment variable references on
164+
/// properties of various data types (string, enum, bool, int).
165+
/// </summary>
166+
public const string CONFIG_ENV_VARS = @"
167+
{
168+
""data-source"": {
169+
""database-type"": ""@env('database-type')"",
170+
""connection-string"": ""@env('connection-string')""
171+
},
172+
""runtime"": {
173+
""rest"": {
174+
""path"": ""/api"",
175+
""enabled"": false
176+
},
177+
""graphql"": {
178+
""path"": ""/graphql"",
179+
""enabled"": true,
180+
""allow-introspection"": true
181+
},
182+
""host"": {
183+
""mode"": ""development"",
184+
""cors"": {
185+
""origins"": [],
186+
""allow-credentials"": false
187+
},
188+
""authentication"": {
189+
""provider"": ""StaticWebApps""
190+
}
191+
}
192+
},
193+
""entities"": {
194+
""MyEntity"": {
195+
""source"": {
196+
""type"": ""stored-procedure"",
197+
""object"": ""s001.book"",
198+
""parameters"": {
199+
""param1"": ""@env('sp_param1_int')"",
200+
""param2"": ""hello"",
201+
""param3"": ""@env('sp_param3_bool')""
202+
}
203+
},
204+
""permissions"": [
205+
{
206+
""role"": ""anonymous"",
207+
""actions"": [
208+
""execute""
209+
]
210+
}
211+
],
212+
""rest"": {
213+
""methods"": [
214+
""post""
215+
]
216+
},
217+
""graphql"": {
218+
""operation"": ""mutation""
219+
}
220+
}
221+
}
222+
}";
223+
160224
/// <summary>
161225
/// A minimal valid config json without any entities. This config string is used in unit tests.
162226
/// </summary>

src/Cli.Tests/ValidateConfigTests.cs

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,12 @@ public void TestCleanup()
3030
{
3131
_fileSystem = null;
3232
_runtimeConfigLoader = null;
33+
34+
// Clear environment variables set in tests.
35+
Environment.SetEnvironmentVariable($"connection-string", null);
36+
Environment.SetEnvironmentVariable($"database-type", null);
37+
Environment.SetEnvironmentVariable($"sp_param1_int", null);
38+
Environment.SetEnvironmentVariable($"sp_param2_bool", null);
3339
}
3440

3541
/// <summary>
@@ -107,4 +113,58 @@ public void TestValidateWithEmptyConfig()
107113
Assert.Fail($"Unexpected Exception thrown: {ex.Message}");
108114
}
109115
}
116+
117+
/// <summary>
118+
/// This method implicitly validates that RuntimeConfigValidator::ValidateConfigSchema(...) successfully
119+
/// executes against a config file referencing environment variables.
120+
/// [CLI] ConfigGenerator::IsConfigValid(...)
121+
/// |_ [Engine] RuntimeConfigValidator::TryValidateConfig(...)
122+
/// |_ [Engine] RuntimeConfigValidator::ValidateConfigSchema(...)
123+
/// ValidateConfigSchema(...) doesn't execute successfully when a RuntimeConfig object has unresolved environment variables.
124+
/// Example:
125+
/// Input file snipppet:
126+
/// "data-source": {
127+
/// "database-type": "@env('DATABASE_TYPE')", // ENUM
128+
/// "connection-string": "@env('CONN_STRING')" // STRING
129+
/// }
130+
/// ...
131+
/// "source": {
132+
/// "type": ""stored-procedure",
133+
/// "object": "s001.book",
134+
/// "parameters": {
135+
/// "param1": "@env('sp_param1_int')", // INT
136+
/// "param2": "@env('sp_param2_bool')" // BOOL
137+
/// }
138+
/// }
139+
/// </summary>
140+
[TestMethod]
141+
public void ValidateConfigSchemaWhereConfigReferencesEnvironmentVariables()
142+
{
143+
// Arrange
144+
Environment.SetEnvironmentVariable($"connection-string", SAMPLE_TEST_CONN_STRING);
145+
Environment.SetEnvironmentVariable($"database-type", "mssql");
146+
Environment.SetEnvironmentVariable($"sp_param1_int", "123");
147+
Environment.SetEnvironmentVariable($"sp_param2_bool", "true");
148+
149+
// Capture console output to get error messaging.
150+
StringWriter writer = new();
151+
Console.SetOut(writer);
152+
153+
((MockFileSystem)_fileSystem!).AddFile(
154+
path: TEST_RUNTIME_CONFIG_FILE,
155+
mockFile: CONFIG_ENV_VARS);
156+
ValidateOptions validateOptions = new(TEST_RUNTIME_CONFIG_FILE);
157+
158+
// Act
159+
ConfigGenerator.IsConfigValid(validateOptions, _runtimeConfigLoader!, _fileSystem!);
160+
161+
// Assert
162+
string loggerOutput = writer.ToString();
163+
Assert.IsFalse(
164+
condition: loggerOutput.Contains("Failed to validate config against schema due to"),
165+
message: "Unexpected errors encountered when validating config schema in RuntimeConfigValidator::ValidateConfigSchema(...).");
166+
Assert.IsTrue(
167+
condition: loggerOutput.Contains("The config satisfies the schema requirements."),
168+
message: "RuntimeConfigValidator::ValidateConfigSchema(...) didn't communicate successful config schema validation.");
169+
}
110170
}

src/Config/Azure.DataApiBuilder.Config.csproj

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@
2727
</PackageReference>
2828
<PackageReference Include="Microsoft.SourceLink.GitHub" PrivateAssets="All" />
2929
<PackageReference Include="Humanizer" />
30+
<PackageReference Include="Npgsql" />
3031
</ItemGroup>
3132

3233
<ItemGroup>

src/Config/RuntimeConfigLoader.cs

Lines changed: 55 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,8 @@
1414
using Azure.DataApiBuilder.Service.Exceptions;
1515
using Microsoft.Data.SqlClient;
1616
using Microsoft.Extensions.Logging;
17+
using Microsoft.IdentityModel.Tokens;
18+
using Npgsql;
1719

1820
[assembly: InternalsVisibleTo("Azure.DataApiBuilder.Service.Tests")]
1921
namespace Azure.DataApiBuilder.Config;
@@ -105,11 +107,15 @@ public static bool TryParseConfig(string json,
105107

106108
DataSource ds = config.GetDataSourceFromDataSourceName(dataSourceKey);
107109

108-
// Add Application Name for telemetry for MsSQL
110+
// Add Application Name for telemetry for MsSQL or PgSql
109111
if (ds.DatabaseType is DatabaseType.MSSQL && replaceEnvVar)
110112
{
111113
updatedConnection = GetConnectionStringWithApplicationName(connectionValue);
112114
}
115+
else if (ds.DatabaseType is DatabaseType.PostgreSQL && replaceEnvVar)
116+
{
117+
updatedConnection = GetPgSqlConnectionStringWithApplicationName(connectionValue);
118+
}
113119

114120
ds = ds with { ConnectionString = updatedConnection };
115121
config.UpdateDataSourceNameToDataSource(dataSourceName, ds);
@@ -235,4 +241,52 @@ internal static string GetConnectionStringWithApplicationName(string connectionS
235241
// Return the updated connection string.
236242
return connectionStringBuilder.ConnectionString;
237243
}
244+
245+
/// <summary>
246+
/// It adds or replaces a property in the connection string with `Application Name` property.
247+
/// If the connection string already contains the property, it appends the property `Application Name` to the connection string,
248+
/// else add the Application Name property with DataApiBuilder Application Name based on hosted/oss platform.
249+
/// </summary>
250+
/// <param name="connectionString">Connection string for connecting to database.</param>
251+
/// <returns>Updated connection string with `Application Name` property.</returns>
252+
internal static string GetPgSqlConnectionStringWithApplicationName(string connectionString)
253+
{
254+
// If the connection string is null, empty, or whitespace, return it as is.
255+
if (string.IsNullOrWhiteSpace(connectionString))
256+
{
257+
return connectionString;
258+
}
259+
260+
string applicationName = ProductInfo.GetDataApiBuilderUserAgent();
261+
262+
// Create a StringBuilder from the connection string.
263+
NpgsqlConnectionStringBuilder connectionStringBuilder;
264+
try
265+
{
266+
connectionStringBuilder = new NpgsqlConnectionStringBuilder(connectionString);
267+
}
268+
catch (Exception ex)
269+
{
270+
throw new DataApiBuilderException(
271+
message: DataApiBuilderException.CONNECTION_STRING_ERROR_MESSAGE,
272+
statusCode: HttpStatusCode.ServiceUnavailable,
273+
subStatusCode: DataApiBuilderException.SubStatusCodes.ErrorInInitialization,
274+
innerException: ex);
275+
}
276+
277+
// If the connection string does not contain the `Application Name` property, add it.
278+
// or if the connection string contains the `Application Name` property, replace it with the DataApiBuilder Application Name.
279+
if (connectionStringBuilder.ApplicationName.IsNullOrEmpty())
280+
{
281+
connectionStringBuilder.ApplicationName = applicationName;
282+
}
283+
else
284+
{
285+
// If the connection string contains the `ApplicationName` property with a value, update the value by adding the DataApiBuilder Application Name.
286+
connectionStringBuilder.ApplicationName += $",{applicationName}";
287+
}
288+
289+
// Return the updated connection string.
290+
return connectionStringBuilder.ConnectionString;
291+
}
238292
}

src/Core/Configurations/RuntimeConfigValidator.cs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -190,7 +190,7 @@ public async Task<bool> TryValidateConfig(
190190
/// </summary>
191191
public async Task<JsonSchemaValidationResult> ValidateConfigSchema(RuntimeConfig runtimeConfig, string configFilePath, ILoggerFactory loggerFactory)
192192
{
193-
string jsonData = _fileSystem.File.ReadAllText(configFilePath);
193+
string jsonData = runtimeConfig.ToJson();
194194
ILogger<JsonConfigSchemaValidator> jsonConfigValidatorLogger = loggerFactory.CreateLogger<JsonConfigSchemaValidator>();
195195
JsonConfigSchemaValidator jsonConfigSchemaValidator = new(jsonConfigValidatorLogger, _fileSystem);
196196

src/Directory.Packages.props

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -64,7 +64,6 @@
6464
<PackageVersion Include="Microsoft.AspNetCore.TestHost" Version="6.0.29" />
6565
<PackageVersion Include="Microsoft.Extensions.Configuration.Json" Version="6.0.0" />
6666
<PackageVersion Include="Microsoft.Extensions.Configuration.Binder" Version="6.0.0" />
67-
<!--Once dotnet restore properly resolves that 'Npgsql' 7.0.7 is not vulnerable, set this ref to 7.0.7.-->
68-
<PackageVersion Include="Npgsql" Version="8.0.3" />
67+
<PackageVersion Include="Npgsql" Version="7.0.7" />
6968
</ItemGroup>
7069
</Project>

0 commit comments

Comments
 (0)