[Cherry-pick] Bug fixes/Upgrades/Enhancements to release 1.1 (#2217)

## Why make this change? Ports bug-fixes to release/1.1: 1.) #2214 2.) #2215 3.) #2208 4.) #2216 --------- Co-authored-by: Yogi K <[email protected]> Co-authored-by: Aniruddh Munde <[email protected]> Co-authored-by: Sean Leonard <[email protected]> Co-authored-by: Jerry Nixon <[email protected]>
Azure · May 14, 2024 · 74ea6c5 · 74ea6c5
1 parent df426a4
commit 74ea6c5
Show file tree

Hide file tree

Showing 13 changed files with 288 additions and 37 deletions.
diff --git a/CODEOWNERS b/CODEOWNERS
@@ -2,3 +2,10 @@
 # the repo. Unless a later match takes precedence,
 # review when someone opens a pull request.
 *       @mbhaskar @Aniruddh25 @seantleonard @mathos1432 @gledis69 @aaronburtle @tarazou9 @ayush3797 @abhishekkumams @aaronpowell @severussundar @ravishetye @rohkhann @neeraj-sharma2592 @sourabh1007 
+
+code_of_conduct.md @jerrynixon
+contributing.md @jerrynixon
+license.txt @jerrynixon
+readme.md @jerrynixon
+security.md @jerrynixon
+support.md @jerrynixon
diff --git a/SUPPORT.md b/SUPPORT.md
@@ -1,18 +1,35 @@
 # Support
 
+Data API builder is an open-source project with contributions from Microsoft and the community to build a configuration-based engine in a Docker-friendly container that easily adds a secure and feature-rich Data API over databases. 
+
 ## How to file issues and get help
 
-### Azure support tickets
+### GitHub Issues
+
+Engineers who contribute to and work on Data API builder periodically monitor and review issues added to [the repository's issues list](https://github.com/Azure/data-api-builder/issues). The period of review is not set by a service level agreement (SLA), and resolution is not guaranteed. When an issue is raised, a member of the team will respond as soon as possible; please be patient.
+
+Repository issues track bugs, questions, and feature requests. Creating issues requires a GitHub account; issue templates prompt for details that will help engineers reproduce your issue and more effectively complete an investigation. Repository issues may be added to the backlog by engineers and become triaged in our future roadmap.
+
+We recommend searching for existing issues similar to yours before creating a new one. Commenting on existing issues helps coalesce recurring issues and helps engineers evaluate severity and find better resolutions.
+
+### Before Submitting an Issue
 
-While Data API builder is in public preview, it is not eligible for Microsoft support. Please use [GitHub Issues](https://github.com/Azure/data-api-builder/issues) to file bugs and feature requests.
+First, please do a search in [open issues](https://github.com/Azure/data-api-builder/issues) to see if the issue or feature request has already been filed. Use this [query](https://github.com/Azure/data-api-builder/issues?q=is%3Aopen+is%3Aissue+sort%3Areactions-%2B1-desc+label%3Aenhancement+) to search for the most popular feature requests.
+If you find your issue already exists, make relevant comments and add your [reaction](https://github.com/blog/2119-add-reactions-to-pull-requests-issues-and-comments). Use a reaction in place of a "+1" comment.
 
-### GitHub issues
+:+1: - upvote
 
-We use [GitHub Issues](https://github.com/azure/data-api-builder/issues) to track bugs, questions, and feature requests. Opening GitHub issues requires you to have a free to create GitHub account. There is a GitHub Issues template to help you provide the information we need to investigate your issue.
+:-1: - downvote
 
-We recommend that you search through the existing issues to see if there is a similar issue that you can comment on. If you find an existing issue, please add a comment to the existing issue instead of creating a new one.
+### Azure services support tickets
 
-When an issue is raised, a member of the team will respond as soon as possible, so please be patient.
+If your solution hosts Data API builder containers using Azure Static Web Apps, Azure Container Apps, Azure Container Instances, Azure Kubernetes Services, or Azure Web Apps for Containers, then the cause of your issue could be that service. If you wish to open a support ticket with that service, do so in your [Azure portal](https://learn.microsoft.com/en-us/azure/azure-portal/supportability/how-to-create-azure-support-request). 
+
+### Azure Database support tickets
+
+If your solution uses an Azure database, including Azure SQL, Azure Cosmos DB, Azure Database for PostgreSQL, Azure Database for MySQL, and Azure SQL Data Warehouse, then the cause of your issue could be with that database. If you wish to open a support ticket with that database, do so in your [Azure portal](https://learn.microsoft.com/en-us/azure/azure-portal/supportability/how-to-create-azure-support-request).
+
+Data API builder containers using Azure Static Web Apps, Azure Container Apps, Azure Container Instances, Azure Kubernetes Services, or Azure Web Apps for Containers, then the cause of your issue could be that service. If you wish to open a support ticket with that service, do so in your [Azure portal](https://learn.microsoft.com/en-us/azure/azure-portal/supportability/how-to-create-azure-support-request). 
 
 ## Resources
 
@@ -23,5 +40,4 @@ When an issue is raised, a member of the team will respond as soon as possible,
 ## Security issues
 
 Security issues and bugs should be reported privately, via email, to the Microsoft Security Response Center (<[email protected]>). You should receive a response within 24 hours.
-
 For more information, review [SECURITY.md](SECURITY.md).
diff --git a/src/Cli.Tests/EndToEndTests.cs b/src/Cli.Tests/EndToEndTests.cs
@@ -178,7 +178,7 @@ public void TestInitializingRestAndGraphQLGlobalSettings()
     [DataRow(CliBool.None, "cosmosdb_nosql", DatabaseType.CosmosDB_NoSQL, DisplayName = "Init command without '--graphql.multiple-create.enabled' option for cosmosdb_nosql database type")]
     public void TestEnablingMultipleCreateOperation(CliBool isMultipleCreateEnabled, string dbType, DatabaseType expectedDbType)
     {
-        List<string> args = new() { "init", "-c", TEST_RUNTIME_CONFIG_FILE, "--connection-string", SAMPLE_TEST_CONN_STRING, "--database-type", dbType };
+        List<string> args = new() { "init", "-c", TEST_RUNTIME_CONFIG_FILE, "--connection-string", dbType == "postgresql" ? SAMPLE_TEST_PGSQL_CONN_STRING : SAMPLE_TEST_CONN_STRING, "--database-type", dbType };
 
         if (string.Equals("cosmosdb_nosql", dbType, StringComparison.OrdinalIgnoreCase))
         {

diff --git a/src/Cli.Tests/TestHelper.cs b/src/Cli.Tests/TestHelper.cs
@@ -13,6 +13,8 @@ public static class TestHelper
 
         public const string SAMPLE_TEST_CONN_STRING = "Data Source=<>;Initial Catalog=<>;User ID=<>;Password=<>;";
 
+        public const string SAMPLE_TEST_PGSQL_CONN_STRING = "Host=<>;Database=<>;username=<>;password=<>";
+
         // test schema for cosmosDB
         public const string TEST_SCHEMA_FILE = "test-schema.gql";
         public const string DAB_DRAFT_SCHEMA_TEST_PATH = "https://github.com/Azure/data-api-builder/releases/download/vmajor.minor.patch/dab.draft.schema.json";
@@ -157,6 +159,68 @@ public static Process ExecuteDabCommand(string command, string flags)
           },
           ""entities"": {}";
 
+        /// <summary>
+        /// Configuration with unresolved environment variable references on
+        /// properties of various data types (string, enum, bool, int).
+        /// </summary>
+        public const string CONFIG_ENV_VARS = @"
+            {
+               ""data-source"": {
+              ""database-type"": ""@env('database-type')"",
+              ""connection-string"": ""@env('connection-string')""
+            },
+          ""runtime"": {
+              ""rest"": {
+                  ""path"": ""/api"",
+                  ""enabled"": false
+              },
+              ""graphql"": {
+                  ""path"": ""/graphql"",
+                  ""enabled"": true,
+                  ""allow-introspection"": true
+              },
+              ""host"": {
+                  ""mode"": ""development"",
+                  ""cors"": {
+                      ""origins"": [],
+                      ""allow-credentials"": false
+                  },
+                  ""authentication"": {
+                      ""provider"": ""StaticWebApps""
+                  }
+              }
+          },
+              ""entities"": {
+              ""MyEntity"": {
+                ""source"": {
+                  ""type"": ""stored-procedure"",
+                  ""object"": ""s001.book"",
+                  ""parameters"": {
+                      ""param1"": ""@env('sp_param1_int')"",
+                      ""param2"": ""hello"",
+                      ""param3"": ""@env('sp_param3_bool')""
+                  }
+                },
+                ""permissions"": [
+                  {
+                    ""role"": ""anonymous"",
+                    ""actions"": [
+                      ""execute""
+                    ]
+                  }
+                ],
+                ""rest"": {
+                    ""methods"": [
+                      ""post""
+                    ]
+                  },
+                  ""graphql"": {
+                    ""operation"": ""mutation""
+                      }
+                    }
+                  }
+          }";
+
         /// <summary>
         /// A minimal valid config json without any entities. This config string is used in unit tests.
         /// </summary>

diff --git a/src/Cli.Tests/ValidateConfigTests.cs b/src/Cli.Tests/ValidateConfigTests.cs
@@ -30,6 +30,12 @@ public void TestCleanup()
     {
         _fileSystem = null;
         _runtimeConfigLoader = null;
+
+        // Clear environment variables set in tests.
+        Environment.SetEnvironmentVariable($"connection-string", null);
+        Environment.SetEnvironmentVariable($"database-type", null);
+        Environment.SetEnvironmentVariable($"sp_param1_int", null);
+        Environment.SetEnvironmentVariable($"sp_param2_bool", null);
     }
 
     /// <summary>
@@ -107,4 +113,58 @@ public void TestValidateWithEmptyConfig()
             Assert.Fail($"Unexpected Exception thrown: {ex.Message}");
         }
     }
+
+    /// <summary>
+    /// This method implicitly validates that RuntimeConfigValidator::ValidateConfigSchema(...) successfully
+    /// executes against a config file referencing environment variables.
+    /// [CLI] ConfigGenerator::IsConfigValid(...)
+    ///     |_ [Engine] RuntimeConfigValidator::TryValidateConfig(...)
+    ///        |_ [Engine] RuntimeConfigValidator::ValidateConfigSchema(...)
+    /// ValidateConfigSchema(...) doesn't execute successfully when a RuntimeConfig object has unresolved environment variables.
+    /// Example:
+    /// Input file snipppet:
+    ///   "data-source": {
+    ///     "database-type": "@env('DATABASE_TYPE')", // ENUM
+    ///     "connection-string": "@env('CONN_STRING')" // STRING
+    ///   }
+    ///   ...
+    ///   "source": {
+    ///     "type": ""stored-procedure",
+    ///     "object": "s001.book",
+    ///     "parameters": {
+    ///         "param1": "@env('sp_param1_int')", // INT
+    ///         "param2": "@env('sp_param2_bool')" // BOOL
+    ///     }
+    ///   }
+    /// </summary>
+    [TestMethod]
+    public void ValidateConfigSchemaWhereConfigReferencesEnvironmentVariables()
+    {
+        // Arrange
+        Environment.SetEnvironmentVariable($"connection-string", SAMPLE_TEST_CONN_STRING);
+        Environment.SetEnvironmentVariable($"database-type", "mssql");
+        Environment.SetEnvironmentVariable($"sp_param1_int", "123");
+        Environment.SetEnvironmentVariable($"sp_param2_bool", "true");
+
+        // Capture console output to get error messaging.
+        StringWriter writer = new();
+        Console.SetOut(writer);
+
+        ((MockFileSystem)_fileSystem!).AddFile(
+            path: TEST_RUNTIME_CONFIG_FILE,
+            mockFile: CONFIG_ENV_VARS);
+        ValidateOptions validateOptions = new(TEST_RUNTIME_CONFIG_FILE);
+
+        // Act
+        ConfigGenerator.IsConfigValid(validateOptions, _runtimeConfigLoader!, _fileSystem!);
+
+        // Assert
+        string loggerOutput = writer.ToString();
+        Assert.IsFalse(
+            condition: loggerOutput.Contains("Failed to validate config against schema due to"),
+            message: "Unexpected errors encountered when validating config schema in RuntimeConfigValidator::ValidateConfigSchema(...).");
+        Assert.IsTrue(
+            condition: loggerOutput.Contains("The config satisfies the schema requirements."),
+            message: "RuntimeConfigValidator::ValidateConfigSchema(...) didn't communicate successful config schema validation.");
+    }
 }
diff --git a/src/Config/Azure.DataApiBuilder.Config.csproj b/src/Config/Azure.DataApiBuilder.Config.csproj
@@ -27,6 +27,7 @@
       </PackageReference>
       <PackageReference Include="Microsoft.SourceLink.GitHub" PrivateAssets="All" />
       <PackageReference Include="Humanizer" />
+      <PackageReference Include="Npgsql" />
   </ItemGroup>
 
     <ItemGroup>

diff --git a/src/Config/RuntimeConfigLoader.cs b/src/Config/RuntimeConfigLoader.cs
@@ -14,6 +14,8 @@
 using Azure.DataApiBuilder.Service.Exceptions;
 using Microsoft.Data.SqlClient;
 using Microsoft.Extensions.Logging;
+using Microsoft.IdentityModel.Tokens;
+using Npgsql;
 
 [assembly: InternalsVisibleTo("Azure.DataApiBuilder.Service.Tests")]
 namespace Azure.DataApiBuilder.Config;
@@ -105,11 +107,15 @@ public static bool TryParseConfig(string json,
 
                 DataSource ds = config.GetDataSourceFromDataSourceName(dataSourceKey);
 
-                // Add Application Name for telemetry for MsSQL
+                // Add Application Name for telemetry for MsSQL or PgSql
                 if (ds.DatabaseType is DatabaseType.MSSQL && replaceEnvVar)
                 {
                     updatedConnection = GetConnectionStringWithApplicationName(connectionValue);
                 }
+                else if (ds.DatabaseType is DatabaseType.PostgreSQL && replaceEnvVar)
+                {
+                    updatedConnection = GetPgSqlConnectionStringWithApplicationName(connectionValue);
+                }
 
                 ds = ds with { ConnectionString = updatedConnection };
                 config.UpdateDataSourceNameToDataSource(dataSourceName, ds);
@@ -235,4 +241,52 @@ internal static string GetConnectionStringWithApplicationName(string connectionS
         // Return the updated connection string.
         return connectionStringBuilder.ConnectionString;
     }
+
+    /// <summary>
+    /// It adds or replaces a property in the connection string with `Application Name` property.
+    /// If the connection string already contains the property, it appends the property `Application Name` to the connection string,
+    /// else add the Application Name property with DataApiBuilder Application Name based on hosted/oss platform.
+    /// </summary>
+    /// <param name="connectionString">Connection string for connecting to database.</param>
+    /// <returns>Updated connection string with `Application Name` property.</returns>
+    internal static string GetPgSqlConnectionStringWithApplicationName(string connectionString)
+    {
+        // If the connection string is null, empty, or whitespace, return it as is.
+        if (string.IsNullOrWhiteSpace(connectionString))
+        {
+            return connectionString;
+        }
+
+        string applicationName = ProductInfo.GetDataApiBuilderUserAgent();
+
+        // Create a StringBuilder from the connection string.
+        NpgsqlConnectionStringBuilder connectionStringBuilder;
+        try
+        {
+            connectionStringBuilder = new NpgsqlConnectionStringBuilder(connectionString);
+        }
+        catch (Exception ex)
+        {
+            throw new DataApiBuilderException(
+                message: DataApiBuilderException.CONNECTION_STRING_ERROR_MESSAGE,
+                statusCode: HttpStatusCode.ServiceUnavailable,
+                subStatusCode: DataApiBuilderException.SubStatusCodes.ErrorInInitialization,
+                innerException: ex);
+        }
+
+        // If the connection string does not contain the `Application Name` property, add it.
+        // or if the connection string contains the `Application Name` property, replace it with the DataApiBuilder Application Name.
+        if (connectionStringBuilder.ApplicationName.IsNullOrEmpty())
+        {
+            connectionStringBuilder.ApplicationName = applicationName;
+        }
+        else
+        {
+            // If the connection string contains the `ApplicationName` property with a value, update the value by adding the DataApiBuilder Application Name.
+            connectionStringBuilder.ApplicationName += $",{applicationName}";
+        }
+
+        // Return the updated connection string.
+        return connectionStringBuilder.ConnectionString;
+    }
 }
diff --git a/src/Core/Configurations/RuntimeConfigValidator.cs b/src/Core/Configurations/RuntimeConfigValidator.cs
@@ -190,7 +190,7 @@ public async Task<bool> TryValidateConfig(
     /// </summary>
     public async Task<JsonSchemaValidationResult> ValidateConfigSchema(RuntimeConfig runtimeConfig, string configFilePath, ILoggerFactory loggerFactory)
     {
-        string jsonData = _fileSystem.File.ReadAllText(configFilePath);
+        string jsonData = runtimeConfig.ToJson();
         ILogger<JsonConfigSchemaValidator> jsonConfigValidatorLogger = loggerFactory.CreateLogger<JsonConfigSchemaValidator>();
         JsonConfigSchemaValidator jsonConfigSchemaValidator = new(jsonConfigValidatorLogger, _fileSystem);
 

diff --git a/src/Directory.Packages.props b/src/Directory.Packages.props
@@ -64,7 +64,6 @@
     <PackageVersion Include="Microsoft.AspNetCore.TestHost" Version="6.0.29" />
     <PackageVersion Include="Microsoft.Extensions.Configuration.Json" Version="6.0.0" />
     <PackageVersion Include="Microsoft.Extensions.Configuration.Binder" Version="6.0.0" />
-    <!--Once dotnet restore properly resolves that 'Npgsql' 7.0.7 is not vulnerable, set this ref to 7.0.7.-->
-    <PackageVersion Include="Npgsql" Version="8.0.3" />
+    <PackageVersion Include="Npgsql" Version="7.0.7" />
   </ItemGroup>
 </Project>