Merge pull request #135 from oracle-devrel/lift-tenancy-cost-usage

lift-tenancy-usage-cost-reports

Author: AlexanderHodicke

manageability-and-operations/tenancy-usage-cost-reports/LICENSE

@@ -0,0 +1,36 @@
+
+Copyright (c) 2023 Oracle and/or its affiliates.
+
+The Universal Permissive License (UPL), Version 1.0
+
+Subject to the condition set forth below, permission is hereby granted to any
+person obtaining a copy of this software, associated documentation and/or data
+(collectively the "Software"), free of charge and under any and all copyright
+rights in the Software, and any and all patent rights owned or freely
+licensable by each licensor hereunder covering either (i) the unmodified
+Software as contributed to or provided by such licensor, or (ii) the Larger
+Works (as defined below), to deal in both
+
+(a) the Software, and
+(b) any piece of software and/or hardware listed in the lrgrwrks.txt file if
+one is included with the Software (each a "Larger Work" to which the Software
+is contributed by such licensors),
+
+without restriction, including without limitation the rights to copy, create
+derivative works of, display, perform, and distribute the Software and make,
+use, sell, offer for sale, import, export, have made, and have sold the
+Software and the Larger Work(s), and to sublicense the foregoing rights on
+either these or other terms.
+
+This license is subject to the following condition:
+The above copyright notice and either this complete permission notice or at
+a minimum a reference to the UPL must be included in all copies or
+substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

manageability-and-operations/tenancy-usage-cost-reports/README.md

@@ -0,0 +1,40 @@
+# Tenancy Usage and Cost Reports
+​
+Usage2ADW is a tool that uses the Python SDK to extract the usage and cost reports from OCI tenancy and load them to Oracle Autonomous Database. It is an extension of OCI's native Billing & Cost management capabilities, to provide customers with a granular understanding of their spending and utilization of OCI. Using ADW and OAC features, this solution can provide a high-level overview, in-depth spending analysis, resource utilization trends, and forecast.
+
+OCI automatically generates usage data and is stored in an Oracle-owned Object Storage bucket. It contains one row per each OCI resource per hour along with consumption information, metadata, namespace, and tags. Usage2ADW load this data to the ADW database and OAC visualizations can be created on top of this database.
+​
+# When to use this asset?
+​
+Usage2ADW provides customers granular understanding of their spending and utilization of OCI. This solution provides an in-depth spending analysis on OCI. It includes a high-level overview for executives, resource and cost trends for line of business application owners, trends in resource utilization, and more. Usage2ADW is ideal for self-service analysis and allows users to build their own visualizations and enrich the analysis by bringing in their own data like departmental budgets.
+
+Customers can leverage a management module to securely retrieve their tenancy cost and usage data. The module can also be configured to retrieve multiple OCI tenancies data for analysis. This solution extends the core Usage2ADW solution with integration to retrieve usage and cost metrics from other sources, in particular from OEM for the customer on-premises database estate.
+​
+This asset will provide the following key features and benefits:
+​
+1) Load Usage Report to ADW.
+2) Load Cost Reports to ADW.
+3) Load Public Rates to ADW.
+4) Report Usage Current State.
+5) Report Usage Over Time.
+6) Report Cost Analysis.
+7) Report Cost Over Time.
+8) Report Rate Card for Used Products.
+​
+# How to use this asset?
+​
+Here is an overview of how to use this asset:
+​
+Step 1: Provision ADW, OAC, and a VM on Oracle cloud.
+Step 2: Deploy Usage2ADW scripts on VM.
+Step 3: Upload and restore the catalog file on OAC.
+Step 4: Create necessary connections between OCI, ADW, and OAC.
+Step 5: Refresh the OAC dashboard to retrieve your tenancy usage and cost details.
+
+Note: Usage2ADW can be downloaded from https://github.com/oracle/oci-python-sdk/tree/master/examples/usage_reports_to_adw
+
+
+## License
+Copyright (c) 2023 Oracle and/or its affiliates.
+Licensed under the Universal Permissive License (UPL), Version 1.0.
+See [LICENSE](LICENSE) for more details.

manageability-and-operations/tenancy-usage-cost-reports/files/images/image-1.png

@@ -0,0 +1,110 @@
+# Introduction
+
+Usage2ADW is a tool that uses the Python SDK to extract the usage and cost reports from OCI tenancy and load them to Oracle Autonomous Database. It is an extension of OCI's native Billing & Cost management capabilities, to provide customers with a granular understanding of their spending and utilization of OCI. Using ADW and OAC features, this solution can provide a high-level overview, in-depth spend analysis, resource utilization trends, and forecast.
+
+OCI automatically generates usage data and is stored in an Oracle-owned Object Storage bucket. It contains one row per each OCI resource per hour along with consumption information, metadata, namespace, and tags. Usage2ADW load this data to the ADW database and OAC visualizations can be created on top of this database.
+
+## Required technical skills
+
+- Python scripting.
+- Oracle SQL.
+- Oracle Analytics Cloud (OAC).
+- Application Express (Apex).
+
+## OCI APIs Used
+
+- IdentityClient.list_compartments - Policy COMPARTMENT_INSPECT
+- IdentityClient.get_tenancy - Policy TENANCY_INSPECT
+- ObjectStorageClient.list_objects - Policy OBJECT_INSPECT
+- ObjectStorageClient.get_object - Policy OBJECT_READ
+
+## SDK Modules Used
+
+- oci.identity.IdentityClient
+- oci.object_storage.ObjectStorageClient
+
+## OCI Resources Used
+
+- Compute VM: The compute VM houses the install script to create the required data structures and installs the package and Apex application. All data integrations to retrieve OCI information and load it into ADW tables are run on the compute VM.
+  
+- Oracle Analytics Cloud: OAC instance will contain Dashboards and Visualization of the cost and usage reports.
+  
+- Autonomous Data Warehouse: ADW instance will be used to store and process the cost and usage data and effectively support a data mart that will provide usage and cost visibility of the tenancy and different dimensions like tenancy, region, project, and environments.
+  
+- ADW Private Endpoint: PE is used in order that ADW to be deployed on a private subnet and hence be only accessible from a private connection and not from the internet.
+  
+- AC Private Access Channel: PAC provides connectivity to private databases from Oracle Analytics Cloud. It will enable a private endpoint in a private subnet of the Virtual Cloud Network (VCN) of OCI that will let OAC query the private ADW using the private endpoint of ADW.
+
+## Main features:
+
+- Load Usage Report to ADW.
+- Load Cost Reports to ADW.
+- Load Public Rates to ADW.
+- Report Usage Current State.
+- Report Usage Over Time.
+- Report Cost Analysis.
+- Report Cost Over Time.
+- Report Rate Card for Used Products.
+
+# Business Value
+
+ With Usage2ADW, the customer will enjoy the benefits of an analytics platform with dashboards and data visualizations that provide cost by OCI Resources, PaaS, and IaaS services and can be grouped by regions, compartments, etc. for budgetary purposes and chargeback. The analysis can be personalized by application, or department, based on custom tags. Additionally, performance monitoring and utilization metrics will be available for applications running on OCI compute instances.
+
+## Use case
+
+When the extension Usage2ADW is implemented, the user through a dashboard will be able to visualize usage and cost reports over the dimension values. The Tag Namespace and Keys can be used to classify and group resources for Usage and Costing visibility.
+The Usage and Cost Reports provide customers granular understanding of their spending and utilization of OCI. This solution provides an in-depth spend analysis on OCI. It includes a high-level overview for executives, resource and cost trends for line of business application owners, trends in resource utilization, and more. Usage2ADW is ideal for self-service analysis and allows users to build their own visualizations and enrich the analysis by bringing in their own data like departmental budgets. Customers can leverage a management module to securely retrieve their tenancy cost and usage data. The module can also be configured to retrieve multiple OCI tenancies data for analysis. This solution extends the core Usage2ADW solution with integration to retrieve usage and cost metrics from other sources, in particular from OEM for the customer on-prem database estate.
+
+The implemented solution satisfies a broader set of functional requirements:
+
+- Provides the cost of a specific instance.
+- Plot the cost evolution of a specific project over 1, 3, 6, 12, 24 months.
+- Plot the usage evolution of a specific instance per day & month.
+- View the current global consumption.
+- Predict the consumption.
+- Plan the budget in a better way.
+  
+  
+  The solution can also be extended with the following set of capabilities using OCI Tags and drill-down features:
+
+- Extract and Load OCI Tags and Resource Identifiers into the database.
+- Define Meta-Data information about OCI Tags on OCI Resource Identifiers.
+- Data model for Oracle Analytics Cloud (OAC), extended tables for Tags, Meta-Data, and Resource Identifying tables.
+- Custom Canvas/Report in OAC based on the Data Model with drill-down capabilities.
+
+## Sample visualizations
+
+Below are a few visualizations that can be derived out of OCI usage data, using Usage2ADW.
+
+![Alt text](image.png)
+
+Figure 1. Representation of total cost incurred by compartments over a period of time.
+
+![Alt text](image-1.png)
+
+Figure 2. Representation of total cost incurred by services over a period of time.
+
+![Alt text](image-2.png)
+
+Figure 3. Representation of Oracle CPU usage in hours by compartments.
+
+![Alt text](image-3.png)
+
+Figure 4. Representation of running hours of each resource.
+
+![Alt text](image-4.png)
+
+Figure 5. Representation of profit earned each month and forecast for future months.
+
+![Alt text](image-5.png)
+
+
+Figure 6. Representation of cost incurred each month.
+
+
+
+
+
+
+
+