AI Alliance Analytics

Purpose

The purpose of this repository is to host code and documentation for measuring and presenting key process indicators (KPIs) for measturing AI Alliance effectiveness. Intent:

🔍 1. Data-Driven Decision Making

• Enables strategic and operational decisions based on actual data.
• Supports A/B testing and performance comparisons.

📊 2. Improved Operational Efficiency

• Identifies working group bottlenecks, redundancies, and inefficiencies.
• Optimizes resource allocation, staffing, and workflows.

📈 3. Performance Monitoring

• Tracks Key Performance Indicators (KPIs) in real time.
• Highlights underperforming areas for targeted improvements.

💡 4. Participant Insights

• Analyzes participant behavior, preferences, and patterns.
• Aids in project segmentation and marketing.

🚀 5. Competitive Advantage

• Provides insights into production and consumption trends and competitors.
• Enables quick adaptation to changing environments.

📉 6. Risk Management

• Flags compliance issues, and anomalies early.
• Supports scenario modeling and forecasting.

📚 7. Knowledge Sharing

• Centralizes insights for consistent reporting.
• Fosters collaboration across departments through shared data.

Metrics Sources

Metrics Source	Metric List
GitHub	aialliance.github_analytics
PyPi	aialliance.pypi
HuggingFace Data Sets	huggingface.datasets
HuggingFace Data Sets Detail	huggingface.datasets_detail

How to Contribute

You can contribute in several ways:

Add an Existing GitHub Repository

Adding a daily job to collect metrics from an existing GitHub repository is easy:

1. Add the collect_metrics.yml as a workflow to your project at ./github/workflows/collect_metrics.yml

2. Add thre collect_metrics.py to your projects at ./github/scripts/collect_metrics.py

3. Add the following secrets to your project:

Secret	Value
AWS_S3_BUCKET	See AI Alliance analytics team
AWS_REGION	See AI Alliance analytics team
SPECIAL_GH_TOKEN	This needs to be a GitHUb user token than has rights for accessing the repo

Add a New Metrics Source

Adding a new metrics source can be accomplished as follows:

1. Use Python and the source API to query out the desired metrics. Here is an example for PyPi.
2. Build the Python code into a Docker container. Here is an example build script for PyPi, and here is an example Dockerfile for PyPi.
3. Test the Docker container locally. Here is an example local run script for PyPi. You will need AWS access to verify the execution. See the AI Alliance analytics team for assistance.
4. Push the Docker container to the AI Alliance Docker store. Here is an example push script for PyPi.
5. To schedule the job for recurring execution, see the AI Alliance analytics team.
6. To access the nightly execution logs for the job, see the AI Alliance analytics team.

Create a New or Modify an Existing Dashboard

The AI Alliance presentation layer uses Grafana. Creating a new dashboard will require access to the AI Alliance Grafana server. See the AI Alliance analytics team.

Creating or modifying a dashboard will require basic SQL query / filtering / aggregating / joining skills.