Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
122 changes: 122 additions & 0 deletions CODE_OF_CONDUCT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
# Contributor Covenant Code of Conduct

## Our Pledge

We as members, contributors, and leaders pledge to make participation in our
community a harassment-free experience for everyone, regardless of age, body
size, visible or invisible disability, ethnicity, sex characteristics, gender
identity and expression, level of experience, education, socio-economic status,
nationality, personal appearance, race, religion, or sexual identity
and orientation.

We pledge to act and interact in ways that are welcoming, inclusive,
diverse, and equitable.

## Our Standards

Examples of behavior that contributes to a positive environment for our
community include:

* Demonstrating empathy and kindness toward other people
* Being respectful of differing opinions, viewpoints, and experiences
* Giving and gracefully accepting constructive feedback
* Accepting responsibility and apologizing to those affected by our mistakes,
and learning from the experience
* Focusing on what is best not just for us as individuals, but for the
overall community

Examples of unacceptable behavior include:

* The use of sexualized language or imagery, and sexual attention or
advances of any kind
* Trolling, insulting or derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or email
address, without their explicit permission
* Other conduct which could reasonably be considered inappropriate in a
professional setting

## Enforcement Responsibilities

Community leaders are responsible for clarifying and enforcing our standards of
acceptable behavior and will take appropriate and fair corrective action in
response to any behavior that they deem inappropriate, threatening, offensive,
or harmful.

Community leaders have the right and responsibility to remove, edit, or reject
comments, commits, code, wiki edits, issues, and other contributions that are
not aligned to this Code of Conduct, and will communicate reasons for moderation
decisions when appropriate.

## Scope

This Code of Conduct applies within all community spaces, and also applies when
an individual is officially representing the community in public spaces.
Examples of representing our community include using an official e-mail address,
posting via an official social media account, or acting as an appointed
representative at an online or offline event.

## Enforcement

Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported to the community leaders responsible for enforcement at
[email protected].
All complaints will be reviewed and investigated promptly and fairly.

All community leaders are obligated to respect the privacy and security of the
reporter of any incident.

## Enforcement Guidelines

Community leaders will follow these Community Impact Guidelines in determining
the consequences for any action they deem in violation of this Code of Conduct:

### 1. Correction

**Community Impact**: Use of inappropriate language or other behavior deemed
unprofessional or unwelcome in the community.

**Consequence**: A private, written warning from community leaders, providing
clarity around the nature of the violation and an explanation of why the
behavior was inappropriate. A public apology may be requested.

### 2. Warning

**Community Impact**: A violation through a single incident or series
of actions.

**Consequence**: A warning with consequences for continued behavior. No
interaction with the people involved, including unsolicited interaction with
those enforcing the Code of Conduct, for a specified period of time. This
includes avoiding interaction in community spaces as well as external channels
like social media. Violating these terms may lead to a temporary or
permanent ban.

### 3. Temporary Ban

**Community Impact**: A serious violation of community standards, including
sustained inappropriate behavior.

**Consequence**: A temporary ban from any sort of interaction or public
communication with the community for a specified period of time. No public or
private interaction with the people involved, including unsolicited interaction
with those enforcing the Code of Conduct, is allowed during this period.
Violating these terms may lead to a permanent ban.

### 4. Permanent Ban

**Community Impact**: Demonstrating a pattern of violation of community
standards, including sustained inappropriate behavior, harassment of an
individual, or aggression toward or disparagement of classes of individuals.

**Consequence**: A permanent ban from any sort of public interaction within
the community.

## Attribution

This Code of Conduct is adapted from the [Contributor Covenant][homepage],
version 2.1, available at
[https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1].

[homepage]: https://www.contributor-covenant.org
[v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html
97 changes: 97 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
# Contributing to Intugle

First off, thank you for considering contributing to Intugle! It's people like you that make Intugle such a great tool.

Intugle is an open source project and we love to receive contributions from our community — you!

There are many ways to contribute, from writing tutorials or blog posts, improving the documentation, submitting bug reports and feature requests or writing code which can be incorporated into Intugle itself.

## How to Contribute

### Reporting Bugs

If you find a bug, please open an issue on our [GitHub repository](https://github.com/Intugle/data-tools/issues). Please include a clear and concise description of the bug, as well as steps to reproduce it.

### Submitting Feature Requests

If you have an idea for a new feature, please open an issue on our [GitHub repository](https://github.com/Intugle/data-tools/issues). Please include a clear and concise description of the feature, as well as a justification for why it would be a good addition to the project.

### Writing Code

If you want to contribute code to the project, please follow the steps below.

## Getting Started

1. Fork the repository on GitHub.
2. Clone your fork locally:

```bash
git clone https://github.com/your-username/intugle.git
```

3. Set up a virtual environment and install the dependencies:

```bash
cd intugle
python -m venv venv
source venv/bin/activate
pip install -r requirements.txt
```

## Development Workflow

1. Create a new branch for your changes:

```bash
git checkout -b my-new-feature
```

2. Make your changes to the code.

3. Run the tests to make sure that your changes don't break anything:

```bash
pytest
```

4. Commit your changes:

```bash
git commit -m "Add some feature"
```

5. Push your changes to your fork:

```bash
git push origin my-new-feature
```

6. Open a pull request on the [Intugle repository](https://github.com/Intugle/data-tools/pulls).

## Coding Style

This project uses `ruff` for linting and code formatting. Before submitting a pull request, please make sure that your code conforms to the project's coding style by running the following command:

```bash
ruff check .
```

## Testing

This project uses `pytest` for testing. To run the tests, use the following command:

```bash
pytest
```

## Submitting a Pull Request

When you submit a pull request, please include the following:

* A clear and concise title for the pull request.
* A clear and concise description of the changes that you made.
* A link to the issue that the pull request addresses (if applicable).

## Code of Conduct

Please note that this project is released with a [Contributor Code of Conduct](CODE_OF_CONDUCT.md). By participating in this project you agree to abide by its terms.
13 changes: 8 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
# Data-Tools
<p align="center">
<img src="https://intugle.ai/assets/Icons/intugle.png" alt="Intugle Logo" width="350">
<h3 align="center"><i>The GenAI-powered toolkit for automated data intelligence.</i></h3>
</p>

[![Release](https://img.shields.io/github/release/Intugle/data-tools)](https://github.com/Intugle/data-tools/releases/tag/v0.1.0)
[![Made with Python](https://img.shields.io/badge/Made_with-Python-blue?logo=python&logoColor=white)](https://www.python.org/)
Expand All @@ -11,7 +14,7 @@

## Overview

Intugle's Data-Tools is a GenAI-powered Python library that simplifies and accelerates the journey from raw data to insights. It empowers data and business teams to build an intelligent semantic layer over their data, enabling self-serve analytics and natural language queries. By automating data profiling, link prediction, and SQL generation, Data-Tools helps you build data products faster and more efficiently than traditional methods.
Intugle provides a set of GenAI-powered Python tools to simplify and accelerate the journey from raw data to insights. This library empowers data and business teams to build an intelligent semantic layer over their data, enabling self-serve analytics and natural language queries. By automating data profiling, link prediction, and SQL generation, Intugle helps you build data products faster and more efficiently than traditional methods.

## Who is this for?

Expand All @@ -35,7 +38,7 @@ This tool is designed for both **data teams** and **business teams**.
### Installation

```bash
pip install data-tools
pip install intugle
```

### Configuration
Expand Down Expand Up @@ -67,7 +70,7 @@ The core workflow of the project involves the following steps:
3. **Predict links:** Use the `LinkPredictor` to discover relationships between your tables.

```python
from data_tools import LinkPredictor
from intugle import LinkPredictor

# Initialize the predictor
predictor = LinkPredictor(datasets)
Expand All @@ -80,7 +83,7 @@ The core workflow of the project involves the following steps:
5. **Generate SQL:** Use the `SqlGenerator` to generate SQL queries from the semantic layer.

```python
from data_tools import SqlGenerator
from intugle import SqlGenerator

# Create a SqlGenerator
sql_generator = SqlGenerator()
Expand Down
20 changes: 10 additions & 10 deletions notebooks/quickstart_content.txt
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ type: markdown
---
# Quickstart: Building a Semantic Layer with Data-Tools

This notebook provides a quick introduction to the `data-tools` project. You'll learn how to use its key features to automatically build a semantic layer over your data.
This notebook provides a quick introduction to the `intugle` project. You'll learn how to use its key features to automatically build a semantic layer over your data.

**What is a Semantic Layer?**

Expand Down Expand Up @@ -67,12 +67,12 @@ type: markdown
---
## 3. Data Profiling

The first step in building a semantic layer is to profile your data. This involves analyzing your data sources to understand their structure, data types, and other characteristics. The `data-tools` project provides a pipeline for this purpose.
The first step in building a semantic layer is to profile your data. This involves analyzing your data sources to understand their structure, data types, and other characteristics. The `intugle` project provides a pipeline for this purpose.
---
type: code
---
import pandas as pd
from data_tools.analysis.models import DataSet
from intugle.analysis.models import DataSet
---
type: code
---
Expand Down Expand Up @@ -109,7 +109,7 @@ type: markdown
---
## 4. Automated Link Prediction

Now that we've profiled our data, let's discover the relationships between tables. The `data-tools` project uses a Large Language Model (LLM) to predict links (foreign keys) between tables.
Now that we've profiled our data, let's discover the relationships between tables. The `intugle` project uses a Large Language Model (LLM) to predict links (foreign keys) between tables.

First, we'll load a few more tables from the sample dataset.
---
Expand All @@ -130,7 +130,7 @@ Now, let's run the link prediction pipeline.
---
type: code
---
from data_tools.link_predictor import LinkPredictor
from intugle.link_predictor import LinkPredictor

# Initialize the predictor
predictor = LinkPredictor(datasets)
Expand Down Expand Up @@ -168,7 +168,7 @@ Now, we can load the YAML files and create a manifest.
---
type: code
---
from data_tools.parser.manifest import ManifestLoader
from intugle.parser.manifest import ManifestLoader

# Load the manifest
manifest_loader = ManifestLoader('.')
Expand All @@ -188,7 +188,7 @@ Let's create an ETL model to define the query we want to generate.
---
type: code
---
from data_tools.libs.smart_query_generator.models.models import ETLModel, FieldModel
from intugle.libs.smart_query_generator.models.models import ETLModel, FieldModel

# Create an ETL model
etl_model = ETLModel(
Expand All @@ -205,7 +205,7 @@ Now, let's use the `SqlGenerator` to generate the SQL query.
---
type: code
---
from data_tools.sql_generator import SqlGenerator
from intugle.sql_generator import SqlGenerator

# Create a SqlGenerator
sql_generator = SqlGenerator('.')
Expand All @@ -220,12 +220,12 @@ type: markdown
---
## Conclusion

This notebook has provided a brief overview of the `data-tools` project. You've learned how to:
This notebook has provided a brief overview of the `intugle` project. You've learned how to:

* Configure your LLM provider.
* Profile your data to understand its characteristics.
* Use an LLM to automatically predict links between tables.
* Generate a semantic layer (`manifest.json`).
* Use the semantic layer to generate SQL queries.

This is just a starting point. The `data-tools` project has many other features to explore. We encourage you to try it with your own data and see how it can help you build a powerful semantic layer.
This is just a starting point. The `intugle` project has many other features to explore. We encourage you to try it with your own data and see how it can help you build a powerful semantic layer.
129 changes: 0 additions & 129 deletions notebooks/sql_generator.ipynb

This file was deleted.

Loading