Merge pull request #604 from subquery/feat/pipeline-rag-data

feat: pipeline rag data

Author: HuberTRoy

.github/workflows/deploy-rag.yml

@@ -0,0 +1,62 @@
+name: Deploy RAG Data - academy.subquery.network
+on:
+  push:
+    branches:
+      - master
+  workflow_dispatch:
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    env:
+      OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Install pnpm
+        uses: pnpm/action-setup@v2
+        with:
+          run_install: true
+
+      - name: Setup Deno
+        uses: denoland/setup-deno@v2
+        with:
+          deno-version: "2.2.2"
+          cache: pnpm
+
+      - name: Install subql-ai
+        run: deno install -g -f --allow-env --allow-sys --allow-net --allow-import --allow-read --allow-write --allow-ffi --allow-run --unstable-worker-options --no-prompt -n subql-ai jsr:@subql/ai-app-framework/cli
+
+      - name: Build rag
+        run: subql-ai embed-mdx -i docs -o dist -t subql-docs --model text-embedding-3-large --openAiApiKey $OPENAI_API_KEY
+
+      - name: Tar dist
+        run: tar -czvf db-openai.tar.gz dist
+
+      - name: Commit changes
+        uses: EndBug/add-and-commit@v5
+        with:
+          message: "Submit Build Rag Data"
+        env:
+          GITHUB_TOKEN: ${{ secrets.REPO_TOKEN }}
+
+      - name: Redeploy subquery-ai
+        run: |
+          curl --request POST \
+            --url 'https://api.bitbucket.org/2.0/repositories/onfinalitydev/pipeline-ai/pipelines' \
+            --header 'Authorization: Bearer ${{ secrets.BITBUCKET_ACCESS_TOKEN }}' \
+            --header 'Accept: application/json' \
+            --header 'Content-Type: application/json' \
+            --data '
+              { 
+                "target": {
+                "selector": {
+                    "type": "custom",
+                    "pattern": "only-redeploy"
+                },
+                "type": "pipeline_ref_target",
+                "ref_name": "main",
+                "ref_type": "branch"
+                }
+              }' 

.gitignore

@@ -4,6 +4,8 @@
 # Reference: https://intellij-support.jetbrains.com/hc/en-us/articles/206544839
 .idea
 
+temp
+
 # CMake
 cmake-build-*/
 

db-openai.tar.gz

@@ -28,3 +28,50 @@ Depends on the provider, SubQuery don't dictate this. But SubQuery gateway encou
 ## Is there a testnet environment?
 
 Yes, be sure to check https://dev.thechaindata.com/
+
+## Is there a meaning behind SubQuery's name?
+
+The name "SubQuery" refers to its function in the blockchain ecosystem. A subquery is essentially a query within another query, allowing users to access data more efficiently. It is a powerful tool for indexing and querying blockchain data.
+
+## What was the first network that SubQuery first supported?
+
+SubQuery initially supported the Polkadot network, focusing on indexing and querying data for the Polkadot and its parachains.
+
+## How many networks do SubQuery support now?
+
+As of now, SubQuery supports multiple networks, including Polkadot, Kusama, Ethereum, and many others, expanding its reach to a wide array of blockchain ecosystems.
+
+## How does SubQuery improve blockchain data access?
+
+SubQuery enhances blockchain data access by providing an indexing layer that aggregates and organizes data from blockchain networks. This allows developers and applications to quickly retrieve and interact with data in a structured and efficient manner.
+
+## How does SubQuery differ from traditional block explorers?
+
+Traditional block explorers allow users to explore blockchain data in a static, non-queryable format. SubQuery, on the other hand, provides dynamic and customizable data querying, giving developers the ability to filter, process, and retrieve data in real-time for their applications.
+
+## What programming languages does SubQuery support?
+
+SubQuery primarily supports JavaScript and TypeScript for building queries and processing data. Developers can use these languages to define the structure and logic for data retrieval from supported networks.
+
+## Is SubQuery open-source?
+
+Yes, SubQuery is open-source. It encourages developers to contribute to its growth and use its tools to build decentralized applications (dApps) more efficiently. The source code is available on GitHub.
+
+## What are the main use cases for SubQuery?
+
+SubQuery is used for various purposes, such as:
+- Building decentralized applications (dApps) that need to access blockchain data.
+- Indexing and querying blockchain data for analysis and reporting.
+- Creating custom data solutions for blockchain ecosystems and decentralized finance (DeFi) platforms.
+
+## Can I deploy SubQuery on my own infrastructure?
+
+Yes, SubQuery can be deployed on your own infrastructure, whether on-premise or in the cloud. It offers flexibility to users who want complete control over their indexing and querying solutions.
+
+## How does SubQuery handle data updates and changes?
+
+SubQuery continuously updates its data by monitoring blockchain events in real-time. This ensures that the indexed data is always up to date, and any changes to the blockchain are reflected immediately in the data queries.
+
+## How can I get started with SubQuery?
+
+To get started with SubQuery, you can visit their official documentation, which provides step-by-step guides on setting up a project, writing queries, and deploying them on supported networks. The community also offers support through forums and Discord.