feat(semsearch): example semantic search

Author: Daniele Briggi

examples/semantic_search/README.md

@@ -0,0 +1,56 @@
+# Semantic Search Example with sqlite-vector
+
+This example in Python demonstrates how to build a semantic search engine using the [sqlite-vector](https://github.com/sqliteai/sqlite-vector) extension and a Sentence Transformer model. It allows you to index and search documents using vector similarity, powered by a local LLM embedding model.
+
+### How it works
+
+- **Embeddings**: Uses [sentence-transformers](https://huggingface.co/sentence-transformers) to generate dense vector representations (embeddings) for text. The default model is [`all-MiniLM-L6-v2`](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2), a fast, lightweight model (384 dimensions) suitable for semantic search and retrieval tasks.
+- **Vector Store and Search**: Embeddings are stored in SQLite using the [`sqlite-vector`](https://github.com/sqliteai/sqlite-vector) extension, enabling fast similarity search (cosine distance) directly in the database.
+- **Sample Data**: The `samples/` directory contains example documents you can index and search immediately.
+
+### Installation
+
+1. Download the `sqlite-vector` extension for your platform [here](https://github.com/sqliteai/sqlite-vector/releases).
+
+2. Extract the `vector.so` file in the main directory of the project.
+
+3. Install the dependencies:
+
+
+```bash
+$ python -m venv venv
+
+$ source venv/bin/activate
+
+$ pip install -r requirements.txt
+```
+
+4. On first use, the required model will be downloaded automatically.
+
+### Usage
+
+Use the interactive mode to keep the model in memory and run multiple queries efficiently:
+
+```bash
+python semsearch.py --repl
+
+# Index a directory of documents
+semsearch> index ./samples
+
+# Search for similar documents
+semsearch> search "neural network architectures for image recognition"
+```
+
+### Example Queries
+
+Try these queries to test semantic similarity:
+
+- "neural network architectures for image recognition"
+- "reinforcement learning in autonomous systems"
+- "explainable artificial intelligence methods"
+- "AI governance and regulatory compliance"
+- "network intrusion detection systems"
+
+**Note:**
+- Supported extension are `.md`, `.txt`, `.py`, `.js`, `.html`, `.css`, `.sql`, `.json`, `.xml`.
+- For more details, see the code in `semsearch.py` and `semantic_search.py`.

examples/semantic_search/requirements.txt

@@ -0,0 +1 @@
+sentence-transformers

examples/semantic_search/samples/sample-1.md

@@ -0,0 +1,3 @@
+# Article 1: Deep Learning Neural Networks
+
+Deep learning utilizes artificial neural networks with multiple layers to process and learn from vast amounts of data. These networks automatically discover intricate patterns and representations without manual feature engineering. Convolutional neural networks excel at image recognition tasks, while recurrent neural networks handle sequential data like text and speech. Popular frameworks include TensorFlow, PyTorch, and Keras. Deep learning has revolutionized computer vision, natural language processing, and speech recognition applications.

examples/semantic_search/samples/sample-10.md

@@ -0,0 +1,3 @@
+# Article 10: Zero Trust Security Architecture
+
+Zero trust security operates on the principle of "never trust, always verify," requiring authentication and authorization for every access request regardless of location. This approach assumes breach scenarios and implements continuous verification throughout the network. Key components include identity verification, device compliance checking, least privilege access, and micro-segmentation. Zero trust frameworks help organizations protect against insider threats and advanced persistent attacks.

examples/semantic_search/samples/sample-11.md

@@ -0,0 +1,3 @@
+# Article 11: Incident Response and Recovery
+
+Effective incident response requires predefined procedures for detecting, containing, and recovering from security breaches. Response teams follow structured phases: preparation, identification, containment, eradication, recovery, and lessons learned. Critical activities include forensic analysis, stakeholder communication, system restoration, and process improvement. Regular tabletop exercises and response plan updates ensure organizations can quickly minimize damage and restore normal operations after security incidents.

examples/semantic_search/samples/sample-12.md

@@ -0,0 +1,3 @@
+# Article 12: Machine Learning for Malware Detection
+
+Machine learning enhances malware detection by analyzing file characteristics, behavioral patterns, and network communications to identify threats. Static analysis examines file properties without execution, while dynamic analysis observes runtime behavior in controlled environments. Ensemble methods combining multiple algorithms improve detection accuracy and reduce false positives. AI-powered systems can identify zero-day threats and polymorphic malware that traditional signature-based solutions miss.

examples/semantic_search/samples/sample-13.md

@@ -0,0 +1,3 @@
+# Article 13: Behavioral Analytics for Anomaly Detection
+
+Behavioral analytics leverages machine learning to establish baseline patterns of normal user and system behavior, flagging deviations that may indicate security threats. User and entity behavior analytics (UEBA) systems monitor login patterns, data access, and application usage to detect insider threats and compromised accounts. Machine learning models adapt to changing behavior patterns while maintaining sensitivity to subtle anomalies that human analysts might overlook.

examples/semantic_search/samples/sample-14.md

@@ -0,0 +1,3 @@
+# Article 14: AI-Driven Security Orchestration
+
+Security orchestration platforms integrate multiple security tools and automate incident response workflows using artificial intelligence. These systems correlate alerts from various sources, prioritize threats based on risk assessment, and execute automated remediation actions. Natural language processing helps analyze threat intelligence reports, while machine learning improves decision-making accuracy over time. Orchestration reduces response times and analyst workload while maintaining consistent security procedures.

examples/semantic_search/samples/sample-15.md

@@ -0,0 +1,3 @@
+# Article 15: Advanced Persistent Threats (APTs)
+
+Advanced persistent threats represent sophisticated, long-term cyberattacks typically conducted by nation-states or organized criminal groups. APTs use multiple attack vectors, maintain persistent access, and employ stealth techniques to avoid detection. Common tactics include spear-phishing, zero-day exploits, living-off-the-land techniques, and lateral movement within networks. Defense requires continuous monitoring, threat hunting, and intelligence-driven security strategies to detect and neutralize these patient adversaries.

examples/semantic_search/samples/sample-16.md

@@ -0,0 +1,3 @@
+# Article 16: Social Engineering Attack Vectors
+
+Social engineering exploits human psychology rather than technical vulnerabilities to gain unauthorized access to systems and information. Common techniques include phishing emails, pretexting phone calls, baiting with infected media, and physical tailgating. Attackers research targets through social media and public information to craft convincing scenarios. Defense requires security awareness training, verification procedures, and creating organizational cultures that encourage reporting suspicious communications.