CyNetDiff blog post

_posts/2025-09-21-cynetdiff.md

@@ -0,0 +1,61 @@
+---
+layout: single
+title: "CYNETDIFF: A Python Library for Accelerated Implementation of Network Diffusion Models"
+excerpt: CYNETDIFF is a new Python package for accelerating network diffusion simulations, recently accepted into the pyOpenSci ecosystem.
+author: Eliot W. Robson
+permalink: /blog/cynetdiff
+header:
+  overlay_image: /images/blog/headers/cynetdiff-header.png
+categories:
+  - blog-post
+  - cynetdiff
+  - network-diffusion
+  - pyos-accepted
+classes: wide
+comments: true
+last_modified: 2024-09-14
+---
+
+In recent years, network diffusion models, which are used to study how information and epidemics spread over social networks, have garnered increasing interest[cite: 1, 21]. [cite_start]Two of the most widely used models are the **independent cascade (IC)** and **linear threshold (LT) models**[cite: 12, 13, 23, 22]. [cite_start]Many research tasks related to these models, particularly influence maximization, require a large number of simulations on large graphs, which can be computationally expensive[cite: 12, 25, 29].
+
+[cite_start]While researchers often prefer high-level languages like Python for their flexibility and quick development, these computationally intensive tasks are better suited for low-level, compiled languages[cite: 14, 15, 68]. [cite_start]To bridge this gap, we've developed **CYNETDIFF**, a Python library that uses components written in Cython to provide the performance of a compiled language with the flexibility of a high-level language[cite: 16, 67, 68].
+
+---
+
+## Statement of Need
+
+[cite_start]Several Python libraries, such as **NDLIB**, have been created to simulate network diffusion[cite: 30, 31]. [cite_start]However, NDLIB, which is written in pure Python, suffers from significant memory overhead and slow execution, especially for iterative algorithms[cite: 35]. [cite_start]It's also inefficient because it loops through every node at each time step, even when only a few nodes are active[cite: 36].
+
+[cite_start]CYNETDIFF addresses these issues by offering a high-performance alternative for network diffusion tasks in the Python ecosystem[cite: 37, 81]. [cite_start]It's designed to be a "drop-in replacement" for NDLIB, with a focus on speed and lower memory usage[cite: 67, 81].
+
+---
+
+## Package Features
+
+[cite_start]The performance-critical parts of CYNETDIFF are written in **Cython**, a language extension that compiles to C and C++ while remaining callable from Python[cite: 69, 70, 71].
+
+[cite_start]To take advantage of Cython's speed, the library represents graphs using array-based data structures optimized for low-level languages[cite: 72, 73]. [cite_start]It uses the **compressed sparse row (CSR) format** to store graphs, which has a smaller memory footprint and allows for efficient, repeated traversals, making it ideal for diffusion simulations[cite: 74, 76, 77].
+
+[cite_start]The implementation also includes an important optimization: it uses a **BFS-based traversal algorithm** that only explores the out-neighbors of nodes that were activated in the previous iteration[cite: 85, 86]. [cite_start]This means the work done is proportional to the number of edges connected to activated nodes, which can be much smaller than the size of the entire graph, especially when there are few seed nodes[cite: 87, 88]. [cite_start]This is particularly beneficial for algorithms like CELF, which involve many simulations with small numbers of activated nodes[cite: 89, 90].
+
+[cite_start]The package also includes utility functions to easily convert well-established NetworkX graphs into the CSR format, allowing for seamless integration into existing research pipelines[cite: 78, 79, 80].
+
+---
+
+## Example Usage
+
+The library's performance can be demonstrated by comparing it to other implementations like NDLIB and a pure Python version. [cite_start]Below is an example of the output from the `simple_benchmark` function, which can be used to run comparative benchmarks on arbitrary graphs[cite: 96, 97, 98].
+
+The benchmarks show that CYNETDIFF is significantly faster. [cite_start]For instance, in a benchmark on a synthetic random graph with 1,000 nodes and 4,958 edges, CYNETDIFF took **0.0672 seconds** to run 1,000 trials, while a pure Python implementation took **8.9998 seconds** and NDLIB took **14.6245 seconds**[cite: 108, 109, 110, 111, 125, 127, 138].
+
+[cite_start]The speed of CYNETDIFF also makes it possible to create visualizations in real-time on large graphs[cite: 158]. [cite_start]For example, a heatmap can be generated by running many simulations and coloring nodes based on how often they were activated[cite: 160, 161].
+
+[cite_start]The library's performance is especially critical for tasks like **influence maximization**, which require a large number of influence function evaluations[cite: 62, 63, 63]. [cite_start]By using CYNETDIFF as a backend for the **CELF algorithm**, which is an optimized version of the greedy algorithm for influence maximization, the runtime is kept at a reasonable level[cite: 168, 170, 188, 189].
+
+---
+
+## Citing
+
+This post is adapted from our paper, which should be used for citations.
+
+**Eliot W. Robson, Dhemath Reddy, and Abhishek K. Umrawal. 2024. CYNETDIFF: A Python Library for Accelerated Implementation of Network Diffusion Models.**