Updated Java code summarization and test generation examples

Signed-off-by: Saurabh Sinha <[email protected]>

Author: sinha108

docs/examples/java/notebook/code_summarization.ipynb

@@ -9,29 +9,20 @@
    "source": [
     "# Using CLDK to explain Java methods\n",
     "\n",
-    "In this tutorial, we will use CLDK to explain or generate code summary for all the methods in a Java Application.\n",
+    "In this tutorial, we will use CLDK to explain or generate code summary for a Java method. You'll explore some of the benefits of using CLDK to perform quick and easy program analysis and build an LLM-based code summarizer. By the end of this tutorial, you will have implemented such a tool and generated code summary for a Java method.\n",
     "\n",
-    "By the end of this tutorial, you will have code summary for all the methods in a Java application. You'll be able to explore some of the benefits of using CLDK to perform fast and easy program analysis and build a LLM-based code summary generation.\n",
+    "Specifically, you will learn how to perform the following tasks on a Java application to create LLM prompts for code summarization:\n",
     "\n",
-    "You will learn how to do the following:\n",
+    "1. Create a new instance of the CLDK class.\n",
+    "2. Create an analysis object for the target Java application.\n",
+    "3. Iterate over all files in the application.\n",
+    "4. Iterate over all classes in a file.\n",
+    "5. Initialize treesitter utils for the class content.\n",
+    "6. Iterate over all methods in a class.\n",
+    "7. Get the code body of a method.\n",
+    "8. Sanitize the class for prompting the LLM.\n",
     "\n",
-    "<ol>\n",
-    "<li> Create a new instance of the CLDK class.\n",
-    "<li> Create an analysis object over the Java application.\n",
-    "<li> Iterate over all the files in the project.\n",
-    "<li> Iterate over all the classes in the file.\n",
-    "<li> Iterate over all the methods in the class.\n",
-    "<li> Get the code body of the method.\n",
-    "<li> Initialize the treesitter utils for the class file content.\n",
-    "<li> Sanitize the class for analysis.\n",
-    "</ol>\n",
-    "Next, we will write a couple of helper methods to:\n",
-    "\n",
-    "<ol>\n",
-    "<li> Format the instruction for the given focal method and class.\n",
-    "<li> Prompts the local model on Ollama.\n",
-    "<li> Use CLDK to analyze code and get context information for generating code summary.\n",
-    "</ol>"
+    "We will write a couple of helper methods to (1) format the LLM instruction for summarizing a given target method and (2) prompt the LLM via Ollama. We will then use CLDK to go through an application and generate the summary for the target method."
    ]
   },
   {
@@ -45,12 +36,11 @@
     "\n",
     "Before we get started, let's make sure you have the following installed:\n",
     "\n",
-    "<ol>\n",
-    "<li> Python 3.11 or later\n",
-    "<li> Ollama 0.3.4 or later\n",
-    "<li> Java 11 or later\n",
-    "</ol>\n",
-    "We will use ollama to spin up a local granite model that will act as our LLM for this turorial."
+    "1. Python 3.11 or later (you can use [pyenv](https://github.com/pyenv/pyenv) to install Python)\n",
+    "2. Java 11 or later (you can use [SDKMAN!](https://sdkman.io) to instal Java)\n",
+    "3. Ollama 0.3.4 or later (you can get Ollama here: [Ollama download](https://ollama.com/download))\n",
+    "\n",
+    "We will use Ollama to spin up a local [Granite code model](https://ollama.com/library/granite-code), which will serve as our LLM for this turorial."
    ]
   },
   {
@@ -60,12 +50,28 @@
     "collapsed": false
    },
    "source": [
-    "### Prerequisite 1: Install ollama\n",
+    "### Download Granite code model\n",
     "\n",
-    "If you don't have ollama installed, please download and install it from here: [Ollama](https://ollama.com/download).\n",
-    "Once you have ollama, start the server and make sure it is running. Once ollama is up and running, you can download the latest version of the Granite 8b Instruct model by running the following command:\n",
-    "There are other granite versions available, but for this tutorial, we will use the Granite 8b Instruct model. You if prefer to use a different version, you can replace `8b-instruct` with any of the other [versions](https://ollama.com/library/granite-code/tags).\n",
-    "Let's make sure the model is downloaded by running the following command:"
+    "After starting the Ollama server, please download the latest version of the Granite code 8b-instruct model by running the following command. There are other Granite code models available, but for this tutorial, we will use Granite code 8b-instruct. If you prefer to use a different Granite code model, you can replace `8b-instruct` with the tag of another version (see [Granite code tags](https://ollama.com/library/granite-code/tags))."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "627e7184",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%bash\n",
+    "ollama pull granite-code:8b-instruct"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "8cc1ca5b",
+   "metadata": {},
+   "source": [
+    " Let's make sure the model is downloaded by running the following command:"
    ]
   },
   {
@@ -88,7 +94,7 @@
     "collapsed": false
    },
    "source": [
-    "### Prerequisite 3: Install ollama Python SDK"
+    "### Install Ollama Python SDK"
    ]
   },
   {
@@ -110,8 +116,8 @@
     "collapsed": false
    },
    "source": [
-    "### Prerequisite 4: Install CLDK\n",
-    "CLDK is avaliable on github at github.com/IBM/codellm-devkit.git. You can install it by running the following command:"
+    "### Install CLDK\n",
+    "CLDK is avaliable at https://github.com/IBM/codellm-devkit. You can install it by running the following command:"
    ]
   },
   {
@@ -134,7 +140,7 @@
    },
    "source": [
     "### Step 1: Get the sample Java application\n",
-    "For this tutorial, we will use apache commons cli. You can download the source code to a temporary directory by running the following command:"
+    "For this tutorial, we will use [Apache Commons CLI](https://github.com/apache/commons-cli) as the sample Java application. You can download the source code to a temporary directory by running the following command:"
    ]
   },
   {
@@ -157,7 +163,8 @@
     "collapsed": false
    },
    "source": [
-    "The project will now be extracted to `/tmp/commons-cli-rel-commons-cli-1.7.0`. We'll remove these files later, so don't worry about the location."
+    "The project will now be extracted to `/tmp/commons-cli-rel-commons-cli-1.7.0`.\n",
+    "<!-- We'll remove these files later, so don't worry about the location. -->"
    ]
   },
   {
@@ -167,12 +174,9 @@
     "collapsed": false
    },
    "source": [
-    "### Generate code summary\n",
-    "Code summarization or code explanation is a task that converts a code written in a programming language to a natural language. This particular task has several\n",
-    "benefits, such as understanding code without looking at its intrinsic details, documenting code for better maintenance, etc. To do that, one needs to\n",
-    "understand the basic details of code structure works, and use that knowledge to generate the summary using various AI-based approaches. In this particular\n",
-    "example, we will be using Large Language Models (LLM), specifically Granite 8B, an open-source model built by IBM. We will show how easily a developer can use\n",
-    "CLDK to expose various parts of the code by calling various APIs without implementing various time-intensive program analyses from scratch."
+    "## Generate code summary\n",
+    "\n",
+    "Code summarization or code explanation is the task of converting code written in a programming language to natural language. It has several benefits, such as understanding code without looking at its intrinsic details, documenting code for better maintenance, etc. To perform code summarization, one needs to understand the basic details of code implementation, and use that knowledge to generate the summary using various AI-based approaches. In this tutorial, we will use LLMs, specifically Granite code 8b-instruct. We will show how a developer can easily use CLDK to analyze code by calling various APIs without having to implement such analyses."
    ]
   },
   {
@@ -182,7 +186,7 @@
     "collapsed": false
    },
    "source": [
-    "Step 1: Add all the neccessary imports"
+    "Step 1: Add the neccessary imports"
    ]
   },
   {
@@ -194,7 +198,6 @@
    },
    "outputs": [],
    "source": [
-    "from pathlib import Path\n",
     "import ollama\n",
     "from cldk import CLDK\n",
     "from cldk.analysis import AnalysisLevel"
@@ -207,8 +210,7 @@
     "collapsed": false
    },
    "source": [
-    "Step 2: Formulate the LLM prompt. The prompt can be tailored towards various needs. In this case, we show a simple example of generating summary for each\n",
-    "method in a Java class"
+    "Step 2: Define a function for creating the LLM prompt, which instructs the LLM to summarize a Java method and includes relevant code for the task."
    ]
   },
   {
@@ -249,7 +251,7 @@
     "collapsed": false
    },
    "source": [
-    "Step 3: Create a function to call LLM. There are various ways to achieve that. However, for illustrative purpose, we use ollama, a library to communicate with models downloaded locally."
+    "Step 3: Define a function to call the LLM (in this case, Granite code 8b-instruct) using Ollama."
    ]
   },
   {
@@ -274,7 +276,7 @@
     "collapsed": false
    },
    "source": [
-    "Step 4: Create an object of CLDK and provide the programming language of the source code."
+    "Step 4: Create an instance of CLDK and provide the programming language of the source code."
    ]
   },
   {
@@ -286,7 +288,7 @@
    },
    "outputs": [],
    "source": [
-    "# Create a new instance of the CLDK class\n",
+    "# Create an instance of CLDK for Java analysis\n",
     "cldk = CLDK(language=\"java\")"
    ]
   },
@@ -297,10 +299,7 @@
     "collapsed": false
    },
    "source": [
-    "Step 5: CLDK uses different analysis engine--Codeanalyzer (built using WALA and Javaparser), Treesitter, and CodeQL (future). By default, codenanalyzer has\n",
-    "been selected as the default analysis engine. Also, CLDK support different analysis levels--(a) symbol table, (b) call graph, (c) program dependency graph, and\n",
-    "(d) system dependency graph. Analysis engine can be selected using ```AnalysisLevel``` enum. In this example, we will generate summarization of all the methods\n",
-    "of an application. "
+    "Step 5: Select the analysis engine and analysis level. CLDK uses different analysis engines---[CodeAnalyzer](https://github.com/IBM/codenet-minerva-code-analyzer) (built over [WALA](https://github.com/wala/WALA) and [JavaParser](https://github.com/javaparser/javaparser)), [Treesitter](https://tree-sitter.github.io/tree-sitter/), and [CodeQL](https://codeql.github.com/) (future)---with CodeAnalyzer being the default analysis engine. CLDK supports different analysis levels: (1) symbol table, (2) call graph, (3) program dependency graph, and (4) system dependency graph. The analysis level can be selected using the `AnalysisLevel` enumerated type. For this example, we select the symbol-table analysis level, with CodeAnalyzer as the default analysis engine."
    ]
   },
   {
@@ -323,9 +322,9 @@
     "collapsed": false
    },
    "source": [
-    "Step 6: Iterate over all the class files and create the prompt. In this case, we want to provide a customized Java class in the prompt. For instance,\n",
+    "Step 6: Iterate over all the class files and create the prompt. In this case, we want to provide a sanitized Java class in the prompt, containing only the relevant information for summarizing the target method. To illustrate, consider the floowing class:\n",
     "\n",
-    "```\n",
+    "```java\n",
     "package com.ibm.org;\n",
     "import A.B.C.D;\n",
     "...\n",
@@ -345,7 +344,7 @@
     "    // do somthing\n",
     " }   \n",
     "```\n",
-    "Given the above class, let's say we want to generate a summary for the ```bar``` method. To understand what it does, we add the callee of this method in the prompt, which in this case is ```baz```. We also remove imports, comments, etc. All of these are done using a single call to ```sanitize_focal_class``` API. In this process, we also use Treesitter to analyze the code. Once the input code has been sanitized, we call the ```format_inst``` method to create the LLM prompt, which has been passed to ```prompt_ollama``` method to generate the summary using LLM."
+    "Let's say we want to generate a summary for method `bar`. To understand what it does, we add the callees of this method in the prompt, which in this case includes `baz`. We remove the other methods, imports, comments, etc. All of this can be achieved with a single call to CLDK's `sanitize_focal_class` API. In this process, we also use Treesitter to analyze the code.  After creating the sanitized code, we call the previously defined `format_inst` method to create the LLM prompt and pass the prompt to `prompt_ollama` to generate the method summary."
    ]
   },
   {
@@ -357,30 +356,34 @@
    },
    "outputs": [],
    "source": [
-    "# For simplicity, we run the code summarization for a single class and method. One can remove that filter to run this code for the entire application\n",
-    "qualified_class_name = 'org.apache.commons.cli.GnuParser'\n",
-    "method_signature = 'flatten(Options, String[], boolean)'\n",
-    "# Iterate over all the files in the project\n",
+    "# For simplicity, we run the code summarization for a single class and method (this filter can be removed to run this code over the entire application)\n",
+    "target_class = \"org.apache.commons.cli.GnuParser\"\n",
+    "target_method = \"flatten(Options, String[], boolean)\"\n",
+    "\n",
+    "# Iterate over all classes in the application\n",
     "for class_name in analysis.get_classes():\n",
-    "    if class_name==qualified_class_name:\n",
+    "    if class_name == target_class:\n",
     "        class_file_path = analysis.get_java_file(qualified_class_name=class_name)\n",
-    "        # Iterate over all the methods in the class\n",
+    "\n",
+    "        # Read code for the class\n",
+    "        with open(class_file_path, 'r') as f:\n",
+    "            code_body = f.read()\n",
+    "\n",
+    "        # Initialize treesitter utils for the class file content\n",
+    "        tree_sitter_utils = cldk.tree_sitter_utils(source_code=code_body)\n",
+    "                \n",
+    "        # Iterate over all methods in class\n",
     "        for method in analysis.get_methods_in_class(qualified_class_name=class_name):\n",
-    "            if method==method_signature:\n",
-    "                # Get code body of the method\n",
-    "                with open(class_file_path, 'r') as f:\n",
-    "                    code_body = f.read()\n",
+    "            if method == target_method:\n",
     "        \n",
-    "                # Initialize the treesitter utils for the class file content\n",
-    "                tree_sitter_utils = cldk.tree_sitter_utils(source_code=code_body)\n",
-    "                \n",
     "                # Get all the method details\n",
     "                method_details = analysis.get_method(qualified_class_name=class_name,\n",
     "                                                     qualified_method_name=method)\n",
-    "                # Sanitize the class for analysis\n",
+    "                \n",
+    "                # Sanitize the class for analysis with respect to the target method\n",
     "                sanitized_class = tree_sitter_utils.sanitize_focal_class(method_details.declaration)\n",
     "        \n",
-    "                # Format the instruction for the given focal method and class\n",
+    "                # Format the instruction for the given target method and class\n",
     "                instruction = format_inst(\n",
     "                    code=sanitized_class,\n",
     "                    focal_method=method_details.declaration,\n",
@@ -417,7 +420,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.4"
+   "version": "3.11.9"
   }
  },
  "nbformat": 4,