adding function for difference tree

Author: vsoch

README.md

@@ -30,10 +30,10 @@ After installation, you should be able to run `shub` on the command line, withou
 	Please specify one or more containers with --image(s)
 
 
-      $ shub --help
+      $  shub --help
 	usage: shub [-h] [--image IMAGE] [--images IMAGES] [--debug]
 		    [--outfolder OUTFOLDER] [--package] [--tree] [--simtree]
-		    [--simcalc] [--size SIZE]
+		    [--subtract] [--simcalc] [--size SIZE]
 
 	Singularity Hub command line tool
 
@@ -49,6 +49,8 @@ After installation, you should be able to run `shub` on the command line, withou
 	  --package             package a singularity container for singularity hub
 	  --tree                view the guts of an singularity image (use --image)
 	  --simtree             view common guts between two images (use --images)
+	  --subtract            subtract one container image from the second to make
+                                a difference tree (use --images first,subtract)
 	  --simcalc             calculate similarity (number) between images based on
 		                file contents.
 	  --size SIZE           If using Docker or shub image, you can change size
@@ -114,10 +116,12 @@ This will open up something that looks like this:
 An [interactive demo](https://singularityware.github.io/singularity-python/examples/container_tree) is also available, and see the [example](examples/container_tree) for updates.
 
 
-### Compare Containers
+### Visualize Containers
 
 #### Container Similarity Tree
 
+![examples/similar_tree/simtree.png](examples/similar_tree/simtree.png)
+
 What do two containers have in common, in terms of files and folders? shub provides a command line function for rendering a view to (immediately) show the similarity between to container images:
 
 
@@ -139,26 +143,52 @@ Or two Docker images:
 If you need output for any of the following, you can add the `--debug` argument. Note that when generating docker comparisons, the back end is obtaining the layers, creating the images, importing and packaging, so the result is not instantanous.
 
 
+An [interactive demo](https://singularityware.github.io/singularity-python/examples/similar_tree/) is also available.
+
+
+#### Container Difference Tree
+What files and folders differ between two containers? What does it look like if I subtract one image from the second? `shub` provides a command line tool to generate a visualization to do exactly this.
+
+
+      shub --subtract --images docker://ubuntu:latest.docker://centos:latest
+
+As with `simtree`, this function supports both docker and singularity images as inputs.
+
+![examples/difference_tree/difftree.png](examples/difference_tree/difftree.png)
+
+An [interactive demo](https://singularityware.github.io/singularity-python/examples/difference_tree/) is also available.
+
+
+### Compare Containers
+The same functions above can be used to show the exact similarities (intersect) and differences (files and/or folders unique to two images) between two images. You can get a data structure with this information as follows:
+
+
+      from shub.views import compare_containers
+  
+      image1 = 'ubuntu.img'
+      image2 = 'centos.img'
+      by = "files.txt" # can also be "folders.txt", or a list with both
+
+      comparison = compare_containers(image1,image2,by=by)
+
+
 
 #### Calculate similarity of images
 
 We can calculate similarity of images based on the file content inside. For an example, see [examples/calculate_similarity](examples/calculate_similarity). We can compare two local images as follows:
 
       $ shub --images /home/vanessa/Desktop/ubuntu.img,/home/vanessa/Desktop/ubuntu.img --simcalc
       
-and the same applies for specification of Docker images, as in the previous example. Note that we are specifying `images` for the argument instead of `image`, and it's a single string of image names separated by a comma. For this argument you can specify an image or package.
+and the same applies for specification of Docker images, as in the previous example. Note that we are specifying `images` for the argument instead of `image`, and it's a single string of image names separated by a comma. 
 
-![examples/similar_tree/simtree.png](examples/similar_tree/simtree.png)
-
-An [interactive demo](https://singularityware.github.io/singularity-python/examples/similar_tree/) is also available.
 
 
 ### Build your container
 More information coming soon.
 
 
 ### Functions Provided
-You can also use the library as a module, and import singularity-python functions into your application.
+You can also use the library as a module, and import singularity-python functions into your application. If you would like to see specific examples for something, [please ask](https://github.com/singularityware/singularity-python)!
 
 
 ## Help and Contribution

setup.py

@@ -7,7 +7,7 @@
     name="singularity",
 
     # Version number:
-    version="0.34",
+    version="0.36",
 
     # Application author details:
     author="Vanessa Sochat",

singularity/app.py

@@ -1,6 +1,7 @@
 from singularity.views import (
     container_tree, 
-    container_similarity
+    container_similarity,
+    container_difference
 )
 
 from flask import (
@@ -51,30 +52,37 @@ def app_container_tree():
                                                  files=app.viz['files'],
                                                  container_name=container_name)
 
-#@app.route('/container/difftree')
-#def difference_tree():
-#    # The server will store the package name and result object for query
-#    if app.viz == None:
-#        app.viz = diff_tree(app.packages[0],app.packages[1])
-#    container1_name = os.path.basename(app.packages[0]).split(".")[0]
-#    container2_name = os.path.basename(app.packages[1]).split(".")[0]
-#    title = "%s minus %s" %(container1_name,container2_name)
-#    return render_template('container_tree.html',graph=app.viz['graph'],
-#                                                 files=app.viz['files'],
-#                                                 container_name=title)
+@app.route('/containers/subtract')
+def difference_tree():
+    # The server will store the package name and result object for query
+    if app.viz == None:
+        app.viz = container_difference(app.images[0],app.images[1])
+    container1_name,container2_name = get_container_names()
+    title = "%s minus %s" %(container1_name,container2_name)
+    return render_template('container_tree.html',graph=app.viz['graph'],
+                                                 files=app.viz['files'],
+                                                 container_name=title)
 
 @app.route('/containers/similarity')
 def app_similar_tree():
     # The server will store the package name and result object for query
     if app.viz == None:
         app.viz = container_similarity(app.images[0],app.images[1])
-    container1_name = os.path.basename(app.images[0]).split(".")[0]
-    container2_name = os.path.basename(app.images[1]).split(".")[0]
+    container1_name,container2_name = get_container_names()
     title = "%s INTERSECT %s" %(container1_name,container2_name)
     return render_template('container_tree.html',graph=app.viz['graph'],
                                                  files=app.viz['files'],
                                                  container_name=title)
 
+def get_container_names():
+    '''return container names for one or more images, depending on what
+    app initialized for.
+    '''
+    if app.images != None:
+        container1_name = os.path.basename(app.images[0]).split(".")[0]
+        container2_name = os.path.basename(app.images[1]).split(".")[0]
+        return container1_name,container2_name
+    return None
 
 # START FUNCTIONS ##################################################
 
@@ -97,6 +105,16 @@ def make_sim_tree(image1,image2,port=None):
     webbrowser.open("http://localhost:%s/containers/similarity" %(port))
     app.run(host="0.0.0.0",debug=False,port=port)
 
+# Function to make difference tree for two images
+def make_diff_tree(image1,image2,port=None):
+    app.images = [image1,image2]
+    if port==None:
+        port=8088
+    print("Pandas... just let them go.")
+    webbrowser.open("http://localhost:%s/containers/subtract" %(port))
+    app.run(host="0.0.0.0",debug=True,port=port)
+
+
 if __name__ == '__main__':
     app.debug = False
     app.run(host='0.0.0.0')

singularity/scripts.py

@@ -50,6 +50,11 @@ def get_parser():
                         help="view common guts between two images (use --images)", 
                         default=False, action='store_true')
 
+    # Compare two images (a similarity tree)
+    parser.add_argument('--subtract', dest='subtract', 
+                        help="subtract one container image from the second to make a difference tree (use --images first,subtract)", 
+                        default=False, action='store_true')
+
     # Compare two images (get numerical comparison)
     parser.add_argument('--simcalc', dest='simcalc', 
                         help="calculate similarity (number) between images based on file contents.", 
@@ -110,6 +115,7 @@ def main():
                make_tree(image)
                clean_up(image,existed)
 
+
            # The user wants to package the image
            elif args.package == True:
                from singularity.package import package
@@ -138,11 +144,17 @@ def main():
                bot.logger.error("Cannot find image. Exiting.")
                sys.exit(1)
 
-           # the user wants to make a tree
+           # the user wants to make a similarity tree
            if args.simtree == True:
                from singularity.app import make_sim_tree
                make_sim_tree(image1,image2)
 
+           # the user wants to make a difference tree
+           if args.subtract == True:
+               from singularity.app import make_diff_tree
+               make_diff_tree(image1,image2)
+
+
            if args.simcalc == True:
                from singularity.views import calculate_similarity
                score = calculate_similarity(image1,image2,by="files.txt")

singularity/views.py

@@ -25,7 +25,7 @@
 # COMPARISON FUNCTIONS ############################################################
 ###################################################################################
 
-def compare_containers(container1,container2,by="files.txt"):
+def compare_containers(container1,container2,by=None):
     '''compare_containers will generate a data structure with common and unique files to
     two images. If environmental variable SINGULARITY_HUB is set, will use container
     database objects.
@@ -35,6 +35,8 @@ def compare_containers(container1,container2,by="files.txt"):
     :param by: what to compare, one or more of 'files.txt' or 'folders.txt'
     default compares just files
     '''
+    if by == None:
+        by = ["files.txt"]
     if not isinstance(by,list):
         by = [by]
 
@@ -65,19 +67,6 @@ def compare_containers(container1,container2,by="files.txt"):
     return comparisons
 
 
-def container_similarity(container1,container2):
-    '''container_sim will return a data structure to render an html tree (graph) of the intersection (commonalities) between two images or packages
-    :param container1: the first container object
-    :param container2: the second container object
-    '''
-    comparison = compare_containers(container1,container2,
-                                    by=['files.txt','folders.txt'])
-    files = comparison["files.txt"]['intersect']
-    folders = comparison['folders.txt']['intersect']
-    tree = make_container_tree(folders=folders,
-                               files=files)
-    return tree
-
 
 def calculate_similarity(container1,container2,by="files.txt"):
     '''calculate_similarity will calculate similarity of two containers by files content, default will calculate
@@ -155,6 +144,39 @@ def get_container_contents(container,gets=None,split_delim=None):
 # COMPARISON TREES
 ###################################################################################
 
+
+def container_difference(container,container_subtract):
+    '''container_difference will return a data structure to render an html 
+    tree (graph) of the differences between two images or packages. The second
+    container is subtracted from the first
+    :param container: the primary container object (to subtract from)
+    :param container_subtract: the second container object to remove
+    '''
+    comparison = compare_containers(container,container_subtract,
+                                    by=['files.txt','folders.txt'])
+    files = comparison["files.txt"]['unique1']
+    folders = comparison['folders.txt']['unique1']
+    tree = make_container_tree(folders=folders,
+                               files=files)
+    return tree
+
+
+
+def container_similarity(container1,container2):
+    '''container_sim will return a data structure to render an html tree 
+    (graph) of the intersection (commonalities) between two images or packages
+    :param container1: the first container object
+    :param container2: the second container object
+    '''
+    comparison = compare_containers(container1,container2,
+                                    by=['files.txt','folders.txt'])
+    files = comparison["files.txt"]['intersect']
+    folders = comparison['folders.txt']['intersect']
+    tree = make_container_tree(folders=folders,
+                               files=files)
+    return tree
+
+
 def container_tree(container):
     '''tree will render an html tree (graph) of a container
     '''