Skip to content

Commit fc36c60

Browse files
RayMNRayMN
authored
Initial Upload
1 parent 4ba48fc commit fc36c60

File tree

6 files changed

+733
-2
lines changed

6 files changed

+733
-2
lines changed

‎ConfigInventory.jar‎

12.5 KB
Binary file not shown.

‎README.md‎

Lines changed: 42 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,42 @@
1-
# DataPower-Configuration-Inventory
2-
stuff here
1+
# DataPower-Gateway-Inventory
2+
Java based tool for getting a list of configured items from a DataPower whole appliance export.
3+
4+
## Synopsis
5+
This is a tool that extracts a list of all of gateways and "important" objects with optional details OR a list of all named objects (everything you have created) on a DataPower appliance from an export of the entire appliance.
6+
7+
## Code Example
8+
There are lots of comments in the .java files, I believe in heavy commenting, but the basic steps ***paraphrased*** are:
9+
```
10+
1) Open the parent zip file
11+
2) Gater some basic appliance information including a list of all domains from the export.xml file
12+
3) From that list of domains:
13+
4) Extract the zip file for each domain
14+
5) Get a the data for the list type selected for that domain
15+
6) Create a csv file with the above information. (actually happens throughout)
16+
7) Clean up all the extracted files
17+
```
18+
## Motivation
19+
A friend (and soon to be co-worker) asked me if there was a way to get a listing of all the gateways on a DataPower appliance. The short answer from IBM when I posted the question to Developer Works was "No, we recomment you open an RFE." Having been down that path and understanding the speed with which these things happen (that's not a dig, just a statement ... I realize there is a lot to do and it gets ***complicated*** when you are dealing with that amount of code) I decided that it would probably be best to do this outside of DataPower. I thought about using CLI or SSH and decided to keep it entirely off of DataPower and use the information found in the export.xml files in an export of the entire appliance.
20+
21+
## Installation
22+
This is a simple command line Java program, it does require Java v1.8+
23+
24+
## Usage
25+
```
26+
Usage: java -jar ConfigInventory.jar zipFile tmpDir csvFile [-h (-details | -all) -debug]");
27+
Where:");
28+
zipFile = Absolute path to the DataPower complete appliance export zip file");
29+
tmpDir = Absolute path to temporary directory to extract zip files including a tailing /");
30+
csvFile = Absolute path and name of the output csv file");
31+
-h = Optional: This message");
32+
-details = Optional: When present adds the object details to the output - ! used with -all");
33+
-all = Optional: When present generates a list of all named objects - ! used with -details");
34+
-debug = Optional: When present generates VERY verbose DEBUG messages in the console");
35+
36+
Example > java -jar /Classes/ConfigInventory.jar /dir/IDG-20-A.zip /dir/dp-export/ /dir/IDG-20-report.csv");
37+
```
38+
## Contributors
39+
The code may not be the ***slickest*** code ever written but I think it works well enough to get started and it's not really meant a high performance application. It's a tool. Like a hammer (mentioned above). I am more than open to constructive criticizm, suggestions for improvement, and/or help in improving the tool.
40+
41+
## License
42+
This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

‎org/prairieland/datapower/ConfigInventory/ConfigInventory.java‎

Lines changed: 340 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,340 @@
1+
/**
2+
* Module: ConfigInventory.java
3+
*
4+
* Written by Ray Wilson
5+
*
6+
* Description: Parse DataPower export files and put the data into a csv file.
7+
* The output file will be stored written in the base directory of the export.
8+
* This file can then be imported into your favorite spreadsheet.
9+
*
10+
* History:
11+
* 2017-07-06 v0.0.1 Created.
12+
* 2017-07-07 v0.1.0 Basic functionality achieved.
13+
* 2017-07-08 v1.0.0 Added auto extract and removal of extracted folders.
14+
* v2.0.0 Added Configuration Details option to the data collected on each gateway
15+
* Added the ability to turn on debug from the command line
16+
* 2017-07-25 v2.5.0 Moving it to the new package structure for PrairieLand.
17+
* Added a -all switch to get all named objects
18+
* Rewrote the core to use the DOM model and XML tools for java better.
19+
*
20+
* KNOWN ISSUES:
21+
* Not the most efficient file handling but it works and it's not a "performance"
22+
* software so for now I'm willing to live with it.
23+
*
24+
* Author: Ray Wilson
25+
* [email protected]
26+
*
27+
* Date: 2017-07-06
28+
*
29+
* Copyright (C) 2017 Paul Ray Wilson
30+
*
31+
* This program is free software: you can redistribute it and/or modify
32+
* it under the terms of the GNU General Public License as published by
33+
* the Free Software Foundation, either version 3 of the License, or
34+
* (at your option) any later version.
35+
*
36+
* This program is distributed in the hope that it will be useful,
37+
* but WITHOUT ANY WARRANTY; without even the implied warranty of
38+
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
39+
* GNU General Public License for more details.
40+
*
41+
* You should have received a copy of the GNU General Public License
42+
* along with this program. If not, see <http://www.gnu.org/licenses/>.
43+
*
44+
*/
45+
46+
package org.prairieland.datapower.ConfigInventory;
47+
import org.prairieland.util.pXML;
48+
import org.prairieland.util.pDPutil;
49+
import org.prairieland.util.pUtil;
50+
51+
import java.io.*;
52+
import java.util.Scanner;
53+
import java.nio.file.*;
54+
import org.w3c.dom.*;
55+
import java.nio.file.attribute.BasicFileAttributes;
56+
import java.util.stream.Collectors;
57+
import java.util.Enumeration;
58+
import java.util.zip.*;
59+
60+
public class ConfigInventory {
61+
public static String zipFile = "";
62+
public static String tmpDir = "";
63+
public static String csvFile = "";
64+
public static boolean DEBUG = false;
65+
public static boolean DETAILS = false;
66+
public static boolean GETALL = false;
67+
public static String EXP = "export.xml"; // Export file name to work with ... all of them should be the same.
68+
69+
/** main()
70+
*
71+
* This is the "main" method that checks for command line arguments and
72+
* for starting the createInventory method below where all the actual work will happen
73+
*
74+
*/
75+
public static void main(String arg[]) throws Exception {
76+
77+
// Validate the command line arguments.
78+
boolean argsValid = checkArgs(arg); // Check command line arugments
79+
80+
if (!argsValid) {
81+
System.out.println("\n===============================================================================");
82+
System.out.println(" ");
83+
System.out.println("Usage: java -jar ConfigInventory.jar zipFile tmpDir csvFile [-h (-details | -all) -debug]");
84+
System.out.println(" Where:");
85+
System.out.println(" zipFile = Absolute path to the DataPower complete appliance export zip file");
86+
System.out.println(" tmpDir = Absolute path to temporary directory to extract zip files including a tailing /");
87+
System.out.println(" csvFile = Absolute path and name of the output csv file");
88+
System.out.println(" -h = Optional: This message");
89+
System.out.println(" -details = Optional: When present adds the object details to the output - ! used with -all");
90+
System.out.println(" -all = Optional: When present generates a list of all named objects - ! used with -details");
91+
System.out.println(" -debug = Optional: When present generates VERY verbose DEBUG messages in the console");
92+
System.out.println(" ");
93+
System.out.println("Example > java -jar /Classes/ConfigInventory.jar /dir/IDG-20-A.zip /dir/dp-export/ /dir/IDG-20-report.csv");
94+
System.out.println(" ");
95+
System.out.println("===============================================================================\n");
96+
System.exit(0);
97+
} else {
98+
try {
99+
zipFile = arg[0];
100+
tmpDir = arg[1];
101+
csvFile = arg[2];
102+
103+
// This is where the work happens ....
104+
String result = createInventory();
105+
System.out.println(result);
106+
} catch (Exception e) {
107+
System.err.println(e.getMessage());
108+
System.exit(0);
109+
}
110+
}
111+
}
112+
/* ===== END OF public static void main(String arg[]) =====*/
113+
114+
115+
/** createInventory()
116+
*
117+
* This is the module where the following happens:
118+
* 1) The zip file from the DataPower Export is extracted
119+
* 2) Basic appliance information collected from export.xml in the root of the extract
120+
* 3) A list of domains is read from the export.xml file in the root of the extract
121+
* 4) For each domain in the list
122+
* 5) The zip file for that domain is extracted
123+
* 6) The export.xml file in that domain is examined and information on gateways collected.
124+
* 7) All the upziped files and folders are removed. (still pending)
125+
*
126+
* return - String - The results of the doWork opperation.
127+
*/
128+
public static String createInventory () {
129+
if(DEBUG) {System.out.println("DEBUG :: Entering createInventory");}
130+
131+
// Create helper classes
132+
pXML px = new pXML();
133+
pDPutil dp = new pDPutil();
134+
pUtil pu = new pUtil();
135+
136+
File fout = new File(csvFile);
137+
try {
138+
// Create the output file if it does not exist.
139+
if(!fout.exists()){
140+
fout.createNewFile();
141+
if(DEBUG){System.out.println("DEBUG :: Creating output file - " + csvFile);}
142+
}
143+
// Output file handle for writing
144+
BufferedWriter out = new BufferedWriter(new FileWriter(csvFile,true));
145+
146+
// Extract the main export file. It will contain multiple zip files with it for each domain
147+
// on the appliance. We will unzip those later in the loop where we gather the gateway informaiton
148+
pu.unZipIt(zipFile, tmpDir);
149+
150+
// ******
151+
// Get the data we want from the root export file about the appliance itself.
152+
// ******
153+
// Open the export file in the root folder to get some some of the data
154+
String content = new Scanner(new File(tmpDir+EXP)).useDelimiter("\\Z").next();
155+
156+
// Get the node that contains the export details from the root export.xml file
157+
NodeList epxDetails = px.getAllNodes(content,"export-details");
158+
159+
// Get the Date-Time of the export and write it to the file.
160+
String expDate = px.getAllNodes(content,"current-date").item(0).getTextContent();
161+
String expTime = px.getAllNodes(content,"current-time").item(0).getTextContent();
162+
out.write("Export Date, "+ expDate + " - " + expTime + "\n"); out.flush();
163+
if(DEBUG){ System.out.println("DEBUG :: Export Date :"+expDate+" "+expTime); }
164+
165+
// Get the Device Name and write it to the file.
166+
String deviceName = px.getAllNodes(content,"device-name").item(0).getTextContent();
167+
out.write("Device Name,"+deviceName+"\n"); out.flush();
168+
if(DEBUG){ System.out.println("DEBUG :: Device Name :"+deviceName ); }
169+
170+
// Get the Product ID and write it to the file.
171+
String productId = px.getAllNodes(content, "product-id").item(0).getTextContent();
172+
out.write("Product ID,"+productId+"\n"); out.flush();
173+
if(DEBUG){ System.out.println("DEBUG :: Product ID :"+productId ); }
174+
175+
// Get the serial number and write it to the file.
176+
String serialNum = px.getAllNodes(content,"serial-number").item(0).getTextContent();
177+
out.write("Serial Number,"+serialNum+"\n"); out.flush();
178+
if(DEBUG){ System.out.println("DEBUG :: Serial No :"+serialNum ); }
179+
180+
// Get the firmware version level and write it to the file.
181+
String firmwareVer = px.getAllNodes(content,"firmware-version").item(0).getTextContent();
182+
out.write("Firmware Version,"+firmwareVer+"\n"); out.flush();
183+
if(DEBUG){ System.out.println("DEBUG :: Firmware :"+firmwareVer ); }
184+
185+
// Get a list of domains on the appliance from the base export.xml file.
186+
NodeList domainList = px.getAllNodes(content,"domain");
187+
188+
// Write the header for the list of gateways based on the DETAILS switch.
189+
if(DETAILS){
190+
out.write("\n,\nDomain, Object Type, Object Name, Object Attribute, Attribute Value"); out.flush();
191+
} else {
192+
out.write("\n,\nDomain, Object Type, Object Name"); out.flush();
193+
}
194+
195+
// ******
196+
// Now get the object information from each domain.
197+
// ******
198+
for(int k = 1; k < domainList.getLength(); k++) {
199+
200+
String thisDomainName = px.getElementAtrribute(domainList.item(k), "name");
201+
if(DEBUG){ System.out.println("\n\nDEBUG :: ++++++++++\nDEBUG :: ++++++++++ Working in Domain :" + thisDomainName + " ++++++++++\nDEBUG :: ++++++++++"); }
202+
203+
// Extract the zip file for this domain
204+
String thisZipFile = tmpDir+thisDomainName+".zip";
205+
String thisTmpDir = tmpDir+thisDomainName+"/";
206+
pu.unZipIt(thisZipFile, thisTmpDir);
207+
208+
// Read in the export.xml file for this domain.
209+
String domainExportFile = thisTmpDir+"/"+EXP;
210+
File expFile = new File(domainExportFile);
211+
212+
// Create a workList of objects to return from each domain.
213+
String[] workList = null;
214+
if(GETALL){
215+
if(DEBUG) {System.out.println("DEBUG :: Using getAllObjectsList to generate the workList.");}
216+
workList = dp.getAllObjectsList(expFile);
217+
} else {
218+
if(DEBUG) {System.out.println("DEBUG :: Using OBJECT_LIST for workList.");}
219+
workList = dp.OBJECT_LIST;
220+
}
221+
222+
// Create on big string from the export.xml file for this domain.
223+
String domainContent = new Scanner(expFile).useDelimiter("\\Z").next();
224+
225+
// Write out the domain name to the file with a blank line before it.
226+
out.write(", \n"+thisDomainName+"\n"); out.flush();
227+
228+
// Generate an object list for each object in the dp.OBJECT_LIST ... found in pDPutil.java
229+
for(int l=0; l< workList.length; l++){
230+
if(DEBUG){ System.out.println("\nDEBUG :: +++++ Searching for " + workList[l] + " objects in " + thisDomainName); }
231+
232+
// The list of object for this OBJECT_LISTtype.
233+
NodeList thisList = px.getAllNodes(domainContent, workList[l]);
234+
if(DEBUG){ System.out.println("DEBUG :: Found " + thisList.getLength() + " object(s) of type : " + workList[l] ); }
235+
236+
// Check to see if there are any gateways in the list.
237+
if(thisList.getLength() > 0){
238+
for(int m = 0; m < thisList.getLength(); m++){
239+
// Only process elements that have name attribute (some are only for when the element is used ... bug in IBM export.xml format.)
240+
if (px.getElementAtrribute(thisList.item(m), "name").length() > 1) {
241+
if(DEBUG){ System.out.println("DEBUG :: === " + (m+1) + " === Working element -" + px.getElementAtrribute(thisList.item(m), "name")+"-"); }
242+
// Write the type and name of each gateway in the list out to the file.
243+
out.write("," + workList[l] + "," + px.getElementAtrribute(thisList.item(m), "name") + "\n"); out.flush();
244+
if(DETAILS){
245+
// Get the element for the object, including all child elements.
246+
if(DEBUG){ System.out.println("DEBUG :: Requesting Element: " + workList[l] + " with the name " + px.getElementAtrribute(thisList.item(m), "name")); }
247+
248+
// From the Node of this OBJECT get all of the applicable Attributes of the object from dp.DETAIL_LIST ... found in pDPutil.java
249+
for(int n = 0; n < dp.DETAIL_LIST.length; n++){
250+
NodeList aList = px.getAllNodes(px.xmlNodeToString(thisList.item(m)), dp.DETAIL_LIST[n]);
251+
for (int p = 0; p < aList.getLength(); p++) {
252+
if(DEBUG){ System.out.println("DEBUG :: Found " + dp.DETAIL_LIST[n] + " - " + aList.item(p).getTextContent()); }
253+
254+
// If the object admin state is disabled make a marker under the name to let someone know.
255+
if(dp.DETAIL_LIST[n].equals("mAdminState") && aList.item(p).getTextContent().equals("disabled")) {
256+
out.write(",X^_DISABLED_^X, ," + dp.DETAIL_LIST[n] + "," + aList.item(p).getTextContent() + "\n"); out.flush();
257+
} else {
258+
out.write(", , ," + dp.DETAIL_LIST[n] + "," + aList.item(p).getTextContent() + "\n"); out.flush();
259+
}
260+
}
261+
}
262+
}
263+
}
264+
}
265+
}
266+
}
267+
}
268+
// House keeping, Flush the buffer for the csvFile and close it.
269+
out.flush();
270+
out.close();
271+
// Remove the extracted files
272+
pu.killDir(tmpDir);
273+
} catch (Exception ex) {
274+
System.out.println(":: ERROR ::");
275+
ex.printStackTrace();
276+
}
277+
return("INVENTORY COMPLETE: The file "+csvFile+" has been created");
278+
}
279+
/* ===== END OF createInventory() =====*/
280+
281+
282+
/** checkArgs(String[])
283+
* This is method validates the command line arguments as best as we can.
284+
* Needless to say, it's barely idiot proof .. but there is no what it's stupid proof. :)
285+
* param - arg[]
286+
* return - boolean - true of no errors detected else false.
287+
*/
288+
public static boolean checkArgs(String[] arg){
289+
boolean valid = true;
290+
291+
if(arg.length < 3) { return false; }
292+
293+
// Check for the zip file
294+
File zFile = new File(arg[0]);
295+
if (!zFile.exists()) { valid = false; System.out.println("ERROR :: zipFile -" + arg[0] + "- is not valid"); }
296+
297+
// Check for a temporary directory name. It does not have to exist, so just check that it's on the command line.
298+
if(!(arg[1].length() > 0)) { valid = false; System.out.println("ERROR :: tmpDir -" + arg[1] + "- is not valid"); }
299+
300+
// Check for a csvFile name.
301+
if(!(arg[2].length() > 0)) { valid = false; System.out.println("ERROR :: csvFile -" + arg[2] + "- is not valid"); }
302+
303+
// OPTIONAL SWITCH PARAMETERS
304+
if(arg.length > 3){
305+
for(int j = 3; j < arg.length; j++) {
306+
switch (arg[j]) {
307+
case "-h": // Check for help switch
308+
valid=false;
309+
break;
310+
case "-details":
311+
DETAILS=true; // Set the DETAILS switch
312+
break;
313+
case "-all":
314+
GETALL=true; // Set the GETALL switch
315+
DETAILS=false; // unSet the DETAILS switch - really fubars the output if you use both.
316+
break;
317+
case "-debug":
318+
DEBUG=true; // Set the DEBUG switch
319+
break;
320+
default:
321+
System.out.println("ERROR :: switch -" + arg[j] + "- is not valid");
322+
valid=false;
323+
break;
324+
}
325+
}
326+
}
327+
if(DEBUG) {
328+
System.out.println();
329+
for(int i=0;i<arg.length;i++) { System.out.println("DEBUG :: arg " + i + " -" + arg[i] + "-"); }
330+
System.out.println("DEBUG :: DETAILS = " + DETAILS);
331+
System.out.println("DEBUG :: GETALL = " + GETALL);
332+
System.out.println("DEBUG :: DEBUG = " + DEBUG);
333+
}
334+
return valid;
335+
}
336+
/* ===== END OF checkArgs(String[]) =====*/
337+
338+
}
339+
/* ===== END OF Class ConfigInventory =====*/
340+

0 commit comments

Comments
 (0)