Bug Fix

issue with true result in case command was ignored,
closing file to prevent potential blocking on io operation,
true transmitted status if dmesg capturing skipped,
issue with a specified thread dump file,
closing javacore file to prevent potential blocking on io operation

Author: ycrash

README.md

@@ -1,75 +1,82 @@
-# yc-data-script
-
-![img](/docs/images/360-degree.png)
-
-## What does the yc-data-script do?
-
-yc-data-script is a simple Golang script that captures 16 different artifacts from your application in a **pristine** manner. These artifacts will be highly useful to troubleshoot performance problems. Below is the list of artifacts captured:
-
-1. Garbage collection log
-2. Thread dump
-3. Heap dump
-4. Heap substitute
-5. top
-6. ps
-7. top -H
-8. Disk usage
-9. dmesg
-10. netstat
-11. ping
-12. vmstat
-13. iostat
-14. Kernel parameters
-15. Application Log
-16. Metadata
-
-## How to run the yc-data-script?
-
-1. Download latest yc-data-script from [this](https://tier1app.com/dist/ycrash/yc-agent-latest.zip) location
-2. Unzip the downloaded ```yc-agent-latest.zip``` file. (Say you are unzipping in '/opt/workspace/yc-agent-latest' folder)
-3. In the unzipped folder you will find yc-data-script by operating system:
-
-a) ```linux/yc``` - If you are running on Unix/Linux, then use this script.
-
-b) ```windows/yc.exe``` - If you are running on Windows, then use this script.
-
-c) ```mac/yc``` - If you are running on MAC, then use this script.
+### Usage of argument options:
+```bash
+Usage of yc:
+  -a string
+        The APP Name of the target
+  -c string
+        The config file path to load
+  -cmd value
+        The command to be executed, should be paired with '-urlParams' together
+  -d    Delete logs folder created during analyse, default is false
+  -gcPath string
+        The gc log file to be uploaded while it exists
+  -hd
+        Capture heap dump, default is false
+  -hdPath string
+        The heap dump file to be uploaded while it exists
+  -j string
+        The java home path to be used. Default will try to use os env 'JAVA_HOME' if 'JAVA_HOME' is not empty, for example: /usr/lib/jvm/java-8-openjdk-amd64
+  -k string
+        The API Key that will be used to make API requests, for example: tier1app@12312-12233-1442134-112
+  -p int
+        The process Id of the target, for example: 3121
+  -s string
+        The server url that will be used to upload data, for example: https://ycrash.companyname.com
+  -tdPath string
+        The thread dump file to be uploaded while it exists
+  -urlParams value
+        The params to be added at the end of upload request url, should be paired with '-cmd' together
+  -version
+        Show the version of this program
 
 ```
-./yc -j {JAVA_HOME} -onlyCapture -p {PID} -hd
+### Example of config file:
+```yaml
+version: "1"
+options:
+  a: name
+  d: false
+  hd: false
+  j: /usr/lib/jvm/java-8-openjdk-amd64
+  k: buggycompany@e094aasdsa-c3eb-4c9a-8254-f0dd107245cc
+  p: 3121
+  s: https://gceasy.io
+  gcPath: /var/log/gc.log
+  hdPath: /var/log/heapdump.log
+  tdPath: /var/log/threaddump.log
+  cmds:
+  - urlParams: dt=vmstat
+    cmd: vmstat 1 1
 ```
-Where,
-
-**JAVA_HOME** is the home directory where JDK is installed
-
-**PID** is the target JVM's process ID
-
-**Example:**
-
-```
-./yc -j /usr/java/jdk1.8.0_141 -onlyCapture -p 15326 -hd
-```
-
-When you pass the above arguments, yc-data-script will capture all the application level and system level artifacts/logs from the server from the target JVM & host for analysis. Captured artifacts will be compressed into a zip file and stored in the current directory where the above command was executed. The zip file will have the name in the format: 'yc-YYYY-MM-DDTHH-mm-ss.zip'. 
-    
-**Example:** 'yc-2021-03-06T14-02-42.zip'.
-
-## How to analyze the artifacts generated by the yc-data-script?
-
-You can analyze the artifacts captured by yc-data-script either manually or through [yCrash server](https://ycrash.io/). yCrash server analyzes all the captured data and generates a root cause analysis report instantly. You can use the [Bundle upload](https://docs.ycrash.io/ycrash-features/bundle-upload.html#step-1-go-to-upload-incident-form) feature in the yCrash server to analyze the captured 360-degree data. 
-
-### Advanced launch modes
-
-You can launch yc-data-script in following [3 different modes](https://docs.ycrash.io/ycrash-agent/launch-modes.html#launch-modes):
-
-1. **On-demand Mode:** In this mode you can directly transmit 360-degree artifacts from your server to yCrash server for analysis.
-2. **API Mode:** In this mode you can integrate yc-data-script with your current monitoring tools such as AppDynamics, New Relic, Dynatrace, …
-3. **M3 (Micro-metrics Monitoring) mode:** In this mode, yc-data-script proactively detect performance outages much earlier before it surfaces
-
-## How to build the yc-data-script?
-
-Please refer to any one of the following links if you want to build the yc-data-script in that corresponding operating system:
 
-1. Build yc-data-script in [Windows](/docs/Build%20yc%20agent%20in%20Windows.pdf)
-2. Build yc-data-script in [Linux](/docs/build-yc-agent-linux.md)
-3. Build yc-data-script in [MacOS](/docs/build-yc-agent-macos.md)
+The config file is using yaml format. The name of the option keys is same as the name of argument options.
+
+'-s': the server url that will be used to upload data.  
+'-k': the API key that will be used to make API requests.
+'-j': the java home path to be used. Default will try to use os env 'JAVA_HOME' if 'JAVA_HOME' is not empty.
+'-a': the app name of the target.  
+'-p': the pid of the target.  
+'-d': delete logs folder created during analyse, default is false.  
+'-hd': capture heap dump, default is false.  
+'-gcPath': the gc log file to be uploaded while it exists, otherwise it will captures one if failed to get the path from '-Xlog:gc' or '-Xloggc'.  
+'-hdPath': the heap dump file to be uploaded while it exists.  
+'-tdPath': the thread dump file to be uploaded while it exists, otherwise it will captures one.  
+
+Only for argument options:  
+'-version' show the version of this program.  
+'-c': the config file path to load.  
+
+### Example to capture info from target with pid 3121:
+
+`yc -p 3121 -s https://gceasy.io -k testCompany@e094a34e-c3eb-4c9a-8254-f0dd107245cc -j /usr/lib/jvm/java-11-openjdk-amd64 -c ./config.yaml`
+
+### Example to execute custom commands after the capturing:
+
+- By arguments. One '-urlParams' should be paired with one '-cmd'.  
+`yc ... -urlParams dt=vmstat -cmd "vmstat 1 1" -urlParams dt=pidstat -cmd "pidstat 1 1" ...` 
+- By config file. 
+```yaml
+  cmds:
+  - urlParams: dt=vmstat
+    cmd: vmstat 1 1
+```

capture/custom.go

@@ -26,7 +26,7 @@ func (c *Custom) Run() (result Result, err error) {
 	}
 	if c.Cmd.IsSkipped() {
 		result.Msg = "skipped capturing custom"
-		result.Ok = true
+		result.Ok = false
 		return
 	}
 	c.Cmd.Wait()

capture/gc.go

@@ -77,6 +77,12 @@ func (t *GC) Run() (result Result, err error) {
 		}
 	}
 
+	if gcFile != nil {
+		defer func() {
+			_ = gcFile.Close()
+		}()
+	}
+
 	result.Msg, result.Ok = shell.PostData(t.Endpoint(), "gc", gcFile)
 	absGCPath, err := filepath.Abs(t.GCPath)
 	if err != nil {

capture/jstack.go

@@ -92,21 +92,11 @@ func (t *JStack) Run() (result Result, err error) {
 					return
 				}
 
-				defer func() {
-					e := jstackFile.Sync()
-					if e != nil {
-						logger.Log("failed to sync file %s", e)
-					}
-					e = jstackFile.Close()
-					if e != nil {
-						logger.Log("failed to close file %s", e)
-					}
-				}()
-
 				_, e := jstackFile.WriteString("\nFull thread dump\n")
 				if e != nil {
 					logger.Log("failed to write file %s", e)
 					e1 <- e
+					_ = jstackFile.Close()
 					return
 				}
 				_, err = (&JStackF{
@@ -117,13 +107,18 @@ func (t *JStack) Run() (result Result, err error) {
 				if err != nil {
 					logger.Log("failed to collect dump using jstack -F : %v", err)
 					e1 <- err
+					_ = jstackFile.Close()
 					return
 				}
 			}
 
-			e := jstackFile.Sync()
-			if e != nil {
-				logger.Log("failed to sync file %v", e)
+			var e error
+			if jstackFile != nil {
+				e := jstackFile.Sync()
+				if e != nil {
+					logger.Log("failed to sync file %v", e)
+				}
+				_ = jstackFile.Close()
 			}
 
 			// necessary to send something into channel to prevent blocking inside waiting loop

capture/kernel.go

@@ -21,7 +21,7 @@ func (k *Kernel) Run() (result Result, err error) {
 	}
 	if k.Cmd.IsSkipped() {
 		result.Msg = "skipped capturing Kernel"
-		result.Ok = true
+		result.Ok = false
 		return
 	}
 	k.Cmd.Wait()

capture/ping.go

@@ -22,7 +22,7 @@ func (c *Ping) Run() (result Result, err error) {
 	}
 	if c.Cmd.IsSkipped() {
 		result.Msg = "skipped capturing Ping"
-		result.Ok = true
+		result.Ok = false
 		return
 	}
 	c.Cmd.Wait()

capture/thread.go

@@ -27,22 +27,28 @@ func (t *ThreadDump) Run() (result Result, err error) {
 		if err != nil {
 			logger.Log("failed to open tdPath(%s), err: %s", t.TdPath, err.Error())
 		} else {
-			defer tdf.Close()
+			defer func() {
+				_ = tdf.Close()
+			}()
 			td, err = os.Create(tdOut)
 			if err != nil {
+				result.Msg = err.Error()
 				return
 			}
-			defer td.Close()
+			defer func() {
+				_ = td.Close()
+			}()
 			_, err = io.Copy(td, tdf)
 			if err != nil {
+				result.Msg = err.Error()
 				return
 			}
 			_, err = td.Seek(0, 0)
 			if err != nil {
+				result.Msg = err.Error()
 				return
 			}
 		}
-		return
 	}
 	if t.Pid > 0 && td == nil {
 		if !shell.IsProcessExists(t.Pid) {
@@ -71,7 +77,9 @@ func (t *ThreadDump) Run() (result Result, err error) {
 		if err != nil {
 			return
 		}
-		defer td.Close()
+		defer func() {
+			_ = td.Close()
+		}()
 	}
 	result.Msg, result.Ok = shell.PostData(t.Endpoint(), "td", td)
 	return

capture/top.go

@@ -21,7 +21,7 @@ type Top struct {
 func (t *Top) Run() (result Result, err error) {
 	if len(shell.Top) < 1 {
 		result.Msg = "skipped capturing TopH"
-		result.Ok = true
+		result.Ok = false
 		return
 	}
 	file, err := os.Create("top.out")
@@ -40,7 +40,7 @@ func (t *Top) Run() (result Result, err error) {
 	}
 	if t.Cmd.IsSkipped() {
 		result.Msg = "skipped capturing Top"
-		result.Ok = true
+		result.Ok = false
 		return
 	}
 	err = t.Cmd.Wait()
@@ -69,7 +69,7 @@ func (t *Top) Run() (result Result, err error) {
 		logger.Log("trying %q, cause %q exit code != 0, read err %s %v", t.Cmd.String(), oCmd, output, rErr)
 		if t.Cmd.IsSkipped() {
 			result.Msg = "skipped capturing Top"
-			result.Ok = true
+			result.Ok = false
 			return
 		}
 		err = t.Cmd.Wait()
@@ -94,7 +94,7 @@ type TopH struct {
 func (t *TopH) Run() (result Result, err error) {
 	if len(shell.TopH) < 1 {
 		result.Msg = "skipped capturing TopH"
-		result.Ok = true
+		result.Ok = false
 		return
 	}
 	if !shell.IsProcessExists(t.Pid) {
@@ -127,7 +127,7 @@ func (t *TopH) Run() (result Result, err error) {
 	}
 	if t.Cmd.IsSkipped() {
 		result.Msg = "skipped capturing TopH"
-		result.Ok = true
+		result.Ok = false
 		return
 	}
 	err = t.Cmd.Wait()
@@ -156,7 +156,7 @@ func (t *TopH) Run() (result Result, err error) {
 		logger.Log("trying %q, cause %q exit code != 0, read err %s %v", t.Cmd.String(), oCmd, output, rErr)
 		if t.Cmd.IsSkipped() {
 			result.Msg = "skipped capturing TopH"
-			result.Ok = true
+			result.Ok = false
 			return
 		}
 		err = t.Cmd.Wait()
@@ -174,7 +174,7 @@ type Top4M3 struct {
 func (t *Top4M3) Run() (result Result, err error) {
 	if len(shell.Top4M3) < 1 {
 		result.Msg = "skipped capturing TopH"
-		result.Ok = true
+		result.Ok = false
 		return
 	}
 	top, err := os.Create("top4m3.out")
@@ -195,7 +195,7 @@ func (t *Top4M3) Run() (result Result, err error) {
 		}
 		if t.Cmd.IsSkipped() {
 			result.Msg = "skipped capturing Top"
-			result.Ok = true
+			result.Ok = false
 			return
 		}
 		err = t.Cmd.Wait()

capture/vmstat.go

@@ -31,7 +31,7 @@ func (t *VMStat) Run() (result Result, err error) {
 	t.Cmd, err = shell.CommandStartInBackgroundToWriter(file, cmd)
 	if t.Cmd.IsSkipped() {
 		result.Msg = "skipped capturing VMStat"
-		result.Ok = true
+		result.Ok = false
 		return
 	}
 	if err != nil {

const.go

@@ -12,7 +12,7 @@ import (
 //  Generic Properties
 // ------------------------------------------------------------------------------
 var (
-	SCRIPT_VERSION      = "yc_agent_1.0.6"
+	SCRIPT_VERSION      = "yc_agent_1.0.11"
 	SCRIPT_SPAN         = 120 // How long the whole script should take. Default=240
 	JAVACORE_INTERVAL   = 30  // How often javacores should be taken. Default=30
 	TOP_INTERVAL        = 60  // How often top data should be taken. Default=60