You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This PR introduces the Trace Archive Tool along with improvements in the
documentation at `traces/stf_trace_archive/README.md`.
The trace archive tools comes with four different commands
* **Upload.** Upload workload and/or trace.
* **List.** List items by category.
* **Get.** Download a specified trace file.
* **Setup.** Create or edit current tool configurations.
This PR files follows this structure:
```text
README.md # Main documentation file
src/
├── data/ # Core data models and classes
│ └── storage/ # Storage backend implementations (local, cloud, etc.)
├── handlers/ # Command handlers (upload, get, list, setup)
├── utils/ # Utility functions and helpers
└── trace_archive.py # Main CLI entry point
```
### Quickstart
```bash
# Install dependencies
pip install -r requirements.txt
# Configure initial storage (local)
python trace_archive.py setup
# Upload a workload and trace
python trace_archive.py upload --workload ../../stf_metadata/example/dhrystone --trace ../../stf_metadata/example/dhrystone.zstf
```
command definitions and examples are present in the README file.
### Assumptions:
* Every metadata file for simpointed traces will contain the property
`stf` -> `trace_interval` -> `start_instruction_index`
* Traces files has a .zstf file type
### Missing Work / Next Steps:
* Add Bundle / Suite grouping
* Add script to generate metadata for traces
* Add search command
* Make possible to upload workload related files, like objdump
* Add a test class
***[List](#list-command).** List items by category.
72
+
***[Get](#get-command).** Download a specified trace file.
73
+
***[Setup](#setup-command).** Create or edit current tool configurations.
36
74
37
-
### `upload`
38
75
39
-
Uploads a trace file along with its associated workload and metadata.
76
+
### Initial Setup
77
+
78
+
To set up the trace archive tool, run the `setup`command to configure the inital storage type and it's path. For example, to set up a local storage type, with the name `local` and path to the storage folder `/home/user/trace_archive`, run:
40
79
41
80
```bash
42
-
$ python trace_archive.py upload --help
43
-
Usage: python trace_archive.py upload [OPTIONS]
81
+
$ python trace_archive.py setup
82
+
Creating a new storage source.
83
+
Registred storage type options: local-storage
84
+
Select your storage type: local-storage
85
+
Enter your storage name: local
86
+
Enter the storage folder path: /home/user/trace_archive
87
+
```
88
+
89
+
All storage sources contains a type and a name. The type is used to identify the storage source, like `local-storage` or `google-drive`, while the name is used to identify the storage configuration in the tools commands.
44
90
45
-
Upload a workload, trace and metadata to the database
91
+
With the initial setup done, you can add new storage sources or change the default storage source using the `setup` command again, with the commands `--add-storage` and `--set-default-storage`, respectively.
> Requires a metadata file located at `<trace>.metadata.yaml`.
98
+
All configurations are stored in the `config.yaml` file, which is created in the current working directory when the `setup` command is run for the first time.
54
99
55
-
> For every upload, a unqiue [trace id](#trace-id) will be generated
100
+
Checkout the [Creating and using second storage source](#creating-and-using-second-storage-source) section for more details on how to create and use a second storage source.
56
101
57
-
---
102
+
### Upload Command
58
103
59
-
### `search`
104
+
The `upload` command is for upload a trace and it's workload. The a trace file, and if not presented in the storage yet, hte workload file. The tools also expects a metadata file, which is a YAML file with the name `<trace>.metadata.yaml`, where `<trace>` is the name of the trace file. Multiple traces can be uploaded at once, as long as they are from the same trace attempt.
60
105
61
-
Search can be used to search for the given regex term in the list of available traces and metadata matches
*`--it`: Interactive files selection mode. If this option is used, the tool will prompt the user to selectthe workload and trace files
66
111
67
-
Search for traces and metadata using a regular expression.
112
+
For every upload, a unique [`trace-id`](#trace-id) will be generated and filled into the metadata file.
68
113
69
-
Arguments:
70
-
REGEX Regex expression to search with.
114
+
### List Command
71
115
72
-
Options:
73
-
--names-only Search only by trace id (ignore metadata).
74
-
```
116
+
The `list`command is used to list the available traces or workloads in the archive.
75
117
76
-
---
118
+
### Get Command
77
119
78
-
### `list`
120
+
The `get`command is used to download a specified trace, workload or metadata file from the archive. The command options are:
79
121
80
-
```bash
81
-
$ python trace_archive.py list --help
82
-
Usage: python trace_archive.py list [OPTIONS]
122
+
*`--trace`: Id of the trace to download.
123
+
*`--workload`: Id of the workload to download.
124
+
*`--metadata`: Id of the metadata (same as the trace id) to download.
125
+
*`-o, --output`: Output file path. If not specified, the file will be downloaded to the current working directory.
83
126
84
-
List database traces or related entities.
127
+
## Examples
85
128
86
-
Options:
87
-
--traces Lists available traces (default)
88
-
--workloads Lists available workloads
129
+
Assuming the trace archive tool is set up with a local storage type named `local`, you can use the following commands:
130
+
131
+
### Uploading a Trace
132
+
133
+
To upload a trace file named `dhrystone.zstf` and its workload `dhrystone`, present in the metadata example folder, you can run:
134
+
135
+
```bash
136
+
$ python trace_archive.py --storage-name local upload --workload ../../stf_metadata/example/dhrystone --trace ../../stf_metadata/example/dhrystone.zstf
137
+
138
+
Uploading workload: ../../stf_metadata/example/dhrystone with id: 0
139
+
Uploading trace: ../../stf_metadata/example/dhrystone.zstf with id: 0.0.0000_dhrystone
89
140
```
90
141
91
-
---
142
+
### Downloading a Trace
143
+
144
+
To download the trace file `000.000.000_dhrystone.zstf` and its metadata, you can run:
145
+
146
+
```bash
147
+
$ python trace_archive.py get --trace 000.000.000_dhrystone.zstf
92
148
93
-
### `get`
149
+
Trace 0.0.0000_dhrystone saved on ./0.0.0000_dhrystone.zstf
150
+
Metadata 0.0.0000_dhrystone saved on ./0.0.0000_dhrystone.zstf.metadata.yaml
151
+
```
152
+
153
+
### Downloading a Workload
94
154
95
-
Downloads a specified trace file.
155
+
To download the workload `0` (dhrystone), you can run:
96
156
97
157
```bash
98
-
$ python trace_archive.py get --help
99
-
Usage: python trace_archive.py get [OPTIONS] TRACE
158
+
$ python trace_archive.py get --workload 0
100
159
101
-
Download a specified trace file.
160
+
Workload 0 saved on ./dhrystone
161
+
```
102
162
103
-
Arguments:
104
-
TRACE Name of the trace to download.
163
+
### Creating and using second storage source
105
164
106
-
Options:
107
-
--revision Revision number. If not specified, the latest revision is used.
108
-
--company Filter by associated company.
109
-
--author Filter by author.
110
-
-o, --output Output file path.
165
+
To create a second storage source you can run the `setup`command with the `--add-storage` option:
166
+
167
+
```bash
168
+
$ python trace_archive.py setup --add-storage
169
+
Creating a new storage source.
170
+
Registred storage type options: local-storage
171
+
Select your storage type: local-storage
172
+
Enter your storage name: private-storage
173
+
Enter the storage folder path: ./private
174
+
```
175
+
176
+
This will create a new storage source named `private-storage` with the path `./private`. You can then use this storage sourcein the `upload`command by specifying the `--storage` option:
0 commit comments