Commit 7d63832
committed
mount: thread safety
There is actual concurrency when using `pyfuse3` in Borg, which necessitates thread safety mechanisms like the recently added `threading.Lock`.
Borg's `pyfuse3` implementation is built on top of the **`trio`** async framework. When Borg is started in `pyfuse3` mode, it calls `trio.run(llfuse.main)`.
1. **Async/Await Model**: Unlike the classic `llfuse` (which Borg runs with `workers=1` to remain single-threaded), `pyfuse3` uses an asynchronous event loop. While it often runs on a single OS thread, `trio` allows multiple tasks to be "in-flight" simultaneously.
2. **Context Switching**: When an operation (like reading metadata or data) hits an `await` point—such as during network I/O with a remote repository or disk I/O—`trio` can suspend that task and switch to another one.
3. **Parallel Archive Loading**: In `borg mount` (without a specific archive specified), archives are loaded lazily. If a user or a process (like `find` or a file manager) triggers a `lookup` or `opendir` on multiple archive directories nearly simultaneously, multiple `check_pending_archive` calls can be active at once.
4. **Race Conditions**: Because `iter_archive_items` is a generator that yields control back to the caller (which, in the `pyfuse3` case, happens within an async wrapper), and because it performs I/O via `get_many`, it creates windows where one archive's loading process can be suspended while another begins.
Even if running on a single OS thread, the interleaved execution of async tasks can lead to the same data corruption issues as multi-threading:
- **Shared State**: Both tasks access the same `ItemCache` instance.
- **Inode Collisions**: If two tasks read the `write_offset` before either has incremented it, they will assign the same inode numbers to different files and overwrite each other's metadata in the `meta` bytearray.
- **Global Interpreter Lock (GIL)**: While the GIL protects Python's internal memory integrity, it does not prevent logical race conditions at the application level when tasks are interleaved.
The use of `threading.Lock` in `check_pending_archive` and the direct instance variable access in `ItemCache` ensure that even when `trio` switches between concurrent FUSE requests, the internal state remains consistent and inode assignment remains unique across all archives.1 parent d54996f commit 7d63832
1 file changed
+17
-18
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
| 9 | + | |
9 | 10 | | |
10 | 11 | | |
11 | 12 | | |
| |||
155 | 156 | | |
156 | 157 | | |
157 | 158 | | |
158 | | - | |
159 | | - | |
160 | 159 | | |
161 | 160 | | |
162 | 161 | | |
163 | 162 | | |
164 | | - | |
165 | | - | |
166 | | - | |
167 | | - | |
168 | | - | |
169 | | - | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
170 | 168 | | |
171 | 169 | | |
172 | 170 | | |
| |||
200 | 198 | | |
201 | 199 | | |
202 | 200 | | |
203 | | - | |
204 | | - | |
| 201 | + | |
| 202 | + | |
205 | 203 | | |
206 | 204 | | |
207 | 205 | | |
| |||
222 | 220 | | |
223 | 221 | | |
224 | 222 | | |
225 | | - | |
| 223 | + | |
226 | 224 | | |
227 | 225 | | |
228 | 226 | | |
229 | | - | |
| 227 | + | |
230 | 228 | | |
231 | | - | |
232 | | - | |
233 | | - | |
| 229 | + | |
| 230 | + | |
234 | 231 | | |
235 | 232 | | |
236 | 233 | | |
| |||
269 | 266 | | |
270 | 267 | | |
271 | 268 | | |
| 269 | + | |
272 | 270 | | |
273 | 271 | | |
274 | 272 | | |
| |||
304 | 302 | | |
305 | 303 | | |
306 | 304 | | |
307 | | - | |
308 | | - | |
309 | | - | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
310 | 309 | | |
311 | 310 | | |
312 | 311 | | |
| |||
0 commit comments