Skip to content

Commit b8cc750

Browse files
authored
Fix docs explaining write modes for Luigi Targets. Closes #2783 (#2931)
1 parent da392e6 commit b8cc750

File tree

2 files changed

+27
-1
lines changed

2 files changed

+27
-1
lines changed

doc/tasks.rst

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -127,6 +127,31 @@ An example:
127127
)
128128
)
129129
130+
It's useful to note that if you're writing to a binary file, Luigi automatically
131+
strips the ``'b'`` flag due to how atomic writes/reads work. In order to write a binary
132+
file, such as a pickle file, you should instead use ``format=Nop`` when calling
133+
LocalTarget. Following the above example:
134+
135+
.. code:: python
136+
137+
class GenerateWords(luigi.Task):
138+
139+
def output(self):
140+
return luigi.LocalTarget('words.pckl', format=Nop)
141+
142+
def run(self):
143+
import pickle
144+
145+
# write a dummy list of words to output file
146+
words = [
147+
'apple',
148+
'banana',
149+
'grapefruit'
150+
]
151+
152+
with self.output().open('w') as f:
153+
pickle.dump(words, f)
154+
130155
.. _Task.input:
131156

132157
Task.input

luigi/target.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -235,7 +235,8 @@ def open(self, mode):
235235
236236
:param str mode: the mode `r` opens the FileSystemTarget in read-only mode, whereas `w` will
237237
open the FileSystemTarget in write mode. Subclasses can implement
238-
additional options.
238+
additional options. Using `b` is not supported; initialize with
239+
`format=Nop` instead.
239240
"""
240241
pass
241242

0 commit comments

Comments
 (0)