Factor out generic utilities for loading recursively nested collections from disk#2688
Factor out generic utilities for loading recursively nested collections from disk#2688
Conversation
|
@xsebek would you have time to look at this? If not, no worries! |
xsebek
left a comment
xsebek
left a comment
There was a problem hiding this comment.
Very nice! I think it's actually easier to read once it is more generic. 👍
| -- Load a collection with items in no particular order, and | ||
| -- optionally warn that the ORDER file is missing. |
There was a problem hiding this comment.
Isn't this an alphabetical order based on file name?
There was a problem hiding this comment.
I mean, in practice it seems to be alphabetical, but really it depends on the order in which files are returned by listDirectory. The documentation for listDirectory does not make any guarantees about the order of the resulting list: https://hackage-content.haskell.org/package/directory-1.3.10.1/docs/System-Directory.html#v:listDirectory I presume it just depends on the particular order of files returned by some OS call.
I suppose we could apply an explicit sort in order to guarantee alphabetical order. @xsebek do you think that would be worthwhile?
| m (CollectionItem a) | ||
| loadCollectionItem cfg path = do | ||
| isDir <- sendIO $ doesDirectoryExist path | ||
| let collectionName = into @Text . dropWhile isSpace . takeBaseName $ path |
There was a problem hiding this comment.
What is dropWhile isSpace doing here? I would expect a map trim nonEmptyLines in readOrderFile would better take care of leading and trailing whitespace.
There was a problem hiding this comment.
I... am not sure. It would be strange if a folder name started with spaces, wouldn't it?
Co-authored-by: Ondřej Weis <44544735+xsebek@users.noreply.github.com>
2 participants
Towards #2504 . As soon as I started working on #2504 I quickly realized that I wanted to load a whole tree of documentation from disk, which was going to involve a lot of code very similar to the code we already use to load the collection of scenarios. In fact #2504 even mentions possibly reusing the idea of having an
00-ORDER.txtfile to specify the ordering of documentation pages within a folder.So, as a first step, this PR factors out the generic recursively-loading-stuff-from-disk part from the scenario-specific parts. In particular:
OrderedMaputilities into their own moduleScenarioCollectionandScenarioItemtypes into genericCollectionandCollectionItemtypesloadCollectionandloadCollectionItemfunctions which implement the generic logic (including reading00-ORDER.txtfiles), parameterized by aCollectionConfigrecord which specifies how to load individual items, and how to decide which items to load and which to ignore.CollectionConfigrecord in the future should we decide that additional configuration options are needed.Swarm.ResourceLoading.Collectionand re-exported fromSwarm.ResourceLoading.