Get rid of run command and convert all scenarios to use import

[byorgey]

In this PR:

• Get rid of the run command (good riddance!)
• Get rid of IO from the runtime engine, and introduce new effects for the things we were still using IO for: (1) mutable global cache and (2) getting the current time
• Split up Language.Syntax.Import into multiple modules
• Rename getDataDirSafe to getDataDirThrow since it actually throws an exception, and create an actually safe variant
• Add syntax ~swarm as an import anchor to refer to things in the Swarm data dir, so you can write e.g. import "~swarm/lib/control" in order to import data/lib/control.sw from wherever it got installed
• Converts all scenarios to use import:
  • Move a bunch of repeated code into the beginnings of a standard library
  • Typically, if we had run "foo.sw" and foo.sw contained def ...; def ...; go we remove go from the end of foo.sw and change run "foo.sw" to import "foo"; go

Closes #495. Closes #328. Closes #2068. Closes #2182. Closes #1560. Closes #2602.

[xsebek]

Wow, this is awesome! 👍 And you even went ahead and got rid of the IO! 🚀

[byorgey]

Currently the tournament-host test suite is failing. It used to work by first testing that it could upload arbitrage.yaml as a scenario, then separately testing that it could upload _arbitrage/solution.sw as a solution to the previously uploaded scenario. Previously, arbitrage.yaml contained something like

solution: |
  run "_arbitrage/solution.sw"

and validating the scenario description did not involve loading solution.sw at all, since the run is not interpreted until runtime. Now, however. arbitrage.yaml contains

solution: |
  import "_arbitrage/solution"; solution

and in order to validate the scenario we must typecheck the solution, which includes recursively loading _arbitrage/solution.sw. Predictably this failed when it could not find the file. I tried modifying the test to upload both arbitrage.yaml and the solution, but it still can't find the right file. It seems that it is not even looking for it in the right place, since it complains about not being able to find swarm/_arbitrage/solution. This is what happens when it doesn't know where it is loading the .yaml file from so it can't figure out the right place to load any relative imports from. I don't yet understand what's going on well enough to know how to fix this.

Edited to add: it seems like the gamestateFromScenarioText function here:

swarm/src/swarm-tournament/Swarm/Web/Tournament/Validate.hs

Lines 183 to 185 in e0b64c7

	gamestateFromScenarioText ::
	LBS.ByteString ->
	ExceptT ScenarioInstantiationFailure IO (GameState, Scenario Elaborated)

already just takes the content of a .yaml file as input; then it in turn calls initScenarioObject:

swarm/src/swarm-tournament/Swarm/Web/Tournament/Validate.hs

Line 200 in e0b64c7

scenarioObject <- initScenarioObject scenarioInputs content

which in turn is forced to call parseJSONE' Nothing:

swarm/src/swarm-tournament/Swarm/Web/Tournament/Validate.hs

Line 179 in e0b64c7

parseEither (parseJSONE' Nothing scenarioInputs) rawYaml

The first argument to parseJSONE' is supposed to be the provenance of the file being parsed, but at this point it is too late, we only have the file's contents.

Edited to add 2: even if I figured out the above issue, I don't think this is going to work... the scenario validation calls processSource, which assumes it will be able to do regular file I/O to recursively load any imports; but in the context of the tournament server it seems like there's some kind of persistence layer providing access to uploaded files, so just doing regular file I/O will not work (I think). If I'm right about that, I suppose the real solution would be to create some kind of filesystem effect with two handlers, one to run filesystem commands via normal IO and one to run it via a provided persistence layer.

@kostmo any thoughts on this?

Edited to add 3: Just commenting out the tournament-host test suite for now: #2681 . Another possibility that was discussed was inlining the solution into the scenario description so that it could be processed without loading an import. However, doing this on the real arbitrage scenario itself would mean that the solution is now duplicated between being inlined in the scenario description and the solution.sw file, which I don't want to do. Alternatively we could copy the scenario to a special location which would only be used for testing the tournament server, but then the tournament server test would not be using a "real" scenario which is guaranteed to be up-to-date. Simply commenting out the test for now seems like the most straightforward and honest thing to do.

[mergify]

Merge Queue Status

✅ The pull request has been merged at 946e4bf

This pull request spent 5 seconds in the queue, with no time running CI.
The checks were run in-place.

Required conditions to merge

• #approved-reviews-by >= 1 [🛡 GitHub branch protection]
  • Get rid of run command and convert all scenarios to use import #2679
• #changes-requested-reviews-by = 0 [🛡 GitHub branch protection]
  • Get rid of run command and convert all scenarios to use import #2679