feat: change from ConvertibleToAssetDep to ConvertibleToAssetKey

[lucargir]

This would close #54.

• Convert SQLMeshDagsterTranslator to extend ConfigurableResource for Dagster resource injection pattern
• Change IntermediateAssetOut and IntermediateAssetDep from Pydantic BaseModel to dataclass for lighter weight
• Use AssetKey instead of AssetDep for external dependencies to prevent duplicate asset definitions in containerized code locations
• Remove translator attribute from SQLMeshResource; use config.get_translator() as single source of truth
• Simplify docstrings throughout

[lucargir]

@ravenac95 I was also checking the ConvertibleToAssetOut and the IntermediateAssetOut, and I am not sure why are these used. If it's to be able to use SQLMesh' cache, shouldn't we just update the config to include the path to the cache directory and instantiate the context with that custom setting?

From their python API docs, it shows that it's defined as ".cache" by default, so not sure if this custom intermediate asset is needed. Or is this because if the AssetOut gets created during the initialisation of Dagster's code location, it takes a long time?

[ravenac95]

@ravenac95 I was also checking the ConvertibleToAssetOut and the IntermediateAssetOut, and I am not sure why are these used. If it's to be able to use SQLMesh' cache, shouldn't we just update the config to include the path to the cache directory and instantiate the context with that custom setting?

From their python API docs, it shows that it's defined as ".cache" by default, so not sure if this custom intermediate asset is needed. Or is this because if the AssetOut gets created during the initialisation of Dagster's code location, it takes a long time?

Ah yes this is a good question. We actually had to do this because we do some work with SQLMesh that requires essentially model generation with python models (this can take some significant time). This isn't something inherently handled by SQLMesh so in some weird way this is a pretty specific use case we needed to support for ourselves. At the moment we don't plan on changing any of that, but it's possible we could pull more of this into our own repository as opposed to leaning on the integration. So for now can we update this to keep the IntermediateAssetOut?

[lucargir]

@ravenac95 just made an updated PR for this, I think this should work with the requirements that you have, but still be flexible enough to fix the issue with external deps.