36 Pods V2

[dendron-bot]

Goals

1. Making pods easier to use
2. Making pods more flexible to use
3. Making pods easier to extend

Context

The proposal here covers a lot of ground. We should break it down into phases.

Proposal

1. Instead of having both a "Export Pod" and a "Publish Pod", only have an Export Pod
   • refactor existing commands
     • Export Pod -> Export
     • Publish Pod -> Export One ^NWDbwthvJ0xm
2. Introduce the equivalent of a Export One to Import
   • refactor existing command:
     • Import Pod -> Import
   • introduce new command:
     • Import One
3. Introduce a new pod command: From Clipboard ^k2tGQOBXQrOK
   • Export From Clipboard
   • Import From Clipboard
   • options to support:
     • splitLine: should we consider each line as its own record? default, false
   • example use case:
     • export a selection of task notes to airtable
     • import a part of google doc
4. Support pod destination
   • Import:
     • New Note
     • Clipboard
   • Export:
     • New Entry
     • Clipboard
5. Extend pod capabilities
   • all pods should have the ability to handle merge conflicts
     • on import, merge would update local note
     • on export, merge would update remote source
   • all pods should have the ability to specify field mappings
     • on import, map fields from the source to fields in the note
     • on export, map fields from note to export destination
     • fields from the note includes anything in the note frontmatter as well as sections within notes
       • in the future, we can also include backlinks
6. Support interactive mode
   • should be a toggle on the quickpick
   • if interactive, instead of reading src/dest from config, prompt user for this
   • eg. orbit import pod, when importing interactively, user enters a username
7. Introduce a webview to configure pods
   • users can use a form widget with live validation instead of YAML
8. Support pod profiles
   • users have different targets for the same pods (eg. publish a task note to airtable task table but publish a user note to airtable user table)
   • pod configuration should support custom profiles
   • should be supported in webview
9. support custom pod plugins
   • a user should be able to contribute their own pods

Alternatives

• instead of having a unique Export/Import command for different use cases, just have a single command wiith different flags and modifiers

Details

Example

Discussion

https://github.com/dendronhq/dendron/discussions/new

[SeriousBug]

I think "Export" and "Export One" might be a little confusing, maybe "Export Many" or "All" and "Export One"? Or we could entirely combine the commands and allow selection of the scope when the command runs, like the Dendron Doctor command.

all pods should have the ability to handle merge conflicts

I think this is harder to do than it sounds, what if there are both changes in remote and local? For example, someone edits a GDocs document using the google web editor while someone else updates it in Dendron, then the user in Dendron tries to export. You need a three-way merge like what git does, and that's not trivial. Not saying we can't do it, just pointing out that it might be a bit of work.

Introduce a webview to configure pods

I think this will greatly improve user-friendliness of the pods, it's a great idea.

support custom pod plugins

Something to add here is that pod plugins would likely be much more complicated than something like a hook, so we might want to build some tooling around this. Could we have something like a template repository for building pod plugins? My main concern is that people might want additional dependencies that we don't already have included in Dendron, like a package that parses a different file format or a wrapper for an API.

[nickolay-kondratyev]

It appears more info is needed around in this RFC to collect feedback

For example:

all pods should have the ability to handle merge conflicts
on import, merge would update local note
on export, merge would update remote source

How exactly are they updates are going to be done? Are we going to do something along the lines of GIT where both of the conflicts appear in the note for the user to resolve? Are both of the conflicts going to be 'committed' right away?

[Harshita-mindfire]

I can speak for Orbit Import pod here, The user gets prompted if there are conflict in the entries and select an option from prompt to either overwrite the local values with remote or completely skip the merge and do it manually. Unlike git this pod takes one action for entire file. So if there are multiple conflicts in same file, Both remote and local entries are displayed for each conflict and single action (either overwrite or skip) is selected per file. Merge Conflict

[Harshita-mindfire]

all pods should have the ability to handle merge conflicts

I think handling merge conflicts would be tricky for us when there's change in the note body. It is relatively easy if the conflict is in some frontmatter properties, but when it comes to note body, which is often a large string and to selectively figure out the difference can be a task.

Introduce a new pod command: From Clipboard ^k2tGQOBXQrOK
Import From Clipboard

Is import from clipboard similar to creating a new note and paste from clipboard?

Also not very pod structure specific but since we are having a discussion on pods,

1. I think we should make them more visible when it comes to onboarding new users coming from other tools. It has been a common question in our community on importing docs to a vault. I am not sure what could work best in this case(onboarding guide for dendron alternatives or something else)

2. Implementing pods by analyzing whether it could be a pod for migration.
   For eg. GithubIssue Import pod certainly not a pod for a user who wants to import his existing docs, but gdoc pod could be one of them.

[ScriptAutomate]

I think we should make them more visible when it comes to onboarding new users coming from other tools. It has been a common question in our community on importing docs to a vault. I am not sure what could work best in this case(onboarding guide for dendron alternatives or something else)

I can see value in adding it into the docs, but linking out to it at the end of the Getting Started guide. Something like, "Looking at import your notes from Google Docs and other platforms? Take a look at Dendron's built-in Pods, our way of importing to and exporting notes to/from your Dendron vaults."

[jonathanyeung]

Reposting some offline comments from earlier:

• I like removal of Publish POD (consolidation with Export) - i always found this confusing
• Second Kaan - I don't think we should add new commands for export one / import one - they should just be params / additional quick pick steps in the existing commands. I think adding additional commands is confusing.
• clipboard - I like the functionality, but once again I don't think it should be a separate command. maybe it's time to add modifiers to the pod quickpick. we can also add contextual UI - i.e. if part of markdown text is highlighted, then as part of the lightbulb or context menu we have a "Dendron: Pod Export To..." option for example

[ScriptAutomate]

I agree with the same comments here:

• Consolidate export and pod, as this has confused me. I've already found myself doing export when I meant publish
• I like the idea of params to export/import, and using "One" might be better worded as "Individual" or "Single" vs. "Collection" or "Multiple"? Granted, it's more verbose than one vs. many or all. @SeriousBug also pointed out some rewording alternative along similar lines in this discussion.

clipboard - I like the functionality, but once again I don't think it should be a separate command. maybe it's time to add modifiers to the pod quickpick. we can also add contextual UI - i.e. if part of markdown text is highlighted, then as part of the lightbulb or context menu we have a "Dendron: Pod Export To..." option for example

This has been on my mind a bit, like using individual commands as a shortcut key to the longer path. These could work like aliases. These have come up in conversation within the community before. For example, this Discord thread on exporting Markdown in gfm. In this thread, we talk about the steps to exporting the markdown to clipboard, and the potential value in a aliasing a common item into something like Dendron: Export Pod Markdown to Clipboard. Though this is a verbose example, it would mean that the single command would appear in the command palette when looking at other publish/export commands.

[kevinslin]

thanks everyone for the feedback! to clarify about merge conflicts, i should emphasize that it’s all pods should have the capability to handle merge conflicts. that is to say it should be in the interface. pod authors can choose to implement or not (in some cases that won’t be possible)

[kevinslin]

@SeriousBug @nickolay-kondratyev 👆

[simkimsia]

is there a way to write our own export?

the current publish to markdown is close but not exactly what i want.

• doesn't include frontmatter which is ✅
• uses markdown when i want the presentation style of the preview ❌

[kevinslin]

currently, you can write a publish pod. docs here -> https://wiki.dendron.so/notes/6322a2de-34fd-4c93-bcbe-d71c30e5f2b6.html

(NOTE: this is experimental and we might be updating this soon to merge publish with export)

[simkimsia]

Thanks... Just update here when it's updated

[hikchoi]

A bit late to the party but all prior comments covered my concerns. 👍🏼