Copilot with cards

[macin]

Hi, did you consider simplifying integration of Copilot with Canvas?

I've recorded my workflow, would love it being simplified ... https://www.youtube.com/watch?v=xSMXLu69b6c

[logancyang]

@macin Amazing video! Thanks for putting it together!

I think it's a great idea. My initial reactions:

• Are there any standalone apps that already do similar things? I vaguely recall seeing some demo like this before.
• By this I mean roughly:
  • interact with ChatGPT in a tree UI, where each child node has all the parent nodes up to the root as context
  • generate new response directly into the child node
• This is good for the use case you showed, a planning session that branches out with suggestions from the ChatGPT response, or more generally, any top-down workflow where the user would like to dig deeper into something. It's actually quite prevalent in a lot of scenarios. If I build it, I'd like it to be easily generalized to a number of use cases. If you have other concrete examples please chime in!
• The bare minimum feature with Obsidian Canvas looks like that it should be able to pass the selection from Chat window to a selected node with 1 click; same in the opposite direction, 1-click moving current node content + all previous node context to the Chat box. It will require adding buttons to the cards and the chat response. Anything else for a minimal MVP?
• I'm starting to think that it might work better completely without the Chat window and directly in the Canvas UI, where it generates response directly into a node, and the user can easily break the node down to multiple nodes with a selection and a "create a new node" button, something like that.

In general I think a tree-like UI for interacting with LLMs is useful for many many use cases, I have to think about it more though.

[macin]

@logancyang glad you like it. I searched for the tool, but no luck so far. Besides, I'd prefer privacy of Obsidian, as we all probably do in the community :)

• For the use cases .. yeah, this is very generic approach for "speaking" to ChatGPT. Imagine, writing a book, any book. Or research of the some subject. Possibilites are virtually endless. The issue could be how you manage your pre and post content prompts ... as there will be some because general prompt will be different for itinerary planning than the one for the scientific book. Also, the rephrasing prompt could differ so I'd leave that to the users to configure and present somewhere on the UI to choose from (no idea how best to make it)

The back and forth buttons for moving selection of text would be good.

To expand a bit on yours I'd think about:
Chat to card

• copy chat selection to selected card
• create child card to selected card based on chat content selection
• create sibbling card to selected card based on chat content selection (this would probably be nice to have as you can achieve the same with the previous one)

Card to chat

• Copy all cards from selected to the root and to chat (this is to either expand or rewrite)
• Copy all cards from selected to the root + include sibblings to chat (this is to fill the gaps on the last level, i.e. ask for missing pieces)

Not sure how this will work without the chat.###

When I was preparing the example I quite often had to put in a different introductory prompt (pre content prompt) and closing prompt (post content prompt) once I pasted a card to chat content. Maybe with some templates, or some adhoc modal window to modify the pre and post content prompt that could work without the chat... I'd say that with chat that would be an MVP. The approah without the chat is certainly worth exploring further for MVP +1 :)

Further extensions

A cherry on the cake would be a plugin which could combine all the cards and build a single content page with all the card content combined for print. That could be a super tool, I think as this would allow sharing the results of your work with the rest of the world. Especially those who don't think in graphs and prefer a more linear content :)

[oscaromsn]

Such a nice workflow — congratulations! I would definitely use a feature like that if it were implemented. There is an online tool that does something like that, but without the privacy of Obsidian — which is mandatory for many users in the community.

[macin]

@oscaromsn great find, didn't know that. Thanks, but yeah, privacy first is the ultimate direction :)

[logancyang]

@oscaromsn oh yes this is exactly what I saw before! Thanks for sharing.

[logancyang]

I'm a big believer in privacy and local-first AI, however monetization is a big problem to make the project sustainable as it grows. Building such features is not a trivial thing.

Obsidian plugins are completely open source, and with a local-first philosophy (completely in-browser with no backend servers), it's kind of impossible to add any paid features. The ideal case for me is to have 90% of the features for free and 10% behind a paywall so the project can be sustainable long-term. But I have no idea how that should work with this setup. Any suggestion?

[macin]

I think that, there would need to be an ecosystem with free (like it is now) and paid plugins. The only way to charge would be to have those plugins available to download only after payment. It would be hard to introduce recurring payment though... But I'd happily pay for good work if there was a way to pay, but unfortunately cannot fund the whole cost of development myself. For now I guess only donations can work.

[oscaromsn]

Yeah, I agree — the plugin model in Obsidian isn't designed for monetization, and that's a limitation for more complex and time-consuming features. I know that the TfT Hacker has done some custom CSS customizations for the canvas feature that is paid and only then you can download.

[logancyang]

Pay-and-download could work but it's not as user-friendly as the one-click installation from the community plugin page. Backend-enabled features seem the only plausible way to monetize. Donation has proved to be quite limited right now. I see some biggest Obsidian plugins with only a handful of sponsors 😮‍💨. Anyways, I will try to build stuff out as much as possible based on feedback here and the reception of the community.

[logancyang]

Here's another one in a similar vein https://hunch.tools/

[oscaromsn]

Looks pretty cool — thank you for sharing!

I was just now taking a look at two papers that explore the possibilities of using LLMs to chain prompts on node structures^{1 2, with code here}.

I see a lot of potential for Obsidian to be the perfect IDE for reasoning with the help of language models. Right now, with Copilot and Smart Connections, I kind of do that by referencing some markdown files containing frameworks, mental models, or checklists to brainstorm and help with complex decision-making (legal strategy, for example). I'm waiting for calmer times at work to document and share these use cases in a repo here.

[logancyang]

@oscaromsn If you share your use cases that would be super helpful!

I'm also interested in the mental models you mentioned, do you publish these writings somewhere? Would love to learn. I'm thinking about making a series of youtube videos talking about various mental models in different disciplines.