RFC: Cause of the documentation issue and proposal for a possible solution

[hacksparrow]

export class MyApplication extends RestApplication {
  constructor(options: ApplicationConfig = {}) {
    super(options);
  }
}

looks much better from a users's perspective, than:

export class MyApplication extends BootMixin(
  ServiceMixin(RepositoryMixin(RestApplication)),
) {
  constructor(options: ApplicationConfig = {}) {
    super(options);
  }
}

Mixins are our tools for composability; it is a great pattern for the people developing the framework but is not a great DX for the users of the framework, especially new users. We should shield them from internal implementation and present a simple RestApplication class, which they can use for creating their REST apps. RestApplication should then be documented as a first class entity (almost as LoopBack itself). Not suggesting we should modify the current RestApplication, just a name used for the example.

"But LoopBack can be used for creating other types of apps that may not use services and/or repositories."

The majority of our users are looking to use LoopBack for developing REST apps, even our app scaffolder creates a REST app by default. Therefore RestApplication should be a principal in the framework.

In my opinion, the confusion in the current documentation is caused by the drastic differences in the view of what LoopBack is, according to "us, the developers of the framework" and our users.

Users: LoopBack is a REST framework.
We: LoopBack is a powerful framework that can be used to build all sorts of apps. REST apps are just one of the many possibilities.

The organization and content of our documentation is a correct reflection of our view, but our users are looking for the documentation of their view.

That makes me wonder if creating two identities out of the current work would help.

1. LoopBack - the REST framework.
2. A meta framework, which can be used for creating LoopBack and other frameworks.

Now creating LoopBack apps would look like this:

export class MyApplication extends LoopBack {
  constructor(options: ApplicationConfig = {}) {
    super(options);
  }
}

A documentation around the LoopBack class is what our users are looking for. The current documentation is that of the meta framework.

[bajtos]

Thank you for starting the discussion, @hacksparrow.

One one hand I agree with you that the current template for building REST+Repository applications may look complex.

On the other hand, we are discussing support for other ORMs like TypeORM and Mongoose/Typegoose, and other transport protocols like Kafka (and pub/sub style in general). Application-level mixins like RepositoryMixin and ServiceMixin are the tool we use to keep different ORMs and possibly also transports decoupled from each other. How do you envision the user code for building an application using @loopback/rest and a TypeORM layer replacing @loopback/repository? How about Typegoose + Kafka?

Implementation-wise, where would you define the class constructor that's combining RestApplication (implemented by @loopback/rest) with RepositoryMixin (from @loopback/repository), ServiceMixin (from @loopback/service-proxy) and BootMixin (from @loopback/boot)?

The current design allows users to choose the version of repository/service-proxy/boot package to use in their project, e.g. they can use the latest version of boot but the previous semver-major version of repository. I am not sure how important this trait is, maybe most users don't care and want to use the latest versions? Either way, we should consider this aspect and be explicit about our decision.

To conclude, I like the idea of simplifying the default application template 👍 Now we need to find a solution that would not break those good parts of the current design that we want to preserve.

The organization and content of our documentation is a correct reflection of our view, but our users are looking for the documentation of their view.

That makes me wonder if creating two identities out of the current work would help.

• LoopBack - the REST framework.
• A meta framework, which can be used for creating LoopBack and other frameworks.

A very loosely related discussion: RFC: Reorganize code & docs along abstraction levels #5550

In my opinion, the fact that LoopBack is primarily perceived as a REST API framework is because we didn't get to implement non-REST parts that have been always on our long-term roadmap, e.g. GraphQL (#5545) and Kafka (#1925). I am not sure if it's a good idea to further reinforce the REST-centric point of view.

[hacksparrow]

How do you envision the user code for building an application using @loopback/rest and a TypeORM layer replacing @loopback/repository? How about Typegoose + Kafka?

The user would express their strategy in a config object (code or file), our framework should be able to figure out the implementation for them. I haven't thought about how to do it, yet. However, I am sure this is something that's not impossible. If done, it would be a wonderful experience for our users.

Implementation-wise, where would you define the class constructor ...

If changing the concept of LoopBack is too disruptive, maybe @loopback/venus - the REST API framework powered by LoopBack 😄 . Maybe even just venus-framework.

Now we need to find a solution that would not break those good parts of the current design that we want to preserve.

Our current design is pretty solid as a meta framework, no doubt about it. We need not change it; what I am suggesting is to carve out a new entity specifically for the REST framework, and we need not name it LoopBack.

A lot of confusion goes away once LoopBack is looked at as a meta framework, not a REST API framework. However, our documentation, tooling, and examples present it primarily as a REST API framework.

One example:

In my opinion, the fact that LoopBack is primarily perceived as a REST API framework is because we didn't get to implement non-REST parts ...

If the REST aspect of LoopBack is as important as any other, the bias should be removed from the documentation, tooling, and examples to avoid confusion. If the bias is an indication of its importance and demand, maybe we should create a new entity for the LoopBack REST framework and give it a name.

What I feel is that we need another level of abstraction for our users. It will have very little (almost insignificant) affect on the implementation, but documentation based around this new abstraction will be a lot simpler and less confusing.

The new framework need not be hard-coded to REST. It could work something like this:

export class MyApplication extends Venus {
  constructor(options: ApplicationConfig = {}) {
    super(options);
  }
}

const app = new MyApplication({
  transport: 'graphql' // 'rest' | 'graphql' | 'kafka'
});

or simply:

const app = new Venus({ transport: 'graphql' });

[dhmlau]

Great discussion.

I can see it could bring confusion for having ServiceMixin and RepositoryMixin for users who don't use those in their applications and wonder why it appears in the docs.

If changing the concept of LoopBack is too disruptive, maybe @loopback/venus - the REST API framework powered by LoopBack 😄 . Maybe even just venus-framework.

I'm not sure if we're ready to split LoopBack into various parts for different purpose. :). Like some other projects, they can have different flavor, e.g. Node.js, Java, PHP, and not necessarily to break the project/framework into parts.

Perhaps as a start, having what we have now (more REST-style centric), we can start building the other flavor, i.e. GraphQL and Kafka as @bajtos mentioned above. If further down the road when we have more contents, we can decide whether there's a need to separate into different guides for other protocol.

[raymondfeng]

Cross-post: #5549 (comment)

[bajtos]

Few more questions to consider:

1. Where do we want to draw a line between what should be included in the "uber" framework out of the box? We probably agree on rest, repository, service-proxy and boot, but how about authorization, authentication, health, metrics, etc?

2. Our modular architecture makes it easier for our users when we make a breaking change. For example, if the application is not using service-proxy, then the developers don't have to read release notes when a breaking changes is released in service-proxy. With an "uber" framework bundling all together, most breaking changes in dependencies will trigger a major release of the "uber" framework, thus putting more load on users.

3. How are we going to support users who outgrow the default bundled components and want to either remove some of the built-in ones or replace them with a 3rd-party alternative? Will they have to rewrite all dependencies and Application constructor manually? That could be cumbersome. Are we going to implement an eject action to switch a project from "uber" framework to "meta" framework? (See create-react-app for the concept of eject: https://create-react-app.dev/docs/available-scripts#npm-run-eject)

[hacksparrow]

1. Include all common utilities, enable/disable with some config/defaults. So authorization, authentication, health, metrics, will be included but not loaded unless they are configured to be enabled.

2. At this stage, breaking changes in the dependencies be wouldn't be that frequent. Since we have some insight into upcoming breaking changes, if many dependencies are going to release breaking changes, we could bundle them all together in a single major release of the "uber" framework. Unless it is a security issue, it is OK to wait for a while for a new release.

3. We should come up with a design for configurable replacement of most components. Number 1 applies here too.