feat: output full bson objects MONGOSH-1285

[lerouxb]

MONGOSH-1285

I built this on top of @addaleax's wip branch, so the types are still mostly from there.

---

The short version of what we're trying to do: For BSON that we get from the database we want to print all of it, untruncated, when it gets evaluated or otherwise inspected with util.inspect().

---

This solution wraps the ServiceProvider with another class that implements all the same methods. Then:

• all methods that return cursors have the returned cursors intercepted and wrapped so that the relevant methods on there that return documents are replaced with new methods that recursively install our inspect function.
• all methods that return promises of bson have their results intercepted and we recursively install our inspect function on the result.
• all other methods are just forwarded with their results unchanged.

I have been testing it with this document which should be affected by the inspectOptions depth, maxArrayLength and maxStringLength:

db.test.insertOne({
  array: Array.from(Array(1000), (_,i) => i),
  string: 'All work and no play makes Jack a dull boy. '.repeat(250),
  object: {
    foo: {
      bar: {
        baz: {
          qux: {
            quux: {
              corge: {
                grault: 'If you can read this, you are too close.'
              }
            }
          }
        }
      }
    }
  }
});

And this doc has every BSON type which is useful for testing that we're not messing up existing BSON formatting:

db.test.insertOne({
    double: Double(1.2),
    doubleThatIsAlsoAnInteger: Double(1),
    string: 'Hello, world!',
    binData: Binary(Buffer.from([1, 2, 3])),
    boolean: true,
    date: Date('2023-04-05T13:25:08.445Z'),
    null: null,
    regex: BSONRegExp('pattern', 'i'),
    javascript: Code('function() {}'),
    symbol: BSONSymbol('symbol'),
    javascriptWithScope: Code('function() {}', { foo: 1, bar: 'a' }),
    int: Int32(12345),
    timestamp: Timestamp(Long('7218556297505931265')),
    long: Long('123456789123456789'),
    decimal: Decimal128(
      Buffer.from([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16])
    ),
    minKey: MinKey(),
    maxKey: MaxKey(),

    binaries: {
      generic: Binary(Buffer.from([1, 2, 3]), 0),
      functionData: Binary(Buffer.from('//8='), 1),
      binaryOld: Binary(Buffer.from('//8='), 2),
      uuidOld: Binary(Buffer.from('c//SZESzTGmQ6OfR38A11A=='), 3),
      uuid: UUID('AAAAAAAA-AAAA-4AAA-AAAA-AAAAAAAAAAAA'),
      md5: Binary(Buffer.from('c//SZESzTGmQ6OfR38A11A=='), 5),
      encrypted: Binary(Buffer.from('c//SZESzTGmQ6OfR38A11A=='), 6),
      compressedTimeSeries: Binary(
        Buffer.from(
          'CQCKW/8XjAEAAIfx//////////H/////////AQAAAAAAAABfAAAAAAAAAAEAAAAAAAAAAgAAAAAAAAAHAAAAAAAAAA4AAAAAAAAAAA==',
          'base64'
        ),
      ),
      custom: Binary(Buffer.from('//8='), 128),
    },

    dbRef: DBRef('namespace', ObjectId('642d76b4b7ebfab15d3c4a78')),
  });

You can test that the find cursor's tryNext was replaced by running:

> db.test.find()

Notice that the whole array, string and object all printed. However, this specific case already worked before because we special-case printing cursors and a few other things.

This will exercise the inspect function on the top-level array that was returned.

You can test that it recursively installed it on the documents inside that array by running:

> db.test.find().toArray()[0]

or even

> config.set('inspectDepth', 2);
> f = { a: { b: { c: { d: 1 } } } }
{ a: { b: { c: [Object] } } } # notice that it truncated
> db.test.find().toArray()[0].object # this will print the whole object, not just 2 levels

This needs a lot of tests. There are almost certainly some cases left that I've missed. And I'm unsure about some details. Just opening to have a discussion. Oh and we might want a way for users to opt out of it.

[lerouxb]

A way around all this is to just make one function similar to bsonMethod() for each unique return type, kinda like what I did for the cursor methods.

[lerouxb]

btw it is VERY easy to accidentally pass initialServiceProvider (ie. the unwrapped value) to something in place of this.initialServiceProvider. Ask me how I know..

[lerouxb]

Maybe better to reuse an isSimpleObject() test like https://github.com/mongodb-js/devtools-shared/blob/ff08cc5e55212890f1d8635f545474201b1e1167/packages/mongodb-redact/src/utils.ts

[Copilot]

Pull request overview

This PR implements deep inspection for BSON objects returned from the database by wrapping the ServiceProvider to intercept and modify document inspection behavior. When BSON data is retrieved, it recursively installs custom inspect functions that override Node.js's default truncation limits for depth, array length, and string length.

Key changes:

• Introduced DeepInspectServiceProviderWrapper and cursor wrappers to intercept database responses
• Added addCustomInspect function that recursively attaches custom inspection behavior to BSON documents
• Updated ShellInstanceState to conditionally wrap service providers based on platform compatibility

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
packages/shell-api/src/shell-instance-state.ts	Wraps the initial service provider with deep inspect functionality when supported
packages/shell-api/src/shell-api.spec.ts	Adds missing test setup properties for service provider stub
packages/shell-api/src/runtime-independence.spec.ts	Updates test assertions to account for wrapped service provider
packages/shell-api/src/pick-methods-by-return-type.ts	Defines utility type for filtering methods by return type
packages/shell-api/src/deep-inspect-service-provider-wrapper.ts	Implements service provider wrapper that intercepts cursor and BSON-returning methods
packages/shell-api/src/deep-inspect-run-command-cursor-wrapper.ts	Wraps run command cursors to apply custom inspect to results
packages/shell-api/src/deep-inspect-find-cursor-wrapper.ts	Wraps find cursors to apply custom inspect to documents
packages/shell-api/src/deep-inspect-change-stream-wrapper.ts	Wraps change streams to apply custom inspect to change events
packages/shell-api/src/deep-inspect-aggregation-cursor-wrapper.ts	Wraps aggregation cursors to apply custom inspect to results
packages/shell-api/src/custom-inspect.ts	Implements recursive custom inspection logic for BSON documents
packages/service-provider-core/src/service-provider.ts	Adds deepInspectWrappable interface property and default implementation
packages/service-provider-core/src/admin.ts	Updates getNewConnection return type to ServiceProvider
packages/java-shell/src/main/kotlin/com/mongodb/mongosh/service/JavaServiceProvider.kt	Disables deep inspect wrapping for Java shell platform
packages/e2e-tests/test/e2e-oidc.spec.ts	Updates test to access wrapped service provider via _sp property

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[addaleax]

Looks good! Should I try to take a stab at trying to deduplicate the cursor implementations a bit?

If we don't do that, I think I'd have a mild preference for grouping all the deep-* files in a shared subdirectory, since they're a fairly isolated thing of their own

[lerouxb]

❤️

[addaleax]

Just approving this for Github, but I'm not going to merge without approval from @lerouxb on my own changes in this PR 🙂