clang-format 19 breaks ipynb files

[njzjz]

We know that ipynb files are JSON files. Give an example ipynb file as below:

{
  "a": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
}

clang-format test.ipynb

In clang-format 18, nothing changes.

{
  "a": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
}

In clang-format 19.1.0, the JSON is not valid anymore after formating:

{
  "a" :
      "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
      "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
      "aaaaaaaaa"
}

But if one rename test.ipynb to test.json, nothing is broken.

---

I encounter this issue when using the pre-commit, which formats ipynb files by default.

[rymiel]

We know that ipynb files are JSON files.

clang-format doesn't know that. It'd probably have to be added somewhere here

llvm-project/clang/lib/Format/Format.cpp

Lines 3958 to 3959 in 5064c4c

	if (FileName.ends_with_insensitive(".json"))
	return FormatStyle::LK_Json;

In clang-format 18, nothing changes.

I cannot reproduce that, every version of clang-format should behave equally here because they all treat .ipynb as a C++ file.

[njzjz]

In clang-format 18, nothing changes.

I cannot reproduce that, every version of clang-format should behave equally here because they all treat .ipynb as a C++ file.

I see. It's related to my .clang-format config:

---
BasedOnStyle: Google
... 

The default config doesn't have such behavior. But it seems that the Google style changes in clang-format 19.

[HDembinski]

I have the same problem as @njzjz and I am surprised that not more people are commenting here. My .clang-format contains this section for json

---
Language: Json
BasedOnStyle: llvm
...

clang-format should not fix ipynb files.

[HDembinski]

A workaround is to create .clang-format-ignore and put the ipynb files in, see answer to this question.
https://stackoverflow.com/questions/50647960/how-to-ignore-files-or-directories-with-clang-format-3-9

[mydeveloperday]

Why would you need to ignore it? Why are you giving clang the files?

[HDembinski]

We are not giving clang the files, we are running clang-format through pre-commit with standard settings, see https://github.com/pre-commit/mirrors-clang-format (as posted by @njzjz)

The bug is that clang-format started recently (I cannot say which version) to break .ipynb files, at least when you configure it to format json files. I want clang-format to format my json files. I don't want it to break my ipynb files. I would never expect clang-format to even touch my ipynb files, because they do not have the extension "json".

clang-format should ignore ipynb files, right? The code that @rymiel cited is looking for the file ending. That logic must be flawed somehow. I am just summarizing info that is already in this ticket.

[njzjz]

pre-commit identifies ipynb files as JSON files, and thus pass these files to clang-format. A list of extensions that pre-commit identifies can be found at https://github.com/pre-commit/identify/blob/main/identify/extensions.py.

2.5k GitHub repositories use this pre-commit hook: https://github.com/search?q=repo%3A+https%3A%2F%2Fgithub.com%2Fpre-commit%2Fmirrors-clang-format+path%3A.pre-commit-config.yaml&type=code

[HDembinski]

Thanks, @njzjz, that explains why the notebooks are passed to clang-format and it is not clang-format which is doing that. Then the question remains why clang-format is destroying long strings. @njzjz's initial description shows exactly what happens.

I neither know the JSON nor the ipynb format very well, so I was wondering whether a notebook is actually valid JSON. It is seems to be, there is a schema for it and all:
https://github.com/jupyter/nbformat/blob/main/nbformat/v4/nbformat.v4.schema.json

The bug seems to be in clang-format, breaking long strings into multiple lines is apparently not allowed, so why is this happening?
https://stackoverflow.com/questions/2392766/are-multi-line-strings-allowed-in-json

Jupyter notebooks accept long strings broken into lists of strings, but that cannot be assumed to work in general, so the only option for clang-format is to not break long strings into multi-line strings.

[owenca]

You need redirect the input to stdin and use -assume-filename, e.g.:

$ clang-format -assume-filename=.json < test.ipynb | diff test.ipynb -
$