Skip to content

Commit a62877b

Browse files
committed
Docs: wrap files.
1 parent 16d5ef0 commit a62877b

File tree

12 files changed

+487
-455
lines changed

12 files changed

+487
-455
lines changed

server/src/ide/filewatcher.rs

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -783,7 +783,8 @@ mod tests {
783783
m
784784
}
785785
_ = sleep(Duration::from_secs(3)) => {
786-
// The backtrace shows what message the code was waiting for; otherwise, it's an unhelpful error message.
786+
// The backtrace shows what message the code was waiting for;
787+
// otherwise, it's an unhelpful error message.
787788
panic!("Timeout waiting for message:\n{}", Backtrace::force_capture());
788789
}
789790
}

server/src/lexer.rs

Lines changed: 130 additions & 136 deletions
Large diffs are not rendered by default.

server/src/lexer/pest/c.pest

Lines changed: 86 additions & 90 deletions
Original file line numberDiff line numberDiff line change
@@ -15,22 +15,22 @@
1515
// [http://www.gnu.org/licenses](http://www.gnu.org/licenses).
1616
//
1717
// `c.pest` - Pest parser definition for the C language
18-
// ====================================================
18+
// =============================================================================
1919
//
2020
// Comments
21-
// --------
21+
// -----------------------------------------------------------------------------
2222
doc_block = _{ inline_comment | block_comment }
2323

24-
// Per the [C standard, section
25-
// 6.4.3](https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3220.pdf#page=65),
24+
// Per the
25+
// [C standard, section 6.4.3](https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3220.pdf#page=65),
2626
// "white-space consists of: (space, horizontal tab, new-line, vertical tab, and
2727
// form-feed)." Omit newlines, since the rest of this parser uses these.
2828
vertical_tab = { "\x0B" }
2929
form_feed = { "\x0C" }
3030
white_space = { (" " | "\t" | vertical_tab | form_feed)* }
3131

32-
// The [C standard, section
33-
// 6.4.9](https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3220.pdf#page=65),
32+
// The
33+
// [C standard, section 6.4.9](https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3220.pdf#page=65),
3434
// defines inline and block comments.
3535
//
3636
// ### Inline comments
@@ -50,110 +50,106 @@ block_comment_closing_delim_1 = { unused }
5050
block_comment_closing_delim_2 = { unused }
5151

5252
// Code
53-
// ----
53+
// -----------------------------------------------------------------------------
5454
//
55-
// Per the [C standard, section
56-
// 5.1.1.2](https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3220.pdf#page=24),
55+
// Per the
56+
// [C standard, section 5.1.1.2](https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3220.pdf#page=24),
5757
// if a line of code ends with a backslash, it continues on the next line. This
5858
// is a logical line; treat it as a single line. Therefore, consider a
5959
// backslash-newline (or anything that's not a newline) a part of the current
6060
// logical line. Note that this parser doesn't apply this rule to comments
6161
// (which, per the spec, it should) for several reasons:
6262
//
63-
// 1. Comments continued onto another line don't look like a comment; this
64-
// would confuse most developers.
65-
//
66-
// 2. The backslash-newline in a comment creates a [hard line
67-
// break](https://spec.commonmark.org/0.31.2/#hard-line-breaks) in Markdown,
68-
// which means inserting a hard line break this way in an inline comment
69-
// requires the next line to omit the inline comment delimiters. For
70-
// example: 
71-
//
72-
// ```C
73-
// // This is a hard line break\
74-
// followed by a comment which must not include the // inline comment
75-
// // delimiter on the line after the line break, but which must
76-
// include them on following lines.
77-
// ```
78-
//
79-
// 3. The CodeChat Editor web-to-code function produces incorrect results in
80-
// this case, adding a comment delimiter when it shouldn't. To fix this, it
81-
// would have to look for a backslash newline only in C/C++-like languages.
63+
// 1. Comments continued onto another line don't look like a comment; this would
64+
// confuse most developers.
65+
//
66+
// 2. The backslash-newline in a comment creates a
67+
// [hard line break](https://spec.commonmark.org/0.31.2/#hard-line-breaks) in
68+
// Markdown, which means inserting a hard line break this way in an inline
69+
// comment requires the next line to omit the inline comment delimiters. For
70+
// example: 
71+
//
72+
// ```C
73+
// // This is a hard line break\
74+
// followed by a comment which must not include the // inline comment
75+
// // delimiter on the line after the line break, but which must
76+
// include them on following lines.
77+
// ```
78+
//
79+
// 3. The CodeChat Editor web-to-code function produces incorrect results in
80+
// this case, adding a comment delimiter when it shouldn't. To fix this, it
81+
// would have to look for a backslash newline only in C/C++-like languages.
8282
logical_line_char = _{ ("\\" ~ NEWLINE) | not_newline }
8383
code_line_token = _{ logical_line_char }
8484

8585
// Dedenter
86-
// --------
86+
// -----------------------------------------------------------------------------
8787
//
8888
// This parser runs separately; it dedents block comments. There are several
8989
// cases:
9090
//
91-
// * A single line: `/* comment */`. No special handling needed.
92-
// * Multiple lines, in two styles.
93-
// * Each line of the comment is not consistently whitespace-indented. No
94-
// special handling needed. For example:
95-
//
96-
// ```C
97-
// /* This is
98-
// not
99-
// consistently indented. */
100-
// ```
101-
//
102-
// * Each line of the comment is consistently whitespace-indented; for
103-
// example:
104-
//
105-
// ```C
106-
// /* This is
107-
// consistently indented. */
108-
// ```
109-
//
110-
// Consistently indented means the first non-whitespace character on a
111-
// line aligns with, but never comes before, the comment's start.
112-
// Another example:
113-
//
114-
// ```C
115-
// /* This is
116-
// correct
117-
//
118-
// indentation.
119-
// */
120-
// ```
121-
//
122-
// Note that the third (blank) line doesn't have an indent; since that
123-
// line consists only of whitespace, this is OK. Likewise, the last line
124-
// (containing the closing comment delimiter of `*/`) consists only of
125-
// whitespace after the comment delimiters are removed.
126-
//
127-
// * Each line of the comment is consistently asterisk-indented; for
128-
// example:
129-
//
130-
// ```C
131-
// /* This is
132-
// * correct
133-
// *
134-
// * indentation.
135-
// */
136-
// ```
137-
//
138-
// Note that in this case, no whitespace-only lines are allowed.
139-
// Instead, the special case is lines which have a newline immediately
140-
// after the `*`.
91+
// * A single line: `/* comment */`. No special handling needed.
92+
// * Multiple lines, in two styles.
93+
// * Each line of the comment is not consistently whitespace-indented. No
94+
// special handling needed. For example:
95+
//
96+
// ```C
97+
// /* This is
98+
// not
99+
// consistently indented. */
100+
// ```
101+
//
102+
// * Each line of the comment is consistently whitespace-indented; for
103+
// example:
104+
//
105+
// ```C
106+
// /* This is
107+
// consistently indented. */
108+
// ```
109+
//
110+
// Consistently indented means the first non-whitespace character on a line
111+
// aligns with, but never comes before, the comment's start. Another
112+
// example:
113+
//
114+
// ```C
115+
// /* This is
116+
// correct
117+
//
118+
// indentation.
119+
// */
120+
// ```
121+
//
122+
// Note that the third (blank) line doesn't have an indent; since that line
123+
// consists only of whitespace, this is OK. Likewise, the last line
124+
// (containing the closing comment delimiter of `*/`) consists only of
125+
// whitespace after the comment delimiters are removed.
126+
//
127+
// * Each line of the comment is consistently asterisk-indented; for example:
128+
//
129+
// ```C
130+
// /* This is
131+
// * correct
132+
// *
133+
// * indentation.
134+
// */
135+
// ```
136+
//
137+
// Note that in this case, no whitespace-only lines are allowed. Instead,
138+
// the special case is lines which have a newline immediately after the `*`.
141139
//
142140
// To implement this dedenting, we must have two paths to accepting the contents
143141
// of a block comment. Otherwise, this parser rejects the block (it cannot be
144142
// dedented). The approach:
145143
//
146-
// 1. The space-indented path. This requires:
147-
// 1. The first line ends with a newline. (`valid_first_line`)
148-
// 2. Non-first lines with contents must be properly indented. If a
149-
// non-first line ends in a newline, it must not be the last line.
150-
// (`dedented_line`)
151-
// 3. A whitespace-only line must not be the last line, unless it has
152-
// exactly the indent needed to align the closing comment delimiter
153-
// (`last_line`).
154-
// 2. The asterisk-indented path. The requirements are the same as the
155-
// space-indented path, though the proper indent includes an asterisk in the
156-
// correct location.
144+
// 1. The space-indented path. This requires:
145+
// 1. The first line ends with a newline. (`valid_first_line`)
146+
// 2. Non-first lines with contents must be properly indented. If a non-first
147+
// line ends in a newline, it must not be the last line. (`dedented_line`)
148+
// 3. A whitespace-only line must not be the last line, unless it has exactly
149+
// the indent needed to align the closing comment delimiter (`last_line`).
150+
// 2. The asterisk-indented path. The requirements are the same as the
151+
// space-indented path, though the proper indent includes an asterisk in the
152+
// correct location.
157153
dedenter = {
158154
SOI ~ indent ~ valid_first_line ~ (valid_space_line+ | valid_star_line+) ~ DROP ~ EOI
159155
}
Lines changed: 35 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -1,54 +1,53 @@
11
Parser design
2-
=============
2+
================================================================================
33

44
The CodeChat Editor uses the [Pest parser](https://pest.rs/), a Rust
55
implementation of a parsing expression grammar (or PEG). The purpose of the
66
parser from a CodeChat Editor perspective is to classify a source file into code
77
blocks and doc blocks. To accomplish this goal, grammar files (`.pest`) are
88
divided into:
99

10-
* A shared grammar ([shared.pest](shared.pest)), which contains basic
11-
definitions applicable to all languages;
12-
* A language-specific grammar, which builds on these shared definitions by
13-
providing necessary language-specific customizations.
10+
* A shared grammar ([shared.pest](shared.pest)), which contains basic
11+
definitions applicable to all languages;
12+
* A language-specific grammar, which builds on these shared definitions by
13+
providing necessary language-specific customizations.
1414

1515
In particular, a language-specific grammar must provide:
1616

17-
* The definition of a `doc_block`; for most languages, `doc_block = _{
18-
inline_comment | block_comment }`. However, languages which lack an inline
19-
comment (such as CSS) or a block comment (such as Python) would contain only
20-
the appropriate comment type.
17+
* The definition of a `doc_block`; for most languages, `doc_block = _{
18+
inline_comment | block_comment }`. However, languages which lack an inline
19+
comment (such as CSS) or a block comment (such as Python) would contain only
20+
the appropriate comment type.
2121

22-
* Inline comment definitions:
22+
* Inline comment definitions:x
2323

24-
* Opening inline delimiter(s) supported by the language. Three inline
25-
comment delimiters must be defined for a language. For C, this is:
24+
* Opening inline delimiter(s) supported by the language. Three inline comment
25+
delimiters must be defined for a language. For C, this is:
2626

27-
```
28-
inline_comment_delims = _{ inline_comment_delim_0 }
29-
inline_comment_delim_0 = { "//" }
30-
inline_comment_delim_1 = { unused }
31-
inline_comment_delim_2 = { unused }
32-
```
33-
34-
* A token which defines characters in the body of on an inline comment.
35-
For Python, this is:
36-
37-
```
38-
inline_comment_char = { not_newline }
39-
```
27+
```
28+
inline_comment_delims = _{ inline_comment_delim_0 }
29+
inline_comment_delim_0 = { "//" }
30+
inline_comment_delim_1 = { unused }
31+
inline_comment_delim_2 = { unused }
32+
```
4033
41-
* Block comment definitions: provide opening and closing delimiter
42-
definitions. For C, this is:
34+
* A token which defines characters in the body of on an inline comment. For
35+
Python, this is:
4336
4437
```
45-
block_comment = { block_comment_0 }
46-
block_comment_opening_delim_0 = { "/*" }
47-
block_comment_opening_delim_1 = { unused }
48-
block_comment_opening_delim_2 = { unused }
49-
block_comment_closing_delim_0 = { "*/" }
50-
block_comment_closing_delim_1 = { unused }
51-
block_comment_closing_delim_2 = { unused }
38+
inline_comment_char = { not_newline }
5239
```
53-
54-
* `code_line_token`, a token used to recognize tokens in a code line.
40+
* Block comment definitions: provide opening and closing delimiter definitions.
41+
For C, this is:
42+
43+
```
44+
block_comment = { block_comment_0 }
45+
block_comment_opening_delim_0 = { "/*" }
46+
block_comment_opening_delim_1 = { unused }
47+
block_comment_opening_delim_2 = { unused }
48+
block_comment_closing_delim_0 = { "*/" }
49+
block_comment_closing_delim_1 = { unused }
50+
block_comment_closing_delim_2 = { unused }
51+
```
52+
53+
* `code_line_token`, a token used to recognize tokens in a code line.

server/src/lexer/pest/python.pest

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -15,29 +15,29 @@
1515
// [http://www.gnu.org/licenses](http://www.gnu.org/licenses).
1616
//
1717
// `python.pest` - Pest parser definition for Python
18-
// =================================================
18+
// =============================================================================
1919
doc_block = _{ inline_comment }
2020

21-
// Per the [Python language
22-
// reference](https://docs.python.org/3/reference/lexical_analysis.html#indentation),
21+
// Per the
22+
// [Python language reference](https://docs.python.org/3/reference/lexical_analysis.html#indentation),
2323
// leading whitespace used to determine the indentation level consists of spaces
2424
// and tabs.
2525
white_space = { (" " | "\t")* }
2626

2727
// Inline comments
28-
// ---------------
28+
// -----------------------------------------------------------------------------
2929
inline_comment_delims = _{ inline_comment_delim_0 }
3030
inline_comment_delim_0 = { "#" }
3131
inline_comment_delim_1 = { unused }
3232
inline_comment_delim_2 = { unused }
33-
// Per the [Python language reference, section
34-
// 2.1.3](https://docs.python.org/3/reference/lexical_analysis.html#comments),
33+
// Per the
34+
// [Python language reference, section 2.1.3](https://docs.python.org/3/reference/lexical_analysis.html#comments),
3535
// comments end at the end of a physical line. There's no C-like backslash that
3636
// can join physical lines into logical lines for comments.
3737
inline_comment_char = { not_newline }
3838

3939
// Block comments
40-
// --------------
40+
// -----------------------------------------------------------------------------
4141
//
4242
// Other languages support block comments; even though Python doesn't, the
4343
// following must be defined. Block comments never combine.
@@ -50,7 +50,7 @@ block_comment_closing_delim_1 = { unused }
5050
block_comment_closing_delim_2 = { unused }
5151

5252
// Code blocks
53-
// -----------
53+
// -----------------------------------------------------------------------------
5454
code_line_token = _{ long_string | short_string | not_newline }
5555
long_string = _{
5656
// The opening string delimiter.
@@ -73,7 +73,7 @@ short_string = _{
7373
}
7474

7575
// Dedenter
76-
// --------
76+
// -----------------------------------------------------------------------------
7777
dedenter = { unused }
7878

7979
/// CodeChat Editor lexer: cpp.

0 commit comments

Comments
 (0)