Skip to content

Commit dcac9f1

Browse files
mosuemmosuem
committed
Merge package:characters into the core monorepo
2 parents 8c5dca8 + 9582360 commit dcac9f1

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

46 files changed

+22919
-0
lines changed

‎pkgs/characters/.github/dependabot.yml‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
# Dependabot configuration file.
2+
# See https://docs.github.com/en/code-security/dependabot/dependabot-version-updates
3+
version: 2
4+
5+
updates:
6+
- package-ecosystem: github-actions
7+
directory: /
8+
schedule:
9+
interval: monthly
10+
labels:
11+
- autosubmit
12+
groups:
13+
github-actions:
14+
patterns:
15+
- "*"

‎pkgs/characters/.github/workflows/publish.yaml‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
# A CI configuration to auto-publish pub packages.
2+
3+
name: Publish
4+
5+
on:
6+
pull_request:
7+
branches: [ master ]
8+
push:
9+
tags: [ 'v[0-9]+.[0-9]+.[0-9]+*' ]
10+
11+
jobs:
12+
publish:
13+
if: ${{ github.repository_owner == 'dart-lang' }}
14+
uses: dart-lang/ecosystem/.github/workflows/publish.yaml@main

‎pkgs/characters/.github/workflows/test-package.yml‎

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
name: Dart CI
2+
3+
on:
4+
# Run on PRs and pushes to the default branch.
5+
push:
6+
branches: [ master ]
7+
pull_request:
8+
branches: [ master ]
9+
schedule:
10+
- cron: "0 0 * * 0"
11+
12+
env:
13+
PUB_ENVIRONMENT: bot.github
14+
15+
jobs:
16+
# Check code formatting and static analysis on a single OS (linux)
17+
# against dev, stable, and 2.19.0 (the package's lower bound).
18+
analyze:
19+
runs-on: ubuntu-latest
20+
strategy:
21+
fail-fast: false
22+
matrix:
23+
sdk: [dev, stable, 3.4]
24+
steps:
25+
- uses: actions/checkout@d632683dd7b4114ad314bca15554477dd762a938
26+
- uses: dart-lang/setup-dart@0a8a0fc875eb934c15d08629302413c671d3f672
27+
with:
28+
sdk: ${{ matrix.sdk }}
29+
- id: install
30+
name: Install dependencies
31+
run: dart pub get
32+
- name: Check formatting
33+
run: dart format --output=none --set-exit-if-changed .
34+
if: matrix.sdk == 'dev' && steps.install.outcome == 'success'
35+
- name: Analyze code
36+
run: dart analyze --fatal-infos
37+
if: always() && steps.install.outcome == 'success'
38+
39+
# Run tests on a matrix consisting of two dimensions:
40+
# 1. OS: ubuntu-latest
41+
# 2. Release channel: dev, stable, and 2.19.0 (the package's lower bound)
42+
test:
43+
needs: analyze
44+
runs-on: ${{ matrix.os }}
45+
strategy:
46+
fail-fast: false
47+
matrix:
48+
os: [ubuntu-latest]
49+
sdk: [dev, stable, 3.4]
50+
steps:
51+
- uses: actions/checkout@d632683dd7b4114ad314bca15554477dd762a938
52+
- uses: dart-lang/setup-dart@0a8a0fc875eb934c15d08629302413c671d3f672
53+
with:
54+
sdk: ${{ matrix.sdk }}
55+
- id: install
56+
name: Install dependencies
57+
run: dart pub get
58+
- name: Run VM tests
59+
run: dart test --platform vm
60+
if: always() && steps.install.outcome == 'success'
61+
- name: Run Chrome tests
62+
run: dart test --platform chrome
63+
if: always() && steps.install.outcome == 'success'

‎pkgs/characters/.gitignore‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
.dart_tool/
2+
.packages
3+
pubspec.lock
4+
doc/api/

‎pkgs/characters/AUTHORS‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
# Below is a list of people and organizations that have contributed
2+
# to the Dart project. Names should be added to the list like so:
3+
#
4+
# Name/Organization <email address>
5+
6+
Google LLC

‎pkgs/characters/CHANGELOG.md‎

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
## 1.3.1-wip
2+
3+
* Fixed README rendering on pub.dev and API docs.
4+
* Require Dart `^3.4.0`
5+
6+
## 1.3.0
7+
8+
* Updated to use Unicode 15.0.0.
9+
10+
## 1.2.1
11+
12+
* Update the value of the pubspec `repository` field.
13+
14+
## 1.2.0
15+
16+
* Fix `Characters.where` which unnecessarily did the iteration and test twice.
17+
* Adds `Characters.empty` constant and makes `Characters("")` return it.
18+
* Changes the argument type of `Characters.contains` to (covariant) `String`.
19+
The implementation still accepts `Object?`, so it can be cast to
20+
`Iterable<Object?>`, but you get warned if you try to call directly with a
21+
non-`String`.
22+
23+
## 1.1.0
24+
25+
* Stable release for null safety.
26+
* Added `stringBeforeLength` and `stringAfterLength` to `CharacterRange`.
27+
* Added `CharacterRange.at` constructor.
28+
* Added `getRange(start, end)` and `characterAt(pos)` to `Characters`
29+
as alternative to `.take(end).skip(start)` and `getRange(pos, pos + 1)`.
30+
* Change some positional parameter names from `other` to `characters`.
31+
32+
## 1.0.0
33+
34+
* Core APIs deemed stable; package version set to 1.0.0.
35+
* Added `split` methods on `Characters` and `CharacterRange`.
36+
37+
## 0.5.0
38+
39+
* Change [codeUnits] getter to [utf16CodeUnits] which returns an iterable.
40+
This avoids leaking that the underlying string has efficient UTF-16
41+
code unit access in the API, and allows the same interface to be
42+
just as efficiently implemented on top of UTF-8.
43+
44+
## 0.4.0
45+
46+
* Added an extension method on `String` to allow easy access to the `Characters`
47+
of the string:
48+
49+
```dart
50+
print('The first character is: ' + myString.characters.first)
51+
```
52+
53+
* Updated Dart SDK dependency to Dart 2.6.0
54+
55+
## 0.3.1
56+
57+
* Added small example in `example/main.dart`
58+
* Enabled pedantic lints and updated code to resolve issues.
59+
60+
## 0.3.0
61+
62+
* Updated API which does not expose the underlying string indices.
63+
64+
## 0.1.0
65+
66+
* Initial release

‎pkgs/characters/LICENSE‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
Copyright 2019, the Dart project authors.
2+
3+
Redistribution and use in source and binary forms, with or without
4+
modification, are permitted provided that the following conditions are
5+
met:
6+
7+
* Redistributions of source code must retain the above copyright
8+
notice, this list of conditions and the following disclaimer.
9+
* Redistributions in binary form must reproduce the above
10+
copyright notice, this list of conditions and the following
11+
disclaimer in the documentation and/or other materials provided
12+
with the distribution.
13+
* Neither the name of Google LLC nor the names of its
14+
contributors may be used to endorse or promote products derived
15+
from this software without specific prior written permission.
16+
17+
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
18+
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
19+
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
20+
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
21+
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
22+
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
23+
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
24+
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
25+
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
26+
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
27+
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

‎pkgs/characters/README.md‎

Lines changed: 133 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,133 @@
1+
[![Build Status](https://github.com/dart-lang/characters/workflows/Dart%20CI/badge.svg)](https://github.com/dart-lang/characters/actions?query=workflow%3A"Dart+CI"+branch%3Amaster)
2+
[![pub package](https://img.shields.io/pub/v/characters.svg)](https://pub.dev/packages/characters)
3+
[![package publisher](https://img.shields.io/pub/publisher/characters.svg)](https://pub.dev/packages/characters/publisher)
4+
5+
[`Characters`][Characters] are strings viewed as
6+
sequences of **user-perceived character**s,
7+
also known as [Unicode (extended) grapheme clusters][Grapheme Clusters].
8+
9+
The [`Characters`][Characters] class allows access to
10+
the individual characters of a string,
11+
and a way to navigate back and forth between them
12+
using a [`CharacterRange`][CharacterRange].
13+
14+
## Unicode characters and representations
15+
16+
There is no such thing as plain text.
17+
18+
Computers only know numbers,
19+
so any "text" on a computer is represented by numbers,
20+
which are again stored as bytes in memory.
21+
22+
The meaning of those bytes are provided by layers of interpretation,
23+
building up to the *glyph*s that the computer displays on the screen.
24+
25+
| Abstraction | Dart Type | Usage | Example |
26+
| --------------------- | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ |
27+
| Bytes | [`ByteBuffer`][ByteBuffer],<br />[`Uint8List`][Uint8List] | Physical layout: Memory or network communication. | `file.readAsBytesSync()` |
28+
| [Code units][] | [`Uint8List`][Uint8List] (UTF&#x2011;8)<br />[`Uint16List`][Uint16List], [`String`][String] (UTF&#x2011;16) | Standard formats for<br /> encoding code points in memory.<br />Stored in memory using one (UTF&#x2011;8) or more (UTF&#x2011;16) bytes. One or more code units encode a code point. | `string.codeUnits`<br />`string.codeUnitAt(index)`<br />`utf8.encode(string)` |
29+
| [Code points][] | [`Runes`][Runes] | The Unicode unit of meaning. | `string.runes` |
30+
| [Grapheme Clusters][] | [`Characters`][Characters] | Human perceived character. One or more code points. | `string.characters` |
31+
| [Glyphs][] | | Visual rendering of grapheme clusters. | `print(string)` |
32+
33+
A Dart `String` is a sequence of UTF-16 code units,
34+
just like strings in JavaScript and Java.
35+
The runtime system decides on the underlying physical representation.
36+
37+
That makes plain strings inadequate
38+
when needing to manipulate the text that a user is viewing, or entering,
39+
because string operations are not working at the grapheme cluster level.
40+
41+
For example, to abbreviate a text to, say, the 15 first characters or glyphs,
42+
a string like "A 🇬🇧 text in English"
43+
should abbreviate to "A 🇬🇧 text in Eng&mldr; when counting characters,
44+
but will become "A 🇬🇧 text in &mldr;"
45+
if counting code units using [`String`][String] operations.
46+
47+
Whenever you need to manipulate strings at the character level,
48+
you should be using the [`Characters`][Characters] type,
49+
not the methods of the [`String`][String] class.
50+
51+
## The Characters class
52+
53+
The [`Characters`][Characters] class exposes a string
54+
as a sequence of grapheme clusters.
55+
All operations on [`Characters`][Characters] operate
56+
on entire grapheme clusters,
57+
so it removes the risk of splitting combined characters or emojis
58+
that are inherent in the code-unit based [`String`][String] operations.
59+
60+
You can get a [`Characters`][Characters] object for a string using either
61+
the constructor [`Characters(string)`][Characters constructor]
62+
or the extension getter `string.characters`.
63+
64+
At its core, the class is an [`Iterable<String>`][Iterable]
65+
where the element strings are single grapheme clusters.
66+
This allows sequential access to the individual grapheme clusters
67+
of the original string.
68+
69+
On top of that, there are operations mirroring the operations
70+
of [`String`][String] that are not index, code-unit or code-point based,
71+
like [`startsWith`][Characters.startsWith]
72+
or [`replaceAll`][Characters.replaceAll].
73+
There are some differences between these and the [`String`][String] operations.
74+
For example the replace methods only accept characters as pattern.
75+
Regular expressions are not grapheme cluster aware,
76+
so they cannot be used safely on a sequence of characters.
77+
78+
Grapheme clusters have varying length in the underlying representation,
79+
so operations on a [`Characters`][Characters] sequence cannot be index based.
80+
Instead, the [`CharacterRange`][CharacterRange] *iterator*
81+
provided by [`Characters.iterator`][Characters.iterator]
82+
has been greatly enhanced.
83+
It can move both forwards and backwards,
84+
and it can span a *range* of grapheme cluster.
85+
Most operations that can be performed on a full [`Characters`][Characters]
86+
can also be performed on the grapheme clusters
87+
in the range of a [`CharacterRange`][CharacterRange].
88+
The range can be contracted, expanded or moved in various ways,
89+
not restricted to using [`moveNext`][CharacterRange.moveNext],
90+
to move to the next grapheme cluster.
91+
92+
Example:
93+
94+
```dart
95+
// Using String indices.
96+
String? firstTagString(String source) {
97+
var start = source.indexOf('<') + 1;
98+
if (start > 0) {
99+
var end = source.indexOf('>', start);
100+
if (end >= 0) {
101+
return source.substring(start, end);
102+
}
103+
}
104+
return null;
105+
}
106+
107+
// Using CharacterRange operations.
108+
Characters? firstTagCharacters(Characters source) {
109+
var range = source.findFirst('<'.characters);
110+
if (range != null && range.moveUntil('>'.characters)) {
111+
return range.currentCharacters;
112+
}
113+
return null;
114+
}
115+
```
116+
117+
[ByteBuffer]: https://api.dart.dev/dart-typed_data/ByteBuffer-class.html "ByteBuffer class"
118+
[CharacterRange.moveNext]: https://pub.dev/documentation/characters/latest/characters/CharacterRange/moveNext.html "CharacterRange.moveNext"
119+
[CharacterRange]: https://pub.dev/documentation/characters/latest/characters/CharacterRange-class.html "CharacterRange class"
120+
[Characters constructor]: https://pub.dev/documentation/characters/latest/characters/Characters/Characters.html "Characters constructor"
121+
[Characters.iterator]: https://pub.dev/documentation/characters/latest/characters/Characters/iterator.html "CharactersRange get iterator"
122+
[Characters.replaceAll]: https://pub.dev/documentation/characters/latest/characters/Characters/replaceAll.html "Characters.replaceAlle"
123+
[Characters.startsWith]: https://pub.dev/documentation/characters/latest/characters/Characters/startsWith.html "Characters.startsWith"
124+
[Characters]: https://pub.dev/documentation/characters/latest/characters/Characters-class.html "Characters class"
125+
[Code Points]: https://unicode.org/glossary/#code_point "Unicode Code Point"
126+
[Code Units]: https://unicode.org/glossary/#code_unit "Unicode Code Units"
127+
[Glyphs]: https://unicode.org/glossary/#glyph "Unicode Glyphs"
128+
[Grapheme Clusters]: https://unicode.org/reports/tr29/#Grapheme_Cluster_Boundaries "Unicode (Extended) Grapheme Cluster"
129+
[Iterable]: https://api.dart.dev/dart-core/Iterable-class.html "Iterable class"
130+
[Runes]: https://api.dart.dev/dart-core/Runes-class.html "Runes class"
131+
[String]: https://api.dart.dev/dart-core/String-class.html "String class"
132+
[Uint16List]: https://api.dart.dev/dart-typed_data/Uint16List-class.html "Uint16List class"
133+
[Uint8List]: https://api.dart.dev/dart-typed_data/Uint8List-class.html "Uint8List class"

‎pkgs/characters/analysis_options.yaml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
include: package:dart_flutter_team_lints/analysis_options.yaml
2+
3+
analyzer:
4+
errors:
5+
prefer_single_quotes: ignore

0 commit comments

Comments
 (0)