Skip to content

Commit e6bd777

Browse files
committed
Use BinaryBuilder-provided iconv
Fixes #36.
1 parent cfa650a commit e6bd777

File tree

6 files changed

+12
-163
lines changed

6 files changed

+12
-163
lines changed

.github/workflows/ci.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ jobs:
1212
fail-fast: false
1313
matrix:
1414
version:
15-
- '1.0'
15+
- '1.3'
1616
- '1' # automatically expands to the latest stable 1.x release of Julia
1717
- 'nightly'
1818
os:

.gitignore

Lines changed: 1 addition & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,4 @@
11
*.jl.cov
22
*.jl.*.cov
33
*.jl.mem
4-
deps/builds
5-
deps/downloads
6-
deps/src
7-
deps/usr
8-
deps/deps.jl
9-
deps/build.log
10-
MANIFEST.TOML
4+
Manifest.toml

Project.toml

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -3,12 +3,11 @@ uuid = "69024149-9ee7-55f6-a4c4-859efe599b68"
33
version = "0.3.3"
44

55
[deps]
6-
BinaryProvider = "b99e7846-7c00-51b0-8f62-c81ae34c0232"
7-
Libdl = "8f399da3-3557-5675-b5ff-fb832c97cbdb"
6+
Libiconv_jll = "94ce4f54-9a6c-5748-9c1c-f9c7231a4531"
87

98
[compat]
10-
BinaryProvider = "0.4.1, 0.5"
11-
julia = "0.7, 1"
9+
Libiconv_jll = "1.16"
10+
julia = "1.3"
1211

1312
[extras]
1413
Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"

README.md

Lines changed: 1 addition & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
[![Build status](https://github.com/JuliaStrings/StringEncodings.jl/workflows/CI/badge.svg)](https://github.com/JuliaStrings/StringEncodings.jl/actions?query=workflow%3ACI+branch%3Amaster)
44
[![Codecov Coverage Status](http://codecov.io/github/JuliaStrings/StringEncodings.jl/coverage.svg?branch=master)](http://codecov.io/github/JuliaStrings/StringEncodings.jl?branch=master)
55

6-
This Julia package provides support for decoding and encoding texts between multiple character encodings. It is currently based on the iconv interface, and supports all major platforms using either the native iconv support or [GNU libiconv](https://www.gnu.org/software/libiconv/). In the future, native Julia support for major encodings will be added.
6+
This Julia package provides support for decoding and encoding texts between multiple character encodings. It is currently based on the iconv interface, and supports all major platforms using [GNU libiconv](https://www.gnu.org/software/libiconv/). In the future, native Julia support for major encodings will be added.
77

88
## Encoding and Decoding Strings
99
*Encoding* a refers to the process of converting a string (of any `AbstractString` type) to a sequence of bytes represented as a `Vector{UInt8}`. *Decoding* refers to the inverse process.
@@ -151,7 +151,3 @@ julia> read(s, String) # Decoding happens automatically here
151151
Do not forget to call `close` on `StringEncoder` and `StringDecoder` objects to finish the encoding process. For `StringEncoder`, this function calls `flush`, which writes any characters still in the buffer, and possibly some control sequences (for stateful encodings). For both `StringEncoder` and `StringDecoder`, `close` checks that there are no incomplete sequences left in the input stream, and raise an `IncompleteSequenceError` if that's the case. It will also free iconv resources immediately, instead of waiting for garbage collection.
152152

153153
Conversion currently raises an error if an invalid byte sequence is encountered in the input, or if some characters cannot be represented in the target enconding. It is not yet possible to ignore such characters or to replace them with a placeholder.
154-
155-
## Notes on Installation on Linux OS
156-
157-
Most Linux distributions provide `iconv` functionalities as part of the base operating system library `glibc`. In normal circumstances, no additional installation of `libiconv` should be required. If you observe such a behavior on your operating system, file an issue with OS details.

deps/build.jl

Lines changed: 0 additions & 124 deletions
This file was deleted.

src/StringEncodings.jl

Lines changed: 6 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -2,20 +2,7 @@
22

33
module StringEncodings
44

5-
using Libdl
6-
7-
# Load in `deps.jl`, complaining if it does not exist
8-
const depsjl_path = joinpath(dirname(@__FILE__), "..", "deps", "deps.jl")
9-
if !isfile(depsjl_path)
10-
error("iconv not installed properly, run Pkg.build(\"StringEncodings\"), restart Julia and try again")
11-
end
12-
include(depsjl_path)
13-
14-
# Module initialization function
15-
function __init__()
16-
# Always check dependencies from `deps.jl`
17-
check_deps()
18-
end
5+
using Libiconv_jll
196

207
using Base.Libc: errno, strerror, E2BIG, EINVAL, EILSEQ
218

@@ -75,13 +62,13 @@ show(io::IO, exc::T) where {T<:Union{IncompleteSequenceError,OutputBufferError}}
7562

7663
function iconv_close(cd::Ptr{Nothing})
7764
if cd != C_NULL
78-
ccall((iconv_close_s, libiconv), Cint, (Ptr{Nothing},), cd) == 0 ||
65+
ccall((:libiconv_close, libiconv), Cint, (Ptr{Nothing},), cd) == 0 ||
7966
throw(IConvError("iconv_close"))
8067
end
8168
end
8269

8370
function iconv_open(tocode::String, fromcode::String)
84-
p = ccall((iconv_open_s, libiconv), Ptr{Nothing}, (Cstring, Cstring), tocode, fromcode)
71+
p = ccall((:libiconv_open, libiconv), Ptr{Nothing}, (Cstring, Cstring), tocode, fromcode)
8572
if p != Ptr{Nothing}(-1)
8673
return p
8774
elseif errno() == EINVAL
@@ -144,7 +131,7 @@ function iconv!(cd::Ptr{Nothing}, inbuf::Vector{UInt8}, outbuf::Vector{UInt8},
144131
inbytesleft_orig = inbytesleft[]
145132
outbytesleft[] = BUFSIZE
146133

147-
ret = ccall((iconv_s, libiconv), Csize_t,
134+
ret = ccall((:libiconv, libiconv), Csize_t,
148135
(Ptr{Nothing}, Ptr{Ptr{UInt8}}, Ref{Csize_t}, Ptr{Ptr{UInt8}}, Ref{Csize_t}),
149136
cd, inbufptr, inbytesleft, outbufptr, outbytesleft)
150137

@@ -176,7 +163,7 @@ function iconv_reset!(s::Union{StringEncoder, StringDecoder})
176163

177164
s.outbufptr[] = pointer(s.outbuf)
178165
s.outbytesleft[] = BUFSIZE
179-
ret = ccall((iconv_s, libiconv), Csize_t,
166+
ret = ccall((:libiconv, libiconv), Csize_t,
180167
(Ptr{Nothing}, Ptr{Ptr{UInt8}}, Ref{Csize_t}, Ptr{Ptr{UInt8}}, Ref{Csize_t}),
181168
s.cd, C_NULL, C_NULL, s.outbufptr, s.outbytesleft)
182169

@@ -278,9 +265,6 @@ read from `stream` into text in the encoding `to`. Calling `close` on the
278265
stream does not close `stream`.
279266
280267
`to` and `from` can be specified either as a string or as an `Encoding` object.
281-
282-
Note that some implementations may accept invalid sequences
283-
in the input data without raising an error.
284268
"""
285269
function StringDecoder(stream::IO, from::Encoding, to::Encoding=enc"UTF-8")
286270
cd = iconv_open(String(to), String(from))
@@ -542,7 +526,7 @@ encode(s::AbstractString, enc::AbstractString) = encode(s, Encoding(enc))
542526

543527
function test_encoding(enc::String)
544528
# We assume that an encoding is supported if it's possible to convert from it to UTF-8:
545-
cd = ccall((iconv_open_s, libiconv), Ptr{Nothing}, (Cstring, Cstring), enc, "UTF-8")
529+
cd = ccall((:libiconv_open, libiconv), Ptr{Nothing}, (Cstring, Cstring), enc, "UTF-8")
546530
if cd == Ptr{Nothing}(-1)
547531
return false
548532
else

0 commit comments

Comments
 (0)