Skip to content

Commit 205ae95

Browse files
dmillerdmiller
committed
Started draft for QME
1 parent 0256405 commit 205ae95

File tree

1 file changed

+332
-0
lines changed

1 file changed

+332
-0
lines changed

‎_drafts/2024-08-05-qualified-methods-for-ClojureCLR.md‎

Lines changed: 332 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,332 @@
1+
---
2+
layout: post
3+
title: Qualfied methods -- for ClojureCLR
4+
date: 2024-08-05 00:00:00 -0500
5+
categories: general
6+
---
7+
8+
Clojure has introduced a new _qualified methods_ feature allows for Java methods to be passed to higher-order functions. This feature also provides alternative ways to invoke methods and constructors, and new ways to specify type hints. We need to enhance this mechanism for ClojureCLR in the same way we enhanced 'classic' interop.
9+
10+
I'll start with a quick review of 'classic' interop, including the additions made for interop with the CLR.
11+
I'll introduce the new qualified methods mechanism.
12+
I'll conclude with looking at how the CLR extras will be incorporated in the new mechanism.
13+
14+
# 'Classic' interop
15+
16+
I'm going to assume basic familiarity with interoperabiltiy with the JVM/CLR, at least prior to the introduction of qualified methods. If you need a refresher, you can look at the [Java interop section](https://clojure.org/reference/java_interop) of the Clojure reference.
17+
18+
There are several pages on the ClojureCLR wiki that talk about the additional features of CLR interop:
19+
20+
- [Basic CLR interop](https://github.com/clojure/clojure-clr/wiki/Basic-CLR-interop).
21+
- [`ByRef` and `params`](https://github.com/clojure/clojure-clr/wiki/ByRef-and-params)
22+
- [Calling generic methods](https://github.com/clojure/clojure-clr/wiki/Calling-generic-methods)
23+
24+
25+
## Class access
26+
27+
Symbols that represent the names of types are resolved to `Type` objects.
28+
29+
30+
```clojure
31+
-> System.String ; => System.String
32+
-> String ; => System.String
33+
-> (class String) ; => System.RuntimeType
34+
```
35+
36+
Classes can be `import`ed into a Clojure namespece so that the namespace of the type can be omitted, as with `String` above. (There is a default set of imports that are always available. See the note at the end for how that set is computed.)
37+
38+
There are types in the CLR that can not be named by symbols. (I guess Java does not yet have this problem.) See the note at the end for a few comments about this.
39+
40+
## Member access
41+
42+
The classic list of ways to access members of a class are:
43+
44+
```clojure
45+
(.instanceMember instance args*)
46+
(.instanceMember Classname args*)
47+
(.-instanceField instance)
48+
(Classname/staticMethod args*)
49+
Classname/staticField
50+
```
51+
52+
The Lisp reader is tasked with translating these into the _dot_ special form; read all about it [here](https://clojure.org/reference/java_interop#_the_dot_special_form). Generally, you should use the forms above rather than using the _dot_ special form directly.
53+
54+
## CLR augmentations
55+
56+
For CLR interop, we had to add some additional functionality, primarily for calling generic methods and for working with `ByRef` and `params` arguments.
57+
58+
If you are familiar with C#, you have seen `ref`, `in`, and `out` used in method signatures. There is no distinction of these at the CLR level. C# adds `in` and `out` for additional compile-time analysis. Given that we don't have uninitialized variables in Clojure and that CLR doesn't distinguish, ClojureCLR only provide a `by-ref` mechanism. The example given on the wiki page looks at a class defined by:
59+
60+
```C#
61+
public class C1
62+
{
63+
public int m3(int x) { return x; }
64+
public int m3(ref int x) { x = x + 1; return x+20; }
65+
public string m5(string x, ref int y) { y = y + 10; return x + y.ToString(); }
66+
public int m5(int x, ref int y) { y = y + 100; return x+y; }
67+
}
68+
```
69+
70+
To call `m3` with a `ref` argument, you would use:
71+
72+
```clojure
73+
(let [n (int n) ]
74+
(.m3 c (by-ref n)))
75+
```
76+
77+
The type hint provided by the `(int n)` is required -- otherwise the it will try to match a `ref object` parameter.
78+
79+
The `by-ref` is a syntactic form that can only be used at the top-level of interop calls, as shown here. It can only wrap a local variable. (`by-ref` can also be used in `definterface`, `deftype`, and the like.) And yes, the value of the local variable `n` is updated by the call -- yep, that binding is not immutable. You do not want to know how this is done.
80+
81+
For `params`, consider the class:
82+
83+
```C#
84+
namespace dm.interop
85+
{
86+
public class C6
87+
{
88+
public static int sm1(int x, params object[] ys)
89+
{
90+
return x + ys.Length;
91+
}
92+
public static int sm1(int x, params string[] ys)
93+
{
94+
int count = x;
95+
foreach (String y in ys)
96+
count += y.Length;
97+
return count;
98+
}
99+
public static int m2(ref int x, params object[] ys)
100+
{
101+
x += ys.Length;
102+
return ys.Length;
103+
}
104+
}
105+
}
106+
```
107+
108+
Method `sm1` is overloaded on the `params` argument.
109+
You can access the first overload with either of these:
110+
111+
```clojure
112+
(dm.interop.C6/sm1 12 #^objects (into-array Object [1 2 3] ))
113+
(dm.interop.C6/sm1 12 #^"System.Object[]" (into-array Object [1 2 3]))
114+
```
115+
116+
The second overload is accessed with:
117+
118+
```clojure
119+
(dm.interop.C6/sm1 12 #^"System.String[]" (into-array String ["abc" "de" "f"]))
120+
(dm.interop.C6/sm1 12 #^"System.String[]" (into-array ["abc" "de" "f"]))
121+
```
122+
123+
Make me one with everything.
124+
125+
```clojure
126+
(defn c6m2 [x]
127+
(let [n (int x)
128+
v (dm.interop.C6/m2 (by-ref n) #^objects (into-array Object [1 2 3 4]))]
129+
[n v]))
130+
```
131+
132+
## Generic methods
133+
134+
We are talking here about methods with type paremeters, not methods that are part of a generic type.
135+
Often you don't need to do anything special:
136+
137+
```clojure
138+
(import 'System.Linq.Enumerable)
139+
(seq (Enumerable/Where [1 2 3 4 5] even?)) ; => (2 4)
140+
```
141+
142+
There are actually two overloads of `Where` in `Enumerable`.
143+
144+
```C#
145+
public static IEnumerable<TSource> Where<TSource>(
146+
this IEnumerable<TSource> source,
147+
Func<TSource, bool> predicate
148+
)
149+
150+
public static IEnumerable<TSource> Where<TSource>(
151+
this IEnumerable<TSource> source,
152+
Func<TSource, int, bool> predicate
153+
)
154+
```
155+
156+
The type inferencing mechanism built into ClojureCLR can figure out that `even?` supports one argument but not two, so it can be coerced to a `Func<Object,bool>` but not a `Func<Object,int,bool>`. Thus, it can select the first overload.
157+
158+
The wiki page for this discusses how to deal with situations where the function might not be so accommodating. Some of these techniques involve macros to define `System.Func` and `System.Action` delegates from Clojure functions. This is a bit of a pain, but it is not too bad.
159+
160+
But there are still situations where more is required.
161+
162+
```clojure
163+
(def r3 (Enumerable/Repeat 2 5)) ;; fails with
164+
165+
Execution error (InvalidOperationException) at System.Diagnostics.StackFrame/ThrowNoInvokeException (NO_FILE:0).
166+
Late bound operations cannot be performed on types or methods for which ContainsGenericParameters is true.
167+
```
168+
169+
We need to know the type parameters for the generic method `Enumerable/Repeat` at compile (load) time. The `type-args` macro can be used to supply type arguments for the method:
170+
171+
```clojure
172+
(seq (Enumerable/Repeat (type-args Int32) 2 5)) ;=> (2 2 2 2 2)
173+
```
174+
175+
# The new qualified methods feature
176+
177+
The new qualified methods feature is used to provide methods to higher-order functions, for example, passing a Java method to `map`.
178+
The new feature is described [here](https://clojure.org/news/2024/04/28/clojure-1-12-alpha10#method_values).
179+
180+
Qualified methods are specified as follows:
181+
182+
- `Classname/method` -- refers to a static method
183+
- `Classname/.method` -- refers to an instance method
184+
- `Classname/new` -- refers to a constructor
185+
186+
These can be used as values (e.g., passed to higher-order functions) or as invocations.
187+
188+
By invocation we mean appearing as the first element in a `(func args)` form. Thus
189+
190+
```clojure
191+
(String/Format format-str arg1) ;; static method invocation
192+
(String/.ToUpper s) ;; instance method invocation
193+
(String/new init-char count) ;; constructor invocation
194+
```
195+
196+
For static methods, this is our original syntax. For instance methods, this would compare to `(.ToUpper ^String s)`.
197+
The third example is equivalent to `(String. \a 12)`. We gain the ability to directly specify the type on the instance method.
198+
(Though see `:param-tags` below for a bonus.)
199+
200+
Using qualified methods as values is the more interesting case. Rather than needing to wrap a method in a function, as in
201+
202+
```clojure
203+
(map #(.ToUpper ^String %) ["abc" "def" "ghi"])
204+
```
205+
206+
you can just use the qualified method directly:
207+
208+
```clojure
209+
(map String/.ToUpper ["abc" "def" "ghi"])
210+
```
211+
Note that we no longer need the type hint on the parameter to avoid reflection.
212+
213+
## `:param-tags`
214+
215+
Using qualified methods gives us the benefit of a type hint on the instance variable for instance method invocation. In fact, the type that the qualified method gives (`String` in the case of `String/.ToUpper`) overrides any type hint on the instance argument.
216+
217+
For invocations, further disambiguation of method signature can be made by providing type hints on the other arguments.
218+
However, for use in value positions, we cannot type hint in this way.
219+
220+
Consider trying to map `Math/Abs` over a sequence. In the old `IFn`-wrapper style
221+
222+
```clojure
223+
(map #(Math/Abs %) [1.23 -3.14])
224+
```
225+
226+
This will get a reflection warning. You can fix this with a type hint.
227+
228+
```clojure
229+
(map #(Math/Abs ^double %) [1.23 -3.14])
230+
```
231+
232+
Now consider using a qualified method:
233+
234+
```clojure
235+
(map Math/Abs [1.23 -3.14])
236+
```
237+
238+
You will get a reflection warning. And there is no place to put a traditional type hint.
239+
240+
Enter `:param-tags`. You can add `:param-tags` metadata to the qualified method to provide type hints. The easiest way to The easiest way is to use new `^[...]` metadata reader syntax.
241+
242+
```clojure
243+
(map ^[double] Math/Abs [1.23 -3.14]))
244+
```
245+
246+
You need to put as many types as there are arguments in the method you wish to select.
247+
If you don't need to type a specific argument, you can use an underscore, as in
248+
249+
```clojure
250+
(^[_ _] clojure.lang.Tuple/create 1 2) ; => (1 2)
251+
```
252+
Here, we just need to indicate that the two-argument version of the `Tuple/create` method is required.
253+
254+
By the way, `:param-tags` can be used in invocations also as an alternative way to specify argument typing. Compare
255+
256+
```clojure
257+
(Math/Abs ^double x)
258+
```
259+
260+
to
261+
262+
```clojure
263+
(^[double] Math/Abs x)
264+
```
265+
For one argument, not a big deal. But with, say, 3 arguments, it might help to pull all the type info in one place
266+
267+
```clojure
268+
(.Method ^Typename (...) ^T1 (...) ^T2 (...) ^T3 (...) )
269+
([T1 T2 T3] Typename/.Method (....) (....) (....) (....))
270+
```
271+
272+
At least you have a choice.
273+
274+
## the add-ons
275+
276+
We need to allow adding `(type-args ...)` and `(by-ref ...)` to our `:param-tags`.
277+
Simply, we just put them into position.
278+
279+
```clojure
280+
#[(type-args T1 T2) T3 T4 (by-ref T5)] T/.m
281+
```
282+
283+
would specify the instance method
284+
285+
```C#
286+
class T
287+
{
288+
Object m<T1,T2>(T3 arg1, T4 arg2, ref T4 arg3) {...}
289+
}
290+
```
291+
292+
I don't know if there is any utility for `by-ref` in things like `map` calls. You would have no way to get any changed value in a `by-ref` parameter.
293+
A fallback to the classic interop using a function wrapper seems best for this situation.
294+
295+
296+
## Availability
297+
298+
The new qualified methods feature is available in ClojureCLR 1.12-alpha10 and later.
299+
300+
# Notes
301+
302+
## Note: Default imports
303+
304+
At system startup, we go through all loaded assemblies and create an default import list of all types that satisfy the following conditions:
305+
306+
- the type's namespace is `System`
307+
- the type is a class, interface, or value type
308+
- the type is public
309+
- the type is not a generic type definition (meaning an uintantiated generic type)
310+
- the type's name does not start with "_" or "<".
311+
312+
In addition, strictly for my own convenience for dealing with 'core.clj' and other startup files, I add `System.Text.StringBuilder`, `clojure.lang.BigInteger` and `clojure.lang.BigDecimal`. (I'll change `clojure.lang.BigInteger` to `System.Numerics.BigInteger` in the ClojureCLR.Next.)
313+
314+
315+
## Note: Specifying type names
316+
317+
The
318+
[specifying types](https://github.com/clojure/clojure-clr/wiki/Specifying-types) page explains how we get around this. It is just ugly. The mechanism ties directly into the CLR's type naming and resolution machinery, thus:
319+
320+
```clojure
321+
|System.Collections.Generic.IList`1[System.Int32]|
322+
```
323+
324+
You have to include fully-qualified type names, generics include a backtick and the number of type arguments, and the type arguments are enclosed in square brackets.
325+
326+
I plan to introduce a new syntax for this in ClojureCLR.Next, that would take advantage of imports and be otherwise nice.
327+
When I designed the `|...|` syntax (stolen from CommonLisp), Clojure did not yet have _tagged literals_. Now we might be able to do something like
328+
329+
#type "IList<int>"
330+
331+
(If you are interested in helping to design this, please let me know.)
332+

0 commit comments

Comments
 (0)