Skip to content

Commit 56b0725

Browse files
timfeltimfel
committed
[GR-23349] Get test_warnings to pass
PullRequest: graalpython/1116
2 parents 692bb88 + b7f8ea7 commit 56b0725

File tree

65 files changed

+1515
-236
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

65 files changed

+1515
-236
lines changed

‎docs/contributor/CONTRIBUTING.md

Lines changed: 66 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -180,10 +180,14 @@ A tag file can be regenerated with
180180

181181
There's also multiple other gates that may fail with changes. One of these is
182182
our *style* gate, which checks formatting rules and copyrights. To auto-fix most
183-
issues, run the following command. Anything that's reported as error after this
184-
command you have to fix manually.
183+
issues, run the following commands. Anything that's reported as error after this
184+
command you have to fix manually. Note that to really match what's in the gate,
185+
you have to set the `JDT` environment variable to the path to an Eclipse
186+
compiler Jar file, and the `ECLIPSE_EXE` environment variable to the path of an
187+
eclipse executable.
185188

186189
mx python-style --fix
190+
mx python-gate --tags style
187191

188192
Another important gate is the gate that checks if you broke the native image
189193
building. To test if building a native image still works, you can use the
@@ -225,3 +229,63 @@ For additional arguments to the Python launcher, you can separate them by
225229
another double-dash:
226230

227231
mx benchmark meso:nbody3 -- --python-vm=graalpython -- --python.EmulateJython -Dgraal.Dump= -Dgraal.MethodFilter=*measure*
232+
233+
#### A note on terminology
234+
235+
Note that there may be a little confusion about the configuration names of
236+
benchmarks.
237+
238+
##### GraalVM Community and GraalVM Enterprise configurations
239+
240+
We have benchmarks for GraalVM Community and Enterprise. For historical reasons,
241+
these are sometimes referred to in some config files as *CE* and *EE*; *core*
242+
and *enterprise*; *graalvm_ce* and *graalvm_ee*; or *graalpython_core* and
243+
*graalpython_enterprise*, respectively.
244+
245+
##### Different GraalVM Python configurations
246+
247+
There are also different options for how the Python interpreter is run, passed
248+
via the `--python-vm-config` parameter:
249+
* `default` - run using the standard options
250+
* `default-multi` - run using a shared engine, which is the mode that is
251+
recommended to embedders that want to spawn multiple isolated Python contexts
252+
* `native` - same as `default`, its name is due to the fact that it runs C
253+
extensions using a mixture of LLVM bitcode interpreted and compiled via
254+
GraalVM and real native libraries
255+
* `sandboxed` - this name is historical - this configuration requires a GraalVM
256+
Enterprise and runs all C extensions purely as LLVM bitcode on the GraalVM,
257+
without any access to the native OS libraries, i.e., using the
258+
`--llvm.managed` option for GraalVM.
259+
260+
##### Configuration of the underlying GraalVM runtime
261+
262+
Finally, there are the `--jvm` and `--jvm-config` configuration options for `mx
263+
benchmark`. By default, the commands presented above will run on the JVM in
264+
*server* mode, using the Graal compiler in what we call *hosted* mode. This is
265+
almost the same but not quite the `--jvm` mode you will get when running the
266+
`graalpython` executable from a full GraalVM, and usually good enough if you
267+
want to look at the compiler graphs or peak performance numbers. In our CI,
268+
however, we always build a full GraalVM and benchmark using that, since that is
269+
what we ship. There, we have two different configurations corresponding to the
270+
launcher flags available for the GraalVM `graalpython` executable: *jvm* and
271+
*native*. The first runs on top of Hotspot using the Graal compiler, the second
272+
runs the AOT compiled GraalVM native image of Python.
273+
274+
Building a GraalVM Python configuration can be achieved for the CE version like
275+
so:
276+
277+
mx --env ../../graal/vm/mx.vm/ce --exclude-components=slgm --dynamicimports /vm graalvm-show
278+
mx --env ../../graal/vm/mx.vm/ce --exclude-components=slgm --dynamicimports /vm build
279+
280+
The first command will print some information about the GraalVM configuration
281+
that is about to be built, and the second will build it. **IMPORTANT:** The
282+
first command should tell you that the `Config name` is `ce_python`, otherwise
283+
the next commands will not work.
284+
285+
To run the JVM configuration:
286+
287+
mx --env ../../graal/vm/mx.vm/ce --exclude-components=slgm --dynamicimports /vm benchmark meso:nbody3 -- --python-vm=graalpython --jvm=graalvm-ce-python --jvm-config=jvm --python-vm-config=default --
288+
289+
To run the Native Image configuration:
290+
291+
mx --env ../../graal/vm/mx.vm/ce --exclude-components=slgm --dynamicimports /vm benchmark meso:nbody3 -- --python-vm=graalpython --jvm=graalvm-ce-python --jvm-config=native --python-vm-config=default --

‎graalpython/com.oracle.graal.python.processor/src/com/oracle/graal/python/processor/ArgumentClinicModel.java

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,7 @@ public static String getCodeSnippet(ArgumentClinic annotation, BuiltinAnnotation
7474
case Index:
7575
return format("IndexConversionNodeGen.create(%s, %s)", annotation.defaultValue(), annotation.useDefaultForNone());
7676
case None:
77+
return format("DefaultValueNode.create(%s, %s)", annotation.defaultValue(), annotation.useDefaultForNone());
7778
default:
7879
throw new IllegalArgumentException(annotation.conversion().toString());
7980
}
@@ -87,7 +88,7 @@ public static void addImports(ArgumentClinic annotation, Set<String> imports) {
8788
if (annotation.defaultValue().startsWith("PNone.")) {
8889
imports.add("com.oracle.graal.python.builtins.objects.PNone");
8990
}
90-
if (annotation.conversion() != ClinicConversion.None) {
91+
if (annotation.conversion() != ClinicConversion.None || (annotation.customConversion().isEmpty() && !annotation.defaultValue().isEmpty())) {
9192
imports.add(CLINIC_PACKAGE + '.' + getConvertorImport(annotation));
9293
}
9394
}
@@ -103,6 +104,7 @@ private static String getConvertorImport(ArgumentClinic annotation) {
103104
case Index:
104105
return "IndexConversionNodeGen";
105106
case None:
107+
return "DefaultValueNode";
106108
default:
107109
throw new IllegalArgumentException(annotation.conversion().toString());
108110
}
@@ -118,6 +120,7 @@ public static PrimitiveType[] getAcceptedPrimitiveTypes(ArgumentClinic annotatio
118120
case Index:
119121
return new PrimitiveType[]{PrimitiveType.Int};
120122
case None:
123+
return new PrimitiveType[]{PrimitiveType.Boolean, PrimitiveType.Int, PrimitiveType.Long, PrimitiveType.Double};
121124
default:
122125
throw new IllegalArgumentException(annotation.conversion().toString());
123126
}
@@ -195,7 +198,7 @@ public static ArgumentClinicData create(ArgumentClinic annotation, TypeElement t
195198
PrimitiveType[] acceptedPrimitives = new PrimitiveType[0];
196199
String castNodeFactory;
197200
if (annotation.customConversion().isEmpty()) {
198-
if (annotation.conversion() == ClinicConversion.None) {
201+
if (annotation.conversion() == ClinicConversion.None && annotation.defaultValue().isEmpty()) {
199202
throw new ProcessingError(type, "ArgumentClinic annotation must declare either builtin conversion or custom conversion.");
200203
}
201204
castNodeFactory = BuiltinConvertor.getCodeSnippet(annotation, builtinAnnotation);

‎graalpython/com.oracle.graal.python.shell/src/com/oracle/graal/python/shell/GraalPythonMain.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -410,7 +410,7 @@ protected void launch(Builder contextBuilder) {
410410
}
411411
}
412412
if (warnOptions == null || warnOptions.isEmpty()) {
413-
warnOptions = "default";
413+
warnOptions = "";
414414
}
415415
String executable = getContextOptionIfSetViaCommandLine("python.Executable");
416416
if (executable != null) {

‎graalpython/com.oracle.graal.python.test/src/com/oracle/graal/python/nodes/literal/FormatStringTests.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -409,7 +409,7 @@ public RuntimeException raiseInvalidSyntax(ErrorType type, Node location, String
409409
}
410410

411411
@Override
412-
public void warn(Object type, String format, Object... args) {
412+
public void warn(PythonBuiltinClassType type, String format, Object... args) {
413413
throw new RuntimeException("Warning: " + String.format(format, args));
414414
}
415415

‎graalpython/com.oracle.graal.python.test/src/com/oracle/graal/python/test/parser/StringUtilsTests.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -73,7 +73,7 @@ public RuntimeException raiseInvalidSyntax(ErrorType type, Node location, String
7373
}
7474

7575
@Override
76-
public void warn(Object type, String format, Object... args) {
76+
public void warn(PythonBuiltinClassType type, String format, Object... args) {
7777
Assert.fail("Unexpected warning: " + String.format(format, args));
7878
}
7979

‎graalpython/com.oracle.graal.python.test/src/graalpytest.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -630,6 +630,8 @@ def run(self):
630630
eval_scope("session")
631631

632632
for module in self.test_modules():
633+
if module is None:
634+
continue
633635
# eval module scope
634636
eval_scope("module")
635637

‎graalpython/com.oracle.graal.python.test/src/tests/unittest_tags/test_warnings.txt

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,17 @@
1+
*graalpython.lib-python.3.test.test_warnings.__init__.BootstrapTest.test_issue_8766
12
*graalpython.lib-python.3.test.test_warnings.__init__.CCatchWarningTests.test_catch_warnings_defaults
23
*graalpython.lib-python.3.test.test_warnings.__init__.CCatchWarningTests.test_catch_warnings_recording
34
*graalpython.lib-python.3.test.test_warnings.__init__.CCatchWarningTests.test_catch_warnings_reentry_guard
45
*graalpython.lib-python.3.test.test_warnings.__init__.CCatchWarningTests.test_catch_warnings_restore
56
*graalpython.lib-python.3.test.test_warnings.__init__.CCatchWarningTests.test_check_warnings
67
*graalpython.lib-python.3.test.test_warnings.__init__.CCatchWarningTests.test_record_override_showwarning_before
78
*graalpython.lib-python.3.test.test_warnings.__init__.CCatchWarningTests.test_record_override_showwarning_inside
9+
*graalpython.lib-python.3.test.test_warnings.__init__.CEnvironmentVariableTests.test_comma_separated_warnings
10+
*graalpython.lib-python.3.test.test_warnings.__init__.CEnvironmentVariableTests.test_conflicting_envvar_and_command_line
11+
*graalpython.lib-python.3.test.test_warnings.__init__.CEnvironmentVariableTests.test_default_filter_configuration
12+
*graalpython.lib-python.3.test.test_warnings.__init__.CEnvironmentVariableTests.test_envvar_and_command_line
13+
*graalpython.lib-python.3.test.test_warnings.__init__.CEnvironmentVariableTests.test_nonascii
14+
*graalpython.lib-python.3.test.test_warnings.__init__.CEnvironmentVariableTests.test_single_warning
815
*graalpython.lib-python.3.test.test_warnings.__init__.CFilterTests.test_always
916
*graalpython.lib-python.3.test.test_warnings.__init__.CFilterTests.test_always_after_default
1017
*graalpython.lib-python.3.test.test_warnings.__init__.CFilterTests.test_append_duplicate
@@ -25,25 +32,35 @@
2532
*graalpython.lib-python.3.test.test_warnings.__init__.CFilterTests.test_simplefilter_duplicate_filters
2633
*graalpython.lib-python.3.test.test_warnings.__init__.CPublicAPITests.test_module_all_attribute
2734
*graalpython.lib-python.3.test.test_warnings.__init__.CWCmdLineTests.test_improper_input
35+
*graalpython.lib-python.3.test.test_warnings.__init__.CWarnTests.test_accelerated
2836
*graalpython.lib-python.3.test.test_warnings.__init__.CWarnTests.test_bad_str
2937
*graalpython.lib-python.3.test.test_warnings.__init__.CWarnTests.test_exec_filename
3038
*graalpython.lib-python.3.test.test_warnings.__init__.CWarnTests.test_filename
3139
*graalpython.lib-python.3.test.test_warnings.__init__.CWarnTests.test_message
3240
*graalpython.lib-python.3.test.test_warnings.__init__.CWarnTests.test_stacklevel
3341
*graalpython.lib-python.3.test.test_warnings.__init__.CWarnTests.test_stacklevel_import
3442
*graalpython.lib-python.3.test.test_warnings.__init__.CWarnTests.test_warn_explicit_non_ascii_filename
43+
*graalpython.lib-python.3.test.test_warnings.__init__.CWarnTests.test_warn_explicit_type_errors
3544
*graalpython.lib-python.3.test.test_warnings.__init__.CWarnTests.test_warn_nonstandard_types
3645
*graalpython.lib-python.3.test.test_warnings.__init__.CWarnTests.test_warning_classes
3746
*graalpython.lib-python.3.test.test_warnings.__init__.CWarningsDisplayTests.test_formatwarning
3847
*graalpython.lib-python.3.test.test_warnings.__init__.CWarningsDisplayTests.test_formatwarning_override
3948
*graalpython.lib-python.3.test.test_warnings.__init__.CWarningsDisplayTests.test_showwarning
49+
*graalpython.lib-python.3.test.test_warnings.__init__.FinalizationTest.test_finalization
50+
*graalpython.lib-python.3.test.test_warnings.__init__.FinalizationTest.test_late_resource_warning
4051
*graalpython.lib-python.3.test.test_warnings.__init__.PyCatchWarningTests.test_catch_warnings_defaults
4152
*graalpython.lib-python.3.test.test_warnings.__init__.PyCatchWarningTests.test_catch_warnings_recording
4253
*graalpython.lib-python.3.test.test_warnings.__init__.PyCatchWarningTests.test_catch_warnings_reentry_guard
4354
*graalpython.lib-python.3.test.test_warnings.__init__.PyCatchWarningTests.test_catch_warnings_restore
4455
*graalpython.lib-python.3.test.test_warnings.__init__.PyCatchWarningTests.test_check_warnings
4556
*graalpython.lib-python.3.test.test_warnings.__init__.PyCatchWarningTests.test_record_override_showwarning_before
4657
*graalpython.lib-python.3.test.test_warnings.__init__.PyCatchWarningTests.test_record_override_showwarning_inside
58+
*graalpython.lib-python.3.test.test_warnings.__init__.PyEnvironmentVariableTests.test_comma_separated_warnings
59+
*graalpython.lib-python.3.test.test_warnings.__init__.PyEnvironmentVariableTests.test_conflicting_envvar_and_command_line
60+
*graalpython.lib-python.3.test.test_warnings.__init__.PyEnvironmentVariableTests.test_default_filter_configuration
61+
*graalpython.lib-python.3.test.test_warnings.__init__.PyEnvironmentVariableTests.test_envvar_and_command_line
62+
*graalpython.lib-python.3.test.test_warnings.__init__.PyEnvironmentVariableTests.test_nonascii
63+
*graalpython.lib-python.3.test.test_warnings.__init__.PyEnvironmentVariableTests.test_single_warning
4764
*graalpython.lib-python.3.test.test_warnings.__init__.PyFilterTests.test_always
4865
*graalpython.lib-python.3.test.test_warnings.__init__.PyFilterTests.test_always_after_default
4966
*graalpython.lib-python.3.test.test_warnings.__init__.PyFilterTests.test_append_duplicate
@@ -64,6 +81,8 @@
6481
*graalpython.lib-python.3.test.test_warnings.__init__.PyFilterTests.test_simplefilter_duplicate_filters
6582
*graalpython.lib-python.3.test.test_warnings.__init__.PyPublicAPITests.test_module_all_attribute
6683
*graalpython.lib-python.3.test.test_warnings.__init__.PyWCmdLineTests.test_improper_input
84+
*graalpython.lib-python.3.test.test_warnings.__init__.PyWCmdLineTests.test_improper_option
85+
*graalpython.lib-python.3.test.test_warnings.__init__.PyWCmdLineTests.test_warnings_bootstrap
6786
*graalpython.lib-python.3.test.test_warnings.__init__.PyWarnTests.test_bad_str
6887
*graalpython.lib-python.3.test.test_warnings.__init__.PyWarnTests.test_exec_filename
6988
*graalpython.lib-python.3.test.test_warnings.__init__.PyWarnTests.test_filename
@@ -72,15 +91,23 @@
7291
*graalpython.lib-python.3.test.test_warnings.__init__.PyWarnTests.test_stacklevel
7392
*graalpython.lib-python.3.test.test_warnings.__init__.PyWarnTests.test_stacklevel_import
7493
*graalpython.lib-python.3.test.test_warnings.__init__.PyWarnTests.test_warn_explicit_non_ascii_filename
94+
*graalpython.lib-python.3.test.test_warnings.__init__.PyWarnTests.test_warn_explicit_type_errors
7595
*graalpython.lib-python.3.test.test_warnings.__init__.PyWarnTests.test_warn_nonstandard_types
7696
*graalpython.lib-python.3.test.test_warnings.__init__.PyWarnTests.test_warning_classes
7797
*graalpython.lib-python.3.test.test_warnings.__init__.PyWarningsDisplayTests.test_formatwarning
7898
*graalpython.lib-python.3.test.test_warnings.__init__.PyWarningsDisplayTests.test_formatwarning_override
7999
*graalpython.lib-python.3.test.test_warnings.__init__.PyWarningsDisplayTests.test_showwarning
100+
*graalpython.lib-python.3.test.test_warnings.__init__.PyWarningsDisplayTests.test_tracemalloc
101+
*graalpython.lib-python.3.test.test_warnings.__init__._WarningsTests.test_default_action
102+
*graalpython.lib-python.3.test.test_warnings.__init__._WarningsTests.test_filename_none
103+
*graalpython.lib-python.3.test.test_warnings.__init__._WarningsTests.test_filter
104+
*graalpython.lib-python.3.test.test_warnings.__init__._WarningsTests.test_issue31285
80105
*graalpython.lib-python.3.test.test_warnings.__init__._WarningsTests.test_issue31411
81106
*graalpython.lib-python.3.test.test_warnings.__init__._WarningsTests.test_issue31416
82107
*graalpython.lib-python.3.test.test_warnings.__init__._WarningsTests.test_issue31566
108+
*graalpython.lib-python.3.test.test_warnings.__init__._WarningsTests.test_onceregistry
83109
*graalpython.lib-python.3.test.test_warnings.__init__._WarningsTests.test_show_warning_output
84110
*graalpython.lib-python.3.test.test_warnings.__init__._WarningsTests.test_showwarning_missing
85111
*graalpython.lib-python.3.test.test_warnings.__init__._WarningsTests.test_showwarning_not_callable
112+
*graalpython.lib-python.3.test.test_warnings.__init__._WarningsTests.test_showwarnmsg_missing
86113
*graalpython.lib-python.3.test.test_warnings.__init__._WarningsTests.test_stderr_none

‎graalpython/com.oracle.graal.python/src/com/oracle/graal/python/builtins/Python3Core.java

Lines changed: 5 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -97,6 +97,7 @@
9797
import com.oracle.graal.python.builtins.modules.TimeModuleBuiltins;
9898
import com.oracle.graal.python.builtins.modules.TraceModuleBuiltins;
9999
import com.oracle.graal.python.builtins.modules.UnicodeDataModuleBuiltins;
100+
import com.oracle.graal.python.builtins.modules.WarningsModuleBuiltins;
100101
import com.oracle.graal.python.builtins.modules.WeakRefModuleBuiltins;
101102
import com.oracle.graal.python.builtins.modules.ZLibModuleBuiltins;
102103
import com.oracle.graal.python.builtins.modules.ZipImportModuleBuiltins;
@@ -175,8 +176,6 @@
175176
import com.oracle.graal.python.builtins.objects.type.TypeNodes.GetNameNode;
176177
import com.oracle.graal.python.builtins.objects.zipimporter.ZipImporterBuiltins;
177178
import com.oracle.graal.python.nodes.BuiltinNames;
178-
import com.oracle.graal.python.nodes.attributes.ReadAttributeFromDynamicObjectNode;
179-
import com.oracle.graal.python.nodes.call.CallNode;
180179
import com.oracle.graal.python.nodes.call.GenericInvokeNode;
181180
import com.oracle.graal.python.runtime.PythonCodeSerializer;
182181
import com.oracle.graal.python.runtime.PythonContext;
@@ -228,7 +227,6 @@ private static String[] initializeCoreFiles() {
228227
"_functools",
229228
"method",
230229
"code",
231-
"_warnings",
232230
"posix",
233231
"_io",
234232
"_frozen_importlib",
@@ -422,6 +420,7 @@ private static PythonBuiltins[] initializeBuiltins() {
422420
new LZMADecompressorBuiltins(),
423421
new MultiprocessingModuleBuiltins(),
424422
new SemLockBuiltins(),
423+
new WarningsModuleBuiltins(),
425424
new GraalPythonModuleBuiltins()));
426425
if (hasCoverageTool) {
427426
builtins.add(new TraceModuleBuiltins());
@@ -543,7 +542,7 @@ public PythonBuiltinClass lookupType(PythonBuiltinClassType type) {
543542
@Override
544543
@TruffleBoundary
545544
public String[] builtinModuleNames() {
546-
return builtinModules.keySet().toArray(new String[0]);
545+
return builtinModules.keySet().toArray(PythonUtils.EMPTY_STRING_ARRAY);
547546
}
548547

549548
@Override
@@ -565,10 +564,8 @@ public PException raise(PythonBuiltinClassType type, String format, Object... ar
565564

566565
@Override
567566
@TruffleBoundary
568-
public void warn(Object type, String format, Object... args) {
569-
PythonModule warningsModule = lookupBuiltinModule("_warnings");
570-
Object warn = ReadAttributeFromDynamicObjectNode.getUncached().execute(warningsModule.getStorage(), "warn");
571-
CallNode.getUncached().execute(warn, String.format(format, args), type);
567+
public void warn(PythonBuiltinClassType type, String format, Object... args) {
568+
WarningsModuleBuiltins.WarnNode.getUncached().warnFormat(null, null, type, 1, format, args);
572569
}
573570

574571
private void publishBuiltinModules() {
@@ -636,7 +633,6 @@ private void populateBuiltins() {
636633

637634
// core machinery
638635
createModule("_descriptor");
639-
createModule("_warnings");
640636
PythonModule bootstrapExternal = createModule("importlib._bootstrap_external");
641637
bootstrapExternal.setAttribute(__PACKAGE__, "importlib");
642638
builtinModules.put("_frozen_importlib_external", bootstrapExternal);

0 commit comments

Comments
 (0)