Skip to content

Commit f834d71

Browse files
yoffyoff
committed
Python: split out data model tests
1 parent 2c5de7f commit f834d71

File tree

2 files changed

+161
-155
lines changed

2 files changed

+161
-155
lines changed

‎python/ql/test/experimental/dataflow/coverage/classes.py

Lines changed: 2 additions & 155 deletions
Original file line numberDiff line numberDiff line change
@@ -5,164 +5,9 @@
55
# These tests should cover all the class calls that we hope to support.
66
# It is based on https://docs.python.org/3/reference/datamodel.html, and headings refer there.
77
#
8-
# Intended sources should be the variable `SOURCE` and intended sinks should be
9-
# arguments to the function `SINK` (see python/ql/test/experimental/dataflow/testConfig.qll).
10-
#
11-
# Functions whose name ends with "_with_local_flow" will also be tested for local flow.
12-
#
138
# All functions starting with "test_" should run and print `"OK"`.
149
# This can be checked by running validTest.py.
1510

16-
# These are defined so that we can evaluate the test code.
17-
NONSOURCE = "not a source"
18-
SOURCE = "source"
19-
20-
def is_source(x):
21-
return x == "source" or x == b"source" or x == 42 or x == 42.0 or x == 42j
22-
23-
def SINK(x):
24-
if is_source(x):
25-
print("OK")
26-
else:
27-
print("Unexpected flow", x)
28-
29-
def SINK_F(x):
30-
if is_source(x):
31-
print("Unexpected flow", x)
32-
else:
33-
print("OK")
34-
35-
# Callable types
36-
# These are the types to which the function call operation (see section Calls) can be applied:
37-
38-
# User-defined functions
39-
# A user-defined function object is created by a function definition (see section Function definitions). It should be called with an argument list containing the same number of items as the function's formal parameter list.
40-
def f(a, b):
41-
return a
42-
43-
SINK(f(SOURCE, 3))
44-
45-
# Instance methods
46-
# An instance method object combines a class, a class instance and any callable object (normally a user-defined function).
47-
class C(object):
48-
49-
def method(self, x, cls):
50-
assert cls is self.__class__
51-
return x
52-
53-
@classmethod
54-
def classmethod(cls, x):
55-
return x
56-
57-
@staticmethod
58-
def staticmethod(x):
59-
return x
60-
61-
def gen(self, x, count):
62-
n = count
63-
while n > 0:
64-
yield x
65-
n -= 1
66-
67-
async def coro(self, x):
68-
return x
69-
70-
c = C()
71-
72-
# When an instance method object is created by retrieving a user-defined function object from a class via one of its instances, its __self__ attribute is the instance, and the method object is said to be bound. The new method’s __func__ attribute is the original function object.
73-
func_obj = c.method.__func__
74-
75-
# When an instance method object is called, the underlying function (__func__) is called, inserting the class instance (__self__) in front of the argument list. For instance, when C is a class which contains a definition for a function f(), and x is an instance of C, calling x.f(1) is equivalent to calling C.f(x, 1).
76-
SINK(c.method(SOURCE, C))
77-
SINK(C.method(c, SOURCE, C))
78-
SINK(func_obj(c, SOURCE, C))
79-
80-
81-
# When an instance method object is created by retrieving a class method object from a class or instance, its __self__ attribute is the class itself, and its __func__ attribute is the function object underlying the class method.
82-
c_func_obj = C.classmethod.__func__
83-
84-
# When an instance method object is derived from a class method object, the “class instance” stored in __self__ will actually be the class itself, so that calling either x.f(1) or C.f(1) is equivalent to calling f(C,1) where f is the underlying function.
85-
SINK(c.classmethod(SOURCE))
86-
SINK(C.classmethod(SOURCE))
87-
SINK(c_func_obj(C, SOURCE))
88-
89-
# Generator functions
90-
# A function or method which uses the yield statement (see section The yield statement) is called a generator function. Such a function, when called, always returns an iterator object which can be used to execute the body of the function: calling the iterator’s iterator.__next__() method will cause the function to execute until it provides a value using the yield statement. When the function executes a return statement or falls off the end, a StopIteration exception is raised and the iterator will have reached the end of the set of values to be returned.
91-
def gen(x, count):
92-
n = count
93-
while n > 0:
94-
yield x
95-
n -= 1
96-
97-
iter = gen(SOURCE, 1)
98-
SINK(iter.__next__())
99-
# SINK_F(iter.__next__()) # throws StopIteration, FP
100-
101-
oiter = c.gen(SOURCE, 1)
102-
SINK(oiter.__next__())
103-
# SINK_F(oiter.__next__()) # throws StopIteration, FP
104-
105-
# Coroutine functions
106-
# A function or method which is defined using async def is called a coroutine function. Such a function, when called, returns a coroutine object. It may contain await expressions, as well as async with and async for statements. See also the Coroutine Objects section.
107-
async def coro(x):
108-
return x
109-
110-
import asyncio
111-
SINK(asyncio.run(coro(SOURCE)))
112-
SINK(asyncio.run(c.coro(SOURCE)))
113-
114-
class A:
115-
116-
def __await__(self):
117-
# yield SOURCE -- see https://groups.google.com/g/dev-python/c/_lrrc-vp9TI?pli=1
118-
return (yield from asyncio.coroutine(lambda: SOURCE)())
119-
120-
async def agen(x):
121-
a = A()
122-
return await a
123-
124-
SINK(asyncio.run(agen(SOURCE)))
125-
126-
# Asynchronous generator functions
127-
# A function or method which is defined using async def and which uses the yield statement is called a asynchronous generator function. Such a function, when called, returns an asynchronous iterator object which can be used in an async for statement to execute the body of the function.
128-
129-
# Calling the asynchronous iterator’s aiterator.__anext__() method will return an awaitable which when awaited will execute until it provides a value using the yield expression. When the function executes an empty return statement or falls off the end, a StopAsyncIteration exception is raised and the asynchronous iterator will have reached the end of the set of values to be yielded.
130-
131-
# Built-in functions
132-
# A built-in function object is a wrapper around a C function. Examples of built-in functions are len() and math.sin() (math is a standard built-in module). The number and type of the arguments are determined by the C function. Special read-only attributes: __doc__ is the function’s documentation string, or None if unavailable; __name__ is the function’s name; __self__ is set to None (but see the next item); __module__ is the name of the module the function was defined in or None if unavailable.
133-
134-
# Built-in methods
135-
# This is really a different disguise of a built-in function, this time containing an object passed to the C function as an implicit extra argument. An example of a built-in method is alist.append(), assuming alist is a list object. In this case, the special read-only attribute __self__ is set to the object denoted by alist.
136-
137-
# Classes
138-
# Classes are callable. These objects normally act as factories for new instances of themselves, but variations are possible for class types that override __new__(). The arguments of the call are passed to __new__() and, in the typical case, to __init__() to initialize the new instance.
139-
140-
# Class Instances
141-
# Instances of arbitrary classes can be made callable by defining a __call__() method in their class.
142-
143-
# If a class sets __iter__() to None, calling iter() on its instances will raise a TypeError (without falling back to __getitem__()).
144-
145-
# 3.3.1. Basic customization
146-
147-
class Customized:
148-
149-
a = NONSOURCE
150-
b = NONSOURCE
151-
152-
def __new__(cls):
153-
cls.a = SOURCE
154-
return super().__new__(cls)
155-
156-
def __init__(self):
157-
self.b = SOURCE
158-
159-
# testing __new__ and __init__
160-
customized = Customized()
161-
SINK(Customized.a)
162-
SINK_F(Customized.b)
163-
SINK(customized.a)
164-
SINK(customized.b)
165-
16611
def OK():
16712
print("OK")
16813

@@ -1254,6 +1099,8 @@ def test_exit():
12541099
pass
12551100

12561101
# 3.4.1. Awaitable Objects
1102+
import asyncio
1103+
12571104
# object.__await__(self)
12581105
class With_await:
12591106

‎python/ql/test/experimental/dataflow/coverage/datamodel.py

Lines changed: 159 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,159 @@
1+
# User-defined methods, both instance methods and class methods, can be called in many non-standard ways
2+
# i.e. differently from simply `c.f()` or `C.f()`. For example, a user-defined `__await__` function on a
3+
# class `C` will be called by the syntactic construct `await c` when `c` is an instance of `C`.
4+
#
5+
# These tests are based on the first part of https://docs.python.org/3/reference/datamodel.html.
6+
# A thorough covering of methods in that document is found in classes.py.
7+
#
8+
# Intended sources should be the variable `SOURCE` and intended sinks should be
9+
# arguments to the function `SINK` (see python/ql/test/experimental/dataflow/testConfig.qll).
10+
11+
# These are defined so that we can evaluate the test code.
12+
NONSOURCE = "not a source"
13+
SOURCE = "source"
14+
15+
def is_source(x):
16+
return x == "source" or x == b"source" or x == 42 or x == 42.0 or x == 42j
17+
18+
def SINK(x):
19+
if is_source(x):
20+
print("OK")
21+
else:
22+
print("Unexpected flow", x)
23+
24+
def SINK_F(x):
25+
if is_source(x):
26+
print("Unexpected flow", x)
27+
else:
28+
print("OK")
29+
30+
# Callable types
31+
# These are the types to which the function call operation (see section Calls) can be applied:
32+
33+
# User-defined functions
34+
# A user-defined function object is created by a function definition (see section Function definitions). It should be called with an argument list containing the same number of items as the function's formal parameter list.
35+
def f(a, b):
36+
return a
37+
38+
SINK(f(SOURCE, 3))
39+
40+
# Instance methods
41+
# An instance method object combines a class, a class instance and any callable object (normally a user-defined function).
42+
class C(object):
43+
44+
def method(self, x, cls):
45+
assert cls is self.__class__
46+
return x
47+
48+
@classmethod
49+
def classmethod(cls, x):
50+
return x
51+
52+
@staticmethod
53+
def staticmethod(x):
54+
return x
55+
56+
def gen(self, x, count):
57+
n = count
58+
while n > 0:
59+
yield x
60+
n -= 1
61+
62+
async def coro(self, x):
63+
return x
64+
65+
c = C()
66+
67+
# When an instance method object is created by retrieving a user-defined function object from a class via one of its instances, its __self__ attribute is the instance, and the method object is said to be bound. The new method’s __func__ attribute is the original function object.
68+
func_obj = c.method.__func__
69+
70+
# When an instance method object is called, the underlying function (__func__) is called, inserting the class instance (__self__) in front of the argument list. For instance, when C is a class which contains a definition for a function f(), and x is an instance of C, calling x.f(1) is equivalent to calling C.f(x, 1).
71+
SINK(c.method(SOURCE, C))
72+
SINK(C.method(c, SOURCE, C))
73+
SINK(func_obj(c, SOURCE, C))
74+
75+
76+
# When an instance method object is created by retrieving a class method object from a class or instance, its __self__ attribute is the class itself, and its __func__ attribute is the function object underlying the class method.
77+
c_func_obj = C.classmethod.__func__
78+
79+
# When an instance method object is derived from a class method object, the “class instance” stored in __self__ will actually be the class itself, so that calling either x.f(1) or C.f(1) is equivalent to calling f(C,1) where f is the underlying function.
80+
SINK(c.classmethod(SOURCE))
81+
SINK(C.classmethod(SOURCE))
82+
SINK(c_func_obj(C, SOURCE))
83+
84+
# Generator functions
85+
# A function or method which uses the yield statement (see section The yield statement) is called a generator function. Such a function, when called, always returns an iterator object which can be used to execute the body of the function: calling the iterator’s iterator.__next__() method will cause the function to execute until it provides a value using the yield statement. When the function executes a return statement or falls off the end, a StopIteration exception is raised and the iterator will have reached the end of the set of values to be returned.
86+
def gen(x, count):
87+
n = count
88+
while n > 0:
89+
yield x
90+
n -= 1
91+
92+
iter = gen(SOURCE, 1)
93+
SINK(iter.__next__())
94+
# SINK_F(iter.__next__()) # throws StopIteration, FP
95+
96+
oiter = c.gen(SOURCE, 1)
97+
SINK(oiter.__next__())
98+
# SINK_F(oiter.__next__()) # throws StopIteration, FP
99+
100+
# Coroutine functions
101+
# A function or method which is defined using async def is called a coroutine function. Such a function, when called, returns a coroutine object. It may contain await expressions, as well as async with and async for statements. See also the Coroutine Objects section.
102+
async def coro(x):
103+
return x
104+
105+
import asyncio
106+
SINK(asyncio.run(coro(SOURCE)))
107+
SINK(asyncio.run(c.coro(SOURCE)))
108+
109+
class A:
110+
111+
def __await__(self):
112+
# yield SOURCE -- see https://groups.google.com/g/dev-python/c/_lrrc-vp9TI?pli=1
113+
return (yield from asyncio.coroutine(lambda: SOURCE)())
114+
115+
async def agen(x):
116+
a = A()
117+
return await a
118+
119+
SINK(asyncio.run(agen(SOURCE)))
120+
121+
# Asynchronous generator functions
122+
# A function or method which is defined using async def and which uses the yield statement is called a asynchronous generator function. Such a function, when called, returns an asynchronous iterator object which can be used in an async for statement to execute the body of the function.
123+
124+
# Calling the asynchronous iterator’s aiterator.__anext__() method will return an awaitable which when awaited will execute until it provides a value using the yield expression. When the function executes an empty return statement or falls off the end, a StopAsyncIteration exception is raised and the asynchronous iterator will have reached the end of the set of values to be yielded.
125+
126+
# Built-in functions
127+
# A built-in function object is a wrapper around a C function. Examples of built-in functions are len() and math.sin() (math is a standard built-in module). The number and type of the arguments are determined by the C function. Special read-only attributes: __doc__ is the function’s documentation string, or None if unavailable; __name__ is the function’s name; __self__ is set to None (but see the next item); __module__ is the name of the module the function was defined in or None if unavailable.
128+
129+
# Built-in methods
130+
# This is really a different disguise of a built-in function, this time containing an object passed to the C function as an implicit extra argument. An example of a built-in method is alist.append(), assuming alist is a list object. In this case, the special read-only attribute __self__ is set to the object denoted by alist.
131+
132+
# Classes
133+
# Classes are callable. These objects normally act as factories for new instances of themselves, but variations are possible for class types that override __new__(). The arguments of the call are passed to __new__() and, in the typical case, to __init__() to initialize the new instance.
134+
135+
# Class Instances
136+
# Instances of arbitrary classes can be made callable by defining a __call__() method in their class.
137+
138+
# If a class sets __iter__() to None, calling iter() on its instances will raise a TypeError (without falling back to __getitem__()).
139+
140+
# 3.3.1. Basic customization
141+
142+
class Customized:
143+
144+
a = NONSOURCE
145+
b = NONSOURCE
146+
147+
def __new__(cls):
148+
cls.a = SOURCE
149+
return super().__new__(cls)
150+
151+
def __init__(self):
152+
self.b = SOURCE
153+
154+
# testing __new__ and __init__
155+
customized = Customized()
156+
SINK(Customized.a)
157+
SINK_F(Customized.b)
158+
SINK(customized.a)
159+
SINK(customized.b)

0 commit comments

Comments
 (0)